Anyone tried Theli for stacking on Linux?

It seems that finally there is a nice stacking software for Linux only :

http://www.astro.uni.../gui/index.html

Here as a really great image using it:

http://www.cloudynig...5/o/all/fpart/1

Anybody tried it?

Alex

Hi Alex,

just give it a try! I'll happily assist you with your first steps :-)
You can contact me at mschirme /at/ gemini . edu


Best regards,

Mischa

Anyone who decides to write up a tutorial on using Theli would be a hero in my book...

dave

Mischa,
I will try and let you know if I run into issues!
How does it stack up against Deep Sky Stacker? Can you see any difference in final result?

Alex

I want to try Theli but I don't have computer running Linux unfortunately

I discovered that I could use VMWare or WUBI to install Linux on my Windows machine. I hope running on a virtual machine doesn't degrade the performance of Theli too much.

I have managed to install ubuntu on my laptop using WUBI. It is in fact a dual boot system, so the performance will not be degraded at all. Of course linux is all greek to me but luckily my DOS knowledge did help me a lot. I managed to install Theli as well. The only thing I was not able to do was setting the Paths for Theli. After some research I guess I found how to do it as well. I ran Theli from the GUI but this is not recommended. So I will shut down Windows 7 now and boot linux and give it another try. Then I have to spend some time to understand how to operate Theli

Dave,

I know the lack of an english-spoken user manual for amateur astronomers is the largest hurdle to overcome. Unfortunately my position at Gemini observatory keeps me very busy so I won't have time to write one from scratch.

However, there is an (old) document where I included some processing examples in the appendix. You can download it from www.astro.uni-bonn.de/~mischa/download/gui.pdf

Read the WFI example (appendix F), and then switch to appendix H (ST10). But please be aware that this is really old and in many parts totally outdated or even wrong. Like, for example, the astrometry section is totally different now. It should help you getting started, though. My colleagues all learn it by simply trying it out, so you can as well.

The current and up-to-date manual (in english) can be found at

www.astro.uni-bonn.de/~theli/gui

It is targetted at professional astronomers, but I think it is a great resource also for amateurs once you know your way around.

I'll happily assist anyone of you. If there are several of you, I will set up a small email exploder somewhere.

Best regards,

mischa

Thank you Mischa, that's very kind of you. I did a little googling and there appear to be a few tutorials online now (the last time I looked was some time back and I had no luck). I've been stacking my pennies and am going to be buying some gear soon and I wanted to play with whatever Linux-based options that were available and see how it goes.

The last time I looked at Theli I didn't have enough time to devote to scaling the learning curve. We'll see how things work out this time around!

Thanks again, and welcome to CN!

dave

Alex, Sedat,

@Alex: I haven't made in-depth comparisons between Theli and other software packages on the amateur market. Results are as good as they can possibly get (otherwise I wouldn't offer it to fellow scientists). An in-depth comparison with commercial products and other free-ware would certainly be interesting (and some comparisons have been made, with some very positive and some very negative results for the other products). I don't want to name any, as I'm not in the position to do so. These tests have never been really quantified or controlled, neither have they been conducted by me, nor were the authors of the other software packages involved. Theli is certainly a much more stony and bumpy way to reduce your data than most other windows solutions. On the other hand, I promise you will learn more about your data, and data reduction as such, as you have ever imagined. And some Linux background knowledge comes for free :-)

@Sedat:
Installing a virtual linux box under windows or mac, using e.g. www.virtualbox.org, works fine. However, theli will run significantly slower.

best regards,

mischa

Dave, and others,

the first step is a complete Linux installation. If you are considering to set up a dual boot system, I recommend you go with any of the Ubuntu/Debian flavors. They have very good packet managers, and the Theli webpage has a list of copy&paste install commands that pull all the relevant tools and libraries needed from the online repositories. No unresolved dependencies.

Once you have a working Theli installation, you'll probably face one or two intense rough days, but then things will get a lot easier. Some basic Linux knowledge will be of great help, but is not mandatory.

In any case, once you try and you get stuck, don't give up but just ask me.

As for the online tutorials, if they are in english (or any other language than german) and not on the theli homepage, I'd be very interested to have a look!

mischa

Mischa, the most recent tutorial I've found is on Theli's documentation site, in English, labeled "UNDER HEAVY CONSTRUCTION" No complaints, I think it's great that they're trying to make this more accessible to us uni-lingual types.

I use Ubuntu, but I'm currently trying to find a version with proper multi-monitor support (looks like I'm going to regress to 10.04, I know multi-mon works in it), or I may jump ship and move to a different Linux flavor altogether. After 4 solid years using Ubuntu, I think I've crested the Linux learning curve and I'm not at all concerned over moving away from Ubuntu.

For anyone thinking of picking up Linux, Mischa's recommendation of Ubuntu is a good one. Canonical has exerted a great deal of effort to make it as user-friendly and "idiot-proof" as they could. I've watched it improve dramatically over the past 4 years and it's now easier to install than XP in most cases. It's a good distribution to cut your teeth on.

As far as my level of experience goes, I can resolve dependencies, compile from source (with decent instructions ), and have become very used to using the Command Line for some things (a lot of my photo tools are command-line).

dave

Hi Dave,

sounds like you will be fine, then! The multi-monitor issue should be no problem under Ubuntu 10.04 or later, I have been using it myself. If you are running on recent hardware, you should always go for the latest Ubuntu release.

As for the "under heavy construction": that has been written there for years. My colleagues hardly find time to write the documentation. Anyway, that is not the location you should look at. That page describes the core of the theli pipeline, which I am building on with the GUI, but that's about it. All the information you need for can be found under www.astro.uni-bonn.de/~theli/gui
(including short instructions as of how to install that stuff "under heavy construction", which installs pretty easily by the way).

best

mischa

Actually, I've found the multimonitor situation to be stable under 10.04 and unpredictable in any of the later releases. It's a documented bug that has gone unresolved for quite a while, however there are several work-arounds that I have yet to try. Current situation in 10.10 is multimon works, but only in lower resolutions. I've installed 11.04, 11.10 and 10.10 with the same issues. I may just go with 11.10 and use one of the kludges to get it running as opposed to reverting to a 2 year old distro (although 10.04 is a long-term support release).

dave

Compelling! I'm a die hard Linux for Astronomy fan, but am currently stuck on wine for both AutoStakkert2 (Planetary) and Iris (DSO). Only thing I can't do with Iris is decode my .CR2 raws.

I'll still put this on my list of tryouts!

Mike

Note: The latest Ubuntu (11.10) release has a new UI that has proven to be rather divisive. 11.04 gave the option to use the new Unity UI or Gnome 2, while the new distro uses Unity exclusively (however, Gnome 3 can be installed after turnup of the OS). Some like it, some do not. I've gone back to 10.04, and my multimonitor issues have disappeared. Xubuntu does not use unity, and uses the same repositories as Ubuntu, so if you find yourself in the "Unity Stinks" camp, Xubuntu seems like the most painless option.

Edit: Mike, I just saw your reply in the 11.10 thread after I posted this reply, so consider it more a post of general interest to other readers, since you already know what I said ^up there!

dave

Compelling! I'm a die hard Linux for Astronomy fan, but am currently stuck on wine for both AutoStakkert2 (Planetary) and Iris (DSO). Only thing I can't do with Iris is decode my .CR2 raws.

Hi Mike,

Theli has full support for .CR2 raws, using a pretty good kernel for the debayering.

mischa

I had intentions of digging into Theli this weekend, but wound up with the flu and just couldn't grok things. I'm going to try to make time this week to make some sense of things before I ask for any hand-holding

I've got some toys coming in the mail and I'm just itching to do some wide-field.

dave

I am planning to setup Theli and give it a try this weekend.

I was wondering though: how do I stack narrowband images taken with DSLR?

Should I stack images with different filters separately and then combine them or can I just stack all the images together?

The latter option seems to be feasible because based on what I read in Theli manual, it assigns weight to frame based on noise level, so assuming that it does it separately for each color channel, it should work because images with red filter would have high noise level in blue and green channels, thus effectively blue and green channels would be suppressed.
Any comments?

Alex

OK, I gave Theli a try yesterday and today.
Here is my experience so far:
1. Installation under Ubuntu 11.10 was very easy - just followed the instructions on the Theli GUI page
2. My test image was a narrow band Ha California Nebula with 5' exposures and 3200 ISO
3. Got stuck during Astrometry because I did not specify pixel scale while configuring the camera. After some googling found a post on German language forum describing my issue. Thanks to Google translate, was able to figure out what to do and set up pixel scale at 1.26". I think this should be good for my T1i and 750mm scope.
4. It looks like processing happens in single thread, I guess it is because I have only one CCD
5. I am still stuck right now on Astrometry step. I am using 2MASS catalog, specify filter as 'r' (I guess it means red), photometric error as 1. Everything else is default. I am getting this log:


Retrieving photometric reference catalog ...
Retrieving astrometric reference catalog ...
1498 reference sources retrieved.
Detecting sources for abs photometry ...
Calculating 1st pass astrometric solution ...
Correcting astrometry in catalogs ...
Estimating direct absolute zeropoints ...
Adjusting CRVAL1/2 key in header ...
Detecting sources ...
Calculating astrometric solution ...
Collecting image statistics ...
Collecting information for coaddition ...
ERROR (line 5488): Showing you the log ...

Finally this is where the error is:


+ /home/alex/THELI/theli//bin/Linux_64//scamp @/home/alex/.theli/tmp//scamp_cats_31931 -c /home/alex/THELI/theli//gui/reduction/scamp.conf -ASTREFCAT_NAME /home/alex/.theli/scripts/theli_mystd_scamp.cat -CDSCLIENT_EXEC aclient -SAVE_REFCATALOG Y -REFOUT_CATPATH //data2/Astro/work//SCIENCE -NTHREADS 4 -FLAGS_MASK 240 -AHEADER_SUFFIX .ahead -DISTORT_KEYS X_IMAGE,Y_IMAGE -DISTORT_DEGREES 3 -DISTORT_GROUPS 1,1 -POSITION_MAXERR 2.0 -POSANGLE_MAXERR 2.0 -PIXSCALE_MAXERR 1.05 -FGROUP_RADIUS 1.0 -CROSSID_RADIUS 3.1 -ASTRINSTRU_KEY FILTER -PHOTINSTRU_KEY FILTER -SN_THRESHOLDS 5,20 -MOSAIC_TYPE UNCHANGED -STABILITY_TYPE INSTRUMENT -MATCH_FLIPPED N -ASTREF_WEIGHT 1.0

cat ${TEMPDIR}/scamp.log_$$
+ cat /home/alex/.theli/tmp//scamp.log_31931
>
----- SCAMP 1.4.6-MP started on 2012-03-04 at 22:17:03 with 4 threads

>
----- 3 inputs:
> Examining Catalog LIGHT_300s_3200iso_+10c_506stdev_20120103-00h40m02s754ms_B_scamp.cat
> Examining Catalog LIGHT_300s_3200iso_+10c_506stdev_20120103-00h40m02s754ms_G_scamp.cat
> Examining Catalog LIGHT_300s_3200iso_+10c_506stdev_20120103-00h40m02s754ms_R_scamp.cat
LIGHT_300s_3200iso_+10c_506stdev_20120103-00h40m02s754ms_B_scamp.cat: "UNKNOWN " no ext. header 1 set 146 detections
LIGHT_300s_3200iso_+10c_506stdev_20120103-00h40m02s754ms_G_scamp.cat: "UNKNOWN " no ext. header 1 set 148 detections
LIGHT_300s_3200iso_+10c_506stdev_20120103-00h40m02s754ms_R_scamp.cat: "UNKNOWN " no ext. header 1 set 452 detections

----- 746 detections loaded

----- 3 instruments found for astrometry:

Instrument A1 :
FILTER = 'Red '

Instrument A2 :
FILTER = 'Green '

Instrument A3 :
FILTER = 'Blue '

----- 3 instruments found for photometry:

Instrument P1 :
FILTER = 'Red '

Instrument P2 :
FILTER = 'Green '

Instrument P3 :
FILTER = 'Blue '

----- 1 field group found:

Group 1: 3 fields at 00:00:00.20 +00:00:00.6 with radius 60.19'
 instruments center coordinates radius scale
LIGHT_300s_3200iso_+ A3 P3 00:00:00.20 +00:00:00.6 60.19' 1.260"
LIGHT_300s_3200iso_+ A2 P2 00:00:00.20 +00:00:00.6 60.19' 1.260"
LIGHT_300s_3200iso_+ A1 P1 00:00:00.20 +00:00:00.6 60.19' 1.260"

> Making mosaic adjustments...
>
----- Reference catalogs:

> Examining Catalog theli_mystd_scamp.cat

> WARNING: FLAGS parameter not found in catalog theli_mystd_scamp.cat

> Catalog theli_mystd_scamp.cat: Object #0 / 0 samples stored
>
 0 astrometric references loaded from theli_mystd_scamp.cat
>
 Group 1: 0 standard found in file (band mag)
>

----- Astrometric matching:

> Starting field matching...
> Matching field LIGHT_300s_3200iso_+10c_506stdev_20120103-00h40m02s754ms_B_scamp.cat...
> Matching field LIGHT_300s_3200iso_+10c_506stdev_20120103-00h40m02s754ms_G_scamp.cat...
> Matching field LIGHT_300s_3200iso_+10c_506stdev_20120103-00h40m02s754ms_R_scamp.cat...
>
 Group 1: 0 standard in file (band mag)
 instruments pos.angle scale cont. shift cont.
LIGHT_300s_3200iso A3 P3 +0.00 deg 1.260" 0.00 +0.0" +0.0" 0.00
LIGHT_300s_3200iso A2 P2 +0.00 deg 1.260" 0.00 +0.0" +0.0" 0.00
LIGHT_300s_3200iso A1 P1 +0.00 deg 1.260" 0.00 +0.0" +0.0" 0.00

> Making preliminary cross-identifications in group 1
> Generating group plots...
> Initializing detection weight factors...
> Initializing the global astrometry matrix...
> Starting field matching...
> Filling the global astrometry matrix: group 1/1, field 1/3
> Filling the global astrometry matrix: group 1/1, field 2/3
> Filling the global astrometry matrix: group 1/1, field 3/3
> Fixing the degrees of freedom
> Solving the global astrometry matrix...

> WARNING: Significant inaccuracy likely to occur in projection


> WARNING: Significant inaccuracy likely to occur in projection


> WARNING: Significant inaccuracy likely to occur in projection


> WARNING: Significant inaccuracy likely to occur in projection


> WARNING: Significant inaccuracy likely to occur in projection


> WARNING: Significant inaccuracy likely to occur in projection

>

----- Astrometric clipping:

> Making cross-identifications in group 1
> Computing astrometric stats for group 1
> Astrometric clipping in group 1
>
Group 1: 0/0 detections removed
> Initializing detection weight factors...
> Initializing the global astrometry matrix...
> Starting field matching...
> Filling the global astrometry matrix: group 1/1, field 1/3
> Filling the global astrometry matrix: group 1/1, field 2/3
> Filling the global astrometry matrix: group 1/1, field 3/3
> Fixing the degrees of freedom

> WARNING: Not enough matched detections in instrument A3


> WARNING: Not enough matched detections in instrument A2


> WARNING: Not enough matched detections in instrument A1

> Solving the global astrometry matrix...
>

----- Astrometric stats (internal) :

 All detections | High S/N 
 dAXIS1 dAXIS2 chi2 ndets | dAXIS1 dAXIS2 chi2 ndets
> Making cross-identifications in group 1
>

----- Astrometric stats (external):

 All detections | High S/N 
 dAXIS1 dAXIS2 chi2 nstars | dAXIS1 dAXIS2 chi2 nstars
Group 1: 0" 0" 0 0 0" 0" 0 0
> Generating astrometric plots...
> Initializing the global photometry matrix...
> Solving the global photometry matrix...
> Initializing the global photometry matrix...
> Solving the global photometry matrix...
> Initializing the global photometry matrix...
> Solving the global photometry matrix...
>

----- Photometric clipping:

> Computing photometric stats for group 1 / P1
> Computing photometric stats for group 1 / P2
> Computing photometric stats for group 1 / P3
> Initializing the global photometry matrix...
> Solving the global photometry matrix...
> Initializing the global photometry matrix...
> Solving the global photometry matrix...
> Initializing the global photometry matrix...
> Solving the global photometry matrix...
>

----- Photometric stats (internal):

 All detections | High S/N 
 Instru mag RMS chi2 ndets | mag RMS chi2 ndets
>

----- Photometric stats (external):

 All detections | High S/N 
 Instru mag RMS chi2 nstars | mag RMS chi2 nstars
> Generating photometric plots...
> Computing colour shifts in group 1
> Computing proper motions in group 1
> Computing proper motions...
> Generating proper-motion plots...
> Saving image headers...
> Saving merged catalog for group 1
> Cleaning up...
>
> All done (in 10 s)

Hi Alex

I'm not an expert in Theli, but with my pictures it works very well. I think, a narrow band image is not really good for a first try, normal rgb images would be better.

A pixelscale calculator: http://www.cmoos.de/...=4.7&pixelsize=

My settings in Astro/Photometry:

catalog: USNO Mag-limit: 15

Absolute Photometry (direct) and (indirect): unchecked

create source cat: checked and default settings

Astro+photometry: checked,
settings:
posangle maxerror: 180
position maxerror: 10-20
match flipped images: checked

other settings: default !!!

I hope it helps

Thomas

Alex,

it does not make much sense to stack different narrow-band filters into one coadded FITS image. You should coadd different filters separately.

Concerning (4): THELI uses full parallelisation only for multi-chip cameras. All amateur cameras are single-chip cameras. For single-chip cameras parallelisation is used during the astrometric solution, sky subtraction and coaddition.

As for (5): You do NOT want to run any of the absolute photometric calibration. Differences in atmospheric transparency (i.e. relative photometric calibration) etc are automatically done when doing the astrometry.

The other hints posted by tommy_h sound very reasonable to me. Have a look at the plots/fgroups_1.png check-plot. This should look reasonable (same orientation for exposures, many green dots inside). Examples can be found here:

http://www.astro.uni...amp-check-plots

mischa

Mischa, Thomas,
Thanks for your help!
I followed your suggestions but I am still not getting past the errors while doing astro+photometry.
My plot looks like this now:
-frames are nicely aligned
-inside the frames I have mostly green crosses, about 300
-I also have some red squares inside and outside, about 10
-there are few, maybe 20 green squares inside frames

What does this plot mean?

Another question is: what do green squares and crosses mean?


Alex

Alex:

green crosses: an object detected in the image and used for astrometry
green diamond: an object detected, matched with an object in the reference catalog
red square: a reference source, not matched by any detections in the images.

your plot looks fine to me, but it appears that you could choose a deeper catalog, or increase the magnitude limit by 2 magnitudes.

how do you know that "it didn't work"? looks fine to me!

mischa

I hook up on this older posting.

I´m going to do a presentation about a standard workflow in Theli at CEDIC in Linz/Austria at the beginning of March. I really will try hard to do a full presentation in english for my Youtube Channel as well. So stay tuned.

CS Frank