Jump to content

  •  

CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.

Photo

New plate solver program

astrophotography
  • Please log in to reply
48 replies to this topic

#26 catalogman

catalogman

    Viking 1

  • *****
  • Posts: 998
  • Joined: 25 Nov 2014

Posted 04 December 2018 - 01:39 AM

Much of what you say is echoed in this Astrometry.net thread:

 

https://groups.googl...try/hGa74stffzE

 

This user noted that the 4100 indexes (Tycho-2) solved his image faster than the 4200 indexes (2-MASS).  He further improved the number of 4100 matches by building a Vt mag version of Tycho-2 rather than the available Bt version.

 

ASTAP could be improved in the same way: include a utility (similar to CatGen in Cartes du Ciel) that allows the user to convert .txt catalogue files to .290 binaries. If an image doesn't solve in G17, then the user can build another catalogue to try. This custom catalogue feature would make ASTAP a much more versatile program that could solve almost any image.

 

--catalogman


Edited by catalogman, 04 December 2018 - 08:51 AM.


#27 han59

han59

    Mariner 2

  • *****
  • topic starter
  • Posts: 285
  • Joined: 09 Mar 2015
  • Loc: the Netherlands

Posted 05 December 2018 - 06:33 AM

In ASTAP version 0.9.142 released today, I have added the possibility to force the star database. Below to number of matches for some of my images:

 

ASTAP test different databases.png

 

The two Gaia databases perform different but very similar. The UCAC4 U16 behaves a little poorer. Exposure time was between 50 and 200 seconds.  The U16 is no longer provided since it was replaced by Gaia. In principle you could also use the TYC++ and TUC (UCAC4 and Tycho hybrid) but they are in 290-9, 290-10 format and I removed that support in ASTAP to save some code. So they will not work. The program goes for my images typical down to magnitude 12-13 so the Tycho with magnitude limit 12 will most likely not work well. ASTAP uses brute force for solving rather then index files as Astrometry.net and changing the database is easy.

 

You could try yourself if this performance is similar for your camera. Note you can select "show extended solver log" in tab alignment.

 

Creating a new star database is a huge task and will take 24 hours CPU time. I have still 60 gbytes files from the Gaia dr2 on my disk. In principle I could reasonable quick create a new version with a specific magnitude value calculated using the three magnitudes (G, B, R) provided. My computer will be one day crunching till it is ready.

 

The G16 (mag 16) contains also the Gaia blue en red difference. At the moment the only thing with this value is done is to colour the star annotation. In principle this could be used if the program detects a blue filter in the image header but it will be very exotic. There is no reason to image in other band then luminance for astrometric solving.

 

Probably the only thing I will do is to create a special G17 (mag 17) star database adding this blue-red difference as in the G16. Main purpose is for the HNSKY planetarium program to colour the stars up to magnitude 17. The penalty will be an 20% increase in file size. Note that Carte du Ciel can read/use these star database files. A G18 (mag 18) with colour info is also possible.

 

Later:

  I'm have uploaded the u16 (mag16) database again at sourceforge for testing. It is a hybrid of Tycho2 and UCAC4 without abbreviations.

 

Han


Edited by han59, 05 December 2018 - 09:47 AM.

  • power1001 and artem2 like this

#28 catalogman

catalogman

    Viking 1

  • *****
  • Posts: 998
  • Joined: 25 Nov 2014

Posted 05 December 2018 - 07:12 PM

My image solves quickly in ansvr/AstroImageJ using any one of the much smaller YBS, HIP, Sky2000 Master, ASCC, or Tycho-2 catalogues. All magnitudes are in the V-band.

 

My image does NOT solve in ansvr/AstroImageJ using the large 2MASS catalogue. Magnitudes are in the G-band.

 

---

 

To solve my image in ASTAP, there needs to be both a .jpg and a .fit of the same image in the directory. If there is only a .jpg, ansvr aborts and complains about the missing .fit:

 

augment-xylist.c:585:backtick Failed to run command: /usr/lib/astrometry/bin/image2pnm.py --sanitized-fits-outfile /tmp/tmp.sanitized.aOuVly --fix-sdss --infile C:/Program\ Files/astapwin32/orion.fit --uncompressed-outfile /tmp/tmp.uncompressed.rIiaHL --outfile /tmp/tmp.ppm.OXhBLp --ppm
  ioutils.c:605:run_command_get_outputs Command failed: return value 255

 

But the .fit needed by the program and created with GIMP is poorly rendered. The same ansvr indexes listed above (used in ASTAP's "use Local Astrometry.net" option) can't find a solution of this poorly rendered image.

 

If ASTAP were fixed so that it samples from only the .jpg, then it should be able to solve the image.

 

--catalogman


Edited by catalogman, 05 December 2018 - 07:16 PM.


#29 han59

han59

    Mariner 2

  • *****
  • topic starter
  • Posts: 285
  • Joined: 09 Mar 2015
  • Loc: the Netherlands

Posted 06 December 2018 - 10:09 AM

.

 

If ASTAP were fixed so that it samples from only the .jpg, then it should be able to solve the image.

 

There is no need to convert a JPG file to a FITS file. This goes fully automatic in memory. The only thing you should do is to set the binning to 2 or 3 for large images (see red marked screenshot below) and set the approximate height of the image in degrees once.

 

1) Load the JPG file in the ASTAP viewer.

2) Enter the approximate position in the viewer.

3) Hit one of the two solve buttons.  (viewer solve button or tab alignment, button "find astrometric solution")

 

If possible load a raw file like CR2 or FITS file. That is better.

 

You could do the same in the ASTAP command line.

 

Han

 

binning.png


Edited by han59, 06 December 2018 - 12:29 PM.

  • artem2 likes this

#30 catalogman

catalogman

    Viking 1

  • *****
  • Posts: 998
  • Joined: 25 Nov 2014

Posted 06 December 2018 - 11:15 PM

If I'm reading the menu correctly, changing the binning shouldn't have any effect when selecting the "Use local Astrometry.net" bullet.

 

When I start the ansvr server, the attachments show what ASTAP's display looks like. Toggling the binning has no effect.

 

--catalogman

Attached Thumbnails

  • orion_jpg.jpg
  • orion_fit.JPG


#31 han59

han59

    Mariner 2

  • *****
  • topic starter
  • Posts: 285
  • Joined: 09 Mar 2015
  • Loc: the Netherlands

Posted 07 December 2018 - 06:54 AM

The binning for the external local Astrometry.net access is configured by text commands in the extra options. This is below marked with a gray ellipse. (--downsample2). I agree it could be confusing and in next version 0.9.143 I will gray out not relevant options depending on the solver selection.

 

The green area contains the relevant settings of the ASTAP internal solver.

 

The red area contains the relevant settings for external access to a local Astrometry.net.

 

The yellow marked settings are only relevant if the program is used as a command line PlateSolve2 substitute. If the "convert to FITS" option is checked, a new solved FITS will be produced intended for displaying in a Planetarium map The binning in the yellow marked area is to keep the new FITS file compact and is not used for solving.

 

Han

astap_settings.png


  • artem2 likes this

#32 catalogman

catalogman

    Viking 1

  • *****
  • Posts: 998
  • Joined: 25 Nov 2014

Posted 11 December 2018 - 12:01 AM

As AstroImageJ opens my image, it reduces the resolution to 8-bit graphics.

 

Could that be why AIJ is able to solve my image but ASTAP cannot?

 

(The downsize option has no effect, and ASTAP still complains that it accepts only a .fits).

 

--catalogman



#33 han59

han59

    Mariner 2

  • *****
  • topic starter
  • Posts: 285
  • Joined: 09 Mar 2015
  • Loc: the Netherlands

Posted 11 December 2018 - 05:02 AM

JPG, TIF, PNG and BMP images will be processed as floating-point arrays but read as 8 bit per colour. The read routine has a 8 bit limitation so even 16 bit PNG or  TIFF will be read as 8 bit. This is no problem as long the image has enough signal. So the full bit depth range is used and the brightest stars are near or in saturation.

 

Raw files (typical 12 bits) and FITS (typical 16 bits) will be read in their native bit depth and also processed as floating-point arrays . Since the original bit depth is higher the signal level is less critical and solving more reliable.

 

>>>(The downsize option has no effect, and ASTAP still complains that it accepts only a .fits).

?? How are you using ASTAP.? JPG, TIF, PNG and BMP should be processed without problems.

 

Han


  • artem2 likes this

#34 catalogman

catalogman

    Viking 1

  • *****
  • Posts: 998
  • Joined: 25 Nov 2014

Posted 13 December 2018 - 12:48 AM

Except for tweaking with the downsampling, all of the ASTAP settings are the default settings. The only other change is selecting the astrometry.net bullet and turning on the ansvr server (start_ansvr).

 

The first screenshot (in #30 above) shows ASTAP demanding a .fit even though a .jpg has been loaded. ASTAP does this with every .jpg that I have tested and with both Win7 x32 and Win7 x64, so the problem is not with my initial image. The problem is that ASTAP refuses to read the .jpg while the local ansvr server is running. (I'm running the Java-free version of ansvr.)

 

With options other than astrometry.net, ASTAP tries to solve the initial image with its large catalogue and then gives up after only a few seconds. OTOH, ansvr with AstroImageJ quickly solved the same image with runs in five different and much smaller catalogues.

 

--catalogman


Edited by catalogman, 13 December 2018 - 12:50 AM.


#35 han59

han59

    Mariner 2

  • *****
  • topic starter
  • Posts: 285
  • Joined: 09 Mar 2015
  • Loc: the Netherlands

Posted 13 December 2018 - 05:40 AM

Now it becomes clear.

 

1) Solving a jpeg using the external Astrometry.net didn't work. The program was default converting the jpeg to an unsaved FITS (array) for the internal solver. Unsaved means not available for Astrometry.net. This conversion should only happen for the internal solver. This is now fixed in v0.9.145.

 

2) The option solving a JPEG using the internal solver was also broken due to a last minute change. Also fixed in version v0.9.145.

 

If your using ASTAP 32 bit version, try the 64 bit version if possible. The 64 bit version is almost twice as fast.

 

Han


Edited by han59, 13 December 2018 - 07:50 AM.


#36 catalogman

catalogman

    Viking 1

  • *****
  • Posts: 998
  • Joined: 25 Nov 2014

Posted 14 December 2018 - 04:37 AM

Thanks for the quick fix. 

 

I made a test with the V-mag edition of the Tycho-2 catalogue:

 

A sample image taken with the Aladin viewer solved in ASTAP/ansvr in only a couple of seconds. 

 

BUT using the same catalogue in ASTAP/ansvr still fails to solve my original image.

 

Why?  The Aladin image is much more pixelated, so it should be the one that fails to solve.

 

OTOH, the AstroImageJ/ansvr combination can solve both images.

 

--catalogman



#37 han59

han59

    Mariner 2

  • *****
  • topic starter
  • Posts: 285
  • Joined: 09 Mar 2015
  • Loc: the Netherlands

Posted 21 December 2018 - 12:22 PM

I have added a new option to show the distortion of an image in version 0.9.147. The lines show the 50xoffset of the image stars and the Gaia star database.

 

This could help to analyze why some images are difficult to solve. My astrograph images are almost perfect but this one shows severe distortion. You could use either the internal solver and the astrometry.net solver.

 

Usage:

  1) Load image and click solve.

  2) Select distortion tool.

 

Han

 

M45 image with optical distortions.jpg

 


  • Rickster, artem2 and bsavoie like this

#38 bsavoie

bsavoie

    Viking 1

  • *****
  • Posts: 613
  • Joined: 30 Nov 2013
  • Loc: North Alabama 35747

Posted 21 December 2018 - 07:35 PM

so correct me if I understand this incorrectly.. the errors that show up on a sky photo are blown up by a factor of 50. This shows mostly air flow, which makes the stars wiggle.

 

I am hopping to find a math expression to describe the quality of a telescope. But if air currents have this large effect, then they swamp the differences left by mirror grinding. It seems to me that much of astronomy is built on hands on experience of looking through an eyepiece. There are schools that believe in Reflectors and another that has Refactors..  but the air flows are so diverse, quality characterization is mostly a religion. People believe what they believe, and there isn't a math we can do that will make it into a science.

 

Of course, color might add enough new information to again point at the telescope. Does wind affect all colors the same? Perhaps filtering by three different colors would produce different distortions which could get beyond wind distortion..

 

Just thinking out loud..

 

Bill



#39 han59

han59

    Mariner 2

  • *****
  • topic starter
  • Posts: 285
  • Joined: 09 Mar 2015
  • Loc: the Netherlands

Posted 22 December 2018 - 09:46 AM

so correct me if I understand this incorrectly.. the errors that show up on a sky photo are blown up by a factor of 50. This shows mostly air flow, which makes the stars wiggle.

Yes the errors are magnified. But due to long exposure, the seeing effects seems to be mostly nullified.  So this could be used to judge the distortion of the optics. An other quality measurement would be the HFD/ FWHM values of the imaged stars. You will need a good sharp image to do this.

 

I'm not the only one experimenting with this:

 

ESO:
https://arxiv.org/pdf/1401.3344.pdf

IRIS, astrometric correction
http://www.astrosurf...0/new550_us.htm

PI distortion correction:

http://www.pixinsigh...tion/index.html

 

 

This measurement could be used to correct an image for distortion for astrometric purposes or mosaic building. I'm not sure if this is practical useful for amateurs. Unless it is very wide field optics, you probably better off with buying good optics.

 

The tool is currently experimental. Here an other example of an image with some minor problems in the corners. there are also some random arrows in the middle. Maybe they are close pair double stars. I didn't investigate.

 

 

 

M27  with distortion.jpg

 

Han


Edited by han59, 22 December 2018 - 11:25 AM.

  • bsavoie likes this

#40 Fusionhead

Fusionhead

    Lift Off

  • -----
  • Posts: 12
  • Joined: 19 Nov 2015

Posted 09 March 2019 - 03:54 PM

Han,

 

This looks fantastic.  Do you have any plans to offer a Fedora RPM installer package?  I have other options if you don't, such as windows installer on wine, or try to convert the deb package, but those are a bit messy. I thought I'd ask before trying those.

 

Thanks for your hard work and making this available to the public!

 

Roy



#41 Bob Denny

Bob Denny

    Vendor (DC-3 Dreams)

  • -----
  • Vendors
  • Posts: 128
  • Joined: 17 Mar 2009
  • Loc: Mesa AZ USA

Posted 09 March 2019 - 07:52 PM

@han59, this is an interesting discussion as I have "some experience" with solving images with severe optical distortion several years before the publication of the work of Calabretta (et al.) in 2004, and of Shupe (et al.) for the SIP Convention in 2005. I have to give you a great compliment for the work you are doing.

 

My exposure was way back around 2001 when asteroid hunting was big. One client had four modified Newtonian telescopes which had some spectacular sin(x)/x and off-center distortion. The reported positions derived from solutions that minimized residuals against WCS planar projections was so far off that the MPC rejected the reports! I was so new at all of this... what a surprise. Two-dimensional high order polynomials and singular value decomposition later, what a difference! Having come from the world of communication and flight control, then early web serving, it was a real learning experience. 

 

I have a great appreciation for the work being done by people like you. I just helped a guy 'arie' recover from a botched Windows uninstall of my software. He was kind enough to tell me that he planned to use ASTAP instead. Congratulations on such good work.



#42 han59

han59

    Mariner 2

  • *****
  • topic starter
  • Posts: 285
  • Joined: 09 Mar 2015
  • Loc: the Netherlands

Posted 13 March 2019 - 02:53 AM

 

 

This looks fantastic.  Do you have any plans to offer a Fedora RPM installer package?  I have other options if you don't, such as windows installer on wine, or try to convert the deb package, but those are a bit messy. I thought I'd ask before trying those.

Roy, I have no experience with Fedora. i assumed it could install deb packages. Today i will try to convert it to rpm or make a rpm package. Could you test the resulting rpm on your system?

 

@han59, this is an interesting discussion as I have "some experience" with solving images with severe optical distortion several years before the publication of

Bob, thanks. P.s  you have been “leading by example” with the first successful solver and that also inspired me. smile.gif

 

--Han


  • Bob Denny likes this

#43 descott12

descott12

    Apollo

  • -----
  • Posts: 1075
  • Joined: 28 Sep 2018
  • Loc: Charlotte, NC

Posted 15 March 2019 - 12:14 PM

It is open source and written in Object Pascal ( Lazarus& FPC). I have documented some principles at the webpage.

 

 

How does it run on multiple platforms? I didn't realize you could develop on multiple platforms with Object Pascal. What developement tools are you using? I am a software developer as well. I have a windows app that I would love to port to other platforms. While I do all my work in C++, I remember really liking Pascal so this might work well.



#44 han59

han59

    Mariner 2

  • *****
  • topic starter
  • Posts: 285
  • Joined: 09 Mar 2015
  • Loc: the Netherlands

Posted 15 March 2019 - 05:03 PM

ASTAP is available for Windows, Linux AMD64,  Linux Raspberry-Pi and MacOS 386 all compiled from a single code in Object Pascal. The code is developed in the Lazarus IDE with the integrated FPC compiler. All freeware and Delphi compatible.

 

I never like C and stayed with Pascal. I assume Pascal is better suited for complicated stuff then C++. Speed is very similar. Furthermore the executables are standalone and don't need something awkward like .NET or other Microsoft libraries.

 

I develop under Windows and just copy the code to a virtual machine running Linux or MacOs or to Raspberry-Pi hardware. Load it in dedicated Lazarus IDE and just compile it. Alternatively you could also develop in a Linux (or MacOS but more limited) system as you like. An other possibility is to compile directly from the Windows IDE (or Linux) directly to different operating systems & CPU's but that's is a little complicated to setup. Patrick Chevalley is doing that for CDC and CCDCiel.

 

I'm not a software developer by profession, but noted that for example Pixinsight and APP are using Java to make it multi platform. The single executables produced by Lazarus/FPC is a better solution. So Lazarus/FPC is a great and free solution for multiple platform development.

 

Hopefully you could make some new nice astro applications.

 

--Han



#45 descott12

descott12

    Apollo

  • -----
  • Posts: 1075
  • Joined: 28 Sep 2018
  • Loc: Charlotte, NC

Posted 15 March 2019 - 05:49 PM

ASTAP is available for Windows, Linux AMD64,  Linux Raspberry-Pi and MacOS 386 all compiled from a single code in Object Pascal. The code is developed in the Lazarus IDE with the integrated FPC compiler. All freeware and Delphi compatible.

 

I never like C and stayed with Pascal. I assume Pascal is better suited for complicated stuff then C++. Speed is very similar. Furthermore the executables are standalone and don't need something awkward like .NET or other Microsoft libraries.

 

I develop under Windows and just copy the code to a virtual machine running Linux or MacOs or to Raspberry-Pi hardware. Load it in dedicated Lazarus IDE and just compile it. Alternatively you could also develop in a Linux (or MacOS but more limited) system as you like. An other possibility is to compile directly from the Windows IDE (or Linux) directly to different operating systems & CPU's but that's is a little complicated to setup. Patrick Chevalley is doing that for CDC and CCDCiel.

 

I'm not a software developer by profession, but noted that for example Pixinsight and APP are using Java to make it multi platform. The single executables produced by Lazarus/FPC is a better solution. So Lazarus/FPC is a great and free solution for multiple platform development.

 

Hopefully you could make some new nice astro applications.

 

--Han

Thanks for all the information. Very helpful. Actually I despise .NET and I don't use that for anything. I use straight C\C++ in Visual studio and it results in a pretty compact package with just 3 dll's. But it is completely unportable so I really like what you have described and I think I will give it a try. I am also surprised to hear that PixInsight is java because java is typically so horribly slow...I wonder how they made that work?? Anyway, thanks alot.



#46 han59

han59

    Mariner 2

  • *****
  • topic starter
  • Posts: 285
  • Joined: 09 Mar 2015
  • Loc: the Netherlands

Posted 16 March 2019 - 02:47 AM

To be correct, PI seems to be written in C++ and Java according some info in GitHub.  I assume they write/build several tools accessible by command line. I see something similar in IRIS or Astrometry.net where several modules are written in C and some in Python2. It is a different programming approach I'm not used to. I prefer all in one IDE. Success Han



#47 Patrick Chevalley

Patrick Chevalley

    Explorer 1

  • -----
  • Vendors
  • Posts: 64
  • Joined: 04 Jul 2017

Posted 16 March 2019 - 03:56 AM

I also like the way FreePascal/Lazarus make it easy to develop cross-platform application. What I like the most is you can concentrate your efforts on the functionality without losing time with technical difficulties.

 

But if you prefer to continue with C++ the wxWidgets library provide good cross-platform functionality. See https://www.wxwidgets.org/

This is use for the PHD2 software for example.


Edited by Patrick Chevalley, 16 March 2019 - 03:57 AM.


#48 descott12

descott12

    Apollo

  • -----
  • Posts: 1075
  • Joined: 28 Sep 2018
  • Loc: Charlotte, NC

Posted 16 March 2019 - 07:21 AM

Yes, I have been using WxWidgets for years. That is what my main app is built in and it does work well. Previous attempts to port the OSX didn't go to well but I think I probably gave up too quickly and the Mac market is so small compared to Windows that it didn't seem so urgent.  I really do love C++. It is a beautiful language - very low level and fast but also object-oriented. For me, it is the best of both worlds.

 

My primary goal is to port to iOS and wxWidgets doesn't support that very well. After looking at Lazarus, I don't it does either, at least not yet.  But Lazarus really does look quite powerful and mature so I think I may tinker with it. I started out with Borland Turbo Pascal back in the 80's and remember really liking it.



#49 han59

han59

    Mariner 2

  • *****
  • topic starter
  • Posts: 285
  • Joined: 09 Mar 2015
  • Loc: the Netherlands

Posted 01 May 2019 - 05:36 AM

In the latest ASTAP version, the annotation has been extended with a HyperLeda galaxy database of 2.440.000 objects. Below a demonstration of the number of galaxies around global cluster M5. Most are recognizable in the image.

 

I hope this new addition will be useful to recognize faint fuzzies.

 

Han

 

Image data:

M5 global cluster

Telescope 100 mm APO astrograph APO100Q, F5,8
Camera ASI1600MM-Cool
Exposure: 28 x 50 sec
Date: 2019-4-28

Solved and annotated with ASTAP

 

A less compressed version of the image is available here

 

M5_50s_20190429_003652  253xFD  191xF  90xD  0x0R  0x0G  0x0B  0x0RGB  28x50L  _annotated cropped.jpg


  • dghent, artem2, bsavoie and 1 other like this


CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.


Recent Topics







Cloudy Nights LLC
Cloudy Nights Sponsor: Astronomics