Jump to content

  •  

CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.

Photo

Astrometry-api-lite - local astrometry.net API with installer

  • Please log in to reply
22 replies to this topic

#1 jusasi

jusasi

    Sputnik

  • -----
  • topic starter
  • Posts: 26
  • Joined: 04 Mar 2018

Posted 04 April 2018 - 11:46 AM

Hello there!

 

I thought I'd drop by here too and let you know that I've been working hard to bring the Astrometry-api-lite to the community.

 

So what is the Astrometry-api-lite? It's a local alternative to using http://nova.astrometry.net for your plate solving purposes. The project's aim is to make an easily locally installable, alternative lightweight API to the Astrometry.net full site and suite. You can then use it from your astronomy software, like KStars/Ekos for example. It's fully open source, and the code is available in Github, https://github.com/J...ometry-api-lite. It runs on Linux as well as Windows 10 Subsystem for Linux.

 

It originally started from my own needs; needing to use the API, but with a spotty and slow internet connection it wasn't ideal - and sometimes the whole Nova was down and there was nothing I could do. So I ended up writing a minimal version of the API, that clones Nova API functionality but leaves all the other fluff behind. So it's effectively a plate solver API born out of need.

 

It has been out and about for a while, but today I finally finished the thing most people were probably craving for: the Windows installer. The installer now installs everything you need in order for the API to run, as well as a shortcut to your desktop to run the API whenever you need it. The only manual thing left to do on your own is to install the Linux Subsystem from Windows Store before you run the installer.

 

In addition I added a simple Dashboard to it where you can monitor the status of the latest solver jobs as well as cancel jobs if need be.

 

You can download the installer here: https://github.com/J...i-lite/releases

 

I leave you with a few screenshots to show off, because screenshots are nice smile.gif

 

LghLupu.png

 

mNOHrk1.png

 

xukSm4U.png

 

Best Wishes,

 

- Jussi


  • psandelle, Oleg Astro, barbarosa and 5 others like this

#2 lambermo

lambermo

    Apollo

  • -----
  • Posts: 1,220
  • Joined: 16 Jul 2007
  • Loc: .nl

Posted 04 April 2018 - 02:41 PM

Hey Jussi,

a Linux user here, I've been running your Astrometry-api-lite intended for the new MountWizzard alignment program for 10micron mounts which is still under development. I've been looking to contact you for a while now with some questions ;-)

Does astrometry-api-lite require that such a program logs in first ?

I'd like to test astrometry-api-lite's access to the local astrometry.net installation, what's the simplest way to do so ?

-- Hans



#3 jusasi

jusasi

    Sputnik

  • -----
  • topic starter
  • Posts: 26
  • Joined: 04 Mar 2018

Posted 04 April 2018 - 03:12 PM

Hey Jussi,

a Linux user here, I've been running your Astrometry-api-lite intended for the new MountWizzard alignment program for 10micron mounts which is still under development. I've been looking to contact you for a while now with some questions ;-)

Does astrometry-api-lite require that such a program logs in first ?

I'd like to test astrometry-api-lite's access to the local astrometry.net installation, what's the simplest way to do so ?

-- Hans

Hi Hans!

 

  • Astrometry-api-lite follows the astrometry.net API convention, so there is the /api/login endpoint that accepts the apikey - however using it is completely optional, the apikey isn't used for anything. The login doesn't really do anything, and it's only there to satisfy other programs so that no error is generated if a login call is attempted.
  • If you have astrometry.net installed locally, then everything should work without a hitch. The worker program runs 'solve-field' and 'wcsinfo', so if astrometry.net is properly installed it can find them.

I hope that answered your question :)

 

- Jussi



#4 lambermo

lambermo

    Apollo

  • -----
  • Posts: 1,220
  • Joined: 16 Jul 2007
  • Loc: .nl

Posted 05 April 2018 - 04:14 PM

Great thanks. I already assumed the /login was a stub :)

 

Any ideas what causes this '500 internal server error' ?  (part of a tcpdump capture)

POST /api/upload HTTP/1.1
Host: 192.168.100.17:8080
Accept: */*
User-Agent: python-requests/2.18.4
Accept-Encoding: gzip, deflate
Connection: keep-alive
Content-Type: multipart/form-data; boundary=13dd9c1834b44d159bc32793c11e9ff3
Content-Length: 913685

Content-Disposition: form-data; name="request-json"

{"use_sextractor": false, "scale_est": 1.5913783783783781, "allow_commercial_use": "d", "allow_modifications": "d", "publicly_v
e": "n", "center_ra": 198.36237500000001, "center_dec": 59.06136111111111, "crpix_center": true, "parity": 2, "downsample_facto
.0, "session": 12345, "scale_units": "arcsecperpix", "scale_err": 50, "radius": 2, "scale_type": "ev"}
Content-Disposition: form-data; name="file"; filename="/home/hans/mountwizzard-3.0a16/mountwizzard/images/2018-03-24-19-53-41/M
Image_000.fit"
Content-Type: application/octet-stream
length 244: HTTP: HTTP/1.1 500 Internal Server Error

I saw no log file or anything on stdout/stderr.

 

-- Hans



#5 jusasi

jusasi

    Sputnik

  • -----
  • topic starter
  • Posts: 26
  • Joined: 04 Mar 2018

Posted 05 April 2018 - 04:48 PM

Great thanks. I already assumed the /login was a stub smile.gif

 

Any ideas what causes this '500 internal server error' ?  (part of a tcpdump capture)

POST /api/upload HTTP/1.1
Host: 192.168.100.17:8080
Accept: */*
User-Agent: python-requests/2.18.4
Accept-Encoding: gzip, deflate
Connection: keep-alive
Content-Type: multipart/form-data; boundary=13dd9c1834b44d159bc32793c11e9ff3
Content-Length: 913685

Content-Disposition: form-data; name="request-json"

{"use_sextractor": false, "scale_est": 1.5913783783783781, "allow_commercial_use": "d", "allow_modifications": "d", "publicly_v
e": "n", "center_ra": 198.36237500000001, "center_dec": 59.06136111111111, "crpix_center": true, "parity": 2, "downsample_facto
.0, "session": 12345, "scale_units": "arcsecperpix", "scale_err": 50, "radius": 2, "scale_type": "ev"}
Content-Disposition: form-data; name="file"; filename="/home/hans/mountwizzard-3.0a16/mountwizzard/images/2018-03-24-19-53-41/M
Image_000.fit"
Content-Type: application/octet-stream
length 244: HTTP: HTTP/1.1 500 Internal Server Error

I saw no log file or anything on stdout/stderr.

 

-- Hans

Can you please post the full request? This one has line endings truncated.

 

One thing that did catch my eye is that "session" is a number, and a string is expected - I can confirm that it will throw an error. Try putting quotes around it and let me know if that helped. If not, please post a full request and I'll happily debug it to see what's going on.

 

The schema for the request is as follows:

 

SZ8aE5g.png

 

- Jussi



#6 lambermo

lambermo

    Apollo

  • -----
  • Posts: 1,220
  • Joined: 16 Jul 2007
  • Loc: .nl

Posted 06 April 2018 - 04:35 PM

Thanks for looking into it. I don't have the full tcpdump anymore but I'll test again once some repaired gear is returned.

-- Hans



#7 Oleg Astro

Oleg Astro

    Astronomy Linux

  • -----
  • Vendors
  • Posts: 503
  • Joined: 12 Aug 2008
  • Loc: Albufeira, Portugal

Posted 06 April 2018 - 04:46 PM

What does it mean?

oleh@oleh:~$ service astrometry-api-lite-manager status
● astrometry-api-lite-manager.service - Astrometry.net lite API worker manager
   Loaded: loaded (/etc/systemd/system/astrometry-api-lite-manager.service; enabled; vendor preset: enabled)
   Active: activating (auto-restart) (Result: exit-code) since Fri 2018-04-06 22:45:18 WEST; 476ms ago
  Process: 1583 ExecStart=/usr/bin/node /opt/astrometry-api-lite/dist/manager/main.js (code=exited, status=217/USER)
Main PID: 1583 (code=exited, status=217/USER)
Apr 06 22:45:18 oleh systemd[1]: astrometry-api-lite-manager.service: Main process exited, code=exited, status=217/USER
Apr 06 22:45:18 oleh systemd[1]: astrometry-api-lite-manager.service: Unit entered failed state.
Apr 06 22:45:18 oleh systemd[1]: astrometry-api-lite-manager.service: Failed with result 'exit-code'.
oleh@oleh:~$


#8 jusasi

jusasi

    Sputnik

  • -----
  • topic starter
  • Posts: 26
  • Joined: 04 Mar 2018

Posted 07 April 2018 - 06:14 AM

 

What does it mean?

oleh@oleh:~$ service astrometry-api-lite-manager status
● astrometry-api-lite-manager.service - Astrometry.net lite API worker manager
   Loaded: loaded (/etc/systemd/system/astrometry-api-lite-manager.service; enabled; vendor preset: enabled)
   Active: activating (auto-restart) (Result: exit-code) since Fri 2018-04-06 22:45:18 WEST; 476ms ago
  Process: 1583 ExecStart=/usr/bin/node /opt/astrometry-api-lite/dist/manager/main.js (code=exited, status=217/USER)
Main PID: 1583 (code=exited, status=217/USER)
Apr 06 22:45:18 oleh systemd[1]: astrometry-api-lite-manager.service: Main process exited, code=exited, status=217/USER
Apr 06 22:45:18 oleh systemd[1]: astrometry-api-lite-manager.service: Unit entered failed state.
Apr 06 22:45:18 oleh systemd[1]: astrometry-api-lite-manager.service: Failed with result 'exit-code'.
oleh@oleh:~$

Looks like the service won't run. I suspect the service configuration is incorrect. Can you run the manager with:

cd /opt/astrometry-api-lite
node dist/manager/main.js

If you can, then you should check that the configuration in astrometry-api-lite-manager.service is correct. If this fails, you'll probably at least get a more sensible error message out.

 

My checklist would be:

- does /usr/bin/node exist (ie. is NodeJS installed?)

- is the working directory set correct?

- is the User correct?


Edited by jusasi, 07 April 2018 - 06:52 AM.

  • Oleg Astro likes this

#9 Oleg Astro

Oleg Astro

    Astronomy Linux

  • -----
  • Vendors
  • Posts: 503
  • Joined: 12 Aug 2008
  • Loc: Albufeira, Portugal

Posted 07 April 2018 - 01:53 PM

1. /usr/bin/node already exist;

2. working directory is /opt/astrometry-api-lite

3. User=oleh

oleh@oleh:~$ cd /opt/astrometry-api-lite
oleh@oleh:/opt/astrometry-api-lite$ node dist/manager/main.js
Manager started
Worker manager is still alive, active/max worker count: 0/4

http://localhost:3000/

Cannot GET /

Attached Thumbnails

  • 001.png

Edited by Oleg Astro, 07 April 2018 - 01:54 PM.


#10 Oleg Astro

Oleg Astro

    Astronomy Linux

  • -----
  • Vendors
  • Posts: 503
  • Joined: 12 Aug 2008
  • Loc: Albufeira, Portugal

Posted 07 April 2018 - 03:43 PM

Is it works?

 

How can I use it in Ubuntu?...

 

Attached Thumbnails

  • 002.png


#11 jusasi

jusasi

    Sputnik

  • -----
  • topic starter
  • Posts: 26
  • Joined: 04 Mar 2018

Posted 07 April 2018 - 04:32 PM

Is it works?

 

How can I use it in Ubuntu?...

Yep, looks like it's running fine.

 

Now you just need to put the API URL to your software, whatever it is. Kstars/Ekos for example has:

9vieJje.png


  • Oleg Astro likes this

#12 Wjdrijfhout

Wjdrijfhout

    Explorer 1

  • -----
  • Posts: 90
  • Joined: 24 Sep 2017
  • Loc: Groningen, The Netherlands

Posted 03 May 2018 - 02:24 PM

Jussi, is it also possible to install Astrometry-Api-Lite on a Mac?



#13 lambermo

lambermo

    Apollo

  • -----
  • Posts: 1,220
  • Joined: 16 Jul 2007
  • Loc: .nl

Posted 04 May 2018 - 07:59 PM

FYI astrometry-api-lite runs very well on Ubuntu 16.04 , I used this :

 

We need a more recent node than the repo's have, so I used
https://nodejs.org/e...u-based-linux-d

curl -sL https://deb.nodesource.com/setup_8.x | sudo -E bash -
sudo apt-get install nodejs

and then in astrometry-api-lite/ directory :

npm install
npm run all:build

I do not yet use a nice upstart or systemd startup script, so I start from

astrometry-api-lite/ directory (git head) in two shells :

node dist/api/server.js
node dist/manager/main.js

-- Hans



#14 freerobby

freerobby

    AstroSwarm (Telescope R/C)

  • *****
  • Vendors
  • Posts: 54
  • Joined: 13 Aug 2016

Posted 15 May 2018 - 06:13 PM

Thanks for putting this together. Would you mind adding license information into the repo? I'm interested in using it for the OSS controller I'm designing.



#15 catalogman

catalogman

    Apollo

  • *****
  • Posts: 1,281
  • Joined: 25 Nov 2014

Posted 18 November 2018 - 03:33 PM

Here are some problems with the installation of astrometry-api-lite:

 

- The api-lite app doesn't install on Linux Mint 19 -- it begins with this error:

 

Running 'sudo apt-get update' to make sure package list is up to date...
E: The repository 'cdrom://Linux Mint 19 _Tara_ - Release i386 20180626
bionic Release' does not have a Release file.
ERROR: failed to run 'sudo apt-get update', cannot continue.

 

- The api-lite app also doesn't install in Windows 10 if WSL fails to run (my 'bash' gives a 0x80070000 error

and updates don't fix it).

 

- The api-lite app isn't available for Win7 users because it requires commands like 'sudo' and 'apt-get', which

aren't available in Cygwin or MSYS2.

 

Even if the api-lite app does install, there are no directions on how to use it. The same can be said for the

AstroImageJ/ansvr setup. Even when I use the WCS tab, the "solution" that AIJ gives doesn't add any

position info to the .fits file unless the image is from a digital mount that has recorded the central position.

So a free plate-solving app that makes full use of the WCS tools is what is needed.

 

--catalogman



#16 catalogman

catalogman

    Apollo

  • *****
  • Posts: 1,281
  • Joined: 25 Nov 2014

Posted 18 November 2018 - 04:01 PM

The astrometry-api-lite app site reports that

 

"Debian based distributions will not have any NGC objects in their annotations if you've installed astrometry.net from their packages; this is due to the non-commercial license requirements to use those object catalogs, and Debian has had to remove that data..."

 

What about the OpenNGC?

 

https://github.com/mattiaverga/OpenNGC

 

Note that in the current version, the four IC numbers IC 5383-86 are missing, but they are easy to add:

IC 5384 = NGC 7813 and IC 5386 = NGC 7832 (so just copy the NGC info). The other two IC's are

non-existent ("other"):

 

https://cseligman.co...atlas/ic53a.htm

 

--catalogman


Edited by catalogman, 18 November 2018 - 04:02 PM.


#17 catalogman

catalogman

    Apollo

  • *****
  • Posts: 1,281
  • Joined: 25 Nov 2014

Posted 20 November 2018 - 11:42 AM

FYI astrometry-api-lite runs very well on Ubuntu 16.04 <snip>

 

In Linux Mint 18.3 (=Ubuntu 16.04) the install fails. (See log att.)

 

OTOH, I did get the more familiar AstroImageJ/ansvr to do a successful solve in Win7. Because my image is a wide field, all I had to do was use the Tycho-2 (4100) instead of 2MASS (4200).

 

Tycho-2 (4100) is 99% complete to V=11.0, so that is the cutoff point.

 

The 2MASS (4200) has a nominal completeness limit of J=15.8, H=15.1, and Ks=14.3 mag.

 

Gaia-DR2 (5000) is essentially complete between mag 12-17. But many brighter sources are missing due to the difficulties of treating saturated CCD images. And the distribution of fainter sources does not match the real sky due to data processing limitations in crowded fields.

 

So always check that you have the correct catalogue if your image doesn't solve. Unfortunately, api-lite offers only one by default.
 

--catalogman

Attached Files

  • Attached File  log.zip   116.23KB   5 downloads


#18 lambermo

lambermo

    Apollo

  • -----
  • Posts: 1,220
  • Joined: 16 Jul 2007
  • Loc: .nl

Posted 24 November 2018 - 05:28 PM

Hi Catalogman, 

In Linux Mint 18.3 (=Ubuntu 16.04) the install fails. (See log att.)

You probably already know this but just in case; it's the node sqlite3 3.1.1 package that fails to install.

And although this should work it may have something to do with you running the 32bits version of Mint which I assume means it's quite less well tested.

 

It may also clash with the previous installed version sqlite@2.9.1 , so you could try uninstalling that first (no idea if this helps but you can try).

 

There is little info in the log of what exactly failed :

19687 info lifecycle sqlite3@3.1.13~install: sqlite3@3.1.13
19688 verbose lifecycle sqlite3@3.1.13~install: unsafe-perm in lifecycle false
19689 verbose lifecycle sqlite3@3.1.13~install: PATH: /usr/lib/node_modules/npm/node_modules/npm-lifecycle/node-gyp-bin:/usr/local/astrometry-api-lite-master/installer/node_modules/sqlite3/node_modules/.bin:/usr/local/astrometry-api-lite-master/installer/node_modules/.bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
19690 verbose lifecycle sqlite3@3.1.13~install: CWD: /usr/local/astrometry-api-lite-master/installer/node_modules/sqlite3
19691 silly lifecycle sqlite3@3.1.13~install: Args: [ '-c', 'node-pre-gyp install --fallback-to-build' ]
19692 silly lifecycle sqlite3@3.1.13~install: Returned: code: 1  signal: null
19693 info lifecycle sqlite3@3.1.13~install: Failed to exec install script 

You could run that by hand to see if it reproduces or gives additional clues.

 

-- Hans



#19 catalogman

catalogman

    Apollo

  • *****
  • Posts: 1,281
  • Joined: 25 Nov 2014

Posted 25 November 2018 - 10:07 PM

Attached File  log2.zip   207.99KB   3 downloadsThanks for your review of the log.

 

Actually, sqlite isn't installed in LM at all! When it is installed by the user, it's v2.x -- the v3 at the link below

 

https://community.li...re/view/sqlite3

 

will not install in LM18.

 

Another log is posted to show how far the install gets before aborting. There's also a screenshot from the end of the run to show some of the output to stderr.

 

Hopefully there's enough info for the developer to make it work.

 

--catalogman


Edited by catalogman, 25 November 2018 - 10:10 PM.


#20 nlondonlad

nlondonlad

    Lift Off

  • -----
  • Posts: 2
  • Joined: 04 Nov 2016

Posted 07 March 2019 - 03:50 PM

Hi Jussi

 

Nice one!!!   I've installed this on Ubuntu, and got it working with KStars.  Really great job!

 

I've got a hitch though, and wonder if you can help point me in the right direction.

 

Solving fails using Maxim DL, because it is uploading a plain text file with comma separated XY pairs.  The file is saved off OK, but when it's passed to solve-field, that fails because it expects a FITS header around the file when dealing with XY pairs rather than just a text file.

 

I've struggled to get the nova.astrometry.net code to work (not a clue how django works!), but looking at the source code, it looks like it has some special casing for plain text files that are uploaded in 'process_submissions.py': 

 

 

# otherwise, check to see if it is a text list
        try:
            fitsfn = get_temp_file()

            text_file = open(str(df.get_path()))
            text = text_file.read()
            text_file.close()

            # add x y header
            # potential hack, assumes it doesn't exist...
            text = "# x y\n" + text

            text_table = text_table_fields("", text=text)
            text_table.write_to(fitsfn)
            logmsg("Creating fits table from text list")

            fits = fits_table(fitsfn)
            source_type = 'text'

 

It seems to basically parse it into a FITS format by adding a FITS header to it, then passes xylist parameters to solve-field to get it to process correctly.

 

I'm struggling to work out where to put something similar in the code for your API Lite version.  Can you help point me in the right direction?

 

Thanks a million in advance!!

 

Stewart.



#21 Kasra

Kasra

    Sputnik

  • -----
  • Posts: 29
  • Joined: 09 Jan 2020
  • Loc: United Kingdom

Posted 22 July 2020 - 05:25 PM

The problem is that I can't even open the nova.astrometry.net website. I have tried many web browsers, PCs and internet connections. I still get the same error. Been trying to gte an API key for months, but the error gets in the way every time.

Error.JPG



#22 lambermo

lambermo

    Apollo

  • -----
  • Posts: 1,220
  • Joined: 16 Jul 2007
  • Loc: .nl

Posted 28 July 2020 - 04:15 PM

The problem is that I can't even open the nova.astrometry.net website. I have tried many web browsers, PCs and internet connections. I still get the same error. Been trying to gte an API key for months, but the error gets in the way every time.

You realize that this thread is about astrometry-api-lite right ? Which is about a local installation of astrometry.net plus an API interface (astrometry-api-lite) that responds just as nova.astrometry.net does.

So you do not need an api key from nova.astrometry.net in order to use a local installation of astrometry-api-lite.

 

Having said that and assuming you still do also want to have an API key for nova.astrometry.net I recommend you mail Dustin on the astrometry.net mailing list by joining https://groups.googl...orum/astrometry

 

-- Hans



#23 NuTek

NuTek

    Explorer 1

  • *****
  • Posts: 50
  • Joined: 16 Jun 2016

Posted 26 August 2020 - 04:11 PM

This conversation is old but I am trying to install the Astrometry-api-lite in Windows 10 release 2004 (latest as of 8/26/2020) under the Ubuntu Windows Subsystem.

I'm having issues with both the Windows installer (install_v1.2.0.exe from https://github.com/J...i-lite/releases) or from running the install script.  

 

Errors include several python missing errors (seems to not like python3 so I ran python-is-python3), and issues building sqlite (I have salite3 installed already)

 

Any successfully install astrometry-api-lite lately on Windows 10?

If so, how? 

 

Ultimately I'd like to have the local Windows 10 Kstars/Ekos invoke the local copy of astrometry .

 

Thanks!

Bernie

 

 




CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.


Recent Topics






Cloudy Nights LLC
Cloudy Nights Sponsor: Astronomics