Jump to content

  •  

CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.

Photo

Introducing MetroPSF - a program for differential ensemble PSF photometry

  • Please log in to reply
18 replies to this topic

#1 blackhaz

blackhaz

    Vanguard

  • *****
  • topic starter
  • Posts: 2,093
  • Joined: 20 May 2006
  • Loc: London, England

Posted 08 April 2021 - 02:34 PM

Hi All,

 

I would like to share a small program I wrote, currently called MetroPSF for the lack of better name, for differential ensemble PSF-fitting photometry. Initially wrote a small script to double-check the results I was getting through Astrometrica for the SN 2021gmj, however, thought it may be useful for public, so releasing the source code. It's a prototype, so the code quality is pretty low, but the results it produces should be ready for analysis. I would appreciate any feedback and suggestions. I thought I'd check if this is of interest to public at all by making a preliminary "alpha" release.

 

The READ ME file with a brief description and installation instructions: http://trafyx.com/wp...04/metropsf.pdf

The program files are here: http://trafyx.com/wp...04/metropsf.zip

 

The installation process may seem a bit strange, but it all boils down to installing Python interpreter which is pretty much automatic process, and using pip to install libraries, which is also automatic. This takes a few minutes on both Windows and Unix-like platforms.

 

MetroPSF works with VizieR and Astrometry.net for retrieving catalogs and solving, and the READ ME file describes a quick use case to familiarize with the interface. The screen shot of the program in action is below - on NGC 3310 with SN 2021gmj.

 

Maxim

 

Screenshot-2021-04-08-20-24-03.png

Larger version: https://i.ibb.co/LJD...08-20-24-03.png


Edited by blackhaz, 08 April 2021 - 02:37 PM.


#2 le93baron

le93baron

    Lift Off

  • *****
  • Posts: 21
  • Joined: 10 Feb 2018

Posted 09 April 2021 - 05:37 AM

Well I tried to load it but couldn't get past this part:

 

3. Run pip install -r requirements.txt command in the folder with MetroPSF
files. This will automatically install all the required libraries.

 

I guess I'm not familiar with Python enough.



#3 blackhaz

blackhaz

    Vanguard

  • *****
  • topic starter
  • Posts: 2,093
  • Joined: 20 May 2006
  • Loc: London, England

Posted 09 April 2021 - 06:27 AM

le93baron, if you are using Windows 10 then open the folder with the MetroPSF files then do File menu –> Open Windows PowerShell. There, type "pip install -r requirements.txt". If you're having troubles, please let me know what error message you're getting, if any.



#4 le93baron

le93baron

    Lift Off

  • *****
  • Posts: 21
  • Joined: 10 Feb 2018

Posted 09 April 2021 - 06:43 PM

Ok I did that and get this message:

pip : The term 'pip' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.



#5 blackhaz

blackhaz

    Vanguard

  • *****
  • topic starter
  • Posts: 2,093
  • Joined: 20 May 2006
  • Loc: London, England

Posted 09 April 2021 - 06:49 PM

le93baron, when installing Python, you probably didn't specify to add it to the system path, so it can't find pip. There are two options:

 

1) Reinstall Python. It should ask whether you want to add its folder to the system path - so you could execute pip without explicitly specifying where it's located.

2) You can try installing Anaconda. It's Python with a bunch of libraries and a more user-friendly installer:

https://www.anaconda...ucts/individual

 

I am pretty sure that will resolve the issue.



#6 le93baron

le93baron

    Lift Off

  • *****
  • Posts: 21
  • Joined: 10 Feb 2018

Posted 09 April 2021 - 07:03 PM

YEAH!.  That worked.

It's downloading and installing the packages....

 

Thanks blackhaz!



#7 le93baron

le93baron

    Lift Off

  • *****
  • Posts: 21
  • Joined: 10 Feb 2018

Posted 09 April 2021 - 07:05 PM

At the end it says:

 

WARNING: You are using pip version 20.2.3; however, version 21.0.1 is available.
You should consider upgrading via the 'c:\users\admin\appdata\local\programs\python\python39\python.exe -m pip install --upgrade pip' command.

 

Is this important?

 

Bob



#8 blackhaz

blackhaz

    Vanguard

  • *****
  • topic starter
  • Posts: 2,093
  • Joined: 20 May 2006
  • Loc: London, England

Posted 10 April 2021 - 03:55 AM

If the program runs fine then no, generally you don't need to upgrade pip - unless necessary. If it doesn't start, try upgrading it and then running the pip command once again.



#9 cometchaserde

cometchaserde

    Vendor (cometchaserde)

  • -----
  • Vendors
  • Posts: 40
  • Joined: 06 Jul 2009
  • Loc: Germany

Posted 13 April 2021 - 02:24 PM

Hi,

I tried it on Debian stable but got 2 errors.

 

Successfully built astroquery photutils subprocess32
astroquery 0.4.1 has requirement astropy>=3.1, but you'll have astropy 2.0.16 which is incompatible.
Installing collected packages: python-dateutil, pandas, scipy, py, attrs, pluggy, more-itertools, atomicwrites, pytest, astropy, astroquery, kiwisolver, subprocess32, cycler, matplotlib, photutils
  The scripts py.test and pytest are installed in '/home/stefan/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  The scripts fits2bitmap, fitscheck, fitsdiff, fitsheader, fitsinfo, samp_hub, volint and wcslint are installed in '/home/stefan/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed astropy-2.0.16 astroquery-0.4.1 atomicwrites-1.4.0 attrs-20.3.0 cycler-0.10.0 kiwisolver-1.1.0 matplotlib-2.2.5 more-itertools-5.0.0 pandas-0.24.2 photutils-0.4.1 pluggy-0.7.1 py-1.10.0 pytest-3.6.4 python-dateutil-2.8.1 scipy-1.2.3 subprocess32-3.5.4

 

This error is not a problem but what about the other one by starting "metrofsp"

 

python metropsf.py
  File "metropsf.py", line 309
SyntaxError: Non-ASCII character '\xce' in file metropsf.py on line 309, but no encoding declared; see http://python.org/dev/peps/pep-0263/ for details

 

Any idea ?

 

Stefan



#10 blackhaz

blackhaz

    Vanguard

  • *****
  • topic starter
  • Posts: 2,093
  • Joined: 20 May 2006
  • Loc: London, England

Posted 14 April 2021 - 03:45 AM

Hi Stefan,

 

Very strange you have warnings with astroquery. This would be something beyond me, as that's a library. Perhaps, try to install these packages via apt? Another option is to install Anaconda. Could be dependencies are resolved better there.

 

On the syntax error, I have added an explicit UTF-8 encoding declaration to the top of the file. It's odd that it ran on other Linux machines fine. Could you please try to run the file attached and see if it works?

 

Thanks!

 

Max

Attached Files


Edited by blackhaz, 14 April 2021 - 03:46 AM.


#11 cometchaserde

cometchaserde

    Vendor (cometchaserde)

  • -----
  • Vendors
  • Posts: 40
  • Joined: 06 Jul 2009
  • Loc: Germany

Posted 14 April 2021 - 02:45 PM

Hi Max,

it looks a little bit better.

 

pip install -r requirements.txt
Requirement already satisfied: numpy in /usr/lib/python2.7/dist-packages (from -r requirements.txt (line 1)) (1.16.2)
Requirement already satisfied: pandas in /home/stefan/.local/lib/python2.7/site-packages (from -r requirements.txt (line 2)) (0.24.2)
Requirement already satisfied: scipy in /home/stefan/.local/lib/python2.7/site-packages (from -r requirements.txt (line 3)) (1.2.3)
Requirement already satisfied: astropy in /home/stefan/.local/lib/python2.7/site-packages (from -r requirements.txt (line 4)) (2.0.16)
Requirement already satisfied: pillow in /usr/lib/python2.7/dist-packages (from -r requirements.txt (line 5)) (5.4.1)
Requirement already satisfied: astroquery in /home/stefan/.local/lib/python2.7/site-packages (from -r requirements.txt (line 6)) (0.4.1)
Requirement already satisfied: matplotlib in /home/stefan/.local/lib/python2.7/site-packages (from -r requirements.txt (line 7)) (2.2.5)
Requirement already satisfied: photutils in /home/stefan/.local/lib/python2.7/site-packages (from -r requirements.txt (line 8)) (0.4.1)
Requirement already satisfied: python-dateutil>=2.5.0 in /home/stefan/.local/lib/python2.7/site-packages (from pandas->-r requirements.txt (line 2)) (2.8.1)
Requirement already satisfied: pytz>=2011k in /usr/local/lib/python2.7/dist-packages (from pandas->-r requirements.txt (line 2)) (2019.3)
Requirement already satisfied: pytest<3.7,>=2.8 in /home/stefan/.local/lib/python2.7/site-packages (from astropy->-r requirements.txt (line 4)) (3.6.4)
Requirement already satisfied: requests>=2.4.3 in /usr/lib/python2.7/dist-packages (from astroquery->-r requirements.txt (line 6)) (2.21.0)
Requirement already satisfied: beautifulsoup4>=4.3.2 in /usr/lib/python2.7/dist-packages (from astroquery->-r requirements.txt (line 6)) (4.7.1)
Requirement already satisfied: html5lib>=0.999 in /usr/lib/python2.7/dist-packages (from astroquery->-r requirements.txt (line 6)) (1.0.1)
Requirement already satisfied: keyring>=4.0 in /usr/lib/python2.7/dist-packages (from astroquery->-r requirements.txt (line 6)) (17.1.1)
Requirement already satisfied: six in /usr/lib/python2.7/dist-packages (from astroquery->-r requirements.txt (line 6)) (1.12.0)
Requirement already satisfied: kiwisolver>=1.0.1 in /home/stefan/.local/lib/python2.7/site-packages (from matplotlib->-r requirements.txt (line 7)) (1.1.0)
Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/lib/python2.7/dist-packages (from matplotlib->-r requirements.txt (line 7)) (2.2.0)
Requirement already satisfied: backports.functools-lru-cache in /usr/lib/python2.7/dist-packages (from matplotlib->-r requirements.txt (line 7)) (1.5)
Requirement already satisfied: subprocess32 in /home/stefan/.local/lib/python2.7/site-packages (from matplotlib->-r requirements.txt (line 7)) (3.5.4)
Requirement already satisfied: cycler>=0.10 in /home/stefan/.local/lib/python2.7/site-packages (from matplotlib->-r requirements.txt (line 7)) (0.10.0)
Requirement already satisfied: py>=1.5.0 in /home/stefan/.local/lib/python2.7/site-packages (from pytest<3.7,>=2.8->astropy->-r requirements.txt (line 4)) (1.10.0)
Requirement already satisfied: setuptools in /usr/lib/python2.7/dist-packages (from pytest<3.7,>=2.8->astropy->-r requirements.txt (line 4)) (40.8.0)
Requirement already satisfied: funcsigs; python_version < "3.0" in /usr/lib/python2.7/dist-packages (from pytest<3.7,>=2.8->astropy->-r requirements.txt (line 4)) (1.0.2)
Requirement already satisfied: attrs>=17.4.0 in /home/stefan/.local/lib/python2.7/site-packages (from pytest<3.7,>=2.8->astropy->-r requirements.txt (line 4)) (20.3.0)
Requirement already satisfied: pluggy<0.8,>=0.5 in /home/stefan/.local/lib/python2.7/site-packages (from pytest<3.7,>=2.8->astropy->-r requirements.txt (line 4)) (0.7.1)
Requirement already satisfied: more-itertools>=4.0.0 in /home/stefan/.local/lib/python2.7/site-packages (from pytest<3.7,>=2.8->astropy->-r requirements.txt (line 4)) (5.0.0)
Requirement already satisfied: atomicwrites>=1.0 in /home/stefan/.local/lib/python2.7/site-packages (from pytest<3.7,>=2.8->astropy->-r requirements.txt (line 4)) (1.4.0)
Requirement already satisfied: secretstorage<3 in /usr/lib/python2.7/dist-packages (from keyring>=4.0->astroquery->-r requirements.txt (line 6)) (2.3.1)

 

python metropsf.py

Traceback (most recent call last):
  File "metropsf.py", line 15, in <module>
    from PIL import Image, ImageTk, ImageMath, ImageOps
ImportError: cannot import name ImageTk

 

-> fixed with

sudo apt-get install python-tkinter

 

Next error:

 

File "metropsf.py", line 26, in <module>
    import tkinter as tk
ImportError: No module named tkinter

 

-> fixed with

 

import Tkinter as tk

from Tkinter import filedialog as fd

 

Next error:

ImportError: cannot import name filedialog

 

 

Looks like you used Python3. Filedialog is a command of Python3.

I try to get it running with Python 2.7 because if I try to set it up with Python3 I get even more errors.

 

Got it running by

 

import Tkinter as tk
import Tkinter, Tkconstants, tkFileDialog as fd

 

 

 

 

Stefan


Edited by cometchaserde, 14 April 2021 - 03:52 PM.


#12 blackhaz

blackhaz

    Vanguard

  • *****
  • topic starter
  • Posts: 2,093
  • Joined: 20 May 2006
  • Loc: London, England

Posted 15 April 2021 - 02:24 AM

Hi Stefan,

 

Python 2.7 has been EOL'ed back in 2020 and is not supported anymore. It's interesting you have made it working there, but you may experience problems along the way, especially given the warnings you've received earlier from pip. I would strongly recommend you to update your Python to 3. I am using 3.7 on my FreeBSD 12.2.



#13 blackhaz

blackhaz

    Vanguard

  • *****
  • topic starter
  • Posts: 2,093
  • Joined: 20 May 2006
  • Loc: London, England

Posted 15 April 2021 - 02:29 AM

Note, the latest MetroPSF versions can be found here:

http://trafyx.com/?p=2669

 

It's currently at v0.13 with many features added.


Edited by blackhaz, 15 April 2021 - 02:29 AM.


#14 cometchaserde

cometchaserde

    Vendor (cometchaserde)

  • -----
  • Vendors
  • Posts: 40
  • Joined: 06 Jul 2009
  • Loc: Germany

Posted 15 April 2021 - 11:35 AM

Hi Max,

tried the latest version 0.13 with Linux Mint 20.1 within Virtualbox. Due to the Python 2 problems with Debian.

 

Runs, but if I load a fits image (taken with my CCD-Kamera) it shows only a black image ( tried it with my normal Debian 10, and the same with the Linux Mint within Virtualbox -> black image ).

 

For Linux Mint I had to install some more python packages.

 

python3-pil python3-pil.imagetk

and

python-tk

 

 

Stefan



#15 blackhaz

blackhaz

    Vanguard

  • *****
  • topic starter
  • Posts: 2,093
  • Joined: 20 May 2006
  • Loc: London, England

Posted 15 April 2021 - 01:19 PM

Did you get any error messages in the MetroPSF or Python consoles? If you don't mind, please share the FITS file and I'll double-check if I can open it here.



#16 cometchaserde

cometchaserde

    Vendor (cometchaserde)

  • -----
  • Vendors
  • Posts: 40
  • Joined: 06 Jul 2009
  • Loc: Germany

Posted 15 April 2021 - 01:58 PM

Hi Max,

 

no really error messages.

 

Try it with this image https://www.cometcha...0sek-215130.fit

 

Stefan



#17 blackhaz

blackhaz

    Vanguard

  • *****
  • topic starter
  • Posts: 2,093
  • Joined: 20 May 2006
  • Loc: London, England

Posted 16 April 2021 - 03:16 AM

Very strange - it does open perfectly fine on my FreeBSD laptop. I'll check on Windows when I'll get home but don't expect any issues. If you don't see an image there should be an error message somewhere. Could you show a screenshot and, also, paste the output from the shell just for any case?


Edited by blackhaz, 16 April 2021 - 03:18 AM.


#18 cometchaserde

cometchaserde

    Vendor (cometchaserde)

  • -----
  • Vendors
  • Posts: 40
  • Joined: 06 Jul 2009
  • Loc: Germany

Posted 16 April 2021 - 02:31 PM

From the terminal on Debian stable:

 

python2.7 metropsf.py
WARNING: Astrometry.net API key not found in configuration file [astroquery.astrometry_net.core]
WARNING: You need to manually edit the configuration file and add it [astroquery.astrometry_net.core]
WARNING: You may also register it for this session with AstrometryNet.key = 'XXXXXXXX' [astroquery.astrometry_net.core]
/home/stefan/.local/lib/python2.7/site-packages/mpl_toolkits/mplot3d/axes3d.py:1069: UserWarning: Axes3D.figure.canvas is 'None', mouse rotation disabled.  Set canvas then call Axes3D.mouse_init().
  "Axes3D.figure.canvas is 'None', mouse rotation disabled.  "
Exception in Tkinter callback
Traceback (most recent call last):
  File "/usr/lib/python2.7/lib-tk/Tkinter.py", line 1550, in __call__
    return self.func(*args)
  File "metropsf.py", line 743, in update_histogram_high
    self.update_display()
  File "metropsf.py", line 616, in update_display
    self.display_image()
  File "metropsf.py", line 118, in display_image
    generate_FITS_thumbnail(self.histogram_slider_low, self.histogram_slider_high, self.zoom_level, self.stretching_stringvar.get())
  File "metropsf.py", line 64, in generate_FITS_thumbnail
    converted_data = image_data.astype(float)
NameError: global name 'image_data' is not defined

 

See the attached image. No error messages.

 

I think this is a problem of Debian.

 

-> On Linux Mint 20.1 I had to change the slider image stretching. Looks good.

Attached Thumbnails

  • Bildschirmfoto von 2021-04-16 21-25-32.png

Edited by cometchaserde, 16 April 2021 - 02:40 PM.


#19 blackhaz

blackhaz

    Vanguard

  • *****
  • topic starter
  • Posts: 2,093
  • Joined: 20 May 2006
  • Loc: London, England

Posted 16 April 2021 - 04:32 PM

Just to make sure, everything works fine on your Python 3 system, right? Python 2 is not supported, as it's EOL, so I have no idea what could have gone wrong there. I don't have any Python 2 systems around to troubleshoot this, I am afraid.  




CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.


Recent Topics






Cloudy Nights LLC
Cloudy Nights Sponsor: Astronomics