Jump to content

  •  

CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.

Photo

PlanetarySystemStacker - Program crashes and other execution problems

imaging astrophotography moon planet solar
  • Please log in to reply
50 replies to this topic

#1 Rolf

Rolf

    Viking 1

  • -----
  • topic starter
  • Posts: 521
  • Joined: 25 Apr 2016
  • Loc: Cologne, Germany

Posted 29 January 2021 - 02:10 PM

Hi,

 

The "epic thread" New stacking software project "PlanetarySystemStacker" has been going on for about two years now. Many of you actively contributed to the discussion which was a big help in the process to make the software mature and useful for many application scenarios. I thank you all very much for your dedication and patience!

 

Now it is time to organize the discussion in a better way. I suggest that we split it in three threads. This one will be the place where you can report program crashes or other execution problems when processing your data. The assumption here is that the installation was successful, the user interface comes up as expected and the GUI interaction seems to be okay. Only when you process some input file, it does not work as described in the User Guide document.

 

I have started two other threads for different topics:

I hope that you will find this new organization useful, and that we don't loose contact in this transition. I'm very much looking forward to the continued exchange with you on PSS in this new setting.

 

All the best,

Rolf


  • bips3453 likes this

#2 Rolf

Rolf

    Viking 1

  • -----
  • topic starter
  • Posts: 521
  • Joined: 25 Apr 2016
  • Loc: Cologne, Germany

Posted 29 January 2021 - 02:28 PM

Hi,

 

In this posting Rapid81 reported a problem when processing a 4K .mp4 file. I don't understand exactly what it means that the "PSS quits to desktop after aligning the frames". Does this mean the program crashes without any error message?

 

Well, I often regret that PSS accepts video formats like .mp4 or .mov at all. Many things can go wrong, and working with compressed video formats is a big disadvantage for any stacking algorithm. Therefore, I recommend to use .ser or .avi instead. Of course even then the program should not just crash without further notice. The strange thing is that the problem occurred after aligning the frames. In other words, at a point where all frames have been read from the file successfully.

 

If the program crashes, the protocol file probably does not help much. One way to get more info is to start the program from a terminal. If PSS was installed in a Python environment (using "pip3") after opening a terminal PSS can be started by entering "PlanetarySystemStacker". If it was installed using the Windows installer, the full path to the PSS executable must be entered.

 

Additionally, I recommend to set the protocol level to the highest value (2).

 

Is there any message in the terminal window after the crash?

 

All the best,

 Rolf



#3 Rapid81

Rapid81

    Lift Off

  • -----
  • Posts: 4
  • Joined: 29 Jan 2021

Posted 29 January 2021 - 03:38 PM

Hi,

 

In this posting Rapid81 reported a problem when processing a 4K .mp4 file. I don't understand exactly what it means that the "PSS quits to desktop after aligning the frames". Does this mean the program crashes without any error message?

 

Well, I often regret that PSS accepts video formats like .mp4 or .mov at all. Many things can go wrong, and working with compressed video formats is a big disadvantage for any stacking algorithm. Therefore, I recommend to use .ser or .avi instead. Of course even then the program should not just crash without further notice. The strange thing is that the problem occurred after aligning the frames. In other words, at a point where all frames have been read from the file successfully.

 

If the program crashes, the protocol file probably does not help much. One way to get more info is to start the program from a terminal. If PSS was installed in a Python environment (using "pip3") after opening a terminal PSS can be started by entering "PlanetarySystemStacker". If it was installed using the Windows installer, the full path to the PSS executable must be entered.

 

Additionally, I recommend to set the protocol level to the highest value (2).

 

Is there any message in the terminal window after the crash?

 

All the best,

 Rolf

When the aligning procedure should reach 100%, the program just closes without any error and the terminal window also closes.

 

I stared the terminal window for the whole procedure, and I could catch these messages, before it closed.

The number of frames in the video is 1854, so if PSS starts the frames with 0, then it's the last one.

 

It seems I really should use PIPP first, but this way, without PIPP, it would be much-much faster.

Attached Thumbnails

  • PSS.jpg

Edited by Rapid81, 29 January 2021 - 03:45 PM.


#4 Rolf

Rolf

    Viking 1

  • -----
  • topic starter
  • Posts: 521
  • Joined: 25 Apr 2016
  • Loc: Cologne, Germany

Posted 30 January 2021 - 04:35 AM

Hi Rapid81,

 

Thank you for showing the terminal output. Unfortunately, you did not set the protocol level to the highest value (2), so I still don't get a clear picture of what happened.

 

It seems that the error occurred deep in a library routine which reads a single frame from the file. The strange thing is that this frame must have been read before without any problem. So, this looks like a bug in that library routine for .mp4 files.

 

At least I found the reason why the program did not handle this problem "gracefully" and instead crashed. I fixed this bug, so starting with the next release the program will show the error message and continue with the next job.

 

The buffering level must have been quite low in your example. If your computer has enough RAM for buffering level 4, you could try running your example again. In this case all frames are read only once, so the error should not occur. Otherwise I can only give the advice which you saw in the terminal output already to try converting the video with PIPP into an AVI or SER file.

 

All the best,

 Rolf



#5 Rapid81

Rapid81

    Lift Off

  • -----
  • Posts: 4
  • Joined: 29 Jan 2021

Posted 30 January 2021 - 05:03 AM

Hi Rapid81,

 

Thank you for showing the terminal output. Unfortunately, you did not set the protocol level to the highest value (2), so I still don't get a clear picture of what happened.

 

It seems that the error occurred deep in a library routine which reads a single frame from the file. The strange thing is that this frame must have been read before without any problem. So, this looks like a bug in that library routine for .mp4 files.

 

At least I found the reason why the program did not handle this problem "gracefully" and instead crashed. I fixed this bug, so starting with the next release the program will show the error message and continue with the next job.

 

The buffering level must have been quite low in your example. If your computer has enough RAM for buffering level 4, you could try running your example again. In this case all frames are read only once, so the error should not occur. Otherwise I can only give the advice which you saw in the terminal output already to try converting the video with PIPP into an AVI or SER file.

 

All the best,

 Rolf

Sorry, I missed the part about the protocol level. Now I changed it.

Also, I tried to adjust the buffering level, but my 16GB of RAM wasn't enough. Only level 1 worked.

Attached Thumbnails

  • PSS.jpg


#6 Stargazer3236

Stargazer3236

    Aurora

  • ****-
  • Posts: 4,816
  • Joined: 07 Aug 2010
  • Loc: Waltham, MA

Posted 31 January 2021 - 08:38 AM

Is there a new and all put together update that I can download? I am not coding savvy so I need to have the least amount of problems installing. Thanks!



#7 Rolf

Rolf

    Viking 1

  • -----
  • topic starter
  • Posts: 521
  • Joined: 25 Apr 2016
  • Loc: Cologne, Germany

Posted 31 January 2021 - 09:37 AM

Hi Rapid81,

 

Sorry, I missed the part about the protocol level. Now I changed it.

Also, I tried to adjust the buffering level, but my 16GB of RAM wasn't enough. Only level 1 worked.

Thank you very much for the detailed output! Everything on the side of PSS looks completely normal. At the point where the program crashes the frame no. 1853 is read for the second time. This should be no problem, and for all the other frames it has worked before. This must be a bug in the OpenCV routine called by PSS, or even the ffmpeg library underneath OpenCV. Reading single frames from a video is not trivial if videos are stored using compression (such as MP4). The problem is that frames are not stored as individual entities, but in larger chunks, so-called "keyframe intervals". Something must have gone wrong there.

 

Did you try converting the video into a uncompressed format (e.g. AVI) and using that as input to PSS?

 

All the best,

 Rolf



#8 Rolf

Rolf

    Viking 1

  • -----
  • topic starter
  • Posts: 521
  • Joined: 25 Apr 2016
  • Loc: Cologne, Germany

Posted 31 January 2021 - 09:45 AM

Hi,

 

Is there a new and all put together update that I can download? I am not coding savvy so I need to have the least amount of problems installing. Thanks!

Do you mean for Windows? The latest "traditional" Windows installer was released on January 1st (version 0.8.22). Since then I fixed several bugs, but the updates are only available on the PyPI server for installation in a Python 3 environment. I plan to create a new Windows installer soon, say, not later than in a month from now.

 

All the best,

 Rolf


  • Stargazer3236 likes this

#9 Rapid81

Rapid81

    Lift Off

  • -----
  • Posts: 4
  • Joined: 29 Jan 2021

Posted 31 January 2021 - 10:47 AM

Hi Rapid81,

 

Did you try converting the video into a uncompressed format (e.g. AVI) and using that as input to PSS?

 

All the best,

 Rolf

I tried with .ser and .avi too. Worked like a charm.



#10 Rolf

Rolf

    Viking 1

  • -----
  • topic starter
  • Posts: 521
  • Joined: 25 Apr 2016
  • Loc: Cologne, Germany

Posted 31 January 2021 - 02:35 PM

Hi Rapid81,

 

Glad to hear that the converted videos worked well! Compressed videos are really difficult to handle, and with the conversion you made it much easier for the ffmpeg library.

 

At least I corrected the error handling in PSS, so that it will not crash anymore, should the same problem strike again.

 

All the best,

 Rolf



#11 mark77

mark77

    Viking 1

  • *****
  • Posts: 818
  • Joined: 28 Jun 2015
  • Loc: PA

Posted 01 March 2021 - 11:43 AM

Rolf

 

I am a long time experienced Linux developer, old school, I use C/C++ not Python.

 

I have downloaded your code and run the scripts, (very cryptic)

 

And I cannot get it to run.

 

I am running on Ubuntu 64 bit. 16.04

 

If I try to run ./setup_linux.py

 

it just sits there and does nothing.

 

 

Any better instructions for Linux?

 

Thanks



#12 Rolf

Rolf

    Viking 1

  • -----
  • topic starter
  • Posts: 521
  • Joined: 25 Apr 2016
  • Loc: Cologne, Germany

Posted 02 March 2021 - 09:06 AM

Hi mark77,

 

Did you read the installation chapter in the User Guide document?

 

PSS is available on PyPI for download. You don't have to produce a wheel file yourself with "./setup_linux.py". I used that file to produce the file on PyPI.

 

All the best,

 Rolf



#13 rbarbera

rbarbera

    Sputnik

  • -----
  • Posts: 44
  • Joined: 05 Nov 2019
  • Loc: Valencia, Spain

Posted 11 April 2021 - 10:23 AM

Hi Rolf,

 

Warming up for the next planets session I was trying to reprocess some SER files with the current version (0.8.17 running on macOS 10.15.7). It reads fine the SER file and all the aligning and stacking procedure works as expected. 

 

But just as I tap the Ok button to go to the post processing screen the app crashes. It doesn't saves any information on the protocol files (I've put level 2, but they are created empty). The only error message that I can see is:

➜  ~ PlanetarySystemStacker
17-12-15.6 Info: libmkl_rt.dylib not found (Intel Math Kernel Library not installed?). Performance might be slightly reduced.

QObject::moveToThread: Cannot move objects with a parent
Traceback (most recent call last):
  File "/Users/rlbe/Library/Python/3.9/lib/python/site-packages/planetary_system_stacker/postproc_editor.py", line 753, in run
    version.image = Miscellaneous.post_process(
  File "/Users/rlbe/Library/Python/3.9/lib/python/site-packages/planetary_system_stacker/miscellaneous.py", line 1264, in post_process
    input_image_hsv = cvtColor(input_image, COLOR_BGR2HSV)
cv2.error: OpenCV(4.4.0) /private/var/folders/xy/mksl4zq557ggryg9tl_y8yqjqtkb6c/T/pip-install-kb_pc36s/opencv-python-headless_b30f3ba5d518499fbe67746ae5e522bd/opencv/modules/imgproc/src/color.simd_helpers.hpp:92: error: (-2:Unspecified error) in function 'cv::impl::(anonymous namespace)::CvtHelper<cv::impl::(anonymous namespace)::Set<3, 4, -1>, cv::impl::(anonymous namespace)::Set<3, -1, -1>, cv::impl::(anonymous namespace)::Set<0, 5, -1>, cv::impl::(anonymous namespace)::NONE>::CvtHelper(cv::InputArray, cv::OutputArray, int) [VScn = cv::impl::(anonymous namespace)::Set<3, 4, -1>, VDcn = cv::impl::(anonymous namespace)::Set<3, -1, -1>, VDepth = cv::impl::(anonymous namespace)::Set<0, 5, -1>, sizePolicy = cv::impl::(anonymous namespace)::NONE]'
> Invalid number of channels in input image:
>     'VScn::contains(scn)'
> where
>     'scn' is 1
[1]    49600 abort      PlanetarySystemStacker

The stacked image is saved, but if I try to open it directly it also crashes. The SER file was recorded with FireCapture from an ASI120MM-S (so, it's monochromatic) and I have a lot of them processed with previous versions of PSS.

 

What can be happening here? Could you give me any advice?

 

Thanks Rolf



#14 Rolf

Rolf

    Viking 1

  • -----
  • topic starter
  • Posts: 521
  • Joined: 25 Apr 2016
  • Loc: Cologne, Germany

Posted 20 April 2021 - 04:13 AM

Hi,

 

First I have to apologize for anwering so late. I did not look into this forum for a few weeks.

 

I have the impression that I have seen this error before, and I think I fixed it meanwhile. The current PSS version is 0.8.31, and you are using 0.8.17. Did you try upgrading to the current version?

 

If I remember the (meanwhile fixed) bug correctly, the problem occurred if at some point you checked the box "luminance only". If later you tried to postprocess a monochrome image, PSS tried to convert it into HSV color coding and failed. As I said, this bug is fixed in the current version.

 

As a workaround, you could delete the ".PlanetarySystemStacker.ini" file in your home directory. Then the history with the "Luminance only" setting should be gone, and postprocessing should work on your file.

 

All the best,

 Rolf


  • rbarbera likes this

#15 rbarbera

rbarbera

    Sputnik

  • -----
  • Posts: 44
  • Joined: 05 Nov 2019
  • Loc: Valencia, Spain

Posted 20 April 2021 - 09:55 AM

First of all, I'm very glad to listen again from you. It was a long absence for those difficult times we are living worldwide (and specially in Europe)

 

I have the impression that I have seen this error before, and I think I fixed it meanwhile. The current PSS version is 0.8.31, and you are using 0.8.17. Did you try upgrading to the current version?

Yes, I've tried it without success. When I try

pip3 install planetary-system-stacker --upgrade

It finish telling me that the requirements are fulfill with the installed version (0.8.17). If I try to force install 0.8.31 it fails

pip3 install planetary-system-stacker==0.8.31 --upgrade
ERROR: Could not find a version that satisfies the requirement planetary-system-stacker==0.8.31
ERROR: No matching distribution found for planetary-system-stacker==0.8.31

I'm running it on macOS 10.15.7

 

Thanks Rolf



#16 Rolf

Rolf

    Viking 1

  • -----
  • topic starter
  • Posts: 521
  • Joined: 25 Apr 2016
  • Loc: Cologne, Germany

Posted 20 April 2021 - 10:57 AM

Hi,

 

Are you using Python 3.9 on your Mac? For some library incompatibility reason in the more recent releases I had to include the condition that Python must be between 3.5 and 3.8.x. By the way: The release tag for the Mac version is 0.8.31.3. I had to include the fourth identifier so that I could specify different conditions for different OS types. So, there is no distribution 0.8.31.

 

Did you try the workaround, i.e. deleting the "ini" file berofe you start the postprocessing?

 

All the best,

 Rolf



#17 rbarbera

rbarbera

    Sputnik

  • -----
  • Posts: 44
  • Joined: 05 Nov 2019
  • Loc: Valencia, Spain

Posted 20 April 2021 - 11:34 AM

Also, If I try to force a reinstall, it doesn't see the new versions after 0.8.17

 ~ pip install --force-reinstall --no-cache-dir planetary-system-stacker
Collecting planetary-system-stacker
  Downloading planetary_system_stacker-0.8.17-py3-none-macosx_10_6_intel.whl (228 kB)
     |████████████████████████████████| 228 kB 4.0 MB/s 
Collecting PyQt5
  Downloading PyQt5-5.15.4-cp36.cp37.cp38.cp39-abi3-macosx_10_13_intel.whl (7.0 MB)
     |████████████████████████████████| 7.0 MB 8.3 MB/s 
Collecting scikit-image
  Downloading scikit_image-0.18.1-cp39-cp39-macosx_10_9_x86_64.whl (12.9 MB)
     |████████████████████████████████| 12.9 MB 17.7 MB/s 


#18 rbarbera

rbarbera

    Sputnik

  • -----
  • Posts: 44
  • Joined: 05 Nov 2019
  • Loc: Valencia, Spain

Posted 20 April 2021 - 11:35 AM

Did you try the workaround, i.e. deleting the "ini" file berofe you start the postprocessing?

Yes, yes, this worked, but I'm trying to get my installation working again with the latest version. I'll try to downgrade to python 3.8 (yes, I'm running 3.9).

 

Thanks



#19 Rolf

Rolf

    Viking 1

  • -----
  • topic starter
  • Posts: 521
  • Joined: 25 Apr 2016
  • Loc: Cologne, Germany

Posted 20 April 2021 - 12:50 PM

Hi,

 

I'm glad that it is not a kew bug, but one I have fixed long ago.

 

If you downgrade Python to 3.8.x you should be able to get PSS 0.8.31.3. I hope it works!

 

All the best,

 Rolf



#20 rbarbera

rbarbera

    Sputnik

  • -----
  • Posts: 44
  • Joined: 05 Nov 2019
  • Loc: Valencia, Spain

Posted 20 April 2021 - 01:49 PM

Hi,

 

Yes, after downgrading to Python 3.8.9 I was able to get PSS 0.8.31.3. 

 

Thanks Rolf



#21 Rolf

Rolf

    Viking 1

  • -----
  • topic starter
  • Posts: 521
  • Joined: 25 Apr 2016
  • Loc: Cologne, Germany

Posted 20 April 2021 - 02:20 PM

Hi,

 

Thanks for your feedback! I'm glad you could get the latest version running on your machine. With this version the "Luminance only" bug should not appear any more (and several others as well).

 

All the best,

 Rolf



#22 ClownFish

ClownFish

    Fly Me to the Moon

  • *****
  • Posts: 6,732
  • Joined: 26 Apr 2005
  • Loc: Edgewood, New Mexico

Posted 17 May 2021 - 03:53 PM

Thanks Rolf for creating an amazing piece of software.  I used it successfully with my video files without any trouble on my MacBook.  But today I tried to stack a folder of fits files and got this:

*******************************************************************************

14-09-10.2 Start processing /Users/peterkennett/Documents/ESO Images/Moon/Light
           Software version used: PlanetarySystemStacker 0.8.13
*******************************************************************************
14-09-10.3 +++ Start reading frames +++
14-09-10.3 +++ Start ranking frames +++
14-09-24.8 Error: Reading image with index: 6, Attempt to read image format other than 'tiff', 'tif', '.png', '.jpg' or 'fit', 'fits', continuing with next job

The files were all .fits files from KStars of the Moon.

Is this not an implemented feature yet?

 

Peter



#23 Rolf

Rolf

    Viking 1

  • -----
  • topic starter
  • Posts: 521
  • Joined: 25 Apr 2016
  • Loc: Cologne, Germany

Posted 18 May 2021 - 09:14 AM

Hi Peter,

 

My only guess is that the folder contains some (hidden?) files with a different extension. Reading ".fits" files is an implemented feature and should work. Also, the error message suggests that the first five files could be read without problems, so the ".fits" format should not be the reason. Could you please check if the folder contains something hidden?

 

All the best,

 Rolf



#24 ClownFish

ClownFish

    Fly Me to the Moon

  • *****
  • Posts: 6,732
  • Joined: 26 Apr 2005
  • Loc: Edgewood, New Mexico

Posted 18 May 2021 - 12:33 PM

Hi Peter,

 

My only guess is that the folder contains some (hidden?) files with a different extension. Reading ".fits" files is an implemented feature and should work. Also, the error message suggests that the first five files could be read without problems, so the ".fits" format should not be the reason. Could you please check if the folder contains something hidden?

 

All the best,

 Rolf

Thanks Rolf,

 

I tried again, and looked for hidden files and didn't see any.  I'm not sure why the first five worked.

Here's a screenshot showing the folder, and error again.   I converted all these files to TIFF with PixInsight and they loaded fine.

 

Screen Shot 2021-05-18 at 11.29.43 AM copy.jpg



#25 Rolf

Rolf

    Viking 1

  • -----
  • topic starter
  • Posts: 521
  • Joined: 25 Apr 2016
  • Loc: Cologne, Germany

Posted 18 May 2021 - 02:02 PM

Hi Peter,

 

I'm glad you could process your TIFF files successfully. But I still don't understand why you got the error message for the sixth FITS file. The strange thing is that the error message has nothing to do with the content of the files, but only indicates that the list of file names in the folder contains something with an invalid extension. That is why I assumed that there must be some hidden file in the folder. What if you copy all those ".fits" files (and only those) into a new directory and process that directory? Does the error still show?

 

All the best,

 Rolf




CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.


Recent Topics





Also tagged with one or more of these keywords: imaging, astrophotography, moon, planet, solar



Cloudy Nights LLC
Cloudy Nights Sponsor: Astronomics