Jump to content

  •  

CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.

Photo

StarNet v2 help requests

  • Please log in to reply
230 replies to this topic

#76 gorann

gorann

    Lift Off

  • -----
  • Posts: 13
  • Joined: 29 Nov 2015
  • Loc: Varmland, Sweden

Posted 06 September 2022 - 03:03 PM

I have just installed the StarNet2 on the latest version of PI on my Mac running Mac OS 12.5.1 and when I try to use it I get the message:

*** Error: Could not find checkpoint file in '/Applications/PixInsight/bin/'!

 

Any suggestion?

 

Cheers, Göran



#77 MAT_Blue

MAT_Blue

    Explorer 1

  • -----
  • Posts: 99
  • Joined: 08 Feb 2019
  • Loc: Bluffton, South Carolina, USA

Posted 06 September 2022 - 06:27 PM

I have just installed the StarNet2 on the latest version of PI on my Mac running Mac OS 12.5.1 and when I try to use it I get the message:

*** Error: Could not find checkpoint file in '/Applications/PixInsight/bin/'!

 

Any suggestion?

 

Cheers, Göran

On Windows, I get that when I don't start Pixinsight with it's default directory set to PixInsight/bin.  Don't know what causes it on a Mac but I offer that in case it gives you any ideas.  



#78 pfile

pfile

    Fly Me to the Moon

  • -----
  • Posts: 6,415
  • Joined: 14 Jun 2009

Posted 06 September 2022 - 11:50 PM

are the checkpoint files really installed in /Applications/PixInsight/bin?

 

it's kind of hard to set the default directory on a macosx app, but instead of that, after starting PI you can go into the console and say "cd /Applications/PixInsight/bin" and then try to run starnet. but i havent seen that problem in a very long time and i think it only plagued the original Starnet, before Juan refactored it. but StarNetV2 was not based on that code so i can see how it might have a problem finding the weights files.



#79 gorann

gorann

    Lift Off

  • -----
  • Posts: 13
  • Joined: 29 Nov 2015
  • Loc: Varmland, Sweden

Posted 07 September 2022 - 02:27 AM

On Windows, I get that when I don't start Pixinsight with it's default directory set to PixInsight/bin.  Don't know what causes it on a Mac but I offer that in case it gives you any ideas.  

 

 

are the checkpoint files really installed in /Applications/PixInsight/bin?

 

it's kind of hard to set the default directory on a macosx app, but instead of that, after starting PI you can go into the console and say "cd /Applications/PixInsight/bin" and then try to run starnet. but i havent seen that problem in a very long time and i think it only plagued the original Starnet, before Juan refactored it. but StarNetV2 was not based on that code so i can see how it might have a problem finding the weights files.

Thanks for the replies!

What is the checkpoint file? No file with that name came with StarNet2 so it is not among the files I moved to Pixinsight/bin

Where do you set the default directory?

CS, Göran



#80 gorann

gorann

    Lift Off

  • -----
  • Posts: 13
  • Joined: 29 Nov 2015
  • Loc: Varmland, Sweden

Posted 07 September 2022 - 02:55 AM

On Windows, I get that when I don't start Pixinsight with it's default directory set to PixInsight/bin.  Don't know what causes it on a Mac but I offer that in case it gives you any ideas.  

 

 

are the checkpoint files really installed in /Applications/PixInsight/bin?

 

it's kind of hard to set the default directory on a macosx app, but instead of that, after starting PI you can go into the console and say "cd /Applications/PixInsight/bin" and then try to run starnet. but i havent seen that problem in a very long time and i think it only plagued the original Starnet, before Juan refactored it. but StarNetV2 was not based on that code so i can see how it might have a problem finding the weights files.

Problem solved. I moved the file "StarNet2_weights.pb" to the Pixinsight/bin and then it worked

Cheers, Göran



#81 TunedDownGuitar

TunedDownGuitar

    Lift Off

  • -----
  • Posts: 2
  • Joined: 13 Dec 2021

Posted 09 September 2022 - 06:34 AM

I'd like to comment on the GPU memory utilization with PixInsight. It's not a memory leak, this is expected behavior from the library. TensorFlow will claim memory, but just because it's showing utilized in Windows doesn't mean it's actually in use, just reserved for use. It will only actually be used when committing tensors for the SN process to run. This looks like a memory leak, but it's not, and those have different behaviors such as slow creeping increases in utilization (Firefox circa 2006 anybody?).

 

On my RTX2070 utilization hovers around 2.1GB until I run the tool, then it jumps to 6.7GB. The only way that 4.5GB~ is reclaimed is by closing PixInsight, and parameters do not matter in terms of how much it uses. When I've used TF with Python in the past I was able to release memory by deleting session objects and/or garbage collection, but that would require development changes to the StarNet2-pxm.dll file.

 

If this could be evaluated I think it would be a positive improvement to release that memory for other programs. I typically multitask and will leave PI running in the background for hours (or days) as I process a session since some of them can take quite a while especially on drizzled images.

 

My build information is below -

* Windows 11, RTX2070, 64GB Memory, Ryzen 2700X

* NVIDIA Drivers v516.94, CUDA v11.7.1, CUDNN 11.4

* PixInsight 1.8.9-1, StarNet++ v2.0.2, Tensorflow v2.7 (libtensorflow-gpu-windows-x86_64-2.7.0)

 

An idea of the utilization vs. processing time is below. You can see when I started the session at 6:44, the process ran until about 6:47, and I closed PixInsight at 6:48.

 

Screenshot 2022-09-09 at 06-54-13 Explore - InfluxDB2 - Grafana.png

Screenshot 2022-09-09 at 06-54-34 Explore - InfluxDB2 - Grafana.png

 

 



#82 pfile

pfile

    Fly Me to the Moon

  • -----
  • Posts: 6,415
  • Joined: 14 Jun 2009

Posted 09 September 2022 - 10:47 AM

i do think this is a tensorflow thing. on the mac tensorflow has to run on the CPU and i noticed that after running starnet all these tensorflow threads were still active, and likely still had a bunch of memory allocated. the behavior when running on GPU is probably the same.

 

i think i asked nikita about this and he said he was tearing all the tensorflow stuff down per the documentation after starnet runs, in which case this would be a tensorflow bug. i guess you'd have to check out other applications that use tensorflow to see if you can replicate the behavior.

 

rob



#83 TunedDownGuitar

TunedDownGuitar

    Lift Off

  • -----
  • Posts: 2
  • Joined: 13 Dec 2021

Posted 09 September 2022 - 04:02 PM

Depending on the way they are tearing it down they might have to explicitly call on the language equivalent of keras.backend.clear_session() or something similar prior to tearing down, then manually invoke the GC.

 

Without seeing the code I can't say for sure so this is mostly speculation, but I suspect this may also be an issue with the CPU version just less visible because it won't claim as much.



#84 pfile

pfile

    Fly Me to the Moon

  • -----
  • Posts: 6,415
  • Joined: 14 Jun 2009

Posted 09 September 2022 - 06:45 PM

well, the code for starnetv1 should still be in the pixinsight gitlab respository. i don't see why nikita would have changed that much (in v2) with respect to the actual tensorflow part of the module, and juan really only refactored the UI code as far as i know.

 

https://gitlab.com/p...NetInstance.cpp

 

i don't see any of the stuff you're talking about in there so it's possible the TF objects are not properly destroyed.

 

rob


Edited by pfile, 09 September 2022 - 06:45 PM.


#85 nekitmm

nekitmm

    Vendor - StarNet Software

  • *****
  • Vendors
  • topic starter
  • Posts: 227
  • Joined: 25 Apr 2018

Posted 11 September 2022 - 01:45 PM

Depending on the way they are tearing it down they might have to explicitly call on the language equivalent of keras.backend.clear_session() or something similar prior to tearing down, then manually invoke the GC.

 

Without seeing the code I can't say for sure so this is mostly speculation, but I suspect this may also be an issue with the CPU version just less visible because it won't claim as much.

The code calls these functions in this sequence (starting at line 239 in the code shared by Rob):

 

TF_CloseSession
TF_DeleteSession
TF_DeleteSessionOptions
TF_DeleteGraph
TF_DeleteStatus

 

I believe this is

 

So the session is closed first and then deleted. I don't think there is anything else to delete, input and output tensors are also removed before that...


  • pfile likes this

#86 erchobi

erchobi

    Lift Off

  • -----
  • Posts: 1
  • Joined: 07 Mar 2022

Posted 14 September 2022 - 08:59 AM

Good morning, I've been using StarNet for a long time and now I'm a version 2 user. A few days ago I realized that the version of StarNet v2 that I use from the terminal of a mbp m1 max of 64 gb returns me a tiff without stars but at 72 dpi, when I put it in StarNet v2 at 300 dpi, is it a known error? Is it an option that I have to choose somewhere and that I don't do? Or is it a particular feature of this plugin?

 

Thank you in advance for your information.



#87 SilverLitz

SilverLitz

    Gemini

  • *****
  • Posts: 3,165
  • Joined: 17 Feb 2018
  • Loc: Louisville, KY

Posted 27 September 2022 - 02:25 PM

I have been trying to get SN2 to work with GPU, and have yet to get it working.  More than 2 years ago, I tried GPU acceleration with SN1, and it was no faster than my using my CPU (though 24x Xeon 2680v3 cores), but it was an old Quadro K4200, about 3min on 183mm combines.  With SN2 on larger QHY268M, 2x drizzled combines, this go real slow (guessing ~20min).  So last week, I got a new 3070Ti (unused, pull from new PC, after seller got 3090ti), and give it another try.

 

Going through William Li's, https://www.williaml...om/starnet-cuda, tutorial and installing the Cuda 11.7, it did not seem that the GPU was being used.  So I uninstalled PI (latest 1.8.9-1, Window10P), and reinstalled, and replaced the TF dll, and still no results.  After reading this thread, I am going to try using libtensorflow-gpu-windows-x86_64-2.8.0, as William Li had me install 2.7.  I have not yet tried this.

 

I have never been able to get SN1 or SN2 to give good results, as they always had ugly artifacts, though SN2 seemed to be better.  I did find that strangely SN2 seemed to be very erratic in leaving small stars.

 

Here is what a test from an unprocessed Oiii stack, straight out of WBPP, with default SN2 looks like.  Notice all the small stars and the black holes:

 

SN2.jpg

 

Here is what the same stack looks like use default SN1.  Notice the weird black squares throughout.  But the pieces not in the black square look good.

 

UPDATE:  This might be user error, as the stack was linear, and linear was NOT checked on SN2.  Currently rerunning.

 

SN1.jpg

 

Both of these use the TF dll from the SN2 download.  I assume this dll is not good for SN???

 

Why does SN2 not remove the tiny stars?  Why the black holes?


Edited by SilverLitz, 27 September 2022 - 02:35 PM.


#88 SilverLitz

SilverLitz

    Gemini

  • *****
  • Posts: 3,165
  • Joined: 17 Feb 2018
  • Loc: Louisville, KY

Posted 27 September 2022 - 02:28 PM

Here is the unprocessed Oiii stack, straight out of WBPP, using QHY268M, Astronomik 6nm Oiii filter, and Esprit 100 (reduced to 413mm), before SN:

 

Pickering_WestVeil_O3.jpg



#89 SilverLitz

SilverLitz

    Gemini

  • *****
  • Posts: 3,165
  • Joined: 17 Feb 2018
  • Loc: Louisville, KY

Posted 27 September 2022 - 02:44 PM

Definitely user error.  Here SN2 on the same file, but Linear checked, no issues!

 

SN2_Linear.jpg

 

My previous disappointment with SN2 and RGB combines were non-linear, but were not stretched nearly as much as STF (MS at 10%).  Is that the reason for the erratic residual small stars?

 

Now I have to get GPU to work for SN2. 



#90 SilverLitz

SilverLitz

    Gemini

  • *****
  • Posts: 3,165
  • Joined: 17 Feb 2018
  • Loc: Louisville, KY

Posted 27 September 2022 - 03:11 PM

With SN1 after STF, its problems went away as well.

 

SN2 is clearly better than SN1, mainly with much better SNR (40.48dB vs. 21.44dB on Pickering crop, below):

 

Here is SN2:

 

Pickering_SN2.jpg

 

Here is SN1:

 

Pickering_SN1.jpg



#91 MAT_Blue

MAT_Blue

    Explorer 1

  • -----
  • Posts: 99
  • Joined: 08 Feb 2019
  • Loc: Bluffton, South Carolina, USA

Posted 27 September 2022 - 03:17 PM

Here's what worked for me.

I have an nVidia RTX3060.  I assume this will work for any RTX's in the 20xx, 30xx, or 40xx series but I can only testify that it works for my 3060.

 

nVidia driver 516.59
- the latest version as of 7/16/22  (last time I checked0
   when I run nvidia-smi, it says it's CUDA version 11.7

 

tensorflow 2.9.0

- https://storage.goog...86_64-2.9.0.zip
(can get any version, just change the version number in the URL)

 

CUDA 11.2
- cuda_11.2.0_460.89_win10
- from nVidia developers website

CUDNN 8.1.1 for CUDA 11.2
- cudnn-11.2-windows-x64-v8.1.1.33
- from nVidia developers website

 

Add environment variables:
  - add "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.2\bin" to PATH (system)
  - create TF_FORCE_GPU_ALLOW_GROWTH environment variable set to "true" (w/o quotes) (system)
 

Full instructions:
- install nVidia driver
- copy tensorflow.dll 2.9.0 to PixInsight\bin folder (rename the one that comes with PixInsight to .save)
- install CUDA 11.2 (runtime only)
- extract the CUDNN zip file and copy the \bin and \lib folders
    to C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.2
- set the two environment variables
- to enable the "CUDA" graph in task manager  (in Windows 10):
    Start->Settings->System->Display-Graphics settings (at bottom)->
      turn "Hardware-accelerated GPU scheduling" OFF
- reboot then run PixInsight

To see what "Hardware-accelerated CPU scheduling" is:
  https://devblogs.mic...gpu-scheduling/


  • SilverLitz likes this

#92 SilverLitz

SilverLitz

    Gemini

  • *****
  • Posts: 3,165
  • Joined: 17 Feb 2018
  • Loc: Louisville, KY

Posted 27 September 2022 - 03:44 PM

Here's what worked for me.

I have an nVidia RTX3060.  I assume this will work for any RTX's in the 20xx, 30xx, or 40xx series but I can only testify that it works for my 3060.

 

nVidia driver 516.59
- the latest version as of 7/16/22  (last time I checked0
   when I run nvidia-smi, it says it's CUDA version 11.7

 

tensorflow 2.9.0

- https://storage.goog...86_64-2.9.0.zip
(can get any version, just change the version number in the URL)

 

CUDA 11.2
- cuda_11.2.0_460.89_win10
- from nVidia developers website

CUDNN 8.1.1 for CUDA 11.2
- cudnn-11.2-windows-x64-v8.1.1.33
- from nVidia developers website

 

Add environment variables:
  - add "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.2\bin" to PATH (system)
  - create TF_FORCE_GPU_ALLOW_GROWTH environment variable set to "true" (w/o quotes) (system)
 

Full instructions:
- install nVidia driver
- copy tensorflow.dll 2.9.0 to PixInsight\bin folder (rename the one that comes with PixInsight to .save)
- install CUDA 11.2 (runtime only)
- extract the CUDNN zip file and copy the \bin and \lib folders
    to C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.2
- set the two environment variables
- to enable the "CUDA" graph in task manager  (in Windows 10):
    Start->Settings->System->Display-Graphics settings (at bottom)->
      turn "Hardware-accelerated GPU scheduling" OFF
- reboot then run PixInsight

To see what "Hardware-accelerated CPU scheduling" is:
  https://devblogs.mic...gpu-scheduling/

Halleluiah!  It works!  I have everything as CUDA 11.7, as that was how I downloaded it last week.

 

Time went from 8min to 50sec.


Edited by SilverLitz, 27 September 2022 - 03:45 PM.

  • Kerry D. Green and MAT_Blue like this

#93 DirtyRod

DirtyRod

    Surveyor 1

  • *****
  • Posts: 1,937
  • Joined: 23 Mar 2021
  • Loc: Arizona

Posted 01 October 2022 - 09:52 AM

Morning. Currently using the Windows standalone version of Starnet V2. My normal process is to copy the layer to a new workspace, export, and then run Starnet. Sometimes, including today, it just gives me the error that it only supports 16 bit Color or Grayscale even though the image is 16 bit color. 

 

I've noticed that I get this message more often when I use a mask prior to exporting. Even if I flatten the layers, tell Gimp not to export the layers, or change the precision it sill will not recognize the file. If I export the file prior to using the mask Starnet has no issues.

 

Open to ideas and suggestions. 



#94 averageplutoenjoyer

averageplutoenjoyer

    Explorer 1

  • -----
  • Posts: 70
  • Joined: 16 Apr 2022

Posted 02 October 2022 - 06:15 AM

Im using a windows 10 laptop, pretty new, ive only had it a few months.

Whenever I try to run starnet im always met with the error 'The application was unable to start correctly (0xc0000142).' 

 

Also in every tutorial ive seen for starnet theres always been two applications, mono starnet and rgb starnet, but in every downlaod off the offical website ive tried there is only ever just 1 application just called 'starnet'.

 

I honestly have no idea what im doing wrong, ive tried following so many different tutorials and always in one way or another im met with that error.



#95 Skybound

Skybound

    Lift Off

  • -----
  • Posts: 1
  • Joined: 09 Dec 2019

Posted 05 October 2022 - 04:48 PM

Hi,  Has anyone attempted to build a mosaic using "Gradient Mosaic Merge" after applying StarNet v2 to the mosaic panels?  I tried this following the star alignment to the 2X2 mosaic panels I was attempting to merge.  The star alignment worked fine on the for panels, and generated both the starless images and star masks.  However, when using Gradient Mosaic Merge in either "overlay" or "average" mode, I got only 1 of the 4 panels in the "merged" image with others "black" for overlay case, and got what looks like a simple pixelmath add in the average case.  Any ideas or help?

 

Thanks,

 

David



#96 AstroTupelo

AstroTupelo

    Lift Off

  • *****
  • Posts: 12
  • Joined: 19 Jan 2021

Posted 05 October 2022 - 11:16 PM

I am trying to use the Windows GUI version and am also getting the "The application was unable to start correctly (0xc0000142)" message on my desktop. I believe my computer meets the minimum system requirements and I have other programs installed that required up to date systems, such as programs using AI, and they work. I was able to get it to work on my laptop but wish I could figure out why my desktop won't run it, otherwise I will be moving images over to the laptop just for the Starnet processing step. I would appreciate any suggestions.


Edited by AstroTupelo, 05 October 2022 - 11:25 PM.


#97 nekitmm

nekitmm

    Vendor - StarNet Software

  • *****
  • Vendors
  • topic starter
  • Posts: 227
  • Joined: 25 Apr 2018

Posted 06 October 2022 - 12:43 AM

Im using a windows 10 laptop, pretty new, ive only had it a few months.

Whenever I try to run starnet im always met with the error 'The application was unable to start correctly (0xc0000142).' 

 

Also in every tutorial ive seen for starnet theres always been two applications, mono starnet and rgb starnet, but in every downlaod off the offical website ive tried there is only ever just 1 application just called 'starnet'.

 

I honestly have no idea what im doing wrong, ive tried following so many different tutorials and always in one way or another im met with that error.

 

I am trying to use the Windows GUI version and am also getting the "The application was unable to start correctly (0xc0000142)" message on my desktop. I believe my computer meets the minimum system requirements and I have other programs installed that required up to date systems, such as programs using AI, and they work. I was able to get it to work on my laptop but wish I could figure out why my desktop won't run it, otherwise I will be moving images over to the laptop just for the Starnet processing step. I would appreciate any suggestions.

What kind of CPUs do you guys have? Do they support AVX instructions? This is the only limitation that I am aware of.



#98 gnn2000

gnn2000

    Lift Off

  • -----
  • Posts: 1
  • Joined: 07 Mar 2009

Posted 07 October 2022 - 10:18 AM

Hi

 

first thing I'm a 5 day old user of Pixinsight so this may be my lack of understanding, so apologies if I'm missing something obvious.

 

I've installed Starnet2 on Pixinisight 1.8.9-1 under Windows 10  Ryzen 5600X 64gb ram

 

I can run Starnet2 and the Stars are removed which is brilliant, however when i try to create a starmask i get the following error.

 

Starting star removal procedure...
Window size: 512
Stride: 256
Image size: 4359x3277
Number of channels: 3
Color space: RGB
Bits per sample: 32
Has alpha channels: false
Float sample: true
Restoring neural network checkpoint...
Processing 234 image tiles: done
*** Error: Singular matrix in RGB working color space initialization.
Reading swap files...
7910.214 MiB/s
<* failed *>

 

Im not sure whether this is a Pixinsight thing or Starnet2 related.

 

Starnet 1 creates a mask fine btw

 

Any advice or guidance would be appreciated.

 

Thanks Glen



#99 averageplutoenjoyer

averageplutoenjoyer

    Explorer 1

  • -----
  • Posts: 70
  • Joined: 16 Apr 2022

Posted 07 October 2022 - 07:17 PM

What kind of CPUs do you guys have? Do they support AVX instructions? This is the only limitation that I am aware of.

I just had a look at the CPUs specs, from what I can see it doesnt support avx instructions. I assume this is the problem, is there any way around it?

 

If not would there be a way to run it on my mac (running 10.13 high sierra)? I haven't had any luck with that either


Edited by averageplutoenjoyer, 07 October 2022 - 07:20 PM.


#100 sarmen2

sarmen2

    Sputnik

  • *****
  • Posts: 49
  • Joined: 14 Mar 2011

Posted 11 October 2022 - 05:32 PM

Hello,

 

Is there a version of StarNet that will run as a Pixinsight module under Linux - Ubuntu? There was a download available at: https://www.starnetastro.com/download/  I followed the instructions in the README and it did not work. PI does not see StarNet (V2) as a module to install. Are there working instructions?

 

thank you




CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.


Recent Topics






Cloudy Nights LLC
Cloudy Nights Sponsor: Astronomics