Jump to content

  •  

CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.

Photo

StarNet: star removing software

astrophotography
  • Please log in to reply
240 replies to this topic

#1 nekitmm

nekitmm

    Explorer 1

  • *****
  • topic starter
  • Posts: 64
  • Joined: 25 Apr 2018

Posted 23 February 2019 - 07:37 PM

Hi, there!

 

For a while by now I was developing a neural network-based software that can make star removal from images simpler. The idea was to train a neural network that can remove stars in one simple step, avoiding time-consuming-thousand-steps procedures with dozens of parameters to tune. It is still not perfect and leaves artifacts when removing large stars, but with a bit of work with clone stamp in Photoshop it can yield a good starless image in a little time.

 

There are two versions of the code:

 

1. Python version for more advanced users. Available on GitHub here. The whole code is open-sourced. It is harder to use as you need to install Python, Tensorflow, etc, but give you more possibilities. For example, you can train the neural network some more on your own data, or modify the code for your needs.

 

2. Pre-compiled binary version that requires no installation, just download and run. Available through here. It is extremely straightforward to use, but for now only Windows x64 version is available. I am working on compiling version for macOS.

 

You can find much more information and lots of examples in my releases on Astrobin:

 

https://www.astrobin.com/339099/

 

https://www.astrobin.com/371108/

 

https://www.astrobin.com/392222/

 

GitHub repo also contains a lot of information and example (scroll to the bottom of the page). I don't want to repeat everything in this post, since that would make it excessively long.

 

Let me know what you think!

 

Best,

Nikita


  • H-Alfa, lambermo, psandelle and 13 others like this

#2 maadscientist

maadscientist

    Viking 1

  • *****
  • Posts: 950
  • Joined: 08 May 2013
  • Loc: Atlanta, Georgia and Deerlick Astronomy Village

Posted 23 February 2019 - 09:13 PM

Great! Thank you very much for sharing.


  • nekitmm likes this

#3 astro_1

astro_1

    Ranger 4

  • -----
  • Posts: 367
  • Joined: 14 Aug 2014
  • Loc: USA, Omaha, Nebraska

Posted 23 February 2019 - 10:16 PM

Yes, thanks for sharing.!!


  • nekitmm likes this

#4 mashirts

mashirts

    Messenger

  • *****
  • Posts: 429
  • Joined: 18 Sep 2014
  • Loc: Elgin, TX

Posted 23 February 2019 - 10:36 PM

Yep another thanks for sharing!
  • nekitmm likes this

#5 nekitmm

nekitmm

    Explorer 1

  • *****
  • topic starter
  • Posts: 64
  • Joined: 25 Apr 2018

Posted 24 February 2019 - 02:37 PM

Thanks all!

 

Please, let me know if you have any questions or problems using it!



#6 Oleg Astro

Oleg Astro

    Astronomy Linux

  • -----
  • Vendors
  • Posts: 418
  • Joined: 12 Aug 2008
  • Loc: Albufeira, Portugal

Posted 24 February 2019 - 05:29 PM

Hello,

I can't run it on Ubuntu 16.04 based system:

./starnet.py transform 2_s.tif
./starnet.py: line 14: $'\r': command not found
' @ error/constitute.c/WriteImage/1028.
' @ error/constitute.c/WriteImage/1028.
' @ error/constitute.c/WriteImage/1028.
./starnet.py: line 18: $'\r': command not found
./starnet.py: line 20: os.environ[TF_CPP_MIN_LOG_LEVEL]: command not found
./starnet.py: line 21: $'\r': command not found
./starnet.py: line 22: $'\r': command not found
./starnet.py: line 23: epochs: command not found
./starnet.py: line 27: batch: command not found
./starnet.py: line 30: steps: command not found
./starnet.py: line 31: output_freq: command not found
./starnet.py: line 32: verbose: command not found
./starnet.py: line 33: images: command not found
./starnet.py: line 34: log_freq: command not found
./starnet.py: line 35: gen_plots: command not found
./starnet.py: line 37: learning_rates: command not found
./starnet.py: line 40: stride: command not found
./starnet.py: line 42: $'\r': command not found
./starnet.py: line 43: syntax error near unexpected token `sys.argv'
'/starnet.py: line 43: `if len(sys.argv) > 1:



#7 nekitmm

nekitmm

    Explorer 1

  • *****
  • topic starter
  • Posts: 64
  • Joined: 25 Apr 2018

Posted 24 February 2019 - 05:38 PM

Hi, Oleg!

You are trying to run it with bash instead of Python. The correct command will be something like:

python starnet.py transform ...

Make sure you have Python and tensorflow installed! You can find more information on the GitHub page!

Edited by nekitmm, 24 February 2019 - 05:39 PM.


#8 Oleg Astro

Oleg Astro

    Astronomy Linux

  • -----
  • Vendors
  • Posts: 418
  • Joined: 12 Aug 2008
  • Loc: Albufeira, Portugal

Posted 24 February 2019 - 06:20 PM

Hi, Oleg!

You are trying to run it with bash instead of Python. The correct command will be something like:

python starnet.py transform ...

Make sure you have Python and tensorflow installed! You can find more information on the GitHub page!

python starnet.py transform 2_s.tif

Restoring previous state of the model...
Traceback (most recent call last):
  File "starnet.py", line 103, in <module>
    stride = stride)
  File "/home/astronom/starnet/transform.py", line 50, in transform
    saver.restore(sess, "./model.ckpt")
  File "/home/astronom/anaconda3/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1717, in restore
    + compat.as_text(save_path))
ValueError: The passed save_path is not a valid checkpoint: ./model.ckpt



#9 nekitmm

nekitmm

    Explorer 1

  • *****
  • topic starter
  • Posts: 64
  • Joined: 25 Apr 2018

Posted 24 February 2019 - 06:31 PM

Oleg,

you did not download the weights for the network. Please see the GitHub page for instructions!

#10 Oleg Astro

Oleg Astro

    Astronomy Linux

  • -----
  • Vendors
  • Posts: 418
  • Joined: 12 Aug 2008
  • Loc: Albufeira, Portugal

Posted 24 February 2019 - 06:56 PM

Ok!

It works.

 

...

...

Transforming input image... 99%
Transforming input image... 99%
Transforming input image... 99%
Transforming input image... 99%
Transforming input image... 100%
Transforming input image... Done!
Saving output image...
Done!
Saving mask...
Done!
Total time taken: 2.2 minutes


  • nekitmm likes this

#11 nekitmm

nekitmm

    Explorer 1

  • *****
  • topic starter
  • Posts: 64
  • Joined: 25 Apr 2018

Posted 24 February 2019 - 07:50 PM

Ok!

It works.

 

...

...

Transforming input image... 99%
Transforming input image... 99%
Transforming input image... 99%
Transforming input image... 99%
Transforming input image... 100%
Transforming input image... Done!
Saving output image...
Done!
Saving mask...
Done!
Total time taken: 2.2 minutes

Excellent, glad to hear that!



#12 Der_Pit

Der_Pit

    Viking 1

  • -----
  • Posts: 837
  • Joined: 07 Jul 2018
  • Loc: La Palma

Posted 26 February 2019 - 08:55 AM

Whee, this is cool!  waytogo.gif

 

Just wanted to try, but the tensorflow package in my distribution doesn't like my Python (3.7).

First need to fix that shocked.gif


  • nekitmm likes this

#13 Der_Pit

Der_Pit

    Viking 1

  • -----
  • Posts: 837
  • Joined: 07 Jul 2018
  • Loc: La Palma

Posted 26 February 2019 - 10:36 AM

So I had to install tensorflow 1.13 to make it work with Python 3.7, but now it runs.  There's some error messages about deprecated stuff, but it runs nicely.  I tried it on one of my images here.  Absolutely amazing how well it works - likely because my scope/cam combination is similar enough to yours.

And as it looks that might be a very nice tool (just) for star mask generation!

 

 

Attached Thumbnails

  • M42_final.tif_starless.jpg

  • DuncanM and nekitmm like this

#14 nekitmm

nekitmm

    Explorer 1

  • *****
  • topic starter
  • Posts: 64
  • Joined: 25 Apr 2018

Posted 26 February 2019 - 04:55 PM

So I had to install tensorflow 1.13 to make it work with Python 3.7, but now it runs.  There's some error messages about deprecated stuff, but it runs nicely.  I tried it on one of my images here.  Absolutely amazing how well it works - likely because my scope/cam combination is similar enough to yours.

And as it looks that might be a very nice tool (just) for star mask generation!

Nice! I am glad to see that you got a very good result! Yes, you are right, it indeed works better when the image is close to the training data used!

 

Not sure about its usage for star mask generation, I think classical methods will do a bit better here... Maybe not, that will be for users to decide.

 

The goal I had in mind when developing it, was to use starless images to push up nebulosity in images without blowing up stars!



#15 futuneral

futuneral

    Apollo

  • *****
  • Posts: 1004
  • Joined: 27 Dec 2014
  • Loc: Phoenix, AZ

Posted 26 February 2019 - 09:57 PM

Nice! I am glad to see that you got a very good result! Yes, you are right, it indeed works better when the image is close to the training data used!

 

Not sure about its usage for star mask generation, I think classical methods will do a bit better here... Maybe not, that will be for users to decide.

 

The goal I had in mind when developing it, was to use starless images to push up nebulosity in images without blowing up stars!

The tool worked great for me too. I just used the binary version.

The idea about star mask generation Der_Pit is talking about is actually very useful and exactly for the purpose you're describing. After you get a starless image, it's easy (basically two clicks in photoshop) to generate a star mask that exactly matches your image. Classic tools will always approximate stars as Gaussian blobs and you need to fiddle with settings to get all the sizes right (often by combining several masks). The reason this is important is because after you adjust the nebulosity, you still need to paint your stars back. And if your mask is not accurate, you'll get nasty halos around stars.

 

I just tried using the mask generated using the starless image produced by your program and it gives me crazy levels of freedom on how to adjust the nebula and still have the natural looking stars. Even extreme manipulations didn't produce noticeable halos.

 

Thanks for this amazing tool!

 

Here is an example of a mask generated using the starless image:

 

star_mask.jpg


Edited by futuneral, 26 February 2019 - 09:58 PM.

  • nekitmm, RossW and Der_Pit like this

#16 nekitmm

nekitmm

    Explorer 1

  • *****
  • topic starter
  • Posts: 64
  • Joined: 25 Apr 2018

Posted 27 February 2019 - 12:49 AM

The tool worked great for me too. I just used the binary version.

The idea about star mask generation Der_Pit is talking about is actually very useful and exactly for the purpose you're describing. After you get a starless image, it's easy (basically two clicks in photoshop) to generate a star mask that exactly matches your image. Classic tools will always approximate stars as Gaussian blobs and you need to fiddle with settings to get all the sizes right (often by combining several masks). The reason this is important is because after you adjust the nebulosity, you still need to paint your stars back. And if your mask is not accurate, you'll get nasty halos around stars.

I see now, yes, that seems to be an interesting way to use it!

 

Thanks!



#17 Jeff2011

Jeff2011

    Gemini

  • *****
  • Posts: 3494
  • Joined: 01 Jan 2013
  • Loc: Sugar Land, TX

Posted 27 February 2019 - 02:46 AM

I had some trouble setting up the environment.  I was also using Anaconda 3.7 but got the following error

 

ModuleNotFoundError: No module named 'numpy.core._multiarray_umath'
ImportError: numpy.core.multiarray failed to import

 

I upgraded numpy to 1.16.2 and the error went away.

 

Pretty amazing results.  I can definitely use this to replace narrowband stars with RGB stars.  Thanks for making this available.  How long did it take you to write it?

 

 

 

Attached Thumbnails

  • Wiz.tif_starless.jpg

  • bilgebay and nekitmm like this

#18 nekitmm

nekitmm

    Explorer 1

  • *****
  • topic starter
  • Posts: 64
  • Joined: 25 Apr 2018

Posted 27 February 2019 - 03:31 AM

Pretty amazing results.  I can definitely use this to replace narrowband stars with RGB stars.  Thanks for making this available.  How long did it take you to write it?

I am glad to see that my work is helpful!

 

It is a continuing work, I was working on it for more than one year now.

 

The term "write" is not exactly applicable though. This is machine learning model, so it is much more about creating training data (which was a hell of a work), then developing an architecture of neural network and training it. Latter takes a lot of computer power and time as well.

 

This is still work in progress, I am hoping to get even better results and also create a PI plugin.

 

Thank you for your interest!


Edited by nekitmm, 27 February 2019 - 03:32 AM.


#19 Oleg Astro

Oleg Astro

    Astronomy Linux

  • -----
  • Vendors
  • Posts: 418
  • Joined: 12 Aug 2008
  • Loc: Albufeira, Portugal

Posted 27 February 2019 - 05:45 AM

Nikita,
your software also removes bright globules frown.gif

 

Attached Thumbnails

  • crop_.jpg

  • bilgebay likes this

#20 Jeff2011

Jeff2011

    Gemini

  • *****
  • Posts: 3494
  • Joined: 01 Jan 2013
  • Loc: Sugar Land, TX

Posted 27 February 2019 - 09:03 AM

 

This is still work in progress, I am hoping to get even better results and also create a PI plugin.

 

 

You read my mind. That would be awesome to run this from PI.  Any thoughts about making it work for linear data?  Then it could be used for tone mapping.

 

Also thanks for the explanation on how to train your dragon smile.gif .  I am an old C programmer so this neural network stuff is foreign to me but fascinating. 


  • nekitmm likes this

#21 nekitmm

nekitmm

    Explorer 1

  • *****
  • topic starter
  • Posts: 64
  • Joined: 25 Apr 2018

Posted 27 February 2019 - 07:26 PM

Nikita,
your software also removes bright globules frown.gif

No one said it was perfect!

 

Although for this image I think your expectations are way too high!



#22 nekitmm

nekitmm

    Explorer 1

  • *****
  • topic starter
  • Posts: 64
  • Joined: 25 Apr 2018

Posted 27 February 2019 - 07:40 PM

Any thoughts about making it work for linear data?

The problem with linear data is preparing a training set. To create it you will need some very good method. For example, in this case I used good old clone stamp in PS. It is extremely tedious and takes a lot of time, but it can yield starless image of very good quality, which works very well for training. The catch here is that if your training data has some artifacts from star removal, the neural network will learn to reproduce them!

 

So you will need a perfect star removal method for linear data to begin with. I did't think about this much, but for now I have no idea how to do that, PS will not do good here I think...

 

Actually, Rick Stevenson from Astrobin (not sure if he is also on CN here) was interested in this, and he said that he actually can do a good job removing stars from linear data in PI. He was also interested in training his own model.


Edited by nekitmm, 27 February 2019 - 07:49 PM.


#23 futuneral

futuneral

    Apollo

  • *****
  • Posts: 1004
  • Joined: 27 Dec 2014
  • Loc: Phoenix, AZ

Posted 27 February 2019 - 09:57 PM

The problem with linear data is preparing a training set. To create it you will need some very good method. For example, in this case I used good old clone stamp in PS. It is extremely tedious and takes a lot of time, but it can yield starless image of very good quality, which works very well for training. The catch here is that if your training data has some artifacts from star removal, the neural network will learn to reproduce them!

 

So you will need a perfect star removal method for linear data to begin with. I did't think about this much, but for now I have no idea how to do that, PS will not do good here I think...

 

Actually, Rick Stevenson from Astrobin (not sure if he is also on CN here) was interested in this, and he said that he actually can do a good job removing stars from linear data in PI. He was also interested in training his own model.

I think you should crowd source it. Many people here do (did) go through the troubles of removing stars in linear images. If you start a thread and request people to submit the "before" and "after" images you may be able to gather enough data for training.

 

P.S. I actually think it will work better on linear images as it's much easier to distinguish stars there vs non-linear data - the brightness difference is much larger. 


  • Jeff2011, nekitmm and RossW like this

#24 nekitmm

nekitmm

    Explorer 1

  • *****
  • topic starter
  • Posts: 64
  • Joined: 25 Apr 2018

Posted 28 February 2019 - 07:31 PM

I think you should crowd source it. Many people here do (did) go through the troubles of removing stars in linear images. If you start a thread and request people to submit the "before" and "after" images you may be able to gather enough data for training.

 

P.S. I actually think it will work better on linear images as it's much easier to distinguish stars there vs non-linear data - the brightness difference is much larger. 

That could be a good idea actually! Thanks!

 

I am still working on current (nonlinear) version, but later on I might try doing that!

 

Thanks!



#25 HxPI

HxPI

    Apollo

  • -----
  • Posts: 1478
  • Joined: 05 Sep 2013
  • Loc: Virginia Beach, VA

Posted 03 March 2019 - 01:18 AM

I tried the program and got nice results. However, I’m not a fan of CLIs! It would be nice if it had a GUI, at some point. A plug-in for Pixinsight would be perfect also! Thanks for sharing.

 

Ciao,

Mel


  • nekitmm likes this


CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.


Recent Topics





Also tagged with one or more of these keywords: astrophotography



Cloudy Nights LLC
Cloudy Nights Sponsor: Astronomics