Jump to content

  •  

CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.

Photo

StarNet: star removing software

astrophotography
  • Please log in to reply
297 replies to this topic

#226 nekitmm

nekitmm

    Vendor - StarNet Software

  • *****
  • Vendors
  • topic starter
  • Posts: 74
  • Joined: 25 Apr 2018

Posted 19 September 2019 - 01:41 PM

1) The star trails can be any length - a function of the comet's arc second per minute drift and the length of the exposure

2) The direction of the star trail can be any angle - a function of the camera orientation and the comet orbit

None of these should be problematic.

 

The first one should not be a problem at all given that you have reasonable number of different star trail length in your training data.

 

The second one is solved using image augmentation - you just randomly rotate input images during training.

 

I would be more worried about 'structure' of star trails since there could be different sequences of filters, lengths of exposures, various gaps, etc. etc. However if all the images are like the example you attached everything should be fine.


Edited by nekitmm, 19 September 2019 - 01:42 PM.


#227 Tonk

Tonk

    Cosmos

  • *****
  • Posts: 9,036
  • Joined: 19 Aug 2004
  • Loc: Leeds, UK, 54N

Posted 19 September 2019 - 02:52 PM

 

The second one is solved using image augmentation - you just randomly rotate input images during training.

Excellent - I'd already come to that conclusion while walking my dogs tonight (lots of thinking time)

 

 

The first one should not be a problem at all given that you have reasonable number of different star trail length in your training data.

That might be a problem - it make take years to collect enough  random length samples. I wonder if some sort if image stretch along the trail direction augmentation trick would be workable

I haven't looked into you stuff in any detail yet - but are there some instructions/directions on how to train?

Thanks!


Edited by Tonk, 19 September 2019 - 02:57 PM.


#228 nekitmm

nekitmm

    Vendor - StarNet Software

  • *****
  • Vendors
  • topic starter
  • Posts: 74
  • Joined: 25 Apr 2018

Posted 19 September 2019 - 04:44 PM

That might be a problem - it make take years to collect enough  random length samples. I wonder if some sort if image stretch along the trail direction augmentation trick would be workable

I thought you already had a number of images and now want to train... Well, you need a decent number of training images anyway, no matter what kind of neural network you want to train, so there are no shortcuts here. One option could be to create stacks of different length from the same data set, but you will still need multiple images to get good results.

 

I haven't looked into you stuff in any detail yet - but are there some instructions/directions on how to train?

Not sure what you mean by this, you should train it as any other neural network. There is no single set of instructions here... This is more like an art at the moment, trial and error, and requires some experience to get best results.


Edited by nekitmm, 19 September 2019 - 04:46 PM.


#229 Tonk

Tonk

    Cosmos

  • *****
  • Posts: 9,036
  • Joined: 19 Aug 2004
  • Loc: Leeds, UK, 54N

Posted 19 September 2019 - 04:58 PM

you should train it as any other neural network


Having never done it ... I guess I need to get a book.

 

 

I thought you already had a number of images and now want to train

I have a large collection of colour DSLR images from the last 15 years - but as of this year I've switched to 7 filter monochrome frame sets (L, R, G, B, C2, CO, CN)- so effectively starting over again - and this in the middle of the longest comet drought for years!

OK I either need to spend some years collecting data again - or go for another approach all together


Edited by Tonk, 19 September 2019 - 05:04 PM.


#230 kingjamez

kingjamez

    Vanguard

  • *****
  • Posts: 2,160
  • Joined: 03 Oct 2006
  • Loc: Fairfax, VA

Posted 19 September 2019 - 06:16 PM

I processed this photo by extracting the stars then doing all of the color manipulation on the starless image then using the max () command in pixelmath to recombine.

 

It's got some artifacts that are easy to spot due to the star removal process leaving behind rings around the stars.

 

Any tips on better preparing an image to match the training dataset? Can I do some sort of optimized preprocessing on an image just for the purpose of making it optimal for Starnet++'s NN?

 

It's an amazing tool!

 

-Jim

Attached Thumbnails

  • Bubble_SHO_Pub1.jpg

  • nekitmm likes this

#231 kingjamez

kingjamez

    Vanguard

  • *****
  • Posts: 2,160
  • Joined: 03 Oct 2006
  • Loc: Fairfax, VA

Posted 19 September 2019 - 07:00 PM

Right you are!

 

The proper PixelMath syntax is max(weight1*starless + weight2*starry)

 

Very different effects from adding or max. Experiment and see what you like best.

No, that doesn't work either. By adding them together there is nothing for max to evaluate.

 

The syntax is "max(weight1*starless, weight2*starry)".

 

-Jim


Edited by kingjamez, 19 September 2019 - 07:01 PM.

  • Salacious B Crumb likes this

#232 nekitmm

nekitmm

    Vendor - StarNet Software

  • *****
  • Vendors
  • topic starter
  • Posts: 74
  • Joined: 25 Apr 2018

Posted 20 September 2019 - 01:06 PM

Having never done it ... I guess I need to get a book.

There is plethora of online materials on the topic!



#233 Tonk

Tonk

    Cosmos

  • *****
  • Posts: 9,036
  • Joined: 19 Aug 2004
  • Loc: Leeds, UK, 54N

Posted 20 September 2019 - 05:20 PM

There is plethora of online materials on the topic!


Never mind - sigh - you seemed to have failed to understand that what I originally asked was does your software provide a built in means to feed it training data (e.g. command switches or menu items or whatever you used). I wasn't asking about the theory of designing training sets.

The possible answers I was expecting were "yes - go here to see my instructions on how to feed it the training data" or " yes you could - but its undocumented" or "no - its canned with no means for you (the user) to change the training data and you can only use it to process your images".

#234 nekitmm

nekitmm

    Vendor - StarNet Software

  • *****
  • Vendors
  • topic starter
  • Posts: 74
  • Joined: 25 Apr 2018

Posted 20 September 2019 - 09:15 PM

Never mind - sigh - you seemed to have failed to understand that what I originally asked was does your software provide a built in means to feed it training data (e.g. command switches or menu items or whatever you used). I wasn't asking about the theory of designing training sets.

The possible answers I was expecting were "yes - go here to see my instructions on how to feed it the training data" or " yes you could - but its undocumented" or "no - its canned with no means for you (the user) to change the training data and you can only use it to process your images".

https://github.com/nekitmm/starnet

 

PS. If you want specific answers make sure to ask specific questions.


Edited by nekitmm, 20 September 2019 - 09:47 PM.


#235 georgian82

georgian82

    Viking 1

  • *****
  • Posts: 619
  • Joined: 16 Jan 2016
  • Loc: Healdsburg, California

Posted 24 September 2019 - 01:08 PM

Hi guys,

 

I was looking into this new StarNet module for PixInsight so that I can work off a bigger file (as opposed to 16-bit) but I could not get it to work on my computer which is a Mac. 

 

Is this module available for Mac users? 

 

Once the folders are downloaded, how do I added into PixInsight?

 

Thanks a lot,

 

Sebastian



#236 pfile

pfile

    Fly Me to the Moon

  • -----
  • Posts: 5,369
  • Joined: 14 Jun 2009

Posted 24 September 2019 - 01:16 PM

it's not yet available for OSX. juan posted the other day in the PI forum that Nikita had released the source but juan is busy with finishing the new version of PI and so has not worked on it. so i guess it will be a little while. it's available for windows and linux only right now.

 

rob


  • georgian82 likes this

#237 georgian82

georgian82

    Viking 1

  • *****
  • Posts: 619
  • Joined: 16 Jan 2016
  • Loc: Healdsburg, California

Posted 24 September 2019 - 01:31 PM

it's not yet available for OSX. juan posted the other day in the PI forum that Nikita had released the source but juan is busy with finishing the new version of PI and so has not worked on it. so i guess it will be a little while. it's available for windows and linux only right now.

 

rob

Thanks a lot Rob for letting me know!

 

Cheers,



#238 nekitmm

nekitmm

    Vendor - StarNet Software

  • *****
  • Vendors
  • topic starter
  • Posts: 74
  • Joined: 25 Apr 2018

Posted 02 October 2019 - 01:57 PM

If it's not too much trouble, please do compile a linux version of starnet++

 

I have been running Mac PI (no starnet) and using mac starnet++ to process a .tif, but I have a headless linux server that's very fast and would be great to run starnet++ on there.

 

Might be best to transfer my whole PI world over there, but since it's headless I need to install loads of other packages first.

Sorry for the delay! I just released the linux version. You can find it here: https://sourceforge....net/files/v1.1/

 

Let me know if it works!


  • Der_Pit likes this

#239 Der_Pit

Der_Pit

    Apollo

  • *****
  • Posts: 1,105
  • Joined: 07 Jul 2018
  • Loc: La Palma

Posted 04 October 2019 - 05:24 PM

Sorry for the delay! I just released the linux version. You can find it here: https://sourceforge....net/files/v1.1/

 

Let me know if it works!

Seems the libtiff is missing.  It has only the link libtiff.so.3 -> libtiff.so.3.9.7, but the latter one isn't in the archive.



#240 nekitmm

nekitmm

    Vendor - StarNet Software

  • *****
  • Vendors
  • topic starter
  • Posts: 74
  • Joined: 25 Apr 2018

Posted 04 October 2019 - 06:09 PM

Seems the libtiff is missing.  It has only the link libtiff.so.3 -> libtiff.so.3.9.7, but the latter one isn't in the archive.

Thanks! I updated the release, should work fine now!


Edited by nekitmm, 04 October 2019 - 06:11 PM.


#241 Der_Pit

Der_Pit

    Apollo

  • *****
  • Posts: 1,105
  • Joined: 07 Jul 2018
  • Loc: La Palma

Posted 05 October 2019 - 05:53 AM

Thanks!

Working it did before, I had installed libtiff3 on my system, so it was using that one.  It runs the examples fine on my system waytogo.gif

 

One possibility for improvement would be the location of the checkpoint files.  Right now it (only) looks in the current directory.  It would make a system-wide install easier if it could also try the 'standard' directories like /usr/local/share/starnet++/ or /usr/share/starnet++ 

Right now I'm using a shellscript-wrapper around it, so I can use it in any directory...



#242 pyrasanth

pyrasanth

    Vanguard

  • *****
  • Posts: 2,214
  • Joined: 08 Jan 2016

Posted 03 November 2019 - 05:15 AM

Did you mean standalone version or PI module?

 

I just released PI module version (you can find it here), but when I returned and re-read your message I suddenly realized that you probably were asking about standalone version... Standalone version I wasn't going to compile since Python version should run in Linux no problem, but if you want it, just let me know, I can do this no problem!

 

PS. If you going to use PI module for Linux, make sure to read README file. There is a problem with finding neural network weights in Linux since for some reason PI does not look for them in it's bin folder by default, the workaround is very simple though, you just need to cd into PI's bin folder in PI's console and then it works. Need to solve that in future versions but kinda no idea how to do that now.

I tested the PI version this morning for windows on an image of M31 crammed full of stars and it worked brilliantly. This module will be a game changer for me as I will be able to shoot narrow band targets & blend nicely coloured RGB star fields which will be awesome. Thanks for all the hard work you have pit into this coding- it is a stunning achievement. My computer is a Ryzen 1800 with 32 Gb of RAM & it worked really fast on a 90 Mb tiff image.


  • nekitmm likes this

#243 pyrasanth

pyrasanth

    Vanguard

  • *****
  • Posts: 2,214
  • Joined: 08 Jan 2016

Posted 03 November 2019 - 06:39 AM

I found a useful side effect of running Starnet through a mask. If you apply an inverted Lmask it shrinks blobby stars very nicely!


  • pfile and nekitmm like this

#244 H-Alfa

H-Alfa

    Messenger

  • -----
  • Posts: 483
  • Joined: 21 Sep 2006
  • Loc: Spain

Posted 03 November 2019 - 10:13 AM

I found a useful side effect of running Starnet through a mask. If you apply an inverted Lmask it shrinks blobby stars very nicely!

That's a good point! I didn't realized that it could be executed that way.

Enviado desde mi ANE-LX1 mediante Tapatalk
  • nekitmm likes this

#245 pfile

pfile

    Fly Me to the Moon

  • -----
  • Posts: 5,369
  • Joined: 14 Jun 2009

Posted 11 November 2019 - 05:27 PM

I found a useful side effect of running Starnet through a mask. If you apply an inverted Lmask it shrinks blobby stars very nicely!

that is a neat trick... i found that if you iterate on extracting the L and re-masking/re-running starnet you get a very nice star reduction effect.

 

rob



#246 nekitmm

nekitmm

    Vendor - StarNet Software

  • *****
  • Vendors
  • topic starter
  • Posts: 74
  • Joined: 25 Apr 2018

Posted 11 November 2019 - 10:41 PM

Hi, everyone!

 

It's been a long time, but I am finally able release PI module for MacOS! You can find it here.

 

Huge thanks to Rob Pfile for compiling this one!


Edited by nekitmm, 11 November 2019 - 10:42 PM.

  • psandelle, HxPI and CvBadengoth like this

#247 HxPI

HxPI

    Surveyor 1

  • -----
  • Posts: 1,555
  • Joined: 05 Sep 2013
  • Loc: Virginia Beach, VA

Posted 11 November 2019 - 11:45 PM

Hi, everyone!

 

It's been a long time, but I am finally able release PI module for MacOS! You can find it here.

 

Huge thanks to Rob Pfile for compiling this one!

Installed in PixInsight 1.8.8 on early 2013 MBP 15" Retina running macOS Sierra. Thanks for sharing.


Edited by HxPI, 11 November 2019 - 11:58 PM.

  • pfile likes this

#248 nekitmm

nekitmm

    Vendor - StarNet Software

  • *****
  • Vendors
  • topic starter
  • Posts: 74
  • Joined: 25 Apr 2018

Posted 12 November 2019 - 12:00 AM

Unable to install in PixInsight 1.8.8 on early 2013 MBP 15" Retina running macOS Sierra. Files are in pixinsight/bin folder but module install function doesn't find the module. Should this be working with my setup?

 

Also the included readme files says its a unix executable and unable to open with text editor. Is this correct?

Lol, of course this is not correct, this is a readme file, it should open with text editor! Are you sure you can not open it? This does not make any sense!



#249 HxPI

HxPI

    Surveyor 1

  • -----
  • Posts: 1,555
  • Joined: 05 Sep 2013
  • Loc: Virginia Beach, VA

Posted 12 November 2019 - 12:13 AM

Lol, of course this is not correct, this is a readme file, it should open with text editor! Are you sure you can not open it? This does not make any sense!

Yeah I figured it out. I renamed the readme file with .txt extension. The instructions allowed me to figure out how to install the module. 


  • nekitmm likes this

#250 nekitmm

nekitmm

    Vendor - StarNet Software

  • *****
  • Vendors
  • topic starter
  • Posts: 74
  • Joined: 25 Apr 2018

Posted 12 November 2019 - 12:20 AM

Yeah I figured it out. I renamed the readme file with .txt extension. The instructions allowed me to figure out how to install the module. 

Ok, I probably need to rename text files in the release to avoid this kind of confusion...




CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.


Recent Topics





Also tagged with one or more of these keywords: astrophotography



Cloudy Nights LLC
Cloudy Nights Sponsor: Astronomics