Jump to content

  •  

CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.

Photo

DeepSNR: New DL based software for noise reduction

  • Please log in to reply
31 replies to this topic

#1 nekitmm

nekitmm

    Vendor - StarNet Software

  • *****
  • Vendors
  • topic starter
  • Posts: 227
  • Joined: 25 Apr 2018

Posted 29 December 2022 - 08:29 PM

Hi, everyone!

 

I got a little free time around Christmas and decided to look into noise reduction.

 

There is a lot of research and open source code available, I ended up training a variation of NAFnet architecture on astro images.

 

I called the result DeepSNR and it is available here: http://deepsnrastro.com/

 

Note that currently there is only PI plugin that works on RGB images. I totally forgot about Greyscale images as always, will try to add later.

 

It is deployed in the same exact way as StarNet, so installation and usage instructions are the same. Note that there are two options to download: with and without Tensorflow libraries. If you have StarNet already you don't need to re-download and replace Tensorflow libs.

 

This is very fresh out the oven, I only did limited testing, so let me know how you find it working. Wei-Hao Wang was kind enough to test MacOS version, but the rest of the release materials are not really tested so far.

 

Also, it works on freshly integrated images with minimum postprocessing and is supposed to be applied early in the processing workflow.

 

Thanks!


Edited by nekitmm, 29 December 2022 - 08:31 PM.

  • Aleforge, Lead_Weight, Sky King and 2 others like this

#2 fmendes

fmendes

    Surveyor 1

  • *****
  • Posts: 1,742
  • Joined: 20 Apr 2020
  • Loc: Houston, TX

Posted 29 December 2022 - 08:52 PM

Great stuff! Just downloaded. Will try later today.


  • nekitmm likes this

#3 nekitmm

nekitmm

    Vendor - StarNet Software

  • *****
  • Vendors
  • topic starter
  • Posts: 227
  • Joined: 25 Apr 2018

Posted 31 December 2022 - 03:40 PM

Update:

 

Based on the feedback I see that is works really well for images from monochrome cameras, but not suitable for anything that has been debayered prior to integration. It does not handle the low-frequency noise produced by that (the same goes for things like walking noise). This makes sense since I only trained it on images taken with monochrome cameras, so should have mentioned that from the start. Will work on the version that handles debayered images as well when I have time!



#4 cyber

cyber

    Ranger 4

  • -----
  • Posts: 397
  • Joined: 01 Jun 2016
  • Loc: Belgium

Posted 31 December 2022 - 04:28 PM

Works great on RGB data from my mono ASI1600MM-Cool. Many thanks for making this available, Nikita!

 

I also tried DeepSNR on DSLR data and standard integrations don't respond well indeed. The result is shown in the bottom left image below. I then followed the comments in your AstroBin post and can confirm that those issues are completely solved when CFA Drizzle integration is used (bottom right image).

 

1672491884771.png


  • nekitmm likes this

#5 Der_Pit

Der_Pit

    Mercury-Atlas

  • *****
  • Posts: 2,874
  • Joined: 07 Jul 2018
  • Loc: La Palma

Posted 01 January 2023 - 11:29 AM

A standalone version, like for Starnet, would be really nice.  At least at some point...


  • nekitmm likes this

#6 nekitmm

nekitmm

    Vendor - StarNet Software

  • *****
  • Vendors
  • topic starter
  • Posts: 227
  • Joined: 25 Apr 2018

Posted 01 January 2023 - 10:13 PM

Works great on RGB data from my mono ASI1600MM-Cool. Many thanks for making this available, Nikita!

 

I also tried DeepSNR on DSLR data and standard integrations don't respond well indeed. The result is shown in the bottom left image below. I then followed the comments in your AstroBin post and can confirm that those issues are completely solved when CFA Drizzle integration is used (bottom right image).

Victor,

 

The result looks awesome! Thanks a lot for testing and confirming! I like how barely perceivable difference in the noise translates to drastically different results!


Edited by nekitmm, 01 January 2023 - 10:13 PM.

  • cyber likes this

#7 nekitmm

nekitmm

    Vendor - StarNet Software

  • *****
  • Vendors
  • topic starter
  • Posts: 227
  • Joined: 25 Apr 2018

Posted 01 January 2023 - 10:14 PM

A standalone version, like for Starnet, would be really nice.  At least at some point...

Yeah, it's definitely coming at some point!


  • Der_Pit likes this

#8 dmcarlsn

dmcarlsn

    Explorer 1

  • -----
  • Posts: 91
  • Joined: 08 Mar 2018

Posted 20 February 2023 - 10:16 PM

Hi Nikita,

 

I'm am trying DeepSNR on my image of C/2022 E3 (ZTF) and am impressed with the results I am seeing.

 

I script all of my image processing in PixInsight to capture the full process and make it easy to do multiple runs with varying parameters.

 

Would it be possible for you to update DeepSNR to support its GUI controls (i.e. Strength and Linear data) in its process parameters so that they can be controlled from a PJSR script?  Like you did in the recent update to StarNet2 (much appreciated by the way).  This would also make DeepSNR work in PI containers as the StarNet2 update did.

 

Thanks!


  • nekitmm likes this

#9 nekitmm

nekitmm

    Vendor - StarNet Software

  • *****
  • Vendors
  • topic starter
  • Posts: 227
  • Joined: 25 Apr 2018

Posted 21 February 2023 - 10:24 PM

Hi Nikita,

 

I'm am trying DeepSNR on my image of C/2022 E3 (ZTF) and am impressed with the results I am seeing.

 

I script all of my image processing in PixInsight to capture the full process and make it easy to do multiple runs with varying parameters.

 

Would it be possible for you to update DeepSNR to support its GUI controls (i.e. Strength and Linear data) in its process parameters so that they can be controlled from a PJSR script?  Like you did in the recent update to StarNet2 (much appreciated by the way).  This would also make DeepSNR work in PI containers as the StarNet2 update did.

 

Thanks!

Yes, you are correct, it has the same problem. Let me transfer all the recent updates from starnet, stay tuned!

 

Thank you!


Edited by nekitmm, 21 February 2023 - 10:25 PM.


#10 nekitmm

nekitmm

    Vendor - StarNet Software

  • *****
  • Vendors
  • topic starter
  • Posts: 227
  • Joined: 25 Apr 2018

Posted 25 February 2023 - 02:04 PM

Just released an update - the new version will apply auto-stretch if no STF set!



#11 jambi99

jambi99

    Viking 1

  • -----
  • Posts: 571
  • Joined: 14 Aug 2010
  • Loc: Ottawa , Canada

Posted 06 March 2023 - 07:11 PM

Just released an update - the new version will apply auto-stretch if no STF set!

Any chances this could be used for EAA? Like a plugin in sharpcap?


Edited by jambi99, 06 March 2023 - 09:52 PM.


#12 cyber

cyber

    Ranger 4

  • -----
  • Posts: 397
  • Joined: 01 Jun 2016
  • Loc: Belgium

Posted 12 June 2023 - 01:43 AM

DeepSNR is absolutely amazing for RGB images. However, it is currently not trained to run well on a monochrome file. 

 

I've figured out a way to improve the performance of DeepSNR on mono images. When you simply convert a mono image to RGB, the algorithm will produce lot of artefacts (fake stars). I figured this was due to autocorrelation between the 3 channels of an RGB converted image. So, my strategy was to create 2 linear copies of the mono file, inject random noise in them to simulate a different filter, and then all 3 channels are stretched with the same HT before RGB combination. 

 

The untreated mono data was used in the Red channel in the examples below. Noise was added to the G and B channel seprately. Adding more noise to the synthetic channels improves denoise performance in the red channel. However, excessive noise seems to introduce some smoothing and detail loss. More experimentation is needed, but this idea might be helpful for other users that want to run DeepSNR on a single monochrome dataset. 

 

Click images for full size (600 pix wide)

Each step doubles the amount of noise in the linear phase.

 

uniform tests1_2.jpg

 

uniform tests2_2.jpg


Edited by cyber, 12 June 2023 - 01:45 AM.

  • *Axel* and nekitmm like this

#13 nekitmm

nekitmm

    Vendor - StarNet Software

  • *****
  • Vendors
  • topic starter
  • Posts: 227
  • Joined: 25 Apr 2018

Posted 17 June 2023 - 05:44 PM

DeepSNR is absolutely amazing for RGB images. However, it is currently not trained to run well on a monochrome file. 

 

I've figured out a way to improve the performance of DeepSNR on mono images. When you simply convert a mono image to RGB, the algorithm will produce lot of artefacts (fake stars). I figured this was due to autocorrelation between the 3 channels of an RGB converted image. So, my strategy was to create 2 linear copies of the mono file, inject random noise in them to simulate a different filter, and then all 3 channels are stretched with the same HT before RGB combination. 

 

The untreated mono data was used in the Red channel in the examples below. Noise was added to the G and B channel seprately. Adding more noise to the synthetic channels improves denoise performance in the red channel. However, excessive noise seems to introduce some smoothing and detail loss. More experimentation is needed, but this idea might be helpful for other users that want to run DeepSNR on a single monochrome dataset. 

 

Click images for full size (600 pix wide)

Each step doubles the amount of noise in the linear phase.

 

attachicon.gif uniform tests1_2.jpg

 

attachicon.gif uniform tests2_2.jpg

Wow, really nice thinking, thanks so much for sharing!

 

One of these days I will find time to train it on Greyscale images, but honestly this hack makes perfect sense and works so well that it could be better just to implement it in the plugin... Will do more testing once I have time!



#14 cyber

cyber

    Ranger 4

  • -----
  • Posts: 397
  • Joined: 01 Jun 2016
  • Loc: Belgium

Posted 21 June 2023 - 04:31 PM

I wrote a script to automate the process. 

Here's a link to the PixInsight forum post where you can download the script: https://pixinsight.c...edeepsnr.21031/


Edited by cyber, 21 June 2023 - 04:31 PM.

  • nekitmm likes this

#15 nekitmm

nekitmm

    Vendor - StarNet Software

  • *****
  • Vendors
  • topic starter
  • Posts: 227
  • Joined: 25 Apr 2018

Posted 29 June 2023 - 11:51 PM

I wrote a script to automate the process. 

Here's a link to the PixInsight forum post where you can download the script: https://pixinsight.c...edeepsnr.21031/

The results look really good, thanks a lot for this awesome contribution!


  • gianpri likes this

#16 astronoom

astronoom

    Lift Off

  • -----
  • Posts: 1
  • Joined: 31 Aug 2023

Posted 31 August 2023 - 01:23 AM

I've recently updated to a newer AMD GPU, and I was able to get hardware acceleration working for StarX and BlurX using the tensorflow dll's included in the latest DirectML download (from their github). This has unfortunately broken DeepSNR for me. My understanding is that it might be due to star/blurX using tensorflow1, while deepSNR/starnet2 are on tensorflow2, which is not included in DirectML.

Would you have any hints, input, or help that might let me get DeepSNR working with my AMD hardware? Ultimately I'd love to see it GPU accelerated, but failing that, is there any way to have star/blurX running accelerated while deepSNR could work on the CPU?

Currently it just fails to run at all with the following error:

 

*** Error: Could not find checkpoint file in 'C:\Program Files\PixInsight\bin\'!

I can fix it by replacing the tensorflow.dll with the one provided, but this puts everything back to CPU and is obviously not ideal.



#17 nekitmm

nekitmm

    Vendor - StarNet Software

  • *****
  • Vendors
  • topic starter
  • Posts: 227
  • Joined: 25 Apr 2018

Posted 09 September 2023 - 06:11 PM

I've recently updated to a newer AMD GPU, and I was able to get hardware acceleration working for StarX and BlurX using the tensorflow dll's included in the latest DirectML download (from their github). This has unfortunately broken DeepSNR for me. My understanding is that it might be due to star/blurX using tensorflow1, while deepSNR/starnet2 are on tensorflow2, which is not included in DirectML.

Would you have any hints, input, or help that might let me get DeepSNR working with my AMD hardware? Ultimately I'd love to see it GPU accelerated, but failing that, is there any way to have star/blurX running accelerated while deepSNR could work on the CPU?

Currently it just fails to run at all with the following error:

I can fix it by replacing the tensorflow.dll with the one provided, but this puts everything back to CPU and is obviously not ideal.

Sorry, but I will not be able to help here, I have never used AMD GPUs and not sure what is going on. What is puzzling me - does all the software work with StarNet's tensorflow.dll on CPU? If so, everything should be equally well working on GPU...



#18 cuba

cuba

    Vostok 1

  • -----
  • Posts: 170
  • Joined: 04 Oct 2018

Posted 11 September 2023 - 04:51 AM

Looks extremely promising! Tested with grayscale script


  • nekitmm likes this

#19 fmendes

fmendes

    Surveyor 1

  • *****
  • Posts: 1,742
  • Joined: 20 Apr 2020
  • Loc: Houston, TX

Posted 18 September 2023 - 10:26 PM

Hi Nikita!

 

A quick comment on the packaging: the version for who has Starnet installed with PI 1.8.9-2 is incomplete. The file DeepSNR_weights.pb is missing. I needed to download the complete version and copy all files except tensorflow.dll.


  • nekitmm likes this

#20 nekitmm

nekitmm

    Vendor - StarNet Software

  • *****
  • Vendors
  • topic starter
  • Posts: 227
  • Joined: 25 Apr 2018

Posted 25 September 2023 - 04:39 PM

Hi Nikita!

 

A quick comment on the packaging: the version for who has Starnet installed with PI 1.8.9-2 is incomplete. The file DeepSNR_weights.pb is missing. I needed to download the complete version and copy all files except tensorflow.dll.

Thanks for pointing out! Corrected for all PI 1.8.9-2 distributions!


  • fmendes likes this

#21 Zegery

Zegery

    Lift Off

  • -----
  • Posts: 3
  • Joined: 08 Aug 2022

Posted 20 October 2023 - 03:00 PM

Noticed DeepSNR introducing something that looks like compression? This data is not compressed at all, mono RGB. I wonder why

Attached Thumbnails

  • image1.png
  • image2.png

  • cuba likes this

#22 nekitmm

nekitmm

    Vendor - StarNet Software

  • *****
  • Vendors
  • topic starter
  • Posts: 227
  • Joined: 25 Apr 2018

Posted 25 October 2023 - 10:25 AM

Noticed DeepSNR introducing something that looks like compression? This data is not compressed at all, mono RGB. I wonder why

Is this stretched using STF? If so, then it's probably an STF artifact and you need to enable 24-bit lookup tables.



#23 Zegery

Zegery

    Lift Off

  • -----
  • Posts: 3
  • Joined: 08 Aug 2022

Posted 25 October 2023 - 07:43 PM

Is this stretched using STF? If so, then it's probably an STF artifact and you need to enable 24-bit lookup tables.

Yes, I did try enabling the 24-bit lookup tables and no difference was made.



#24 cuba

cuba

    Vostok 1

  • -----
  • Posts: 170
  • Joined: 04 Oct 2018

Posted 25 October 2023 - 09:57 PM

Same here, both on color and mono (using script) images



#25 Skholman

Skholman

    Lift Off

  • -----
  • Posts: 5
  • Joined: 11 Apr 2012

Posted 26 October 2023 - 08:00 AM

Hello Nikita!

 

First, many thanks for all your efforts on AI products, which are really leap forward, especially Starnet++.

Do you have any plans to issue standalone version of DeepSNR?


  • nekitmm likes this


CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.


Recent Topics






Cloudy Nights LLC
Cloudy Nights Sponsor: Astronomics