19 replies to this topic

[nekitmm]

Hi, everyone!

 

I got a little free time around Christmas and decided to look into noise reduction.

 

There is a lot of research and open source code available, I ended up training a variation of NAFnet architecture on astro images.

 

I called the result DeepSNR and it is available here: http://deepsnrastro.com/

 

Note that currently there is only PI plugin that works on RGB images. I totally forgot about Greyscale images as always, will try to add later.

 

It is deployed in the same exact way as StarNet, so installation and usage instructions are the same. Note that there are two options to download: with and without Tensorflow libraries. If you have StarNet already you don't need to re-download and replace Tensorflow libs.

 

This is very fresh out the oven, I only did limited testing, so let me know how you find it working. Wei-Hao Wang was kind enough to test MacOS version, but the rest of the release materials are not really tested so far.

 

Also, it works on freshly integrated images with minimum postprocessing and is supposed to be applied early in the processing workflow.

 

Thanks!

Edited by nekitmm, 29 December 2022 - 08:31 PM.

[fmendes]

Great stuff! Just downloaded. Will try later today.

[nekitmm]

Update:

 

Based on the feedback I see that is works really well for images from monochrome cameras, but not suitable for anything that has been debayered prior to integration. It does not handle the low-frequency noise produced by that (the same goes for things like walking noise). This makes sense since I only trained it on images taken with monochrome cameras, so should have mentioned that from the start. Will work on the version that handles debayered images as well when I have time!

[cyber]

Works great on RGB data from my mono ASI1600MM-Cool. Many thanks for making this available, Nikita!

 

I also tried DeepSNR on DSLR data and standard integrations don't respond well indeed. The result is shown in the bottom left image below. I then followed the comments in your AstroBin post and can confirm that those issues are completely solved when CFA Drizzle integration is used (bottom right image).

 

[Der_Pit]

A standalone version, like for Starnet, would be really nice.  At least at some point...

[nekitmm]

Works great on RGB data from my mono ASI1600MM-Cool. Many thanks for making this available, Nikita!

 

I also tried DeepSNR on DSLR data and standard integrations don't respond well indeed. The result is shown in the bottom left image below. I then followed the comments in your AstroBin post and can confirm that those issues are completely solved when CFA Drizzle integration is used (bottom right image).

Victor,

 

The result looks awesome! Thanks a lot for testing and confirming! I like how barely perceivable difference in the noise translates to drastically different results!

Edited by nekitmm, 01 January 2023 - 10:13 PM.

[nekitmm]

A standalone version, like for Starnet, would be really nice.  At least at some point...

Yeah, it's definitely coming at some point!

[dmcarlsn]

Hi Nikita,

 

I'm am trying DeepSNR on my image of C/2022 E3 (ZTF) and am impressed with the results I am seeing.

 

I script all of my image processing in PixInsight to capture the full process and make it easy to do multiple runs with varying parameters.

 

Would it be possible for you to update DeepSNR to support its GUI controls (i.e. Strength and Linear data) in its process parameters so that they can be controlled from a PJSR script?  Like you did in the recent update to StarNet2 (much appreciated by the way).  This would also make DeepSNR work in PI containers as the StarNet2 update did.

 

Thanks!

[nekitmm]

Hi Nikita,

 

I'm am trying DeepSNR on my image of C/2022 E3 (ZTF) and am impressed with the results I am seeing.

 

I script all of my image processing in PixInsight to capture the full process and make it easy to do multiple runs with varying parameters.

 

Would it be possible for you to update DeepSNR to support its GUI controls (i.e. Strength and Linear data) in its process parameters so that they can be controlled from a PJSR script?  Like you did in the recent update to StarNet2 (much appreciated by the way).  This would also make DeepSNR work in PI containers as the StarNet2 update did.

 

Thanks!

Yes, you are correct, it has the same problem. Let me transfer all the recent updates from starnet, stay tuned!

 

Thank you!

Edited by nekitmm, 21 February 2023 - 10:25 PM.

[nekitmm]

Just released an update - the new version will apply auto-stretch if no STF set!

[jambi99]

Just released an update - the new version will apply auto-stretch if no STF set!

Any chances this could be used for EAA? Like a plugin in sharpcap?

Edited by jambi99, 06 March 2023 - 09:52 PM.

[cyber]

DeepSNR is absolutely amazing for RGB images. However, it is currently not trained to run well on a monochrome file. 

 

I've figured out a way to improve the performance of DeepSNR on mono images. When you simply convert a mono image to RGB, the algorithm will produce lot of artefacts (fake stars). I figured this was due to autocorrelation between the 3 channels of an RGB converted image. So, my strategy was to create 2 linear copies of the mono file, inject random noise in them to simulate a different filter, and then all 3 channels are stretched with the same HT before RGB combination. 

 

The untreated mono data was used in the Red channel in the examples below. Noise was added to the G and B channel seprately. Adding more noise to the synthetic channels improves denoise performance in the red channel. However, excessive noise seems to introduce some smoothing and detail loss. More experimentation is needed, but this idea might be helpful for other users that want to run DeepSNR on a single monochrome dataset. 

 

Click images for full size (600 pix wide)

Each step doubles the amount of noise in the linear phase.

 

 

Edited by cyber, 12 June 2023 - 01:45 AM.

[nekitmm]

DeepSNR is absolutely amazing for RGB images. However, it is currently not trained to run well on a monochrome file. 

 

I've figured out a way to improve the performance of DeepSNR on mono images. When you simply convert a mono image to RGB, the algorithm will produce lot of artefacts (fake stars). I figured this was due to autocorrelation between the 3 channels of an RGB converted image. So, my strategy was to create 2 linear copies of the mono file, inject random noise in them to simulate a different filter, and then all 3 channels are stretched with the same HT before RGB combination. 

 

The untreated mono data was used in the Red channel in the examples below. Noise was added to the G and B channel seprately. Adding more noise to the synthetic channels improves denoise performance in the red channel. However, excessive noise seems to introduce some smoothing and detail loss. More experimentation is needed, but this idea might be helpful for other users that want to run DeepSNR on a single monochrome dataset. 

 

Click images for full size (600 pix wide)

Each step doubles the amount of noise in the linear phase.

 

uniform tests1_2.jpg

 

uniform tests2_2.jpg

Wow, really nice thinking, thanks so much for sharing!

 

One of these days I will find time to train it on Greyscale images, but honestly this hack makes perfect sense and works so well that it could be better just to implement it in the plugin... Will do more testing once I have time!

[cyber]

I wrote a script to automate the process. 

Here's a link to the PixInsight forum post where you can download the script: https://pixinsight.c...edeepsnr.21031/

Edited by cyber, 21 June 2023 - 04:31 PM.

[nekitmm]

I wrote a script to automate the process. 

Here's a link to the PixInsight forum post where you can download the script: https://pixinsight.c...edeepsnr.21031/

The results look really good, thanks a lot for this awesome contribution!

[astronoom]

I've recently updated to a newer AMD GPU, and I was able to get hardware acceleration working for StarX and BlurX using the tensorflow dll's included in the latest DirectML download (from their github). This has unfortunately broken DeepSNR for me. My understanding is that it might be due to star/blurX using tensorflow1, while deepSNR/starnet2 are on tensorflow2, which is not included in DirectML.

Would you have any hints, input, or help that might let me get DeepSNR working with my AMD hardware? Ultimately I'd love to see it GPU accelerated, but failing that, is there any way to have star/blurX running accelerated while deepSNR could work on the CPU?

Currently it just fails to run at all with the following error:

 

*** Error: Could not find checkpoint file in 'C:\Program Files\PixInsight\bin\'!

I can fix it by replacing the tensorflow.dll with the one provided, but this puts everything back to CPU and is obviously not ideal.

[nekitmm]

I've recently updated to a newer AMD GPU, and I was able to get hardware acceleration working for StarX and BlurX using the tensorflow dll's included in the latest DirectML download (from their github). This has unfortunately broken DeepSNR for me. My understanding is that it might be due to star/blurX using tensorflow1, while deepSNR/starnet2 are on tensorflow2, which is not included in DirectML.

Would you have any hints, input, or help that might let me get DeepSNR working with my AMD hardware? Ultimately I'd love to see it GPU accelerated, but failing that, is there any way to have star/blurX running accelerated while deepSNR could work on the CPU?

Currently it just fails to run at all with the following error:

I can fix it by replacing the tensorflow.dll with the one provided, but this puts everything back to CPU and is obviously not ideal.

Sorry, but I will not be able to help here, I have never used AMD GPUs and not sure what is going on. What is puzzling me - does all the software work with StarNet's tensorflow.dll on CPU? If so, everything should be equally well working on GPU...

[cuba]

Looks extremely promising! Tested with grayscale script

[fmendes]

Hi Nikita!

 

A quick comment on the packaging: the version for who has Starnet installed with PI 1.8.9-2 is incomplete. The file DeepSNR_weights.pb is missing. I needed to download the complete version and copy all files except tensorflow.dll.

[nekitmm]

Hi Nikita!

 

A quick comment on the packaging: the version for who has Starnet installed with PI 1.8.9-2 is incomplete. The file DeepSNR_weights.pb is missing. I needed to download the complete version and copy all files except tensorflow.dll.

Thanks for pointing out! Corrected for all PI 1.8.9-2 distributions!