Jump to content

  •  

CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.

Photo

Image Processing - Lots of noise in the nebula

  • Please log in to reply
14 replies to this topic

#1 dcm_guitar

dcm_guitar

    Explorer 1

  • -----
  • topic starter
  • Posts: 60
  • Joined: 08 Sep 2020

Posted 21 January 2021 - 07:57 PM

I've been working on astrophotography for about a month.  I started from basically ground zero, so the learning curve has been STEEP!!

 

I'm in the "I don't know what I don't know" phase of learning, so unfortunately I don't know exactly what to ask to help with my current issue.

 

Current Issue:

Nebula photos have LOTS of noise in them.

 

Here's my current acquisition project:  The cone nebula:

 

 

https://www.astrobin.com/8d601o/

 

The above image is after pre-processing in Pixinsight.  Here's what I've done so far:

 

I captured 60 images at 119s exposure with the camera gain at 96 and offset at 20 (ZWO ASI071MC Pro).  The camera sensor was cooled to -10C.  The gain and offset values came from a Sharpcap image sensor analysis.  The exposure time came from the Sharpcap Smart Histogram recommendation after measuring the night sky and using the image sensor analysis results.  The OTA is a WO GT81IV using the WO Flat6AIII 0.8 flattener/reducer.

  1. Subs brought into Pixinsight
  2. Subs were calibrated using a superbias and master flat
  3. Subs were cosmetic corrected with a master dark
  4. Subs were debayered
  5. Subs went through the subframe selection process and weighted based on a score taking FWHM, eccentricity and SNR ratio into consideration (the Light Vortex tutorial weighting scheme)
  6. Subs were registered and then stacked based on their weights, drizzle data was generated
  7. Local normalization was generated
  8. Drizzle files with local normalization were integrated
  9. Integrated image went through background neutralization
  10. Image was then photometric calibrated
  11. Image then went through DBE (dynamic background extraction)

The image in the link above was also quickly cropped and then stretched just so you could see it.

 

I like the colors.  BUT, the nebulosity has a LOT of noise.  It's very, very grainy.

 

Is this:

 

  • The expected amount of noise for 2 hours of exposure?
  • Is it reasonable or unreasonable to expect to get smooth nebulosity from this image using Pixinsight noise reduction processes (TGVDenoise, MLT Transform)?
  • Should I even bother trying to get this image to look good (smooth nebulosity)?  Am I better off not trying to process this data yet and add additional subs before trying to process?
  • I know there's no hard and fast rule for how much exposure is needed.  But given that the above image is two hours of exposure, how much more exposure should I expect to need before I can reasonably expect to get smooth nebulosity in my image (after Pixinsight processing)?

 

I can't get this image to look good, and I'm not sure if it's my Pixinsight limitations, my data limitations or my capture method is flawed.  Do I need to invest more time in more exposure, or is this enough data to get a decent result if I get better at applying noise reduction techniques in Pixinsight?

 

Eventually, I want to combine this with RGB data.  I also have 75 subs of RGB at 30 seconds exposure each.  That data is also noisy.

 

Thoughts?  Next Steps?

 

I'd REALLY appreciate someone who's been through this talking me through the recommended next steps.

 

 



#2 Midnight Dan

Midnight Dan

    Voyager 1

  • *****
  • Posts: 14,189
  • Joined: 23 Jan 2008
  • Loc: Hilton, NY, Yellow Zone (Bortle 4.5)

Posted 21 January 2021 - 08:13 PM

I'd say that looks pretty normal.  2 hours is not a lot of integration time.  For targets like that, I've been trying for at least 10 hours to get much better signal to noise.

 

Another factor is how much light pollution you have, or what the phase of the moon was at the time.  Shot noise increases as the square root of the signal.  LP can add a lot of signal, and with it comes that shot noise.  That's why imaging from darker sites will show so much less noise.

 

Noise reduction in PI is an extremely complex endeavor and it takes a lot of time and practice to get it right.  There are MANY ways to go about it and different targets require different approaches.  One approach I've found very effective is shown on this page.

 

https://jonrista.com...duction-part-2/

 

-Dan


Edited by Midnight Dan, 21 January 2021 - 08:17 PM.

  • bobzeq25 and dcm_guitar like this

#3 pyrasanth

pyrasanth

    Soyuz

  • *****
  • Posts: 3,627
  • Joined: 08 Jan 2016

Posted 21 January 2021 - 08:16 PM

My experience with OSC is that you need a lot more exposure. 2 hours is not nearly enough- realistically your probably looking at closer to 8 or maybe 10 hours to get the noise down on that target.

 

If you were using mono you could use demure denoise which is superb however it does not work on OSC data.

 

The other possibility is to take the noise reduction outside of PI using a utility in Photoshop such as Topaz AI, however your best win is more exposure time to increase the SNR.


  • dcm_guitar likes this

#4 copper280z

copper280z

    Vostok 1

  • -----
  • Posts: 159
  • Joined: 01 Mar 2019
  • Loc: Rochester, NY

Posted 21 January 2021 - 08:20 PM

Light pollution is a huge contributor, what type of area are you imaging from?

Additionally, you can increase SNR at the expense of angular resolution by binning/downsampling your image.



#5 zxx

zxx

    Surveyor 1

  • -----
  • Posts: 1,757
  • Joined: 27 Jan 2010

Posted 21 January 2021 - 08:27 PM

That's a nice result for only one month in the hobby waytogo.gif


  • Dynan likes this

#6 SteveL42

SteveL42

    Explorer 1

  • *****
  • Posts: 80
  • Joined: 18 Sep 2020
  • Loc: Colorado

Posted 21 January 2021 - 08:40 PM

The RAW filter in Photoshop can help with noise, as can Topaz Denoise AI (which can run standalone).  I suggest downloading the Topaz trial and giving it a whirl.  The low light filter works quite well. I ran your pic thru it, but not comfortable uploading your work in my albums to display here...

 

Steve


Edited by SteveL42, 21 January 2021 - 08:49 PM.


#7 jonnybravo0311

jonnybravo0311

    Viking 1

  • *****
  • Posts: 978
  • Joined: 05 Nov 2020
  • Loc: NJ, US

Posted 21 January 2021 - 09:12 PM

If you were using mono you could use demure denoise which is superb however it does not work on OSC data.

Couldn't you split the OSC into separate R, G and B channels, then run the mure denoise on each and combine them afterwards?


  • MeenSeen likes this

#8 ntph

ntph

    Vostok 1

  • -----
  • Posts: 147
  • Joined: 20 Oct 2012
  • Loc: Sudbury, ON

Posted 21 January 2021 - 09:29 PM

You might have tried this already, but it would be interesting to see what you get when you: 

load everything up in the WBPP script, then take what you get, crop it as needed, do your A or DBE, then PCC then EZ Denoise and compare. EZDenoise script is basically Jon Rista's approach to linear noise reduction with very few knobs to twirl and buttons to push. Works a charm on most images. It is certainly a good thing to know how to do preprocessing and why, but for most datasets, it seems like WBPP does a pretty good job, and most datasets don't need the extra steps of LocalNormalization and DrizzleIntegration, according to all the experts around here and in print (there, that should light a fire somewhere....)shocked.gif

 

Nice image any way you look at it and it will only get betterwaytogo.gif



#9 randcpoll

randcpoll

    Ranger 4

  • *****
  • Posts: 386
  • Joined: 20 Jul 2013
  • Loc: New York

Posted 21 January 2021 - 09:32 PM

That's a good result from only 2 hours integration. That is a very difficult target. Dithering will help some, as it will spread the color noise around over several pixels and smooth out your future results. And as others have commented Topaz Noise AI will do a pretty good job smoothing it out after the fact.  



#10 dcm_guitar

dcm_guitar

    Explorer 1

  • -----
  • topic starter
  • Posts: 60
  • Joined: 08 Sep 2020

Posted 22 January 2021 - 11:45 AM

First of all, THANKS!!!!!

 

I'd say that looks pretty normal.  2 hours is not a lot of integration time.  For targets like that, I've been trying for at least 10 hours to get much better signal to noise.

 

Another factor is how much light pollution you have, or what the phase of the moon was at the time.  Shot noise increases as the square root of the signal.  LP can add a lot of signal, and with it comes that shot noise.  That's why imaging from darker sites will show so much less noise.

 

Noise reduction in PI is an extremely complex endeavor and it takes a lot of time and practice to get it right.  There are MANY ways to go about it and different targets require different approaches.  One approach I've found very effective is shown on this page.

 

https://jonrista.com...duction-part-2/

 

-Dan

 

Dan, Thanks!!!.  The comment about "shot noise increasing as the square root of the signal" prompted more reading which completely changed my thought process on setting exposure time.  The good news is I already have thoughts on changing my approach to see how it changes my data.  The bad news is that it's going to be cloudy here for the next week.  Thanks for the noise tutorial link!!

 

 

My experience with OSC is that you need a lot more exposure. 2 hours is not nearly enough- realistically your probably looking at closer to 8 or maybe 10 hours to get the noise down on that target.

 

If you were using mono you could use demure denoise which is superb however it does not work on OSC data.

 

The other possibility is to take the noise reduction outside of PI using a utility in Photoshop such as Topaz AI, however your best win is more exposure time to increase the SNR.

 

Okay, this really helps in suggesting a total exposure target is for this nebula.

 

I JUST got all this gear, AND I'm already looking at monochrome cameras and filter wheels.  This is dangerous!!!

 

Somebody else asked if demure denoise would work on the grayscale layers of a OSC result.  Could I use PI to separate the image into R, G & B channels and use demure denoise?

 

 

Light pollution is a huge contributor, what type of area are you imaging from?

Additionally, you can increase SNR at the expense of angular resolution by binning/downsampling your image.

 

According to the "Clear Outside" app I'm in a bortle class 6 location.  I'm also imaging ridiculously close to two street lights which I KNOW is causing issues.  I am imaging using a duoband filter, but I know a filter is not going to cure my light pollution issues.  Right now, I'm still trying to wrap my head around image gathering, pre-processing and then processing.  I've imaged at a local park away from the street lights a few times, and that felt like a major excursion away from home!

 

You might have tried this already, but it would be interesting to see what you get when you: 

load everything up in the WBPP script, then take what you get, crop it as needed, do your A or DBE, then PCC then EZ Denoise and compare. EZDenoise script is basically Jon Rista's approach to linear noise reduction with very few knobs to twirl and buttons to push. Works a charm on most images. It is certainly a good thing to know how to do preprocessing and why, but for most datasets, it seems like WBPP does a pretty good job, and most datasets don't need the extra steps of LocalNormalization and DrizzleIntegration, according to all the experts around here and in print (there, that should light a fire somewhere....)shocked.gif

 

Nice image any way you look at it and it will only get betterwaytogo.gif

I've actually enjoyed running the pre-process part manually (using the LIght Vortex tutorial process).  I've compared my images without and without drizzle integration and I can certainly see a substantial difference with the drizzle integration.  I have not done a personal comparison on whether I can see a big difference using local normalization.  I don't want to get involved in any firestorms!! ;-).

 

I am learning that there are LOTS of different ways to accomplish the same result.  For an experienced person this is fantastic.  As a beginner, the fact that there isn't necessarily a "standard" makes each step in the process almost overwhelming.  I don't know enough to make an educated choice with regards to process.  I'm still in the "follow the recipe exactly" phase.  Just getting an identifiable result is a thrill.  But there are so many levers to pull and parameters to tweak it makes my head spin.  

 

The idea of an EZDenoise script is VERY appealing.  I'll do a search and find where to download and give it a whirl.

 

That's a good result from only 2 hours integration. That is a very difficult target. Dithering will help some, as it will spread the color noise around over several pixels and smooth out your future results. And as others have commented Topaz Noise AI will do a pretty good job smoothing it out after the fact.  

 

I actually dither after every shot.  Early on I was getting "raining noise" along with dark blotches in my images.  I thought something was wrong with my brand new camera!!  I started dithering every third photo, and it helped.  Dithering after every image has removed the raining noise and the dark blotches.

 

I've not heard of Topaz Noise AI, so more reading and research is warranted.  Thanks!!!!

 

Based on information from this thread (which prompted more reading last night) I think my issue is two-fold.  First and foremost I simply don't have enough exposure time.  Second, my sub exposures are probably too short.  Based on the math, I can substantially improve my signal to noise ratio in each sub by increasing my exposure time.

 

Given that it's going to be cloudy here for the next week, I'll test out the denoise options provided here. I'll do this as an educational exercise.  But once the skies clear, I'll get more subs with longer exposure times.



#11 dcm_guitar

dcm_guitar

    Explorer 1

  • -----
  • topic starter
  • Posts: 60
  • Joined: 08 Sep 2020

Posted 22 January 2021 - 11:50 AM

Oh and another question.....

 

My current data is based on 119 second exposures.

 

When I get new data I plan on taking 500 second exposures.

 

Is it "okay" to mix these two different exposure values in processing?  Do I calibrate them separately using different master darks (a separate master dark for each exposure time), and then mix them all together in the subframe selector process?  Is that the right thought process?



#12 jonnybravo0311

jonnybravo0311

    Viking 1

  • *****
  • Posts: 978
  • Joined: 05 Nov 2020
  • Loc: NJ, US

Posted 22 January 2021 - 12:56 PM

Oh and another question.....

 

My current data is based on 119 second exposures.

 

When I get new data I plan on taking 500 second exposures.

 

Is it "okay" to mix these two different exposure values in processing?  Do I calibrate them separately using different master darks (a separate master dark for each exposure time), and then mix them all together in the subframe selector process?  Is that the right thought process?

Sure, you can mix exposure lengths. Yes, you calibrate each separately. So, your 119s exposures would be calibrated against a 119s master dark, flats from that session, and biases. Your 500s exposures would be calibrated against 500s darks, flats from that session and biases. Then once all the images from both sessions are calibrated, cosmetically corrected, debayered, etc... run the entire batch through subframe selector.



#13 dcm_guitar

dcm_guitar

    Explorer 1

  • -----
  • topic starter
  • Posts: 60
  • Joined: 08 Sep 2020

Posted 22 January 2021 - 01:46 PM

I downloaded the trial of Topaz AI denoise, and.......... WOW!!



#14 dcm_guitar

dcm_guitar

    Explorer 1

  • -----
  • topic starter
  • Posts: 60
  • Joined: 08 Sep 2020

Posted 22 January 2021 - 10:43 PM

Okay, probably my last update in this thread.......

 

I combined my RGB data with my narrowband (narrowband comes from a OSC using a duo band filter).  I processed the image in Pixinsight and then used a free trial of Topaz AI to denoise it.  Thanks to all on this thread who helped!!!!!

 

I still have lots of reading to do, but I'm pretty happy with this photo given where I am on this journey.

 

https://astrob.in/fjxvpp/0/

 

Thanks!!!


  • ntph likes this

#15 Midnight Dan

Midnight Dan

    Voyager 1

  • *****
  • Posts: 14,189
  • Joined: 23 Jan 2008
  • Loc: Hilton, NY, Yellow Zone (Bortle 4.5)

Posted 23 January 2021 - 09:51 AM

Okay, probably my last update in this thread.......

 

I combined my RGB data with my narrowband (narrowband comes from a OSC using a duo band filter).  I processed the image in Pixinsight and then used a free trial of Topaz AI to denoise it.  Thanks to all on this thread who helped!!!!!

 

I still have lots of reading to do, but I'm pretty happy with this photo given where I am on this journey.

 

https://astrob.in/fjxvpp/0/

 

Thanks!!!

Big improvement!  Nice Job! waytogo.gif

 

-Dan




CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.


Recent Topics






Cloudy Nights LLC
Cloudy Nights Sponsor: Astronomics