Return to the Cloudy Nights Telescope Reviews home page


Astrophotography and Sketching >> CCD Imaging & Processing

Pages: 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | (show all)
Alph
Carpal Tunnel


Reged: 11/23/06

Loc: Melmac
Re: 5 or 30 minute exposures for narrowband? new [Re: vpcirc]
      #5650805 - 01/29/13 05:38 PM

Quote:

For a great understanding of the benefit of longer images and why they are much better than multiple short images, John has an easy to understand writeup here

http://www.hiddenloft.com/notes/SNR.txt




Mike,
I would recommend The Signal to Noise Connection part I & II articles by Mike Newberry published in CCD Astronomy magazine (long gone) in the summer and the fall of 1994. IMO a must read for everyone.

It is true that pitch dark sky and long exposures might be the only option to capture the dimmest parts of an already very dim object. However there is no science behind the number 16 and stacking a large number of shorter exposures can produce equally good results in many cases.

Here is a quote from the cited article:

Quote:

If our exposures are background limited because the readout noise is small in comparison to the noise from skyglow, it is usually preferable to stack exposures rather make a single long one.




Mike! Thanks for the pass to the last year's RTMC


Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: freestar8n]
      #5650947 - 01/29/13 06:57 PM

I was thinking more about this thread on the way home from work. I realize we really don't define "signal" and "noise" and these aren't actually intuitive concepts. From a statistical perspective, signal is really the theoretical average of some parameter, from some population, given an infinite number of unbiased samples. In this case, i think the population is the photons hitting the photon collector, over the integration time. So, this signal is the theoretical average "rate" of photon arrival, as we have a count/time. The bias is then anything that deviates our ability to detect this rate, in a fixed way, so, this is really the "pattern noise".. For example, there could be gaps between the pixels, so, some photons will miss the collector in a fixed way and there are other fixed errors counting the photons and translating to an image to display etc.... Once we take out the bias, the remaining deviations from the theoretical average rate of photon arrival, are random errors In our detection. If something is random and we take one sample, then that random error will have a large contribution, but if we take another and it isn't there, and average ( or sum ) the two, then it will have a smaller contribution to the total detected population and so on... But the signal is conceptual and is the theoretical average rate of photon arrival and the noise is just everything else that effects a given sampling of this population. So, the signal, being conceptual is always there, it doesn't really "increase"', it's just through various methods, we are able to decrease the noise to "see" the average rate of photon arrival and translate it into a perceivable image.

Post Extras: Print Post   Remind Me!   Notify Moderator  
vpcirc
Post Laureate


Reged: 12/09/09

Loc: Merced CA
Re: 5 or 30 minute exposures for narrowband? new [Re: Inverted]
      #5651145 - 01/29/13 08:51 PM

Be careful about using the word sunn and average. If, for example you chose sum in CCDstack verse Average, your result would be totally different. Adam Block gives excellent examples of the differences in alogarythims in his ccdstack tutorials.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: vpcirc]
      #5651332 - 01/29/13 10:54 PM

Quote:

Be careful about using the word sunn and average. If, for example you chose sum in CCDstack verse Average, your result would be totally different. Adam Block gives excellent examples of the differences in alogarythims in his ccdstack tutorials.




Mike, mathematically, this isn't that complicated once you take out all the intimidating terminology. When we're talking about summing or averaging here, we're really just talking about a ratio of fractions I.e.

Edit, I realize I had made this confusing and it may not have actually been obvious what I was getting at, as written. I'm trying not to write out too much math. So, I rewrote this slightly to emphasize better in English. The point, is summing and averaging are just linear functions of the other. If you have a perfect computer system, you can always go back and forth, one is a scaled version prof the other
(sum(xi)/n) = sum(x1)/n + sum(x2)/n .... sum(xn)/n

Where n = the number of exposures. Whether you sum or average, one is just an equivelent linear function of the other and in each case, x is part signal and part noise. The signal part stays constant though and the random part of the noise changes for each exposure. So, when we sum or average them, we have a scaled linear function of the other and if there isn't truncation of the data, through computational limitations, we can always go back and forth, if we know n (the number of exposure)

You can also re-write s/n as (signal mean) / (standard deviation of noise) if you google summation of variance and work your way through it though, you'll find it works out the same, in that one (I.e, summing or averaging) is a scaled function of the other with respect to the ratio.

Again, As i alluded to in the previous post there can be differences because of scaling, rounding, truncation of big numbers, analog to digital scaling/conversion, but these are limitations of the specific hardware and software design used, not any fundamental properties.

Edited by Inverted (01/30/13 05:55 AM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
David PavlichAdministrator
Transmographied
*****

Reged: 05/18/05

Loc: Mandeville, LA USA
Re: 5 or 30 minute exposures for narrowband? new [Re: Inverted]
      #5651344 - 01/29/13 11:07 PM

Just to jump into the middle of this, around the end of October or early November, the Advanced Imaging Conference takes place around San Jose, CA. If there's one astroconference you want to attend if you enjoy imaging, this is the one! All the best imagers are there and many of them have workshops. You'll learn a ton of stuff and meet some really great people in the process. And I won't mention the Vendor's Hall. Leave your credit card at home.

David


Post Extras: Print Post   Remind Me!   Notify Moderator  
mikeschuster
super member


Reged: 08/25/11

Loc: SF Bay area
Re: 5 or 30 minute exposures for narrowband? new [Re: David Pavlich]
      #5651467 - 01/30/13 12:36 AM

Note also dark current noise. Going longer to overcome read noise means there is even more dark current noise to overcome. In my case at f/5, 3nm Ha, dark skies, 40 minute exposures. Read + dark noise is about 14e-. Typically at 40 minutes my sky background noise is about 20e-. This is *far* from being sky limited. But I can't expose any longer most of the time due to temperature dependent focus changes.
Mike


Post Extras: Print Post   Remind Me!   Notify Moderator  
freestar8n
Post Laureate
*****

Reged: 10/12/07

Re: 5 or 30 minute exposures for narrowband? new [Re: mikeschuster]
      #5651548 - 01/30/13 02:29 AM

Dark current is no different from sky glow - at least in terms of the noise model used in these discussions. You can lump all the noises that increase with exposure time - and are independent from the number of sub-exposures used - on one side - and on the other side put read noise - which is independent of total exposure time and *only* depends on the number of sub-exposures (for a given camera). Read noise is what drives the desired number of exposures down - but the other terms don't care.

There are many sources of noise and their impact is reduced by dithering, good flats, master frames, etc. But read noise is special because it doesn't accumulate due to total exposure time, but number of frames. Without read noise, the length of sub-exposure wouldn't matter at all. At least according to these noise models.

For the other noise sources - just increase the total integration time - by longer subs and/or more frames - and the SNR will improve. But other factors not included in the noise model may make the improvement less than expected.

Frank


Post Extras: Print Post   Remind Me!   Notify Moderator  
freestar8n
Post Laureate
*****

Reged: 10/12/07

Re: 5 or 30 minute exposures for narrowband? new [Re: Inverted]
      #5651557 - 01/30/13 02:52 AM

Quote:

The bias is then anything that deviates our ability to detect this rate, in a fixed way, so, this is really the "pattern noise"..




I think this is a more subtle issue and is something that is consistently handled very badly in typical amateur level write ups of ccd imaging. The only purpose of a master bias and master dark is to remove the fixed pattern noise from those two terms. A dark frame will look noisy and ugly - but if you look at many such frames, the noise looks similar. The goal is to take a really clear "picture" of that pattern noise and subtract it - and it works very well to reduce the overall noise in a sub-exposure.

Bias frames also have pattern noise, but a big reason for bias frames is to set the zero of the photon signal so the calibration process is linear. Even if the bias frames were perfectly flat at a level of 1000, with no noise, you would need to subtract them (if you are using flats and you don't have flat darks).

But a single dark frame will contain the pattern noise in the dark current - plus Poisson noise also - and that Poisson noise cannot be reduced by subtracting the master dark. But there is Poisson noise in the sky glow also, and in the nebulosity signal you are trying to image. And if you expose long enough, the SNR of the nebulosity against the other noise terms will increase and you will see improvement in the image.

Frank


Post Extras: Print Post   Remind Me!   Notify Moderator  
vpcirc
Post Laureate


Reged: 12/09/09

Loc: Merced CA
Re: 5 or 30 minute exposures for narrowband? new [Re: freestar8n]
      #5651579 - 01/30/13 03:31 AM

I wish we could get Adam Block to dispel all the science fact from science fiction on here. He is a true wiz at this (just watch one of his tutorials). I think the average beginner to intermediate user is now totally confused with all the "theory" being spread here. David has the perfect solution attend AIC and listen to the John Smith and go tell him he doesn't know what he's talking about! Hey let's call the Myth Busters!

Post Extras: Print Post   Remind Me!   Notify Moderator  
freestar8n
Post Laureate
*****

Reged: 10/12/07

Re: 5 or 30 minute exposures for narrowband? new [Re: vpcirc]
      #5651588 - 01/30/13 03:46 AM

"I wish we could get Adam Block to dispel all the science fact from science fiction on here. He is a true wiz at this (just watch one of his tutorials). I think the average beginner to intermediate user is now totally confused with all the "theory" being spread here. David has the perfect solution attend AIC and listen to the John Smith and go tell him he doesn't know what he's talking about! Hey let's call the Myth Busters!"

If you feel you understand this material and see flaws in what I'm saying - feel free to point out specific items of disagreement - preferably with explicit references. But once again - you are just casting aspersions without any actual content and disrupting what appears to be a fruitful discussion among people who are interested in learning and discussing the subject.

Moderators?

Frank


Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: freestar8n]
      #5651645 - 01/30/13 06:18 AM

"I think this is a more subtle issue and is something that is consistently handled very badly in typical amateur level write ups of ccd imaging. The only purpose of a master bias and master dark is to remove the fixed pattern noise from those two terms."

Yes, and i'm using bias a little different from what we refer to as "bias" in the hobby. Technically bias frames don't necessarily remove all fixed pattern error/noise (bias). It is a way to try to remove bias, but in practice, not all noise, fixed, or random can be dealt with. Bias frames are a way to deal with bias, but there may be other bias as well for example.


Post Extras: Print Post   Remind Me!   Notify Moderator  
vpcirc
Post Laureate


Reged: 12/09/09

Loc: Merced CA
Re: 5 or 30 minute exposures for narrowband? new [Re: freestar8n]
      #5651975 - 01/30/13 10:23 AM

Quote:

"I wish we could get Adam Block to dispel all the science fact from science fiction on here. He is a true wiz at this (just watch one of his tutorials). I think the average beginner to intermediate user is now totally confused with all the "theory" being spread here. David has the perfect solution attend AIC and listen to the John Smith and go tell him he doesn't know what he's talking about! Hey let's call the Myth Busters!"

If you feel you understand this material and see flaws in what I'm saying - feel free to point out specific items of disagreement - preferably with explicit references. But once again - you are just casting aspersions without any actual content and disrupting what appears to be a fruitful discussion among people who are interested in learning and discussing the subject.

Moderators?

Frank




Don't be so sensitive Frank, I wasn't talking about you, I was just pointing out there were several differing opinions on here as to the correct methods. Heck, I don't even know if I'm right, I'm just going by what pro's who know a lot more than me tell me, maybe some of them are wrong? I could see how you thought that since the reply was to your post, but that happens when you reply, the last poster is auto selected.


Post Extras: Print Post   Remind Me!   Notify Moderator  
JWalk
Pooh-Bah


Reged: 06/06/10

Loc: San Antonio, TX
Re: 5 or 30 minute exposures for narrowband? new [Re: vpcirc]
      #5652049 - 01/30/13 11:02 AM

I have been shooting from very dark skies for the last 6 months and I can say I've shot at all different exposure durations and I can say you should go as deep as you can without blowing out your background. This is target specific of course. I've been going 20min lately with a OSC at F3.59 and it does great.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Fogboundturtle
professor emeritus


Reged: 05/20/09

Loc: Burnaby, BC
Re: 5 or 30 minute exposures for narrowband? new [Re: JWalk]
      #5652065 - 01/30/13 11:09 AM

I am confused now

Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: Fogboundturtle]
      #5652145 - 01/30/13 11:50 AM

I'm confused as well. I've been reading through the hidden loft articles and they seem pretty darn consistent with what has been said here. A fundamental concept of modern statistics is that lots of small samples will tend to produce better estimates than fewer large samples, unless for example, there is bias introduced by the sampling process itself. As we see, there does appear to be bias introduced with regards to image "sampling" (really what we are doing). This article seems to summarize that for example: http://www.hiddenloft.com/notes/SubExposures.pdf

As I said though, I am familiar with stats, but not really so much the properties of ccds and related noise models etc... I'd like to understand those parts better, but at the moment, still don't understand why you think the experts disagree with what's been said here?


Post Extras: Print Post   Remind Me!   Notify Moderator  
freestar8n
Post Laureate
*****

Reged: 10/12/07

Re: 5 or 30 minute exposures for narrowband? new [Re: Inverted]
      #5652202 - 01/30/13 12:18 PM

The noise model discussed here and on web pages is pretty simple, and I'm not aware of anything I have said in this thread that conflicts with such web pages. The main source of confusion, which some people have and others don't, is to think that a longer optimal sub-exposure at a dark site means that your subs will *have more visible noise* at a dark site. To many people, this is absurd - and it is. But for others, it gives them the impression their subs are noisier due to the darkness of the sky allowing read noise "to be seen." This is a really bad thing to be confused about - let alone to spread around. It is true that a dark sky lets the read noise "become visible" - but only because the sky noise - and overall noise - has been reduced - leaving read noise as something worth addressing - with longer subs.

The key is to emphasize that dark sites *allow* you to go longer and get an even greater benefit not possible at a bright site. They don't *require* you to go longer.

Frank


Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: freestar8n]
      #5652265 - 01/30/13 12:53 PM

Thanks Frank, much of the discussion makes more sense now. I agree many of the published noise models seem pretty simple, I'm not sure what assumptions "hobby-level" models are making to simply things though. Also, another thing I don't quite get is, is read noise really random? or can it have an underlying pattern? Random noise can mask out our "perception" of pattern noise for example. So, if you do remove more random noise, I could see people "perceiving" pattern noise more, even though it is actually a cleaner image. Of course, you can always run a white noise algorithm and add some back if it bothers you LOL. Also then, what other sources of pattern noise could remain? So anyways though, I would agree it makes no sense that you "need" to go longer if you have less sky noise.

Post Extras: Print Post   Remind Me!   Notify Moderator  
vpcirc
Post Laureate


Reged: 12/09/09

Loc: Merced CA
Re: 5 or 30 minute exposures for narrowband? new [Re: freestar8n]
      #5652267 - 01/30/13 12:54 PM

On that Frank I would disagree with your statement. Stan Moore and Bob Smith, Don Goldman and Kevin Nelson all told me the same thing. I had to go longer to over come the read noise in my background because I was at a dark site. JWalk worked with some of my data as did Warren Keller and they can both tell you it was a significant problem especially with narrowband. I was over smoothing my images with noise reduction. I went to the 4 guys I thought would know ccd's better than I to look for a solution. They all told me the same thing, longer subs to overcome that noise. When I asked why I hadn't seen it before, that's when they told me about signal and sky glow. I had enough frames, but I needed longer exposures to improve my SNR. Maybe it's because the QE is so low on my chip, I don't know, but I do know their advice in my case helped immensely.

Post Extras: Print Post   Remind Me!   Notify Moderator  
freestar8n
Post Laureate
*****

Reged: 10/12/07

Re: 5 or 30 minute exposures for narrowband? [Re: Inverted]
      #5652278 - 01/30/13 01:01 PM

Read noise usually shows a pattern across the ccd - and a master bias captures that pattern so it can be subtracted. But there will remain a random, Poisson component that cannot be subtracted. The pattern noise is the "fixed pattern noise" in the bias - and it can be subtracted away. The Poisson noise is the "read noise" - and it cannot. If you have a good master bias, the pattern noise is removed well, but the read noise remains. And you get one hit of read noise every time you expose an image - regardless of the exposure time.

So - if the read noise is visible in the sub-exposures, as happens at a dark site, you want to go longer so that sky glow, which is a fundamental limit at your sites, overcomes it.

If the site is so dark that you can't really do that - no problem - it still means your image will have low noise and will turn out well.

Frank


Post Extras: Print Post   Remind Me!   Notify Moderator  
freestar8n
Post Laureate
*****

Reged: 10/12/07

Re: 5 or 30 minute exposures for narrowband? [Re: vpcirc]
      #5652286 - 01/30/13 01:06 PM

If you think that the darkness of your sky is increasing the noise in your images - I recommend turning on floodlights around your observatory. That will increase the sky glow so that read noise is no longer a factor - and allow you to use optimal sub exposures that are short.

Frank


Post Extras: Print Post   Remind Me!   Notify Moderator  
Pages: 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | (show all)


Extra information
19 registered and 20 anonymous users are browsing this forum.

Moderator:  Oldfield, Scott in NC, bilgebay, WOBentley 

Print Thread

Forum Permissions
      You cannot start new topics
      You cannot reply to topics
      HTML is disabled
      UBBCode is enabled


Thread views: 7388

Jump to

CN Forums Home




Cloudy Nights Sponsor: Astronomics