Return to the Cloudy Nights Telescope Reviews home page


Astrophotography and Sketching >> CCD Imaging & Processing

Pages: 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | (show all)
Peter in Reno
Postmaster
*****

Reged: 07/15/08

Loc: Reno, NV
Re: 5 or 30 minute exposures for narrowband? new [Re: shams42]
      #5654104 - 01/31/13 11:09 AM

I also use Bad Pixel Mapping with my Atik 460EX low noise camera. I get better results than dark subtraction.

Peter


Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: freestar8n]
      #5654114 - 01/31/13 11:17 AM

Quote:

Not really. The message of the web sites...




I was referring to mathematical agreement to the contextual matter being discusses at that point. The formulas don't appear to disagree with that. How people interpret that and perceive it is another matter, based other various practical considerations and the expected performance and tolerance equipment used.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: shams42]
      #5654155 - 01/31/13 11:35 AM

Quote:


I just think that defining our terms such that subtracting noise can add it makes the matter more confusing than it needs to be.




I agree, that's we why like to stick with "eliminating bias" and "reducing variance". Nice and parsimonious. Then all these other fields go and muck everything up though


Post Extras: Print Post   Remind Me!   Notify Moderator  
Alph
Carpal Tunnel


Reged: 11/23/06

Loc: Melmac
Re: 5 or 30 minute exposures for narrowband? new [Re: Inverted]
      #5654272 - 01/31/13 12:52 PM

Quote:

that's we why like to stick with "eliminating bias" and "reducing variance". Nice and parsimonious.



Yes I agree with you. It is simple and precise. Unfortunately most engineers have been taught dumbed down mathematics and they are the ones who came up with that terminology. It is inconsistent and non-intuitive. To be consistent they should say readout bias frame instead of just bias frame and dark current bias frame instead of just master dark frame.


Post Extras: Print Post   Remind Me!   Notify Moderator  
freestar8n
Post Laureate
*****

Reged: 10/12/07

Re: 5 or 30 minute exposures for narrowband? new [Re: shams42]
      #5654544 - 01/31/13 03:30 PM

Quote:

But here, aren't you distinguishing "noise" from "random Poisson noise?" So in practice, it sounds like you too make some distinction between systematic or repeatable sources of noise which can be removed via calibration and random noise which cannot. We only seem to differ in that I call the former "unwanted signal."




It's not just me - it's standard ccd noise models as described in e.g. Janesick and Holst/Lomheim - in which many other types of noise are described.

You can even have moving pattern noise - and you can subtract it, if you can model it, - as described here. The noise has a recognizable pattern that can be modeled and removed - even though it changes in each frame. For that you could not use a master bias and you would want to deal with it in each frame individually. It's still a form of pattern noise removal.

Pattern noise is treated as a separate noise term from the read noise - and both are found in a bias frame. The master bias captures an image of the pattern noise. Since the master bias will not be a perfect "image" of the noise, when you subtract it from a frame you greatly reduce the pattern noise - but you slightly increase the Poisson noise present in a light. If the ccd were uniform and you had no pattern noise, you would be better off not using a master bias at all - and simply subtract the mean bias value across the entire ccd. It's the presence of pattern noise in the bias and darks that motivates the subtraction - and it has a net benefit by reducing the pattern noise while slightly increasing the Poisson noise in each light.

There is separate pattern noise in the bias, the dark, and the flat.

I think the term "bias" is used because it offsets the pixel response from zero. The amount of this offset is different for each pixel - that's the pattern noise in the bias. In these models, each pixel is assumed to generate the same amount of read noise on top of that with the same Poisson distribution. In fact the distribution may not be the same for each pixel - and that is one way the noise model might not work perfectly.

Frank


Post Extras: Print Post   Remind Me!   Notify Moderator  
cn register 5
scholastic sledgehammer


Reged: 12/26/12

Re: 5 or 30 minute exposures for narrowband? new [Re: orlyandico]
      #5654549 - 01/31/13 03:31 PM

Quote:

i just did some 20 minute narrowband subs last night.

compared them to last year's effort on the same subject (horsehead) where i used 10 minute subs.

the 20 minute ones are definitely cleaner.



It would be interesting to compare 2 x 10 minute exposures with one 20 minute exposure, that way you have the same exposure and so the same amount of data from the sky. What's different is one has two helpings of read noise and the other has one.

What does the difference look like then?

Chris


Post Extras: Print Post   Remind Me!   Notify Moderator  
Calypte
Pooh-Bah


Reged: 03/20/07

Loc: Anza, California
Re: 5 or 30 minute exposures for narrowband? new [Re: cn register 5]
      #5654583 - 01/31/13 03:53 PM

Wodaski's The New CCD Astronomy deals with these exact questions. I'm surprised that some people treat this as a deep mystery, when in fact it's already been answered. No, a series of short exposures don't equal longer exposures of the same total time. You may have good reasons for choosing shorter subs, and you may achieve a very good final result, but it's not the same. Just my humble opinion.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Peter in Reno
Postmaster
*****

Reged: 07/15/08

Loc: Reno, NV
Re: 5 or 30 minute exposures for narrowband? new [Re: Calypte]
      #5654624 - 01/31/13 04:21 PM

A good read is available at:

http://deepskystacker.free.fr/english/index.html

Click on "How to create better images" link at left side. It's a very well and easy to read document. Here is a snippet:

"Are 100 x 1 minute and 10x10 minutes giving the same result?

Yes when considering the SNR but definitely No when considering the final result.

The difference between a 10 minutes exposure and a 1 minute exposure is that the SNR in the 10 minutes exposure is 3.16 higher than in 1 minute exposure.

Thus you will get the same SNR if you combine 10 light frames of 10 minutes or 100 light frames of 1 minute. However you will probably not have the same signal (the interesting part). Simply put you will only get a signal if your exposure is long enough to catch some photons on most of the light frames so that the signal is not considered as noise.

For example for a very faint nebula you might get a few photons every 10 minutes. If you are using 10 minutes exposures, you will have captured photons on each of your light frames and when combined the signal will be strong.

If you are using 1 minute exposures you will capture photons only for some of your light frames and when combined the photons will be considered as noise since they are not in most of the light frames."

Peter


Post Extras: Print Post   Remind Me!   Notify Moderator  
Alph
Carpal Tunnel


Reged: 11/23/06

Loc: Melmac
Re: 5 or 30 minute exposures for narrowband? new [Re: freestar8n]
      #5654664 - 01/31/13 04:44 PM

Quote:

I think the term "bias" is used because it offsets the pixel response from zero.



Yes, I know they call it bias for a different reason, the one you mentioned. I don't expect them to know statistics

Quote:

each pixel is assumed to generate the same amount of read noise on top of that with the same Poisson distribution.



The readout noise is not modeled as Poisson distribution. That is why it stands out from other noise sources and adds a different wrinkle to image stacking.


Post Extras: Print Post   Remind Me!   Notify Moderator  
cn register 5
scholastic sledgehammer


Reged: 12/26/12

Re: 5 or 30 minute exposures for narrowband? new [Re: cn register 5]
      #5654676 - 01/31/13 04:50 PM

A lot of this discussion seems to be presenting what I see as arguments from authority - stating that something is so or quoting other people (some unnamed) who say that it is so, with no justification. This changes the discussion from a scientific/technical one to a matter of if you trust what the authority is saying - a matter of faith.

I'm not very keen on using arguments from authority in a scientific/technical discipline such as Astronomy.

There are some useful things here, Mark posted a couple of very useful links, one of which was a way of comparing the read noise with the signal you are actually getting.

http://www.hiddenloft.com/notes/SNR.txt


What it said is that the sky noise should be three times the read noise to make the read signal about 10% of the sky signal. (edited because I confused noise and signal in a couple of places). This allows us to calculate how much signal is required to make the read noise a sufficiently small fraction of the signal. For wide band filters and/or a light polluted site a short exposure will be needed to get here but at a dark site with narrow filters a very long exposure may be needed.

If I apply some numbers, my ATK 383L+ has a read noise of 7e. 7x3 is 21 so that's how much noise will be in the minimum sky signal. Squaring this gives a signal of about 400e. So If I have 400 or more electrons generated by photons from the sky in each pixel the read noise contribution is sufficiently small. Multiplying 400e by the camera gain (about 2.5) says that the DAC value needs to be at least 1000 units above the dark current in the darkest areas of the image. That value should be the same everywhere, regardless of the filters or how dark the sky is.

If you are in a really dark sky area with very narrow filters this will take ages, if you are in a light area with wide filters it will take a lot less time. In either case it gives you an indication of if your exposures were long enough that read noise is not significant.

Last night I had the first clear evening for a fortnight so I set up doing 5 minute exposures of the Horse head area. Of the 20 exposures only seven were acceptable because of passing clouds. The signal level in the dark areas, such as the horse head, were about 1200 so that doesn't look too far away from what should be needed.

Hope this helps,

Chris

Edited by cn register 5 (01/31/13 05:48 PM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
Alph
Carpal Tunnel


Reged: 11/23/06

Loc: Melmac
Re: 5 or 30 minute exposures for narrowband? new [Re: cn register 5]
      #5654715 - 01/31/13 05:14 PM

Quote:

What it said is that the sky noise should be three times the read noise to make the read noise about 10% of the sky noise.



Do you realize that your statement is incoherent?


Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: cn register 5]
      #5654726 - 01/31/13 05:20 PM

Quote:

Simply put you will only get a signal if your exposure is long enough to catch some photons on most of the light frames so that the signal is not considered as noise.




The last part is important. "so that the signal is not considered as noise"

SNR is kind of an odd metric, and not just because your sort of pretending you know the actual signal power and noise power, but because what we seem to really want to know is the comparison of the weakest parts of the signal, versus the noise; not just the estimation of the observed signal versus the noise.

More technically, the SNR statistic is actually just the sample mean over the sample standard deviation. This is, in itself a biased estimator of the asymptotic mean over the asymptotic standard deviation.

Mathematically though, if you have sufficient calibration, and sufficient samples (exposures), at some point, more shorter exposures will tend to provide a cleaner signal (image) than fewer longer. Any result that shows otherwise likely assumes insufficient calibration or samples, or has a flaw somewhere in the logic.

If we took a low noise camera, take lots of calibration frames, and lots of exposures, at some point, the results over a given integration time, should start to converge to a better sampling of the underlying signal, than fewer long exposures. It's really not a humble opinion, just a mathematical result. I believe this is already being done in some medical imaging applications, for example, actually. To get good calibration though, my understanding is that very low noise cameras are being cooled to around -90 Deg C, flats taken using extremely well calibrated light boxes, and lots of bias frames used etc... So, this isn't necessarily something we can do in the hobby at this time. However, blindly assuming longer is better, also does not seem completely palatable, even with narrow band filters...


Post Extras: Print Post   Remind Me!   Notify Moderator  
freestar8n
Post Laureate
*****

Reged: 10/12/07

Re: 5 or 30 minute exposures for narrowband? new [Re: cn register 5]
      #5654738 - 01/31/13 05:26 PM

Quote:

This changes the discussion from a scientific/technical one to a matter of if you trust what the authority is saying - a matter of faith.




Well I certainly agree with that - which is why I provide direct links to professional sources - which themselves contain references. Many of links provided in these discussions are just to web pages that themselves have no references. An argument from authority should be distinguished from a "reference to the literature" - and primary sources. On this topic, the amateur sources, and arbitrary web pages, can depart from professional literature - and cause confusion or state incorrect conclusions.

The link you provided may be perfectly correct - but its validity is completely unknown since it is just a discussion grabbed from the web - which in fact is what this thread is. The only implied validation of the content of that text is the assumed expertise of the people writing - which is a form of appeal to authority.

As for the numbers you present - great - that is the first thing I suggested doing in my first posting of this thread:

Quote:

It's good to know the actual read noise of the camera and how much signal is coming from sky glow rather than nebulosity. A few exposures should give a feel for these values.




So - now that you have numbers for your site and your equipment, you can make concrete decisions about the optimal exposure time. The fact that you were hit by passing clouds is a perfect example of why, despite the models and calculators, other factors may make an even shorter exposure more practical and increase your achieved SNR in the long run.

The main thing is - your calculation is based on making read noise small in each sub compared to sky and other noise - and that is the key point about sub-exposure time that keeps being lost or confused in this thread.

Frank


Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: freestar8n]
      #5654748 - 01/31/13 05:31 PM

Quote:


I think the term "bias" is used because it offsets the pixel response from zero.




0 is the expected value of the pixels, given no signal. Even moving pattern noise will converge to random variation or some mean deviation from the expected pixel value, given enough samples. If it converges to a mean value, other than 0, it is in statistical terms "bias", but in this case, can be removed via calibration. At least given that enough calibration samples are taken, "bias frames" so that the master bias frame, converges to nearly the asymptotic mean value of each pixel. Also, we probably need a sufficient pedestal to be used so that we don't clip any data.


Post Extras: Print Post   Remind Me!   Notify Moderator  
cn register 5
scholastic sledgehammer


Reged: 12/26/12

Re: 5 or 30 minute exposures for narrowband? new [Re: Alph]
      #5654766 - 01/31/13 05:45 PM

Quote:

Quote:

What it said is that the sky noise should be three times the read noise to make the read noise about 10% of the sky noise.



Do you realize that your statement is incoherent?



I think I should have said three times the read noise to make the read signal about 10% of the sky signal, it's because the variance of the signal is the square root of the average value. You can square the noise to get the signal.

This is all in the reference I posted.

Chris

Edited by cn register 5 (01/31/13 05:49 PM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
cn register 5
scholastic sledgehammer


Reged: 12/26/12

Re: 5 or 30 minute exposures for narrowband? new [Re: Inverted]
      #5654787 - 01/31/13 06:00 PM

Quote:


I think the term "bias" is used because it offsets the pixel response from zero.



There's a nice story about this.
There were some analyses being done of the mine waste from gold mines in South Africa with the idea that improved methods of extraction could make it profitable to rework the old mine spoil heaps. Some Au values were negative so they were discarded - after all you can't have less than nothing. I'm not sure how much mine tailings they dug up before they realised there wasn't as much Glod there as they thought...

Chris


Post Extras: Print Post   Remind Me!   Notify Moderator  
Alph
Carpal Tunnel


Reged: 11/23/06

Loc: Melmac
Re: 5 or 30 minute exposures for narrowband? new [Re: Inverted]
      #5654793 - 01/31/13 06:02 PM

Quote:

0 is the expected value of the pixels, given no signal.



No, 0 is not an expected value. Cameras add an offset value (bias) to readout by design and the expected value has spatial bias (fixed pattern noise if you will).


Post Extras: Print Post   Remind Me!   Notify Moderator  
Mike7Mak
Pooh-Bah
*****

Reged: 12/07/11

Loc: New York
Re: 5 or 30 minute exposures for narrowband? new [Re: shams42]
      #5654826 - 01/31/13 06:14 PM

Quote:

See, my position is that this is MORE confusing that attempting to distinguish between "unwanted signal" and "random noise" as I have proposed.



I have to agree. It seems quite unscientific to broaden the meaning of noise to the point that 'noise' has noise. IMO use of the term 'noise' as a catch-all for unwanted signal approaches the level of slang.

I can say from personal experience until the noob grasps the significant difference between 'unwanted signal' and 'variance/uncertainty', discussions like this leave us scratching our head. Noise remains a mystical evil spirit that we ward off with long exposure and image stacking on faith alone...cuz the guys who make pretty pics say so.


Post Extras: Print Post   Remind Me!   Notify Moderator  
freestar8n
Post Laureate
*****

Reged: 10/12/07

Re: 5 or 30 minute exposures for narrowband? new [Re: cn register 5]
      #5654839 - 01/31/13 06:23 PM

Hi-

I haven't gone through your numbers but I wanted to confirm your 1200 ADU value is measured from a calibrated, rather than raw, sub exposure. You want to make sure there are no artificial offsets in there. You could just measure the difference between a single dark and a raw sub of the horsehead to make sure there is no offset involved.

Also - I recommend sticking to gain as e/adu - so yours is about 0.4 e/adu. To convert e to adu you would divide by gain.

Also - what f/ratio and what bandpass? I assume no moon involved? With those numbers, more general conclusions could be drawn for your camera and location.

Frank


Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? [Re: Alph]
      #5654878 - 01/31/13 06:44 PM

Quote:

Quote:

0 is the expected value of the pixels, given no signal.



No, 0 is not an expected value. Cameras add an offset value (bias) to readout by design and the expected value has spatial bias (fixed pattern noise if you will).




Gotcha, that makes sense. I did say, probably would need to add a pedestal though, "offset" is a much better term. "Bias" in that regard is then extremely confusing


Post Extras: Print Post   Remind Me!   Notify Moderator  
Pages: 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | (show all)


Extra information
13 registered and 14 anonymous users are browsing this forum.

Moderator:  Oldfield, Scott in NC, bilgebay, WOBentley 

Print Thread

Forum Permissions
      You cannot start new topics
      You cannot reply to topics
      HTML is disabled
      UBBCode is enabled


Thread views: 7389

Jump to

CN Forums Home




Cloudy Nights Sponsor: Astronomics