Return to the Cloudy Nights Telescope Reviews home page


Astrophotography and Sketching >> CCD Imaging & Processing

Pages: 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | (show all)
cn register 5
scholastic sledgehammer


Reged: 12/26/12

5 or 30 minute exposures for narrowband?
      #5645533 - 01/27/13 06:47 AM

What the benefits (if any) of doing 15 to 30 minute exposures for narrowband compared to shorter exposures, such as 5 minutes?

The only advantage I can see is that fewer exposures will have less read noise but that's it. The other sources of noise are proportional to the total exposure time aren't they?. Read noise is pretty small compared to everything else isn't it?

The disadvantage I can see are that if something goes wrong more is lost with long exposures.

Chris


Post Extras: Print Post   Remind Me!   Notify Moderator  
vpcirc
Post Laureate


Reged: 12/09/09

Loc: Merced CA
Re: 5 or 30 minute exposures for narrowband? new [Re: cn register 5]
      #5645559 - 01/27/13 07:33 AM

It's all about signal. Narrowband blocks a lot of the light spectrum and requires more time to accumulate enough signal. Also the number of frames you can stack to improve your SNR is limited before benefits disappear. With any image, the longer the better to a point. If you have a lot of sky glow, that is signal as well and well overwhelm your background. One reason guys can shoot narrowband in light polluted areas is they can block a lot of that sky glow. You could take 30 5 min NB images and 5 15 min shots, and the 5 15 min shots will look much better than the 30 5 min shots even though you have less total time. You want your exposure to go until you've reached your sky noise limit. In my case I'm at a dark site, so my typical NB image is 30-40 for each. For an expert explanation read John Smith's article here: http://www.hiddenloft.com/notes/SubExposures.pdf

Post Extras: Print Post   Remind Me!   Notify Moderator  
dawziecat
Carpal Tunnel
*****

Reged: 10/20/10

Loc: Rural Nova Scotia
Re: 5 or 30 minute exposures for narrowband? new [Re: cn register 5]
      #5645600 - 01/27/13 08:23 AM Attachment (76 downloads)

There is theory . . . and there is what YOU can do in YOUR world with YOUR equipment under YOUR skies!

I quite accidentally discovered that I could get some nice results with (hold your breath) ONE MINUTE NB exposures!

It amazed me!

I've posted the attached image here before. It was taken with a 180mm lens at f3.5 and a SBIG ST-8300M camera with a 3nm Ha filter.

I'm not saying it's ideal to do this. Not hardly! But, if short exposures are the best your gear can do, you can still get some pretty nice images.


Post Extras: Print Post   Remind Me!   Notify Moderator  
vpcirc
Post Laureate


Reged: 12/09/09

Loc: Merced CA
Re: 5 or 30 minute exposures for narrowband? new [Re: dawziecat]
      #5645637 - 01/27/13 08:57 AM

Using a very bright DSO is hardly a comparison. There's so much signal there already, you don't need long exposures. If your're content with only shooting mag 4 or greater images and your optics are very fast, you can shoot short exposures. The key point is signal. The problem is you're going to run out of targets very quickly.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: vpcirc]
      #5645664 - 01/27/13 09:26 AM

My understanding is it depends on the read noise of your camera and also some thing like tracking. Most noise sources, sky noise, dark noise etc.. really just depend on exposure time. But it doesn't really matter if it is one long exposure, or lots of little exposures, the noise is just dependent on the sum of the exposures. However, every time you take an image, you get some read noise from the camera. This does not increase with time though, it increases with the number of exposures. If you have a camera with really low read noise, then it shouldn't matter as much if you take one exposure or lots of exposures. From the specs, some of the new cameras are getting really low. The 8300 has about half of the read noise, that we considered "low" just a few years ago, and some of the Atik cams, for example, have half that!

So now, when you take a long exposure, your also more prone to image shift, tracking error etc, that blur the image and kill some detail. The purpose of a longer exposure though, is to bring out that detail, by increasing SNR.

Not all setups can handle really long exposures as well as well as others though. I think for most setups, there is a happy medium between what the equipment can handle and the minimization a of camera reads and therefore "read noise". Not everyone has a Paramount mount, and zero flex in their optical train etc... And not everyone will see a difference between a super long exposure and lots of shorter exposures. Think people are best advised to experiment and see what works best for their particular equipment combination.


Post Extras: Print Post   Remind Me!   Notify Moderator  
freestar8n
Post Laureate
*****

Reged: 10/12/07

Re: 5 or 30 minute exposures for narrowband? new [Re: Inverted]
      #5645706 - 01/27/13 09:56 AM

Everything depends on the read noise of the camera and the amount of sky glow given your site, the light pollution/moon, and how narrow your filters are. You can leave "signal" out of the discussion and just focus on the noise terms - and the goal is to have the sky noise be much greater than the read noise. If you have dark skies and high read noise you will find more benefit from longer subs. If you have bright skies and low read noise, you won't.

f/ratio plays a role because a faster system will accumulate sky photons faster per pixel than a slow one. Again - this is independent of the nebulosity signal itself and only depends on the "black" parts of the sky - but it means faster systems can use shorter subs.

If your guiding and polar alignment aren't ideal then that would bias things toward the shorter exposures - due to guiding and resolution rather than noise.

It's good to know the actual read noise of the camera and how much signal is coming from sky glow rather than nebulosity. A few exposures should give a feel for these values.

Frank


Post Extras: Print Post   Remind Me!   Notify Moderator  
vpcirc
Post Laureate


Reged: 12/09/09

Loc: Merced CA
Re: 5 or 30 minute exposures for narrowband? new [Re: freestar8n]
      #5645748 - 01/27/13 10:19 AM

Well Frank, for once I agree with you. You can find a calculator here; http://www.stanmooreastro.com/sky_back_photom.html , as well as discussions. I hate it when people mislead by saying time is time 10 1 min exposures does not equal 1 10 min exposure. There's also a limit to how many exposures you can take before there is no increase in SNR and you're wasting your time. That's why Hallas says "take as long of exposures as your equipment will allow" You don't have to have a Paramount to take long exposures, it just makes it easier. JWalk is prime example of someone who could take a CGEM and take wonderful images. Why? Because he had it hypertuned then learned how to properly polar align and guide.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: vpcirc]
      #5645775 - 01/27/13 10:35 AM

Just another though, as I sit here, playing with some test images, taken last night to try out my new setup.... I'm looking at some 4 min, versus 10 minute m81 pics,. In the 4 minute exposures, I can use the histogram to bring the core to a pinpoint. However, with the 10 minute, the core is saturated and at best maybe 20-30x the size. Granted the setup did handle the 10 minute exposure fine, the stars don't show more trailing in the 10 minute, there is better detail in the arms etc... But the minimum resolution of bright objects, just isn't there.. Of course, you could always take both long and short exposures and blend them to get certain detail, which may end up more efficient than taking lots of short exposures, but as mentioned, it depends,

Post Extras: Print Post   Remind Me!   Notify Moderator  
vpcirc
Post Laureate


Reged: 12/09/09

Loc: Merced CA
Re: 5 or 30 minute exposures for narrowband? new [Re: Inverted]
      #5645845 - 01/27/13 11:16 AM Attachment (52 downloads)

Quote:

Just another though, as I sit here, playing with some test images, taken last night to try out my new setup.... I'm looking at some 4 min, versus 10 minute m81 pics,. In the 4 minute exposures, I can use the histogram to bring the core to a pinpoint. However, with the 10 minute, the core is saturated and at best maybe 20-30x the size. Granted the setup did handle the 10 minute exposure fine, the stars don't show more trailing in the 10 minute, there is better detail in the arms etc... But the minimum resolution of bright objects, just isn't there.. Of course, you could always take both long and short exposures and blend them to get certain detail, which may end up more efficient than taking lots of short exposures, but as mentioned, it depends,




On a very bright DSO such as M 81, you are going over saturate the core with long exposures. Short exposures will leave out all the detail in the rest. The best method is to combine short and long. In my example below, M 31 was shot at 15 min exposures and 2 min exposures. The core was blended into the image in using a reveal all mask and feathering as a layer.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: vpcirc]
      #5646193 - 01/27/13 02:45 PM

Buitiful image! Also, by the way, this is I assume this was probably infered, but to clarify. As we are talking about narrow band, the range will tend to be lower, so the odds of saturating anything is lower, so, you'd be less likley to benefit from shorter exposures for NB. So, that would be more justification to go longer.

Post Extras: Print Post   Remind Me!   Notify Moderator  
shams42
Pooh-Bah
*****

Reged: 01/05/09

Loc: Kingsport, TN
Re: 5 or 30 minute exposures for narrowband? new [Re: Inverted]
      #5646461 - 01/27/13 05:11 PM

For narrowband imaging, in most cases read noise IS the dominant source of noise. With sufficiently narrow filters the read noise exceeds the shot noise from the background skyglow.

For me, I want to balance the benefits of shooting longer exposures (injecting a smaller multiple of the read noise into the final stack for a fixed total integration time) against the desire to have enough subs to capitalize on dithering by using an outlier-rejection method for image integration, removing uncalibrated hot pixels, cosmic ray strikes, satellite trails, etc. I have therefore standardized on 20 minute narrowband exposures and 3 hours minimum per channel, which means I'll have at minimum 9 frames to stack.

Longer would be better. If I planned 5 hours of exposure, I'd probably do 30 minute subs because then I'd have 10 to stack.

All this assumes that your mount and guiding solution is capable of exposing for arbitrary lengths of time without trailing. If that's not the case, your hardware will set an upper limit on your exposure time.


Post Extras: Print Post   Remind Me!   Notify Moderator  
shams42
Pooh-Bah
*****

Reged: 01/05/09

Loc: Kingsport, TN
Re: 5 or 30 minute exposures for narrowband? new [Re: dawziecat]
      #5646482 - 01/27/13 05:22 PM

Quote:

There is theory . . . and there is what YOU can do in YOUR world with YOUR equipment under YOUR skies!

I quite accidentally discovered that I could get some nice results with (hold your breath) ONE MINUTE NB exposures!




Do you happen to have links to full-resolution versions of these shots, ideally without noise reduction? It's really hard to judge whether the 1 minute exposure versions are noisier at the image scale you posted.


Post Extras: Print Post   Remind Me!   Notify Moderator  
cn register 5
scholastic sledgehammer


Reged: 12/26/12

Re: 5 or 30 minute exposures for narrowband? new [Re: shams42]
      #5646549 - 01/27/13 05:56 PM

Thanks to everybody for the information, especially the paper by John Smith and the images.

What I'm hearing is that the idea is to make the read noise a small part of the contribution to each image and that the time this takes depends on the amount of other noise, such as sky glow.

In my case I'm in the UK, on the edge of a town, so am not getting the sort of dark skies you get in New Mexico Skies - in fact it's often more cloud glow than sky glow.

I haven't measured it but would guess that I can get away with more like 5 minutes for now, even with narrowband filters. In any case I'd rather stack 10 out of 12 5 minute exposures than loose two out of three 20 minute exposures because of clouds or passing planes.

Thanks for your help,

Chris


Post Extras: Print Post   Remind Me!   Notify Moderator  
dawziecat
Carpal Tunnel
*****

Reged: 10/20/10

Loc: Rural Nova Scotia
Re: 5 or 30 minute exposures for narrowband? new [Re: shams42]
      #5647217 - 01/28/13 12:08 AM

Quote:



Do you happen to have links to full-resolution versions of these shots, ideally without noise reduction? It's really hard to judge whether the 1 minute exposure versions are noisier at the image scale you posted.




Original posting.

Rest assured, theory IS correct! The longer exposures DO produce a smoother image than the ones fifteen times shorter!

Point is, people who don't have high $$$ mounts can still get some nice pics with rather shockingly short exposures . . . even with NB filters.


Post Extras: Print Post   Remind Me!   Notify Moderator  
wolfman_4_ever
Pooh-Bah


Reged: 07/15/11

Loc: El Segundo, Ca, So. Cal
Re: 5 or 30 minute exposures for narrowband? new [Re: dawziecat]
      #5647278 - 01/28/13 12:56 AM

all great posts here!

Post Extras: Print Post   Remind Me!   Notify Moderator  
J.P.M
Post Laureate
*****

Reged: 11/14/06

Loc: 65"01'N25"47'E
Re: 5 or 30 minute exposures for narrowband? new [Re: wolfman_4_ever]
      #5647455 - 01/28/13 06:24 AM

Just my 2 euro cents...

Longer is better since, if there is missing some very weak signal in short exposure, it doesn't show, up no matter how many of them you going to stack. Specially in NB imaging. Usually I'm using 1200 seconds subs but now and then, I have gone up to an hour, if the target is an extremely dim one.


Post Extras: Print Post   Remind Me!   Notify Moderator  
freestar8n
Post Laureate
*****

Reged: 10/12/07

Re: 5 or 30 minute exposures for narrowband? new [Re: J.P.M]
      #5647512 - 01/28/13 07:38 AM

That would work if the limiting noise source is read noise, but if sky glow dominates the read noise then a sum of 2 5-minute frames won't look much different from a single 10-minute frame. Within the assumptions of typical models for ccd imaging SNR, read noise is the only thing you can reduce by going longer - and if it's a small part of the overall noise, then the only way to pull out faint signal is with longer total integration time - and longer subs won't help much.

The ccd noise models could be wrong or incomplete - but in that case it would be nice to know what is missing - so it can be added to the model.

Frank


Post Extras: Print Post   Remind Me!   Notify Moderator  
vpcirc
Post Laureate


Reged: 12/09/09

Loc: Merced CA
Re: 5 or 30 minute exposures for narrowband? new [Re: freestar8n]
      #5649281 - 01/28/13 09:18 PM

I had a discussion today with a narrowband expert. With Narrowband you are not likely to ever exceed the sky glow level so taking the longest images possible is very important. The longer exposures bring out much more nebulosity that will never be seen with short exposures. There is no measurable improvement in image quality once your stack goes beyond 16, therefore longer images become even more important.

Post Extras: Print Post   Remind Me!   Notify Moderator  
freestar8n
Post Laureate
*****

Reged: 10/12/07

Re: 5 or 30 minute exposures for narrowband? new [Re: vpcirc]
      #5649647 - 01/29/13 02:25 AM

I would be interested in any numbers you can provide, along with an appropriate noise model, to help make your (his?) case. I consider it pretty obvious that a fast lens with a wide passband, a ccd with low read noise, and a full moon would likely be limited by Poisson noise from the sky background rather than read noise - and not benefit from long exposures. I welcome you to invite the person you are citing to explain his reasoning.

The importance of read noise and longer subs depends entirely on the scale of read noise compared to other noise sources. Without those numbers, you can't say if longer subs would help much - and as described by several people above, they may make the end result worse due to imperfect guiding and having fewer subs for a good statistical combine with noise rejection.

Frank


Post Extras: Print Post   Remind Me!   Notify Moderator  
vpcirc
Post Laureate


Reged: 12/09/09

Loc: Merced CA
Re: 5 or 30 minute exposures for narrowband? new [Re: freestar8n]
      #5649691 - 01/29/13 04:24 AM

Frank I know you have a lot knowledge, but this gentleman happens to be an optics expert. I will quote from his website;

Also, narrowband signals are inherently weak, requiring long exposures of 20 Ė 40 minutes.


Post Extras: Print Post   Remind Me!   Notify Moderator  
dawziecat
Carpal Tunnel
*****

Reged: 10/20/10

Loc: Rural Nova Scotia
Re: 5 or 30 minute exposures for narrowband? new [Re: vpcirc]
      #5649862 - 01/29/13 08:45 AM

Quote:


Also, narrowband signals are inherently weak, requiring long exposures of 20 Ė 40 minutes.





Yes, but at what focal ratio?

You most certainly do not require such exposure durations with f/2.8 optics. Lots of iconic objects in astronomy are not small in angular diameter and not especially faint in Ha either.

These can be imaged very satisfactorily in NB with fast camera lenses and almost all of us do image these objects when starting out in this hobby.

Sure, it is ideal that one can do long subs and great if your gear is up to it. But, for a great many of us, our gear limits what we can reasonably accomplish.

My biggest thrill in this hobby to date was when I first tried an Ha filter with a DSLR and a 300mm f/2.8 camera lens. No one should deny themselves this out of fear their gear is not up to 20-40 minute exposures. Stack up a dozen or two exposures of a few minutes each (or even less) and be prepared for a kick!

Seems to me there is a conflict of expectations here. What you need to image a small SNR in NB and what you need to do The Rosette, The NA Nebula complex, the Horsehead, and the like are pretty different in regards to FL, associated focal ratio and attendant sub exposure length.


Post Extras: Print Post   Remind Me!   Notify Moderator  
vpcirc
Post Laureate


Reged: 12/09/09

Loc: Merced CA
Re: 5 or 30 minute exposures for narrowband? new [Re: dawziecat]
      #5649887 - 01/29/13 09:07 AM

That's a great example, he said to try and image the horsehead at short exposures and see what happens compared to longer exposures. A very bright DSO like the Rosette isn't going to show the signal problem because so much is there.
But again the point is why stack 2 dozen images. There is zero gain from going beyond 16.


Post Extras: Print Post   Remind Me!   Notify Moderator  
David PavlichAdministrator
Transmographied
*****

Reged: 05/18/05

Loc: Mandeville, LA USA
Re: 5 or 30 minute exposures for narrowband? new [Re: cn register 5]
      #5649891 - 01/29/13 09:10 AM

Remember to play nice and leave the personal comments out of the discussion. It adds nothing to the conversation.

David


Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: dawziecat]
      #5649898 - 01/29/13 09:16 AM

I'm no expert on CCDs or ccd noise models. However, just to state the obvious, it seems clear that if read noise is the dominant source of noise, and read noise varies considerably between cameras, then it would seem that the same snr, over a fixed integration time will vary considerably by camera. For example, if you have a camera with 3e- vs one with 15e- , take 1 hour of integration, with 6x10 minute exposures, with the 15e- camera, then the same read noise contribution, with the 3e- camera should be archived with 30x2 minute exposures. However, through sampling, all random noise should tail further towards the asymptote with the 30x2 minute exposures. Also, if you either your shots, fixed pattern noise should as well. So, your total snr I think may actually be better, as long as the total read noise contribution is roughly the same or better. also, depending on how low the read noise is and how close it is to the point where read noise is no longer the dominant noise is, it should also start to become more beneficial to "sample out" other sources of noise, by taking more exposures. I have no idea what that level is though, or how close modern cameras are getting to it.

Anyways though, still in practice, you may well be better off taking longer exposures wot the low e- cameras, but as mentioned, I'm sure there a lot of factors to consider, even without considering camera noise models.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: vpcirc]
      #5649910 - 01/29/13 09:30 AM

Quote:

There is zero gain from going beyond 16.




No doubt gains decrease as you go, but anyone who understand statistics and the central limit theory, should understand that there is alway a gain until you reach the asymptote. Again, i'm no expert on ccds, but do know enough to know there will be a gain... Whether the gain improves snr noticeabley depends on all sorts of factors, but I would be interested to hear how he came up with 16. I suspect that is a rough example, based on a certain setup, sky conditions, target brightness etc.. That may be true for that one instance,, but clearly is not generalizeable. I know I can certainly see a difference between a statck of 16x5 min exposures and a stack of 50... Whether or not the stack of 50 would be better than a single 250 minute exposure, I don't know "generally" but with my setup the 50x5 would be better LOL


Post Extras: Print Post   Remind Me!   Notify Moderator  
vpcirc
Post Laureate


Reged: 12/09/09

Loc: Merced CA
Re: 5 or 30 minute exposures for narrowband? new [Re: Inverted]
      #5649917 - 01/29/13 09:40 AM

See Tony's research here;
http://www.astrosurf.com/davidguerit/doc/astromag.pdf

You can also email him at tonyhallas@sebastiancorp.net
He's very gracious and responds quickly.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Alex McConahay
Carpal Tunnel


Reged: 08/11/08

Loc: Moreno Valley, CA
Re: 5 or 30 minute exposures for narrowband? new [Re: Inverted]
      #5650001 - 01/29/13 10:29 AM

I think the number 16 is taken as a cutoff point after which there is little APPRECIABLE advantage----not that there is none at all.

The point is that to see an appreciable reduction in noise, you have to go from sixteen to thirty two exposures--so you are at the point of diminishing returns, and it just is not worth it.

Alex


Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: Alex McConahay]
      #5650048 - 01/29/13 10:58 AM

Thanks for the link Mike. I definitely wish I knew more about this stuff. I know as far as sampling goes, random noise should end up as 1/sqrt(n) where n is the number of exposures.

So at 4 exposures, you'd have half the noise of 1 exposure. To get to half the noise of 4, you'd need 16. But, then to start really seeing an improvement, it does drift off, to cut that remaining noise in half, you'd then need to go to 64, and to cut that in half, you'd need to go to 256 etc...

It looks like the example is for dark noise, but the same applies to any unbiased normally distributed random noise, so, read noise etc... in theory anyways...

It looks like he uses "8" as a start value, I'm not sure what units, but the scale will be larger with a larger starting noise. I think in practice, there are lots of factors, such as sky fog, dynamic range of computer monitors, pixel size, tracking, seeing etc... that will limit real-world appreciation of gains after some point. I know though, you can see a gain after 16, as I've added integration to images, taken previous night for example, and watched the images improve. It does definitely take more and more to see an improvement as you go, and if the image is good enough to begin with, the improvement certainly may not be perceivable.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Tandum
sage


Reged: 03/01/10

Loc: Brisbane, Australia
Re: 5 or 30 minute exposures for narrowband? new [Re: Inverted]
      #5650144 - 01/29/13 11:54 AM

My 2 cents worth. Noise is something you deal with after you have signal. I'm sure an f3 lens gives you lots of signal quickly with little noise but we don't all use f3 lenses.

Capture what you can and if it's noisey get more, simple as that. The longer the exposure the deeper you will see. My exposures are limited by polar alignment mainly. Longer than 15 minutes and I see rotation.

Here's an example with noise. Weather stopped getting more data but it still looks ok.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: Tandum]
      #5650221 - 01/29/13 12:36 PM

Quote:

Noise is something you deal with after you have signal.




Exactly. One thing to point out though is that, the accepted theory is that signal increases with time linearly. So, if your using the same equipment, the signal will be the same if you have 1 60 minute exposure, or sum 60 1 minute exposures. The remaining difference between 60 1 minute exposures and 1 6 min, is noise. Different sources of noise, can have different properties and can be best eliminated with different methods. If you reduce one source of noise to negligible levels, but still don't have a nearly perfect representation of the signal, then there must be another source of noise that hasn't been reduced sufficiently.

Of course, I honestly don't know much about all the sources of noise with this regard and how they all fit together. And since they say this hobby is all signal to noise, and we can see that we can write signal out of the equation... all I really know is I don't really know much of anything


Post Extras: Print Post   Remind Me!   Notify Moderator  
freestar8n
Post Laureate
*****

Reged: 10/12/07

Re: 5 or 30 minute exposures for narrowband? new [Re: Inverted]
      #5650257 - 01/29/13 12:53 PM

Quote:

Anyways though, still in practice, you may well be better off taking longer exposures wot the low e- cameras, but as mentioned, I'm sure there a lot of factors to consider, even without considering camera noise models.




Hi-

I think your note sums up key points - and mentions the key word, "asymptote". The basic noise model used in this context - by everyone basically - is that shot noise in the signal and in the sky glow depends only on total integration time, while the read noise is independent of integration time and only depends on the number of exposures. There are other noise terms, but this works well for typical imaging sessions involving a small number of exposures - and it completely ignores other factors such as guiding and field rotation.

A more practical view would be to consider how much sky glow signal you are getting - and that is heavily dependent on f/ratio. At f/15 with narrowband, you will need much longer sub-exposures than f/2 hyperstar in order to make read noise unimportant.

What is "unimportant"? Well - as you make the sub-exposures longer (and fix the total integration time) there will be an improvement as you reduce the impact of read noise - but at some point you won't notice an improvement anymore - and instead your image goes bad due to other factors such as guiding and field rotation.

For anyone interested in this stuff there are many web sites that describe it reasonably well - and refer to the same basic noise model. Some web pages get misinterpreted and make people think you *need* to go longer at a dark site, for example, when in fact they are just saying you *can* go longer and still get benefit - because the sky glow is small.

I think the example given earlier by Dawziecat is great and should make several points. You can do very well with short exposures if you have a fast system - even with 3nm filters. His example stacks a very large number of subs - and I think the point of diminishing returns would be met much earlier - but the key point is that you would need to know the read noise and sky glow to determine how much of a win there is in using very long sub-exposures.

If there are any web pages or people who disagree with this my assessment here - I'm happy to address them.

Frank


Post Extras: Print Post   Remind Me!   Notify Moderator  
vpcirc
Post Laureate


Reged: 12/09/09

Loc: Merced CA
Re: 5 or 30 minute exposures for narrowband? new [Re: Inverted]
      #5650286 - 01/29/13 01:09 PM

Quote:

Thanks for the link Mike. I definitely wish I knew more about this stuff. I know as far as sampling goes, random noise should end up as 1/sqrt(n) where n is the number of exposures.

So at 4 exposures, you'd have half the noise of 1 exposure. To get to half the noise of 4, you'd need 16. But, then to start really seeing an improvement, it does drift off, to cut that remaining noise in half, you'd then need to go to 64, and to cut that in half, you'd need to go to 256 etc...

It looks like the example is for dark noise, but the same applies to any unbiased normally distributed random noise, so, read noise etc... in theory anyways...

It looks like he uses "8" as a start value, I'm not sure what units, but the scale will be larger with a larger starting noise. I think in practice, there are lots of factors, such as sky fog, dynamic range of computer monitors, pixel size, tracking, seeing etc... that will limit real-world appreciation of gains after some point. I know though, you can see a gain after 16, as I've added integration to images, taken previous night for example, and watched the images improve. It does definitely take more and more to see an improvement as you go, and if the image is good enough to begin with, the improvement certainly may not be perceivable.




Since most stacking programs "average" the images you might see an improvement in image quality as the new images may be of higher quality than the previous. I think the result would be even better though if you again measured all images and deleted all but the best 16. I do not profess to have any understanding of the science behind this but I can assure you the folks agreeing with Tony's findings are some of the top imagers in the country. I did ask Tony if that theory applied to light frames in person at PATS this year. The answer was yes.


Post Extras: Print Post   Remind Me!   Notify Moderator  
vpcirc
Post Laureate


Reged: 12/09/09

Loc: Merced CA
Re: 5 or 30 minute exposures for narrowband? new [Re: freestar8n]
      #5650306 - 01/29/13 01:18 PM





Some web pages get misinterpreted and make people think you *need* to go longer at a dark site, for example, when in fact they are just saying you *can* go longer and still get benefit - because the sky glow is small.

Frank




I was told directly by John Smith at AIC that indeed you do need to go longer at a dark site, especially with narrowband. John is an expert in this field. For those who do not know John, you can find his many useful articles and tips here; http://www.hiddenloft.com/


Post Extras: Print Post   Remind Me!   Notify Moderator  
freestar8n
Post Laureate
*****

Reged: 10/12/07

Re: 5 or 30 minute exposures for narrowband? new [Re: vpcirc]
      #5650333 - 01/29/13 01:33 PM

Quote:

I was told directly by John Smith at AIC that indeed you do need to go longer at a dark site, especially with narrowband.




Yes - I think you misunderstood what he was telling you. I would be very surprised if he, or anyone familiar with the underlying noise model, would say you need to use longer subs at a dark site to achieve the same quality in the same time as at a bright site. The message he was probably trying to convey is - you *can* and you *should* go longer - and your results would be even better at a dark site.

The benefit you get from longer subs depends on read noise and sky glow. Short subs at a dark site will end up with a much better image than the same subs at a bright site. But going longer at a bright site won't help much and isn't really worth it - whereas at a dark site it is worth it.

It all comes down to knowing the noise terms - which in this case is read noise and sky glow - for your camera, equipment, and location.

Frank


Post Extras: Print Post   Remind Me!   Notify Moderator  
vpcirc
Post Laureate


Reged: 12/09/09

Loc: Merced CA
Re: 5 or 30 minute exposures for narrowband? new [Re: freestar8n]
      #5650342 - 01/29/13 01:41 PM

I don't think I "misunderstood him" as I was surprised to hear him say that as I thought a dark site meant I could go shorter. He was helping me evaluate my images. I will find out, but it had to do with sky glow "hiding noise", and since sky glow is "signal" I had very little to hide any in noise in the background and therefore needed longer images to overcome that noise. I was shooting 15 Min LRGB and 30 min NB.

Post Extras: Print Post   Remind Me!   Notify Moderator  
vpcirc
Post Laureate


Reged: 12/09/09

Loc: Merced CA
Re: 5 or 30 minute exposures for narrowband? new [Re: vpcirc]
      #5650359 - 01/29/13 01:46 PM

For a great understanding of the benefit of longer images and why they are much better than multiple short images, John has an easy to understand writeup here

http://www.hiddenloft.com/notes/SNR.txt


Post Extras: Print Post   Remind Me!   Notify Moderator  
freestar8n
Post Laureate
*****

Reged: 10/12/07

Re: 5 or 30 minute exposures for narrowband? new [Re: vpcirc]
      #5650378 - 01/29/13 01:52 PM

From what you are describing, it sounds like he assessed the noise in your subs and deduced that the sky glow was small compared to your read noise - and you would benefit from longer exposures. That sounds perfectly fine and I would probably agree with it - assuming it really is read noise dominated. If you took the same setup to a place with big sky glow, or used a faster system or wider filter, you would be doubly impacted in a bad way. Your subs would be much noisier - due to sky glow - and you would not get any benefit from longer subs. All you can hope for is to take many more subs. But - as you say - beyond some number of subs, the stacking just doesn't gain much - so you are just stuck with what you get.

So dark sites are great: your subs look great, and you can make them look even better by going longer. At a bright site - you can't use that trick.

Many people are imaging from brighter sites and/or using fast systems - and they shouldn't feel a need to expose long sub-exposures if it doesn't really help - and in fact can make the end result worse due to other factors. So I just say people should be aware of the noise terms in their imaging so they can make an educated guess of how long they should go - and it all depends.

Frank


Post Extras: Print Post   Remind Me!   Notify Moderator  
Mike Wiles
scholastic sledgehammer


Reged: 02/04/09

Loc: Goodyear, AZ
Re: 5 or 30 minute exposures for narrowband? new [Re: freestar8n]
      #5650613 - 01/29/13 03:41 PM

This thread has given me more to think about all by itself than the last 100 threads I've read put together. Excellent discussion. As I'm likely about to demonstrate, I'm anything but an expert on any of this topic.

In terms of getting a shot noise limited narrowband image....I have everything working against me. I have a slow-ish system (f/7.5) with narrow bandpass filters (3nm) and I image from dark to really dark skies (mag 5.4 at home, 6.5+ in the field). With a measured read noise of 9 electrons my camera isn't overly noisy either. I've done the calculations using test exposures for my own setup and from my magnitude 5.4 backyard at home I'd have to shoot upwards of 5 hour subexposures in order to bury the read noise in the shot noise. From the dark sites that I go to where limiting magnitude is up around 7.5 - there's not enough dark hours in a single day to take a shot noise limited subexposure. So there is some credence to the claim that it's extremely difficult to capture shot noise limited subexposures using narrowband filters. I wouldn't say it's impossible - but extremely difficult and almost certainly impractical in real world conditions. With a 7nm filter on an f/2.8 system in Manhattan...it's possibly do-able. It seems unlikely to me that you're going to get a shot noise limited narrowband image without going 20 or 30 minutes on a subexposure regardless of the equipment configuration.

Having said all that, the S/N ratio of an image is 0 if it's not attempted. As a result, a stack of 300 ten second subexposures will always have a higher S/N than not trying at all. You work with the equipment and environment that you have. To my experience - which is limited compared to many here - narrowband exposures should be as long as you dare to take them based on your equipment, conditions and comfort level. You should also take a significant number of subexposures to gain the maximum benefit from modern statistical data rejection algorithms. I think the point of diminishing returns is somewhere up around 30 subs....but I would agree that 16 is the minimum.

With it being impractical at best to take a shot-noise limited narrowband exposure it seems to me that a decent alternative is to:

  • Create a master bias frame taken from an enormous number of bias subs. This will give a really clean master calibration frame to do as much as possible for reducing read noise.
  • Shoot a lot of subexposures on the light frames. Statistical data rejection works more in your favor - and you're going to need it for random read noise from frame to frame.
  • Dither between subexposures. Any pattern noise and random noise left after the above two steps will disappear for the most part.


Mike


Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: vpcirc]
      #5650627 - 01/29/13 03:48 PM

Quote:


Since most stacking programs "average" the images you might see an improvement in image quality as the new images may be of higher quality than the previous.




I used the term "sum" because most people seem to think of signal as something that changes and intensifies. Really, the signal is more conceptual though and is basically fixed. I guess you could look at it as summing, but relative to the noise. The important part is the SNR. However, mathematically, if we had infinite range, it works out the same whether we sum or average.

To illustrate, if I take 3 images, and the signal is 5 in all and the noise is 1,2,1 respectively, then the sum is 15/4 and the average is 5/1.3, but either way, the ratio (SNR) is 3.75. However, our dynamic range isn't infinite, so, for data storage and processing etc... it becomes easier to average. They key though, is statistically speaking, the signal is really a conceptual idea rather than a physical entity.


Edit: by the way, the above is a simplified example, it is intended to illustrate summation versus averaging, not how noise is actually distributed.

Edited by Inverted (01/29/13 04:15 PM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
freestar8n
Post Laureate
*****

Reged: 10/12/07

Re: 5 or 30 minute exposures for narrowband? new [Re: Mike Wiles]
      #5650726 - 01/29/13 04:50 PM

Quote:

I have everything working against me. I have a slow-ish system (f/7.5) with narrow bandpass filters (3nm) and I image from dark to really dark skies (mag 5.4 at home, 6.5+ in the field). With a measured read noise of 9 electrons my camera isn't overly noisy either. I've done the calculations using test exposures for my own setup and from my magnitude 5.4 backyard at home I'd have to shoot upwards of 5 hour subexposures in order to bury the read noise in the shot noise.




Hi-

I think the main confusing thing about these sub-exposure calculators is it makes people feel *bad* when the *have to* expose a long time. If the problem is that you have a very high read noise camera - that is indeed a bad thing. But if you are "stuck with" narrowband filters and a dark site - those are both *great* things. As long as your read noise isn't really bad, your images will be *much better* than at a bright site where you are "blessed" with short sub-exposures.

So if you are in this situation - where you have a decent camera but slow optics and a dark sky - you will still be doing much better than if you did not have a dark sky. The only downside is that you know that if your camera had lower read noise, or if you had greater aperture, a given total exposure would be slightly better. But that same total exposure at a bright site would be much worse - so be thankful.

I think for most people the general guideline would be to go as long as you can as long as you end up with more than perhaps 5 sub-exposures. If you are using a guidescope on an sct you will probably be limited by flexure and have to keep it short. If you have field rotation - same thing. If your stars saturate - same thing.

But - yes - go as long as you can if the calculations suggest you should go longer - but you shouldn't feel bad about not using the optimal sub-exposure time if it isn't practical.

Frank


Post Extras: Print Post   Remind Me!   Notify Moderator  
Alph
Carpal Tunnel


Reged: 11/23/06

Loc: Melmac
Re: 5 or 30 minute exposures for narrowband? new [Re: vpcirc]
      #5650805 - 01/29/13 05:38 PM

Quote:

For a great understanding of the benefit of longer images and why they are much better than multiple short images, John has an easy to understand writeup here

http://www.hiddenloft.com/notes/SNR.txt




Mike,
I would recommend The Signal to Noise Connection part I & II articles by Mike Newberry published in CCD Astronomy magazine (long gone) in the summer and the fall of 1994. IMO a must read for everyone.

It is true that pitch dark sky and long exposures might be the only option to capture the dimmest parts of an already very dim object. However there is no science behind the number 16 and stacking a large number of shorter exposures can produce equally good results in many cases.

Here is a quote from the cited article:

Quote:

If our exposures are background limited because the readout noise is small in comparison to the noise from skyglow, it is usually preferable to stack exposures rather make a single long one.




Mike! Thanks for the pass to the last year's RTMC


Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: freestar8n]
      #5650947 - 01/29/13 06:57 PM

I was thinking more about this thread on the way home from work. I realize we really don't define "signal" and "noise" and these aren't actually intuitive concepts. From a statistical perspective, signal is really the theoretical average of some parameter, from some population, given an infinite number of unbiased samples. In this case, i think the population is the photons hitting the photon collector, over the integration time. So, this signal is the theoretical average "rate" of photon arrival, as we have a count/time. The bias is then anything that deviates our ability to detect this rate, in a fixed way, so, this is really the "pattern noise".. For example, there could be gaps between the pixels, so, some photons will miss the collector in a fixed way and there are other fixed errors counting the photons and translating to an image to display etc.... Once we take out the bias, the remaining deviations from the theoretical average rate of photon arrival, are random errors In our detection. If something is random and we take one sample, then that random error will have a large contribution, but if we take another and it isn't there, and average ( or sum ) the two, then it will have a smaller contribution to the total detected population and so on... But the signal is conceptual and is the theoretical average rate of photon arrival and the noise is just everything else that effects a given sampling of this population. So, the signal, being conceptual is always there, it doesn't really "increase"', it's just through various methods, we are able to decrease the noise to "see" the average rate of photon arrival and translate it into a perceivable image.

Post Extras: Print Post   Remind Me!   Notify Moderator  
vpcirc
Post Laureate


Reged: 12/09/09

Loc: Merced CA
Re: 5 or 30 minute exposures for narrowband? new [Re: Inverted]
      #5651145 - 01/29/13 08:51 PM

Be careful about using the word sunn and average. If, for example you chose sum in CCDstack verse Average, your result would be totally different. Adam Block gives excellent examples of the differences in alogarythims in his ccdstack tutorials.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: vpcirc]
      #5651332 - 01/29/13 10:54 PM

Quote:

Be careful about using the word sunn and average. If, for example you chose sum in CCDstack verse Average, your result would be totally different. Adam Block gives excellent examples of the differences in alogarythims in his ccdstack tutorials.




Mike, mathematically, this isn't that complicated once you take out all the intimidating terminology. When we're talking about summing or averaging here, we're really just talking about a ratio of fractions I.e.

Edit, I realize I had made this confusing and it may not have actually been obvious what I was getting at, as written. I'm trying not to write out too much math. So, I rewrote this slightly to emphasize better in English. The point, is summing and averaging are just linear functions of the other. If you have a perfect computer system, you can always go back and forth, one is a scaled version prof the other
(sum(xi)/n) = sum(x1)/n + sum(x2)/n .... sum(xn)/n

Where n = the number of exposures. Whether you sum or average, one is just an equivelent linear function of the other and in each case, x is part signal and part noise. The signal part stays constant though and the random part of the noise changes for each exposure. So, when we sum or average them, we have a scaled linear function of the other and if there isn't truncation of the data, through computational limitations, we can always go back and forth, if we know n (the number of exposure)

You can also re-write s/n as (signal mean) / (standard deviation of noise) if you google summation of variance and work your way through it though, you'll find it works out the same, in that one (I.e, summing or averaging) is a scaled function of the other with respect to the ratio.

Again, As i alluded to in the previous post there can be differences because of scaling, rounding, truncation of big numbers, analog to digital scaling/conversion, but these are limitations of the specific hardware and software design used, not any fundamental properties.

Edited by Inverted (01/30/13 05:55 AM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
David PavlichAdministrator
Transmographied
*****

Reged: 05/18/05

Loc: Mandeville, LA USA
Re: 5 or 30 minute exposures for narrowband? new [Re: Inverted]
      #5651344 - 01/29/13 11:07 PM

Just to jump into the middle of this, around the end of October or early November, the Advanced Imaging Conference takes place around San Jose, CA. If there's one astroconference you want to attend if you enjoy imaging, this is the one! All the best imagers are there and many of them have workshops. You'll learn a ton of stuff and meet some really great people in the process. And I won't mention the Vendor's Hall. Leave your credit card at home.

David


Post Extras: Print Post   Remind Me!   Notify Moderator  
mikeschuster
super member


Reged: 08/25/11

Loc: SF Bay area
Re: 5 or 30 minute exposures for narrowband? new [Re: David Pavlich]
      #5651467 - 01/30/13 12:36 AM

Note also dark current noise. Going longer to overcome read noise means there is even more dark current noise to overcome. In my case at f/5, 3nm Ha, dark skies, 40 minute exposures. Read + dark noise is about 14e-. Typically at 40 minutes my sky background noise is about 20e-. This is *far* from being sky limited. But I can't expose any longer most of the time due to temperature dependent focus changes.
Mike


Post Extras: Print Post   Remind Me!   Notify Moderator  
freestar8n
Post Laureate
*****

Reged: 10/12/07

Re: 5 or 30 minute exposures for narrowband? new [Re: mikeschuster]
      #5651548 - 01/30/13 02:29 AM

Dark current is no different from sky glow - at least in terms of the noise model used in these discussions. You can lump all the noises that increase with exposure time - and are independent from the number of sub-exposures used - on one side - and on the other side put read noise - which is independent of total exposure time and *only* depends on the number of sub-exposures (for a given camera). Read noise is what drives the desired number of exposures down - but the other terms don't care.

There are many sources of noise and their impact is reduced by dithering, good flats, master frames, etc. But read noise is special because it doesn't accumulate due to total exposure time, but number of frames. Without read noise, the length of sub-exposure wouldn't matter at all. At least according to these noise models.

For the other noise sources - just increase the total integration time - by longer subs and/or more frames - and the SNR will improve. But other factors not included in the noise model may make the improvement less than expected.

Frank


Post Extras: Print Post   Remind Me!   Notify Moderator  
freestar8n
Post Laureate
*****

Reged: 10/12/07

Re: 5 or 30 minute exposures for narrowband? new [Re: Inverted]
      #5651557 - 01/30/13 02:52 AM

Quote:

The bias is then anything that deviates our ability to detect this rate, in a fixed way, so, this is really the "pattern noise"..




I think this is a more subtle issue and is something that is consistently handled very badly in typical amateur level write ups of ccd imaging. The only purpose of a master bias and master dark is to remove the fixed pattern noise from those two terms. A dark frame will look noisy and ugly - but if you look at many such frames, the noise looks similar. The goal is to take a really clear "picture" of that pattern noise and subtract it - and it works very well to reduce the overall noise in a sub-exposure.

Bias frames also have pattern noise, but a big reason for bias frames is to set the zero of the photon signal so the calibration process is linear. Even if the bias frames were perfectly flat at a level of 1000, with no noise, you would need to subtract them (if you are using flats and you don't have flat darks).

But a single dark frame will contain the pattern noise in the dark current - plus Poisson noise also - and that Poisson noise cannot be reduced by subtracting the master dark. But there is Poisson noise in the sky glow also, and in the nebulosity signal you are trying to image. And if you expose long enough, the SNR of the nebulosity against the other noise terms will increase and you will see improvement in the image.

Frank


Post Extras: Print Post   Remind Me!   Notify Moderator  
vpcirc
Post Laureate


Reged: 12/09/09

Loc: Merced CA
Re: 5 or 30 minute exposures for narrowband? new [Re: freestar8n]
      #5651579 - 01/30/13 03:31 AM

I wish we could get Adam Block to dispel all the science fact from science fiction on here. He is a true wiz at this (just watch one of his tutorials). I think the average beginner to intermediate user is now totally confused with all the "theory" being spread here. David has the perfect solution attend AIC and listen to the John Smith and go tell him he doesn't know what he's talking about! Hey let's call the Myth Busters!

Post Extras: Print Post   Remind Me!   Notify Moderator  
freestar8n
Post Laureate
*****

Reged: 10/12/07

Re: 5 or 30 minute exposures for narrowband? new [Re: vpcirc]
      #5651588 - 01/30/13 03:46 AM

"I wish we could get Adam Block to dispel all the science fact from science fiction on here. He is a true wiz at this (just watch one of his tutorials). I think the average beginner to intermediate user is now totally confused with all the "theory" being spread here. David has the perfect solution attend AIC and listen to the John Smith and go tell him he doesn't know what he's talking about! Hey let's call the Myth Busters!"

If you feel you understand this material and see flaws in what I'm saying - feel free to point out specific items of disagreement - preferably with explicit references. But once again - you are just casting aspersions without any actual content and disrupting what appears to be a fruitful discussion among people who are interested in learning and discussing the subject.

Moderators?

Frank


Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: freestar8n]
      #5651645 - 01/30/13 06:18 AM

"I think this is a more subtle issue and is something that is consistently handled very badly in typical amateur level write ups of ccd imaging. The only purpose of a master bias and master dark is to remove the fixed pattern noise from those two terms."

Yes, and i'm using bias a little different from what we refer to as "bias" in the hobby. Technically bias frames don't necessarily remove all fixed pattern error/noise (bias). It is a way to try to remove bias, but in practice, not all noise, fixed, or random can be dealt with. Bias frames are a way to deal with bias, but there may be other bias as well for example.


Post Extras: Print Post   Remind Me!   Notify Moderator  
vpcirc
Post Laureate


Reged: 12/09/09

Loc: Merced CA
Re: 5 or 30 minute exposures for narrowband? new [Re: freestar8n]
      #5651975 - 01/30/13 10:23 AM

Quote:

"I wish we could get Adam Block to dispel all the science fact from science fiction on here. He is a true wiz at this (just watch one of his tutorials). I think the average beginner to intermediate user is now totally confused with all the "theory" being spread here. David has the perfect solution attend AIC and listen to the John Smith and go tell him he doesn't know what he's talking about! Hey let's call the Myth Busters!"

If you feel you understand this material and see flaws in what I'm saying - feel free to point out specific items of disagreement - preferably with explicit references. But once again - you are just casting aspersions without any actual content and disrupting what appears to be a fruitful discussion among people who are interested in learning and discussing the subject.

Moderators?

Frank




Don't be so sensitive Frank, I wasn't talking about you, I was just pointing out there were several differing opinions on here as to the correct methods. Heck, I don't even know if I'm right, I'm just going by what pro's who know a lot more than me tell me, maybe some of them are wrong? I could see how you thought that since the reply was to your post, but that happens when you reply, the last poster is auto selected.


Post Extras: Print Post   Remind Me!   Notify Moderator  
JWalk
Pooh-Bah


Reged: 06/06/10

Loc: San Antonio, TX
Re: 5 or 30 minute exposures for narrowband? new [Re: vpcirc]
      #5652049 - 01/30/13 11:02 AM

I have been shooting from very dark skies for the last 6 months and I can say I've shot at all different exposure durations and I can say you should go as deep as you can without blowing out your background. This is target specific of course. I've been going 20min lately with a OSC at F3.59 and it does great.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Fogboundturtle
professor emeritus


Reged: 05/20/09

Loc: Burnaby, BC
Re: 5 or 30 minute exposures for narrowband? new [Re: JWalk]
      #5652065 - 01/30/13 11:09 AM

I am confused now

Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: Fogboundturtle]
      #5652145 - 01/30/13 11:50 AM

I'm confused as well. I've been reading through the hidden loft articles and they seem pretty darn consistent with what has been said here. A fundamental concept of modern statistics is that lots of small samples will tend to produce better estimates than fewer large samples, unless for example, there is bias introduced by the sampling process itself. As we see, there does appear to be bias introduced with regards to image "sampling" (really what we are doing). This article seems to summarize that for example: http://www.hiddenloft.com/notes/SubExposures.pdf

As I said though, I am familiar with stats, but not really so much the properties of ccds and related noise models etc... I'd like to understand those parts better, but at the moment, still don't understand why you think the experts disagree with what's been said here?


Post Extras: Print Post   Remind Me!   Notify Moderator  
freestar8n
Post Laureate
*****

Reged: 10/12/07

Re: 5 or 30 minute exposures for narrowband? new [Re: Inverted]
      #5652202 - 01/30/13 12:18 PM

The noise model discussed here and on web pages is pretty simple, and I'm not aware of anything I have said in this thread that conflicts with such web pages. The main source of confusion, which some people have and others don't, is to think that a longer optimal sub-exposure at a dark site means that your subs will *have more visible noise* at a dark site. To many people, this is absurd - and it is. But for others, it gives them the impression their subs are noisier due to the darkness of the sky allowing read noise "to be seen." This is a really bad thing to be confused about - let alone to spread around. It is true that a dark sky lets the read noise "become visible" - but only because the sky noise - and overall noise - has been reduced - leaving read noise as something worth addressing - with longer subs.

The key is to emphasize that dark sites *allow* you to go longer and get an even greater benefit not possible at a bright site. They don't *require* you to go longer.

Frank


Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: freestar8n]
      #5652265 - 01/30/13 12:53 PM

Thanks Frank, much of the discussion makes more sense now. I agree many of the published noise models seem pretty simple, I'm not sure what assumptions "hobby-level" models are making to simply things though. Also, another thing I don't quite get is, is read noise really random? or can it have an underlying pattern? Random noise can mask out our "perception" of pattern noise for example. So, if you do remove more random noise, I could see people "perceiving" pattern noise more, even though it is actually a cleaner image. Of course, you can always run a white noise algorithm and add some back if it bothers you LOL. Also then, what other sources of pattern noise could remain? So anyways though, I would agree it makes no sense that you "need" to go longer if you have less sky noise.

Post Extras: Print Post   Remind Me!   Notify Moderator  
vpcirc
Post Laureate


Reged: 12/09/09

Loc: Merced CA
Re: 5 or 30 minute exposures for narrowband? new [Re: freestar8n]
      #5652267 - 01/30/13 12:54 PM

On that Frank I would disagree with your statement. Stan Moore and Bob Smith, Don Goldman and Kevin Nelson all told me the same thing. I had to go longer to over come the read noise in my background because I was at a dark site. JWalk worked with some of my data as did Warren Keller and they can both tell you it was a significant problem especially with narrowband. I was over smoothing my images with noise reduction. I went to the 4 guys I thought would know ccd's better than I to look for a solution. They all told me the same thing, longer subs to overcome that noise. When I asked why I hadn't seen it before, that's when they told me about signal and sky glow. I had enough frames, but I needed longer exposures to improve my SNR. Maybe it's because the QE is so low on my chip, I don't know, but I do know their advice in my case helped immensely.

Post Extras: Print Post   Remind Me!   Notify Moderator  
freestar8n
Post Laureate
*****

Reged: 10/12/07

Re: 5 or 30 minute exposures for narrowband? new [Re: Inverted]
      #5652278 - 01/30/13 01:01 PM

Read noise usually shows a pattern across the ccd - and a master bias captures that pattern so it can be subtracted. But there will remain a random, Poisson component that cannot be subtracted. The pattern noise is the "fixed pattern noise" in the bias - and it can be subtracted away. The Poisson noise is the "read noise" - and it cannot. If you have a good master bias, the pattern noise is removed well, but the read noise remains. And you get one hit of read noise every time you expose an image - regardless of the exposure time.

So - if the read noise is visible in the sub-exposures, as happens at a dark site, you want to go longer so that sky glow, which is a fundamental limit at your sites, overcomes it.

If the site is so dark that you can't really do that - no problem - it still means your image will have low noise and will turn out well.

Frank


Post Extras: Print Post   Remind Me!   Notify Moderator  
freestar8n
Post Laureate
*****

Reged: 10/12/07

Re: 5 or 30 minute exposures for narrowband? new [Re: vpcirc]
      #5652286 - 01/30/13 01:06 PM

If you think that the darkness of your sky is increasing the noise in your images - I recommend turning on floodlights around your observatory. That will increase the sky glow so that read noise is no longer a factor - and allow you to use optimal sub exposures that are short.

Frank


Post Extras: Print Post   Remind Me!   Notify Moderator  
Peter in Reno
Postmaster
*****

Reged: 07/15/08

Loc: Reno, NV
Re: 5 or 30 minute exposures for narrowband? new [Re: freestar8n]
      #5652304 - 01/30/13 01:15 PM

Mike,

Can you upload a good unprocessed sample sub in FIT file format that you were having issues? Maybe include master bias, dark and flat if possible. I am having difficulty understanding your issues. I cannot imagine a brighter site would result better than your dark site. I have processed other people's images taken at a dark site and they were so easy to process because there were so much data and so little noise, light pollution or sky glow.

Like Frank said, you are still much better at a dark site than where I live.

Peter


Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: freestar8n]
      #5652323 - 01/30/13 01:26 PM

Quote:

Read noise usually shows a pattern across the ccd - and a master bias captures that pattern so it can be subtracted. But there will remain a random, Poisson component that cannot be subtracted. The pattern noise is the "fixed pattern noise" in the bias - and it can be subtracted away. The Poisson noise is the "read noise" - and it cannot. If you have a good master bias, the pattern noise is removed well, but the read noise remains. And you get one hit of read noise every time you expose an image - regardless of the exposure time.

So - if the read noise is visible in the sub-exposures, as happens at a dark site, you want to go longer so that sky glow, which is a fundamental limit at your sites, overcomes it.

If the site is so dark that you can't really do that - no problem - it still means your image will have low noise and will turn out well.

Frank




I see, so basically the sky-glo is being used to add a pedestal to the data? Also, I realize bias subtracts the pattern part of read noise, what I meant was to what extent is there a noticeable remaining after subtracting bias? I'm having trouble comprehending the relative scales. For example, if we take 16 bias frams, we'd still have 25% of the pattern noise. Is that enough to notice, if the sky glo pedestal is low enough? What if we take 64, now we're down to 12.5%, or heck, I could set the camera to just take lots of bias frames while at work, at 10,000 now I'd be down to 0.01%. (Also, I am interested if read pattern noise changes much over time - i.e. how long can I use the same bias frames, but maybe that's another thread..) If we can sufficiently eliminate pattern noise, then I think the pedestal shouldn't be perceived to be beneficial even; usually random noise just isn't that easy to see... I guess as you say though, if there is no sky noise, against a perfectly black background, maybe it is still more noticeable...


Post Extras: Print Post   Remind Me!   Notify Moderator  
Fogboundturtle
professor emeritus


Reged: 05/20/09

Loc: Burnaby, BC
Re: 5 or 30 minute exposures for narrowband? new [Re: Inverted]
      #5652376 - 01/30/13 01:47 PM

I am having hard understanding the concept. Skyglow light spectrum would have been excluded from the Narrowband filter in the first place so can it have any influence of the final result ?

Post Extras: Print Post   Remind Me!   Notify Moderator  
vpcirc
Post Laureate


Reged: 12/09/09

Loc: Merced CA
Re: 5 or 30 minute exposures for narrowband? new [Re: Peter in Reno]
      #5652398 - 01/30/13 01:58 PM

Peter it's not that a brighter site is "better" It's that it requires longer sub exposures in my case. The quality of the data is better. Peter I'll send you a private message with a link

Post Extras: Print Post   Remind Me!   Notify Moderator  
Mike Wiles
scholastic sledgehammer


Reged: 02/04/09

Loc: Goodyear, AZ
Re: 5 or 30 minute exposures for narrowband? new [Re: Fogboundturtle]
      #5652401 - 01/30/13 02:01 PM

Quote:

Skyglow light spectrum would have been excluded from the Narrowband filter in the first place so can it have any influence of the final result ?




Its influence in the final result is the lack of skyglow. Because the narrowband filter does such a good job of blocking the sky glow, it becomes nearly impossible to overcome the read noise with skyglow in a narrowband image.

Mike


Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: Fogboundturtle]
      #5652421 - 01/30/13 02:08 PM

Quote:

I am having hard understanding the concept. Skyglow light spectrum would have been excluded from the Narrowband filter in the first place so can it have any influence of the final result ?




I know looking at the background ADU in some images taken with a HA filter recently, and plugging in the numbers to the Starizona CCD calculator, I was still getting a recommended exposure time of about 10 minutes. This was taken from a very light polluted site with a full moon though.

Looking around I did find this

from the sbig site here:
http://sbig.impulse.net/dss7/dss7.htm

The dark site is much darker, but if not zero, presumably, given time, some skyglow will get through as it appears pretty broadband.

I don't know to what extent. An image can't have a lower SNR by removing noise though...


Post Extras: Print Post   Remind Me!   Notify Moderator  
Fogboundturtle
professor emeritus


Reged: 05/20/09

Loc: Burnaby, BC
Re: 5 or 30 minute exposures for narrowband? new [Re: Mike Wiles]
      #5652425 - 01/30/13 02:09 PM

Quote:

Quote:

Skyglow light spectrum would have been excluded from the Narrowband filter in the first place so can it have any influence of the final result ?




Its influence in the final result is the lack of skyglow. Because the narrowband filter does such a good job of blocking the sky glow, it becomes nearly impossible to overcome the read noise with skyglow in a narrowband image.

Mike




are you suggesting it would be preferable to capture with 7mn instead of 3nm filter because it would than be easier to offset the read noise?


Post Extras: Print Post   Remind Me!   Notify Moderator  
shams42
Pooh-Bah
*****

Reged: 01/05/09

Loc: Kingsport, TN
Re: 5 or 30 minute exposures for narrowband? new [Re: Inverted]
      #5652426 - 01/30/13 02:09 PM

Quote:

Also, I realize bias subtracts the pattern part of read noise, what I meant was to what extent is there a noticeable remaining after subtracting bias?




You have to distinguish between noise and unwanted signal. The noise is random and cannot be subtracted out. Some aspects of unwanted signal, including any fixed pattern that occurs on readout as well as dark current, can be removed. Indeed, calibration with a master dark frame removes both of those aspects of unwanted signal. It is desirable to use an adequate number of exposures for the master dark so that both of these aspects of unwanted signal are well-sampled from a statistical point of view.

The noise components remain. Noise can't be subtracted.

As Frank said, the a master bias frame has two functions. First, to represent any unwanted signal resulting from reading the CCD (often called "pattern noise"). This is only really important if you aren't using dark subtraction. It's really only useful for removing any readout pattern from the flats. Second is to represent the pedestal added to each pixel's value. Knowing this pedestal value is crucial for calibrating the master flat(s). Remember the lights will be divided by a normalized flat, and if the pedestal is not removed the division will not be correct.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Alph
Carpal Tunnel


Reged: 11/23/06

Loc: Melmac
Re: 5 or 30 minute exposures for narrowband? new [Re: Fogboundturtle]
      #5652441 - 01/30/13 02:20 PM

Quote:

I am having hard understanding the concept. Skyglow light spectrum would have been excluded from the Narrowband filter in the first place so can it have any influence of the final result ?



If the background noise is not much higher than the readout noise, then stacking is not a very effective way to increase the S/N.


Post Extras: Print Post   Remind Me!   Notify Moderator  
vpcirc
Post Laureate


Reged: 12/09/09

Loc: Merced CA
Re: 5 or 30 minute exposures for narrowband? new [Re: Fogboundturtle]
      #5652459 - 01/30/13 02:29 PM

Quote:

Quote:

Quote:

Skyglow light spectrum would have been excluded from the Narrowband filter in the first place so can it have any influence of the final result ?




Its influence in the final result is the lack of skyglow. Because the narrowband filter does such a good job of blocking the sky glow, it becomes nearly impossible to overcome the read noise with skyglow in a narrowband image.

Mike




are you suggesting it would be preferable to capture with 7mn instead of 3nm filter because it would than be easier to offset the read noise?




Going to a a 3nm compared to say a 7 will give you greater contrast from what Don told me. Sky glow is signal and from what I understand can actually hide noise because the SNR changes with "that signal" It's also a limiting factor and prevents some of the signal you do want. It's not that the noise isn't still there, it's that the "signal from sky glow hides some of it so it's not as noticeable. In my case at a dark site, I needed more signal to help overcome the read noise. At least that's the way I understood it. I've only been doing narrowband for a year so I'm still learning a lot. I don't profess to understand any of this, I'm only sharing what I was told.


Post Extras: Print Post   Remind Me!   Notify Moderator  
mikeschuster
super member


Reged: 08/25/11

Loc: SF Bay area
Re: 5 or 30 minute exposures for narrowband? new [Re: Alph]
      #5652460 - 01/30/13 02:30 PM

Frank,
Treating dark noise as sky due to its time dependence is a nice idea. But on the other hand it is a defect that would not be present in a "perfect" camera, whereas sky remains unchanged. So maybe I will continue to treat dark noise as a defect to be overcome. And LOL about floodlights. So true!
Thanks,
Mike


Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: shams42]
      #5652468 - 01/30/13 02:35 PM

Quote:

You have to distinguish between noise and unwanted signal.




The definition of noise is unwanted signal. In statistics there is random noise and bias ("pattern noise"). Sorry, this is semantic, but if we're not speaking the same language we're going to misunderstand each other...


Quote:


The noise components remain. Noise can't be subtracted.





Right, random noise remains and this is can be"averaged out" due to properties of the Central Limit Theorem (or if you like, the less rigorously defined "law of Large Numbers").


Quote:


As Frank said, the a master bias frame has two functions. First, to represent any unwanted signal resulting from reading the CCD (often called "pattern noise"). This is only really important if you aren't using dark subtraction. It's really only useful for removing any readout pattern from the flats. Second is to represent the pedestal added to each pixel's value. Knowing this pedestal value is crucial for calibrating the master flat(s). Remember the lights will be divided by a normalized flat, and if the pedestal is not removed the division will not be correct.




That's interesting, the flat does capture bias as well, as does the darks. However, as we saw earlier, remaining noise follows the rule 1/sqrt(n), and I can take bias frames a lot faster than darks. There are diminishing returns, but clearly the n=16 rule doesn't make much sense, as 25% of the noise is still remaining, but it's true the returns diminish substantially after that, to cut that in half I need to go to 64 for example. However, bias frames being easy to take, I could take 10000 and drop the remaining noise down to 0.01%. of the original noise. And really, even if I don't take that many I can still drop the noise down pretty substantially from n=16. At n=400, I'd be at 0.05% which is a heck of a lot less proportionally than 25% at n=16. Taking 400 darks or flats seems like a royal PITA, but taking 400 bias frames is no big deal at all. So, I'm not sure I agree that "there is no benefit to bias frames if taking darks. Unless I'm missing something, totally possible... The big question seems to be under "what" circumstances, does "what" level of noise become perceivable?


Post Extras: Print Post   Remind Me!   Notify Moderator  
Alph
Carpal Tunnel


Reged: 11/23/06

Loc: Melmac
Re: 5 or 30 minute exposures for narrowband? new [Re: mikeschuster]
      #5652488 - 01/30/13 02:45 PM

Quote:

Treating dark noise as sky due to its time dependence is a nice idea



It is not just an idea. You have to subtract skyglow (and dark current) from the captured signal in order to calculate the S/N. Mike Newberry in the above-mentioned article did fold skyglow and dark current into the background noise.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: Alph]
      #5652505 - 01/30/13 02:52 PM

Quote:


If the background noise is not much higher than the readout noise, then stacking is not a very effective way to increase the S/N.




I feel like we need a table of random noise and fixed noise Another for example would be "seeing". Is atmospheric distortion random or pattern noise? I guess patter noise, as that would seem to explain adaptive optics. It seems there are possibly lots of other sources of random noise though, although, unclear that they significantly contribute, but that may be relative to the pedestal. And as mentioned, dithering, sigma reject stacking, dropping bad subs can still help eliminate sources of pattern noise too.


Post Extras: Print Post   Remind Me!   Notify Moderator  
mikeschuster
super member


Reged: 08/25/11

Loc: SF Bay area
Re: 5 or 30 minute exposures for narrowband? new [Re: Inverted]
      #5652518 - 01/30/13 03:09 PM

Alph, I am doing the same thing. I just choose to think of dark current as a camera defect rather than a sky source.
Mike


Post Extras: Print Post   Remind Me!   Notify Moderator  
shams42
Pooh-Bah
*****

Reged: 01/05/09

Loc: Kingsport, TN
Re: 5 or 30 minute exposures for narrowband? new [Re: Inverted]
      #5652537 - 01/30/13 03:23 PM

Quote:

Quote:

You have to distinguish between noise and unwanted signal.




The definition of noise is unwanted signal. In statistics there is random noise and bias ("pattern noise"). Sorry, this is semantic, but if we're not speaking the same language we're going to misunderstand each other...




When I say "signal", I mean that aspect that is fixed, non-stochastic, always the same each time it is measured.

When I say "noise", I mean that aspect that is random, the variable and unpredictable aspect, different each time it is measured.

The wanted signal comes from stars and DSO. Unwanted signal comes from the CCD and camera electronics. Among other sources.

Each signal brings with it shot noise. Shot noise from the object, shot noise from the background, etc. Read noise not a "tag-along" shot noise but errors that occur when reading the CCD.

Are you using the terms differently?

I hear lots of people saying that they "use darks to remove the thermal noise" or that "bias frames remove read noise." No -- bias and darks can remove certain aspects of unwanted signal but not noise. Distinguishing between true noise and unwanted signal is clarifying IMO.

Edited by shams42 (01/30/13 03:43 PM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
pfile
Post Laureate


Reged: 06/14/09

Re: 5 or 30 minute exposures for narrowband? new [Re: shams42]
      #5652608 - 01/30/13 03:54 PM

everyone should own a copy of HAIP. reading the first few chapters of that book would answer all the above questions.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: shams42]
      #5652626 - 01/30/13 04:00 PM

Quote:


Are you using the terms differently?





Signal
http://www.statistics.com/index.php?page=glossary&term_id=844

Noise
http://www.statistics.com/index.php?page=glossary&term_id=805

Bias
http://www.statistics.com/index.php?page=glossary&term_id=717


Really though, I think the "signal" definition given above, is bit vague, without considering the broader context.
In practice, a more useful way of defining, in the context of AP, I think, would be to say "it is the asymptotic mean rate of photons hitting the collector, given an infinite number of unbiased samples, collected over the integration time." The estimator of this will deviate due to noise and bias as defined above.

Usually we define noise more as anything unwanted, but random and bias as anything unwanted and not random.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Peter in Reno
Postmaster
*****

Reged: 07/15/08

Loc: Reno, NV
Re: 5 or 30 minute exposures for narrowband? new [Re: Inverted]
      #5652648 - 01/30/13 04:10 PM

Hi Mike,

Thanks for uploading the images. I think your NB images look similar to my NB images and they appear to be normal to me. If you took enough subs, do proper calibration of each sub, and stack them, the end result will look much better.

I think it's tricky to evaluate a single NB sub because the signal is very weak. When stretching a single NB sub heavily to reveal DSO, it appears to look bad because more low ADU bad pixels show up. Calibration and stacking will take care of this.

My bottom line is I think your subs look normal to me.

Peter


Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: pfile]
      #5652708 - 01/30/13 04:33 PM

Quote:

everyone should own a copy of HAIP. reading the first few chapters of that book would answer all the above questions.




Thanks! I've been out of the hobby for a while though and am not familiar with this. What does it stand for?


Post Extras: Print Post   Remind Me!   Notify Moderator  
Mike7Mak
Pooh-Bah
*****

Reged: 12/07/11

Loc: New York
Re: 5 or 30 minute exposures for narrowband? new [Re: Inverted]
      #5652852 - 01/30/13 05:43 PM

Quote:

Quote:

everyone should own a copy of HAIP. reading the first few chapters of that book would answer all the above questions.




Thanks! I've been out of the hobby for a while though and am not familiar with this. What does it stand for?




That would be this book...

http://www.willbell.com/aip/index.htm

...which also includes a fairly complex image processing program, the poor man's Pixinsight, if you will. (Never actually seen PixInsight so I'm sorta guessing on that.) Whether you dive into the program or not the book is well worth reading.

The problem, for me anyway, is both the book and the program rely extensively on math that quickly makes my eyes glaze over. You can skim the equations in the book and still learn a lot but the program itself expects you to know the math functions and how to enter three decimal place numerical parameters for all but the most basic processing steps.

I think I gain more real understanding from plain language conceptual discussions like this one.

Thanks, btw, for everyone's contributions to this topic, this 'beginner/intermediate' imager is less confused as a result.


Post Extras: Print Post   Remind Me!   Notify Moderator  
freestar8n
Post Laureate
*****

Reged: 10/12/07

Re: 5 or 30 minute exposures for narrowband? new [Re: shams42]
      #5652970 - 01/30/13 06:33 PM

Quote:

I hear lots of people saying that they "use darks to remove the thermal noise" or that "bias frames remove read noise." No -- bias and darks can remove certain aspects of unwanted signal but not noise. Distinguishing between true noise and unwanted signal is clarifying IMO.




Actually this interpretation of "noise" and "signal" is where I depart from typical amateur write ups and instead stick to the usage more commonly found in professional astronomical and engineering literature. I often hear amateurs insist that "noise must be random" - but I don't know where that idea comes from - and I think it makes it very hard to talk about - and understand - why people do all this calibration stuff to make their images look better.

I'm aware that many amateur web sites and even some amateur books on ccd imaging will use that "random" interpretation of noise - but I don't place much value in that if it is at odds with textbooks and journal articles in the field. Even worse - it goes against what I consider to be common sense.

I could cite many examples but here are two:

"Fixed pattern noise is removed from images by a technique called flat fielding, where a computer adjusts pixel sensitivites to be equal." Photon Transfer, James Janesick, SPIE 2007.

That is a casual summary of flat fielding in a well-regarded graduate level text on imaging sensors that has no problem talking about the removal of fixed pattern noise.

Many texts on noise and statistics don't even bother defining what noise is - because it all depends on context. My example is, if you are on the couch watching "the game" and someone starts describing a chore you should do - the game is the signal and the voice is noise. If instead the game is boring and the voice is offering a beverage - the voice becomes signal and the game is noise. Nothing about randomness here - just a thing that you want is being obfuscated by a thing you don't want.

One text that I like is Probability, Statistical Optics, and Data Testing by Frieden, in which he does attempt to give "A Definition of Noise." To me, he sums it up nicely and in a way that applies directly to the removal of pattern noise:

"This is a definition of noise which also shows its arbitrariness: the received message that is independent of one set of events may be dependent upon and describe a different set. For the latter events, the messages are not noise and do contain finite information."

Later, he says:

"The concept of noise is always defined in a specific context. As a consequence, what is considered noise in one case may be considered "signal" in another. One man's weed is another man's wildflower."

So - why do you want to subtract a master dark? To reduce the noise in it - and that noise is Fixed Pattern Noise. Flats are applied to correct vignetting - but also to make the pixel response more uniform - and reduce the noise.

So - sure - you can reduce and even remove noise, and noise need not be random. That's why there are terms like "random noise" and "noise reduction." And that's why images look better after a dark subtraction.

Frank


Post Extras: Print Post   Remind Me!   Notify Moderator  
vpcirc
Post Laureate


Reged: 12/09/09

Loc: Merced CA
Re: 5 or 30 minute exposures for narrowband? new [Re: freestar8n]
      #5652979 - 01/30/13 06:38 PM

We need Richard Crisp to chime in, he's a CCD engineer.

Post Extras: Print Post   Remind Me!   Notify Moderator  
korborh
scholastic sledgehammer
*****

Reged: 01/29/11

Loc: Arizona
Re: 5 or 30 minute exposures for narrowband? new [Re: freestar8n]
      #5653030 - 01/30/13 07:03 PM

Quote:

I often hear amateurs insist that "noise must be random" - but I don't know where that idea comes from




I believe this confusion has been disseminated by one very popular book in amateur CCD imaging. Never heard this association of noise with randomness in all my years in EE.
Noise is unwanted signal, random or otherwise.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Mike7Mak
Pooh-Bah
*****

Reged: 12/07/11

Loc: New York
Re: 5 or 30 minute exposures for narrowband? new [Re: vpcirc]
      #5653039 - 01/30/13 07:11 PM

Quote:

We need Richard Crisp to chime in, he's a CCD engineer.



Good grief, enough with the name-dropping already, it's becoming embarrassing. Address the topic.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: vpcirc]
      #5653057 - 01/30/13 07:25 PM

Quote:

I often hear amateurs insist that "noise must be random"




Quote:

Many texts on noise and statistics don't even bother defining what noise is




Interdiscipline terminology is always confusing. In population statistics, we do define noise, but call it "random error" which for all intents and purposes is really synonymous with "variance". Technically, you could have systematic error, but if this still exists once we get into sampling (which we always get into) then it become "bias". So, really we usually just talk about random error/variance and bias. Generally I do not think we define "noise" though unless trying to communicate with other disciplines such as engineering LOL. Really, I've never heard a coworker mention"noise" except casually. When we do define noise though, it really is random error/variance. So, from a population stats view (and sampling stats usually I think), in practice, noise is really random.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: Mike7Mak]
      #5653060 - 01/30/13 07:26 PM

Quote:


That would be this book...

http://www.willbell.com/aip/index.htm

...which also includes a fairly complex image processing program, the poor man's Pixinsight, if you will. (Never actually seen PixInsight so I'm sorta guessing on that.) Whether you dive into the program or not the book is well worth reading.

The problem, for me anyway, is both the book and the program rely extensively on math that quickly makes my eyes glaze over. You can skim the equations in the book and still learn a lot but the program itself expects you to know the math functions and how to enter three decimal place numerical parameters for all but the most basic processing steps.

I think I gain more real understanding from plain language conceptual discussions like this one.

Thanks, btw, for everyone's contributions to this topic, this 'beginner/intermediate' imager is less confused as a result.




Thanks


Post Extras: Print Post   Remind Me!   Notify Moderator  
mikeschuster
super member


Reged: 08/25/11

Loc: SF Bay area
Re: 5 or 30 minute exposures for narrowband? new [Re: Inverted]
      #5653148 - 01/30/13 08:24 PM

FPN definitely is interesting. If you walk pixel by pixel across the detector, pixel sensitivities do change, and the change does seem "random", like "noise". But on the other hand these sensitivities remain "fixed" across time, so it is also "signal" that can be removed. IMO very strange. No wonder it is a source of confusion.
Mike


Post Extras: Print Post   Remind Me!   Notify Moderator  
vpcirc
Post Laureate


Reged: 12/09/09

Loc: Merced CA
Re: 5 or 30 minute exposures for narrowband? new [Re: Mike7Mak]
      #5653316 - 01/30/13 10:07 PM

Quote:

Quote:

We need Richard Crisp to chime in, he's a CCD engineer.



Good grief, enough with the name-dropping already, it's becoming embarrassing. Address the topic.




I was kidding... This conversation has gotten well beyond my level of interest. I'm neither an engineer or science expert. I'll stick with the guys who make the prettiest images tell me. Longer is better from what I've learned, but people are free to image how ever they want and if they're happy with the results that's all that really matters!


Post Extras: Print Post   Remind Me!   Notify Moderator  
orlyandico
Postmaster
*****

Reged: 08/10/09

Loc: Singapore
Re: 5 or 30 minute exposures for narrowband? new [Re: vpcirc]
      #5653494 - 01/31/13 01:09 AM

i just did some 20 minute narrowband subs last night.

compared them to last year's effort on the same subject (horsehead) where i used 10 minute subs.

the 20 minute ones are definitely cleaner.


Post Extras: Print Post   Remind Me!   Notify Moderator  
freestar8n
Post Laureate
*****

Reged: 10/12/07

Re: 5 or 30 minute exposures for narrowband? new [Re: orlyandico]
      #5653566 - 01/31/13 03:00 AM

Quote:

the 20 minute ones are definitely cleaner.




Hi-

Just to be clear - even if you didn't have any read noise, individual subs at 20 would look better than 10 because SNR increases with total exposure time. In terms of asking "how long should my subs be for a 3 hour total exposure time" you would need to compare the stacked versions of both images with the same total time. That is what all these "sub exposure calculators" do - or many of them anyway.

For you, or anyone, the benefit of 20 compared to 10 may be big, or negligible - it all depends on how the read noise in each sub compares to the other noise terms. Dark site, high read noise camera, slow optics - all point to expecting a win going to 20, but I don't know what you used or where. If you did both imaging sessions with the same equipment it would be nice to compare the two final results with the same total time.

If you used a fast system like hyperstar and compared a 10 minute sub to a 20 minute one - both may look great already - as long as they didn't saturate - because with such a fast system even a single sub could achieve high snr.

Frank


Post Extras: Print Post   Remind Me!   Notify Moderator  
freestar8n
Post Laureate
*****

Reged: 10/12/07

Re: 5 or 30 minute exposures for narrowband? new [Re: Inverted]
      #5653569 - 01/31/13 03:15 AM

Quote:

So, from a population stats view (and sampling stats usually I think), in practice, noise is really random.




Yes - I can believe that some disciplines would want a term like noise to be used with a specific meaning. Same for a word like, "significance." But in physics and engineering it is usually used more casually and in context. But in some ways fixed pattern noise *is* random, or it can be. It is random (ish) across the pixels - but repeats in time. If a random sequence appears once it looks totally random. But if it repeats then you can learn what it is and remove it. That is the idea behind capturing a master dark and subracting it.

But my main point is that graduate level texts on ccd's and image processing, and journal articles, refer to pattern noise removal - and I go by that. Plus, I think it helps people understand why the image improves after dark subtract. If you tell people the image looks better but the noise hasn't actually decreased - I think that is confusing.

On this topic, the master bias usually has an ugly pattern to it - and that ugly pattern can be subtracted away from each sub, but a random Poisson noise will remain - and that is the read noise that you are stuck with in each calibrated sub.

Frank


Post Extras: Print Post   Remind Me!   Notify Moderator  
vpcirc
Post Laureate


Reged: 12/09/09

Loc: Merced CA
Re: 5 or 30 minute exposures for narrowband? new [Re: orlyandico]
      #5653642 - 01/31/13 06:00 AM

Quote:

i just did some 20 minute narrowband subs last night.

compared them to last year's effort on the same subject (horsehead) where i used 10 minute subs.

the 20 minute ones are definitely cleaner.




That's exactly what the a U.S. filter manufacture told me would happen!


Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: vpcirc]
      #5653775 - 01/31/13 08:11 AM

Quote:

I'll stick with the guys who make the prettiest images tell me.




Just to point out, there is a difference between "prettiest" and "most accurate". I think most would agree that longer is better, as long as conditions, equipment and various properties of the target, such as brightness, allow. The issues was saying that you "need" to go longer. From the website you posted, they showed under broadband conditions that, you can go shorter, without much extra data and this may be preferable even. They did not give a dark site example, there doesn't seem to be anything in the formulas that suggests you "need" to go longer, it does suggest you can go longer. If someone has some data from a dark site, it would be fun to plug in though. And certainly if there are experts saying you "need" to go longer, it would be great to hear from them.

Anyways, then the subtle issue is that from a statistical point of view, I think the formulas should change anyways, if pattern noise induced by reading data off the ccd becomes low enough. Also to point out, that could happen via technology, or via extreme read bias calibration, such as hundreds, thousands, or even tens of thousands of bias frames. That would be an interesting experiment, but if it is fixed pattern noise, should be pretty effective, when combined with a large amount of image capture exposures.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: freestar8n]
      #5653782 - 01/31/13 08:21 AM

Quote:


But my main point is that graduate level texts on ccd's and image processing, and journal articles, refer to pattern noise removal - and I go by that. Plus, I think it helps people understand why the image improves after dark subtract. If you tell people the image looks better but the noise hasn't actually decreased - I think that is confusing.




I agree, with the caviar of the next point below.

Quote:

On this topic, the master bias usually has an ugly pattern to it - and that ugly pattern can be subtracted away from each sub, but a random Poisson noise will remain - and that is the read noise that you are stuck with in each calibrated sub.

Frank




The reason we don't really worry about pattern noise, is because as the samples reach asymptotic convergence, the data becomes more normally distributed and further, averages out to zero. It is only really an issue for individual samples, not averages of a large number of samples. So, if you can effectivly remove bias from the sampling technique, then by all means shoot more, smaller images, you will actually get better pictures. We probably aren't at that point yet, but that may be changing with technology and could perhapse be sped up with more aggressive calibration techniques. As we are already seeing 3e- cameras on the hobby market, starting at $1000, that suggests we could be heading in the direction of a paradigm shift (or expansion as more noises cameras are probably going to stick around too).

Edit: note fixed "3e-"

Edited by Inverted (01/31/13 09:13 AM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
vpcirc
Post Laureate


Reged: 12/09/09

Loc: Merced CA
Re: 5 or 30 minute exposures for narrowband? new [Re: Inverted]
      #5653790 - 01/31/13 08:28 AM

I'll repeat what I was told. "Shorter exposures will not pick up the faint nebulosity, if you have a doubt, compare equal amounts of time on the horsehead and see what happens" this is referring to narrowband. I don't think most top imagers get involved in CN forum discussions. I can only speak for myself and listening to the advice I received they were correct. I'm sure it's just my camera and my situation and in no way applies to anyone else. In my case I need to waste my time shooting longer narrowband images. I'm sure many others don't need to undertake such a frivolous approach.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: vpcirc]
      #5653834 - 01/31/13 08:50 AM

Quote:

I'll repeat what I was told. "Shorter exposures will not pick up the faint nebulosity, if you have a doubt, compare equal amounts of time on the horsehead and see what happens"




Of course, but It's because we can't see it, until the SNR is high enough, not because it isn't actually there is a short exposure. Photons are hitting the sensor very quickly, so, there is a signal even in a super short exposure. The SNR though may be near zero though. The formulas on the website you linked, seem to agree that we just need to expose long enough to get the weakest signal over the noise pedestal, sufficiently to be discernible from the noise. With a theoretical low enough noise pedestal, that could be a fraction of a second, in practice it is clearly a lot longer due to relatively high noise levels of our equipment....


Post Extras: Print Post   Remind Me!   Notify Moderator  
freestar8n
Post Laureate
*****

Reged: 10/12/07

Re: 5 or 30 minute exposures for narrowband? new [Re: Inverted]
      #5654056 - 01/31/13 10:43 AM

Not really. The message of the web sites and calculators is to expose long enough for other noise sources in the sub to grow larger than the read noise in the sub. If the sub exposure is noisy due to sky glow then longer subs won't help at all. All you can do is accumulate more total exposure time. But if your sub exposure is noisy due mainly to read noise, then longer subs will help.

That is what the various web sites and calculators are trying to convey - and they treat read noise specially. You can't just tell by how noisy a sub is that it should have been longer - unless you know the noise you see is due in large part to read noise.

That's why people should know the gain, read noise, and sky glow of their equipment and site - so they can plug in the numbers. If the numbers end up being longer than practical, just go shorter and accumulate as many subs as you can.

Frank


Post Extras: Print Post   Remind Me!   Notify Moderator  
Peter in Reno
Postmaster
*****

Reged: 07/15/08

Loc: Reno, NV
Re: 5 or 30 minute exposures for narrowband? new [Re: Inverted]
      #5654082 - 01/31/13 10:57 AM

This has been an interesting thread. It gave me some ideas of how long each sub exposure is too long. Remember this thread is about Narrowband, not broadband. NB filters are insanely narrow that does a great job of blocking unwanted signal (light pollution and moon). Forget about read noise or any other noise to worry about. I pretty much understand all this noise talks in this thread so let's forget about noise for now.

I've been imaging with OSC camera for three years before migrating to Atik 460EX mono last September. My new mono camera has extemely low read noise. Last September/October I captured Bubble Nebula with LRGB, Ha, Oiii and Sii filters which took me two months due to bad weather. I went back to look at the data. I looked at the histogram for each filter to see where the left edge of the graph is. Luminance was the worst because the left side of the graph shifted too far to the right. That tells me the exposure was too long due to heavy light pollution. RGB were much better but the left side was shifted a little bit too far to the right. All LRGB sub-exposures were at 10 minutes each. This will give me ideas of maximum sub-exposures for each filter in my area.

NB filters are a different ball game. I started with Ha at 15 minutes and the left side of histogram was almost all the way to the left. The minimum ADU was in between 200 and 300 and pretty much the same as minimum ADU of Bias. Then Oiii was captured at 15 minutes subs and once again the left side of graph was like Ha. Finally I captured Sii at 30 minutes and the left side of histogram remained the same as Ha and Oiii. It tells me that NB filters allows you to image reeeeeeaaaaaallllly long sub-exposures especially under heavy light pollution where I live.

So I believe the bottom line is to pay attention to histogram to make sure the left side of the graph didn't shift too far to the right. The more it shifts to the right the greater the sky glow you have. You should know camera's minimum ADU of your bias. So if your camera's Bias minimum ADU is 1500 and the minimum background of your light sub is around 1500 to 1700 (wild guess), then you may be okay with the sub-exposures. Keep taking longer sub-exposures until the left side of the histogram graph starts to shift too far to the right.

Once you have established the maximum sub-exposure times for LRGB filters, then use this and take as many subs as you can. For NB, take as long as your mount can handle and take as many subs as well. Low noise cameras are nice for NB imaging. My camera's (Sony CCD) minimum bias ADU is between 200 and 300. Kodak CCD minimum Bias ADU can be pretty high of around 1500. NB imaging with high read noise can be difficult.

If you have a better or easier suggestion to determine the maximum sub-exposure times before it hits the sky noise, please feel free to say it. I don't think you have to worry about the maximum sub-exposures for NB filters because they do a fantastic job of blocking unwanted signals like light pollution or moon light. Probably maximum bandwidth for NB filters should be 5nm. 3nm filters are nice but quite a bit more expensive than 5nm.

See my Bubble Nebula images in "Peter's Galleries" in my signature. Look under "Nebulae". It shows NB, Bi-color, and RGB images of Bubble Nebula.

Peter

Edited by Peter in Reno (01/31/13 11:00 AM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
shams42
Pooh-Bah
*****

Reged: 01/05/09

Loc: Kingsport, TN
Re: 5 or 30 minute exposures for narrowband? new [Re: freestar8n]
      #5654087 - 01/31/13 11:00 AM

Quote:

Yes - I can believe that some disciplines would want a term like noise to be used with a specific meaning. Same for a word like, "significance." But in physics and engineering it is usually used more casually and in context. But in some ways fixed pattern noise *is* random, or it can be. It is random (ish) across the pixels - but repeats in time. If a random sequence appears once it looks totally random. But if it repeats then you can learn what it is and remove it. That is the idea behind capturing a master dark and subracting it.





Very interesting point, Frank. I had been considering only the temporal aspect -- the fixed pattern -- not the positional or pixel-to-pixel variability that does indeed look "random."

Quote:


But my main point is that graduate level texts on ccd's and image processing, and journal articles, refer to pattern noise removal - and I go by that. Plus, I think it helps people understand why the image improves after dark subtract. If you tell people the image looks better but the noise hasn't actually decreased - I think that is confusing.




See, my position is that this is MORE confusing that attempting to distinguish between "unwanted signal" and "random noise" as I have proposed.

The fact is that people who own CCDs with very low dark current (most Sony CCDs) should NOT calibrate by subtracting a master dark. Why? Because although subtracting the master dark may improve the aesthetic quality of the image by removing hot pixels, the random noise in the calibrated sub actually increases! Instead they should use bad pixel mapping, an alternative technique which can remove hot pixels without adding noise.

It's that seeming paradox that I found so confusing -- that by subtracting noise (what I call "unwanted signal"), you actully add noise.

Quote:


On this topic, the master bias usually has an ugly pattern to it - and that ugly pattern can be subtracted away from each sub, but a random Poisson noise will remain - and that is the read noise that you are stuck with in each calibrated sub.




But here, aren't you distinguishing "noise" from "random Poisson noise?" So in practice, it sounds like you too make some distinction between systematic or repeatable sources of noise which can be removed via calibration and random noise which cannot. We only seem to differ in that I call the former "unwanted signal."

I think this is largely semantics, and I will no longer belabor the point. I respect your desire to maintain consistency with professional references and discussion regarding CCDs, and there is obvious value in doing so.

I just think that defining our terms such that subtracting noise can add it makes the matter more confusing than it needs to be.

Edited by shams42 (01/31/13 11:20 AM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
Peter in Reno
Postmaster
*****

Reged: 07/15/08

Loc: Reno, NV
Re: 5 or 30 minute exposures for narrowband? new [Re: shams42]
      #5654104 - 01/31/13 11:09 AM

I also use Bad Pixel Mapping with my Atik 460EX low noise camera. I get better results than dark subtraction.

Peter


Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: freestar8n]
      #5654114 - 01/31/13 11:17 AM

Quote:

Not really. The message of the web sites...




I was referring to mathematical agreement to the contextual matter being discusses at that point. The formulas don't appear to disagree with that. How people interpret that and perceive it is another matter, based other various practical considerations and the expected performance and tolerance equipment used.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: shams42]
      #5654155 - 01/31/13 11:35 AM

Quote:


I just think that defining our terms such that subtracting noise can add it makes the matter more confusing than it needs to be.




I agree, that's we why like to stick with "eliminating bias" and "reducing variance". Nice and parsimonious. Then all these other fields go and muck everything up though


Post Extras: Print Post   Remind Me!   Notify Moderator  
Alph
Carpal Tunnel


Reged: 11/23/06

Loc: Melmac
Re: 5 or 30 minute exposures for narrowband? new [Re: Inverted]
      #5654272 - 01/31/13 12:52 PM

Quote:

that's we why like to stick with "eliminating bias" and "reducing variance". Nice and parsimonious.



Yes I agree with you. It is simple and precise. Unfortunately most engineers have been taught dumbed down mathematics and they are the ones who came up with that terminology. It is inconsistent and non-intuitive. To be consistent they should say readout bias frame instead of just bias frame and dark current bias frame instead of just master dark frame.


Post Extras: Print Post   Remind Me!   Notify Moderator  
freestar8n
Post Laureate
*****

Reged: 10/12/07

Re: 5 or 30 minute exposures for narrowband? new [Re: shams42]
      #5654544 - 01/31/13 03:30 PM

Quote:

But here, aren't you distinguishing "noise" from "random Poisson noise?" So in practice, it sounds like you too make some distinction between systematic or repeatable sources of noise which can be removed via calibration and random noise which cannot. We only seem to differ in that I call the former "unwanted signal."




It's not just me - it's standard ccd noise models as described in e.g. Janesick and Holst/Lomheim - in which many other types of noise are described.

You can even have moving pattern noise - and you can subtract it, if you can model it, - as described here. The noise has a recognizable pattern that can be modeled and removed - even though it changes in each frame. For that you could not use a master bias and you would want to deal with it in each frame individually. It's still a form of pattern noise removal.

Pattern noise is treated as a separate noise term from the read noise - and both are found in a bias frame. The master bias captures an image of the pattern noise. Since the master bias will not be a perfect "image" of the noise, when you subtract it from a frame you greatly reduce the pattern noise - but you slightly increase the Poisson noise present in a light. If the ccd were uniform and you had no pattern noise, you would be better off not using a master bias at all - and simply subtract the mean bias value across the entire ccd. It's the presence of pattern noise in the bias and darks that motivates the subtraction - and it has a net benefit by reducing the pattern noise while slightly increasing the Poisson noise in each light.

There is separate pattern noise in the bias, the dark, and the flat.

I think the term "bias" is used because it offsets the pixel response from zero. The amount of this offset is different for each pixel - that's the pattern noise in the bias. In these models, each pixel is assumed to generate the same amount of read noise on top of that with the same Poisson distribution. In fact the distribution may not be the same for each pixel - and that is one way the noise model might not work perfectly.

Frank


Post Extras: Print Post   Remind Me!   Notify Moderator  
cn register 5
scholastic sledgehammer


Reged: 12/26/12

Re: 5 or 30 minute exposures for narrowband? new [Re: orlyandico]
      #5654549 - 01/31/13 03:31 PM

Quote:

i just did some 20 minute narrowband subs last night.

compared them to last year's effort on the same subject (horsehead) where i used 10 minute subs.

the 20 minute ones are definitely cleaner.



It would be interesting to compare 2 x 10 minute exposures with one 20 minute exposure, that way you have the same exposure and so the same amount of data from the sky. What's different is one has two helpings of read noise and the other has one.

What does the difference look like then?

Chris


Post Extras: Print Post   Remind Me!   Notify Moderator  
Calypte
Pooh-Bah


Reged: 03/20/07

Loc: Anza, California
Re: 5 or 30 minute exposures for narrowband? new [Re: cn register 5]
      #5654583 - 01/31/13 03:53 PM

Wodaski's The New CCD Astronomy deals with these exact questions. I'm surprised that some people treat this as a deep mystery, when in fact it's already been answered. No, a series of short exposures don't equal longer exposures of the same total time. You may have good reasons for choosing shorter subs, and you may achieve a very good final result, but it's not the same. Just my humble opinion.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Peter in Reno
Postmaster
*****

Reged: 07/15/08

Loc: Reno, NV
Re: 5 or 30 minute exposures for narrowband? new [Re: Calypte]
      #5654624 - 01/31/13 04:21 PM

A good read is available at:

http://deepskystacker.free.fr/english/index.html

Click on "How to create better images" link at left side. It's a very well and easy to read document. Here is a snippet:

"Are 100 x 1 minute and 10x10 minutes giving the same result?

Yes when considering the SNR but definitely No when considering the final result.

The difference between a 10 minutes exposure and a 1 minute exposure is that the SNR in the 10 minutes exposure is 3.16 higher than in 1 minute exposure.

Thus you will get the same SNR if you combine 10 light frames of 10 minutes or 100 light frames of 1 minute. However you will probably not have the same signal (the interesting part). Simply put you will only get a signal if your exposure is long enough to catch some photons on most of the light frames so that the signal is not considered as noise.

For example for a very faint nebula you might get a few photons every 10 minutes. If you are using 10 minutes exposures, you will have captured photons on each of your light frames and when combined the signal will be strong.

If you are using 1 minute exposures you will capture photons only for some of your light frames and when combined the photons will be considered as noise since they are not in most of the light frames."

Peter


Post Extras: Print Post   Remind Me!   Notify Moderator  
Alph
Carpal Tunnel


Reged: 11/23/06

Loc: Melmac
Re: 5 or 30 minute exposures for narrowband? new [Re: freestar8n]
      #5654664 - 01/31/13 04:44 PM

Quote:

I think the term "bias" is used because it offsets the pixel response from zero.



Yes, I know they call it bias for a different reason, the one you mentioned. I don't expect them to know statistics

Quote:

each pixel is assumed to generate the same amount of read noise on top of that with the same Poisson distribution.



The readout noise is not modeled as Poisson distribution. That is why it stands out from other noise sources and adds a different wrinkle to image stacking.


Post Extras: Print Post   Remind Me!   Notify Moderator  
cn register 5
scholastic sledgehammer


Reged: 12/26/12

Re: 5 or 30 minute exposures for narrowband? new [Re: cn register 5]
      #5654676 - 01/31/13 04:50 PM

A lot of this discussion seems to be presenting what I see as arguments from authority - stating that something is so or quoting other people (some unnamed) who say that it is so, with no justification. This changes the discussion from a scientific/technical one to a matter of if you trust what the authority is saying - a matter of faith.

I'm not very keen on using arguments from authority in a scientific/technical discipline such as Astronomy.

There are some useful things here, Mark posted a couple of very useful links, one of which was a way of comparing the read noise with the signal you are actually getting.

http://www.hiddenloft.com/notes/SNR.txt


What it said is that the sky noise should be three times the read noise to make the read signal about 10% of the sky signal. (edited because I confused noise and signal in a couple of places). This allows us to calculate how much signal is required to make the read noise a sufficiently small fraction of the signal. For wide band filters and/or a light polluted site a short exposure will be needed to get here but at a dark site with narrow filters a very long exposure may be needed.

If I apply some numbers, my ATK 383L+ has a read noise of 7e. 7x3 is 21 so that's how much noise will be in the minimum sky signal. Squaring this gives a signal of about 400e. So If I have 400 or more electrons generated by photons from the sky in each pixel the read noise contribution is sufficiently small. Multiplying 400e by the camera gain (about 2.5) says that the DAC value needs to be at least 1000 units above the dark current in the darkest areas of the image. That value should be the same everywhere, regardless of the filters or how dark the sky is.

If you are in a really dark sky area with very narrow filters this will take ages, if you are in a light area with wide filters it will take a lot less time. In either case it gives you an indication of if your exposures were long enough that read noise is not significant.

Last night I had the first clear evening for a fortnight so I set up doing 5 minute exposures of the Horse head area. Of the 20 exposures only seven were acceptable because of passing clouds. The signal level in the dark areas, such as the horse head, were about 1200 so that doesn't look too far away from what should be needed.

Hope this helps,

Chris

Edited by cn register 5 (01/31/13 05:48 PM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
Alph
Carpal Tunnel


Reged: 11/23/06

Loc: Melmac
Re: 5 or 30 minute exposures for narrowband? new [Re: cn register 5]
      #5654715 - 01/31/13 05:14 PM

Quote:

What it said is that the sky noise should be three times the read noise to make the read noise about 10% of the sky noise.



Do you realize that your statement is incoherent?


Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: cn register 5]
      #5654726 - 01/31/13 05:20 PM

Quote:

Simply put you will only get a signal if your exposure is long enough to catch some photons on most of the light frames so that the signal is not considered as noise.




The last part is important. "so that the signal is not considered as noise"

SNR is kind of an odd metric, and not just because your sort of pretending you know the actual signal power and noise power, but because what we seem to really want to know is the comparison of the weakest parts of the signal, versus the noise; not just the estimation of the observed signal versus the noise.

More technically, the SNR statistic is actually just the sample mean over the sample standard deviation. This is, in itself a biased estimator of the asymptotic mean over the asymptotic standard deviation.

Mathematically though, if you have sufficient calibration, and sufficient samples (exposures), at some point, more shorter exposures will tend to provide a cleaner signal (image) than fewer longer. Any result that shows otherwise likely assumes insufficient calibration or samples, or has a flaw somewhere in the logic.

If we took a low noise camera, take lots of calibration frames, and lots of exposures, at some point, the results over a given integration time, should start to converge to a better sampling of the underlying signal, than fewer long exposures. It's really not a humble opinion, just a mathematical result. I believe this is already being done in some medical imaging applications, for example, actually. To get good calibration though, my understanding is that very low noise cameras are being cooled to around -90 Deg C, flats taken using extremely well calibrated light boxes, and lots of bias frames used etc... So, this isn't necessarily something we can do in the hobby at this time. However, blindly assuming longer is better, also does not seem completely palatable, even with narrow band filters...


Post Extras: Print Post   Remind Me!   Notify Moderator  
freestar8n
Post Laureate
*****

Reged: 10/12/07

Re: 5 or 30 minute exposures for narrowband? new [Re: cn register 5]
      #5654738 - 01/31/13 05:26 PM

Quote:

This changes the discussion from a scientific/technical one to a matter of if you trust what the authority is saying - a matter of faith.




Well I certainly agree with that - which is why I provide direct links to professional sources - which themselves contain references. Many of links provided in these discussions are just to web pages that themselves have no references. An argument from authority should be distinguished from a "reference to the literature" - and primary sources. On this topic, the amateur sources, and arbitrary web pages, can depart from professional literature - and cause confusion or state incorrect conclusions.

The link you provided may be perfectly correct - but its validity is completely unknown since it is just a discussion grabbed from the web - which in fact is what this thread is. The only implied validation of the content of that text is the assumed expertise of the people writing - which is a form of appeal to authority.

As for the numbers you present - great - that is the first thing I suggested doing in my first posting of this thread:

Quote:

It's good to know the actual read noise of the camera and how much signal is coming from sky glow rather than nebulosity. A few exposures should give a feel for these values.




So - now that you have numbers for your site and your equipment, you can make concrete decisions about the optimal exposure time. The fact that you were hit by passing clouds is a perfect example of why, despite the models and calculators, other factors may make an even shorter exposure more practical and increase your achieved SNR in the long run.

The main thing is - your calculation is based on making read noise small in each sub compared to sky and other noise - and that is the key point about sub-exposure time that keeps being lost or confused in this thread.

Frank


Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: freestar8n]
      #5654748 - 01/31/13 05:31 PM

Quote:


I think the term "bias" is used because it offsets the pixel response from zero.




0 is the expected value of the pixels, given no signal. Even moving pattern noise will converge to random variation or some mean deviation from the expected pixel value, given enough samples. If it converges to a mean value, other than 0, it is in statistical terms "bias", but in this case, can be removed via calibration. At least given that enough calibration samples are taken, "bias frames" so that the master bias frame, converges to nearly the asymptotic mean value of each pixel. Also, we probably need a sufficient pedestal to be used so that we don't clip any data.


Post Extras: Print Post   Remind Me!   Notify Moderator  
cn register 5
scholastic sledgehammer


Reged: 12/26/12

Re: 5 or 30 minute exposures for narrowband? new [Re: Alph]
      #5654766 - 01/31/13 05:45 PM

Quote:

Quote:

What it said is that the sky noise should be three times the read noise to make the read noise about 10% of the sky noise.



Do you realize that your statement is incoherent?



I think I should have said three times the read noise to make the read signal about 10% of the sky signal, it's because the variance of the signal is the square root of the average value. You can square the noise to get the signal.

This is all in the reference I posted.

Chris

Edited by cn register 5 (01/31/13 05:49 PM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
cn register 5
scholastic sledgehammer


Reged: 12/26/12

Re: 5 or 30 minute exposures for narrowband? new [Re: Inverted]
      #5654787 - 01/31/13 06:00 PM

Quote:


I think the term "bias" is used because it offsets the pixel response from zero.



There's a nice story about this.
There were some analyses being done of the mine waste from gold mines in South Africa with the idea that improved methods of extraction could make it profitable to rework the old mine spoil heaps. Some Au values were negative so they were discarded - after all you can't have less than nothing. I'm not sure how much mine tailings they dug up before they realised there wasn't as much Glod there as they thought...

Chris


Post Extras: Print Post   Remind Me!   Notify Moderator  
Alph
Carpal Tunnel


Reged: 11/23/06

Loc: Melmac
Re: 5 or 30 minute exposures for narrowband? new [Re: Inverted]
      #5654793 - 01/31/13 06:02 PM

Quote:

0 is the expected value of the pixels, given no signal.



No, 0 is not an expected value. Cameras add an offset value (bias) to readout by design and the expected value has spatial bias (fixed pattern noise if you will).


Post Extras: Print Post   Remind Me!   Notify Moderator  
Mike7Mak
Pooh-Bah
*****

Reged: 12/07/11

Loc: New York
Re: 5 or 30 minute exposures for narrowband? new [Re: shams42]
      #5654826 - 01/31/13 06:14 PM

Quote:

See, my position is that this is MORE confusing that attempting to distinguish between "unwanted signal" and "random noise" as I have proposed.



I have to agree. It seems quite unscientific to broaden the meaning of noise to the point that 'noise' has noise. IMO use of the term 'noise' as a catch-all for unwanted signal approaches the level of slang.

I can say from personal experience until the noob grasps the significant difference between 'unwanted signal' and 'variance/uncertainty', discussions like this leave us scratching our head. Noise remains a mystical evil spirit that we ward off with long exposure and image stacking on faith alone...cuz the guys who make pretty pics say so.


Post Extras: Print Post   Remind Me!   Notify Moderator  
freestar8n
Post Laureate
*****

Reged: 10/12/07

Re: 5 or 30 minute exposures for narrowband? new [Re: cn register 5]
      #5654839 - 01/31/13 06:23 PM

Hi-

I haven't gone through your numbers but I wanted to confirm your 1200 ADU value is measured from a calibrated, rather than raw, sub exposure. You want to make sure there are no artificial offsets in there. You could just measure the difference between a single dark and a raw sub of the horsehead to make sure there is no offset involved.

Also - I recommend sticking to gain as e/adu - so yours is about 0.4 e/adu. To convert e to adu you would divide by gain.

Also - what f/ratio and what bandpass? I assume no moon involved? With those numbers, more general conclusions could be drawn for your camera and location.

Frank


Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: Alph]
      #5654878 - 01/31/13 06:44 PM

Quote:

Quote:

0 is the expected value of the pixels, given no signal.



No, 0 is not an expected value. Cameras add an offset value (bias) to readout by design and the expected value has spatial bias (fixed pattern noise if you will).




Gotcha, that makes sense. I did say, probably would need to add a pedestal though, "offset" is a much better term. "Bias" in that regard is then extremely confusing


Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: Mike7Mak]
      #5654905 - 01/31/13 07:08 PM

Quote:

I have to agree. It seems quite unscientific to broaden the meaning of noise to the point that 'noise' has noise. IMO use of the term 'noise' as a catch-all for unwanted signal approaches the level of slang.




I think everything is signal, it just may not be the signal you "want". But if we go by what you "want" how the heck do we know what you want? We can't read minds.... We can rigorously define a parameter of interest and come up with a practical way to measure it, and try to do so with the least "error" possible. As mentioned in stats we don't really talk about "noise" usually. We have error terms, which really simplify to variance and bias.

Take for example darts. If the goal is to hit the bullseye, we throw the dart a lot and it goes all over. Then we have high variance. If it always hits the same place, we have, very low variance. However, even with very low variance, your darts may always land in the same place, but miss the dartboard completely. Then you have large bias. If they land very close to the bullseye, you have low bias and if they hit the bullseye, you have no bias. So, together you have a measure of " accuracy". The deviation from the bullseye is the measurement.

If your target is well defined, such as a bullseye, then It's theoretically possible to determine your accuracy by measuring your deviation over multiple dart throws (I.e. your taking "samples" of your dart throwing ability) However, in reality, the true value of most things we study, are probably not known, it's sort of like measuring your dart throws while blindfolded. While you can't do this, you may be able to measure your dart throws, under different circumstances and use this to "calibrate" your aim latter, when you put the blindfold back on. You will still have random variation to contend with, but on average, if you throw enough darts, your deviation from the bullseye, if calibrated correctly, will average out to zero, as there is no longer any fixed variation, all the deviation is random and looking at the average of your measurements, you will now look like a perfect player!

This may seem like a slight of hand, but if our goal is to figure out where the bullseye really is, we now have a perfect estimate.


Post Extras: Print Post   Remind Me!   Notify Moderator  
pfile
Post Laureate


Reged: 06/14/09

Re: 5 or 30 minute exposures for narrowband? new [Re: Inverted]
      #5655081 - 01/31/13 09:08 PM

"bias" - isn't this meant in the EE sense, like forward biasing a diode?

my crude understanding of how a CCD works is that before you expose the frame, the capacitive sites are biased with a positive voltage.

there's some uncertainty in this bias voltage from pixel to pixel, hence, "bias noise".

as photons fall onto the detector, they cause electrons in the silicon to move to a higher valence, thus progressively making the voltage more negative. but even if the number of photons that landed on each site was the same, the end voltage is different because of difference in the bias voltages. (and of course, dark current).


Post Extras: Print Post   Remind Me!   Notify Moderator  
pfile
Post Laureate


Reged: 06/14/09

Re: 5 or 30 minute exposures for narrowband? new [Re: pfile]
      #5655091 - 01/31/13 09:11 PM

inverted, you are simply describing photon shot 'noise' which really has nothing to do with detectors. it's just part of how this universe is put together. bias has nothing to do with it. bias and bias noise are artifacts of the photon detectors we have invented.

Post Extras: Print Post   Remind Me!   Notify Moderator  
vpcirc
Post Laureate


Reged: 12/09/09

Loc: Merced CA
Re: 5 or 30 minute exposures for narrowband? new [Re: Inverted]
      #5655101 - 01/31/13 09:18 PM

I'd like to see some examples of the great images the experts here to see how they apply your obvious extensive knowledge?
The guys "who make pretty pics" must know what they're talking about the evidence to me lies in what they produce. Many are pioneers in this field and have years of extensive experiance. Not all are mathematicians or engineers, but I'll trust experiance and advice from someone producing what I hope too, any day over someone trying to show how they know more than anyone else. People are critizing others for sharing links for advice, but those folks are producing amazing images and being acknowledged with APODS. The question is how many do you have?
I'm from Missouri "Show Me"


Post Extras: Print Post   Remind Me!   Notify Moderator  
shams42
Pooh-Bah
*****

Reged: 01/05/09

Loc: Kingsport, TN
Re: 5 or 30 minute exposures for narrowband? new [Re: vpcirc]
      #5655114 - 01/31/13 09:29 PM

Mike, please show us something from one of these top imagers that you keep referring to that does not agree with what everyone here is saying.

What is this mysterious advice?

You keep insinuating that we all have some profound ignorance of this topic and yet you do not expose your own view for discussion and criticism. So out with it!


Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: pfile]
      #5655166 - 01/31/13 10:05 PM

Quote:

"bias" - isn't this meant in the EE sense, like forward biasing a diode?

my crude understanding of how a CCD works is that before you expose the frame, the capacitive sites are biased with a positive voltage.

there's some uncertainty in this bias voltage from pixel to pixel, hence, "bias noise".

as photons fall onto the detector, they cause electrons in the silicon to move to a higher valence, thus progressively making the voltage more negative. but even if the number of photons that landed on each site was the same, the end voltage is different because of difference in the bias voltages. (and of course, dark current).




That's a confusion I've been having. "Bias" in an EE sense, is apparently used in one way, however, "bias frames"', are actually apparently named for the EE definition, but function in a statistical way. To really make things confusing, mathematically, they function statistically to remove something else called "bias" in stats, although this actually differs from the definition of bias used in EE


Post Extras: Print Post   Remind Me!   Notify Moderator  
Mike7Mak
Pooh-Bah
*****

Reged: 12/07/11

Loc: New York
Re: 5 or 30 minute exposures for narrowband? new [Re: Inverted]
      #5655208 - 01/31/13 10:27 PM

Quote:

Quote:

I have to agree. It seems quite unscientific to broaden the meaning of noise to the point that 'noise' has noise. IMO use of the term 'noise' as a catch-all for unwanted signal approaches the level of slang.




Take for example darts.



I'm not quite sure why what I said prompted that explanation but thanks.

I might have used the analogy of sighting in a rifle, where the offset of the group from the bullseye represents unwanted signal (subtracted with clicks of the scope crosshairs) and the individual hits within the group represent noise (the variance/uncertainty type that must be averaged out).

My point however was if 'noise' is going to mean everything from hot pixels to measurement uncertainty then without further qualification the term 'noise' is profoundly imprecise. Saying an image is noisey becomes only slightly more descriptive than saying it sucks.

In the grand scheme of this discussion this is just a nit pick, so please feel free to ignore it and carry on.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: Mike7Mak]
      #5655236 - 01/31/13 10:42 PM

Quote:

inverted, you are simply describing photon shot 'noise' which really has nothing to do with detectors. it's just part of how this universe is put together. bias has nothing to do with it. bias and bias noise are artifacts of the photon detectors we have invented.




I wouldn't claim to understand all of the noise sourses in the hobby, but I do know enough to understand that you either do not understand what i am saying, and/or do not understand modern statistical theory. I've tried to give a very general overview of statistical theory, as relating to sampling theories, several times, I'm not sure what else I can say without getting into more technical nuts and bolts.... I think the results are fairly intuitive, so, I'm apparently not doing a very good job anyways perhaps your a technical person and would prefer a more technical explanation?. I think a good introduction would be Sampling Design and Analysis by Shannon Lohr, most of the other sampling texts people seem to use really get complex quick, and get into fairly hardcore math and especially matrix algebra though. Really, most of the ideas seem to come from the central limit theorems and various theorems referred to as the "law" of large numbers (but I think these are probably more results of the central limit theorems, the proofs go over my head, but my graduate school stats professors, who specialized in sampling theories, seemed to think so). Anyways, I can't really add anything to the conversation, other than try to put some of the statistical concepts in perspective. The models used are very statistical, but attempts to use them, seem to try to treat them in more deterministic perspective. I think that is really only necessary if your under-sampled though (and this could be sampling with regards to calibration, not just image acquisition)).


Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: vpcirc]
      #5655253 - 01/31/13 10:49 PM

Quote:

I'd like to see some examples of the great images the experts here to see how they apply your obvious extensive knowledge?




The hobby can be an art or a science or both. A scientist who creates the paints for an artist need not be a proficient artist and the artist need not be a proficient scientist. When people of different backgrounds and perspectives work together, though, great things can happen.


Post Extras: Print Post   Remind Me!   Notify Moderator  
David PavlichAdministrator
Transmographied
*****

Reged: 05/18/05

Loc: Mandeville, LA USA
Re: 5 or 30 minute exposures for narrowband? new [Re: Inverted]
      #5655267 - 01/31/13 11:00 PM

Sometimes we all tend to overthink this stuff. If you have a chip like the 8300 class and you're shooting narrowband, take as long as you can stand and your equipment can handle. Neil Flemming recommends 30 minutes. Then take darks and flats and process the stuff.

If you have a camera with a Sony chip like the QHY8 or the Starlight Express MC25, then you don't really need the darks, just a bunch of bias and flats and process.

At the AIC, I learned a couple of things of note. For LRGB, there's a point of diminishing returns beyond 10 minutes. I believe Craig Stark was the presenter that had a chart showing how the curve was very steep until it got to around 10 minutes. The curve tapers to near flat after 10 minutes. Of course, that can change with the scope. A Powernewt is not going to need the full 10 minutes where as an f10 SC may need more time.

Tony Hallas stated that 16 lights seemed to be a good number. In order to get big returns, you needed to go way past 16. In other words, doing 20 lights won't make much difference.

The one thing from every presenter was for a given time, longer exposures ARE best as long as the equipment can handle it and skyglow doesn't take over. Of course, NB isn't affected by skyglow...well not like LRGB.

Anyway, take your images, process them and enjoy them. This stuff will give you a headache if you overdo it.

David


Post Extras: Print Post   Remind Me!   Notify Moderator  
pfile
Post Laureate


Reged: 06/14/09

Re: 5 or 30 minute exposures for narrowband? new [Re: Inverted]
      #5655278 - 01/31/13 11:06 PM

there is nothing wrong with the statistics you are talking about but you are using "bias" in a statistical sense while in the process of calibrating ccd images, bias means something very different. that's all.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: Mike7Mak]
      #5655279 - 01/31/13 11:06 PM

Quote:


I'm not quite sure why what I said prompted that explanation but thanks.





It was in part an attempt to explain why the definition of noise does not necessarily need to be precise, if "target" is precisely defined. Also, I agreed with Chris's statements about authority, I'm not big on authority either. However, I think a lot of the best results do not need to rely on authority or complicated technical definitions, but simple logic and though experiments. I though it better to provide a starting point for such, rather than just state a bunch of technical jargon. I may or may not have done so successfully, but in the end, when methods produce results people like, they will be used, when they don't they won't. In the meantime though, there are no new results without new perspective.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: pfile]
      #5655301 - 01/31/13 11:24 PM

Quote:

there is nothing wrong with the statistics you are talking about but you are using "bias" in a statistical sense while in the process of calibrating ccd images, bias means something very different. that's all.




But it is also statistical sense. There is random noise in the bias frame that converges to a null value, and there is noise that doesn't. If you take enough bias frames and average them, you are mostly left with the stuff that doesn't. This seems important if read noise truly is the dominant source of noise, as many claim. There may be some left that could theoretically if more bias frames were stacked, but this does at least become more Gaussian. Then, when we calibrate the light frames the statistical "bias" from the "reads" are eliminated to an extent (minus some now mostly Normal random error, provided n is large enough). Much of the remaining "noise" and other random noise, from other sources, then still tends further towards the null as more calibrated light frames are stacked. These frames may be intended to linearize the data, as it sounds, however, they along with other calibration frames and sampling techniques should also help remove anything things that converge to a non-null value, independent of the sampled parameter(s) of interest. More or less so, depending on how they are used. More so for example if I were to stack say 400, less so, if I were to stack a few. If you take enough bias frames though (or any other frames that contain the asymptotically fixed and random components of read noise) then it seems you can reduce read noise pretty significantly, especially if read noise is relatively low to begin with. Then when you plug the numbers into the equations, the implications may change substantially.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: David Pavlich]
      #5655326 - 01/31/13 11:37 PM

For sure and the general rules are tried and true and for the most part surly good to follow. It is a technical hobby though.... It's good to exercise the noggin once and a while and try to understand the technical aspects I think. Otherwise, I could save a lot of money and just do basket weaving or something

Post Extras: Print Post   Remind Me!   Notify Moderator  
Mike7Mak
Pooh-Bah
*****

Reged: 12/07/11

Loc: New York
Re: 5 or 30 minute exposures for narrowband? new [Re: Inverted]
      #5655339 - 01/31/13 11:42 PM

Quote:

Quote:


I'm not quite sure why what I said prompted that explanation but thanks.





It was in part an attempt to explain why the definition of noise does not necessarily need to be precise, if "target" is precisely defined.




Gotcha. Yes, I'm all for simple logic and though experiments. My interest in topics like this is in understanding the principles and applying the techniques, to the extant that is possible without knowing advanced statistical theory and higher math. I don't have the educational background to go there nor the motivation to start learning such things now.


Post Extras: Print Post   Remind Me!   Notify Moderator  
cn register 5
scholastic sledgehammer


Reged: 12/26/12

Re: 5 or 30 minute exposures for narrowband? new [Re: vpcirc]
      #5655541 - 02/01/13 03:39 AM

Quote:

I'd like to see some examples of the great images the experts here to see how they apply your obvious extensive knowledge?
The guys "who make pretty pics" must know what they're talking about the evidence to me lies in what they produce. Many are pioneers in this field and have years of extensive experiance. Not all are mathematicians or engineers, but I'll trust experiance and advice from someone producing what I hope too, any day over someone trying to show how they know more than anyone else. People are critizing others for sharing links for advice, but those folks are producing amazing images and being acknowledged with APODS. The question is how many do you have?
I'm from Missouri "Show Me"



Mike, this is an excellent example of the argument from authority, with a side order of ad hominem.

You don't need to do this, after all you gave us both of the links to web pages with good technical information.

Chris


Post Extras: Print Post   Remind Me!   Notify Moderator  
freestar8n
Post Laureate
*****

Reged: 10/12/07

Re: 5 or 30 minute exposures for narrowband? new [Re: cn register 5]
      #5655569 - 02/01/13 04:16 AM

Well I took a look at the numbers and I can't find good specs on the 383L ccd. The ad says 7e read noise, but I don't see a spec. on the gain. Various users on the web measure higher read noise - 8-11e, and gain around 0.45 e/adu.

If we assume 7e for the read noise, and aim for that to be 10% of the sky glow shot noise - and remember that 10% is just a number from a 12-year old discussion on the web - then you want the sky shot noise to be 70e. Assuming Poisson statistics, the sky signal would then be 4900e. Dividing 4900e by the gain, 0.45e/adu, we get a target ADU for the sky glow in the 'black' regions of the image to be 11000 adu.

If a calibrated sub-exposure gave about 1200 adu sky glow in a 5m exposure, then the target sub-exposure to bury the read noise to 10% sky noise would be about 45 minutes.

Note that in this calculation, which is along the line of the web discussion Chris has chosen to use as a reference, there is no mention of the nebulosity signal itself - or anything involving "exposing until the faint stuff shows." It is all about making the read noise small compared to other camera noise, because read noise is the only thing that can have reduced impact through longer sub-exposures. It also assumes a cold camera so dark current can be ignored - which may not be the case in general - and sky noise is all that matters.

The problem is - that web discussion is somewhat stream of consciousness and includes errors that are corrected later in the thread. It says right there:

"A simple dictum would be to expose long enough so that camera noise is only about 10% of the background noise."

- but that is a mistake that is later retracted in the same thread. This is why I prefer the more scientific approach of using textbooks and journal references that themselves have references.

Ultimately, the target they "decide" on is that sky noise should be 3x read noise - or a threshold of 33% instead of 10%. This is much easier to achieve.

TSub(33%) = (7/0.33)^2/0.45/1200*5 = 4m

So one of the biggest factors here is the choice of that 10-33% threshold value - and there is nothing magic about that choice.

I don't know the f/number used here, but assuming it is f/8 - if we switched to f/2 hyperstar, the sky background would be reached 16 times as fast, and the original 45 minutes, assuming 10% target, would be reached in about 3 minutes. Assuming 33% target, about 15 seconds. This would still be narrowband with the same filter - just a faster system - so narrowband does not absolutely require long exposures - even with this 10% threshold and 7e read noise.

So the target numbers depend on many system parameters, plus a user-specified threshold. They require good estimates of the gain and read noise, plus a personal choice for the threshold. Since those numbers vary, I think it's good to measure your own camera - and it isn't hard - and to decide what threshold value works best for you.

I could have made a mistake above so I welcome corrections.

Note that if the original 1200 adu background value came from a raw sub that included a pedestal of, say, 600 - then all estimates above would be doubled because it would take twice as long to gather the needed sky background electrons.

Frank


Post Extras: Print Post   Remind Me!   Notify Moderator  
vpcirc
Post Laureate


Reged: 12/09/09

Loc: Merced CA
Re: 5 or 30 minute exposures for narrowband? new [Re: shams42]
      #5655636 - 02/01/13 06:27 AM

Quote:

Mike, please show us something from one of these top imagers that you keep referring to that does not agree with what everyone here is saying.

What is this mysterious advice?

You keep insinuating that we all have some profound ignorance of this topic and yet you do not expose your own view for discussion and criticism. So out with it!




I am in no way saying that all these posts are wrong or incorrect. But I have heard repeatedly "you don't need to go longer" or several short exposures stacked with equal time are the same, My philopshy mirrors what David has posted. Hallas says there's a limit to how many exposures can be stacked before there's no benefit, therefore shoot as long of exposures as you can. In my case I would trust someone who's been shooting AP since the film days, produces top notch images, has several APODS, is used by NASA and Apple any day over someone who can quote the statistical reasoning of how noise accumulates, but can't produce a quality image themselves.


But when I see statements like the following;

Well I certainly agree with that - which is why I provide direct links to professional sources - which themselves contain references. Many of links provided in these discussions are just to web pages that themselves have no references. An argument from authority should be distinguished from a "reference to the literature" - and primary sources. On this topic, the amateur sources, and arbitrary web pages, can depart from professional literature - and cause confusion or state incorrect conclusions.

Or

Noise remains a mystical evil spirit that we ward off with long exposure and image stacking on faith alone...cuz the guys who make pretty pics say so.

That appears to me as a direct slap at any of the people whom have posted links to information sources to try and help people. Most imagers could care less about the science behind noise they just want help in understanding what works best.


Post Extras: Print Post   Remind Me!   Notify Moderator  
dan17
super member


Reged: 03/30/09

Re: 5 or 30 minute exposures for narrowband? new [Re: vpcirc]
      #5655675 - 02/01/13 07:14 AM

Many things will be clear and easy to understand if You all have a look at this:

http://www.cloudynights.com/item.php?item_id=1966

http://www.cloudynights.com/item.php?item_id=1973

http://www.cloudynights.com/item.php?item_id=2001

http://www.cloudynights.com/item.php?item_id=2042

http://www.cloudynights.com/item.php?item_id=2394


Post Extras: Print Post   Remind Me!   Notify Moderator  
Mike7Mak
Pooh-Bah
*****

Reged: 12/07/11

Loc: New York
Re: 5 or 30 minute exposures for narrowband? new [Re: vpcirc]
      #5655979 - 02/01/13 09:59 AM

Quote:

Or

Noise remains a mystical evil spirit that we ward off with long exposure and image stacking on faith alone...cuz the guys who make pretty pics say so.

That appears to me as a direct slap at any of the people whom have posted links to information sources to try and help people. Most imagers could care less about the science behind noise they just want help in understanding what works best.



And again... good grief, passive aggressive much?

Dude, that's completely out of context, it meant nothing of the sort. And to the degree that last bit was a poke in the ribs, it was at you, not "any of the people" who post links.

So let's pretend you know what 'most imagers' couldn't care less about. 'Most imagers' aren't participating in this thread. The ones that are obviously DO care about the science. In addition to understanding 'what works best' some of us also want to know why.

Both those issues, 'don't need to go longer' and the diminishing returns of stacking, were thoroughly explored and apparently you didn't understand a word of it. If you had you'd realise no one blasphemed the imaging gods.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: freestar8n]
      #5655995 - 02/01/13 10:12 AM

Quote:

It is all about making the read noise small compared to other camera noise, because read noise is the only thing that can have reduced impact through longer sub-exposures. It also assumes a cold camera so dark current can be ignored - which may not be the case in general - and sky noise is all that matters.





That's the thing, if the models are correct and the only thing that prevents us from using more short subs, is read noise, the better calibration should eliminate much of that issue.

If we go back to the dart example, it's sort of like throwing the darts, at a bar, with your friends. Intuitively, you can get the same result whether you throw lots of darts at once, or stop and start again latter. However, read noise is like the friends you brought along. Every time you stop, they decide it will be fun to spin you around. Now, when you go back, you are dizzy, so your throws become more variable and you still feel like your spinning in one direction, so, there is a "bias" added as your throws tend to drift I one direction.

However, you may realize that your friends calm down after some number of beers, but they get more rowdy if they have too many. So, you can use the same principles used to find the bullseye, to find the optimal amount of beer, so that your friends have less of an effect. The problem is you get diminishing returns as you try to further refine your estimation of the optimal amount of beer. If you try to measure this 16 times, there is still around 25% variation in your ability to determine the correct amount of beer. If you do 20 measures, you should still have a 22% variation in your certainty of the correct amount. So, it seems like there is no real gain. However, I you went to 400 measures, your guess would now be accurate to within 5% of the optimal level. If your estimation of the bullseye location is mainly limited by your friends, you've now effectively reduced they influence by 95% compared to 75% with just 16 measures. And if the bullseye is really small, that little percentage may make a huge improvement in your ability to find it. It may still not be enough enough of a reduction though, to make shorter throwing sessions equivalent to longer ones, but it gets you closer. You can now go much shorter than before calibration and achieve the same result.

If you further and further improve your calibration of your friends (read noise) at some point, other, more nominal sources of error may become a bigger proportion of the remaining limitation for further accuracy. Perhaps now that your friends are not bothering you, the drunks at the bar are becoming more noticeable (think of them as sky noise, cosmic rays, or passing clouds, or periods of poor "seeing" etc...). So, perhaps it is beneficial to pause and deal with you now mellow friends a little more, so that you can redo these sets of throws. Or perhaps, you want to do this because you are out of shape and your muscles are getting tired (think poor polar alignment. Or equipment limitations). Then though, a narrow band filter is sort of like some headphones, you can put on some nice soothing music and forget all about the drunks and just carry on however, this doesn't help your muscles and unfortunately the headphones aren't perfect, so, at some point we may still want to retry some sets do to other although more nominal influence of the drunks....

Edited by Inverted (02/01/13 10:32 AM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
Mike7Mak
Pooh-Bah
*****

Reged: 12/07/11

Loc: New York
Re: 5 or 30 minute exposures for narrowband? new [Re: Inverted]
      #5656085 - 02/01/13 10:42 AM

Whoa, I hope you're not expanding that dart analogy strictly for my benefit cuz I'm finding it harder to fathom than the technical explanation it's meant to represent.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: Mike7Mak]
      #5656115 - 02/01/13 10:55 AM

No LOL. I figure, as I apparently can't summarize my thoughts do to different uses of terminology... I'd summarize with humor, so I can put this to away and go back to looking at pretty pictures

Post Extras: Print Post   Remind Me!   Notify Moderator  
Fogboundturtle
professor emeritus


Reged: 05/20/09

Loc: Burnaby, BC
Re: 5 or 30 minute exposures for narrowband? new [Re: Inverted]
      #5656126 - 02/01/13 11:00 AM

This thread is the result of too many clouds. Otherwise everyone would be busy photon capturing.

Post Extras: Print Post   Remind Me!   Notify Moderator  
korborh
scholastic sledgehammer
*****

Reged: 01/29/11

Loc: Arizona
Re: 5 or 30 minute exposures for narrowband? new [Re: Fogboundturtle]
      #5656180 - 02/01/13 11:29 AM

This thread is awesome. While I do like capturing photons, a large part of what attracts me in this hobby is the opportunity of being able to experiment and understand the underlying physics conspiring against accurate observations of nature.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Mike7Mak
Pooh-Bah
*****

Reged: 12/07/11

Loc: New York
Re: 5 or 30 minute exposures for narrowband? new [Re: korborh]
      #5656216 - 02/01/13 11:57 AM

There was something mentioned early on that I meant to question but forgot. And if I missed the explanation along the way my apologies for the repetition.

It had to do with longer, hence fewer, exposures "injecting a smaller multiple of the read noise into the final stack for a fixed total integration time".

That seems to imply that read noise accumulates/increases with each frame added to the stack instead of diminishing at the square root blah blah rate of other noise.

I'd chalk it up to me reading it wrong except I've read similar statements elsewhere that gave me the same impression. What am I missing?


Post Extras: Print Post   Remind Me!   Notify Moderator  
Peter in Reno
Postmaster
*****

Reged: 07/15/08

Loc: Reno, NV
Re: 5 or 30 minute exposures for narrowband? new [Re: dan17]
      #5656283 - 02/01/13 12:37 PM

Quote:

Many things will be clear and easy to understand if You all have a look at this:

http://www.cloudynights.com/item.php?item_id=1966

http://www.cloudynights.com/item.php?item_id=1973

http://www.cloudynights.com/item.php?item_id=2001

http://www.cloudynights.com/item.php?item_id=2042

http://www.cloudynights.com/item.php?item_id=2394




Craig Stark is one of the best writer in the Astronomy market. He not only writes well, but explains in layman's terms. Please read his articles.

Peter


Post Extras: Print Post   Remind Me!   Notify Moderator  
pfile
Post Laureate


Reged: 06/14/09

Re: 5 or 30 minute exposures for narrowband? new [Re: Peter in Reno]
      #5656379 - 02/01/13 01:34 PM

i think if you could calibrate away read noise then the concept of sky-limited vs. read noise limited exposures would not even exist - the idea of exposing long enough to clear the read noise would not exist.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Alph
Carpal Tunnel


Reged: 11/23/06

Loc: Melmac
Re: 5 or 30 minute exposures for narrowband? new [Re: Mike7Mak]
      #5656552 - 02/01/13 03:06 PM

Quote:

That seems to imply that read noise accumulates/increases with each frame added to the stack instead of diminishing at the square root blah blah rate of other noise.




Noise (random) always adds up whether you are subtracting or adding images. There are basically two types of noise in CCD: Poisson noise and readout noise. Poisson noise is dark current shot noise, sky-glow photon noise and object photon noise. Regardless of random noise type, the noise always adds quadratically.


Post Extras: Print Post   Remind Me!   Notify Moderator  
mikeschuster
super member


Reged: 08/25/11

Loc: SF Bay area
Re: 5 or 30 minute exposures for narrowband? new [Re: Alph]
      #5656599 - 02/01/13 03:37 PM

Alph, are you implying that stacking beats down poisson noise more or at a higher rate than read noise? If so that is not what I experience, they seem to go down at the same rate. On the other hand, in my experience, FPN does not get beat down at all with stacking.
Mike

Edited by mikeschuster (02/01/13 03:40 PM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
cn register 5
scholastic sledgehammer


Reged: 12/26/12

Re: 5 or 30 minute exposures for narrowband? new [Re: mikeschuster]
      #5656673 - 02/01/13 04:18 PM Attachment (9 downloads)

I spent the afternoon collecting darks, and used DSS to process the images I took the other evening. I managed to use 14 of the frames but some are pretty obscured.

I ended up with this, by APOD standards it's terrible, but it's about the best I've managed so far.
Oynx 80mm F/6.25 scope, ATK383L+ binned 2x2, 17 x 5 minutes & 16 x 5 minutes darks. Aligned and stacked with DSS. Minimal processing, a bit in DSS and a curve in PS3.

Chris


Edited by cn register 5 (02/01/13 04:29 PM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: pfile]
      #5656689 - 02/01/13 04:28 PM

Quote:

i think if you could calibrate away read noise then the concept of sky-limited vs. read noise limited exposures would not even exist - the idea of exposing long enough to clear the read noise would not exist.




Yes, I think so! Of course we can't really calibrate out read noise entirely, but I think you can do a lot more than some people do. If someone assumes that the can't do better because of diminishing returns, and stop at some number, such as 16, then our estimate of the FPN part of the read noise, still has a lot of variance ("noise"). On my camera, it is about a 8 second download, plus figure, in say idk 2 seconds between frames, that's 10 seconds per bias frame. There are 86400 second in a day, so, if I had a cloudy day to waste, I could theoretically take 8640 bias frames. If I took that many, assuming the random component was fairly normal to begin with (or at least more or less followed some exponential family PDF), then I would reduce the random component to 100*(1/sqrt(8640)) = 1.07% of the initial random noise value noise value. So, my asymptotic estimate of the FPN would be pretty good LOL. Of course, things such as "moving pattern noise" may follow very different distributions and take a lot longer to converge. So, having a chip with nice, Gaussian properties to begin with should help a lot too...


Edit: by the way, just to clarify, I'm asuming there is a higher ratio of pattern noise to random noise. I think that was obvious, it re-reading I see how it could be interpreted that I am saying we are calibrating out random noise. Of course you can't. That's why I keep saying if we have low read noise AND good calibration. As I said I sti don't have a sense o the magnitudes. That is what I'm interested in though. Hopefully once I have a chance to read Craig's article, that will be more clear.

Edited by Inverted (02/01/13 05:50 PM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: mikeschuster]
      #5656730 - 02/01/13 04:45 PM

Quote:

Alph, are you implying that stacking beats down poisson noise more or at a higher rate than read noise? If so that is not what I experience, they seem to go down at the same rate. On the other hand, in my experience, FPN does not get beat down at all with stacking.
Mike




Every-time you take an image, the signal can be thought of as having a standardized value of "1". The "random" part of the noise is just "variance" of a normal distribution. It is 1/sqrt(N) * stdev So, if we standardize the mean signal to "1" then the stdev of it is also "1". So, the noise is just 1/sqrt(n). It's my understanding from this thread that the FPN is another component of "read noise", separate form the random noise. However, this pattern is the same everytime you take an exposure, so, to make things simple, we can also standardize this to "1". So, each time you take an image, you get 1 unit of FPN from the camera read and 1 unit of signal from the DSO. The random part of the noise though is 1/sqrt(n). So, the signal and FPN accumulate faster and eventually the random noise becomes tiny in comparison.


So, as I just said above ^^ you should be able to improve your bias frames, in order to remove the FPN. You shouldn't need to take 8640 though LOL, 100 should remove 90% and should only take maybe a little over 15 minutes depending on your camera. I suspect that may help. It sounds like the bias should also be in your darks, but it takes longer to take 100 darks. Also, once you create a master bias frame, you should be able to delete the 100, so, it shouldn't kill your HD space. Of course, I'm no expert astrophotographer, so, an expert may have other suggestions..


Post Extras: Print Post   Remind Me!   Notify Moderator  
bilgebayModerator
Post Laureate
*****

Reged: 11/06/08

Loc: Turkiye - Istanbul and Marmari...
Re: 5 or 30 minute exposures for narrowband? new [Re: Inverted]
      #5656733 - 02/01/13 04:46 PM

and here is a frame with no dark calibration at all, from a different camera and scope



Hires photo

6x 1200 + 1x 600seconds Light calibrated with flat and bias master frames
FSQ106EDXIII
Astrodon 3nm Ha filter - SX Filterwheel
Atik 460 EXM
Vixen New Atlux Mount + NexAtlux board - controlled via Sky Safari
Guided via 50mm Borg + SSAG/PHD
Framed, focused and captured with Sequence Generator Pro
Calibrated, aligned and stacked CCDStack
Curves and levels in Photoshop.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Alph
Carpal Tunnel


Reged: 11/23/06

Loc: Melmac
Re: 5 or 30 minute exposures for narrowband? new [Re: mikeschuster]
      #5656740 - 02/01/13 04:48 PM

Quote:

If so that is not what I experience, they seem to go down at the same rate.



I have updated my post. You are essentially right, however noise never goes down. It is the S/N that goes up when stacking.


Post Extras: Print Post   Remind Me!   Notify Moderator  
bilgebayModerator
Post Laureate
*****

Reged: 11/06/08

Loc: Turkiye - Istanbul and Marmari...
Re: 5 or 30 minute exposures for narrowband? new [Re: bilgebay]
      #5656750 - 02/01/13 04:51 PM

a DSLR frame



Hires photo

This one was shot under moon light and terrible seeing conditions.

6x 1200 seconds Light @ ISO 1600, dark, flat, and bias master frames
AT110EDQ+AT Field Flattener
Astronomik 6nm Ha filter
Canon T2i, Hutech modified
Vixen New Atlux Mount + NexAtlux board - controlled via Sky Safari
Guided via Lacerta OAG + SSAG/PHD
Framed, focused and captured with BackyardEOS
Calibrated, aligned and stacked with Images Plus
Curves and levels in Photoshop.

File Name HORSEHEAD_LIGHT_1200_1600iso_HA_6NM_+25c_12c_AT110+FF_AMBIENT12c_777_1829_20111106_05h39m05s825_000429.CR2
Camera Model Canon EOS REBEL T2i
Firmware Firmware Version 1.0.8
Shooting Date/Time 06.11.2011 06:19:44
Author BILGEBAY
Owner's Name
Shooting Mode Manual Exposure
Tv( Shutter Speed ) 1197
Av( Aperture Value ) 0.0
Metering Mode Evaluative Metering
ISO Speed 1600
Auto ISO Speed OFF
Image Size 5184x3456
Image Quality RAW
Flash Off
FE lock OFF
White Balance Mode Custom
AF Mode Manual focusing
Picture Style Faithful
Sharpness 0
Contrast 0
Saturation 0
Color tone 0
Color Space Adobe RGB
Long exposure noise reduction 0:Off
High ISO speed noise reduction 3:Disable
Highlight tone priority 0:Disable
Auto Lighting Optimizer 3:Disable
Peripheral illumination correction Disable
File Size 29342 KB
Dust Delete Data No
Drive Mode Single shooting


Post Extras: Print Post   Remind Me!   Notify Moderator  
bilgebayModerator
Post Laureate
*****

Reged: 11/06/08

Loc: Turkiye - Istanbul and Marmari...
Re: 5 or 30 minute exposures for narrowband? new [Re: bilgebay]
      #5656770 - 02/01/13 05:00 PM

One last frame from yet another camera, QSI683.... Luminance only, properly calibrated with bias, dark and flat frames.

13x 600 seconds



Hires


Post Extras: Print Post   Remind Me!   Notify Moderator  
pfile
Post Laureate


Reged: 06/14/09

Re: 5 or 30 minute exposures for narrowband? new [Re: Inverted]
      #5656878 - 02/01/13 06:02 PM

Quote:


Edit: by the way, just to clarify, I'm asuming there is a higher ratio of pattern noise to random noise. I think that was obvious, it re-reading I see how it could be interpreted that I am saying we are calibrating out random noise. Of course you can't. That's why I keep saying if we have low read noise AND good calibration. As I said I sti don't have a sense o the magnitudes. That is what I'm interested in though. Hopefully once I have a chance to read Craig's article, that will be more clear.




here is how you quantify the read noise.

make a really good master bias frame.
subtract the master bias frame from a bias sub.
the result is the read noise in that bias sub.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Mike7Mak
Pooh-Bah
*****

Reged: 12/07/11

Loc: New York
Re: 5 or 30 minute exposures for narrowband? new [Re: Alph]
      #5656900 - 02/01/13 06:15 PM

Quote:

...however noise never goes down. It is the S/N that goes up when stacking.



Doh, ok that's the bit I was missing.

So since the corresponding signal (bias/pedestal) that created the read noise is subtracted out the noise is left behind to accumulate. Which is why covering it with sky noise is the only way to eliminate its effect. Is that close?

Thanks.

Edited by Mike7Mak (02/01/13 06:17 PM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
Alph
Carpal Tunnel


Reged: 11/23/06

Loc: Melmac
Re: 5 or 30 minute exposures for narrowband? new [Re: Mike7Mak]
      #5656964 - 02/01/13 06:59 PM

Quote:

So since the corresponding signal (bias/pedestal) that created the read noise is subtracted out the noise is left behind to accumulate. Which is why covering it with sky noise is the only way to eliminate its effect. Is that close?




Yes, you are pretty close.
The random residue of readout noise and/or dark current noise is always left behind and covering it with the sky glow or a nebula is the only way to hide it from our eyes


Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: pfile]
      #5657003 - 02/01/13 07:21 PM

Quote:

Quote:


Edit: by the way, just to clarify, I'm asuming there is a higher ratio of pattern noise to random noise. I think that was obvious, it re-reading I see how it could be interpreted that I am saying we are calibrating out random noise. Of course you can't. That's why I keep saying if we have low read noise AND good calibration. As I said I sti don't have a sense o the magnitudes. That is what I'm interested in though. Hopefully once I have a chance to read Craig's article, that will be more clear.




here is how you quantify the read noise.

make a really good master bias frame.
subtract the master bias frame from a bias sub.
the result is the read noise in that bias sub.




Thanks that makes sense now and looking back, I see Frank mentioned that the FPN is technically considered separate from the read noise. I think in practice though, it seems if people are taking insufficient bias frames, then a significant amount of FPN will remain in the bias subs. I think this may look like random noise, until it converges into a fixed pattern though. Either way, it could be worse than the poison noise, if your not modeling the FPN well as fixed noise tends to be more noticeable than random, even if it "looks" roughly random. It would also seem that it would make the read noise look higher than it is, if measured how your describing though. Again unless I'm missing something...


Post Extras: Print Post   Remind Me!   Notify Moderator  
vpcirc
Post Laureate


Reged: 12/09/09

Loc: Merced CA
Re: 5 or 30 minute exposures for narrowband? new [Re: Inverted]
      #5657130 - 02/01/13 09:04 PM

I just had a long discussion with a very advanced imager who shared a few of his thoughts and experiance.
1. Science says one thing reality says another.
2. Exposures times necessary can change from night to night and can be dramatically different depending on your equipment so there is no easy answer to "how long and how many"
3. There is no such thing as too long with narrowband
4. Longer images and more of them as a general rule will always produce higher contrast and more resolution than shorter images with the same total time.
5. With modern rejection methods, even bad data is usuable to a point.
6. It never hurts to add in extra data and the limit on how much is not proven.
7. It's now very common for luminance data for example to contain 20 hours of data.

I eat a little crow stating the 16 frame rule, but I do feel correct in the philosophy that total time does not equal total time. The longest images you can take with your given equipment to reach that total time will achieve far great results than stacking a bunch of short exposures.


Post Extras: Print Post   Remind Me!   Notify Moderator  
korborh
scholastic sledgehammer
*****

Reged: 01/29/11

Loc: Arizona
Re: 5 or 30 minute exposures for narrowband? new [Re: vpcirc]
      #5657200 - 02/01/13 09:50 PM

I disagree with #1: Science is based on observations from reality (nature).

Regarding #3, there are lots of other reasons to avoid very long subs (tracking issues, focus shift, star saturation, airplanes, cosmic rays etc.). Question is for a particular equipment, shorter exposures may indeed be better than longer (for the final outcome) so going too long with narrow-band will not always apply.

#4 cannot be always true (true if read noise is too large). Not sure about resolution - like mentioned above, long exposures have more chances of being messed up (tracking, focus etc.) that can make resolution worse.

Edited by korborh (02/01/13 10:08 PM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
David PavlichAdministrator
Transmographied
*****

Reged: 05/18/05

Loc: Mandeville, LA USA
Re: 5 or 30 minute exposures for narrowband? new [Re: korborh]
      #5657327 - 02/01/13 10:59 PM

Like I said, you expose for as long as your equipment and skyglow (NB being the exception) will allow or to the point of diminished returns.

David


Post Extras: Print Post   Remind Me!   Notify Moderator  
Alph
Carpal Tunnel


Reged: 11/23/06

Loc: Melmac
Re: 5 or 30 minute exposures for narrowband? new [Re: vpcirc]
      #5658696 - 02/02/13 06:30 PM

Quote:

I eat a little crow stating the 16 frame rule,




It wasnít really your statement. You simply quoted someone you thought was an authority on the subject. This goes to show what you read in hobby magazines has to be taken with a grain of salt.


Quote:

but I do feel correct in the philosophy that total time does not equal total time. The longest images you can take with your given equipment to reach that total time will achieve far great results than stacking a bunch of short exposures.




It depends. If the Poisson noise dominates the readout noise then stacking will be statistically equivalent to one longer exposure. It is simple arithmetic. If your anonymous SME knows something what scientists donít know then you should encourage him to publish a paper in a professional journal.


Post Extras: Print Post   Remind Me!   Notify Moderator  
vpcirc
Post Laureate


Reged: 12/09/09

Loc: Merced CA
Re: 5 or 30 minute exposures for narrowband? new [Re: Alph]
      #5658748 - 02/02/13 07:16 PM

I should clarify that. When he said total time doesn't equal total time, 10 hours of 10 min shots will not show the same resolution and contrast as 10 hours of 15 min shots. I don't think he meant from a noise perspective.

Post Extras: Print Post   Remind Me!   Notify Moderator  
cn register 5
scholastic sledgehammer


Reged: 12/26/12

Re: 5 or 30 minute exposures for narrowband? new [Re: vpcirc]
      #5658813 - 02/02/13 07:56 PM

Why? What evidence do you, or this unknown prophet, have?

Post Extras: Print Post   Remind Me!   Notify Moderator  
orion69
scholastic sledgehammer


Reged: 05/09/10

Loc: Croatia
Re: 5 or 30 minute exposures for narrowband? new [Re: cn register 5]
      #5658839 - 02/02/13 08:09 PM

Quote:

Why? What evidence do you, or this unknown prophet, have?




Why don't you try if you don't believe. Of course that depends of the sky quality. If sky is poor and your subs are too long noise would prevail.

Imagine this: you have Astrodon 3nm NB filter and shoot one object for 6 hours. First you take 180x1 min and then 6x30min subs. If sky is fair do you really think that those series of subs would produce image of same quality?


Post Extras: Print Post   Remind Me!   Notify Moderator  
vpcirc
Post Laureate


Reged: 12/09/09

Loc: Merced CA
Re: 5 or 30 minute exposures for narrowband? new [Re: orion69]
      #5658895 - 02/02/13 08:43 PM

If you have any questions in regards to narrowband I suggest contacting Don Goldman at Astrodon. He is an optics expert and would be able to answer your questions much better than I from a scientific standpoint. I'm just a struggling imager trying to figure this all out myself.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: orion69]
      #5658919 - 02/02/13 08:59 PM

Quote:

Quote:

Why? What evidence do you, or this unknown prophet, have?




Why don't you try if you don't believe. Of course that depends of the sky quality. If sky is poor and your subs are too long noise would prevail.

Imagine this: you have Astrodon 3nm NB filter and shoot one object for 6 hours. First you take 180x1 min and then 6x30min subs. If sky is fair do you really think that those series of subs would produce image of same quality?




I wish the sky around here was fair. Last night I had to throw out 20% of my subs due to passing clouds, that would have really sucked if a cloud passed through a 180 minute exposure. Also, cosmic rays, satalites etc.. are a problem everywhere, as are moments of bad seeing IME. Further not everyone's equipment will work consistently well for that long of an exposure. The good thing though is camera noise is becoming lower and lower and less of an issue if stacking images (and many technuqies such as sigma stacking and dithering may not work well without at least a reasonable amount of subs, maybe somewhere around 10 or so subs...) From the website,Mike posted as a reference, therer are examples using real world data, which showed cutting the exposure time in half, may only increase the required total integration time by a measly 10%. the irony is they even made recommendations similar to the ones people have been making in this thread.http://www.hiddenloft.com/notes/SubExposures.pdf

No one doubts that under perfect circumstances with perfect tracking and perfect skies going long with nb will improve the image some. In the real world things aren't perfect, I think that is all anyone has said in this thread.

(Also BTW, I think a big part of the issue hasn't been that Mike is saying it's better to go long, everyone is entitled to their opinion, and in many cases I'm sure that's true. I believe much of the discussion stared via his insistence that you have to go long and anyone that says otherwise is apparently wrong. I know I'm not right all the time, that's why discussions like this are useful, I really learned a lot once I finally put two and two together, but if you approach it with your mind made up already, then it all doesn't matter...)


Post Extras: Print Post   Remind Me!   Notify Moderator  
vpcirc
Post Laureate


Reged: 12/09/09

Loc: Merced CA
Re: 5 or 30 minute exposures for narrowband? new [Re: Inverted]
      #5658956 - 02/02/13 09:29 PM

That would be a misunderstanding. I said in my case John Smith and others all told me I needed to go longer with my camera and my dark skies after looking at my data. Longer images in narrowband, from what I'm being told, will produce far better contrast and resolution. The same applies to broadband but has more limits based on equipment skies etc. the disagreement has been over the benefits of long vs short. I know for example Don Goldman shoots his narrowband subs at 30 min. Why, because even though he could easily shoot far longer but he doesn't want to lose images to satellite trails etc. The majority of us firmly believe shooting longer duration images will produce better images. Too often I see new imagers wanting to believe its just as good to take short images because when they try to shoot a 10 min sub they have star trails and other issues because they haven't learned or its too much of a pain to properly polar align their mount. I can speak from experiance, it wasn't that long ago that was me.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: vpcirc]
      #5659055 - 02/02/13 10:45 PM

No worries, I've learned a lot from the resultant discussion, which is all I really care about. I'm not really sure what the continued debate is about then though but that is another matter.

Post Extras: Print Post   Remind Me!   Notify Moderator  
freestar8n
Post Laureate
*****

Reged: 10/12/07

Re: 5 or 30 minute exposures for narrowband? new [Re: Inverted]
      #5659325 - 02/03/13 03:26 AM

I have put some plots together to help convey this stuff in pictures, which maybe I should have done a long time ago. This is all based on the core, simple noise model used by many web sites to calculate the impact of read noise - so if I made a mistake I welcome corrections - but if people have a problem with the noise model itself - I am simply using it, not saying it is perfect.

For people who want to do a similar calculation based on their own parameters, please take note of the following:

1) Measure your own camera parameters (gain/read noise) because they may differ from spec. It is easy to do and you don't need master frames - just a few biases and lights.

2) If you bin, measure the gain and read noise in binned mode also because the results may not be what you expect.

3) To measure sky background, make sure you measure with a calibrated frame so you don't include the bias offset in the sky background. Checking the pixel value of a raw light vs. a raw bias will help confirm that the delta you measure is due to sky background. You want to measure with a "black" area of the light.

The main points of the noise model are:

1) The dominant noise is due to sky background and read noise

2) The sky signal increases linearly with exposure time, and since it is Poisson the sky noise goes as the sqrt of exposure time

3) Read noise only contributes once when each exposure is read, and is independent of exposure time

4) The sky and read noise add in quadrature rather than a simple sum. So if they are both 10, the combined noise is 14. If one is 10 and the other is 20, the combined noise is 22.3.

The following plots are only of the background noise (sky+read) - and they don't show signal at all. SNR is not described here - only background noise. What is important here, and in calculating subexposure, is the *relative impact of read noise* on the total noise. I show this as the ReadImpact value, which is the added impact of read noise divided by the sky noise. There is no hard value for what is acceptable here - but if the sky noise is big, there is little to be gained by reducing the read noise impact to a very small value.

Frank


Post Extras: Print Post   Remind Me!   Notify Moderator  
freestar8n
Post Laureate
*****

Reged: 10/12/07

Re: 5 or 30 minute exposures for narrowband? new [Re: freestar8n]
      #5659328 - 02/03/13 03:30 AM Attachment (17 downloads)

Here is a plot showing a high noise camera at a bright site and the benefit of increasing the sub exposure time from 100s to 300s. The blue curve is the sky background noise and the black arrows are the read noise being added at the end of each subexposure. The red curve shows the combined noise. The gap from the blue to the red curve is the penalty you get from read noise. If this is big compared to sky noise, then you are being hurt by read noise and should use longer exposures. But if the sky noise is big already - longer exposures won't help much.

In this case, longer exposures are a help, and lower the total noise.

Frank


Post Extras: Print Post   Remind Me!   Notify Moderator  
freestar8n
Post Laureate
*****

Reged: 10/12/07

Re: 5 or 30 minute exposures for narrowband? new [Re: freestar8n]
      #5659331 - 02/03/13 03:34 AM Attachment (11 downloads)

Here is the same situation, but at a dark site. Note that longer exposures have a much greater impact on reducing the noise, so you have a bigger motivation to go longer. But in each case, the total noise is less than the noise at a bright site. This helps convey that a dark site you want to go longer - but even if you don't, you will benefit from the dark site.

Frank


Post Extras: Print Post   Remind Me!   Notify Moderator  
freestar8n
Post Laureate
*****

Reged: 10/12/07

Re: 5 or 30 minute exposures for narrowband? new [Re: freestar8n]
      #5659333 - 02/03/13 03:37 AM Attachment (11 downloads)

Here is one more, showing a high read noise camera at a dark site, vs. a low read noise camera at a bright site - with the same exposure. Note that the total noise is the same - but you would have been much better off with the low noise camera at the dark site.

Frank


Post Extras: Print Post   Remind Me!   Notify Moderator  
cn register 5
scholastic sledgehammer


Reged: 12/26/12

Re: 5 or 30 minute exposures for narrowband? new [Re: freestar8n]
      #5659463 - 02/03/13 07:37 AM

Thanks for this Frank, your bright sky, high noise example seems to be fairly close to what I have. Going from 100 to 500 secs has an obvious good effect but it looks as if going much longer than this is at the point of diminishing returns because there's not much room between the red and blue lines and that's all that can possibly be gained by exposing for longer.

The dark sky results show that increasing exposure beyond the 5 minutes of your example would be useful, and this explains why the experts at dark sky sites insist that longer exposures are beneficial, they are much more so for them.

I wonder, how long professional astronomers expose for, they have perfect equipment at very dark sites and are trying to get the most out of very dim objects. Does anyone know how long the individual exposures in the Hubble deep field were?

As for the claim that increasing exposure length, even from from 10 to 15 minutes, makes a difference to resolution and contrast we have been given no evidence that this is so, other than claims that experts say so. I can't see any rational reason for this so am not prepared to take the word of experts without any reason.

Chris


Post Extras: Print Post   Remind Me!   Notify Moderator  
freestar8n
Post Laureate
*****

Reged: 10/12/07

Re: 5 or 30 minute exposures for narrowband? new [Re: cn register 5]
      #5659482 - 02/03/13 07:56 AM Attachment (5 downloads)

Thanks - I think that plot does probably match your situation - especially since you binned - which effectively increases your sky flux, or effectively halves your f/number. But it may also increase your read noise so I recommend measuring it.

Here is another plot with a longer time scale of 2 hours, showing that same 300s subexposure plot (5 minutes) compared to 20 minute subexposures. The noise is heavily dominated by sky and the increase from 5m to 20m only drops the noise by 10, from 98 to 88. It's not clear to me this improvement would even be visible after stretching - but even if it was visible - you would just stretch a bit less and it still wouldn't have much tangible effect.

Frank


Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: freestar8n]
      #5659518 - 02/03/13 08:44 AM

Thanks Frank great work! It really helps to visualize the models impact, rather than just talk about parameters. I'm curious how much changing the read noise would effect the dark site and bright sites. What if we had a 5e- or 3e- camera? Surveying cameras with listed read noise, 10e seems to be the high end for any camera on the market today. The average looks closer to 7 and even relatively inexpensive Atiks median is about 5, and go down to 3. Heck, if you really want, I think Apogee has some high end cameras <2e-

Post Extras: Print Post   Remind Me!   Notify Moderator  
freestar8n
Post Laureate
*****

Reged: 10/12/07

Re: 5 or 30 minute exposures for narrowband? new [Re: Inverted]
      #5659531 - 02/03/13 08:52 AM

Thanks - yes, I sure hope this clarifies the discussion and key points.

As for lower read noise - as I've been saying - it all depends on your camera, optics, and site. You can turn a "bright" site into a dark one just by increasing the f/ratio or using a narrower filter. It's all about the actual sky flux you are receiving and the actual read noise you have.

The equations involved are very simple - but there are some confusing sub-topics. The most confusing one is "optimal sub-exposure length." A long one may indicate a problematic situation - or a very good one.

I mainly encourage people to take some careful measurements if they are interested in this stuff - and be sure to do all calculations in electrons. It is a big no-no to take the square root of adu to find the noise. No, no, no. Must be converted to electrons first.

Frank


Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: freestar8n]
      #5659545 - 02/03/13 09:05 AM

Quote:


As for lower read noise - as I've been saying - it all depends on your camera, optics, and site. You can turn a "bright" site into a dark one just by increasing the f/ratio or using a narrower filter. It's all about the actual sky flux you are receiving and the actual read noise you have.





Yes, those parameters are interesting to visualize too. That makes sense from the posted models. I was tempted to try to plug in the models and graph the numbers myself, but didn't get around to it yet.

I don't mean to give Frank homework either by the way, but if he happens to play around with the numbers and sees anything interesting let us know! Or let me know what model you are using and I may play around with it.


Post Extras: Print Post   Remind Me!   Notify Moderator  
freestar8n
Post Laureate
*****

Reged: 10/12/07

Re: 5 or 30 minute exposures for narrowband? new [Re: Inverted]
      #5659800 - 02/03/13 11:34 AM Attachment (14 downloads)

Well for now I'll provide my own worked example using a Pacman Ha imaging session. This is C11 at f/5.7 on cge-pro with OAG and, of course, MetaGuide in 15m exposures with 2.0" fwhm. SXVF-H9 Sony ccd with measured values of gain = 0.4 e/adu and read noise of 13.5e. It is supposed to be 12e, but unfortunately is 13.5e. I measured a sky flux of 0.18e/s from a calibrated sub, and bias frames have a value around 750 adu, so it is a huge offset that must be removed. The Ha filter is Baader 7nm and I have a fair amount of light pollution - so I have neither a low read noise camera, nor a dark sky with narrow filter.

The question is - is 15m long enough or would I get a big win going much longer? I can guide longer, but I'm mainly concerned about losing subs at 30m - which would defeat any win I would get from reducing read noise. So here is the comparison between 15m subs and 30m subs for a session of about 3 hours:


Post Extras: Print Post   Remind Me!   Notify Moderator  
freestar8n
Post Laureate
*****

Reged: 10/12/07

Re: 5 or 30 minute exposures for narrowband? new [Re: freestar8n]
      #5659813 - 02/03/13 11:40 AM Attachment (12 downloads)

It's clear that I'm hurting from both sky glow and read noise - but there isn't much win going from 15m subs to 30m, and in fact with only 3 hours allocated I would have many fewer subs to average. If I used a narrower filter that would help - but then read noise would really dominate. So - as always - it's good to have dark site, low read noise, and narrow filter - but it's also good to make use of what you have in an optimal way for the given conditions - and 15m exposures worked ok here. Here is a close up of a simple average of 11 subs with some DDP stretch. It has some noise in the darkest regions, but not too bad. It is at 50% size - the original being 0.83"/pixel.

Frank


Post Extras: Print Post   Remind Me!   Notify Moderator  
Peter in Reno
Postmaster
*****

Reged: 07/15/08

Loc: Reno, NV
Re: 5 or 30 minute exposures for narrowband? new [Re: freestar8n]
      #5659986 - 02/03/13 01:20 PM

Bottom line is NB wins for bright site. Look at my Bubble Nebula in "Peter's Galleries" in my signature. Ha/Oiii were 15 minutes subs and Sii was 30 minutes subs. My Atik 460EX camera has very low read noise and it really helps. My web site shows Hubble Palette, Bi-color Ha/Oiii/Oiii, HaRGB, RGB using pseudo Luminance extracted from RGB. I could not use Luminance because even at 10 minutes sub-exposures, the stars were badly bloated and the sky background was too bright. The histogram for Luminance showed the left side of graph shifted too far to the right. My calibrated processes use Bad Pixel Mapping instead of dark subtraction. I get better results with BPM.

When I imaged with Sii filter, I started at 15 minutes. I examined the first image downloaded from my camera and decided to image at 30 minutes subs for the rest of the night. The calibrated 30 minute sub appear to look better than calibrated 15 minute sub.

I uploaded calibrated Sii with BPM, flat and bias of 15 and 30 minutes subs for you to evaluate at:

https://www.dropbox.com/s/ot1ftleoov4cale/Bubble_SII_15minutes_calibrated.fit

https://www.dropbox.com/s/7idi0rwnnm690nl/Bubble_SII_30minutes_calibrated.fit

Note that each image is not stacked, just one calibrated sub of each. I don't have fancy tools like Frank does to do measurements so when you have time, please look at my sample subs and report back if you can. I appreciate it. I think it was worthwhile imaging at 30 minutes over 15 minutes. Fortunately not one single airplane or satellite went by all night! I live directly under the airplanes' flight paths. Yes, my house vibrates when planes fly over.

This is probably the best thread I've read in a long time. Thanks to OP for starting this thread. Also, it turned out that I really screwed up the calibration of Sii for 30 minute subs of Bubble Nebula when I was processing Hubble Palette. Now I have to re-process it all over again but that's okay and best of all it's fun.

Thanks,
Peter


Post Extras: Print Post   Remind Me!   Notify Moderator  
korborh
scholastic sledgehammer
*****

Reged: 01/29/11

Loc: Arizona
Re: 5 or 30 minute exposures for narrowband? new [Re: Peter in Reno]
      #5660040 - 02/03/13 01:57 PM

Frank, great discussion and thanks for taking the effort to plot graphs and explain the various scenarios. It has really given a crisp and clear picture of the trade-offs based on equipment/conditions.

I will doing some more analysis on my recent narrowband images. I am currently going 30min subs with 5nm Ha filter from bright sky and low read noise (~3.5e-) camera. Seems like I may be better off with something like 20mins with less risk of losing subs to focus-shift and UFO's. Need to try in the next imaging session.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Alph
Carpal Tunnel


Reged: 11/23/06

Loc: Melmac
Re: 5 or 30 minute exposures for narrowband? new [Re: cn register 5]
      #5660175 - 02/03/13 03:12 PM

Quote:

The dark sky results show that increasing exposure beyond the 5 minutes of your example would be useful, and this explains why the experts at dark sky sites insist that longer exposures are beneficial, they are much more so for them.

I wonder, how long professional astronomers expose for, they have perfect equipment at very dark sites and are trying to get the most out of very dim objects. Does anyone know how long the individual exposures in the Hubble deep field were?




First of all, professional astronomers are not interested in pretty pictures. They have no scientific value.
Secondly, professional astronomers are not concerned how the image background looks like. All what they care is the S/N of the object of interest. Somehow the object S/N is completely missing from the Frankís presentation. I am afraid this will only add to the misconception that you need sky glow in your images.


Quote:

As for the claim that increasing exposure length, even from from 10 to 15 minutes, makes a difference to resolution and contrast we have been given no evidence that this is so




There is nothing fundamentally wrong with that statement. A higher S/N results in a higher contrast that can be interpreted as a higher spatial resolution.
As a related matter that has not been mentioned in this thread. A brighter sky requires more total exposure time than a dark sky to achieve the similar contrast.


Post Extras: Print Post   Remind Me!   Notify Moderator  
freestar8n
Post Laureate
*****

Reged: 10/12/07

Re: 5 or 30 minute exposures for narrowband? new [Re: Peter in Reno]
      #5660345 - 02/03/13 04:56 PM

Hi Peter-

I'd be happy to look at your numbers - but I do need gain and read noise for your camera. If you can measure them that would be great.

I didn't mention it - but I don't use dark subtraction either with my Sony ccd. If the dark current is low and there is no pattern noise in the dark except for hot pixels - then it's just a hot pixel map and it doesn't help to subtract.

Thanks,
Frank


Post Extras: Print Post   Remind Me!   Notify Moderator  
freestar8n
Post Laureate
*****

Reged: 10/12/07

Re: 5 or 30 minute exposures for narrowband? new [Re: korborh]
      #5660355 - 02/03/13 05:00 PM

Thanks Korhorh-

I have appreciated your positive comments in this thread - and those from others. 3.5e sounds great - and would nicely complement the rather high noise data I posted here.

Thanks,
Frank


Post Extras: Print Post   Remind Me!   Notify Moderator  
Peter in Reno
Postmaster
*****

Reged: 07/15/08

Loc: Reno, NV
Re: 5 or 30 minute exposures for narrowband? new [Re: freestar8n]
      #5660492 - 02/03/13 06:05 PM

Hi Frank,

Try these numbers I found from another person's Atik 460EX camera:

Gain = 0.26
Readout Noise = 4.79e-
Total System Noise = 5.10e-

Temperature was at -15C. My images were at 0C. It's my understanding that temperature colder than 0C makes small difference for Sony CCD cameras as compared to Kodak CCDs.

Atik 460EX Factory spec:
Read Noise: 5e-

Atik does not specify the gain for Atik 460EX. SX H694 specify the System Gain of 0.3 electrons per ADU.

My Bias frames have minimum of 300 ADU, Mean of 335 ADU and maximum of 365 ADU.

Thanks,
Peter

Edited by Peter in Reno (02/03/13 06:09 PM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
freestar8n
Post Laureate
*****

Reged: 10/12/07

Re: 5 or 30 minute exposures for narrowband? new [Re: Peter in Reno]
      #5661128 - 02/04/13 04:27 AM Attachment (6 downloads)

Hi Peter-

It looks like your 15m has a background level of about 70 adu and 30m has 140 adu so the sky background ends up at 0.02e/s using gain 0.26. With read noise of 4.8e I generated the plots below. You have a much darker background but your read noise is also much lower than mine - so 15m is pretty good. 30m helps a bit, but still doesn't get you down to the actual sky background.

Regarding SNR - the noise models I'm referring to are exactly that - a calculation of the fundamental background noise in an image. This is important both for aesthetic imaging, since it determines how noisy the dark regions look, and for scientific work, since it places a limit on the SNR of a given object. The background noise is well defined, but the SNR depends entirely on which signal you are interested in. In a given image, the SNR will vary from 0 in an area of pure background, to perhaps hundreds in a bright patch of nebulosity.

The other complication is that the nebulosity signal itself is Poisson, so it has intrinsic shot noise that affects the SNR. So - a faint signal will have SNR limited by the background and:

SNR(faint) = SFaint/NBkg

an intermediate signal will have a mixture:

SNR(intermediate) = SMed/sqrt(NBkg^2+SMed^2)

and a very strong signal will just be limited by its own shot noise:

SNR(High) = sqrt(SHigh)

- where all these values are electrons - not adu.

Frank


Post Extras: Print Post   Remind Me!   Notify Moderator  
freestar8n
Post Laureate
*****

Reged: 10/12/07

Re: 5 or 30 minute exposures for narrowband? new [Re: freestar8n]
      #5661129 - 02/04/13 04:32 AM

Pater -one other note: you should probably study the actual dark current you are getting at that temperature - by comparing 15m darks and 30m darks. If there is, in fact, some dark current causing the background signal, then you might have a way to reduce it - as opposed to sky background that you are stuck with.

Frank


Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: freestar8n]
      #5661258 - 02/04/13 08:11 AM

Neat, so, it looks like the ratio between the dark 15 and 30m subs is the same in both cases. However, the magnatude of the effect is almost 3.5x larger in the higer read noise case. That would seem to make sense, now if we could figure out exactly how much of an effect can become apparent after proccessing and all that good stuff

Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: Alph]
      #5661290 - 02/04/13 08:44 AM

Quote:

Somehow the object S/N is completely missing from the Frankís presentation. I am afraid this will only add to the misconception that you need sky glow in your images.





True, I think. Really it seems it would make more sense to normalize and then subtract the total noise from 1.

Of course, it's also worth mentioning all this seems to assume we've met nyquest criteria for sub exposure times (adjusting for quantization error) and n number of sub exposures is a reasonable number (i.e. > 10ish).


Post Extras: Print Post   Remind Me!   Notify Moderator  
freestar8n
Post Laureate
*****

Reged: 10/12/07

Re: 5 or 30 minute exposures for narrowband? new [Re: Inverted]
      #5661314 - 02/04/13 09:02 AM

The goal here is to provide a number that conveys the relative impact of read noise in a given imaging situation - and the potential benefit of longer exposures. It does this by expressing the net impact of read noise relative to sky noise. I think it serves the purpose well - but will still require some level of understanding by anyone who might use it, along with a sense of proportionality.

The combination of the numbers with the plots above will, I hope, make the whole thing even clearer.

I think there is already evidence that based on people reading about "optimum sub exposures" and various write ups on the web, some concluded that images would be noisier at a dark site because the darkness lets the noise "show" more. That is incorrect, and direct plots of the accumulated noise in different scenarios should show that clearly.

Frank


Post Extras: Print Post   Remind Me!   Notify Moderator  
vpcirc
Post Laureate


Reged: 12/09/09

Loc: Merced CA
Re: 5 or 30 minute exposures for narrowband? new [Re: freestar8n]
      #5661456 - 02/04/13 10:30 AM

You tell that to John Smith, Frank. I'd bet he'll disagree with you. You can email him at john@ccdware.com It's not that the same noise isn't there at a dark site, it just shows more because the sky glow is signal and that "signal" hides the noise at a more LP site. Of course that sky glow signal has other problems with it such as limiting exposures.

Post Extras: Print Post   Remind Me!   Notify Moderator  
shams42
Pooh-Bah
*****

Reged: 01/05/09

Loc: Kingsport, TN
Re: 5 or 30 minute exposures for narrowband? new [Re: vpcirc]
      #5661467 - 02/04/13 10:37 AM

The reason skyglow is undesirable is because it carries Poisson shot noise equal to the square root of the skyglow signal. Otherwise it could be dealt with by a levels adjustment.

If your sub has 1000e- skyglow, it will have 100e- noise due to skyglow. If you go to a dark site and now get 100e- skyglow in your sub, you will only have 10e- noise from skyglow.

Skyglow adds noise. Why else would one seek out a dark site?


Post Extras: Print Post   Remind Me!   Notify Moderator  
vpcirc
Post Laureate


Reged: 12/09/09

Loc: Merced CA
Re: 5 or 30 minute exposures for narrowband? new [Re: shams42]
      #5661505 - 02/04/13 10:49 AM

Dark sites allow much more of the signal from the DSO to reach the chip as the natural light is not being "polluted" by man made light sources. Contrast and resolution improve dramatically. But I'm done making my point. Frank is the expert here who is never wrong. You're right Frank and all the information I've been given is totally false.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Peter in Reno
Postmaster
*****

Reged: 07/15/08

Loc: Reno, NV
Re: 5 or 30 minute exposures for narrowband? new [Re: freestar8n]
      #5661578 - 02/04/13 11:30 AM

Quote:

Pater -one other note: you should probably study the actual dark current you are getting at that temperature - by comparing 15m darks and 30m darks. If there is, in fact, some dark current causing the background signal, then you might have a way to reduce it - as opposed to sky background that you are stuck with.

Frank




Hi Frank,

Thanks for the report.

I uploaded a Master Dark of 15 minutes (stack of 8 subs) and 30 minutes (stack of 6) if you are interested. I didn't need a lot of dark subs because Bad Pixel Mapping does not require many dark subs. Both of them at 0C.

https://www.dropbox.com/s/1wk8pyx9961qksj/Atik460EX_Master_Dark_15min.fit

https://www.dropbox.com/s/us8vna4npqdn2nq/Atik460EX_Master_Dark_30min_0C.fit

I also uploaded a Master bias of 30 subs at 0C:

https://www.dropbox.com/s/qqnxbg56i3me2k0/Atik460EX_Master_Bias.fit

Peter


Post Extras: Print Post   Remind Me!   Notify Moderator  
bilgebayModerator
Post Laureate
*****

Reged: 11/06/08

Loc: Turkiye - Istanbul and Marmari...
Re: 5 or 30 minute exposures for narrowband? new [Re: Peter in Reno]
      #5661589 - 02/04/13 11:37 AM

Quote:

Hi Frank,

Try these numbers I found from another person's Atik 460EX camera:

Gain = 0.26
Readout Noise = 4.79e-
Total System Noise = 5.10e-

Temperature was at -15C.

Thanks,
Peter




Hi Peter,

Do these figures belong to my 460EX?


Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: vpcirc]
      #5661593 - 02/04/13 11:38 AM

Quote:

You tell that to John Smith, Frank. I'd bet he'll disagree with you. You can email him at john@ccdware.com It's not that the same noise isn't there at a dark site, it just shows more because the sky glow is signal and that "signal" hides the noise at a more LP site. Of course that sky glow signal has other problems with it such as limiting exposures.





Sky-glow should converge to a fixed mean. In statistical terms, this "biases" the results (as described previously). In other words, a bright site, may have more "bias" (or "unwanted signal" if you prefer), so the "observed" signal to noise ratio may appear higher, due to this bias.

However, this is because, as mentioned a few pages back, we are only estimating the underlying signal to noise ratio, based on the observed data, which includes the bias (sky-glow). So, this estimate includes some "unwanted signal" from the sky-glow, and the variance added by the sky glow is proportionally less than the signal. So, SNR being the (signal mean) / sqrt(variance) may look higher if estimated from the observed data. However, really, if you were to properly adjust for the bias, it actually isn't. So, this is a very misleading result.

In some ways, it may make more sense, as Alph alluded to, to use to the contrast ratio. The contrast will be higher given the same amount of exposure time, from a dark site, than from a bright site. Also an unbiased SNR estimate would be higher given the same amount of exposure time, from a dark site, than from a light site. A biased SNR estimate may show that you "need" to go longer from a dark-site, to get the same SNR, as from a bright site, but this is about as mathematically relevant as showing 0 equals 1 and likewise, the solution relies on faulty logic.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Peter in Reno
Postmaster
*****

Reged: 07/15/08

Loc: Reno, NV
Re: 5 or 30 minute exposures for narrowband? new [Re: bilgebay]
      #5661608 - 02/04/13 11:51 AM

Quote:

Quote:

Hi Frank,

Try these numbers I found from another person's Atik 460EX camera:

Gain = 0.26
Readout Noise = 4.79e-
Total System Noise = 5.10e-

Temperature was at -15C.

Thanks,
Peter




Hi Peter,

Do these figures belong to my 460EX?




Hi Sedat,

I am not sure. I got it from "Measure the performance of your CCD camera!" thread started by Konihlav. Are they yours?

Peter


Post Extras: Print Post   Remind Me!   Notify Moderator  
freestar8n
Post Laureate
*****

Reged: 10/12/07

Re: 5 or 30 minute exposures for narrowband? new [Re: Inverted]
      #5661670 - 02/04/13 12:30 PM

I don't think there is any way to explain this stuff that will work for everyone - but clearly the existing methods aren't working perfectly. I think a focus on the noise itself makes perfect sense and will be understandable - and this background noise is completely distinct from whatever choice of signal is made.

It sounds like the main source of confusion you are alluding to is that subs look cleaner in light pollution simply because they are a uniform gray. But anyone with a little experience in this stuff is familiar with stretching an image to bring out the details - and that process will remove the "bias" you are referring to (sky glow signal) and reveal the true noise in the exposure due to sky glow shot noise and read noise - and there shouldn't be any doubt the one with light pollution is inherently more noisy even though the raw image looks more uniform gray.

In fact - many viewers will automatically stretch the sub - so the noise would be apparent.

Either way - I'm trying to indicate the actual amount of background noise in various scenarios - and I think a rising sqrt plot with bumps caused by read noise is a good way to do it.

Frank


Post Extras: Print Post   Remind Me!   Notify Moderator  
bilgebayModerator
Post Laureate
*****

Reged: 11/06/08

Loc: Turkiye - Istanbul and Marmari...
Re: 5 or 30 minute exposures for narrowband? new [Re: Peter in Reno]
      #5661733 - 02/04/13 01:00 PM

Quote:



Hi Sedat,

I am not sure. I got it from "Measure the performance of your CCD camera!" thread started by Konihlav. Are they yours?

Peter




My bad! I thought I have sent you my subs and you calculated those values from those whereas all I have sent to you was combined fits files.

I am getting old

Meanwhile, I owe these files to Pavel. As soon as get to my obs I will collect those files and send tehm to Pavel so that he has better values with his spreadsheet.

Cheers

Sedat


Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: freestar8n]
      #5661747 - 02/04/13 01:09 PM

Right, the dynamic range of the signal is effectively clipped in the LP scenario. Normalizing and rescaling to show relative signal proportions of the various scenarios, will show that the maximum signal strength is limited by sky-glow and the contrast is reduced i.e. the standard deviation of the pixels is lower. It's fine to show the total background noise though, as long as people understand that. You could reduce the background noise from the dark-site, by using a histogram transfer function to clip the range and achieve a similar result. I don't think it is anything that need be particularly mystical, but if we ignore the obvious then it's easy to misinterpret the results....

I think your method is great though and perfectly valid in full context.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Alph
Carpal Tunnel


Reged: 11/23/06

Loc: Melmac
Re: 5 or 30 minute exposures for narrowband? new [Re: Inverted]
      #5661872 - 02/04/13 02:11 PM

Quote:

Right, the dynamic range of the signal is effectively clipped in the LP scenario. Normalizing and rescaling to show relative signal proportions of the various scenarios, will show that the maximum signal strength is limited by sky-glow and the contrast is reduced i.e. the standard deviation of the pixels is lower. It's fine to show the total background noise though, as long as people understand that. You could reduce the background noise from the dark-site, by using a histogram transfer function to clip the range and achieve a similar result. I don't think it is anything that need be particularly mystical, but if we ignore the obvious then it's easy to misinterpret the results....





Let me use the statistician's lingo with you. Do you know why stars are not visible with naked eye in broad daylight? The reason is that the variance or the standard deviation of photon count from the sky background becomes extremely large (Poisson distribution). That makes the difference in the light levels between a star and the sky statistically insignificant. As a statistician you should know how to remedy the problem. You need to increase a sample size which in our case means increasing the total exposure time. A much more pragmatic solution though is to make a trip to a dark site to cut out the sky background from the equation.


Post Extras: Print Post   Remind Me!   Notify Moderator  
cn register 5
scholastic sledgehammer


Reged: 12/26/12

Re: 5 or 30 minute exposures for narrowband? new [Re: Alph]
      #5662148 - 02/04/13 05:18 PM

Or just - wait until it's dark

It's an interesting idea that daylight is a rather extreme form of light pollution.
We can't all go to a dark sky site, or even send our kit there. Working out our best - or least worst - options for coping with where we are and the equipment we have seems useful to me.

From what I can see rules of thumb are:
Use narrow band filters, especially if the site isn't dark because of light pollution, moonlight or twilight.
Longer exposures will help, especially at a dark site, but less so at a light sight. Exposures of 5 to 10 minutes will be good enough that other noise sources will dominate at a light site but at a dark site you will be significantly better off using longer exposures - 20 minutes or more. Even with shorter exposures you will be better off than at a light site.

The thing I find hard to believe is that light pollution will reduce the amount of light received from the deep sky objects. I don't see how that can happen and I think this is a misinterpretation of what the experts are saying. I think that what happens is that the light from the deep sky object stays the same but that the light pollution is added to this.
While the average value of the light pollution can be removed it has it's own noise and this noise is added to the image (in quadrature of course) and this makes the image more noisy. This reduces the signal to noise ratio because the noise has increased, not because the signal has decreased.

Hope this makes sense,

Chris


Post Extras: Print Post   Remind Me!   Notify Moderator  
Mike7Mak
Pooh-Bah
*****

Reged: 12/07/11

Loc: New York
Re: 5 or 30 minute exposures for narrowband? new [Re: cn register 5]
      #5662193 - 02/04/13 05:43 PM

Quote:

...and this noise is added to the image (in quadrature of course)...



Is there a quick explanation for what 'quadrature' means in this context? The only other time I've seen that word it involved feed systems for multi-element antenna arrays. That was long enough ago that I don't remember if I actually understood it then either.


Post Extras: Print Post   Remind Me!   Notify Moderator  
freestar8n
Post Laureate
*****

Reged: 10/12/07

Re: 5 or 30 minute exposures for narrowband? new [Re: Mike7Mak]
      #5662211 - 02/04/13 05:55 PM

The sum of the noise goes as the square root of the sum of the squares of all the independent noise terms.

It's just how independent errors accumulate. If you had aluminum rods that were 1m long with an error of 1mm, if you stacked two together the length would be 2m with an error of 1.4mm - rather than an error of 2mm.

So it isn't just an esoteric Poisson noise thing - it applies to stacked aluminum rods also.

Frank


Post Extras: Print Post   Remind Me!   Notify Moderator  
freestar8n
Post Laureate
*****

Reged: 10/12/07

Re: 5 or 30 minute exposures for narrowband? new [Re: Peter in Reno]
      #5662221 - 02/04/13 06:02 PM

Hi Peter-

I looked at your two darks at 0c, 15m and 30m, and I see a difference of about 14 adu or 5.6e in 15 minutes. That amounts to 0.006e/s. Although that is tiny, it is only about 1/3 of your total "sky" signal - i.e. your actual sky signal may be only 0.014e/s but it is added to a dark current signal of 0.006e/s. In terms of the noise model there is no difference between sky and dark current signal.

Note that your darks don't appear to have pattern noise - so this isn't about needing to subtract the darks. But if you can work at a lower temperature it might help. It would help because your noise is dominated by this sky/current term at your current f/ratio and filter width.

It's just another example where it's good to know the values in your system so you can determine what the dominant noise terms are, so you can address them with priority. If you can go cooler without too much difficulty, you might see a bigger win than with longer exposures.

Frank


Post Extras: Print Post   Remind Me!   Notify Moderator  
Mike7Mak
Pooh-Bah
*****

Reged: 12/07/11

Loc: New York
Re: 5 or 30 minute exposures for narrowband? new [Re: freestar8n]
      #5662231 - 02/04/13 06:06 PM

Thanks Frank, I'm glad you included the aluminum rods.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Peter in Reno
Postmaster
*****

Reged: 07/15/08

Loc: Reno, NV
Re: 5 or 30 minute exposures for narrowband? new [Re: Mike7Mak]
      #5662314 - 02/04/13 07:04 PM

Hi Frank,

Thanks for analyzing my data. It looks like I have a pretty good imaging equipment.

I can get the camera running colder if I can image under clear sky in the Winter. This Winter has been miserable with cloudy nights.

Peter


Post Extras: Print Post   Remind Me!   Notify Moderator  
korborh
scholastic sledgehammer
*****

Reged: 01/29/11

Loc: Arizona
Re: 5 or 30 minute exposures for narrowband? new [Re: Peter in Reno]
      #5662512 - 02/04/13 09:24 PM

Hi Peter, do you run your camera at 100% power unregulated? You may want to try that - maximize cooling and no darks.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Peter in Reno
Postmaster
*****

Reged: 07/15/08

Loc: Reno, NV
Re: 5 or 30 minute exposures for narrowband? new [Re: korborh]
      #5662552 - 02/04/13 09:50 PM

Well, Sony CCDs don't need to run at super cool temperature like Kodak CCDs run better at colder temperature. Plus I don't want the cooler to run too hard so that it would last longer.

The reason for 0C is that when I was imaging Bubble Nebula in late August and through September, the ambient temperature was quite high and it was pushing pretty hard at -5C, so I set it at 0C.

How can you tell if the cooler is running unregulated? I thought the cooler isn't running until the camera is connected to the PC and the capture software is running. I thought in order to cool the camera, you have to enable TEC and set the temperature set point otherwise the cooler may not be running. I just tested it with Nebulosity. When I turn on TEC, there is a temperature set point box and the cooler cools down to the set point temperature. I don't see a way to run it unregulated.

Peter


Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: Alph]
      #5662594 - 02/04/13 10:14 PM

Yes, that makes sense.

Edited by Inverted (02/04/13 10:45 PM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
shams42
Pooh-Bah
*****

Reged: 01/05/09

Loc: Kingsport, TN
Re: 5 or 30 minute exposures for narrowband? new [Re: Peter in Reno]
      #5662603 - 02/04/13 10:19 PM

Quote:

I don't see a way to run it unregulated.





The TEC cooler has no moving parts so I don't think it can wear out.

To run it unregulated, just specify the set point lower than it can achieve. It will run at 100% to try to get there.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Mike7Mak
Pooh-Bah
*****

Reged: 12/07/11

Loc: New York
Re: 5 or 30 minute exposures for narrowband? new [Re: Peter in Reno]
      #5662606 - 02/04/13 10:20 PM

Quote:

I don't see a way to run it unregulated.



I think he means pick a set point lower than the camera can possibly achieve so the cooler runs continuously at 100%.

Although I would hope the Atik cooling system can survive being run maxed out continuously, I kinda agree doing so seems a bit like abuse.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Peter in Reno
Postmaster
*****

Reged: 07/15/08

Loc: Reno, NV
Re: 5 or 30 minute exposures for narrowband? new [Re: shams42]
      #5662610 - 02/04/13 10:21 PM

I was referring to heat. The more you want TEC to cool, the hotter TEC runs, therefore can shorten the life. My SXVR-M25C camera body runs pretty hot when exceeding 100% so I had to lower the set point temperature so the TEC circuit does not run too hot.

I prefer to run at less than 100% and Sony CCDs do not need to run too cold thanks to their low noise CCD design at warmer temperature. Take a look at my sample darks and bias I uploaded earlier.

Peter

Edited by Peter in Reno (02/04/13 10:25 PM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
shams42
Pooh-Bah
*****

Reged: 01/05/09

Loc: Kingsport, TN
Re: 5 or 30 minute exposures for narrowband? new [Re: Inverted]
      #5662651 - 02/04/13 10:40 PM

Hi Frank,

Could you run the numbers for me? I wonder if going from 20min to 30min subs would be helpful.

Here's my data:

TOA-130 @ f/5.76 ~2.3" FWHM
QSI-583 with Astrodon 5nm Ha filter
suburban site

TEC cooler set to -20 deg C

Camera gain: 0.46 e-/ADU
Camera read noise: 9.6e-
Sky flux: 0.153 e- / s


Post Extras: Print Post   Remind Me!   Notify Moderator  
korborh
scholastic sledgehammer
*****

Reged: 01/29/11

Loc: Arizona
Re: 5 or 30 minute exposures for narrowband? new [Re: shams42]
      #5662686 - 02/04/13 11:16 PM

Hi Peter - unregulated as in setting a very low temperature so power is always 100% and temperature chip varies as per ambient. It will run at maximum cooling but not above 100%. Are you saying your camera ran above 100% and hot ? That is strange - I never saw that with Maxim on my Atik.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Peter in Reno
Postmaster
*****

Reged: 07/15/08

Loc: Reno, NV
Re: 5 or 30 minute exposures for narrowband? new [Re: korborh]
      #5662757 - 02/05/13 12:31 AM

Nebulosity does not show temp percentage capacity for my SX camera but it does for my Atik camera. For example if I set the SX camera temp set point to -20C and the CCD temp reading stays at -15C or warmer then it tells me I set at maximum cooling but the camera's body gets very warm. So I would set to -5C to -10C to make sure TEC is not abused.

Atik may have a better TEC design than SX since Atik's camera's body never gets warm. Atik cools a lot slower than SX but that's okay because once Atik's temperature hits the set point, it stays there very consistently.

Nebulosity displays over 100% (even over 200%) capacity when I select Atik native driver. It may be a bug in Nebulosity. I usually ignore this and pay attention to outside temperature and make sure delta temperature between outside and CCD does not exceed at least Atik spec of 25C.

Peter


Post Extras: Print Post   Remind Me!   Notify Moderator  
microstar
Pooh-Bah


Reged: 01/05/08

Loc: Canada
Re: 5 or 30 minute exposures for narrowband? new [Re: korborh]
      #5662766 - 02/05/13 12:38 AM

This has been a very interesting thread (in fact I nominated it for inclusion in the "Best Threads" thread). I think the insight I have taken away is that when trying to conceptualize SNR and noise I was heading down the wrong road to be focusing on the averages, it's all about the variance!
...Keith


Post Extras: Print Post   Remind Me!   Notify Moderator  
Alph
Carpal Tunnel


Reged: 11/23/06

Loc: Melmac
Re: 5 or 30 minute exposures for narrowband? new [Re: cn register 5]
      #5662796 - 02/05/13 01:08 AM

Quote:

From what I can see rules of thumb are:
Use narrow band filters, especially if the site isn't dark because of light pollution, moonlight or twilight.
Longer exposures will help, especially at a dark site, but less so at a light sight. Exposures of 5 to 10 minutes will be good enough that other noise sources will dominate at a light site but at a dark site you will be significantly better off using longer exposures - 20 minutes or more. Even with shorter exposures you will be better off than at a light site.




Yes, pretty much so. I would summarize like this. At a light site your exposure time is depended on the sky background and is pretty much the same for all objects. At a dark site especially with NB filters the exposure time is depended on the object of interest and it varies from an object to an object. A light site requires a much longer total exposure time than a dark site. The exposure times at a dark site should be longer than at a light site to make stacking equally efficient.

B.T.W. I donít know exposure times used by the HST. I am sure they vary from an object to an object. What I do know is that they stack hundreds of images.


Post Extras: Print Post   Remind Me!   Notify Moderator  
cn register 5
scholastic sledgehammer


Reged: 12/26/12

Re: 5 or 30 minute exposures for narrowband? new [Re: Alph]
      #5662869 - 02/05/13 03:13 AM

Quote:

B.T.W. I donít know exposure times used by the HST. I am sure they vary from an object to an object. What I do know is that they stack hundreds of images.



There's an Exposure Time Calculator online, it's surprising how short the times are, but then its a bigger scope and a really dark sky site - they worry about Zodiacal light!

Chris


Post Extras: Print Post   Remind Me!   Notify Moderator  
freestar8n
Post Laureate
*****

Reged: 10/12/07

Re: 5 or 30 minute exposures for narrowband? new [Re: cn register 5]
      #5662881 - 02/05/13 03:34 AM

Hi Chris-

I meant to reply about the HST times. I am too lazy to collect links, but I think this can be found on the web. The main points about exposure time are:

1) The HST might as well be on earth, with the earth spinning once every 90 minutes - because the horizon is still about 180 degrees. I think this limits the exposure of most of the sky to about 20 minutes.

2) There is still "sky" glow when the telescope gets within some distance of the earth's limb - so that reduces it below the possible 45 minutes.

3) There is a region in the north and south that can be exposed continuously but it isn't like on earth because the latitude of the hst goes up and down in its orbit.

4) When the hst was first launched it had giant solar panels that flapped badly every time it re-entered sunlight. That would mess up a long exposure. Since then, smaller panels have been installed and they have modeled the effect so it can be anticipated somewhat and corrected dynamically.

5) I think the original deep field used very long exposures in a northern region that could be exposed continuously - but the more recent even deeper field did not rely on such a location and used shorter exposures. I don't know the reasoning but my guess is the newer ccd is better and they wanted to minimize sky/background glow as much as possible - and shorter exposures "won". But I don't know that for sure. If it's true, it certainly fits in with the theme of this thread.

Frank


Post Extras: Print Post   Remind Me!   Notify Moderator  
freestar8n
Post Laureate
*****

Reged: 10/12/07

Re: 5 or 30 minute exposures for narrowband? new [Re: Peter in Reno]
      #5662885 - 02/05/13 03:42 AM

Hi Peter-

I didn't mean for you to stress your camera or anything. You are fine at 0c - it's just that you are in an unusual situation where no single noise term really dominates. The main sources are sky glow and read noise, but longer exposures will only drop the noise a little bit - and you do have dark current contributing. So if it is "no sweat" to run at a lower temperature, you would drop a few electrons in noise - similar to longer exposures. But it's not clear you would notice the difference, since you have such low background noise anyway.

Frank


Post Extras: Print Post   Remind Me!   Notify Moderator  
freestar8n
Post Laureate
*****

Reged: 10/12/07

Re: 5 or 30 minute exposures for narrowband? new [Re: freestar8n]
      #5662889 - 02/05/13 03:51 AM

With regard to running unregulated - if you have a camera with pattern noise then you want to take a perfect image of the dark current pattern so it will subtract away. Since the pattern noise depends on temperature, you need the darks and lights to be at exactly the same temperature so the pattern noise matches and will subtract away.

But if your camera doesn't have pattern noise and it only has dark current Poisson noise (plus some hot pixels) then there is no need to subtract the master dark - and therefore no need to run it at a fixed temperature. You just want the lights to have as little dark current noise as possible - and that means running them as cold as possible even if it varies a bit due to changing ambient or something.

This is the principle anyway - but if you end up cooking your camera then it is not a good idea. And other bad things may happen also. But in terms of the noise model - if you aren't subtracting a master dark, then you just want each light as cold as possible.

Frank


Post Extras: Print Post   Remind Me!   Notify Moderator  
freestar8n
Post Laureate
*****

Reged: 10/12/07

Re: 5 or 30 minute exposures for narrowband? new [Re: shams42]
      #5662896 - 02/05/13 04:06 AM Attachment (12 downloads)

Hi-

Here is what I get for your system. Again, not a huge win going from 15m to 30m - and dominated by sky more than read noise.

I guess I should relabel the "sky" signal since it also includes some amount of dark current - but yours is so cold that I guess it is more negligible.

Frank


Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: freestar8n]
      #5663051 - 02/05/13 08:28 AM

Skimming through The info on Wikipedia about the HST deep field, it sound like 11.5 days of 20min exposures. At this point, I'd just settle for a full cloud free night, an hour or so of integrate ion per night just isn't cutting it for NB

Post Extras: Print Post   Remind Me!   Notify Moderator  
CCDMan
member


Reged: 03/10/10

Loc: Oregon, USA
Re: 5 or 30 minute exposures for narrowband? *DELETED* new [Re: vpcirc]
      #5710501 - 03/03/13 11:52 AM

Post deleted by CCDMan

Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: CCDMan]
      #5710675 - 03/03/13 01:32 PM

Quote:

Quote:

I wish we could get Adam Block to dispel all the science fact from science fiction on here. He is a true wiz at this (just watch one of his tutorials). I think the average beginner to intermediate user is now totally confused with all the "theory" being spread here. David has the perfect solution attend AIC and listen to the John Smith and go tell him he doesn't know what he's talking about! Hey let's call the Myth Busters!




Well, all I can say is that I tend to pay much less attention to posts from people who are all theory with no links to actual images they have done. They may be right or they may be wrong but a post from someone who has links to actual images that they have taken with their own equipment and processed themselves (like Adam Block) carry much more weight in my mind than posts from someone, however well informed, that does not have the available online results of their efforts. Not to say they are wrong, they may be quite correct, but I don't know anything about them or their expertise if the evidence of that is not available to me. Anyone who does imaging (and I have done it for 20 years now) knows that the science is critical to the effort but is not, by itself, enough. Experience and creativity are also essential to produce a great image. I try to find the people that have all three and emulate them.

Yes, going to AIC is a great idea, I have been there a few times and gave a short talk myself once in the early years. Most of the bright lights of imaging can be found there at one time or another and it is a great place to bankrupt ones self on equipment! <g>





Well, I'm certainly no expert, so, I am not saying anything, just asking questions to understand better. What I would say is the people posting scientific info have posted info and links. This to me, is useful. The ones who are just name dropping have not (or when they have the links don't appear to support what they are claiming). This is not useful. So, far, all the name dropping stuff is hearsay as far as I can tell, i have no way of knowing if those being quoted are being quoted correctly in the correct context. Why would anyone go to AIC and tell John Smith he doesn't know what he's talking about, based on what someone posting info on the web says he said? There is no evidence that I can tell, that John Smith necessarily disagrees with what is being said here, there is only someone claiming to speak for him, saying so. How on earth can someone speak for someone else? If you have evidence that someone disagrees with something being said post a link or Something. Scientists produce scientifically accurate images, and much of the stuff being quoted appears to come from scientific resources. I want to produce accurate images, so, these are the resources that are important to me. If they are being misinterpreted, don't tell me "so and so told me X" so Y is wrong because so and so told me and he produces better images". That is not useful information unless it comes from the hoursess mouth and we are given the opportunity to ask questions and understand the context better. Perhaps being new I've just missed past discussions and don't have the full context, but, at least in that context, a lot of the commentary just sound ridiculous to me and to be honest, sounds more along the lines of trolling than anything else.

Edited by Inverted (03/03/13 01:47 PM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
freestar8n
Post Laureate
*****

Reged: 10/12/07

Re: 5 or 30 minute exposures for narrowband? new [Re: CCDMan]
      #5710682 - 03/03/13 01:40 PM

Quote:

Not to say they are wrong, they may be quite correct, but I don't know anything about them or their expertise if the evidence of that is not available to me.




Not sure who you are referring to - but I like to go by results that show minimal processing - including raw subexposures and gray scale. Some of my images are on the MetaGuide site - and include raw subexposures, with fwhm's in arc-seconds - etc. They are all captured with relatively humble equipment - c11 with reducer on cge/cge pro and small sx camera in light polluted, sea level skies - at fairly high resolution of around 0.8" per pixel.

So - yes - I also like to see results that people achieve with their equipment. In my case, I offer such results - along with the software that I wrote to achieve those results - free of charge.

I don't put my site in a signature because it may appear promotional - even though it is free - but people who follow my posts are aware of my site and results I have posted, and I occasionally refer to the site and particular images when it is relevant to the thread.

But if you have specific concerns about the plots I made and the conclusions I draw from them - they are based on the same noise model everyone is using, including users of the HST in the paper I cited. If you see an error or specific reason for disagreement - I welcome any corrections - either theoretical or based on raw images you have captured that indicate a failure of the underlying noise model on which the plots are based.

Frank


Post Extras: Print Post   Remind Me!   Notify Moderator  
Mike7Mak
Pooh-Bah
*****

Reged: 12/07/11

Loc: New York
Re: 5 or 30 minute exposures for narrowband? new [Re: CCDMan]
      #5710727 - 03/03/13 02:12 PM

Quote:

[Anyone who does imaging (and I have done it for 20 years now) knows that the science is critical to the effort but is not, by itself, enough.



In a discussion of the science it is most certainly enough. I don't need a picture to grasp 2+2=4.

The lack of a link to a couple pretty pictures does not disprove anything. Nor does it justify peanut gallery heckling (which is what you quoted, btw).

'That's not what so-and-so says' isn't a valid rebuttal in and of itself.


Post Extras: Print Post   Remind Me!   Notify Moderator  
vpcirc
Post Laureate


Reged: 12/09/09

Loc: Merced CA
Re: 5 or 30 minute exposures for narrowband? new [Re: Mike7Mak]
      #5710818 - 03/03/13 03:05 PM

I think most amateur imagers could care less about all the theory, links and long drawn out arguments that never end on on this subject and those who troll the posts looking for a chance to write a thesis. They just want the best advice. The "name dropping" is quoted only to show what the best are saying, and sharing from my own learning. Their statements are neither made up nor exaggerated. People can take their advice and learn what works best or do whatever they please. If you're happy with the results shooting 5 min subs that's all that really matters, not what anyone else thinks. Maybe they should start a new category "scientific discussions", then those who want discuss the measurement of noise can provide all those great resources. In the meantime I'm going to continue to learn, listen and emulate those whom are producing what I want to produce. So far it's brought me a long way in 3 years.

Post Extras: Print Post   Remind Me!   Notify Moderator  
shams42
Pooh-Bah
*****

Reged: 01/05/09

Loc: Kingsport, TN
Re: 5 or 30 minute exposures for narrowband? new [Re: CCDMan]
      #5710914 - 03/03/13 04:10 PM

Quote:

Well, all I can say is that I tend to pay much less attention to posts from people who are all theory with no links to actual images they have done.




I'd be more sympathetic to this view if the quality of one's images only depended on one's knowledge. Regardless of how much you know, the fact is that the highest quality work requires the highest quality equipment, optimal seeing, and dark skies. Just because someone may not own a $60k rig at New Mexico Skies does not mean that they don't know what they are talking about.


Post Extras: Print Post   Remind Me!   Notify Moderator  
vpcirc
Post Laureate


Reged: 12/09/09

Loc: Merced CA
Re: 5 or 30 minute exposures for narrowband? new [Re: shams42]
      #5710937 - 03/03/13 04:22 PM

Quote:

Quote:

Well, all I can say is that I tend to pay much less attention to posts from people who are all theory with no links to actual images they have done.




I'd be more sympathetic to this view if the quality of one's images only depended on one's knowledge. Regardless of how much you know, the fact is that the highest quality work requires the highest quality equipment, optimal seeing, and dark skies. Just because someone may not own a $60k rig at New Mexico Skies does not mean that they don't know what they are talking about.




You are 100% correct. Most of the people giving me the best advice are not remote. You can have the best equipment in the world, but if you don't understand how to get the most out of it what good is it? I've seen plenty of guys on here that can take modest equipment in their back yard and produce wonders. Sal Grasso is a perfect example, Bill Synder is another. BTW, I only have $22k invested and I still produce a lot of garbage, which is why I must learn from the best imagers if I ever want to hope to come close to what they do.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: vpcirc]
      #5711045 - 03/03/13 05:22 PM

Quote:

I think most amateur imagers could care less about all the theory




I have no idea. I've never done a poll. It is a pretty technical, scientific type hobby though, so, I would assume it attracts some that are interested in such. I am to an extent, although honestly I have a pretty technical job, so when I get home I usually don't actually want to think about that stuff too much. Once and a while when it is cloudy maybe. I am to an extent more interested in accurate images than "pretty pictures" though. That is just a personal preference. for me though, I'd rather understand the technology rather than just listen to someone who has great equipment, and/or is good at photoshop. There is something to understanding the equipment and how it works, and knowing that a faint background nebula is really nebula and not just smoothed out pattern noise for example. Perhapse it is not necessary to discuss such things on these threads, but some of us do appreciate the time others take to explain the stuff nevertheless.


Post Extras: Print Post   Remind Me!   Notify Moderator  
cn register 5
scholastic sledgehammer


Reged: 12/26/12

Re: 5 or 30 minute exposures for narrowband? new [Re: Inverted]
      #5711127 - 03/03/13 06:02 PM

Why is there such hostility to science?

Anyone can get good images by throwing money, exposure and dark skies at the problem. You don't need to understand anything if you have those advantages.

But this is of absolutely no use at all for those who don't have those advantages. Should we just give up because we don't have the resources to have a top of the line imaging system at somewhere like New Mexico Skies?

I think not. I think we should do what we can and take every advantage of our understanding of the science of image acquisition to extract the best we can from our limited resources.

If Cloudy Nights is not the place to exchange the sort of information that's useful in astro imaging - and from what I see it isn't - then can people suggest a better forum for discussing the technical aspects of Astro imaging

I'm not belittling the good work done by the technical people here but I'm fed up with the arguments from authority and ad hominem attacks.

Chris


Post Extras: Print Post   Remind Me!   Notify Moderator  
vpcirc
Post Laureate


Reged: 12/09/09

Loc: Merced CA
Re: 5 or 30 minute exposures for narrowband? new [Re: cn register 5]
      #5711154 - 03/03/13 06:16 PM

Quote:

Why is there such hostility to science?

Anyone can get good images by throwing money, exposure and dark skies at the problem. You don't need to understand anything if you have those advantages.

But this is of absolutely no use at all for those who don't have those advantages. Should we just give up because we don't have the resources to have a top of the line imaging system at somewhere like New Mexico Skies?

I think not. I think we should do what we can and take every advantage of our understanding of the science of image acquisition to extract the best we can from our limited resources.

If Cloudy Nights is not the place to exchange the sort of information that's useful in astro imaging - and from what I see it isn't - then can people suggest a better forum for discussing the technical aspects of Astro imaging

I'm not belittling the good work done by the technical people here but I'm fed up with the arguments from authority and ad hominem attacks.

Chris




Chris you are dead wrong there. It takes a lot more than equipment and skies to make a good image. You can have the fastest car on the track and still finish last if you don't know how to race. If it was so easy I wouldn't have to ask the teachers what I'm doing wrong. Learning from those with far more wisdom and experience than I seems like a pretty smart thing to do.


Post Extras: Print Post   Remind Me!   Notify Moderator  
korborh
scholastic sledgehammer
*****

Reged: 01/29/11

Loc: Arizona
Re: 5 or 30 minute exposures for narrowband? new [Re: cn register 5]
      #5711238 - 03/03/13 06:57 PM

Quote:



I'm fed up with the arguments from authority and ad hominem attacks.

Chris




They all come from just one CN member; just ignore these and continue this productive discussion.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Konihlav
Pooh-Bah


Reged: 03/05/09

Re: 5 or 30 minute exposures for narrowband? new [Re: korborh]
      #5711893 - 03/04/13 02:47 AM

Hi, "nice" thread of 12 pages. I haven't read any post except of the first one.

My answer is simple - for NB work, go as deep as you can go (10, 15, 20, ... 30?). The only exception is - extremely adverse conditions (full moon, snow all around reflecting light, SQM like 16 mag skies) AND/OR you are a owner of say Atik 460EXM or 314L+ (these cameras I thorougly tested and used and these two (or other with ICX674, ICX694, ICX285, ICX445 etc.) are really different beasts for NB work apart from the rest that I find useless for NB ;D but I am extreme guy wanting only the best for the particular task). In that case, you are OK with 10minutes no matter if you use 12nm wide or 3nm narrow filters. Period. No more important stuff to add on this one.

hope I was helpful for at least someone


Post Extras: Print Post   Remind Me!   Notify Moderator  
Pages: 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | (show all)


Extra information
12 registered and 11 anonymous users are browsing this forum.

Moderator:  Oldfield, droid, Scott in NC, bilgebay, WOBentley 

Print Thread

Forum Permissions
      You cannot start new topics
      You cannot reply to topics
      HTML is disabled
      UBBCode is enabled


Thread views: 7558

Jump to

CN Forums Home




Cloudy Nights Sponsor: Astronomics