Return to the Cloudy Nights Telescope Reviews home pageAstronomics discounts for Cloudy Nights members
· Get a Cloudy Nights T-Shirt · Submit a Review / Article

Click here if you are having trouble logging into the forums

Privacy Policy | Please read our Terms of Service | Signup and Troubleshooting FAQ | Problems? PM a Red or a Green Gu… uh, User

Astrophotography and Sketching >> DSLR & Digital Camera Astro Imaging & Processing

Pages: 1 | 2 | 3 | 4 | (show all)
mpgxsvcd
Carpal Tunnel


Reged: 12/21/11

Loc: Raleigh, North Carolina
How much difference do more subs make?
      #6265953 - 12/21/13 08:03 PM Attachment (67 downloads)

So I wanted to see where the point of diminishing returns comes into play. I had stacked 79 6 second ISO 6400 images of M42 before.

I decided to see what 60, 28, and 16 looked like instead. I stacked the 60, 28, and 16 in the exact same manner and saved the output of DSS without any edits.

Here are what they looked like. I also have the original 79 image file. However, the processing on that was slightly different.

I think that 30-60 images is sufficient. 16 images definitely shows some difference.

60 Images


Post Extras: Print Post   Remind Me!   Notify Moderator  
mpgxsvcd
Carpal Tunnel


Reged: 12/21/11

Loc: Raleigh, North Carolina
Re: How much difference do more subs make? new [Re: mpgxsvcd]
      #6265954 - 12/21/13 08:03 PM Attachment (54 downloads)

28 Images

Edited by mpgxsvcd (12/21/13 08:03 PM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
mpgxsvcd
Carpal Tunnel


Reged: 12/21/11

Loc: Raleigh, North Carolina
Re: How much difference do more subs make? new [Re: mpgxsvcd]
      #6265955 - 12/21/13 08:03 PM Attachment (52 downloads)

16 Images

Edited by mpgxsvcd (12/21/13 08:04 PM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
mpgxsvcd
Carpal Tunnel


Reged: 12/21/11

Loc: Raleigh, North Carolina
Re: How much difference do more subs make? new [Re: mpgxsvcd]
      #6265956 - 12/21/13 08:04 PM Attachment (39 downloads)

Here is the 79 images but with slightly different processing.

Edited by mpgxsvcd (12/21/13 08:05 PM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
Tom and Beth
Post Laureate


Reged: 01/08/07

Loc: Tucson, AZ
Re: How much difference do more subs make? new [Re: mpgxsvcd]
      #6266097 - 12/21/13 09:58 PM

Interesting. undoubtedly a significant jump between 16 and 28, and at least to me another significant jump going to 60 looking only at these down-sampled examples to fit CN guidelines.

In particular the "V" shape just off center at the 11 o'clock position


Post Extras: Print Post   Remind Me!   Notify Moderator  
jbalsam
sage
*****

Reged: 07/06/12

Loc: Darnestown, MD
Re: How much difference do more subs make? new [Re: Tom and Beth]
      #6266124 - 12/21/13 10:22 PM

The 60-frame image certainly looks the best. I don't think you've hit the point of diminishing returns yet because you can definitely see a difference between 28-60. Try 120 and 250 frames. 6s is a short exposure time so they're easy to capture.

In other news, what are you using for a coma corrector?


Post Extras: Print Post   Remind Me!   Notify Moderator  
Samir Kharusi
Carpal Tunnel
*****

Reged: 06/14/05

Loc: Oman
Re: How much difference do more subs make? new [Re: jbalsam]
      #6266196 - 12/21/13 11:34 PM

I doubt that there is EVER a point of diminishing returns. The longer the integration time, the fainter/dimmer the wisps you can dig out at a good Signal-to-Noise-Ratio. But one should not think of x frames more, think of doubling the integration time for each step. After 79 think of 160, then 320, etc. Here's what I mean, small crops displayed at almost 1:1 from APS-C stacks, identical processing for all, 2-minute subs at f4 at a dark site:


Post Extras: Print Post   Remind Me!   Notify Moderator  
Tonk
Postmaster
*****

Reged: 08/19/04

Loc: Leeds, UK, 54N
Re: How much difference do more subs make? new [Re: Samir Kharusi]
      #6266423 - 12/22/13 05:26 AM

Yes - I don't count number fo subframes but the total integration times.

Put it this way, the Gendler's, Croman's and Hallas's of this world do 20 to 30 hours integration. A few DSLR folks here have run up to 20 hours with stunning results.

Personally I find things look very good after 10 hours worth.


Post Extras: Print Post   Remind Me!   Notify Moderator  
garret
sage


Reged: 07/26/09

Loc: Netherlands
Re: How much difference do more subs make? new [Re: Tonk]
      #6266498 - 12/22/13 07:41 AM Attachment (37 downloads)

Multi hour integration time? not needed for a brite object:

Here is a crop of my imaghe of M17, only 14 subs of 60 seconds, canon 5 dmk2, F 3.6 telescope, m17 was only 19 degrees above the horizon.
In the image the right part is noise free more subs is not needed, left could have more subs.

Garret van der Veen


Post Extras: Print Post   Remind Me!   Notify Moderator  
Starhawk
Space Ranger
*****

Reged: 09/16/08

Loc: Tucson, Arizona
Re: How much difference do more subs make? new [Re: mpgxsvcd]
      #6266537 - 12/22/13 09:03 AM

Additional frames improve signal to noise ratio proportionally to the square root of the number of subs. So, if you take 4, you get twice the signal to noise ratio of one. Remember, this is statistical improvement. To get 4 times the improvement, you need 16 subs.

Adding up the integration time grossly overestimates the improvement. With 100 subs, you would predict the image was 5 magnitudes deeper than one sub. In reality, the difference is closer to 2.5 or less. The only time this is valid is when it is all in the same exposure. So, twice the exposure length gives twice the signal to read noise.

I've ended up doing 16 sub frames and putting as much integration time into them as possible.

-Rich


Post Extras: Print Post   Remind Me!   Notify Moderator  
Starhawk
Space Ranger
*****

Reged: 09/16/08

Loc: Tucson, Arizona
Re: How much difference do more subs make? new [Re: Starhawk]
      #6266539 - 12/22/13 09:05 AM

Note, there are other sources of noise besides read noise, so the real improvement from stacking isn't quite as much as the square root of the number of subs.

-Rich


Post Extras: Print Post   Remind Me!   Notify Moderator  
jbalsam
sage
*****

Reged: 07/06/12

Loc: Darnestown, MD
Re: How much difference do more subs make? new [Re: Starhawk]
      #6266581 - 12/22/13 09:44 AM

There's always a point of diminishing returns. In this case, it's when you don't notice a difference in the image quality by adding a significant amount more data. It probably takes a large amount of data to accomplish this for a low SNR object (20-30 hours as someone mentioned) where the shot noise of the object is significant.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Cotts
Just Wondering
*****

Reged: 10/10/05

Loc: Toronto, Ontario
Re: How much difference do more subs make? new [Re: jbalsam]
      #6266650 - 12/22/13 10:52 AM

I am very interested in this 'minimalism' discussion. I will be travelling to Chile in March for 11 nights at Atacama Lodge. I plan to devote about 5 of the nights to astrophotography through an AT65EDQ/Canon60Da combination. (The other nights will be visual observing with the 24" dob while shooting wide-angle stills and time-lapses..)

There are waaaay more targets than I will have time for so I will have to compromise on S/N ratio by shooting some bare minimum of subs, flats, darks etc. ISO 1600 works well with my camera...

The big question is: How many subs is the minimum acceptable number? And how long should they be? Which would be better, e.g., 2x6min, 3x4min, 4x3min, 6x2min or 12x1min?

I have a 'wish list' of about 25 targets with the AT65 so, with my planned schedule, I'll do about 5 targets a night -- about 1.5 hours maximum per target. Whatever shall I do?

Thanks

Dave


Post Extras: Print Post   Remind Me!   Notify Moderator  
Hilmi
Post Laureate
*****

Reged: 03/07/10

Loc: Muscat, Sultanate of Oman
Re: How much difference do more subs make? new [Re: Cotts]
      #6266895 - 12/22/13 01:26 PM

Dave,

You are sometimes limited by saturation in the stars. If stars get saturated they look bad post processing, so my personal advice is to take an image at the maximum exposure at which the stars don't saturate and work on from there.

Hilmi


Post Extras: Print Post   Remind Me!   Notify Moderator  
Footbag
Postmaster
*****

Reged: 04/13/09

Loc: Scranton, PA
Re: How much difference do more subs make? new [Re: Cotts]
      #6266906 - 12/22/13 01:31 PM

Quote:

I am very interested in this 'minimalism' discussion. I will be travelling to Chile in March for 11 nights at Atacama Lodge. I plan to devote about 5 of the nights to astrophotography through an AT65EDQ/Canon60Da combination. (The other nights will be visual observing with the 24" dob while shooting wide-angle stills and time-lapses..)

There are waaaay more targets than I will have time for so I will have to compromise on S/N ratio by shooting some bare minimum of subs, flats, darks etc. ISO 1600 works well with my camera...

The big question is: How many subs is the minimum acceptable number? And how long should they be? Which would be better, e.g., 2x6min, 3x4min, 4x3min, 6x2min or 12x1min?

I have a 'wish list' of about 25 targets with the AT65 so, with my planned schedule, I'll do about 5 targets a night -- about 1.5 hours maximum per target. Whatever shall I do?

Thanks

Dave




Just expose until your histogram is separated from the left by 1/3. How many? That's up to you. How noise free do you want your images to be?

Edited by Footbag (12/22/13 02:06 PM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
Tonk
Postmaster
*****

Reged: 08/19/04

Loc: Leeds, UK, 54N
Re: How much difference do more subs make? new [Re: Footbag]
      #6266960 - 12/22/13 02:03 PM

+10 ^

Post Extras: Print Post   Remind Me!   Notify Moderator  
nofxrx
Vendor (HyperCams & Mods)
*****

Reged: 07/12/05

Loc: Palm Bay,Florida
Re: How much difference do more subs make? new [Re: Tonk]
      #6267110 - 12/22/13 03:28 PM

But do you use the BOC histo to determine the 1/3 rule, or use an unstretched histo?

Also,that 1/3 rule is really dependant on the model used... For instance my 5D2 could EASILY take that to 1/2(center) or even slightly more to the right (like 9/16th to 2/3 from left)
But my old XS could only do 1/3 to slightly closer to center. ?.
The T2i was closer to 1/2 as well without issues..

And this was using the BOC histo. (Which is stretched non linearly (IIRC))

If you look at the Raw images on a PC they are MUCH darker...
So you would have to expose for nearly 2/3++ to get those to look similar to the stretched histo. Just not sure if there is any benefit our negatives in doing so(Though logically would think that the images would be completely washed out with LP... which would make removal near impossible... Again just a guess..


Post Extras: Print Post   Remind Me!   Notify Moderator  
Cotts
Just Wondering
*****

Reged: 10/10/05

Loc: Toronto, Ontario
Re: How much difference do more subs make? new [Re: Footbag]
      #6267131 - 12/22/13 03:41 PM

Adam, you said, "Just expose until your histogram is separated from the left by 1/3. "

Can I observe my histogram 'on the fly', as I shoot or must I shoot a 'guess frame' and then look at the histogram then adjust exposure?

Thanks,

Dave


Post Extras: Print Post   Remind Me!   Notify Moderator  
petemumbower
Carpal Tunnel


Reged: 07/12/09

Loc: West Michigan
Re: How much difference do more subs make? new [Re: Tonk]
      #6267133 - 12/22/13 03:41 PM

Agree with above too, we all want the awesome images we see posted, but rarely do many of us put in the integration times needed to achieve those results. But we all got to decide if we want a handful of great images or lots of ok/good ones.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Footbag
Postmaster
*****

Reged: 04/13/09

Loc: Scranton, PA
Re: How much difference do more subs make? new [Re: Cotts]
      #6267144 - 12/22/13 03:46 PM

Quote:

Adam, you said, "Just expose until your histogram is separated from the left by 1/3. "

Can I observe my histogram 'on the fly', as I shoot or must I shoot a 'guess frame' and then look at the histogram then adjust exposure?

Thanks,

Dave




You will need to take a guess frame. The exposure length will depend on sky brightness, F-Ratio, iso, etc...

At Atacama, I suspect you will be using longish exposures, unless you are imaging with very fast optics.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Wmacky
Carpal Tunnel
*****

Reged: 11/24/07

Loc: Florida
Re: How much difference do more subs make? new [Re: Footbag]
      #6267598 - 12/22/13 09:22 PM

I hate to ask, but a google search comes up with nothing.

Whats BOC?


Post Extras: Print Post   Remind Me!   Notify Moderator  
pfile
Post Laureate


Reged: 06/14/09

Re: How much difference do more subs make? new [Re: Wmacky]
      #6267609 - 12/22/13 09:29 PM

the 1/3 rule is for back of camera ("BOC") histogram. or the histogram that BYE shows you. those histograms represent a default stretch (the same stretch that the camera would apply if you let it create a jpeg for you)

the 1/3 rule is sometimes also given as "20%" or "well-detached".

note that if you are at a very dark sky site it may be impossible to push the histogram that far to the right without really long sub exposures.

rob

edit: dave you will have to take test shots; the histogram is not displayed until the exposure ends.


Edited by pfile (12/22/13 09:30 PM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
Wmacky
Carpal Tunnel
*****

Reged: 11/24/07

Loc: Florida
Re: How much difference do more subs make? new [Re: pfile]
      #6267618 - 12/22/13 09:36 PM

Quote:

the 1/3 rule is for back of camera ("BOC") histogram. or the histogram that BYE shows you. those histograms represent a default stretch (the same stretch that the camera would apply if you let it create a jpeg for you)

the 1/3 rule is sometimes also given as "20%" or "well-detached".

note that if you are at a very dark sky site it may be impossible to push the histogram that far to the right without really long sub exposures.

rob

edit: dave you will have to take test shots; the histogram is not displayed until the exposure ends.





Thanks! I have one other quick question. When taking a exposure test shot. Is there a quick way to tell if you have over saturated the brighter stars, other than a visual check? The histo turn into a hard to see thin line as it approaches the right hand side.

Edited by Wmacky (12/22/13 09:37 PM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
Jerry Lodriguss
Vendor


Reged: 07/19/08

Loc: Voorhees, NJ
Re: How much difference do more subs make? new [Re: Cotts]
      #6268183 - 12/23/13 09:31 AM

Quote:

I am very interested in this 'minimalism' discussion. I will be travelling to Chile in March for 11 nights at Atacama Lodge. I plan to devote about 5 of the nights to astrophotography through an AT65EDQ/Canon60Da combination. (The other nights will be visual observing with the 24" dob while shooting wide-angle stills and time-lapses..)

There are waaaay more targets than I will have time for so I will have to compromise on S/N ratio by shooting some bare minimum of subs, flats, darks etc. ISO 1600 works well with my camera...

The big question is: How many subs is the minimum acceptable number? And how long should they be? Which would be better, e.g., 2x6min, 3x4min, 4x3min, 6x2min or 12x1min?

I have a 'wish list' of about 25 targets with the AT65 so, with my planned schedule, I'll do about 5 targets a night -- about 1.5 hours maximum per target. Whatever shall I do?





Hi Dave,

The brighter objects, like the Eta Carina Nebula, won't need as much exposure as the fainter objects.

Prioritize your list by which objects you really want the most, and then research how faint they are. Give more exposure to the faint objects.

Under true dark skies at altitude, you should get excellent results with shorter total integration times than at a typical observing location in North America.

You will typically need longer exposures for fainter extended objects, and shorter exposures for clusters.

Jerry


Post Extras: Print Post   Remind Me!   Notify Moderator  
mpgxsvcd
Carpal Tunnel


Reged: 12/21/11

Loc: Raleigh, North Carolina
Re: How much difference do more subs make? new [Re: Samir Kharusi]
      #6268216 - 12/23/13 09:48 AM

Quote:

I doubt that there is EVER a point of diminishing returns. The longer the integration time, the fainter/dimmer the wisps you can dig out at a good Signal-to-Noise-Ratio. But one should not think of x frames more, think of doubling the integration time for each step.




The equation for Stacking says otherwise. It varies with the Square root of the number of subs. It has nothing to do with total time at all.

Direct quote from the DSS site.

http://deepskystacker.free.fr/english/theory.htm

Quote:


"The more, the better but above some threshold it is less efficient.
The signal to noise ration in increasing with the square root of the number of combined frames regardless of the exposure time of each frame.
This is true with all the combining methods (average, median, kappa-sigma clipping, auto-adaptive weighted average, ...) except entropy weighted average since this one in using the entropy to weight each pixel and thus is increasing the noise that is a big entropy contributor.."




Edited by mpgxsvcd (12/23/13 10:26 AM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
mpgxsvcd
Carpal Tunnel


Reged: 12/21/11

Loc: Raleigh, North Carolina
Re: How much difference do more subs make? new [Re: Tonk]
      #6268218 - 12/23/13 09:49 AM

Quote:

Yes - I don't count number fo subframes but the total integration times.

Put it this way, the Gendler's, Croman's and Hallas's of this world do 20 to 30 hours integration. A few DSLR folks here have run up to 20 hours with stunning results.

Personally I find things look very good after 10 hours worth.




Your wasting your time then. Check the equations for stacking. They have nothing to do with total time. Just shear number of subs. However, at some point even adding subs won't help enough.

I believe that point is somewhere around 60 subs for High ISO subs(ISO 6400) and lower(Perhaps 32 subs or less) for lower ISO subs for my camera. That is just my observation though. The camera makes a huge difference in whether that observation is accurate or not.

Edited by mpgxsvcd (12/23/13 10:26 AM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
mpgxsvcd
Carpal Tunnel


Reged: 12/21/11

Loc: Raleigh, North Carolina
Re: How much difference do more subs make? new [Re: Cotts]
      #6268288 - 12/23/13 10:24 AM

Quote:

I am very interested in this 'minimalism' discussion. I will be travelling to Chile in March for 11 nights at Atacama Lodge. I plan to devote about 5 of the nights to astrophotography through an AT65EDQ/Canon60Da combination. (The other nights will be visual observing with the 24" dob while shooting wide-angle stills and time-lapses..)

There are waaaay more targets than I will have time for so I will have to compromise on S/N ratio by shooting some bare minimum of subs, flats, darks etc. ISO 1600 works well with my camera...

The big question is: How many subs is the minimum acceptable number? And how long should they be? Which would be better, e.g., 2x6min, 3x4min, 4x3min, 6x2min or 12x1min?

I have a 'wish list' of about 25 targets with the AT65 so, with my planned schedule, I'll do about 5 targets a night -- about 1.5 hours maximum per target. Whatever shall I do?

Thanks

Dave




I have been interested in this solely because of the Messier Marathon. I want to not only capture all ~110 objects. I want to stack them as well.

From my dark site I can get at least 1 minute ISO 6400 subs at F4.0 before blowing out the sky. That is actually a lot of exposure. If I can take at least 60 of those images with good tracking I know I can get a fantastic image once I figure out my collimation issue.

However, the fact is that at most I am only going to be able to spend about 4-5 minutes on each object. 4 subs is not great but it does help.

For the brighter objects like M42, M31, M45 and the clusters I can definitely knock the time down to 30, 15 or even 8 seconds per sub to retain the brightest elements of those objects. That might give me a few extra minutes on the darkest objects.

My hope is that next spring I can stack at least 4 subs for all of the Messier objects plus the Horse Head in a single night without guiding. To me that seems like an admirable task. I only got through stacking 55 objects last year before my mount failed do to a collision with the tripod leg. It was well below freezing and I was not dressed appropriately so I had to give up then.

I have the warm clothes now and I am closer on figuring out the collimation so I think it will work this year. I don't expect perfect images. However, I do think this will give me nice results of an awful lot of objects for a single night.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Footbag
Postmaster
*****

Reged: 04/13/09

Loc: Scranton, PA
Re: How much difference do more subs make? new [Re: mpgxsvcd]
      #6268329 - 12/23/13 10:47 AM

You do need enough subs that average and median stacking methods begin to work properly. I've found this to be tough with narrowband where you have fewer longer subs.

Once you cross that threahold, assuming properly exposed subs; you can treat a sub as an interval of time.

Keep in mind that we are doing this for aethetic photographs. If 5 hours of subs didn't reduce the noise to visually acceptable levels, then quadrupling the exposure is really the only option.

Is it diminishing returns, or diminishing motivation? At some point, its a lot of work for a gain you may not be able to see.


Post Extras: Print Post   Remind Me!   Notify Moderator  
mpgxsvcd
Carpal Tunnel


Reged: 12/21/11

Loc: Raleigh, North Carolina
Re: How much difference do more subs make? new [Re: Footbag]
      #6268396 - 12/23/13 11:33 AM

Quote:


Equations for stacking give you only part of what going on. By your reconing then we should all be shooting 1000's of 1 second exposures to get the best results ?? Makes sense that if you shoot 0 second exposures then you get no noise





I never said that. Stacking more subs will reduce the noise. However, it reduces the noise less and less as you increase the number of subs. At some point you will fail to see a difference no matter how many subs you add.

How long you image for doesn't change the way the stacking process works. If you have less noise to begin with of course you will end up with less noise in the end if you stack the same number of frames. However, the ratio of original noise to final noise will be be the exact same for either scenario if you take the exact same number of subs.

Again, Imaging time has absolutely nothing to do with how the stacking process works. It is simply a way to increase the signal to noise after the images have already been taken. There are many things that will affect the signal to noise while you are taking the image. Stacking is the only efficient way to change it after you have already taken the images.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Tonk
Postmaster
*****

Reged: 08/19/04

Loc: Leeds, UK, 54N
Re: How much difference do more subs make? new [Re: Footbag]
      #6268410 - 12/23/13 11:42 AM

Have you read this summary

http://www.astropix.com/HTML/I_ASTROP/SIGNAL.HTM

Do so and then tell me I'm wasting my time

You have only discussed half the story - noise reduction. So far you have neglected considering collecting sufficient signal - thats a function of total integration time.

If we are not talking about very bright subjects (where lack of signal is not an issue) but the vast range of DSOs such as faint nebula and galaxies then total integration time gets the upper hand. Put it into practice and this becomes very evident. Targets such as ngc 7023 just get better with the amount time you put in capturing photons. On the otherhand bright targets such as M13 don't need this level of effort at all unless you are seeking the faint periphery stars (which will require HDR techniques to balance the core with faint edges in your final image)


Post Extras: Print Post   Remind Me!   Notify Moderator  
mpgxsvcd
Carpal Tunnel


Reged: 12/21/11

Loc: Raleigh, North Carolina
Re: How much difference do more subs make? new [Re: Tonk]
      #6268437 - 12/23/13 12:08 PM

Quote:

Have you read this summary

http://www.astropix.com/HTML/I_ASTROP/SIGNAL.HTM

Do so and then tell me I'm wasting my time

You have only discussed half the story - noise reduction. So far you have neglected considering collecting sufficient signal - thats a function of total integration time.

If we are not talking about very bright subjects (where lack of signal is not an issue) but the vast range of DSOs such as faint nebula and galaxies then total integration time gets the upper hand. Put it into practice and this becomes very evident. Targets such as ngc 7023 just get better with the amount time you put in capturing photons. On the otherhand bright targets such as M13 don't need this level of effort at all unless you are seeking the faint periphery stars (which will require HDR techniques to balance the core with faint edges in your final image)




All I have said is that adding subs of the same exposure duration past the point of diminishing returns will not help you. That says nothing about adding more exposure duration to each sub. However, the standard assumption is that you are using the maximum exposure for your given focal ratio and sky limits. If you aren't doing that then do so by all means. That won't affect how your subs do when stacking though.

You can also use filters to adjust the signal just like you can increase the exposure of each sub. However, that too does not affect how DSS increases the signal to noise ratio.

In my opinion 60 subs is sufficient even with ISOs as high as ISO 6400. In addition the benefit from using longer and longer exposures with lower ISO really decreases after about 4 minutes with my camera. I will post the images for that test when I get the chance. Your camera my vary from mine but I don't think it will vary much unless you are cooling it actively.

At 4 minute subs I believe that 4 hours is the most you should ever spend on an object with my particular camera. I believe that after that you are in fact just wasting your time.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Footbag
Postmaster
*****

Reged: 04/13/09

Loc: Scranton, PA
Re: How much difference do more subs make? new [Re: mpgxsvcd]
      #6268452 - 12/23/13 12:18 PM

Quote:

Quote:

Have you read this summary

http://www.astropix.com/HTML/I_ASTROP/SIGNAL.HTM

Do so and then tell me I'm wasting my time

You have only discussed half the story - noise reduction. So far you have neglected considering collecting sufficient signal - thats a function of total integration time.

If we are not talking about very bright subjects (where lack of signal is not an issue) but the vast range of DSOs such as faint nebula and galaxies then total integration time gets the upper hand. Put it into practice and this becomes very evident. Targets such as ngc 7023 just get better with the amount time you put in capturing photons. On the otherhand bright targets such as M13 don't need this level of effort at all unless you are seeking the faint periphery stars (which will require HDR techniques to balance the core with faint edges in your final image)




In my opinion 60 subs is sufficient even with ISOs as high as ISO 6400. In addition the benefit from using longer and longer exposures with lower ISO really decreases after about 4 minutes with my camera. I will post the images for that test when I get the chance. Your camera my vary from mine but I don't think it will vary much unless you are cooling it actively.

At 4 minute subs I believe that 4 hours is the most you should ever spend on an object with my particular camera. I believe that after that you are in fact just wasting your time.




This ignores the object brightness. That is a mistake because as objects get dimmer, more exposure is required to improve the SNR.

Tonk's example of the iris nebula is a good one. I have probably 15 hrs on that object and very little to show for it.


Post Extras: Print Post   Remind Me!   Notify Moderator  
tazer
sage


Reged: 12/22/11

Loc: Central North Carolina
Re: How much difference do more subs make? new [Re: Footbag]
      #6269674 - 12/23/13 11:16 PM

I had the same question myself and created an animated GIF of a high ISO stacking comparison at one point. Each sub-exposure was for 30s @ ISO 3200:

http://www.astrobin.com/full/32323/0/

It's a 17MB GIF so I decided not to embed the image.


Post Extras: Print Post   Remind Me!   Notify Moderator  
whwang
sage


Reged: 03/20/13

Re: How much difference do more subs make? new [Re: tazer]
      #6270104 - 12/24/13 08:20 AM

There will be no "point of diminishing returns" if you calibrate (bias, dark, flat) your images perfectly. You can essentially get as many subs as you like, and continuously go deeper.

The only thing is, the image quality (S/N) will go with square root of the number of subs. How much you can improve 1 sub by taking another 1, is the same as how much you can improve 60 subs by taking another 60. If I have 1 sub in hand, usually I would not hesitate to add one more. If I have 60 subs in hand, on the other hand, I would think very hard whether or not to take another 60 more. (I think this applies to most of us.) That's a mental effect, however, not a scientific one.


Post Extras: Print Post   Remind Me!   Notify Moderator  
tazer
sage


Reged: 12/22/11

Loc: Central North Carolina
Re: How much difference do more subs make? new [Re: whwang]
      #6270218 - 12/24/13 09:33 AM

I think "point of diminishing return" should actually be "point of diminishing return on investment of time" where 'return' is whatever we're trying to glean from a stack. Most often this is an aesthetic return. Is it worth spending 5 hours in the field to double your SNR 3 times? For me it is. Is it worth spending an additional 80 hours in the field to double it two more times? For me it isn't, as the net gains won't be that apparent in my stacked image (based on my scope/imaging camera/computer monitor/etc.)

Post Extras: Print Post   Remind Me!   Notify Moderator  
Steve OK
scholastic sledgehammer


Reged: 09/22/07

Loc: OKC, OK
Re: How much difference do more subs make? new [Re: tazer]
      #6270749 - 12/24/13 02:11 PM

Well said. Stacking the first 100 subs gives a 10x increase in S/N, while the next 100 subs gives an additional 1.4x increase in S/N. As Whwang points out, all additional subs lead to greater S/N, but they do so at a decreasing rate.

Steve


Post Extras: Print Post   Remind Me!   Notify Moderator  
rkayakr
sage
*****

Reged: 10/27/10

Loc: Northeast Ohio
Re: How much difference do more subs make? new [Re: Steve OK]
      #6270766 - 12/24/13 02:21 PM

Dave
You can shoot the darks and flats at another time and devote your time in Chile to lights.
Enjoy
Bob


Post Extras: Print Post   Remind Me!   Notify Moderator  
Jerry Lodriguss
Vendor


Reged: 07/19/08

Loc: Voorhees, NJ
Re: How much difference do more subs make? new [Re: rkayakr]
      #6272124 - 12/25/13 12:29 PM

Here is my take on the subject of diminishing returns. I agree with Samir and Wei-Hao.

I had just posted this to another similar question and thought it would be appropriate here also.

Some people argue that you reach a point of diminishing returns, but I don't think there is ever a point where more subs hurt.

Personally, I don't even agree with the point of diminishing returns philosophy. More photons are always going to give you a better signal-to-noise ratio because the signal can only come from the photons, and the only way to collect more photons is by collecting more subs.

In the April 2012 issue of Astronomy Magazine, Tony Hallas argued that you reach an asymptotic boundary at about 25 frames and that shooting more frames was basically a waste of time.

But this is not true. He even mentions that the signal-to-noise ratio improves with the square root of the total number of frames, but then seems to ignore it.

For example 4 frames will give you twice the s/n of 1 frame.

Likewise, 9 frames will give you 3x.

16 frames will give you 4x.

25 frames will give you 5x.

36 frames will give you 6x.

So going from 25 to 36 frames you have needed about 50 percent more total exposure time, buy your s/ has only gone up 20 percent.

Still, it has gone up.

If you follow the logic of 25 frames as a practical limit, you would never want to attempt any deep-sky objects from light polluted areas, yet there are plenty of people doing exactly that.

Look at it like this... 25 frames is 5x better than 1 frame. But, 100 frames is 10x better than one frame, and twice as good as 25 frames.

It certainly would be nice if the improvement was linear, but you still get an improvement by shooting as many frames as you can.

Now, you won't see that much improvement with 26 frames vs 25 frames. But you certainly will with 100 vs 25. The s/n will be twice as good. Is that "worth it?" Only the individual photographer can answer what his/her own level of acceptable quality is.

And this is especially important under light-polluted skies because an individual sub exposure will be limited in duration. So you will need a "hella lotta" (scientific term) subs under those conditions.

I have a shot of M27 from my suburban driveway that is a stack of 90 one-minute exposures at f/8 at ISO 800. And 90 frames certainly was not too many.

Jerry


Post Extras: Print Post   Remind Me!   Notify Moderator  
tazer
sage


Reged: 12/22/11

Loc: Central North Carolina
Re: How much difference do more subs make? new [Re: Jerry Lodriguss]
      #6272664 - 12/25/13 07:50 PM

Quote:

Some people argue that you reach a point of diminishing returns, but I don't think there is ever a point where more subs hurt.




I don't think more subs ever hurt, rather the contribution of each successive sub to the resulting stack diminishes.


Quote:

More photons are always going to give you a better signal-to-noise ratio because the signal can only come from the photons, and the only way to collect more photons is by collecting more subs.




As I understand it, the primary purpose of stacking is to eliminate noise. To increase signal you take longer subs.


Quote:

Now, you won't see that much improvement with 26 frames vs 25 frames. But you certainly will with 100 vs 25. The s/n will be twice as good.




And 400 is twice as good at 100, but that's a huge step for the same increase in SNR. Ultimately you want to get to the point where you can stretch your image aggressively enough to bring out the faintest signal. Any further reduction of noise won't be detectable in the resulting image.


Quote:

Is that "worth it?" Only the individual photographer can answer what his/her own level of acceptable quality is.




But I think that's the essence of diminishing returns. You invest time to increase SNR. While your time investment remains constant, the rate of return diminishes.


Post Extras: Print Post   Remind Me!   Notify Moderator  
whwang
sage


Reged: 03/20/13

Re: How much difference do more subs make? new [Re: tazer]
      #6272859 - 12/25/13 10:21 PM

Quote:


Quote:

More photons are always going to give you a better signal-to-noise ratio because the signal can only come from the photons, and the only way to collect more photons is by collecting more subs.




As I understand it, the primary purpose of stacking is to eliminate noise. To increase signal you take longer subs.






We should keep in mind that the most important source of noise is signal itself, in deep-sky imaging.
(This assumes that each sub is long enough so the readout noise of the camera becomes negligible.)

Photons carry noise with them. The amount of noise they carry is the square-root of the number
of photons. The more photons we collect, the stronger noise we get. However, since the noise
is the square-root of photons, the S/N is then number of photons divided by the square-root of the
same number. In other words, S/N improves as we collect more photons. This is exactly why we
want longer total integration time, and also exactly why we want more subs. They are the same thing.

Cheers,
Wei-Hao


Post Extras: Print Post   Remind Me!   Notify Moderator  
dmilligan
member


Reged: 11/21/13

Re: How much difference do more subs make? new [Re: tazer]
      #6272897 - 12/25/13 11:00 PM

Stacking does not reduce noise, it increases noise - for a given total integration time. Stacking simply allows a longer effective total integration time when single exposure time is limited by some other factor (tracking, LP, etc). It's the longer effective integration time that reduces noise, not the stacking. If you take a single exposure of some given time and compare it to a stack of some N > 1 sub-exposures with the equivalent total integration time, the single frame will win in terms of SNR.

If you understand what stacking actually is mathematically you will understand why. Imaging is basically just counting photons. As time passes, photons (actually photoelectrons) accumulate on the sensor, at some point we end the exposure, and convert the photoelectrons into a digital number that is related to the number the photons that landed on the pixel during that time (plus some amount of error due to the electronics). Mathematically this accumulation is simply addition. We add up all the photons that land and end up with a total. Stacking is the same thing, simply addition. We add up all the counts in each subframe to come up with a total (then we normalize it to some range, usually 0 to 1, but that's beside the point). The reason that stacking works is that it is the exact same process that is happening in the physical realm (photons accumulating, i.e. addition), in the digital realm (addition).

The problem is that when we stack we add in the digital realm, so there is repeated quantization and read errors (i.e. sensor read noise) as well as rounding errors in the mathematical computation (though this is usually negligible). When we simply take a longer single exposure, the photons 'add' in the physical realm and we only get read and quantization error 1 time. Using big-o notation we would say that read noise grows at O(n) when we stack. For a single exposure the read noise is simply O(1). If the total exposure time is the same, the inherent photon noise (or shot noise) will also be the same. Photon noise diminishes with time at a rate O(sqrt(t)).

With proper calibration we can eliminate a lot of the read noise, but it is impossible to eliminate all of it. So it may be a small contribution, but it is still linear ( O(n) ). The improvement from more time is sub-linear O(sqrt(t)), and while it will grow a lot faster than the read noise contribution at first, there will be a point at which the increase in read noise catches up and eventually dominates, this is the point of 'diminishing returns'.

So lets say that a single frame exposure time is limited to some value (b/c in reality it always is). To get more time and thus less photon noise, we are forced to use more subs. There will eventually come a point at which the linearly increasing read noise starts to dominate the sub-linearly decreasing photon noise, and adding more subs won't make a lick of difference. Obviously this point is going to depend on many things like sensor performance and temperature, but I imagine that it is quite large for well calibrated images and low temperatures. The only way you could assert that there is never a point of diminishing returns is that if you could show how to build a sensor with absolutely 0 read noise (if you can do this, I would be interested in going into business with you), that's the only way to make the 0(n) term go away.

There is another point of diminishing returns, and that is when we've collected enough photons for even the dimmest part of the image to have such a high photon SNR that noise is not perceivable. It might still be possible to improve SNR, but the noise would be so small that it would not be even distinguishable on something like the 8 bit computer displays we use to view our pretty pictures on (that would be a SNR that is smaller than the number of discrete brightness values or 256:1 which would be about 50dB, and would require collection of 65,536 photons - let's just say: good luck with that - such an image would be *perfectly* noise free, though I'm sure *acceptably* noise free is a lot sooner than that).


Post Extras: Print Post   Remind Me!   Notify Moderator  
Samir Kharusi
Carpal Tunnel
*****

Reged: 06/14/05

Loc: Oman
Re: How much difference do more subs make? new [Re: dmilligan]
      #6272928 - 12/25/13 11:45 PM

The equations involved for SNR are well understood and quoted here. In particular, I would refer to Line 3. You may note that the Read Noise, R, is not multiplied by N, number of subs. Additionally, provided the subs are long enough (20% of BoC Histogram or more), the dominant term in the denominator is the Skyfog signal, Sky. It is overwhelmingly important to make sure that the Skyfog signal in each sub is overwhelmingly larger than the Read Noise, R, otherwise one gets lousy Stacking Efficiency. For dim objects, those very faint tendrils, the signal from the Object itself, Obj, can also be small compared to Sky. Basically, provided the subs are long enough, the dominant term in the denominator is the the Skyfog signal, Sky. Of course the larger Sky, the larger its quantum statistical noise. In brief, there is NO upper limit as to how many subs will improve the SNR, and while the photographer may be happy with a SNR of, say, 30 on the brighter parts of his nebula, the fainter parts may have a SNR of 3 (barely acceptable). That's why some people would say that x hours is sufficient for a bright nebula, but one has to go longer for fainter stuff. IMHO, all nebulae ought to be treated as super faint and dim. They all have very faint tendrils, and it is only the photographer himself who can determine how deep he wishes to go on that object.

In conclusion please note the statement in Line 8. of the referenced URL:

Quote
8. The curve predicted from equation 6 has an asymptote that can be easily calculated as subexposure time approaches infinity. Specifically, as subexposure time approaches infinity, the contribution of R becomes negligible, and this factor can be ignored in determining the asymptote. The equation becomes: SNR asymptote = sqrt[K]*(Obj) / sqrt[(Sky+Obj)]. Thus, in photon noise limited conditions, the SNR is minimally affected by the read noise or subexposure duration (i.e., assuming that the subexposure duration is sufficiently long).
Unquote

If Steve Cannistra has his equations wrong, and they have stood the test of time, I would appreciate hearing where he went wrong. Certainly in the time I spent digesting all this I could not find any faults. But science progresses and what was obvious yesterday may be proven wrong tomorrow.


Post Extras: Print Post   Remind Me!   Notify Moderator  
dmilligan
member


Reged: 11/21/13

Re: How much difference do more subs make? new [Re: Samir Kharusi]
      #6273661 - 12/26/13 01:26 PM

You're quote is for calculating the effect of increasing subexposure time. Not the effect of increasing the number of subs (which is actually doing the opposite, decreasing subexposure time). It also clearly states the equations are 'ignoring the contribution from dark noise'

Obviously, if the subexposure time approaches infinity, the read noise component goes away (that actually agrees with what I said), but that is not at all what we are talking about. We have short limited sub exposure times due to our *BLEEP* equipment (we cannot 'assume the subexposure duration is sufficiently long'). In those situations, read noise is very important.

All this page really asserts is that sky background also puts a limit on the subexposure duration. Which is true and is one of the limitations I stated in my post (simply as 'LP') in addition to many other limitations such as tracking error. But it doesn't have much to do with what we are talking about here.

I would love to see where those equations came from and how he got from step 1 to step 2, but there are no citations or references listed on that page, and he does not derive them himself.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Footbag
Postmaster
*****

Reged: 04/13/09

Loc: Scranton, PA
Re: How much difference do more subs make? new [Re: dmilligan]
      #6273892 - 12/26/13 03:47 PM

Quote:

You're quote is for calculating the effect of increasing subexposure time. Not the effect of increasing the number of subs (which is actually doing the opposite, decreasing subexposure time). It also clearly states the equations are 'ignoring the contribution from dark noise'






I think you are talking about two separate things. The original post is about adding subs to reduce noise. It's not about dividing the total exposure time into as many subs as possible. In that case, yes, more subs would mean more noise.

But, if you are asking how much does an additional [or extra] sub help, SNR increases proportionally to the square root of the number of subs. So, it depends on whether it's your second sub or your hundredth. If it's your second sub, it helps a lot. If it's your hundredth sub, it helps a little.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Jerry Lodriguss
Vendor


Reged: 07/19/08

Loc: Voorhees, NJ
Re: How much difference do more subs make? new [Re: dmilligan]
      #6274093 - 12/26/13 05:42 PM

Quote:

We have short limited sub exposure times due to our *BLEEP* equipment (we cannot 'assume the subexposure duration is sufficiently long'). In those situations, read noise is very important.




This is exactly what we are NOT talking about.

We are talking about using subs that are sky-noise limited, not read-noise limited.

Jerry


Post Extras: Print Post   Remind Me!   Notify Moderator  
dmilligan
member


Reged: 11/21/13

Re: How much difference do more subs make? new [Re: Footbag]
      #6274605 - 12/26/13 11:02 PM

Quote:

But, if you are asking how much does an additional [or extra] sub help, SNR increases proportionally to the square root of the number of subs. So, it depends on whether it's your second sub or your hundredth. If it's your second sub, it helps a lot. If it's your hundredth sub, it helps a little.




No, SNR increases proportionally to the square root of the amount of time, not the number of subs. These are close, but not the same thing. Adding more subs adds more time, but at a price: read noise. This means that SNR increases almost to the square root of the number of subs, but not quite.

You did agree that more subs for a given amount of total exposure time increases noise:

Quote:

In that case, yes, more subs would mean more noise.




The reason I brought that up was to prove that adding more subs does not quite improve SNR at the rate of sqrt(N), but by some amount slightly less.

Example:

9min exposure has 3x the SNR of a 1 min exposure
9x1min exposure has slightly more noise than a 9 min exposure.

Therefore if we are limited to 1 min exposures, going from 1 mins to 9 mins by adding 8 more subs (rather than by adding time, since we can't) means we will not improve the SNR by fully 3x, since we already agreed that 9x1min exposure has more noise than a single 9 minute one, therefore we will only improve it by something slightly less than 3x.

Since the contribution of read noise remains the same per sub (and the error keeps compounding the more subs we take), while the advantage of more time decreases per sub the more subs we take:
Quote:

If it's your second sub, it helps a lot. If it's your hundredth sub, it helps a little.



This means there will be a point of diminishing returns: The effect of photon noise will continue to decrease while the effect of read noise will remain constant, eventually the image will go from being photon noise dominated to read noise dominated in the deepest shadows. No matter how many more subs we add, we won't get rid of the read noise which is now the dominant source of noise, because we can't. Stacking does not eliminate read noise (if it did then 9x1min would have a better SNR than 1x9min, but we already agreed it doesn't).


Post Extras: Print Post   Remind Me!   Notify Moderator  
Samir Kharusi
Carpal Tunnel
*****

Reged: 06/14/05

Loc: Oman
Re: How much difference do more subs make? new [Re: dmilligan]
      #6274804 - 12/27/13 03:23 AM

Cannistra's equations are so straightforward that I never thought he needed to provide references. It's almost like providing a reference to 2+2=4. But of course what is obvious to one person may not be to another. His entire presentation there is aimed at quantifying Stacking Efficiency, basically an answer to your query as to how well One exposure of 9 minutes is replicated by 9 subs of one minute. But I feel that part of it is somewhat above the level of a very basic discussion we had here above. Nevertheless I tried to answer that particular aspect in a separate post here as related to DSLRs. I feel we have flogged this current horse to death. Some people will be happy with shooting a very bright nebulae, like M42, with an integraion time of mere seconds (me too!), others will prefer to go much deeper to capture the fainter tendrils and use an integration time of 20 hours using many different filters. But personally I would still not dare say that the latter wasted his time or that he might as well have stopped at ten hours. Here is a 20 sec integration on M42 (5x4sec):


and a very festive M13 (29x10sec):


A very happy New Year to all, full of dark, still and clear skies!


Post Extras: Print Post   Remind Me!   Notify Moderator  
Footbag
Postmaster
*****

Reged: 04/13/09

Loc: Scranton, PA
Re: How much difference do more subs make? new [Re: Samir Kharusi]
      #6275124 - 12/27/13 10:33 AM

Quote:

This means there will be a point of diminishing returns: The effect of photon noise will continue to decrease while the effect of read noise will remain constant, eventually the image will go from being photon noise dominated to read noise dominated in the deepest shadows. No matter how many more subs we add, we won't get rid of the read noise which is now the dominant source of noise, because we can't. Stacking does not eliminate read noise (if it did then 9x1min would have a better SNR than 1x9min, but we already agreed it doesn't).





To bring the theory into practice, you are saying that at some point in stacking; read noise becomes the dominant source of noise. Are you suggesting the SNR stops improving at this point? Not just that it follows the curve asymptotically(diminishing returns)?

Nearly everyone that commented in this thread qualified their statements saying "...given a properly exposed image". It is important that you overcome read noise with brute force.

So at what point does read noise become an issue? Given a specific setup, is it the 1,000th sub? 10,000th sub? 1,000,000th sub? Or is it a non-issue, because given a properly exposed image, with enough time, read noise wouldn't be significant? It would've been overcome with the combined signal. In other words, having read noise dominat a short exposure is a bad thing. But having read noise dominate an extremely long exposure is a sign that all of the other noises have been removed, and you probably have a very nice image at that point.

Also, if we use dark frames, we are removing the read noise. Does this further push back the point where the read noise becomes significant?

Also, if we have the read noise within the SNR formula, then why wouldn't a sub be the exact same thing as an interval of time?

Edited by Footbag (12/27/13 10:41 AM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
tazer
sage


Reged: 12/22/11

Loc: Central North Carolina
Re: How much difference do more subs make? new [Re: whwang]
      #6275171 - 12/27/13 11:01 AM

Quote:

In other words, S/N improves as we collect more photons. This is exactly why we
want longer total integration time, and also exactly why we want more subs. They are the same thing.




SNR improves by either reducing N via stacking or increasing S by exposing for longer. Mathematically the ratio is the same, but stacking discards S if it's at or below the threshold of N (varies by stacking algorithm of course.) A stack of 20 5-minute subs will retain more S than a stack of 100 1-minute subs.


Post Extras: Print Post   Remind Me!   Notify Moderator  
dmilligan
member


Reged: 11/21/13

Re: How much difference do more subs make? new [Re: Footbag]
      #6275382 - 12/27/13 01:25 PM

Quote:

To bring the theory into practice, you are saying that at some point in stacking; read noise becomes the dominant source of noise. Are you suggesting the SNR stops improving at this point? Not just that it follows the curve asymptotically(diminishing returns)?



Technically it will be asymptotic. When one term dominates another term doesn't mean that the dominated term suddenly vanishes at a certain point, but it will continue to get smaller and smaller (basically that is the definition of an asymptote).

Quote:

So at what point does read noise become an issue?



It depends, I'm not trying to calculate it here, I'm just trying to show that there is such a point, as many ppl have implied that there is not such a point. Surely this point will vary and will depend on quite a number of things: sensor performance, temperature, light pollution, etc. I imagine it is quite large (many hundreds if not thousands of subs) for a decent setup and dark skies. It's also very likely that most ppl will stop well before ever reaching this point, esp. with bright objects, b/c they would have deemed their result sufficiently noise free (Even the extreme guys who take days worth of total integration time).

Most counter arguments have been something like "well this guy's image is 40 hours". But all of those amazing images, while really long, typically have rather reasonable number of subs usually < 100 (per filter of course, when collecting more subs with a different filter, you are collecting a different signal, you don't stack images with different filters anyway, you merge them in different ways, like by assigning them to different color channels). They just have incredibly long sub-exposures (on the order of 30 minutes).

I'm not saying there is a point of diminishing returns on total integration time, I'm saying there's a point of diminishing returns on increasing the number of subs to get more total integration time. At some point the only way to get more useful total integration time is by increasing subexposure length, not by taking more subs.

Quote:

Also, if we use dark frames, we are removing the read noise. Does this further push back the point where the read noise becomes significant?



Yes it does, but it doesn't eliminate it completely, because it is impossible to eliminate read noise completely (we can do a very good job, esp. with modern sensor with really low read noise, but perfection is impossible). So there will still be a point somewhere where it does happen, it may be many hundreds of subs later, but it will surely happen.

Quote:

Also, if we have the read noise within the SNR formula, then why wouldn't a sub be the exact same thing as an interval of time?



If a sub were exactly the same as an interval of time, we could divide a given total integration time into as many subs as we wanted and never loose any quality -> I could take many thousands of 1/10 sec subs and achieve the same result as an equivalent single exposure. Clearly I can't do that. That is the proof that more subs is not exactly the same as more integration time. It's close, but it's not exactly, and the difference is important, the difference is what puts an upper bound on the number of subs.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Footbag
Postmaster
*****

Reged: 04/13/09

Loc: Scranton, PA
Re: How much difference do more subs make? new [Re: dmilligan]
      #6275536 - 12/27/13 03:01 PM

Quote:


It depends, I'm not trying to calculate it here, I'm just trying to show that there is such a point, as many ppl have implied that there is not such a point. Surely this point will vary and will depend on quite a number of things: sensor performance, temperature, light pollution, etc. I imagine it is quite large (many hundreds if not thousands of subs) for a decent setup and dark skies. It's also very likely that most ppl will stop well before ever reaching this point, esp. with bright objects, b/c they would have deemed their result sufficiently noise free (Even the extreme guys who take days worth of total integration time).






At this point, I'd really like to see the math.

You seem to be falling back on the idea that the fact that you cannot divide an exposure time into further exposures without increasing noise. That is true and obvious. But, with multiple properly exposed image, where read noise is overcome by signal; I just don't understand how read noise could build relative to the signal. I agree that at some point read noise becomes dominant over other types of noise.

You seem to be arguing that sub-exposures aren't as effective at removing noise as additional integration time. In a sky limited image, which is what we are all dealing with, what is the alternative? How do you add more time without taking more subs?


Post Extras: Print Post   Remind Me!   Notify Moderator  
tazer
sage


Reged: 12/22/11

Loc: Central North Carolina
Re: How much difference do more subs make? new [Re: Footbag]
      #6276267 - 12/27/13 10:41 PM

Quote:

f a sub were exactly the same as an interval of time, we could divide a given total integration time into as many subs as we wanted and never loose any quality -> I could take many thousands of 1/10 sec subs and achieve the same result as an equivalent single exposure. Clearly I can't do that.




Stacking reduces random noise at the cost of any signal that's below the noise threshold of whichever stacking algorithm you're using. Since random noise remains at a constant level, increasing exposure allows you to bring fainter signal levels above the random noise threshold thus preserving it in the stacking process.

At least that's my understanding of why 1 1-sec sub is not equal to 10 1/10-sec subs.


Post Extras: Print Post   Remind Me!   Notify Moderator  
whwang
sage


Reged: 03/20/13

Re: How much difference do more subs make? new [Re: tazer]
      #6276358 - 12/27/13 11:36 PM

Quote:


At least that's my understanding of why 1 1-sec sub is not equal to 10 1/10-sec subs.




However, once the sub is long enough such that readout noise is no longer important,
1 30-min sub is equal to 10 3-min subs.


Post Extras: Print Post   Remind Me!   Notify Moderator  
dmilligan
member


Reged: 11/21/13

Re: How much difference do more subs make? new [Re: whwang]
      #6278318 - 12/29/13 12:20 AM

Quote:


But, with multiple properly exposed image, where read noise is overcome by signal




It's not like read noise just goes away once you get to a high enough signal, sure the ratio gets better, but read noise is still there, think of read noise as simply an error in the measurement (which is what it is). the ratio goes down as the signal goes up, but the amount stays constant.

A signal of 2 e- on a sensor with a read noise of 2 e- is completely 'lost in the noise', you're measuring 2±2 e-. When you measure a signal of 10 e-, the ratio is much better (5:1), but you're measurement is still 10±2 e-

So lets say you have a 'well exposed' sky background 'signal' of 100 e- per frame, and some signal, above that at 200 e- per frame with read noise of 2 e-. With one frame we'd have 200±2e-. But the 200e- signal is going to have photon noise of sqrt(200) ~= 14. So really we'd be measuring 200±16e-. The photon noise is way more dominant than our read error, and the SNR is 200:16 = 12.5. So we have a lot of room for improvement. But there's going to be a limit, we can keep removing photon noise with more subs, and get that 14 down really small, but there's still going to be that 2e- of measurement noise. Our SNR is not going to get better than 200:2 no matter how many more subs we add.

Quote:


I just don't understand how read noise could build relative to the signal. I agree that at some point read noise becomes dominant over other types of noise.




It doesn't build relative to signal, it remains constant relative to the signal. Since other types of noise diminish thanks to stacking, it becomes the limiting factor. Since more subs won't eliminate it, the only thing that would eliminate it is longer subs, to push up the difference between the signal level and the read noise level.
Quote:


You seem to be arguing that sub-exposures aren't as effective at removing noise as additional integration time. In a sky limited image, which is what we are all dealing with, what is the alternative? How do you add more time without taking more subs?




You don't, you can't, nothing you can do can improve your image any further. Does it not make sense that this should be true? That in an image with a sky background, there is some theoretical limit to the maximum amount of information you can get, some limit to how deep you can go? If this weren't true, then I should be able to do astrophotography during the day given enough subs. It's pretty easy to get a 'well exposed' image of the sky during the day, do you think I should just be able to combine 10 billion 1/100s 'well exposed' photos of the sky during the day and get an image of M42? What is the difference between daytime skyglow and nighttime skyglow other than intensity?

If we aren't limited by skyglow, does it not make sense that there is still some maximum limit to what we can do that is based on the limitations of the sensor?

Quote:


However, once the sub is long enough such that readout noise is no longer important,
1 30-min sub is equal to 10 3-min subs.




Nope, that never happens, they will be really really close, but because you read the sensor 3 times instead of one, you repeated some tiny error three times instead of just once, and the SNR will be ever so slightly lower. Read noise might not be "important", but it doesn't just vanish. If you just compared the two you'd be hard pressed to see the difference, but small as it may be it has an important effect, and that is that it puts an upper bound on the maximum number of useful subs. It makes the best SNR you can get asymptotic to some value.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Footbag
Postmaster
*****

Reged: 04/13/09

Loc: Scranton, PA
Re: How much difference do more subs make? new [Re: dmilligan]
      #6278668 - 12/29/13 08:33 AM

The SNR continues to improve in your example even if read noise stays constant relative to signal. This is due to the other forms of noise infinitely approaching 0 as the read noise remains constant relative to it. I think everyone must have a different definition of diminishing returns. Perhaps we are asymptotically coming to an agreement.

Again, at this point, you have a pretty darn nice image and wouldn't be looking at a formula to improve it. At this point you speak of, read noise WOULD be insignificant. It would be buried in strong signal. What you may call very close, others would call insignificant.

You make a good point about read noise, though. Proper calibration is very important to remove the noise that doesn't stack out. By limiting the read noise, you are really getting a jump on the SNR. And proper exposure is important for getting the signal up above that read noise. By sticking to a few rules, you can mitigate the read noise.

Edited by Footbag (12/29/13 08:41 AM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
tazer
sage


Reged: 12/22/11

Loc: Central North Carolina
Re: How much difference do more subs make? new [Re: whwang]
      #6278704 - 12/29/13 09:02 AM

Quote:

Quote:


At least that's my understanding of why 1 1-sec sub is not equal to 10 1/10-sec subs.




However, once the sub is long enough such that readout noise is no longer important,
1 30-min sub is equal to 10 3-min subs.




Not exactly. In a 30-minute sub you've got 30-minutes of signal. In 10 unprocessed 3-minute subs you have 30-minutes of signal, but when you stack them you discard a portion of that signal for the sake of noise reduction. If all of the signal in your 3-minute sub is above the noise threshold, then yes, 10 3-minute subs are equal to 1 30-minute sub.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Footbag
Postmaster
*****

Reged: 04/13/09

Loc: Scranton, PA
Re: How much difference do more subs make? new [Re: tazer]
      #6279586 - 12/29/13 05:25 PM

Quote:

Quote:

Quote:


At least that's my understanding of why 1 1-sec sub is not equal to 10 1/10-sec subs.




However, once the sub is long enough such that readout noise is no longer important,
1 30-min sub is equal to 10 3-min subs.




Not exactly. In a 30-minute sub you've got 30-minutes of signal. In 10 unprocessed 3-minute subs you have 30-minutes of signal, but when you stack them you discard a portion of that signal for the sake of noise reduction. If all of the signal in your 3-minute sub is above the noise threshold, then yes, 10 3-minute subs are equal to 1 30-minute sub.




I belive that theoretically the stacking would add all the pixel values and divide by the number of subframes. This is what the forulas for SNR seem to assume.

Perhaps average or median stacking method or others would dump some of the pixel values. Visually, it seems to help. I wouldn't know hor to carry the SNR ratio through these methods of stacking.


Post Extras: Print Post   Remind Me!   Notify Moderator  
scottk
sage


Reged: 08/29/09

Loc: Tennessee
Re: How much difference do more subs make? new [Re: Footbag]
      #6279715 - 12/29/13 06:34 PM

I'm very curious about this subject. I have about 3 hours of 104 second subs on m33. I'm in a red/white l.p. zone. The outer arms are still pretty faint.

I wonder if I go up to like 20 hrs of 104 second subs what will happen.


Post Extras: Print Post   Remind Me!   Notify Moderator  
tazer
sage


Reged: 12/22/11

Loc: Central North Carolina
Re: How much difference do more subs make? new [Re: Footbag]
      #6279741 - 12/29/13 06:53 PM

Quote:

Perhaps average or median stacking method or others would dump some of the pixel values. Visually, it seems to help.




Visually it does help but I guess my point is that, short of a perfect QE/noiseless sensor, you can't "go deeper" just by adding more subs. I think some of the higher-end, low noise, and cooled CCD's go a long way towards achieving this goal but not in the DSLR world.

Quote:

I wouldn't know hor to carry the SNR ratio through these methods of stacking.




Good question. I hadn't actually thought through that, but it does appear to be calculated as others have done in this thread (i.e., sqrt(# of subs)). The DSS site is in agreement:

Quote:

The signal to noise ration in increasing with the square root of the number of combined frames regardless of the exposure time of each frame.
This is true with all the combining methods (average, median, kappa-sigma clipping, auto-adaptive weighted average, ...) except entropy weighted average since this one in using the entropy to weight each pixel and thus is increasing the noise that is a big entropy contributor.




Post Extras: Print Post   Remind Me!   Notify Moderator  
whwang
sage


Reged: 03/20/13

Re: How much difference do more subs make? new [Re: scottk]
      #6280314 - 12/30/13 12:46 AM

Quote:

I'm very curious about this subject. I have about 3 hours of 104 second subs on m33. I'm in a red/white l.p. zone. The outer arms are still pretty faint.

I wonder if I go up to like 20 hrs of 104 second subs what will happen.




You will see improvement. The theoretical maximum improvement you may see is a factor of 2.6x better in S/N. To reach this, your flat field, bias, and dark need to be very well calibrated, otherwise errors in the calibration will "eat" the S/N improvement.

Edited by whwang (12/30/13 09:10 PM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
garret
sage


Reged: 07/26/09

Loc: Netherlands
Re: How much difference do more subs make? new [Re: scottk]
      #6280459 - 12/30/13 04:34 AM


Quote:

I'm very curious about this subject. I have about 3 hours of 104 second subs on m33. I'm in a red/white l.p. zone. The outer arms are still pretty faint.

I wonder if I go up to like 20 hrs of 104 second subs what will happen.





Your subs are way too short, even with a fast F3.6 telescope you need subs of 180 seconds on M33 (with color filters).
If your telescope is F7.0 start with 600 seconds subs.

Garret van der Veen


Post Extras: Print Post   Remind Me!   Notify Moderator  
tazer
sage


Reged: 12/22/11

Loc: Central North Carolina
Re: How much difference do more subs make? new [Re: scottk]
      #6280950 - 12/30/13 11:54 AM

Quote:

I'm very curious about this subject. I have about 3 hours of 104 second subs on m33. I'm in a red/white l.p. zone. The outer arms are still pretty faint.

I wonder if I go up to like 20 hrs of 104 second subs what will happen.




You'd be going from about 100 subs to 700 subs, so you'll definitely improve your SNR a _lot_ (10:1 to 26:1, relative to a single sub.) Of course the visible change will probably be slight, but you should be able to stretch much more aggressively.

Don't forget that since stacking is reducing random noise, you'll probably want to take a similar amount of dark subs as well. No point in spending all that time imaging and reducing noise only to reintroduce it when you subtract a stack of 25 dark frames (SNR 5:1.)

If you do this I'd love to see the results.


Post Extras: Print Post   Remind Me!   Notify Moderator  
scottk
sage


Reged: 08/29/09

Loc: Tennessee
Re: How much difference do more subs make? new [Re: garret]
      #6281179 - 12/30/13 01:54 PM

I'm at f/4. I would love nothing more than to take longer subs, but with the light pollution here, my histogram is creeping closer to 1/2 from left at iso 400 104sec.

Why would I use color filters with an unmodified dslr? Do you mean light pollution filters?.

Edited by scottk (12/30/13 02:00 PM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
tazer
sage


Reged: 12/22/11

Loc: Central North Carolina
Re: How much difference do more subs make? new [Re: scottk]
      #6282137 - 12/30/13 09:02 PM

Thought I'd generate a stacking comparison of M27 to add to the discussion. The subs were taken with an unmodified Sony NEX-5 through an ES Levy Comet Hunter (f/4.8, 731mm) on a CG5-ASGT with no filters. No darks, flats, or bias frames were used. The only post processing was shifting the histogram over a bit, otherwise no stretching or saturation applied. The image was edited in GIMP so there may be some posterization present.

Each frame in the comparison is a 100% crop of M27 from its respective stack. I chose the stacking numbers so they correlate with 100% increases in SNR. The frame text is as follows: SNR (signal to noise ratio of the stacked image), SUBS (number of sub exposures in the stack the image was cropped from), INT (total integration time of the stack the image was cropped from.) The frames range from a single 30s sub to 121 30s subs (60.5m total integration.)




Post Extras: Print Post   Remind Me!   Notify Moderator  
Tonk
Postmaster
*****

Reged: 08/19/04

Loc: Leeds, UK, 54N
Re: How much difference do more subs make? new [Re: scottk]
      #6282149 - 12/30/13 09:05 PM

Quote:

Don't forget that since stacking is reducing random noise, you'll probably want to take a similar amount of dark subs as well. No point in spending all that time imaging and reducing noise only to reintroduce it when you subtract a stack of 25 dark frames (SNR 5:1.)




I think you have got this a little wrong. There is no relationship between the number of lights you take and the number darks you need to take.

Remember the master dark is subtracted from each of the lights before the lights are stacked - not afterwards (which wouldn't work anyway as the hotpixels are no longer aligned)


Post Extras: Print Post   Remind Me!   Notify Moderator  
whwang
sage


Reged: 03/20/13

Re: How much difference do more subs make? new [Re: Tonk]
      #6282180 - 12/30/13 09:17 PM

Quote:

Quote:

Don't forget that since stacking is reducing random noise, you'll probably want to take a similar amount of dark subs as well. No point in spending all that time imaging and reducing noise only to reintroduce it when you subtract a stack of 25 dark frames (SNR 5:1.)




I think you have got this a little wrong. There is no relationship between the number of lights you take and the number darks you need to take.

Remember the master dark is subtracted from each of the lights before the lights are stacked - not afterwards (which wouldn't work anyway as the hotpixels are no longer aligned)




This is correct. As long as the darks are taken under similar temperature of the light, a few to 10 dark frames are sufficient. This is especially true under a light-polluted sky, where sky noise can overwhelm dark noise. Adding more darks will not improve the quality of the stacked image.

The only exception is when the tracking is super accurate and no dither is applied. In such a case, the dark noise in the master dark will have its imprint on the final stacked image. However, in reality, such highly accurate tracking (sub-pixel accuracy over several hours) hardly happens.


Post Extras: Print Post   Remind Me!   Notify Moderator  
scottk
sage


Reged: 08/29/09

Loc: Tennessee
Re: How much difference do more subs make? new [Re: Tonk]
      #6282191 - 12/30/13 09:21 PM

I'm wondering if this is where the "groups" in dss comes into play. The images I have already should go fine with darks I have already, but now, at a different temp, on a different night, perhaps I should take more darks, and process them in a separate group with the new lights I take?

Post Extras: Print Post   Remind Me!   Notify Moderator  
Tonk
Postmaster
*****

Reged: 08/19/04

Loc: Leeds, UK, 54N
Re: How much difference do more subs make? new [Re: scottk]
      #6282787 - 12/31/13 05:35 AM

Quote:

I'm wondering if this is where the "groups" in dss comes into play. The images I have already should go fine with darks I have already, but now, at a different temp, on a different night, perhaps I should take more darks, and process them in a separate group with the new lights I take?




Exactly right - DSS groups are for multi-nights stacking


Post Extras: Print Post   Remind Me!   Notify Moderator  
tazer
sage


Reged: 12/22/11

Loc: Central North Carolina
Re: How much difference do more subs make? new [Re: Tonk]
      #6282891 - 12/31/13 08:10 AM

Quote:

There is no relationship between the number of lights you take and the number darks you need to take.




Correct and I didn't mean to imply there was. I was trying to emphasize that investing 20 hours in reducing SNR of the lights is a big effort and a similar effort should be put into reducing SNR of the darks.

Quote:

Remember the master dark is subtracted from each of the lights before the lights are stacked - not afterwards (which wouldn't work anyway as the hotpixels are no longer aligned)




Of course it's subtracted from each light before stacking, but don't forget that while subtracting a master dark from a light does reduce dark signal it also _adds_ noise. Thus it decreases the SNR of each light before stacking.

QSI, while they don't produce DSLR's, has a good page on this:

http://www.qsimaging.com/combined-darks-snr.html

Quote:

How many dark frames do I need?

The answer to this question really comes down to how much time you're willing to invest in collecting dark frames in order to achieve an acceptable level of noise from dark current in your final images. The best answer is to measure the noise in your dark frames and decide how much noise you're willing to add when calibrating your light frames. Because of how noise combines, a small difference in the noise in your master dark won't make a huge difference in the SNR of your final images, but the difference can be measured.




So, for a stack of 700 lights, 700 darks is probably overkill, but you _do_ want to stack as many as you can. Again, why spend a big effort up front and take a shortcut at the end?


Post Extras: Print Post   Remind Me!   Notify Moderator  
Tonk
Postmaster
*****

Reged: 08/19/04

Loc: Leeds, UK, 54N
Re: How much difference do more subs make? new [Re: tazer]
      #6283259 - 12/31/13 11:30 AM

Quote:

and a similar effort should be put into reducing SNR of the darks.




Yes go for a good master dark but 25 to 50 darks are all that's ever needed no matter how many lights you shoot. The amount of noise from say a 25 fame master added to the lights is tiny (1/5th of the existing noise in your light).


Post Extras: Print Post   Remind Me!   Notify Moderator  
Alfredo Beltran
sage


Reged: 08/01/10

Loc: Bogota, Colombia
Re: How much difference do more subs make? new [Re: Tonk]
      #6285215 - 01/01/14 10:38 AM

Here you can see an spreadsheet developed by Craig Stark that starts with some data of the camera (gain, read noise and dark current) and some data from the sky conditions to calculate the SNR of a single frame and then of the whole stack. It also shows the impact of the sky glow on the SNR of the stack.

http://www.stark-labs.com/craig/resources/Articles-&-Reviews/SNR-Calculat...

All of the base for those calculations is here


I measured those parameters for the T1i .

At the end this shows that the shorter the exposure, the more subs you need to reach the same SNR of the stack (as expected). Also that the higher the light pollution, the longer the total exposure time to achieve a higher SNR (again as expected).

This could help to answer the original question, but as others have said, more subs don't hurt and will always increase a little the SNR (even though each time the contribution can be less).

I hope this helps

Happy New year!

Alfredo


Post Extras: Print Post   Remind Me!   Notify Moderator  
tazer
sage


Reged: 12/22/11

Loc: Central North Carolina
Re: How much difference do more subs make? new [Re: Tonk]
      #6285601 - 01/01/14 01:56 PM

Quote:

Yes go for a good master dark but 25 to 50 darks are all that's ever needed no matter how many lights you shoot. The amount of noise from say a 25 fame master added to the lights is tiny (1/5th of the existing noise in your light).




True, especially with small stacks of light frames. However if you're putting in a significant effort into maximizing SNR by stacking 700 lights, and you wish to maintain that SNR, then you'll find you need to add additional 30 lights to your stack. 200 dark subs reduces that number to 10, etc. It's up to each imager to determine where his/her time is invested and what level of noise is acceptable.

The image I linked to earlier demonstrates that, to me, there is little visible decrease in noise between 64 lights and 121 lights. Subtracting a stack of 25 darks or a stack of 200 darks has about the same effect, an effect I probably wouldn't be able to detect.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Pages: 1 | 2 | 3 | 4 | (show all)


Extra information
8 registered and 31 anonymous users are browsing this forum.

Moderator:  Dave M, WOBentley, tecmage 

Print Thread

Forum Permissions
      You cannot start new topics
      You cannot reply to topics
      HTML is disabled
      UBBCode is enabled


Thread views: 2075

Jump to

CN Forums Home


Cloudy Nights LLC
Cloudy Nights Sponsor: Astronomics