Jump to content


Photo

stacking/processing multiple exposure lengths/ISO

  • Please log in to reply
22 replies to this topic

#1 MaestroMyth

MaestroMyth

    Viking 1

  • *****
  • Posts: 510
  • Joined: 16 Feb 2006
  • Loc: CT

Posted 02 July 2013 - 01:29 PM

hi all,

i recently received my orion autoguider and have re-taken the plunge into astrophotography.

as part of my re-acclimation, i wanted to get a feel for how different exposure lengths and ISO's impacted my imaging.

as such, i captured about 1.5 hours worth of subs over two nights on the whirlpool galaxy, admittedly, probably too hard of an object for me at this stage.

anyway... i have about 90 30 sec subs at 1600. 30 60 sec subs at 800. and about 15 120 sec subs at 400.

i've read the forum and see that this topic has been covered, but all the answers and solutions seem to revolve around objects like m42, where preserving core details amongst longer exposed extended nebulosity dictates the creation of three layers in processing.

i'm not sure i need this layering for an object like the whirlpool, but the prevailing wisdom seems to be to make three SEPARATE stacks and combine each in processing using something like 'layer masking'. (??)

so far, i have combined all my exposures and various ISO subs into ONE big stack and processed that.

wondering what you would do in this situation. like i said, i was experimenting to see which settings work with my light pollution and set up, but i have some great subs that i'd like to process into a decent image.

thanks in advance for your thoughts.

#2 mmalik

mmalik

    Fly Me to the Moon

  • *****
  • Posts: 5344
  • Joined: 13 Jan 2012
  • Loc: USA

Posted 02 July 2013 - 02:03 PM

Are you doing ICNR or OCNR?

#3 MaestroMyth

MaestroMyth

    Viking 1

  • *****
  • Posts: 510
  • Joined: 16 Feb 2006
  • Loc: CT

Posted 02 July 2013 - 02:31 PM

OCNR now with flats, dark flats, darks. (was doing it the other way previously...)

#4 mmalik

mmalik

    Fly Me to the Moon

  • *****
  • Posts: 5344
  • Joined: 13 Jan 2012
  • Loc: USA

Posted 02 July 2013 - 03:10 PM

With OCNR you should be calibrating, aligning, and stacking images of identical ISO/duration as separate sets. Statistical stacking method like 'Min Max Excluded Average' is recommended for such sets.


Next step is something I would like other folks to chime in if blending outputs of individual sets using 'Average' will make sense? Thx

#5 MaestroMyth

MaestroMyth

    Viking 1

  • *****
  • Posts: 510
  • Joined: 16 Feb 2006
  • Loc: CT

Posted 02 July 2013 - 03:36 PM

thanks, mmalik. yes, i've aligned each sub-set with corresponding ISO separately. but then in nebulosity i had a bunch of separate recon_pproc images that i combined to make one big image. not sure if that's the way to do it...

thanks also for the min/max excluded avg tip! didn't know that.

#6 srosenfraz

srosenfraz

    Soyuz

  • *****
  • Posts: 3999
  • Joined: 06 Mar 2011
  • Loc: United States

Posted 02 July 2013 - 06:53 PM

As you noted, there isn't much of a need to use HDR techniques to combine your data sets for an object like the Whirlpool. In this case, you simply have a range of subexposures that have differing SNRs.

You'll want to keep in mind that capturing data for astrophotography is all about optimizing your signal to noise ratio. Capturing more subs does not make your image brighter - it improves your SNR so that you can stretch that image more aggressively without the noise showing as obviously.

If you were to stack all of your subs as one big stack, most processing programs will assume that all your subs should have equal weight. However, that isn't a correct assumption. Your 30 second subs will have captured 1/4 the photons that your 120 second subs will have captured (note that all the ISO does is to remap the counts for the photons that are captured - it does not change the number of photons in the subexposure). Given this, you want a 30 second sub to not be as "important" as a 60 second sub or 120 second sub.

Now, within each stack, all of the subs can be given equal weight because they have the same exposure length and, accordingly, the same SNR. So, stacking your 30 second subs separately from your 60 and 120 second subs is correct.

So, the question becomes "how much should I weight each stack". Assuming they're at the same f/ratio (which they are in your case), each stack should represent the percentage of the TOTAL exposure time it represents. For your stacks, you'll have:

90 x 30 sec = 45 minutes
30 x 60 sec = 30 minutes
15 x 120 sec = 30 minutes

Given this, your total exposure time is 105 minutes (1 hour, 45 minutes), and the stacks should be weighted as follows (slight rounding):

90 x 30 sec = 45 minutes = 43%
30 x 60 sec = 30 minutes = 29%
15 x 120 sec = 30 minutes = 29%

I use Photoshop to combine my stacks (I don't usually use different sub length, but I usually have stacks from several nights I need to combine). In Photoshop, you weight the stacks by assigning the opacity to that layer of the stack based upon how much of the total exposure time it represents of all of the layers below it. So, if I were combining your stacks in Photoshop, I would weight them as follows (I'll put the shortest subs on the bottom, but it doesn't matter the order).

Bottom layer - 90 x 30 sec = 45 minutes. This layer represents 45 minute of 45 total minutes so far, so its opacity is 100%

Middle layer - 30 x 60 sec = 30 minutes. This layer represents 30 minutes of 75 total minutes so far, so its opacity would be 40%.

Bottom layer - 15 x 120 sec = 30 minutes. This layer represents 30 minutes of 105 total minutes, so its opacity would be 29%.

Hope this helps.

#7 avarakin

avarakin

    Surveyor 1

  • *****
  • Posts: 1884
  • Joined: 13 Jul 2009
  • Loc: Parsippany NJ, USA

Posted 02 July 2013 - 08:41 PM

I agree with Scott regarding the correct approach. On the other hand this process is a PITA. Also it does not help in cases if subs are taken during periods with variable light pollution conditions, e.g. object rises or sets, or bright moon rises or sets.
It would be nice to find software which can automatically weight the subs based on SNR. It seems Theli does it, but it looks like you need to be a PHD to install and use it.
Does anyone know stacking software which stacks based on weight derived from snr?
I am thinking of putting together a script using imagemagick which would do this job, the only question is how do I determine snr of an image.

Alex

#8 whwang

whwang

    Ranger 4

  • -----
  • Posts: 332
  • Joined: 20 Mar 2013

Posted 02 July 2013 - 09:33 PM

Hi everyone,

It is correct that the images need to be weighted proportionally to their integration time. However, the effect of different ISO needs to be taken into account too. An image taken with higher ISO appears to have higher pixel brightness, so its weight in the stacking process would be artificially elevated.

So, the proper weights should be proportional to the integration times, and inversely proportional to the ISO.

Cheers,
Wei-Hao

#9 mmalik

mmalik

    Fly Me to the Moon

  • *****
  • Posts: 5344
  • Joined: 13 Jan 2012
  • Loc: USA

Posted 02 July 2013 - 09:45 PM

I use Photoshop to combine my stacks (I don't usually use different sub length, but I usually have stacks from several nights I need to combine). In Photoshop, you weight the stacks by assigning the opacity to that layer of the stack based upon how much of the total exposure time it represents of all of the layers below it. So, if I were combining your stacks in Photoshop, I would weight them as follows (I'll put the shortest subs on the bottom, but it doesn't matter the order).


Scott, can you describe click-level steps to combine/weigh stacks in Photoshop? If one were to do this in ImagesPlus how would it be done? Thanks

#10 MaestroMyth

MaestroMyth

    Viking 1

  • *****
  • Posts: 510
  • Joined: 16 Feb 2006
  • Loc: CT

Posted 02 July 2013 - 11:21 PM

scott--

thanks so much for your reply. so, yes, i do have to create three separate 'stacks' and then import them to photoshop and arrange them as suggested with requisite opacity accorded each stack.

i'm new to photoshop and i'm wondering how to process in this scenario. do i simply add my layers (curves, levels, hue, etc.) on to the 'top' layer? in other words, do i just 'process' the top layer? do i merge them at some point? please excuse my ignorance! just started out using that program...

#11 srosenfraz

srosenfraz

    Soyuz

  • *****
  • Posts: 3999
  • Joined: 06 Mar 2011
  • Loc: United States

Posted 03 July 2013 - 01:25 AM

It would be nice to find software which can automatically weight the subs based on SNR. It seems Theli does it, but it looks like you need to be a PHD to install and use it.
Does anyone know stacking software which stacks based on weight derived from snr?


Hi Alex -

I believe that Pixinsight may do this. Don't quote me on that, as I haven't used it. But, I seem to recall stumbling across this once.

#12 srosenfraz

srosenfraz

    Soyuz

  • *****
  • Posts: 3999
  • Joined: 06 Mar 2011
  • Loc: United States

Posted 03 July 2013 - 01:28 AM

Hi everyone,

It is correct that the images need to be weighted proportionally to their integration time. However, the effect of different ISO needs to be taken into account too. An image taken with higher ISO appears to have higher pixel brightness, so its weight in the stacking process would be artificially elevated.

So, the proper weights should be proportional to the integration times, and inversely proportional to the ISO.

Cheers,
Wei-Hao


Hi Wei-Hao -

Best as I can tell, either method should work. Here's an example calculation and explanation that I hope will be enlightening. Please let me know if you see any flaws in my logic.

Let's say we have 2 stars that we're photographing. Star A is actually three times as bright as Star B. We can determine if we've weighted our stacking correctly when we're done if Star A's value is three times that of Star B (irrespective of what that final value is). To be precise, we're actually talking about a representative pixel from each of the stars.

Let's assume we take two 5 minute exposures. One is at ISO 100, and one is at ISO 1,000 (for simplicity). By my calculation, we would weight those two exposures equally (50% for each). By your calculations, we would weight the ISO 100 as ten times the weight of the ISO 1,000 image (or, the ISO 1,000 stack at 10% of the ISO 100 stack). So, let's do the calcs.

Let's say that Star A produces a count of 3,000 in the 5 minute exposure period, and Star B produces a count of 1,000. Again, for simplicity, let's assume that unity gain for my camera is at ISO 100. So, the ISO 100 exposure records 3,000 for Star A, and 1,000 for Star B. The ISO 1,000 would multiply the value by 10 and gives me a value of 30,000 for Star A, and a value of 10,000 for Star B.

For my stack, the weights are 50% each, so here's the calculations

Star A - (3000 * .50) + (30000 * .50) = 16,500

Star B - (1000 * .50) + (10000 * .50) = 5,500

So, I calculate star A to be three times as bright as Star B.


For your stack, we give the ISO 100 a weight of 100% x 50% = 50% and the ISO 1,000 a weight of 10% x 50% = 5% (I hope I'm interpreting your statements correctly). So, you would calculate:

Star A - (3000 * .50) + (30000 * .05) = 3,000

Star B - (1000 * .50) + (10000 * .05) = 1,000

Your method correctly yields Star A as 3 times as bright as Star B.


As such, we can see that both methods end up with a correct relationship between the two stars - both have Star A three times as bright as Star B. The difference between the two methods is that my stack is brightened by the higher ISO values, and your stack is "normalized" to the lower ISO's value (again, assuming I interpret your weighting correctly).

That one stack is brighter than the other isn't significant. ALL of the values in one image will be proportionately brighter, so the relationship between the values are correct. Its just a matter of applying different stretching and levels adjustments in post processing.

Hope this makes sense.

#13 srosenfraz

srosenfraz

    Soyuz

  • *****
  • Posts: 3999
  • Joined: 06 Mar 2011
  • Loc: United States

Posted 03 July 2013 - 01:39 AM

Scott, can you describe click-level steps to combine/weigh stacks in Photoshop?


1) Calibrate and Stack each group of subs.

2) Align the stacks (you can do this in IP - I usually do it in Registar).

3) Open one of the aligned Tif files. This will be your "Background" or base image. You can see that the opacity for this level will always be 100%, so leave the opacity of this level as 100%.

4) Open the second stacked Tif file. Copy it to the clipboard (Edit | Select All, Edit | Copy). Close this tif file (you should be back at the first tif now).

5) Paste the second tif (Edit | Paste). This will paste the second stack as a layer on top of the first. You can check your alignment by turning on and off the eyeball on this second layer (you shouldn't see any movement between the two images).

6) Set the opacity of this layer based upon the calculations we did previously (Total exposure time of this stack / Total exposure time of this layer plus all layers below it). I find its easier just to type the opacity percentage in the box, rather than trying to use the slider to set it.

Continue copying and pasting your additional stacks and setting the opacities.



If one were to do this in ImagesPlus how would it be done? Thanks


Sorry, I can't help you with this one. I haven't ever tried it in IP.

#14 srosenfraz

srosenfraz

    Soyuz

  • *****
  • Posts: 3999
  • Joined: 06 Mar 2011
  • Loc: United States

Posted 03 July 2013 - 01:54 AM

i'm new to photoshop and i'm wondering how to process in this scenario. do i simply add my layers (curves, levels, hue, etc.) on to the 'top' layer? in other words, do i just 'process' the top layer? do i merge them at some point? please excuse my ignorance! just started out using that program...


No problem - we all started from ground zero! After I combine the stacks, I usually flatten the image before I go on to my next steps. I also save each major version of the file with a new name (i.e., I would name this one Objectname-01-CombineStacks.psd - the next one I would name Objectname-02-GradX-Levels-Curves.psd).

I usually do a handful of layers in each file before I save it, flatten it, and then go on to the next set of steps. By doing this, if I end up deciding I don't like something, I can pick it up where I messed it up without having to start all over.

Hope this helps.

#15 whwang

whwang

    Ranger 4

  • -----
  • Posts: 332
  • Joined: 20 Mar 2013

Posted 03 July 2013 - 02:03 AM

Hi Scott,

The point is not brightness of objects (brightness of star A vs B). What's key here is S/N, so you also need to consider the effect of noise.

Let's consider a simple case of equal exposure but unequal ISO. Let's say both image A and B have 10 minutes of exposure, A at ISO 100 and B at ISO 200. Let's also assume that the exposure is long enough such that the effect of readout noise is negligible. We know that in principle both images contain equal amount of information. However, the signal in image B is amplified 2x more than image A (because of its 2x higher ISO). The photon noise in image B is also amplified 2x more. From basic statistical principle, when averaging two measurements whose noises have a relative strengths of 1:2, the weights applied should be 4:1.

Here the point is effect of noise.

Cheers,
Wei-Hao

#16 srosenfraz

srosenfraz

    Soyuz

  • *****
  • Posts: 3999
  • Joined: 06 Mar 2011
  • Loc: United States

Posted 03 July 2013 - 02:15 AM

Hi Wei-Hao -

After I made that prior post, I realized I was off on a different tangent- my apologies for that.

I agree with you that the issue is SNR. Correct me if I am wrong. In your example of equal exposure but unequal ISO, both images start out with identical SNR. The ISO 200 has amplified the signal by 2, and the noise by 2, so the S/N is 2S/2N = S/N. Would this be correct so far?

If I have two images that have the same SNR, wouldn't I want to weight them equally?

#17 whwang

whwang

    Ranger 4

  • -----
  • Posts: 332
  • Joined: 20 Mar 2013

Posted 03 July 2013 - 02:12 PM

Hi Wei-Hao -

After I made that prior post, I realized I was off on a different tangent- my apologies for that.

I agree with you that the issue is SNR. Correct me if I am wrong. In your example of equal exposure but unequal ISO, both images start out with identical SNR. The ISO 200 has amplified the signal by 2, and the noise by 2, so the S/N is 2S/2N = S/N. Would this be correct so far?

If I have two images that have the same SNR, wouldn't I want to weight them equally?



Right, you want to weight them equally. However, if one image is 2x brighter than the other (despite the same S/N), a straight average doesn't weight them equally. The brighter image gets 2x more weight.

Imagine an extreme case: you have two images with identical exposure times, one taken at ISO 1, the other at ISO 1000000. The second image would appear 1000000 times brighter than the first one. Now you add the two image together and divide it by 2. What will happen? The result will be almost the same as directly dividing the second image by 2. It will give you an S/N that's almost identical to the S/N of the second image, not the root-2 better S/N we want. In such a case, the first one almost does nothing to the final result. But we know that in principle, the first one should contribute equally as the second one. To make the two images contribute equally, the weights of the two should be 1000000:1. Or, equivalently, we can bring down the brightness of the second one by 1000000 times first (this is called normalization), and then average them with equal weights.

Cheers,
Wei-Hao

#18 srosenfraz

srosenfraz

    Soyuz

  • *****
  • Posts: 3999
  • Joined: 06 Mar 2011
  • Loc: United States

Posted 03 July 2013 - 08:29 PM

Thank you for that excellent example, Wei-Hao - I see what you're saying now. I stand corrected on this subject.

Of course, your ISO 1,000,000 camera would end up clipping, so you probably should use a lower ISO.

:-)

#19 Zaid Jawed

Zaid Jawed

    Explorer 1

  • -----
  • Posts: 58
  • Joined: 20 Jun 2010

Posted 03 July 2013 - 11:50 PM

Technically, would the same principle be applied to situations with equal ISO's but distinct exposure times? For example, Wei-Hao & Scott, let's consider a situation where Image A was obtained with a 5 minute exposure and Image B with a 10 minute exposure (Both @ IS0 200). Would it be correct to say that Image B has twice the brightness of Image A? Would I apply Image A:Image B with a ratio of 4:1 in this case too? Or would I look at the sky background in each individual case (standard deviations of a point in the background) and then apply ratios as a function of the inverse square of the two?

If we consider weighting images by exposure time(ISO being the same), would you guys say it would be prudent to measure sky background more so than just assuming one image is twice as bright as the other and hence apply a 4:1 ratio? The way I see it as we increase exposure time, noise sources such as sky glow contribute much more and this reduces SNR if the rate at which wanted signal increases is matched by that of sky glow induced noise.

Hence, although I might assume that twice the exposure produces twice the counts (My assumption is linear behaviour b/w signal counts and time...Noise on the other hand, from what I've been reading has different relationships with time depending on whether it is readout noise, shot noise or dark current noise), would I be correct in saying that in situations with different exposures (all other things being equal), one should weigh stacks by measuring noise instead of going directly by measuring brightness?

It would be interesting to hear what you guys have to say.

#20 Zaid Jawed

Zaid Jawed

    Explorer 1

  • -----
  • Posts: 58
  • Joined: 20 Jun 2010

Posted 04 July 2013 - 12:06 AM

I also realize a quick fix to my question is found here

http://www.cloudynig...5952218/page...

Within this thread, Tonk suggested one use the Entropy Weighted method for stacking.

Doing it manually would definitely have more error than using a program that "weights" the images according to calculated SNR. however, I am curious to know if my thinking in the above post is correct.

#21 whwang

whwang

    Ranger 4

  • -----
  • Posts: 332
  • Joined: 20 Mar 2013

Posted 04 July 2013 - 05:25 AM

Of course, your ISO 1,000,000 camera would end up clipping, so you probably should use a lower ISO.

:-)


That's true. You know what? I found my most comfortable ISO with Nikon D800 is 200, not the more commonly used ISO 800 or 1600. I have some images taken in the past month. Once I finish the processing, I will share it with people here.

#22 MaestroMyth

MaestroMyth

    Viking 1

  • *****
  • Posts: 510
  • Joined: 16 Feb 2006
  • Loc: CT

Posted 04 July 2013 - 08:49 AM

Of course, your ISO 1,000,000 camera would end up clipping, so you probably should use a lower ISO.

:-)


That's true. You know what? I found my most comfortable ISO with Nikon D800 is 200, not the more commonly used ISO 800 or 1600. I have some images taken in the past month. Once I finish the processing, I will share it with people here.


The reason why I shot different ISOs was to try and figure out which one to shoot for my current conditions. Still experimenting! Interesting that you shoot at 200.

#23 pfile

pfile

    Gemini

  • -----
  • Posts: 3163
  • Joined: 14 Jun 2009

Posted 04 July 2013 - 03:56 PM

by the way, by default, the ImageIntegration module in PixInsight uses SNR as the weighting method during integration.

in fact along the way it was found that the process of registering the images changed the noise evaluation results enough due to the interpolation that has to take place. to counter this, juan changed the ImageCalibration module to evaluate the noise of the calibrated subs and write a FITS keyword into the calibrated subs which ImageIntegration will key off of if present.

rob






Cloudy Nights LLC
Cloudy Nights Sponsor: Astronomics