Jump to content

  •  

CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.

Photo

I Need a Primer on Read Noise Calc (ASI1600)

  • Please log in to reply
19 replies to this topic

#1 GeneralT001

GeneralT001

    Mercury-Atlas

  • *****
  • topic starter
  • Posts: 2,590
  • Joined: 06 Feb 2012
  • Loc: Nanaimo, BC

Posted 06 November 2017 - 02:03 PM

I'm trying to square this away but have hit a wall. Its my understanding that we should be exposing (ideally) to swamp the read noise by a factor of up to 20x.

 

So, as an example, if I want to shoot at 300 gain on the ASI1600, then I am getting a read noise of about 1.35e. according to the charts. 

 

  • multiply 20 x 1.35e = 27DN @ 12bit or 432DN @ 16 bit so IOT swamp the read noise sufficiently I should be aiming for a background mean of 432DN 16 bit? Is this right - seems pretty low?
     

What I am not getting is I have to account for the offset, which is 50 at 300 gain. If I subtract 50 from 27 I'm getting a negative number - what am I missing? Probably everything:)

 

 

 

Or do I need to divide that 27DN by the Gain (e/ADU), so .2 which gives me 135DN then add the offset 50 to that for 185Dn then multiply that by 16, which gives me 2,960DN @16 bit. ??

 

16-bit ADU = 16* (20*[ReadNoise]/[Gain] + [Offset])    so    16 x 135 + 50 = 16 x 185 = 2,960DN


Edited by GeneralT001, 06 November 2017 - 03:16 PM.


#2 bobzeq25

bobzeq25

    ISS

  • *****
  • Posts: 20,223
  • Joined: 27 Oct 2014

Posted 06 November 2017 - 03:37 PM

Well, you need to compare apples to apples.  So everything needs to be either electrons or DN (ADU).  And, everything needs to be in one bit depth.  You've got them all mixed up.

 

Here's how I'd do it.  I like simple.

 

20XRN =27 electrons.  Divide by 0.2, that's 135DN.  Add offset, you don't say if that's DN and 12 bit, although I presume it is.  Assume that.  185DN.

 

PI will let you look at the image statistics for a light in 12 bit, so that's what you'd aim for.  No need to convert to 16.

 

Minor point.  Some like 20X RN,  Some 5X RN squared.  Some 10X RN squared.  In The Astrophotography Manual (highly recommended) Chris Woodhouse has a long discussion, which I cannot possibly summarize here.  He recommends something between the last two, paying attention to clipping at the high end, and notes that minimizing that would sometimes put you below the lower limit.

 

Crucial point.  Getting the subexposure exactly "right" (impossible, since people don't agree on what "right" is), is secondary to total imaging time.   The best thing you can do is SMS, shoot more subs.


Edited by bobzeq25, 06 November 2017 - 03:43 PM.

  • zakry3323 likes this

#3 Jon Rista

Jon Rista

    ISS

  • *****
  • Posts: 24,324
  • Joined: 10 Jan 2014
  • Loc: Colorado

Posted 06 November 2017 - 03:59 PM

I would use a slightly different formula. I have a mostly-written article on this, at some point my work will slow down and I'll be able to finish it. Anyway, to account for the conflicting needs to swamping read noise vs. not clipping stars, I advocate getting your signal to somewhere between 3xRN^2 and 10xRN^2. Ideally, the highest background signal you can get is of course better, but you want to balance that against clipped stars. So first off, you will want to calculate two levels, which would be your threshold. The basic formula is:

 

DN = (Nread^2 * Swamp / Gain + Offset) * (2^16/2^Bits)

 

Where:

 

DN = required background signal in 16-bit DN.

Nread = read noise in e-

Swamp = swamping factor

Gain = camera gain in e-/ADU

Offset = bias offset in ADU

Bits = ADC bit depth

 

This formula should work for any camera, not just the ASI1600. So, to calculate your absolute lower limit, the "never go below this" threshold for background sky, use a Swamp factor of 3 (the minimum I recommend going, period, even if  you are clipping stars):

 

DNmin = (1.13e-^2 * 3 / 0.15e-/ADU + 50) * 16 = 1209

 

To calculate the "ideal" background level, the one you want if you can achieve it, but would forego if you start clipping too many stars or by too much, is a Swamp factor of 10 (this is NOT a maximum, although going much beyond this has diminishing returns in terms of final SNR):

 

DNideal = (1.13e-^2 * 10 / 0.15e-/ADU + 50) * 16 = 2162

 

I suspect swamping by 10x at Gain 300 is probably going to be more difficult here. You could see what you would require at Gain 200:

 

DNideal = (1.3e-^2 * 10 / 0.483e-/ADU + 50) * 16 = 1360

 

And at Gain 139:

 

DNideal = (1.55e-^2 * 10 / 1e-/ADU + 50) * 16 = 1185

 

Now, since the gain is changing here, even though the required ADU count is LOWER at the lower gain settings, you will actually need LONGER subs to get those levels. Up to you to determine where your exposure length threshold is, and factor that into which gain you choose to use. 


  • Dean J., tkottary, zakry3323 and 3 others like this

#4 donlism

donlism

    Viking 1

  • *****
  • Posts: 538
  • Joined: 20 Oct 2015
  • Loc: San Jose, CA

Posted 06 November 2017 - 04:25 PM

Offset is an offset.  It's very much like measuring with a ruler starting at the 20cm mark.  If the end of the thing is at the 23cm mark, it's 3cm long.

 

So... with an offset of 50, if no other factors were in play, the pixels would all be 50.  If there's a noise value of 10 ADU's, it will be 60.  And so on.  So the reason you got a negative number subtracting 50 from 27 is that you didn't add the 50 offset first.  You want to be subtracting 50 from 77 in that situation.



#5 bobzeq25

bobzeq25

    ISS

  • *****
  • Posts: 20,223
  • Joined: 27 Oct 2014

Posted 06 November 2017 - 05:51 PM

Jon is perfectly right.  In theory.

 

In practice 10 minute lights through my 6nm Ha filter were at 2X RN squared.  And the image didn't come out so bad, although I did need to add a pedestal to avoid a lot of black clipping, and I'm still not sure I got that perfectly right.  There's a lot of salt and pepper noise in the background.

 

Only 2.3 hours, definitely needed to SMS.

Crescent v4 (14 lights).jpg



#6 GeneralT001

GeneralT001

    Mercury-Atlas

  • *****
  • topic starter
  • Posts: 2,590
  • Joined: 06 Feb 2012
  • Loc: Nanaimo, BC

Posted 06 November 2017 - 06:03 PM

Well, you need to compare apples to apples.  So everything needs to be either electrons or DN (ADU).  And, everything needs to be in one bit depth.  You've got them all mixed up.

 

Here's how I'd do it.  I like simple.

 

20XRN =27 electrons.  Divide by 0.2, that's 135DN.  Add offset, you don't say if that's DN and 12 bit, although I presume it is.  Assume that.  185DN.

 

PI will let you look at the image statistics for a light in 12 bit, so that's what you'd aim for.  No need to convert to 16.

 

Minor point.  Some like 20X RN,  Some 5X RN squared.  Some 10X RN squared.  In The Astrophotography Manual (highly recommended) Chris Woodhouse has a long discussion, which I cannot possibly summarize here.  He recommends something between the last two, paying attention to clipping at the high end, and notes that minimizing that would sometimes put you below the lower limit.

 

Crucial point.  Getting the subexposure exactly "right" (impossible, since people don't agree on what "right" is), is secondary to total imaging time.   The best thing you can do is SMS, shoot more subs.

Thanks Bob,

 

I am shooting more subs. 100 x 5 min per each NB, so about 8 hrs or 24 hrs total - that's gotta help clean up the noise! :)



#7 GeneralT001

GeneralT001

    Mercury-Atlas

  • *****
  • topic starter
  • Posts: 2,590
  • Joined: 06 Feb 2012
  • Loc: Nanaimo, BC

Posted 06 November 2017 - 06:26 PM

Jon is perfectly right.  In theory.

 

In practice 10 minute lights through my 6nm Ha filter were at 2X RN squared.  And the image didn't come out so bad, although I did need to add a pedestal to avoid a lot of black clipping, and I'm still not sure I got that perfectly right.  There's a lot of salt and pepper noise in the background.

 

Only 2.3 hours, definitely needed to SMS.

attachicon.gifCrescent v4 (14 lights).jpg

 

When you talk about the black clipping are you referring to the black pixels like in the zoomed shot here:

 

clip.JPG

 

 

Aside from exposing longer I can lessen these by adding a pedestal (in my case I would add 800) when calibrating in PI?



#8 GeneralT001

GeneralT001

    Mercury-Atlas

  • *****
  • topic starter
  • Posts: 2,590
  • Joined: 06 Feb 2012
  • Loc: Nanaimo, BC

Posted 06 November 2017 - 06:28 PM

I would use a slightly different formula. I have a mostly-written article on this, at some point my work will slow down and I'll be able to finish it. Anyway, to account for the conflicting needs to swamping read noise vs. not clipping stars, I advocate getting your signal to somewhere between 3xRN^2 and 10xRN^2. Ideally, the highest background signal you can get is of course better, but you want to balance that against clipped stars. So first off, you will want to calculate two levels, which would be your threshold. The basic formula is:

 

DN = (Nread^2 * Swamp / Gain + Offset) * (2^16/2^Bits)

 

Where:

 

DN = required background signal in 16-bit DN.

Nread = read noise in e-

Swamp = swamping factor

Gain = camera gain in e-/ADU

Offset = bias offset in ADU

Bits = ADC bit depth

 

This formula should work for any camera, not just the ASI1600. So, to calculate your absolute lower limit, the "never go below this" threshold for background sky, use a Swamp factor of 3 (the minimum I recommend going, period, even if  you are clipping stars):

 

DNmin = (1.13e-^2 * 3 / 0.15e-/ADU + 50) * 16 = 1209

 

To calculate the "ideal" background level, the one you want if you can achieve it, but would forego if you start clipping too many stars or by too much, is a Swamp factor of 10 (this is NOT a maximum, although going much beyond this has diminishing returns in terms of final SNR):

 

DNideal = (1.13e-^2 * 10 / 0.15e-/ADU + 50) * 16 = 2162

 

I suspect swamping by 10x at Gain 300 is probably going to be more difficult here. You could see what you would require at Gain 200:

 

DNideal = (1.3e-^2 * 10 / 0.483e-/ADU + 50) * 16 = 1360

 

And at Gain 139:

 

DNideal = (1.55e-^2 * 10 / 1e-/ADU + 50) * 16 = 1185

 

Now, since the gain is changing here, even though the required ADU count is LOWER at the lower gain settings, you will actually need LONGER subs to get those levels. Up to you to determine where your exposure length threshold is, and factor that into which gain you choose to use. 

Awesome Jon, thanks. It makes a lot of sense. Why do I get the feeling by the time I get a good grip on this I will be getting a CCD? Unless, of course that mythical FF CMOS camera comes into being.:)



#9 Jon Rista

Jon Rista

    ISS

  • *****
  • Posts: 24,324
  • Joined: 10 Jan 2014
  • Loc: Colorado

Posted 06 November 2017 - 06:41 PM

This stuff applies to CCDs as well You can use the same calculations to determine what you need for CCD. It's just that with CCD, you will usually be looking at larger values to swamp, and thus longer exposures. Say you had 5e- RN at a gain of say 0.25e-/ADU:

 

DN = (5^2 * 10 / 0.25 + 1200) = (25 * 10 / 0.25 + 1200) = (250 / 0.25 + 1200) = (1000 + 1200) = 2200



#10 Jon Rista

Jon Rista

    ISS

  • *****
  • Posts: 24,324
  • Joined: 10 Jan 2014
  • Loc: Colorado

Posted 06 November 2017 - 07:04 PM

Oh, and you can also determine the theoretical required exposure length if you have a general idea of what your skyfog flux is (and this is something you can measure over time to get an idea). Exposure length is the required number of electrons divided by the flux. So, in the case of say a 5e- read noise camera being swamped by 10x, with say a 0.1e-/s photon flux (which might be a bit high, even, for a dark site or narrow band imaging):

 

Exp = Nread^2 * Swamp / Flux

 

Where:

 

Nread = read noise in e-

Swamp = swamp factor

Flux = background sky flux in e-

 

So, for 5e- @ 10x and 0.1e-/s:

 

Exp = 5e-^2 * 10 / 0.1e-/s = 250e-  / 0.1e-/s = 2500s

 

You would need to expose for 2500 seconds, or almost 42 minutes, to swamp the read noise by 10x. If you went with a 3x swamp factor instead:

 

Exp = 5^2 * 3 / 0.1 = 75 / 0.1 = 750s

 

You would need to expose for 12m30s if you only swamped by 3x. Now, lets take the ASI1600 at unity, with 1.55e- read noise, swamped by 10x, at say half the flux (smaller pixels):

 

Exp = 1.55^2 * 10 / 0.05 = 24.025 / 0.05 = 480.5s

 

Significantly less time than 2500 seconds to swamp by 10x. And if we swamped by only 3x:

 

Exp = 1.55^2 * 3 / 0.05 = 7.2075 / 0.05 = 144.15s

 

Significantly less time than 750s to swamp by 3x. Now, I don't really recommend aiming for only 3x unless you have to. For some cameras with 7-8e- or more read noise, you might indeed have to go for less swamping just to be able to get the exposure at all (otherwise you might end up exposing for hours on end for single subs!) Not to mention the practicalities of dealing with airplane, satellite and meteor trails, hot pixels and remnant FPN, column defects, etc. all of which require dithering and pixel rejection, which works best when you have more than just a few subs. With low read noise cameras, getting at least 5xRN^2 should be easy enough, and 10x is certainly within relatively easy reach (just watch  your clipping.) This is where CMOS as the potential to be better than CCD, in the ability to swamp the read noise by a lot, without having to use very long exposures or risk losing hours of data due to trails or wind or other issues.

 

Also keep in mind, variations in LP, light cloud cover passing through, etc. will all affect background sky level as well as transmission rate of deep space photons. So there will be variations in each of your subs around these levels at a given exposure length. Once you figure out your average flux, then you can determine your ideal exposure lengths, and once you determine those, stick with em. You will have some subs that are brighter, some that are fainter, just the name of the game. In the end, though, they will all be normalized and the differences will be rejected and averaged out.


Edited by Jon Rista, 07 November 2017 - 12:21 AM.

  • Dean J. and zakry3323 like this

#11 GeneralT001

GeneralT001

    Mercury-Atlas

  • *****
  • topic starter
  • Posts: 2,590
  • Joined: 06 Feb 2012
  • Loc: Nanaimo, BC

Posted 06 November 2017 - 09:53 PM

Great stuff Jon. I've made a few slides to keep this stuff avail.



#12 Dean J.

Dean J.

    Messenger

  • -----
  • Posts: 444
  • Joined: 13 Aug 2011
  • Loc: Vista, CA

Posted 07 November 2017 - 05:11 AM

Jon, as a new ASI1600MM user I have been reading your posts about ideal exposure times calculation with interest.

 

I have a question about the calculation -->   DN = (Nread^2 * Swamp / Gain + Offset) * (2^16/2^Bits)

 

Is this the ideal sky background value of an uncalibrated image or a calibrated image?  

 

I have seen you offer this equation before but this is the first time I have seen "Offset" added to the result of (Nread^2 * Swamp / Gain).



#13 Christian-UAE

Christian-UAE

    Mariner 2

  • -----
  • Posts: 267
  • Joined: 04 Jan 2017
  • Loc: Abu Dhabi

Posted 07 November 2017 - 05:26 AM

 

Jon is perfectly right.  In theory.

 

In practice 10 minute lights through my 6nm Ha filter were at 2X RN squared.  And the image didn't come out so bad, although I did need to add a pedestal to avoid a lot of black clipping, and I'm still not sure I got that perfectly right.  There's a lot of salt and pepper noise in the background.

 

Only 2.3 hours, definitely needed to SMS.

attachicon.gifCrescent v4 (14 lights).jpg

 

When you talk about the black clipping are you referring to the black pixels like in the zoomed shot here:

 

attachicon.gifclip.JPG

 

 

Aside from exposing longer I can lessen these by adding a pedestal (in my case I would add 800) when calibrating in PI?

 

Hi.

 

I would be interested too in the answer of this question.



#14 Jon Rista

Jon Rista

    ISS

  • *****
  • Posts: 24,324
  • Joined: 10 Jan 2014
  • Loc: Colorado

Posted 07 November 2017 - 10:28 AM

This is for an uncalibrated sub. That is why the offset is in there (it should have always been in there, if it was not in past formulas, then that was probably a mistake on my part). The idea is to give yourself a number that you can aim for while you are imaging, with a simple measurement of the median or a spot measurement on the background, of any sub in your acquisition program. SGP is a popular one, and you can easily spot measure or view the image mean and median in the image statistics panel. I believe most other acquisition programs have something similar. 

 

Because this formula includes the offset and converts it all to 16-bit, you should be able to directly compare the 16-bit readings from programs like SGP, and immediately gain some basic knowledge about whether you are swamping the read noise enough or not. 



#15 bobzeq25

bobzeq25

    ISS

  • *****
  • Posts: 20,223
  • Joined: 27 Oct 2014

Posted 07 November 2017 - 11:00 AM

 

Jon is perfectly right.  In theory.

 

In practice 10 minute lights through my 6nm Ha filter were at 2X RN squared.  And the image didn't come out so bad, although I did need to add a pedestal to avoid a lot of black clipping, and I'm still not sure I got that perfectly right.  There's a lot of salt and pepper noise in the background.

 

Only 2.3 hours, definitely needed to SMS.

attachicon.gifCrescent v4 (14 lights).jpg

 

When you talk about the black clipping are you referring to the black pixels like in the zoomed shot here:

 

attachicon.gifclip.JPG

 

 

Aside from exposing longer I can lessen these by adding a pedestal (in my case I would add 800) when calibrating in PI?

 

 

 

 

 

Jon is perfectly right.  In theory.

 

In practice 10 minute lights through my 6nm Ha filter were at 2X RN squared.  And the image didn't come out so bad, although I did need to add a pedestal to avoid a lot of black clipping, and I'm still not sure I got that perfectly right.  There's a lot of salt and pepper noise in the background.

 

Only 2.3 hours, definitely needed to SMS.

attachicon.gifCrescent v4 (14 lights).jpg

 

When you talk about the black clipping are you referring to the black pixels like in the zoomed shot here:

 

attachicon.gifclip.JPG

 

 

Aside from exposing longer I can lessen these by adding a pedestal (in my case I would add 800) when calibrating in PI?

 

Hi.

 

I would be interested too in the answer of this question.

 

I believe so.  This is an ongoing project, don't have a definitive answer.  I need to take the first step of checking to see that those are at zero.  If so, they're clipped.  The play with pedestals to see if I can fix it.  Note that you want to use the pedestal that's in the "Output" section of ImageCalibration, not the one in the "Pedestal" section.  More PI obscurity.


Edited by bobzeq25, 07 November 2017 - 11:30 AM.


#16 GeneralT001

GeneralT001

    Mercury-Atlas

  • *****
  • topic starter
  • Posts: 2,590
  • Joined: 06 Feb 2012
  • Loc: Nanaimo, BC

Posted 07 November 2017 - 11:04 AM

Thanks for that Bob....I, of course, was going to put it in the "Pedestal" section! tongue2.gif 

 

So, does anything need to be done with the "Pedestal" section?



#17 bobzeq25

bobzeq25

    ISS

  • *****
  • Posts: 20,223
  • Joined: 27 Oct 2014

Posted 07 November 2017 - 11:30 AM

Thanks for that Bob....I, of course, was going to put it in the "Pedestal" section! tongue2.gif

 

So, does anything need to be done with the "Pedestal" section?

Nope.  It's for taking one out that a camera put in.  The output section is where you put one in.

 

That cost me an hour or so of headscratching.  PI does that.


Edited by bobzeq25, 07 November 2017 - 11:31 AM.


#18 Dean J.

Dean J.

    Messenger

  • -----
  • Posts: 444
  • Joined: 13 Aug 2011
  • Loc: Vista, CA

Posted 07 November 2017 - 11:56 AM

This is for an uncalibrated sub. That is why the offset is in there (it should have always been in there, if it was not in past formulas, then that was probably a mistake on my part). The idea is to give yourself a number that you can aim for while you are imaging, with a simple measurement of the median or a spot measurement on the background, of any sub in your acquisition program. SGP is a popular one, and you can easily spot measure or view the image mean and median in the image statistics panel. I believe most other acquisition programs have something similar. 

 

Because this formula includes the offset and converts it all to 16-bit, you should be able to directly compare the 16-bit readings from programs like SGP, and immediately gain some basic knowledge about whether you are swamping the read noise enough or not. 

Thanks, I figured it was for uncalibrated but I wanted to make sure.

 

Some of your posts from a year ago don't include the offset value - https://www.cloudyni...ight/?p=7559624



#19 Jon Rista

Jon Rista

    ISS

  • *****
  • Posts: 24,324
  • Joined: 10 Jan 2014
  • Loc: Colorado

Posted 07 November 2017 - 11:29 PM

Definitely go with my more recent posts, from this year, namely the last six months or so. After feedback from people over the previous year and a half, I decided to refine my recommendations in terms of how to get sufficient signal, how long to expose, what gain settings to use, etc. A single exposure rule at a single multiplier isn't quite sufficient, too many variables. I think it is better to figure out a min and max range of sufficient signals, and figure out how much will fit in the dynamic range of the gain you choose to use (and there are reasons to choose one over the other.) Most of this is going into a few articles on the subject. Work picked up in the middle of summer, and just keeps getting busier, so I haven't had much time to finish any of the articles. So little tidbits leak out here and there on CN whenever I have time. :p


  • zakry3323 and vagrom like this

#20 Dean J.

Dean J.

    Messenger

  • -----
  • Posts: 444
  • Joined: 13 Aug 2011
  • Loc: Vista, CA

Posted 08 November 2017 - 10:00 AM

Definitely go with my more recent posts, from this year, namely the last six months or so. After feedback from people over the previous year and a half, I decided to refine my recommendations in terms of how to get sufficient signal, how long to expose, what gain settings to use, etc. A single exposure rule at a single multiplier isn't quite sufficient, too many variables. I think it is better to figure out a min and max range of sufficient signals, and figure out how much will fit in the dynamic range of the gain you choose to use (and there are reasons to choose one over the other.) Most of this is going into a few articles on the subject. Work picked up in the middle of summer, and just keeps getting busier, so I haven't had much time to finish any of the articles. So little tidbits leak out here and there on CN whenever I have time. tongue2.gif

Yes, I do have some threads concerning the ASI1600MM-C bookmarked that are more than a year old.  All great information though and I appreciate that you and others take the time to help the members of this list.


  • vagrom likes this


CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.


Recent Topics






Cloudy Nights LLC
Cloudy Nights Sponsor: Astronomics