Jump to content

  •  

CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.

Photo

To bin or not to bin? That is the question.

  • Please log in to reply
31 replies to this topic

#1 kingjamez

kingjamez

    Vanguard

  • *****
  • topic starter
  • Posts: 2160
  • Joined: 03 Oct 2006
  • Loc: Fairfax, VA

Posted 06 October 2019 - 10:32 AM

I just got a C11 and plan to start imaging with it in combination with a 0.75x Starizona corrector and my ASI1600mm-c.

Natively that would result in a 0.38”/px scale. I’ve seen my local skies be that good twice ever in 3 years. More usual is 1”.

My CEM60 is capable of keeping up with my seeing and is almost always limited by seeing unless I’ve not balanced it correctly.

To that end, I “think” that I can bin at 2x2 for a 0.76”/px with no actual loss of resolution except on the rarest of nights.

Is that accurate? How does binning a CMOS play into my swamping the noise calculations?

-Jim

#2 ChrisWhite

ChrisWhite

    Aurora

  • *****
  • Posts: 4817
  • Joined: 28 Feb 2015
  • Loc: Colchester, VT

Posted 06 October 2019 - 11:01 AM

With an ASI 1600 there is absolutely no benefit to binning.  It's not like a CCD where you get lower relative read noise and higher SNR.  With the 1600 you software binning, not hardware.  Also, the read noise is so rediculously low with that camera that even at such a great pixel scale and slow system you should have no trouble swamping enough. 

 

If I were in your shoes, I would not bin. 


  • kingjamez and cfosterstars like this

#3 kingjamez

kingjamez

    Vanguard

  • *****
  • topic starter
  • Posts: 2160
  • Joined: 03 Oct 2006
  • Loc: Fairfax, VA

Posted 06 October 2019 - 12:25 PM

Thanks. I see a benefit from smaller files sizes and faster processing of hundreds of subs. If no real additional resolution is attainable then I’d rather spend less time processing.

-Jim
  • happylimpet likes this

#4 Coconuts

Coconuts

    Ranger 4

  • *****
  • Posts: 335
  • Joined: 23 Sep 2012

Posted 06 October 2019 - 01:33 PM

Chris:  "With an ASI 1600 there is absolutely no benefit to binning".  I would disagree.  2X binning increases the imaging etendue (ie, throughput, or imaging speed) by a whopping 4X.  Yes, read noise is also increased 4X, but that isn't especially important with modern CMOS sensors.  The 4X increase in imaging speed trumps read noise considerations by a huge margin.  In the OP's case, with a corrected speed of f/7.5, the imaging speed gain and 4X shorter subs would be really helpful.  Pixel size matters a lot, whether the sensor be CCD or CMOS.  Smaller files and faster downloads are a minor benefit, but real.

 

Here is a link to an imaging etendue calculator.  I provided the alternate formulation on the right.

https://docs.google....pUC4/edit#gid=0

 

All the best,

 

Kevin


Edited by Coconuts, 06 October 2019 - 01:40 PM.

  • kingjamez and Jon Rista like this

#5 avarakin

avarakin

    Vanguard

  • *****
  • Posts: 2116
  • Joined: 13 Jul 2009
  • Loc: Parsippany NJ, USA

Posted 06 October 2019 - 01:57 PM

One thing which is not considered in bin vs no-bin comparison is the cosmetic correction. Cosmetic correction helps a lot in case of non-binned data because the hot and cold pixels stand out from the rest and are getting smoothed out. I recently started doing more aggressive cosmetic correction on non-binned data and I feel that I am getting better results.

 

Alex


  • happylimpet likes this

#6 kingjamez

kingjamez

    Vanguard

  • *****
  • topic starter
  • Posts: 2160
  • Joined: 03 Oct 2006
  • Loc: Fairfax, VA

Posted 06 October 2019 - 02:54 PM

One thing which is not considered in bin vs no-bin comparison is the cosmetic correction. Cosmetic correction helps a lot in case of non-binned data because the hot and cold pixels stand out from the rest and are getting smoothed out. I recently started doing more aggressive cosmetic correction on non-binned data and I feel that I am getting better results.

 

Alex

Why can't cosmetic correction work on binned data?



#7 avarakin

avarakin

    Vanguard

  • *****
  • Posts: 2116
  • Joined: 13 Jul 2009
  • Loc: Parsippany NJ, USA

Posted 06 October 2019 - 11:08 PM

Why can't cosmetic correction work on binned data?

Imagine a scenario where most pixels of background have value of 1000 and hot pixel has value of 2000. Without binning it would be 2x brighter than the rest of the pixels and would be fixed by cosmetic correction. If you use 2x binning, then the binned value would be 4000 for background and 5000 for pixel, containing hot pixel, which is only 25% difference, so cosmetic correction may not fix it. Furthermore, stars in binned image would be smaller, thus cosmetic correction needs to be reduced in order to avoid eating the stars. 

Alex


  • happylimpet likes this

#8 Jon Rista

Jon Rista

    ISS

  • *****
  • Posts: 23853
  • Joined: 10 Jan 2014
  • Loc: Colorado

Posted 06 October 2019 - 11:49 PM

Chris:  "With an ASI 1600 there is absolutely no benefit to binning".  I would disagree.  2X binning increases the imaging etendue (ie, throughput, or imaging speed) by a whopping 4X.  Yes, read noise is also increased 4X, but that isn't especially important with modern CMOS sensors.  The 4X increase in imaging speed trumps read noise considerations by a huge margin.  In the OP's case, with a corrected speed of f/7.5, the imaging speed gain and 4X shorter subs would be really helpful.  Pixel size matters a lot, whether the sensor be CCD or CMOS.  Smaller files and faster downloads are a minor benefit, but real.

 

Here is a link to an imaging etendue calculator.  I provided the alternate formulation on the right.

https://docs.google....pUC4/edit#gid=0

 

All the best,

 

Kevin

Read noise is increased 2x, signal is increased 4x, so SNR is increased 2x. You do get 4 units of read noise, but remember they add in quadrature: SQRT(Nread^2*4). So if you have 2e- read noise, and combine 4 pixels, you don't have 2*4 (or 8), you have SQRT(2^2*4) (or 4).  So it is a 2x increase in imaging speed. 


  • happylimpet likes this

#9 Jon Rista

Jon Rista

    ISS

  • *****
  • Posts: 23853
  • Joined: 10 Jan 2014
  • Loc: Colorado

Posted 06 October 2019 - 11:53 PM

I just got a C11 and plan to start imaging with it in combination with a 0.75x Starizona corrector and my ASI1600mm-c.

Natively that would result in a 0.38”/px scale. I’ve seen my local skies be that good twice ever in 3 years. More usual is 1”.

My CEM60 is capable of keeping up with my seeing and is almost always limited by seeing unless I’ve not balanced it correctly.

To that end, I “think” that I can bin at 2x2 for a 0.76”/px with no actual loss of resolution except on the rarest of nights.

Is that accurate? How does binning a CMOS play into my swamping the noise calculations?

-Jim

With a 0.38"/px image scale, definitely  bin. Imaging at that scale is pretty advanced, most people don't have the skies for it, nor the skill to track well enough to get the most out of good seeing.

 

You can simply downsample the image by 50% in the end, which will give you similar results to binning 2x2. Don't bin in camera, that will either use simple addition of the individual pixel samples in the driver (which is not the best option), or the hardware binning option will actually sum post-ADC (digitally, not analog), but also reduces bit depth to 10. So hardware binning with current CMOS cameras is not recommended. 

 

Downsampling will give you pretty good results, though, and is basically the same thing. It increases SNR by reducing resolution. In your case, since it is highly doubtful you will actually benefit from sampling at 0.38"/px, you should be just fine. You'll end up with a 0.76"/px scale which is pretty good for moderately high resolution imaging under the kind of skies most of us have. 

 

As for read noise calculations. Best if you compute those based off the unbinned statistics. 


  • kingjamez likes this

#10 happylimpet

happylimpet

    Skylab

  • *****
  • Posts: 4067
  • Joined: 29 Sep 2013
  • Loc: Southampton, UK

Posted 07 October 2019 - 05:14 AM

I just got a C11 and plan to start imaging with it in combination with a 0.75x Starizona corrector and my ASI1600mm-c.

Natively that would result in a 0.38”/px scale. I’ve seen my local skies be that good twice ever in 3 years. More usual is 1”.

My CEM60 is capable of keeping up with my seeing and is almost always limited by seeing unless I’ve not balanced it correctly.

To that end, I “think” that I can bin at 2x2 for a 0.76”/px with no actual loss of resolution except on the rarest of nights.

Is that accurate? How does binning a CMOS play into my swamping the noise calculations?

-Jim

I would definitely bin, were it not for the hot pixel issue raised above - however I dont have a feeling how much of an issue that would be.

 

I image at 0.72"/pix, and its a great sampling frequency. When i attempt 2second sub 'almost lucky' imaging, it can be slightly coarse, so under those circumstances you might wish to image unbinned.



#11 Coconuts

Coconuts

    Ranger 4

  • *****
  • Posts: 335
  • Joined: 23 Sep 2012

Posted 07 October 2019 - 07:41 AM

Jon: "Read noise is increased 2x, signal is increased 4x, so SNR is increased 2x. You do get 4 units of read noise, but remember they add in quadrature: SQRT(Nread^2*4). So if you have 2e- read noise, and combine 4 pixels, you don't have 2*4 (or 8), you have SQRT(2^2*4) (or 4).  So it is a 2x increase in imaging speed."

 

I would agree that read noise adds in quadrature, so you are right, 2X, not 4X, but my argument wasn't really about read noise, or SNR.  It was simply about imaging etendue, which scales as pixel size squared.  Etendue doesn't factor in read noise, and given how low read noise has been driven these days, it is becoming less and less relevant for anything other than narrowband imaging.  My bottom line is that 2X binning, which increases etendue 4X, is equivalent to halving the focal ratio, something that we are used to paying a lot of money for.  But it's free.  It isn't toally free, as you have to have a long enough focal length that you don't end up undersampling, but it is still seems to me to be a 4X increase in imaging speed.  Would you agree?  Or is having 4X the photoelectrons in a given exposure length still only meaningful when considered as an SNR relative to the read noise?  Meaning it isn't the absolute number of photoelectrons that counts, but merely their ratio to read noise?

 

All the best,

 

Kevin


Edited by Coconuts, 07 October 2019 - 07:42 AM.


#12 bortle2

bortle2

    Vostok 1

  • -----
  • Posts: 108
  • Joined: 18 Sep 2019

Posted 07 October 2019 - 08:21 AM

My bottom line is that 2X binning, which increases etendue 4X, is equivalent to halving the focal ratio, something that we are used to paying a lot of money for.  But it's free.  It isn't toally free, as you have to have a long enough focal length that you don't end up undersampling, but it is still seems to me to be a 4X increase in imaging speed.  Would you agree?  Or is having 4X the photoelectrons in a given exposure length still only meaningful when considered as an SNR relative to the read noise?  Meaning it isn't the absolute number of photoelectrons that counts, but merely their ratio to read noise?

I for one do see your point, but the advantage you're talking about stems from the fact that downsampling algorithms average pixels, not sum them up. A downsampling algorithm adding pixel values and thus increasing brightness would be strictly equivalent to [SW] binning. Pretty sure batch processing with something like ImageMagick could do this easily.



#13 Coconuts

Coconuts

    Ranger 4

  • *****
  • Posts: 335
  • Joined: 23 Sep 2012

Posted 07 October 2019 - 09:05 AM

"but the advantage you're talking about stems from the fact that downsampling algorithms average pixels, not sum them up."

 

Does it matter?  It may sound counterintuitive, but the binning needn't be done in the camera; doing it during image processing achieves the same end: double the pixel size, and you have four times the photoelectrons, and four times the signal, equivalent to having magically halved the f-ratio of your scope.  This all comes at a 2X loss of resolution, but as long as this hasn't pushed you into undersampling, it is pretty close to a free lunch.  Your subs can be one quarter as long, reducing mount effects.  4X subs per hour.  Smaller files, faster downloads.  When your focal length and/or seeing support it, BIN!  And it works just as well after the fact.

 

All the best,

 

Kevin



#14 Jon Rista

Jon Rista

    ISS

  • *****
  • Posts: 23853
  • Joined: 10 Jan 2014
  • Loc: Colorado

Posted 07 October 2019 - 09:30 AM

Jon: "Read noise is increased 2x, signal is increased 4x, so SNR is increased 2x. You do get 4 units of read noise, but remember they add in quadrature: SQRT(Nread^2*4). So if you have 2e- read noise, and combine 4 pixels, you don't have 2*4 (or 8), you have SQRT(2^2*4) (or 4).  So it is a 2x increase in imaging speed."

 

I would agree that read noise adds in quadrature, so you are right, 2X, not 4X, but my argument wasn't really about read noise, or SNR.  It was simply about imaging etendue, which scales as pixel size squared.  Etendue doesn't factor in read noise, and given how low read noise has been driven these days, it is becoming less and less relevant for anything other than narrowband imaging.  My bottom line is that 2X binning, which increases etendue 4X, is equivalent to halving the focal ratio, something that we are used to paying a lot of money for.  But it's free.  It isn't toally free, as you have to have a long enough focal length that you don't end up undersampling, but it is still seems to me to be a 4X increase in imaging speed.  Would you agree?  Or is having 4X the photoelectrons in a given exposure length still only meaningful when considered as an SNR relative to the read noise?  Meaning it isn't the absolute number of photoelectrons that counts, but merely their ratio to read noise?

 

All the best,

 

Kevin

Low read noise doesn't necessarily mean it is meaningless. Dynamic range has remained in a range of about 11-12.5 stops, so even though read noise is lower, FWCs are usually smaller with smaller pixels. A smaller FWC requires shorter exposures, which means on a PER-SUB basis background sky signal is smaller. So, even though read noise drops, your signal per sub also drops, which counteracts, to a degree, having low read noise. The main benefit of low read noise is the ability to use shorter exposures, which has certain benefits...easier to acquire, allows sharper results, higher throughput (in the face of sub loss for any reason), etc. Low read noise does not implicitly mean that your SNR is higher, though. That depends on other factors like FWC, how you use the dynamic range you have, etc. 

 

Binning increases SNR by 2x. SNR is what drives speed. If you can double your SNR for a given amount of imaging time, you have doubled the speed at which you image, no quadrupled it. Absolute photon counts are only half the equation. Noise is the other half, and noise must always be accounted for. (Not just read noise, but total noise...remember, combining four samples, you  are not just adding read noise in quadrature...you  are adding ALL noise in quadrature.) It's about SNR, not just total signal. 


Edited by Jon Rista, 07 October 2019 - 09:32 AM.

  • Rickster likes this

#15 Jon Rista

Jon Rista

    ISS

  • *****
  • Posts: 23853
  • Joined: 10 Jan 2014
  • Loc: Colorado

Posted 07 October 2019 - 09:42 AM

I would definitely bin, were it not for the hot pixel issue raised above - however I dont have a feeling how much of an issue that would be.

 

I image at 0.72"/pix, and its a great sampling frequency. When i attempt 2second sub 'almost lucky' imaging, it can be slightly coarse, so under those circumstances you might wish to image unbinned.

 

"but the advantage you're talking about stems from the fact that downsampling algorithms average pixels, not sum them up."

 

Does it matter?  It may sound counterintuitive, but the binning needn't be done in the camera; doing it during image processing achieves the same end: double the pixel size, and you have four times the photoelectrons, and four times the signal, equivalent to having magically halved the f-ratio of your scope.  This all comes at a 2X loss of resolution, but as long as this hasn't pushed you into undersampling, it is pretty close to a free lunch.  Your subs can be one quarter as long, reducing mount effects.  4X subs per hour.  Smaller files, faster downloads.  When your focal length and/or seeing support it, BIN!  And it works just as well after the fact.

 

All the best,

 

Kevin

With CMOS cameras you don't have hardware binning (pre-ADC, done in charge or voltage domain) like a CCD. You may have "hardware" binning, which is really just digital summing after ADC, so there are not the same "reduces read noise" benefits that you get with a CCD, and usually hardware binning on CMOS sensors is intended for video purposes, and tends to come with the consequence of lower bit depth for increased readout speed and higher frame rates (outside of a few niche use cases, utterly useless for most astrophotography). 

 

That leaves downsampling, not binning, as the best way to improve SNR by trading off spatial resolution (potentially). So this is very simply a 4x Signal over 2x Noise equation. You get twice the SNR for half the resolution, but if you are very oversampled to start with then you are not actually going to lose any actual detail. Further, since we are talking downsampling, then you acquire and pre-process everything at 1x1. So nothing changes in regards to how long you expose each SUB, how you calibrate, etc. There are no worries about hot pixels being averaged out or improper calibration. 

 

You still gain  2x imaging speed boost, and you also gain the benefits of less stringent tracking demands, despite needing to use subs that are the same length. Because SNR is what drives imaging speed. Twice the SNR, you reach a reasonable SNR, or a target SNR, twice as fast, in terms of TOTAL exposure time (despite the same SUB exposure time)...period. And small tracking issues, they will get absorbed when you reduce spatial resolution by a factor of two. Something that may have mattered at twice the resolution will matter half as much once you downsample. Further, since this is downsampling...a filter, a kernel based process...there are additional benefits to detail and image quality that are not inherent in strait up summing of the samples (i.e. charge domain binning or raw digital sample summing). 

 

Downsampling for the OP is a total win-win. Double the imaging speed, about half the demands on tracking, and totally free, while not actually having any impact on his actual acquisition or calibration procedures at all. 


Edited by Jon Rista, 07 October 2019 - 09:44 AM.

  • kingjamez likes this

#16 Jon Rista

Jon Rista

    ISS

  • *****
  • Posts: 23853
  • Joined: 10 Jan 2014
  • Loc: Colorado

Posted 07 October 2019 - 09:50 AM

I for one do see your point, but the advantage you're talking about stems from the fact that downsampling algorithms average pixels, not sum them up. A downsampling algorithm adding pixel values and thus increasing brightness would be strictly equivalent to [SW] binning. Pretty sure batch processing with something like ImageMagick could do this easily.

The difference between strait summing, and averaging (which only includes a simple additional division step to re-scale the result of the sum), are moot (when done properly in a high precision floating point space). An averaged result smooths out the noise. So, stretch twice as much to regain the brightness. It is very simple. 


Edited by Jon Rista, 07 October 2019 - 09:51 AM.


#17 bortle2

bortle2

    Vostok 1

  • -----
  • Posts: 108
  • Joined: 18 Sep 2019

Posted 07 October 2019 - 12:21 PM

the binning needn't be done in the camera; doing it during image processing achieves the same end

Yes. That's what I'm saying ("downsampling algorithm adding pixel values and thus increasing brightness would be strictly equivalent to [SW] binning").



#18 spokeshave

spokeshave

    Vanguard

  • *****
  • Posts: 2151
  • Joined: 08 Apr 2015

Posted 07 October 2019 - 01:08 PM

One thing that hasn't been mentioned is that the ASI1600MM binning process is a very poor one. It reduces the bit depth to 10 bits, which will add significant quantization noise to each sub. If you want the advantages of a larger effective pixel (increased SNR) it is much better to resample your master images after calibration and integration. All photo editing software packages have very good resampling interpolation routines and can do so in 16-bit or even 32-bit space. This will produce much better results than in-camera binning.

 

Tim


  • happylimpet likes this

#19 Jon Rista

Jon Rista

    ISS

  • *****
  • Posts: 23853
  • Joined: 10 Jan 2014
  • Loc: Colorado

Posted 07 October 2019 - 02:27 PM

One thing that hasn't been mentioned is that the ASI1600MM binning process is a very poor one. It reduces the bit depth to 10 bits, which will add significant quantization noise to each sub. If you want the advantages of a larger effective pixel (increased SNR) it is much better to resample your master images after calibration and integration. All photo editing software packages have very good resampling interpolation routines and can do so in 16-bit or even 32-bit space. This will produce much better results than in-camera binning.

 

Tim

I mentioned this, although not the ASI1600 by name as the 10-bit thing isn't exclusive to it. Most of these current gen CMOS sensors drop the bit depth when doing binning. 



#20 Coconuts

Coconuts

    Ranger 4

  • *****
  • Posts: 335
  • Joined: 23 Sep 2012

Posted 07 October 2019 - 03:34 PM

Jon: "Most of these current gen CMOS sensors drop the bit depth when doing binning".

 

The QHY600 doesn't have that issue:

"One benefit of the back-illuminated CMOS structure is improved full well capacity. This is particularly helpful for sensors with small pixels. Even with unbinned, 3.76um pixels, the QHY600 has a full well capacity of more than 51ke-. When binned 2x2 to 7.5um the full well is 196ke- and when binned 3x3 to 11um the full well is 441ke-".

 

https://www.qhyccd.c...&catid=94&id=55

 

51k e- full well is pretty impressive from a photoelectrons per square um perspective (although this is also a function of your chosen gain).  In extended mode, that is an even larger 80 ke-.

 

My initial reaction when the Sony sensors appeared on their roadmap was that the pixels seemed on the small side.  I now regard them as nearly ideal, providing flexibility from wide field imaging to large scopes. 

 

All the best,

 

Kevin

 

"One can easily make small pixels into big ones, but not the other way around".


Edited by Coconuts, 07 October 2019 - 03:35 PM.


#21 spokeshave

spokeshave

    Vanguard

  • *****
  • Posts: 2151
  • Joined: 08 Apr 2015

Posted 08 October 2019 - 07:17 AM

I mentioned this, although not the ASI1600 by name as the 10-bit thing isn't exclusive to it. Most of these current gen CMOS sensors drop the bit depth when doing binning. 

Sorry Jon. I missed that. 

 

Tim



#22 Jon Rista

Jon Rista

    ISS

  • *****
  • Posts: 23853
  • Joined: 10 Jan 2014
  • Loc: Colorado

Posted 08 October 2019 - 08:45 AM

Jon: "Most of these current gen CMOS sensors drop the bit depth when doing binning".

 

The QHY600 doesn't have that issue:

"One benefit of the back-illuminated CMOS structure is improved full well capacity. This is particularly helpful for sensors with small pixels. Even with unbinned, 3.76um pixels, the QHY600 has a full well capacity of more than 51ke-. When binned 2x2 to 7.5um the full well is 196ke- and when binned 3x3 to 11um the full well is 441ke-".

 

https://www.qhyccd.c...&catid=94&id=55

 

51k e- full well is pretty impressive from a photoelectrons per square um perspective (although this is also a function of your chosen gain).  In extended mode, that is an even larger 80 ke-.

 

My initial reaction when the Sony sensors appeared on their roadmap was that the pixels seemed on the small side.  I now regard them as nearly ideal, providing flexibility from wide field imaging to large scopes. 

 

All the best,

 

Kevin

 

"One can easily make small pixels into big ones, but not the other way around".

Right. But this isn't a "current gen" CMOS sensor, it's a next gen sensor that is still forthcoming. ;)



#23 cfosterstars

cfosterstars

    Mercury-Atlas

  • *****
  • Posts: 2648
  • Joined: 05 Sep 2014
  • Loc: Austin, Texas

Posted 09 October 2019 - 04:52 PM

I would definitely bin, were it not for the hot pixel issue raised above - however I dont have a feeling how much of an issue that would be.

 

I image at 0.72"/pix, and its a great sampling frequency. When i attempt 2second sub 'almost lucky' imaging, it can be slightly coarse, so under those circumstances you might wish to image unbinned.

If you downsample during processing after your have integrated your master light frames, you can get all the gain, but still get cosmetic correction.



#24 kingjamez

kingjamez

    Vanguard

  • *****
  • topic starter
  • Posts: 2160
  • Joined: 03 Oct 2006
  • Loc: Fairfax, VA

Posted 09 October 2019 - 05:08 PM

If you downsample during processing after your have integrated your master light frames, you can get all the gain, but still get cosmetic correction.


Which is exactly what I’ll be doing after reading this thread. My focal reducer arrives tomorrow and I should be able to give this all a try soon.

Thanks for the help folks!

-Jim

#25 happylimpet

happylimpet

    Skylab

  • *****
  • Posts: 4067
  • Joined: 29 Sep 2013
  • Loc: Southampton, UK

Posted 10 October 2019 - 02:54 AM

If you downsample during processing after your have integrated your master light frames, you can get all the gain, but still get cosmetic correction.

Quite right, and I often do that. Its just that one of the big benefits of binnng, as originally raised, was to reduce file sizes and processing time, and these benefits are not realised with downsampling. Its a different beast.

 

Im not complaining though, just more hard drives!




CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.


Recent Topics






Cloudy Nights LLC
Cloudy Nights Sponsor: Astronomics