Jump to content

  •  

CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.

Photo

Gain and Sub-exposure calculation spreadsheet for the ZWO ASI183 and 294

  • Please log in to reply
86 replies to this topic

#1 StevenBellavia

StevenBellavia

    Messenger

  • *****
  • topic starter
  • Posts: 448
  • Joined: 11 Mar 2014
  • Loc: New York

Posted 12 August 2019 - 03:02 PM

I revised the Gain and Sub-exposure calculation spreadsheet for the ZWO  ASI183 and developed a new one for the ASI294.

 

They are too large to attach, so I setup a read-only Google share drive:

 

https://drive.google...8VJ?usp=sharing

 

Notes:

 

1. Much of the theory and equations came from Jon Rista, Frank, Craig Stark, and others (Thank you!)

2. Blue boxes are for user entry, everything else is calculated for you.  If you don't want to use image data to calculate the sky background and camera dark current, just enter those values directly to the best of your ability.

3. The "Transmission Efficiency" of the entire optical system is likely the largest potential source of error.  Play with it until your data matches a hand-held SQM, or known sky brightness value.  This may change with different filters. I find 0.5 to 0.7 works for me.

4. The common thinking is you want a Swamp Factor of around 10.  However, with narrow-band, especially at darker sites, this becomes impracticable.

5. The 183 tables go from Gain 0 to 270, but Jon Rista warns to not exceed Gain 178 with that camera.

6. I arbitrarily fixed the "object signal" to 22.0 magnitude/square arc-second.  This is to compare imaging an object from two different sites, with different sky brightness, so as to have an apples-to-apples comparison.  If you actually know your object's brightness, change the "22" in that equation to the desired value. This will only affect SNR calculations.

7. It is just a guide.  No precision was intended.

8. The 183 spreadsheet is not all that different form the Gain and Sub-exposure tables posted here:

 

https://www.cloudyni...olour-versions/

 

 

Special thanks to Nik Coli, who has been running these spreadsheets on his 183 and 294 and finding and fixing all my errors.

 

Steve


Edited by StevenBellavia, 12 August 2019 - 03:33 PM.

  • H-Alfa, h2ologg, PirateMike and 5 others like this

#2 niccoc1603

niccoc1603

    Ranger 4

  • -----
  • Posts: 318
  • Joined: 16 Aug 2018
  • Loc: Firenze, Italy

Posted 12 August 2019 - 03:53 PM

Thanks Steven, I couldn't think of imaging without your spreadsheets :) they are of such a great help!
I want to mention that I did a comparison (the 183) with Sharpcap sensor analysis and brain and it perfectly matched in terms of suggested optimal sub exposures

Nik

Edited by niccoc1603, 12 August 2019 - 03:58 PM.

  • StevenBellavia likes this

#3 H-Alfa

H-Alfa

    Ranger 4

  • -----
  • Posts: 386
  • Joined: 21 Sep 2006
  • Loc: Spain

Posted 12 August 2019 - 05:21 PM

Thank you Steve for sharing this. I will do a deep reading on it after holydays.

Enviado desde mi ANE-LX1 mediante Tapatalk

#4 basskep

basskep

    Explorer 1

  • -----
  • Posts: 72
  • Joined: 23 Aug 2018

Posted 10 September 2019 - 10:49 AM

Hi Steve thanks for sharing :-)

I have also a 294mcpro and always use the unity gain at 120. I can't figure out which is the best offset to use ? 8 ?

 

Thanks,

 

Alex 



#5 niccoc1603

niccoc1603

    Ranger 4

  • -----
  • Posts: 318
  • Joined: 16 Aug 2018
  • Loc: Firenze, Italy

Posted 10 September 2019 - 11:30 AM

8/9 will be fine

#6 basskep

basskep

    Explorer 1

  • -----
  • Posts: 72
  • Joined: 23 Aug 2018

Posted 11 September 2019 - 03:45 AM

Thank you for the advise. And why don't we use offset 30 as it is notified in the unity gain default value ?



#7 niccoc1603

niccoc1603

    Ranger 4

  • -----
  • Posts: 318
  • Joined: 16 Aug 2018
  • Loc: Firenze, Italy

Posted 11 September 2019 - 04:42 AM

You want the offset to ensure a min ADU value of a couple of hundreds, to avoid clipping of data, but not too high to cut into the Dynamic range of the camera

 

I did a series of measurement  targeting a min ADU value of ~300 at different gains and came up with the attached curve for Gain/Offset. 

 

An offset of 30 ensures no clipping for the whole range of gains, and will give you a min ADU of about 1600 at gain 120, which is ok, but I prefer to use a more precise value based on my measurements

Attached Thumbnails

  • offset.JPG

Edited by niccoc1603, 11 September 2019 - 04:48 AM.

  • StevenBellavia likes this

#8 basskep

basskep

    Explorer 1

  • -----
  • Posts: 72
  • Joined: 23 Aug 2018

Posted 11 September 2019 - 05:16 AM

Thank you for your reply. 

I will test with an offset at 8 and gain 120 it should be working fine.



#9 jdupton

jdupton

    Surveyor 1

  • *****
  • Posts: 1950
  • Joined: 21 Nov 2010
  • Loc: Central Texas, USA

Posted 11 September 2019 - 09:57 AM

Hi, 

 

   I am not sure I follow the reasoning behind this discussion of choosing an offset. The offsets suggested here seem exceedingly small to me. From my own testing, I would expect to see very significant clipping at offsets much below the ZWO default of 30.

 

   Maybe I am just looking at this incorrectly. Please tell me where I am missing something.

 

   I run my camera at a gain of 200 at the moment. I use the default Offset in the ASCOM driver of 30. I still see clipping in my calibrated images although it is minimal at a few hundred pixels out of the 11,694,368 present on the sensor.

 

   I note that these low Offset values discussed above are based on not clipping data in the individual raw images. That is, of course, good. However, that is not what I see as being most important. I do not see lowering the Offset to a point that no pixels are clipped is meaningful in any real way. Because of the random noise present in all the data, the thing to look at, in my mind, is that our calibrated images are not clipped. It is during image calibration that gross clipping can take place.

 

   To illustrate, I have these measurements from actual data taking as part of my latest project with my ASI294MC-Pro:

  • Gain (ASCOM) = 200
  • Offset (ASCOM) = 30
  • Exposure Time = 90 seconds
     
  • Integrated Master Dark (75 frames):
    Mean = 1942.0, Median = 1936.0, Minimum = 1874.2, Maximum = 65535.0
     
  • Sample single Raw Light Frame from 08/29/2019:
    Mean = 3652.5, Median = 3724.0, Minimum = 2328.0, Maximum = 65532.0
     
  • Calibrated Light Frame (Same Frame as Above) (Includes the effects of Flat Calibration):
    Mean = 2216.5, Median = 2342.0, Minimum = 0.0, Maximum = 65535.0
    Pixels Clipped = 189

   These measurements were taken within PixInsight using the Statistics process. Because of the clipping observed, I usually add a Pedestal during Image Calibration. If using an Offset much lower than the default of 30, I would think that a massive Pedestal would need to be added during Image Calibration to avoid clipping.

 

   Would someone here with data taken at a very low Offset value please post the information from their calibrated images? If you have PixInsight, just open a calibrated single image and open the Statistics Process. Then do the following:

  • Click the Track View icon in the lower right corner of the Process Window
  • Select the calibrated single image to make it active 
  • Check the "Unclipped" Option at the top of the Process Windows
  • Record the Mean, Median, Minimum, and Maximum values and report those here
     
  • Un-Check the "Unclipped" Option at the top of the Process Windows
  • Record the Count (px) value and subtract it from the total number of pixels on the chip (11694368). (This is the total number of Clipped Pixels in the image.)
  • Report back to us the number of clipped pixels you calculated.

 

   I have thought about increasing the default Offset of 30 I use so that I do not have to add a Pedestal during Image Calibration. A quick test I did perform of raising and lowering the Offset showed that at Gain = 200, the Offset added or subtracted about 64 ADU for each Offset change of 1. That value is likely different at other gain settings and it may not be completely linear as has been shown here.

 

   What measurements do others get from their calibrated images? Am I just looking at this methodology incorrectly? Where did I mess up my analysis?

 

 

John


Edited by jdupton, 11 September 2019 - 10:55 AM.


#10 niccoc1603

niccoc1603

    Ranger 4

  • -----
  • Posts: 318
  • Joined: 16 Aug 2018
  • Loc: Firenze, Italy

Posted 11 September 2019 - 10:34 AM

Hi, this approach is based on these guidelines
 
http://www.stark-lab...inAndOffset.php
 
Here my numbers:

Gain = 120
Offset = 8
Exposure Time = 120 seconds

Integrated Master Dark (50 frames):
mdark.jpg

Sample single Raw Light Frame :
light.jpg

Calibrated Light Frame (Same Frame as Above) (Includes the effects of Flat Calibration):
calibrated.jpg

unclipped.jpg

Pixel clipped:86 equal to 0.007% of total pixels, it could likely be some cold pixels that will be removed during Cosmetic Correction, I am not really worried about it.

Edited by niccoc1603, 11 September 2019 - 11:41 AM.

  • jdupton likes this

#11 jdupton

jdupton

    Surveyor 1

  • *****
  • Posts: 1950
  • Joined: 21 Nov 2010
  • Loc: Central Texas, USA

Posted 11 September 2019 - 04:28 PM

niccoc1603,

 

   Those measurements look very good. I did not expect to see so little clipping. You are correct -- 86 clipped pixels is nothing. I would start to look more closely when the number goes above about 600 (as a personal criteria) but less than a couple hundred will hardly be noticeable as you note.

 

   I ran some similar tests of Offsets more than year ago and found quite a bit of clipping. My tests may have been fundamentally different from what you ran. I will try to go back and rerun those tests later today to see if my camera's results agree with yours now. I won't be able to take any actual sky photos at low offset as it is cloudy today but can simulate a sky shot with a very dim flat frame for testing.

 

 

John



#12 jdupton

jdupton

    Surveyor 1

  • *****
  • Posts: 1950
  • Joined: 21 Nov 2010
  • Loc: Central Texas, USA

Posted 11 September 2019 - 09:56 PM

niccoc1603,

 

   I ran a test this evening and was unable to duplicate your results. I will describe what I tested and see if you can point out any differences compared to your procedures.

 

My Experiment:

  • Using SGP to capture the images of the test
     
  • ASI294MC-Pro Camera
    ASI ASCOM Driver
    Gain = 120 (Unity)
    Offset = 9
     
  • I took multiple frames of each type:
    50 Bias Frames
    25 Dark Frames @60 sec
    25 Light Frames @ 60 sec
         (These were actually just dim Flat Frames without an optical system -- just a diffuser.)
         (None the raw Light Frames Showed Clipping of Data.)
     
  • Integrated the Bias Frames
    Mean ADU = 569.269
    Median ADU = 569.061
    Minimum ADU = 532.720
    Maximum ADU = 743.818
     
  • Integrated the Dark Frames
    Mean ADU = 572.843
    Median ADU = 571.520
    Minimum ADU = 500.000
    ​Maximum ADU = 60888.321
     
  • Calibrated the Light Frames with only the Integrated (Master) Dark Frame
     
  • Used Image Statistics to examine each of the 25 Calibrated Light Frames.
     
  • All Calibrated light frames had clipped ADU values
    Minimum Number of Clipped Pixels = 698
    Maximum Number of Clipped Pixels = 27414
     
  • Integrated the 25 Calibrated Light Frames
    Mean ADU = 341.247
    Median ADU = 270.287
    Minimum ADU = 11.453 (No clipping of the integration)
    ​Maximum ADU = 5105.923

 

   What is different between what I tried and your own data? My test this evening seems to match my memory of what I found last year while playing with offsets on my camera.

 

   Is it possible that there is that much variation between samples of the camera? 

 

   Are there others who are using low value Offsets willing to try a quick test like this to see what they show? I am not sure what to make of the differences seen in our respective testing / data and maybe a third set of measurements will point to what is happening.

 

 

John


Edited by jdupton, 11 September 2019 - 09:58 PM.


#13 niccoc1603

niccoc1603

    Ranger 4

  • -----
  • Posts: 318
  • Joined: 16 Aug 2018
  • Loc: Firenze, Italy

Posted 11 September 2019 - 11:16 PM

These are my general recommendations:

1) do not use BIAS with a CMOS 
 
2) with the 294, flat frames exposure should be around 5s   (I use a led traceboard with a layer of 5%VLT film). I take 15 flats
 
3) take dark-flat frames (same settings as Flats), integrate them into a MasterDarkFlat and calibrate your ~5s flat frames with the MasterDarkFlat frame. Uncheck Optimize and Calibrate
 
flats.jpg
 
4) Integrate your calibrated flats into a MasterFlat
 
5) calibrate your light frames with MasterFlat and MasterDark (I take 50 dark fames and integrate them into a MasterDark). Do not check "Optimize" and "Calibrate" 
 
lights.jpg
 
references:
 
https://pixinsight.c...p?topic=11968.0
 
https://www.cloudyni...h-294mc-camera/

Edited by niccoc1603, 11 September 2019 - 11:22 PM.

  • basskep likes this

#14 jdupton

jdupton

    Surveyor 1

  • *****
  • Posts: 1950
  • Joined: 21 Nov 2010
  • Loc: Central Texas, USA

Posted 12 September 2019 - 08:01 AM

niccoc1603,

 

   You seem to have completely missed the point of my post regarding my test. Let me clarify.

 

For my test above, I did not:

  1. Use Bias frames for any calibration.
    The Bias frames were taken only to provide a value of the base ADU level generated by the Offset being set to 9.
  2. Take Flat Frames for use as calibration.
    I was only testing for clipping using dark calibration of a frame that simulated a normal light frame.

   I agree with all your points regarding process flow. Since I cannot take images outdoors at the moment due to rain, I took a Light Frame in a flat-like configuration to use as a simulated Light in my testing. I simply wanted light frames flooded with dim illumination to a point that it simulated the background ADU level you had used in your earlier post.

 

   To clarify the test processing procedure I used; it consisted of:

  • Take 25 60 sec Dark Frames
  • Take 25 60 sec Light Frames
  • Integrate the 25 Dark Frames (creating the Master Dark Frame)
  • Calibrate the 25 Light Frames with the Master Dark Frame (using your settings as above)
  • Use the Statistic Process to examine each of the 25 Calibrated Light Frames for Clipping.

 

   When doing actual imaging, I use 5 second flats and flat-darks and take 50 of each at the beginning of each imaging session. And yes, I am quite familiar with normal preprocessing flow and have no problems with calibrating my own normal images. The data and process used above were for testing clipping at low Offsets only.

 

   What the above test was meant to accomplish is an independent duplication of your result regarding no clipping of data when using very low values of Offsets. I was not able to do that. I found very significant clipping when using an Offset of 9.

 

   One thing has occurred to me since I ran the test last night. I was trying to think of what might be different between the data you posted and the test I ran. One thing different is that you were using longer exposures and your sampled Light Frame had a higher median ADU than mine. I think you were using 120 seconds and I ran my test at 60 seconds. That resulted in median ADU values in the Light Frames of 936 ADU for your data and 799 for mine. I am going to rerun my test using 90 second exposure times for the simulated lights and see if less clipping is seen. The increased exposure time should put my light frames closer to the median ADU values from your data.

 

   Other than the exposure difference and resultant median ADU difference, I cannot think of a reason your data shows essentially no clipping while my testing shows significant clipping of data during image calibration. Noise is a big factor. Maybe my camera is inherently noisier. That would probably increase the likelihood of clipping during calibration.

 

   Any other ASI294MC-Pro users willing to do this simple test and report results?

 

 

John



#15 basskep

basskep

    Explorer 1

  • -----
  • Posts: 72
  • Joined: 23 Aug 2018

Posted 12 September 2019 - 10:58 AM

Hi guys,

 

I could do some tests for you if you want with my ASI294MCPRO. I generally use the unity gain at 120 and offset at 30 (the default Ascom unity gain values). And I didn't noticed any visual changes when I process the frames in Pixinsight with an offset of 5 or 10. I need to grab the real mathematical datas from pix to be sure but I did all my shots last year with gain 120 and offset 5. Recently I used the unity gain at gain 120 and offset 30.

 

Niccoc1603, following your recommendations :

 

"These are my general recommendations:

1) do not use BIAS with a CMOS

Why is it not recommended to use bias frames with cmos ? I ask that because a lot of people are still doing bias frames with cmos (included me) and it doesn't seem to affect the image processing (maybe I'm wrong ?)


2) with the 294, flat frames exposure should be around 5s   (I use a led traceboard with a layer of 5%VLT film). I take 15 flats

Why 5s ? It seems to be a very long exposure for flats, doesn't it ? I usually take flat frames with exposures in milliseconds and then adjust the exposure to match the signal around 3/4 of the full histogram (using a flat field generator).

3) take dark-flat frames (same settings as Flats), integrate them into a MasterDarkFlat and calibrate your ~5s flat frames with the MasterDarkFlat frame. Uncheck Optimize and Calibrate"

I am very curious to follow your recommandations with dark-flat frames and see if there any difference in a resulting frame processed by each methods (bias and dark-flat)



#16 jdupton

jdupton

    Surveyor 1

  • *****
  • Posts: 1950
  • Joined: 21 Nov 2010
  • Loc: Central Texas, USA

Posted 12 September 2019 - 12:53 PM

basskep,

 

   I would love for you to examine some of your past data! Since you already have targets that were shot at Unity Gain (120) and an Offset of 5, not much work is required on your part to check the data.

 

   All you would need to do is open (or post to Dropbox or Google Drive or similar) one of your Calibrated Subs and examine it for clipped pixel data. Do you have PixInsight? that is the easiest way to check for clipping after calibration. If you have PixInsight, do the following:

  • Open a copy of one of the subs you have already calibrated
     
  • Open the PixelMath Process
    Enter the Following equation in the RGB/K entry area (first line) Don't enter the quotes
    iif($T==0, 0.9999999, 0.0)
     
  • Now apply the PixelMath process to the opened calibrated frame
     
  • Next, open the Statistics Process in PixInsight
     
  • Turn off the "Unclipped" option box in the upper right of the Statistics process window
     
  • Turn on the "Track View" option in the lower right of the Statistics process
     
  • Click on the image you applied the PixelMath onto
     
  • Read off and report the number shown in the Statistics Process to the right of "count (px)"
    This is the number of clipped pixels in your calibrated image.
     
  • close but do not save your image -- it has been modified.

   If you can, repeat the above procedure for several more of your calibrated subs. Also try it on calibrated subs from other imaging sessions / targets. Each will show a different clipping level. Ideally, all the clipped pixel counts should be lower than 600 to 800. If they are more than 1000, then the clipping will be more likely to persist even if you add the maximum possible Pedestal in the ImageCalibration process.

 

 

John


  • basskep likes this

#17 niccoc1603

niccoc1603

    Ranger 4

  • -----
  • Posts: 318
  • Joined: 16 Aug 2018
  • Loc: Firenze, Italy

Posted 12 September 2019 - 12:57 PM

@basskep

you can find detailed explanations to points 1 and 3 in this thread
https://pixinsight.c...p?topic=11968.0
Also, CMOS sensors behavior at very short exposures is not linear, resulting in a not proper calibration


For 5s flats there are several discussions here on CN on calibration issues with flats and the 294 sensor, and the general consensus is that those issues are mitigated or eliminated by taking 5s flats
for example here: https://www.cloudyni...h-294mc-camera/

@jdupton
I am glad you know your way around calibration. n your first test you were mentioning 189 clipped pixels after calibration, that is still 0.01% of total pixels.
I personally wouldn't be worried about this number, but that is up to you to define your boundaries.
I am using sky lights form an imaging session in the above statistics
  • basskep likes this

#18 jdupton

jdupton

    Surveyor 1

  • *****
  • Posts: 1950
  • Joined: 21 Nov 2010
  • Loc: Central Texas, USA

Posted 12 September 2019 - 01:19 PM

basskep,

 

   I hope you don't mind my also answering your questions.

 

Niccoc1603, following your recommendations :

 

"These are my general recommendations:

1) do not use BIAS with a CMOS

Why is it not recommended to use bias frames with cmos ? I ask that because a lot of people are still doing bias frames with cmos (included me) and it doesn't seem to affect the image processing (maybe I'm wrong ?)


2) with the 294, flat frames exposure should be around 5s   (I use a led traceboard with a layer of 5%VLT film). I take 15 flats

Why 5s ? It seems to be a very long exposure for flats, doesn't it ? I usually take flat frames with exposures in milliseconds and then adjust the exposure to match the signal around 3/4 of the full histogram (using a flat field generator).

3) take dark-flat frames (same settings as Flats), integrate them into a MasterDarkFlat and calibrate your ~5s flat frames with the MasterDarkFlat frame. Uncheck Optimize and Calibrate"

I am very curious to follow your recommandations with dark-flat frames and see if there any difference in a resulting frame processed by each methods (bias and dark-flat)

 

   I agree with niccoc1603's recommendations. In addition to the reference he has given you, you might want to also read through the thread I started last year specifically concerning calibration of data from the ASI294MC-Pro camera. I did extensive testing last summer and the thread summarizes my characterization of the camera. Be warned, it is a very long detailed thread but explains in detail why Bias Frames should not be used with with this camera and why Flat Frames should be longer than 3 seconds minimum. I generally use 5 to 7 second Flat Frames but anything longer than three is pretty safe. Using Flat-Dark Frames for Flat Frame Calibration is a consequence of not being able to reliably use Bias Frames.

 

   The long story and thread is here:
https://www.cloudynights.com/topic/636301-asi294mc-calibration-%E2%80%93-testing-notes-thoughts-and-opinions/

 

   While Bias Frames are not used anywhere in calibration with this camera, I do strongly recommend that they be captured in a very specific way to help prevent another odd quirk with this camera. Details are discussed in the above thread near the end. I take Bias frames every single session interspersed between normal frames. The Bias Frames prevent a potential problem with incomplete reset of the pixel wells between exposures.

 

 

John


  • basskep likes this

#19 jdupton

jdupton

    Surveyor 1

  • *****
  • Posts: 1950
  • Joined: 21 Nov 2010
  • Loc: Central Texas, USA

Posted 12 September 2019 - 01:44 PM

niccoc1603,

 

@jdupton
I am glad you know your way around calibration. n your first test you were mentioning 189 clipped pixels after calibration, that is still 0.01% of total pixels.
I personally wouldn't be worried about this number, but that is up to you to define your boundaries.
I am using sky lights form an imaging session in the above statistics

 

   You are correct, 189 clipped pixels is of no consequence. That level of clipping was found in a recent set of data I captured at and Offset of 30. Like your value of 86 clipped pixels in your data, it will not affect the final image.

 

   I am concerned by the test results I got last night where up to 27,000 pixels were clipped when using an Offset of 9 at Unity Gain. That was worrisome.

 

   I reran the test again this morning and used a higher median ADU for the average light frame. I pushed it brighter from about 800 to about 950 ADU. This did reduce the amount of clipping during image calibration. For this run, the number of pixels clipped per frame ranged from about 100 to just over 15,000. I am ready to conclude that brighter Light Frame medians reduce the amount of data clipped during calibration.

 

 

Steve,

 

   This then circles back the your original post for this thread. I was worried that the recommended Offsets coupled with using only exposures long enough to overcome read noise might lead to excessive clipping in the calibrated images. My rough initial testing seems to show this can happen. I am not sure how this concept might be incorporated into the spreadsheet or even if it should be. 

 

   What are your thoughts, Steve? Choosing Offsets just large enough to prevent clipping of the data being gathered doesn't seem to be the whole story. Shouldn't the optimum exposure take into account any clipping that will occur during image calibration? I don't know the answer but would like to see your and others' opinions.

 

 

John



#20 basskep

basskep

    Explorer 1

  • -----
  • Posts: 72
  • Joined: 23 Aug 2018

Posted 13 September 2019 - 03:33 AM

Thank you John for your comments :-)

Sorry guys I will read in details all the thread to better understand the recommendations done here.

 

So for the offset at unity gain what do you recommend to use ? Offset 8 or 9 or 30 (default unitygain preset) ? 

 

Just for information here is my last captures from IC5146 with a 45x300s at unity gain 120 Offset 30 (200 bias, 61 flats, 21 darks) But I understand your recommandations to not use bias and do dark flats and flats >5s

I will try these settings asap and see the results 

 

OubtBeQ0NQkp_1824x0_wmhqkGbg.jpg


  • artem2 likes this

#21 basskep

basskep

    Explorer 1

  • -----
  • Posts: 72
  • Joined: 23 Aug 2018

Posted 13 September 2019 - 08:49 AM

basskep,

 

   I would love for you to examine some of your past data! Since you already have targets that were shot at Unity Gain (120) and an Offset of 5, not much work is required on your part to check the data.

 

   All you would need to do is open (or post to Dropbox or Google Drive or similar) one of your Calibrated Subs and examine it for clipped pixel data. Do you have PixInsight? that is the easiest way to check for clipping after calibration. If you have PixInsight, do the following:

  • Open a copy of one of the subs you have already calibrated
     
  • Open the PixelMath Process
    Enter the Following equation in the RGB/K entry area (first line) Don't enter the quotes
    iif($T==0, 0.9999999, 0.0)
     
  • Now apply the PixelMath process to the opened calibrated frame
     
  • Next, open the Statistics Process in PixInsight
     
  • Turn off the "Unclipped" option box in the upper right of the Statistics process window
     
  • Turn on the "Track View" option in the lower right of the Statistics process
     
  • Click on the image you applied the PixelMath onto
     
  • Read off and report the number shown in the Statistics Process to the right of "count (px)"
    This is the number of clipped pixels in your calibrated image.
     
  • close but do not save your image -- it has been modified.

   If you can, repeat the above procedure for several more of your calibrated subs. Also try it on calibrated subs from other imaging sessions / targets. Each will show a different clipping level. Ideally, all the clipped pixel counts should be lower than 600 to 800. If they are more than 1000, then the clipping will be more likely to persist even if you add the maximum possible Pedestal in the ImageCalibration process.

 

 

John

Hi John, 

 

I could send you all the required datas you need.



#22 jdupton

jdupton

    Surveyor 1

  • *****
  • Posts: 1950
  • Joined: 21 Nov 2010
  • Loc: Central Texas, USA

Posted 13 September 2019 - 09:01 AM

basskep,

 

I could send you all the required datas you need.

 

   I don't have a mailbox large enough to receive files directly. Do you have a DropBox free account or a other file sharing account like Google Drive? If so, you could upload one or two sample calibrated subs there and send me a PM (private message; option available when you hover over my CN ID) with a link to the files that I can then download.

 

   I am interested in single calibrated subs taken using a low offset. I have lots of data of my own taken at an Offset of 30 but need real sky target data taken at an offset of 4 to 10 to examine. I only need a couple of calibrated subs at most -- not a full set of data.

 

 

John



#23 jdupton

jdupton

    Surveyor 1

  • *****
  • Posts: 1950
  • Joined: 21 Nov 2010
  • Loc: Central Texas, USA

Posted 13 September 2019 - 09:42 AM

niccoc1603, StevenBellavia,

 

   I must withdraw my concern about excessive clipping of data during Image Calibration.

 

   After yet another data run yesterday and more intensive slicing and dicing of the data, I have found that the large number of pixels clipped to zero during ImageCalibration is not a great concern after all, if and only if a sizable Pedestal is used during the calibration process.

 

   My error in thinking about the clipping is that clipped pixels would fall into a somewhat normalized distribution. They will not. The distribution is quite skewed such that most are clustered near the clipping point.

 

   I went back to yesterday's data and selected the sub that had over 15,000 pixels clipped during calibration. I created a Mask identifying the location of all 15,000+ clipped pixels. Next, I applied that mask to the corresponding uncalibrated Light Frame to restore the uncalibrated values of the pixels that had been clipped later in processing. I examined the statistics of those original pixels. Further, I subtracted 800 ADU from each of those pixels to see how many would remain clipped following Image Calibration with an 800 ADU Pedestal added.

 

   I had expected a significant portion of those pixels would be "very clipped" and survive even a large Pedestal. Since the distribution of the clipped pixels is not "normal", that doesn't happen. When evaluated with an expected Pedestal of 800 ADU, the number of clipped pixels dropped from 15,000+ to less than a dozen.

 

   So, my conclusion now is that while using a low value for Offset can clip a very large number of pixels, the use of a sizable Pedestal during Image Calibration will prevent any large loss of background data. This meshes well with the recommendation from Jon Rista that for many CMOS cameras, we should probably always use a Pedestal of somewhere between 800 up to the maximum of 1000 for all calibration work.

 

   The significant clipping during calibration doesn't happen (at least in my experience) when using larger default offsets such as the ZWO recommended 30. The larger Offset makes the use of a large (or any) Pedestal during calibration much less important as the larger Offset acts to "cushion" the background data.

 

   I would still like to see some actual calibrated data from other users who routinely use low Offset values with or without using a Pedestal during Image Calibration.

 

   Given this information, I think I will now try using a lower value of Offset for my next imaging session. (It may be a week before we have good weather again, though.) Knowing that background signal clipping due to low offset values can be controlled with the use of a large Pedestal during calibration, I can see the value of this approach. Until I am completely comfortable with the concept, I will probably spot check the first few frames of an imaging session to ensure only minimal clipping will occur during later processing. (I tend to be overly cautious about losing all data from an otherwise promising imaging session.)

 

 

John


Edited by jdupton, 13 September 2019 - 08:38 PM.

  • Jon Rista and OldManSky like this

#24 basskep

basskep

    Explorer 1

  • -----
  • Posts: 72
  • Joined: 23 Aug 2018

Posted 13 September 2019 - 09:42 AM

Yes John I will upload the files and give you the link



#25 phf

phf

    Lift Off

  • -----
  • Posts: 2
  • Joined: 30 Jul 2018

Posted 13 September 2019 - 01:36 PM

Hi Steve and others,

 

Thanks for the sheet!

 

I am sorry for asking but I am a newbie. I own an ASI294MC pro. My understanding of the table in the Exposure Calculator tab is that we theoretically don't need the long subs (60s, 180s, 300s...) that people normally use. With an urban sky, subs exposure of 10s-15s are enough at more or less any gain (above unity gain). The inconvenient is that we then need hundreds or thousands of subs to reach the desired integration time. Is that a correct interpretation? Can we have longer exposure though? I would love to have a Gain and Sub-exposure tables such as the one proposed in https://www.cloudyni...olour-versions/ applicable for the 294MC would be great to help me understand better. 

 

Many thanks again. 

 

Philippe




CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.


Recent Topics






Cloudy Nights LLC
Cloudy Nights Sponsor: Astronomics