Jump to content

  •  

CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.

Photo

QHY183M Amp Glow elimination workflow

Astrophotography CMOS Equipment Imaging Software
  • Please log in to reply
18 replies to this topic

#1 adamrosner

adamrosner

    Lift Off

  • -----
  • topic starter
  • Posts: 10
  • Joined: 19 Sep 2018
  • Loc: Wellington New Zealand

Posted 25 January 2023 - 03:30 AM

Greetings all

 

I'm experiencing some difficulty with amp glow not calibrating out and I want to get a better idea of what I'm doing wrong.

 

I am operating from the working hypothesis that it's because I am over-stretching my images, but I'd like to get a bit of wisdom from others here who may have defeated this particular monster.

 

My gear is entry-level mono: QHY183M with William Optics Zenith Star 6.1 (=focal length 360mm, f/5.9)

Mount and guiding etc are unimportant to this conversation. I can get perfectly round stars at all exposure lengths I've tried (up to 360sec) and I've no cause to doubt I could easily push that to 600

Filters are Optolong 1.25" - the 6.5nm ones (Ha, SII and OIII)

Rounding it off is a Pegasus FocusCube2.

 

Attached is a processed Ha stack.

 

Subs are 300Sec at gain 20 (unity on this cam is 10, valid range = 10 to 54) with the sensor at -20ºC and an offset (bias) of 40

This is a stack of 20 subs

 

Darks are the same settings as the lights, just with the lens cover on

 

Bias frames are same temp and offset and gain, min exposure (0.0005sec)

 

Flats are the same as well, using an iPad screen with camera at zenith, usually around 1sec

 

No dark flats

 

USB Readout speed is 0 (zero) for all - lights, darks, flats, bias.

 

Stacking - I'm using DSS at its recommended settings, ensuring "dark frame optimization" is not checked.

 

Still with me? Cool. So if you take a look at the jpg you can clearly see the amp glow hasn't fully calibrated out. And worse, it's thrown lines across the image as well.

 

So - my hypothesis (please either validate or correct if wrong) is that I'm not gathering enough light, and consequently having to over-stretch the image, to the point where I've exceeded the ability of DSS to calibrate out the remaining amp glow. And that the fix for this is either or all of:

  • Shoot at higher gain, or
  • Shoot longer (600s subs?), or
  • Use a faster optic

Side note I haven't had this problem shooting LRGB (or maybe it's still there but I am just not seeing it because I don't stretch those so hard)

 

I can put up a sample FIT file from the camera, and the PSD that generated this jpg, onto a cloud somewhere if anyone wants to download it and have a play.

 

If anyone has any thoughts I'll welcome it.

 

Edit\Add: I've put the PSD, and the darks, lights and bias frames on a download link for anyone so inclined to download 1.2GB https://1drv.ms/u/s!...fETy_A?e=Vn93v8

Attached Thumbnails

  • Ha2048.jpg

Edited by adamrosner, 25 January 2023 - 03:50 AM.

  • Sheridan likes this

#2 OhmEye

OhmEye

    Apollo

  • *****
  • Posts: 1,074
  • Joined: 15 Sep 2019
  • Loc: Western NY Southern Tier

Posted 25 January 2023 - 08:25 AM

I think not using dark flats is the issue and you need to remove amp glow from the flats as well as the lights. I was not successful at using bias to calibrate flats with my 183.


  • Yoddha likes this

#3 FiveByEagle

FiveByEagle

    Viking 1

  • *****
  • Posts: 792
  • Joined: 04 Mar 2020

Posted 25 January 2023 - 09:52 AM

I second that - Bias didnt do anything for calibration on my 183.

 

It was a fickle beast - make sure your darks are perfect as well. I had a tiny, TINY light leak that raised ADU by about 100 (not alot) and it would cause amp glow removal artifacts.



#4 unimatrix0

unimatrix0

    Soyuz

  • *****
  • Posts: 3,642
  • Joined: 03 Jan 2021

Posted 25 January 2023 - 09:57 AM

Try taking new darks, as well as check the exposure time on your flats and just make a new set of flat-darks (dark flats whatever) that matches the exposure time of the flats and see what happens.

The 183 sensor is a bit finicky to say the least but can make good images, it's just dialing in the correct settings. 



#5 idclimber

idclimber

    Soyuz

  • *****
  • Posts: 3,882
  • Joined: 08 Apr 2016
  • Loc: McCall Idaho

Posted 25 January 2023 - 11:43 AM

I do not see flats in your one drive link. I also do not see that you mentioned them in your original post. It would be best if you uploaded one Light, Dark, Flat, and Bias frame for us to analyze and calibrate. 

 

Alternatively you can add a dark flat, which is nothing more that a exposure matched dark frame for the calibration of the flat. 

 

If you do not have flats, then there is no need for a bias or a dark flat. 



#6 matt_astro_tx

matt_astro_tx

    Fly Me to the Moon

  • -----
  • Posts: 5,369
  • Joined: 19 Jan 2021
  • Loc: Dallas, Texas

Posted 25 January 2023 - 01:00 PM

Strange.  I have the ASI183 and darks completely eliminate my amp glow.  I don't think switching from bias to dark flats is going to have any effect on it.

 

How old are the darks you're using?  I typically reshoot my darks every six months.



#7 bulrichl

bulrichl

    Messenger

  • *****
  • Posts: 439
  • Joined: 27 May 2018
  • Loc: La Palma (Canary Islands)

Posted 25 January 2023 - 01:33 PM

I downloaded the archives and generated MasterBias and MasterDark. These are the statistics:

            MB          MD
count (%)   100.00000   100.00000
count (px)  20424096    20424096
mean        2564.514    2511.815
median      2564.267    2498.667
variance    238.607     50364.525
stdDev      15.447      224.420
avgDev      15.024      35.550
MAD         15.023      20.559
minimum     1564.000    1935.429
maximum     4147.734    65504.000

It is weird that the MB has larger mean and median values than the MD. According to the FITS header, all images are captured using the same image acquisition software, same temperature, gain and offset. I am puzzled by this result.

 

Calibrating the light frames with the MD in PixInsight's ImageCalibration process worked correctly - no "amplifier glow" can be detected in the calibrated images nor (after registration) in the integration results. I had to use a pedestal of 500 DN in order to avoid clipping in the low range.

 

When the light frames were calibrated with MD and MB using dark frame optimization, a trace of "amplifier glow" is detectable as slight brightening. However, for images of cameras showing strong "amplifier glow" the use of dark frame optimization is not recommended anyway.

 

Which software was used for image calibration?

 

Bernd

 


  • jdupton likes this

#8 jdupton

jdupton

    Aurora

  • *****
  • Posts: 4,897
  • Joined: 21 Nov 2010
  • Loc: Central Texas, USA

Posted 25 January 2023 - 01:57 PM

adamrosner,

 

I'm experiencing some difficulty with amp glow not calibrating out and I want to get a better idea of what I'm doing wrong.

 

[snip]

 

My gear is entry-level mono: QHY183M with William Optics Zenith Star 6.1 (=focal length 360mm, f/5.9)

Mount and guiding etc are unimportant to this conversation. I can get perfectly round stars at all exposure lengths I've tried (up to 360sec) and I've no cause to doubt I could easily push that to 600

Filters are Optolong 1.25" - the 6.5nm ones (Ha, SII and OIII)

Rounding it off is a Pegasus FocusCube2.

   First of all, I am not convinced you are doing anything wrong. I downloaded all the frames you made available and see some problems but this puzzle you have presented us doesn't seem to be something obvious.

 

   I notice that you did not upload any Flat frames. Did you use Flats for the image in your opening post? It really doesn't matter because your images do not show any dust shadows at all and any vignetting appears to be very minor if any is present at all.

 

   Since Flat frames didn't seem to be needed and were not provided, I processed your Light using only the Dark frames. I could not duplicate your result of obvious under-correction by the Dark frame calibration but see other issues with the Amp Glow. (More on that later. It is mostly an issue with residual Amp Glow noise in my final results.)

 

    Here is the puzzle I see in your file set:

  • Good news, all frames matched in terms of Gain, Offset, and Temperature. This is always the first thing to check.
     
  • More good news; all frames were captured with the same program and driver mode, it appears. In addition, all frames were taken within a day of one another. The calibration frames were fresh.
     
  • Bad News; even though I didn't use the Bias frames, I found that they are brighter on average than the Dark frames. That should never ever happen. They matched in every respect but were just plain wrong. That is a red flag in this puzzle.
     
  • Even more Bad News; Even though there is no clipping in either the Lights or the Darks, massive clipping is happening when the Darks are subtracted from the Lights during image calibration. I even add the maximum PixInsight allowable Output Pedestal of 1000 ADU during calibration and still had significant clipping.

    I think this clipping is what was causing the issues with image integration (stacking). I seem to recall Jon Rista mentioning a similar issue when he tested the IMX183 sensor. He recommended adding an 800 ADU Output Pedestal during calibration. In your data, even that was not enough to prevent the clipping and loss of data.

   I tried several things to try to get a good integration of the calibrated lights. I came closest to success using one of two methods. First, I simply used an extremely tight pixel rejection sigma factor during stacking. That masked the bright streaks but left dark areas at the right edge of the image. Secondly, I violated all normal processing and stacked without removing the 1000 ADU pedestal. That gave a pleasing image but the right edge showed a blurred dark area where the peak of the Amp Glow once resided.

 

   In the end, there is something different between the Lights, Darks, and Bias frames you supplied us to examine. I cannot see a clear root cause but have a couple of suspicions. They are:

  • Your cooling to -20°C may have maxed out the cooling system and the sensor may be a different temperature than what is being reported. I note that the ambient is recorded as being about +16°C for the Lights and about +25°C for the Darks and Bias.

    You might try a different session where you cool to only about -5°C or -10°C for all frame types. I am not familiar with the internals of the QHY cameras to know how the cooling is attached to the back of the sensor and where the probe that records sensor temperature is placed. Those things can lead to the sensor being at a temperature different from what is being reported under some circumstances with ZWO cameras. Maybe the same is true for QHY. In any case, using much more modest cooling may give you different results.
     
  • Try taking a new set of Bias and Darks an check their average brightness level. A Bias frame should always have a lower average ADU value than a Dark frame. In turn, the Dark frame should always have a lower average brightness value than the Light frames. Somehow, yours are not correct. I don't see any obvious cause but I think this is a big part of the problem.

    I will note, though, that if the Dark frame showed up as bright or brighter than the Bias frames as it normally should, the clipping problem of losing a massive number of pixel data in image calibration would be made much worse. That means there is a puzzle on top of a puzzle. I see nothing obvious in the frames to reconcile any of this.

   If you happen to reshoot Bias and Dark frames, let us know what you see or upload them for us to examine. I will continue watching this thread since this one really has me scratching my head as to what could be going on.

 

 

John


Edited by jdupton, 25 January 2023 - 02:06 PM.

  • idclimber likes this

#9 adamrosner

adamrosner

    Lift Off

  • -----
  • topic starter
  • Posts: 10
  • Joined: 19 Sep 2018
  • Loc: Wellington New Zealand

Posted 25 January 2023 - 08:40 PM

In that stack I didn't use any flats or dark flats. I think the question about the ambient temp during shooting darks and bias frames might be the issue, I'll try re-shooting them and find somewhere cooler to put the camera. Middle of summer so it's quite warm... might stand the camera directly under the air conditioning unit.

 

I'll report back. Might take a while though. Gotta shoot a whole new darkframe library



#10 imtl

imtl

    Fly Me to the Moon

  • *****
  • Moderators
  • Posts: 6,634
  • Joined: 07 Jun 2016
  • Loc: Down in a hole

Posted 25 January 2023 - 09:18 PM

Try to reduce the gain to unity. In your case gain of 10. The 183 does not play that well high above unity and you're actually gaining nothing with this camera by going above unity. If anything you're just losing dynamic range.

Put gain of 10. Take calibration frames again. Recapture your lights with the new settings. And see if ampglow is persistent.



#11 Kevin_A

Kevin_A

    Apollo

  • *****
  • Posts: 1,007
  • Joined: 22 Aug 2018
  • Loc: Belmont, Ontario Canada

Posted 25 January 2023 - 09:25 PM

No bias frames with this sensor…. Just darks, flats at about 4-10 seconds and matching 4-10 second flat darks…. Done. Zero amp glow!


Edited by Kevin_A, 25 January 2023 - 09:26 PM.


#12 adamrosner

adamrosner

    Lift Off

  • -----
  • topic starter
  • Posts: 10
  • Joined: 19 Sep 2018
  • Loc: Wellington New Zealand

Posted 26 January 2023 - 02:31 AM

I did a reprocess - all I did was exclude the bias frames from DSS, then re-stacked and did a quick 'n dirty stretch then put them together in Photoshop in HOO and removed the stars.

 

Result: amp glow gone. Not a trace.

 

So in this instance, empirical results contradicts established wisdom.

 

Though, in this quick 'n dirty, I used no flats because I haven't shot them yet at this camera rotation.

Attached Thumbnails

  • RosetteReproNoBias_STARLESS_2048.jpg

  • Sheridan likes this

#13 idclimber

idclimber

    Soyuz

  • *****
  • Posts: 3,882
  • Joined: 08 Apr 2016
  • Loc: McCall Idaho

Posted 26 January 2023 - 02:51 AM

If you are using bias without flats then you were not using established wisdom. 

 

The current calibration flow with CMOS sensors is to subtract a Master bias or Master dark flat from each flat frame. Then integrate to a Master Flat. The Master Dark is subtracted from the light and division step is used to apply the Master Flat. 

 

There were methods that incorporate bias frames and scaling darks, but that only worked on CCD sensors. 



#14 adamrosner

adamrosner

    Lift Off

  • -----
  • topic starter
  • Posts: 10
  • Joined: 19 Sep 2018
  • Loc: Wellington New Zealand

Posted 26 January 2023 - 03:52 AM

Also - thanks everyone for your comments and suggstions. I love this community :)



#15 bulrichl

bulrichl

    Messenger

  • *****
  • Posts: 439
  • Joined: 27 May 2018
  • Loc: La Palma (Canary Islands)

Posted 26 January 2023 - 05:39 AM

In that stack I didn't use any flats or dark flats. I think the question about the ambient temp during shooting darks and bias frames might be the issue, I'll try re-shooting them and find somewhere cooler to put the camera. Middle of summer so it's quite warm... might stand the camera directly under the air conditioning unit.

 

I'll report back. Might take a while though. Gotta shoot a whole new darkframe library

Like John, I also noticed a pecularity regarding pixel rejection.

I used Winsorized Sigma clipping (Sigma low: 4.0, Sigma high: 3.0). The rejection_high map looked normal, but the rejection_low map contained signal outside the nebula and especially in the "amplifier glow" region. I did not pay attention to that because the total fraction of rejected pixels was not too high:


Total :   3280355   0.803% (   104517 +   3175838 =   0.026% +   0.777%)

The observation of John that ambient temperature was in the range of 25 °C for dark and bias frames whereas in the range of 16.5 °C for light frames seems indeed to be the crucial point. The manufacturer specifies a delta T of 40 K below ambient. In case of the light frames (16.5 °C - 40 K = -23.5 °C) this should not pose a problem for the cooling system, but in case of the calibration frames (25 °C - 40 K = -15 °C) a set temperatur of -20 °C is not reachable according to the specification.


Nevertheless, the keyword CCD-TEMP documented in the FITS header has a value of -20 °C for ALL frames. However, I suppose that the temperature sensor in the QHY183 does not correctly measure the true temperatur of the CMOS sensor. So the value of CCD-TEMP does not tell us whether the cooling system is working in the specified range or if it is overloaded. One should always keep an eye on the cooling power. For Thermoelectric Coolers (Peltier modules) it is important not to be used in the maximum cooling power range. So I watch the cooling power and ensure that a value of 70 % is not exceeded.

In my view, the conclusion of the OP is right: take the calibration frames at lower ambient temperature, similar to the conditions of light frame acquisition!


I am curious whether the other weirdness (mean and median of the bias frames > mean and median of the dark frames) will be fixed as well with such new calibration frames. Please post the results!

Bernd


  • jdupton likes this

#16 jdupton

jdupton

    Aurora

  • *****
  • Posts: 4,897
  • Joined: 21 Nov 2010
  • Loc: Central Texas, USA

Posted 26 January 2023 - 11:27 AM

adamrosner,

 

In that stack I didn't use any flats or dark flats. I think the question about the ambient temp during shooting darks and bias frames might be the issue, I'll try re-shooting them and find somewhere cooler to put the camera. Middle of summer so it's quite warm... might stand the camera directly under the air conditioning unit.

   I think you missed the point I was making about the ambient temperature observation. I was not suggesting always shooting calibration frames inside an artificially cooled environment. I was advocating for not running any of your image capture at a -20°C set-point temperature. For many of us, that is needlessly low.

 

   I think running at a -20°C set-point in a 25°C ambient temperature environment is out of the specifications of your camera. The TEC cooler in the camera will have trouble reaching that temperature. In addition, for most CMOS cameras, running at -20°C is not needed. You can get excellent very low thermal current and noise results by running at only -10°C or -5°C instead.

 

   In Bernd's later post (just above here), he touched on why I mentioned the ambient temperatures in my first response. More on that below...

 

I did a reprocess - all I did was exclude the bias frames from DSS, then re-stacked and did a quick 'n dirty stretch then put them together in Photoshop in HOO and removed the stars.

 

Result: amp glow gone. Not a trace.

 

So in this instance, empirical results contradicts established wisdom.

   What you tried is not contrary to established wisdom. It is exactly what most will say is best practice. When you are not using Flat frames, it is best not to give the stacking software any Bias frames to work with. Many of the programs that do stacking will attempt to scale (modify) the Dark frames when also given Bias frames but no Flats. They can use the Bias to attempt to better match the Dark frames to Lights. That techniques does not work well when using a camera that shows Amp Glow. 

 

   So, best practice is that when not using Flat frames, give the stacking software only Lights and Darks to work with. It will be forced to only do normal calibration and not attempt to use the Dark Frame Noise Scaling methodology to try for a better result. When the camera being used has Amp Glow present, the result will almost never be better.

 

Nevertheless, the keyword CCD-TEMP documented in the FITS header has a value of -20 °C for ALL frames. However, I suppose that the temperature sensor in the QHY183 does not correctly measure the true temperatur of the CMOS sensor. So the value of CCD-TEMP does not tell us whether the cooling system is working in the specified range or if it is overloaded. One should always keep an eye on the cooling power. For Thermoelectric Coolers (Peltier modules) it is important not to be used in the maximum cooling power range. So I watch the cooling power and ensure that a value of 70 % is not exceeded.

In my view, the conclusion of the OP is right: take the calibration frames at lower ambient temperature, similar to the conditions of light frame acquisition!


I am curious whether the other weirdness (mean and median of the bias frames > mean and median of the dark frames) will be fixed as well with such new calibration frames. Please post the results!

   Bernd nailed my thoughts in his summary here. I know some ZWO cameras use a thermal sensor placement that does not actually measure the temperature of the image sensor of the camera. There can be a difference. My suspicion was that the image sensor in your camera is not really at the -20°C temperature the camera seems to be reporting. The TEC cooler may have reached that temperature but the sensor was likely lagging behind by some amount that we can never know and may be variable.

 

   The real check of whether you are operating the actual image sensor at a stable set-point temperature is to look at the TEC Cooler power being used. If it is at 100%, then the actual temperature will not (and cannot) be stable. Stable is what we want above all else.

 

   Bernd's recommendation for running the TEC Cooler at no more than 70% power is optimum. That means that you have enough "headroom" in the cooling system to keep the temperature very stable. When running at 100% power, the actual temperature will not be stable. (In fact, for ZWO cameras, running at 100% will actually heat the edges of image sensor via waste heat conduction from the camera case while it is being cooled by the TEC in the middle of the sensor. I don't know the internal design of the QHY cameras but that same effect could happen with them also.)

 

 

John


  • idclimber likes this

#17 adamrosner

adamrosner

    Lift Off

  • -----
  • topic starter
  • Posts: 10
  • Joined: 19 Sep 2018
  • Loc: Wellington New Zealand

Posted 26 January 2023 - 03:50 PM


 

   Bernd's recommendation for running the TEC Cooler at no more than 70% power is optimum. That means that you have enough "headroom" in the cooling system to keep the temperature very stable. When running at 100% power, the actual temperature will not be stable. (In fact, for ZWO cameras, running at 100% will actually heat the edges of image sensor via waste heat conduction from the camera case while it is being cooled by the TEC in the middle of the sensor. I don't know the internal design of the QHY cameras but that same effect could happen with them also.)

 

 

John

Indeed - I did notice when shooting the bias frames, that the power indication on the cooler was hovering around 92%.

 

So, -20°C is un-necessarily low? Good to know. I based it on what others with the same cam were using after searching around for representative images from it on Astrobin.

 

I'll try doing a run at -10° instead, but it means I'll need a whole new dark frame library as well. I believe the forecast is for a cloudy weekend.
 



#18 Sheridan

Sheridan

    Apollo

  • *****
  • Posts: 1,302
  • Joined: 12 Oct 2005
  • Loc: Houston Tx.

Posted Yesterday, 09:27 AM

What are you imagine with Program wise? Just curious.

#19 matt_astro_tx

matt_astro_tx

    Fly Me to the Moon

  • -----
  • Posts: 5,369
  • Joined: 19 Jan 2021
  • Loc: Dallas, Texas

Posted Yesterday, 09:39 AM

So, -20°C is un-necessarily low? Good to know. I based it on what others with the same cam were using after searching around for representative images from it on Astrobin.

Run the cooler at a temp that is below ambient and as John said doesn't utilize too much power.  In Texas I am (surprisingly) able to maintain -10C year round with my 183.  But frankly 0C would be fine.


Edited by matt_astro_tx, Yesterday, 09:40 AM.



CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.


Recent Topics





Also tagged with one or more of these keywords: Astrophotography, CMOS, Equipment, Imaging, Software



Cloudy Nights LLC
Cloudy Nights Sponsor: Astronomics