17 replies to this topic

[RoscoeD]

I've seen multiple posts where folks state that they use a master dark library.  Since Darks have to be at the same temp and same exposure duration (correct?), I assume that folks are then shooting the same duration & temp for all shots? I would guess this would mean finding an exposure time that won't blow out the stars and stick with that for all imaging?

 

Just trying to plan ahead...note for reference I have a 294MC Pro and an F/4.7 scope (with flattener, F/5.9 without)

[Michael Covington]

Right.  Using a DSLR that doesn't have temperature control, I've standardized on 2-minute exposures, and I make a note of the outdoor temperature when I take each set of darks, and reuse the master dark when the temperature is similar to one that I already have.   If you have a cooled astrocamera, you can simply choose a temperature (I hear -10 C is popular) and make a set of darks, then a master dark, for each exposure duration that you think you'll be using regularly.

[dswtan]

Yes as Michael says. And OP’s 294MC has a nice deep well, unlike the older 1600, say. So you have flexibility even with just one exposure combination. But just a terminology note - a “master dark library” is a library, it’s not a master dark itself.

 

A master dark is a stack of individual darks (all at the same temp, gain, etc.). So a library typically contains sets of darks at collections of the same exposures (same temp, gain, etc.) - for example, I maintain libraries of darks at 10s, 30s, 60s,  90s, 120s. Others might store collections at different gains. 
 

Just want to check we’re not conflating a library with a master itself. 
 

PS. I use the 60s unity gain set predominantly. The rest are just there in case I ever need them — and because I’m a hoarder.

[Alex McConahay]

>>>>>I assume that folks are then shooting the same duration & temp for all shots?

 

Yes, Many people use the same exposure values for each session. After all, many of us key "ideal" exposure to the background brightness of the sky-----which is generally the same for any given site on a dark (moonless) night. There may be some variation based on targets.....but most of us shoot the same exposure time night after night. 

 

Of course this means you have temperature regulation. 

 

If you do not have regulated temps on the camera, it is possible to have masters shot at different temps, and use the one appropriate to that night's ambient temp. They do not need to be precise---every five degrees or something helps. 

 

Furthermore, it is possible to "scale" darks and use darks taken at one temp with lights taken at another. 

Alex

[acrh2]

Shooting darks as we speak.
f/4.8
 
240 s for narrowband. 60, 120 s for broadband.
 
Here are raw individual subs of 
240 s with L-eNhance filter, Veil nebula region.
120 s with L-Pro filter, Andromeda galaxy region.
 


EDIT: oh yeah, forgot to say - unity gain.

Edited by acrh2, 25 May 2022 - 10:05 PM.

[Snakey93]

Slightly off topic question...Should I bother to still keep the fits calibration files once I compile into a master?

[acrh2]

Slightly off topic question...Should I bother to still keep the fits calibration files once I compile into a master?

I would keep them in case another stacking program requires them.

[DirtyRod]

As dswtan described, I shoot at -10C in the winter and 0C in the AZ summer and maintain a master dark library and dark flats for each temp at 30, 60, 120, 180, and 300 second exposures at gains of 150, 200, and 300 for both my 294 and 533. 90% of the time I’m shooting 180s at 150 gain though.

 

Twice a year I recreate the library.

Edited by DirtyRod, 26 May 2022 - 01:32 AM.

[bobzeq25]

I've seen multiple posts where folks state that they use a master dark library.  Since Darks have to be at the same temp and same exposure duration (correct?), I assume that folks are then shooting the same duration & temp for all shots? I would guess this would mean finding an exposure time that won't blow out the stars and stick with that for all imaging?

 

Just trying to plan ahead...note for reference I have a 294MC Pro and an F/4.7 scope (with flattener, F/5.9 without)

A library has more than one book.

 

A master dark library can have more than one master dark.  You can have a set of them which covers a range of temperatures, gains and exposures.

 

If you shoot broadband and narrowband you'll surely need at least two.

Edited by bobzeq25, 25 May 2022 - 10:59 PM.

[RoscoeD]

As an aside, two of you mentioned unity gain.  Just to be clear, on my camera (294MC) that value is 117, but 120 is the break point for a significant read noise reduction (~5.8 > ~1.8). What do users of this camera use fro "default" gain?

[OldManSky]

I've seen multiple posts where folks state that they use a master dark library.  Since Darks have to be at the same temp and same exposure duration (correct?), I assume that folks are then shooting the same duration & temp for all shots? I would guess this would mean finding an exposure time that won't blow out the stars and stick with that for all imaging?

 

Just trying to plan ahead...note for reference I have a 294MC Pro and an F/4.7 scope (with flattener, F/5.9 without)

I do darks on cloudy nights (or days sometimes).

I do darks for a variety of exposure lengths.  Right now for my 2600MC-Pro, my dark library contains master darks for 30s, 60s, 120s, 180s, 240s, 300s, 600s exposures -- at gain 0 and gain 100.  All at 0C temperature, since I can run at that temp all year long, winter and summer. Those are the exposure times and gains I use when shooting with that camera, with a variety of equipment.

Edited by OldManSky, 26 May 2022 - 07:34 AM.

[c131frdave]

There is an interesting 3-part video by Adam Block (well known PixInsight expert) that dealt with an image where the flat appeared to "over-correct".  This over-correction seems to be a common issue.  You see lots of posts in this forum about it.  He spent quite some time going over this troubleshooting process looking at the flats, looking at other things- and ultimately he found that the issue was that the darks were too old.  The person who supplied the data for him supplied darks from a dark library that was 6 months old.  When he inspected the dark file itself, he found that the pixel depths (or something) weren't matching what they should mathematically.  His conclusion was that these chips change over time, and therefore the darks should be no more than a few months old.

 

The moral of the story was that you should probably create your dark library quarterly.

Edited by c131frdave, 26 May 2022 - 08:21 AM.

[acrh2]

There is an interesting 3-part video by Adam Block (well known PixInsight expert) that dealt with an image where the flat appeared to "over-correct".  This over-correction seems to be a common issue.  You see lots of posts in this forum about it.  He spent quite some time going over this troubleshooting process looking at the flats, looking at other things- and ultimately he found that the issue was that the darks were too old.  The person who supplied the data for him supplied darks from a dark library that was 6 months old.  When he inspected the dark file itself, he found that the pixel depths (or something) weren't matching what they should mathematically.  His conclusion was that these chips change over time, and therefore the darks should be no more than a few months old.

 

The moral of the story was that you should probably create your dark library quarterly.

That particular video didn't make a whole lot of sense mathematically. The darks should not affect flat correction, only flat darks should. I asked him about it and he gave a vague non-answer. Also, it was clear that the person who supplied the data simply didn't do the calibration frames correctly - he must have had light leak when he took flat darks.

[jdupton]

acrh2,

 

That particular video didn't make a whole lot of sense mathematically. The darks should not affect flat correction, only flat darks should. I asked him about it and he gave a vague non-answer. Also, it was clear that the person who supplied the data simply didn't do the calibration frames correctly - he must have had light leak when he took flat darks.

   No, the Dark frames do make a difference in how a Flat works.

 

   In order for a Master Flat to work properly, the illumination profile of both the Master Flat and the Dark-calibrated Light match. (You are allowed to apply a scaling factor (via multiplication or division) to the Master Flat.) If the illumination profiles don't match, the Flat will either over or under correct the Dark-calibrated Light. What this means is that if the offset (Bias and Dark Current) are not exactly removed from the Light, then the Flat will not work, even if it is absolutely perfect.

 

   You can test this for yourself. Pick a Light, matching Master Dark, and your matching Master Flat to use. Then:

1. Calibrate the Light with the Master Dark and Master Flat.
   You should get a Light with no calibration issues (no over or under correction).
   [EDIT] You can use PixelMath in PixInsight to do the Calibration manually. Use the Equation:
   (Light - Dark) / (Master_Flat / mean(Master_Flat))
   where Master_Flat is the integration of the Flat-Dark calibrated Flats [/EDIT]
    
2. Now add or subtract a Constant of something like 100 DN to 300 DN from the Master Dark.
   [EDIT2] (This created an artificial mismatch between the Light and Dark) [/EDIT2]
   This represents a mismatched Master Dark frame.
   [EDIT] PI PM is:
   Dark + 150/65535 or Dark - 150/65535 [/EDIT]
    
3. Calibrate the Light with the modified / adjusted Master Dark and original Master Flat.
   You should be able to detect the artificial over or under correction by the Master Flat.
   Do this twice, once with the additional DN modified Master Dark and then once with the subtracted DN modified Master Dark. You may find that one over corrects while the other under-corrects.

   Flat Darks can also have a similar effect on correction. Again, the cause of poor correction is due to a mismatch between the illumination profiles between the Dark-calibrated and Light or the Flat-Dark-calibrated Flat. The mismatch can creep in via either the Dark or the Flat-Dark.

 

   I thought I had posted examples of this some time back but cannot find them now.

 

 

John

Edited by jdupton, 26 May 2022 - 01:29 PM.

[acrh2]

acrh2,

 

   No, the Dark frames do make a difference in how a Flat works.

 

   In order for a Master Flat to work properly, the illumination profile of both the Master Flat and the Dark-calibrated Light match. (You are allowed to apply a scaling factor (via multiplication or division) to the Master Flat.) If the illumination profiles don't match, the Flat will either over or under correct the Dark-calibrated Light. What this means is that if the offset (Bias and Dark Current) are not exactly removed from the Light, then the Flat will not work, even if it is absolutely perfect.

 

   You can test this for yourself. Pick a Light, matching Master Dark, and your matching Master Flat to use. Then:

1. Calibrate the Light with the Master Dark and Master Flat.
   You should get a Light with no calibration issues (no over or under correction).
   [EDIT] You can use PixelMath in PixInsight to do the Calibration manually. Use the Equation:
   (Light - Dark) / (Master_Flat / mean(Master_Flat))
   where Master_Flat is the integration of the Flat-Dark calibrated Flats [/EDIT]
    
2. Now add or subtract a Constant of something like 100 DN to 300 DN from the Master Dark.
   [EDIT2] (This created an artificial mismatch between the Light and Dark) [/EDIT2]
   This represents a mismatched Master Dark frame.
   [EDIT] PI PM is:
   Dark + 150/65535 or Dark - 150/65535 [/EDIT]
    
3. Calibrate the Light with the modified / adjusted Master Dark and original Master Flat.
   You should be able to detect the artificial over or under correction by the Master Flat.
   Do this twice, once with the additional DN modified Master Dark and then once with the subtracted DN modified Master Dark. You may find that one over corrects while the other under-corrects.

   Flat Darks can also have a similar effect on correction. Again, the cause of poor correction is due to a mismatch between the illumination profiles between the Dark-calibrated and Light or the Flat-Dark-calibrated Flat. The mismatch can creep in via either the Dark or the Flat-Dark.

 

   I thought I had posted examples of this some time back but cannot find them now.

 

 

John

I think you are right. My mistake - a little brain fart, probably originating from the assumption that other people CMOS sensors might be as good as ASI533.

 

In the video, Adam Block compares two sets of darks and flat darks:

old - darks mean value 0.003, flat darks mean value 0.011

new - darks mean value 0.011, flat darks mean value 0.011

 

In your own vernacular,

image  = (Light - Dark) / (Master_Flat / mean(Master_Flat))
image-C =(Light - Dark + C) / (Master_Flat / mean(Master_Flat))

image-C = image + C / (Master_Flat / mean(Master_Flat))

 

where image-C is the image with the overcorrection problem, C is a positive constant subtracted from the dark to emulate the old dark. In this case C = 0.011 - 0.003 = 0.008.

 

That would indeed result in the flat overcorrection.

But that also means that the change in the dark © is so large as to be comparable to the actual DSO signal.

And to think that it happened in 6 months time.

 

I just compared my own darks from one year ago and now. There's almost no difference, some hot pixels moved around a bit.

 

I'm now thinking that the guy simply used old darks with a different offset value.

[jdupton]

acrh2,

 

But that also means that the change in the dark © is so large as to be comparable to the actual DSO signal.

And to think that it happened in 6 months time.

 

I just compared my own darks from one year ago and now. There's almost no difference, some hot pixels moved around a bit.

 

I'm now thinking that the guy simply used old darks with a different offset value.

   I agree. In the case of the mismatch seen in Adam's video, I expect more was at play than just camera aging. I see less than 4 DN on average between the initial Dark frames I took with my ASI294MC-Pro camera and those taken after two years of use.

 

   Such a large mismatch as seen in the video could have been caused by any number of mismatched factors which you mentioned -- gain, offset, temperature, capture mode, driver changes, etc. The net, though, is I like to remind folks seeing poor Flat correction to first look closely at all of the frame types being used. If there are any mismatches found, those should be treated first before looking into other sundry causes.

 

 

John

Edited by jdupton, 26 May 2022 - 03:09 PM.

[acrh2]

If anyone is curious about what old and new darks look like for ASI533, I got you covered.

These are 3 months after purchase (labeled "old") and 14 months after purchase (labeled "new") 240 s darks, -5 C, offset 20. These are dabayered master darks (30 subs.)

It's a 400% crop on the top-left corner with a stretch that shows a total range of 50 ADU (out of 65536.) The histogram center is about 800 ADU in the linear image.

 

Edited by acrh2, 26 May 2022 - 03:41 PM.

[Aquawind]

I am doing my first library right now and I really suggest doing so. They all look the same and that is not the case when my camera is on the train but, that's another issue. So glad I did this! It's so easy and nice to know you have quality darks before you even choose the target. Darks Library Forever!