Jump to content

  •  

CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.

Photo

DSLR dark frame library

  • Please log in to reply
21 replies to this topic

#1 DuPont

DuPont

    Vostok 1

  • -----
  • topic starter
  • Posts: 197
  • Joined: 19 Mar 2020
  • Loc: Fond du Lac, Wi.

Posted 06 August 2020 - 07:43 AM

    Hi everyone, I've been thinking about how I can build a dark frame library other then spending an hour or two after taking my light's. To be honest, it's a royal pain in the b... sitting around at 3-4 in the morning waiting for the dark's to finish. I'm a morning person & hate sleeping in late, not getting out of bed until 11:00 A.M sucks. I do not have a permanent set up which means I need to break it down in the drive way every time I'm finished. I know dark's are temperature critical & it's the sensor temp. that were concerned about along with ambient, ISO & exposure time.

On those nights when it's not feasible to image (cloudy, full moon,etc) I can sleep instead & get up at 4:00 (I love my morningssmile.gif), set my DSLR/telescope with laptop outside & start taking dark's. Of course, this would be done at various ambient temperatures, ISO & exposures.

The one problem I have with this method is the sensor temp. because at the start it would no be heated up to a stable temperature. So let's say I want 30 dark frames but instead take 40 & discard the first 10. Would that work?

What is the temp. range at which dark's can be used? If the ambient is 60F do I have, let say a +/- 4F range? Is this a workable plan or do I just bit the bullet & stay up until the birds sing?

The cameras/scopes I use are in my signature..............Al.



#2 Michael Covington

Michael Covington

    Author

  • *****
  • Vendors
  • Posts: 6,571
  • Joined: 13 May 2014
  • Loc: Athens, Georgia, USA

Posted 06 August 2020 - 08:02 AM

You have a range of several degrees.  With the Nikons you describe in your signature, dark frames play a much smaller role than with older sensors and you should have quite a bit of leeway.



#3 DuPont

DuPont

    Vostok 1

  • -----
  • topic starter
  • Posts: 197
  • Joined: 19 Mar 2020
  • Loc: Fond du Lac, Wi.

Posted 06 August 2020 - 08:29 AM

Thanks Mike, go to know, that will cut down on the dark library that I'll eventually build.


Edited by DuPont, 06 August 2020 - 08:29 AM.


#4 wsbrown

wsbrown

    Vostok 1

  • *****
  • Posts: 106
  • Joined: 04 Jan 2019
  • Loc: Baxley, GA

Posted 06 August 2020 - 10:09 AM

Lately I take two right before imaging and two after, in my mind this captures any variation in noise from the 5-6 degree drop in temperature from start to finish. Seems to work really well and as stated above Nikons are have very little noise anyway.



#5 Alen K

Alen K

    Surveyor 1

  • *****
  • Posts: 1,542
  • Joined: 25 Nov 2009

Posted 06 August 2020 - 11:12 AM

You have a range of several degrees.  With the Nikons you describe in your signature, dark frames play a much smaller role than with older sensors and you should have quite a bit of leeway.

I believe Mark Shelley demonstrated that darks don't work properly for most situations on at least some Nikon models. (The thread is somewhere here in this forum.) Besides which, AFAIK none report temperature in the EXIF, making matching the temperature of lights and darks an uncertain thing.

 

I use a Pentax camera with a Sony sensor. It does report temperature in the EXIF but I don't do dark frames, I just dither and use kappa-sigma rejection during stacking. That gets rid of hot, cold and stuck pixels. However, if I had amp glow that would not be sufficient. 


Edited by Alen K, 06 August 2020 - 11:20 AM.

  • 17.5Dob likes this

#6 DuPont

DuPont

    Vostok 1

  • -----
  • topic starter
  • Posts: 197
  • Joined: 19 Mar 2020
  • Loc: Fond du Lac, Wi.

Posted 06 August 2020 - 11:17 AM

Lately I take two right before imaging and two after, in my mind this captures any variation in noise from the 5-6 degree drop in temperature from start to finish. Seems to work really well and as stated above Nikons are have very little noise anyway.

Thanks for replying. Little confused, are you saying you only take two dark frames before & after due to low noise in Nikon (Sony) sensors? If so, that would be problem solved for mesmile.gif



#7 DuPont

DuPont

    Vostok 1

  • -----
  • topic starter
  • Posts: 197
  • Joined: 19 Mar 2020
  • Loc: Fond du Lac, Wi.

Posted 06 August 2020 - 11:29 AM

I believe Mark Shelley demonstrated that darks don't work properly for most situations on at least some Nikon models. (The thread is somewhere here in this forum.) Besides which, AFAIK none report temperature in the EXIF, making matching the temperature of lights and darks an uncertain thing. I use a Pentax camera with a Sony sensor. It does report temperature in the EXIF but I don't do dark frames, I just dither and use kappa-sigma rejection during stacking. That gets rid of hot cold and stuck pixels. However, if I had amp glow that would not be sufficient. 

Hi Alan, when I have time I'll process a set of images I already took without dark's & see what the outcome is. Yes, I dither also when using APT & in DSS use kappa-sigma for the lights.

I've never heard of any Nikon reporting temp. but I believe Canon's do.

I'll try to find Mark Shelley's thread......Thanks.


Edited by DuPont, 06 August 2020 - 11:30 AM.


#8 wsbrown

wsbrown

    Vostok 1

  • *****
  • Posts: 106
  • Joined: 04 Jan 2019
  • Loc: Baxley, GA

Posted 06 August 2020 - 12:10 PM

Thanks for replying. Little confused, are you saying you only take two dark frames before & after due to low noise in Nikon (Sony) sensors? If so, that would be problem solved for mesmile.gif

Two before, two after: Four total. Sorry for the confusion



#9 DuPont

DuPont

    Vostok 1

  • -----
  • topic starter
  • Posts: 197
  • Joined: 19 Mar 2020
  • Loc: Fond du Lac, Wi.

Posted 06 August 2020 - 12:21 PM

Two before, two after: Four total. Sorry for the confusion

No problem wsbrown, I'll give it a try next time out....thanks.



#10 Okcman

Okcman

    Explorer 1

  • *****
  • Posts: 72
  • Joined: 01 Dec 2016
  • Loc: Oklahoma City, Oklahoma

Posted 06 August 2020 - 01:37 PM

I too am trying to streamline my process -and effort -Since darks are exposure time, temp, and iso sensitive - have you considered shooting in one iso and build it that way - for example I have a set of darks I shot in the spring at 30 second , iso 800 , and it was in the 60s - can I stack those in dss and then use it as a master dark for when I shoot again n the fall - ?

#11 endlessky

endlessky

    Messenger

  • -----
  • Posts: 456
  • Joined: 24 May 2020
  • Loc: Padova, Italy

Posted 06 August 2020 - 03:18 PM

If I remember correctly, I read somewhere in this Forum that noise doubles every 8°C.

 

Using a Nikon DSLR and having no possible way of

 

a) knowing at what (sensor) temperature the lights were taken, throughout the night

b) knowing at what (sensor) temperature I am taking the darks

c) knowing how fast (sensor) temperature increases while I am taking lights or darks

d) most importantly, being able to have a constant set point for the temperature (active cooling)

 

from the moment I read that, I never bothered taking darks ever again. I had big dreams of building a dark library as well, but considering all of the above, and considering that every 8°C increase in themperature, the noise doubles, I don't know if calibrating with darks is doing more good or more harm.



#12 DuPont

DuPont

    Vostok 1

  • -----
  • topic starter
  • Posts: 197
  • Joined: 19 Mar 2020
  • Loc: Fond du Lac, Wi.

Posted 06 August 2020 - 06:23 PM

I too am trying to streamline my process -and effort -Since darks are exposure time, temp, and iso sensitive - have you considered shooting in one iso and build it that way - for example I have a set of darks I shot in the spring at 30 second , iso 800 , and it was in the 60s - can I stack those in dss and then use it as a master dark for when I shoot again n the fall - ?

As far as ISO goes, ideally I would like to shoot at 200 or 400 but I've been using an LP filter so I also use ISO 800. If I made a library I would start off with ISO 800 then 400.



#13 DuPont

DuPont

    Vostok 1

  • -----
  • topic starter
  • Posts: 197
  • Joined: 19 Mar 2020
  • Loc: Fond du Lac, Wi.

Posted 06 August 2020 - 06:32 PM

If I remember correctly, I read somewhere in this Forum that noise doubles every 8°C.

 

Using a Nikon DSLR and having no possible way of

 

a) knowing at what (sensor) temperature the lights were taken, throughout the night

b) knowing at what (sensor) temperature I am taking the darks

c) knowing how fast (sensor) temperature increases while I am taking lights or darks

d) most importantly, being able to have a constant set point for the temperature (active cooling)

 

from the moment I read that, I never bothered taking darks ever again. I had big dreams of building a dark library as well, but considering all of the above, and considering that every 8°C increase in themperature, the noise doubles, I don't know if calibrating with darks is doing more good or more harm.

I agree with your 4 points but I still do not see any definitive proof saying that dark"s are not needed even with better sensor technology or because your dithering.



#14 17.5Dob

17.5Dob

    Fly Me to the Moon

  • ****-
  • Posts: 6,758
  • Joined: 21 Mar 2013
  • Loc: Colorado,USA

Posted 06 August 2020 - 10:08 PM

Unless your camera exhibits significant amp glow...darks are a waste of time with a dSLR...especially with modern Nikons that already use a very low level of spatial filtering ...

 

Depending on your exposure time, with a modern Nikon, you will need darks once you start shooting 4-5 min ++++ subs..to remove the amp glow....or you can just crop out the tiny margin that shows it...

Unless I really need the full entire frame..it's a lot faster to just skip the darks and crop out the 10% of the left side of the frame...


Edited by 17.5Dob, 06 August 2020 - 10:11 PM.

  • Ettu likes this

#15 Readerp

Readerp

    Ranger 4

  • *****
  • Posts: 317
  • Joined: 22 Jun 2019
  • Loc: So Cal

Posted 06 August 2020 - 11:31 PM

Intruiging. If I skip darks with my D5300 or D810 (I usually shoot 4-5 min subs, and I do dither, do I still use the bias frames in DSS?



#16 endlessky

endlessky

    Messenger

  • -----
  • Posts: 456
  • Joined: 24 May 2020
  • Loc: Padova, Italy

Posted 07 August 2020 - 02:25 AM

I agree with your 4 points but I still do not see any definitive proof saying that dark"s are not needed even with better sensor technology or because your dithering.

Oh, my point wasn't that darks are not needed, even if technology increases or if I dither. I wish I could take darks, if they helped. My problem is that I cannot have any repeatability, with an uncooled, thermally uncontrolled DSLR, in the temperature at which the lights and the darks are taken. And with the noise levels being so temperature sensitive, I don't know if these potentially big changes in temperature between lights and darks are actually going to cause noise to increase in the calibrated light, rather than noise decrease.



#17 sharkmelley

sharkmelley

    Soyuz

  • *****
  • Posts: 3,583
  • Joined: 19 Feb 2013
  • Loc: UK

Posted 07 August 2020 - 03:19 AM

There's a useful article by CN contributor whwang here: How good do your DSLR darks need to be?

 

Note that he was using a Nikon D800 which automatically applies raw data filtering.  My own investigation of how raw data filtering affects the efficacy of darks on a Nikon D5300 is here: Dark subtraction and spatial filtering

 

Dark subtraction is also not very effective on Sony cameras, again because of the raw data filtering (a.k.a. Sony star eater). However it should work well on Canons. 

 

Obviously dithering is very important on all cameras.

 

Mark


Edited by sharkmelley, 07 August 2020 - 03:29 AM.


#18 endlessky

endlessky

    Messenger

  • -----
  • Posts: 456
  • Joined: 24 May 2020
  • Loc: Padova, Italy

Posted 07 August 2020 - 04:09 AM

Very interesting reads, Mark. Thank you!

 

So, it seems that sigma-clipping plus dithering is pretty close to best scenario (darks taken at the correct temperature). I guess that's what I'll be doing, then (since I don't have control over temperature).

 

May I ask another question: what about bias (or super-bias)? Are these useful for DSLR calibration, or not? And if they are, which one of the two?

 

So far, I have been calibrating my images only with flats. I use a Nikon D5300, astromodified. Should I add bias (or super-bias) in my calibration routine? What about dark-flats?

 

Sorry to "hijack" the thread, just trying to figure out what calibration would give me the best result, when using a DSLR.



#19 KLWalsh

KLWalsh

    Viking 1

  • -----
  • Posts: 842
  • Joined: 19 Feb 2014
  • Loc: North Georgia, USA

Posted 07 August 2020 - 07:51 AM

I have two Nikons. I haven’t bother taking darks in a long time.
Flats and bias are sufficient, and both can be done quickly at the start of an observing session.

#20 DuPont

DuPont

    Vostok 1

  • -----
  • topic starter
  • Posts: 197
  • Joined: 19 Mar 2020
  • Loc: Fond du Lac, Wi.

Posted 07 August 2020 - 07:52 AM

There's a useful article by CN contributor whwang here: How good do your DSLR darks need to be?

 

Note that he was using a Nikon D800 which automatically applies raw data filtering.  My own investigation of how raw data filtering affects the efficacy of darks on a Nikon D5300 is here: Dark subtraction and spatial filtering

 

Dark subtraction is also not very effective on Sony cameras, again because of the raw data filtering (a.k.a. Sony star eater). However it should work well on Canons. 

 

Obviously dithering is very important on all cameras.

 

Mark

Mark, thank you for providing this experiment. I'm quite new to deep sky imaging & this gives me an understanding into the use of calibration frames. I'll still continue to take dark's & experiment with & without using them.



#21 the Elf

the Elf

    Mercury-Atlas

  • *****
  • Posts: 2,625
  • Joined: 06 Sep 2017
  • Loc: Germany

Posted 09 August 2020 - 08:45 AM

There is no general answer to this question. If you shoot bright objects with a recent DSLR you may not even see a difference between images with and without darks. Find my test here:

http://www.elf-of-lo.../Art_Darks.html

For my mono modded DSLR I made a dark library taking them at daytime in the fridge, more than a thousand of them. Using Exiftool I sorted them by temperature. More on this here:

https://youtu.be/-6FXbs8fu_Y?t=3312

I took 4 hours of the Cygnus Wall two nights ago under tropical temperatures. Tired I forgot picking darks in the image calibration and the result was unusable. Picking the temperature matched darks made it a nice image. That is because the sky background is low when using a narrow band filter. Now the dark current is visible especially when it is so hot and exposure time is that long (15 minutes). For a bright background and short subs the dark current is small compared to the background, if you take very short subs it may be even as low as the read noise. In that case you won't see a difference.

Anyway, you do not need to take new darks for every session when you have a library and you do not need to take them at night. Just make sure ISO, temperature and exposure time is the same. Some programs can adjust for a different exposure time if you have bias frames.


  • Ettu likes this

#22 DuPont

DuPont

    Vostok 1

  • -----
  • topic starter
  • Posts: 197
  • Joined: 19 Mar 2020
  • Loc: Fond du Lac, Wi.

Posted 09 August 2020 - 10:50 AM

There is no general answer to this question. If you shoot bright objects with a recent DSLR you may not even see a difference between images with and without darks. Find my test here:

http://www.elf-of-lo.../Art_Darks.html

For my mono modded DSLR I made a dark library taking them at daytime in the fridge, more than a thousand of them. Using Exiftool I sorted them by temperature. More on this here:

https://youtu.be/-6FXbs8fu_Y?t=3312

I took 4 hours of the Cygnus Wall two nights ago under tropical temperatures. Tired I forgot picking darks in the image calibration and the result was unusable. Picking the temperature matched darks made it a nice image. That is because the sky background is low when using a narrow band filter. Now the dark current is visible especially when it is so hot and exposure time is that long (15 minutes). For a bright background and short subs the dark current is small compared to the background, if you take very short subs it may be even as low as the read noise. In that case you won't see a difference.

Anyway, you do not need to take new darks for every session when you have a library and you do not need to take them at night. Just make sure ISO, temperature and exposure time is the same. Some programs can adjust for a different exposure time if you have bias frames.

Elf, thank you for providing this information as I find it very useful. Using my D5600 I imaged the N. America Nebula recently & included 30 dark's along with flat's & bias frames. Yesterday I processed it in Star Tools without the dark's & I'm finding the results acceptable. What I need to do is process once again, with & without dark"s, noting the steps I take are the same.




CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.


Recent Topics






Cloudy Nights LLC
Cloudy Nights Sponsor: Astronomics