Jump to content

  •  

CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.

Photo

Understanding “Dark Flats”

  • Please log in to reply
14 replies to this topic

#1 Ken_nneth

Ken_nneth

    Vostok 1

  • -----
  • topic starter
  • Posts: 168
  • Joined: 15 Nov 2014
  • Loc: Western Norway

Posted 24 October 2017 - 08:31 AM

Trying to get a better grip on calibration files and want to understand “Dark Flats” and how they work. I understand that they should be taken with cap on at the same temp, gain and exposure as normal flats. But what about filters, my flats are taken with different exposures depending on filter choice. Do I take “Dark Flats” for all filters”? How do I use my subs to create a master or masters? How do I then use these masters, where do they fit in the calibration workflow? Does anyone have a good tutorial or instructions to help me with this.


  • Mondeclerk likes this

#2 xiando

xiando

    Fly Me to the Moon

  • *****
  • Posts: 6,822
  • Joined: 27 May 2015
  • Loc: Cloudy NEOhio

Posted 24 October 2017 - 08:52 AM

Afaik, both darks and dark-flats are independent of the optical train and mechanically are performed identically, by blocking the camera input. The first is shot at the exposure time of your lights, the second at the exposure time used for your flats.

 

So, unless I'm running on erroneous info, filters have no meaning wrt dark flats. One set should be fine. Shoot your dark flats at the same time as you determine from creating your flats (whatever exposure time you've settled on to reach the 1/3-1/2 peak histogram point) and use a "darkfilter" in your wheel or otherwise optically shutter the camera.

 

Now, how to create masters....

 

If you use Deep Sky Stacker, masters are created as a result of the stacking process. (darks and flat masters as well, IFF dark and flat raws are added, of course)

 

If you use Siril, calibration masters are create in the same manner as light stacking, with the main exception that normalization is avoided when stacking calibration files. That is, one selects the files to be used, then stacks them. tada.

 

Fwiw, neither's calibration masters with with the other, ie, Siril masters don't work with DSS, and DSS masters don't work with Siril

 

I can't advise on any other programs.

 

Edit: I need to read things fully...

 

Using the masters...

 

It depends on the program you use to stack your images. Again, I can't advise on a program like PI, but I can for the two noted above

 

DSS - (presumes the master has already been created from a previous stacking operation)

  1. load your lights
  2. click the darks button - instead of loading all the dark raws, choose the MasterDark_ISOx.tif  (mine "x" is always 0, but I don't know how it is for DSLR users)
  3. click the flats button - as with darks, choose the MasterFlat_ISOx.tif
  4. click the dark flat  button - as before, choose the MasterDarkflat_ISOx.tif

 

Siril - after the masters are created, they are selected during the calibration phase. It would be best to download the documentation and read through it.

 

Note: AFAIK, dark flats (or more properly, flat darks) are used with DSLRs, bias files are used with CCD cameras. Can't say for CMOS

 

 

I'm afraid I can't offer a pointer to a good tutorial. I've picked most of this up in bits and pieces, from a friend who acted as my mentor early on, to posts at CN, to google search results.

 

good luck!


Edited by xiando, 24 October 2017 - 09:07 AM.

  • james7ca, nubycakes and Gindra like this

#3 xiando

xiando

    Fly Me to the Moon

  • *****
  • Posts: 6,822
  • Joined: 27 May 2015
  • Loc: Cloudy NEOhio

Posted 24 October 2017 - 10:04 AM

BTW, I forgot one thing...

 

IF you have not yet created masters in DSS, when you get to the step where you click the dark/flat/dark-flat or bias files, load the associated raws (dark raws, flat raws etc) 

 

things also get a little funkier when you stack sets of files shot at different exposure times. (DSS specific)

 

First, drop each new set of lights (for instance, 10-02-2017 raws, 10-06-2017, etc.) into it's own group tab.

 

then, drop associated masters (or cal raws) into the first associated group.

 

Only unique cal raws or masters are needed

 

For instance, suppose you had 7 nights of data (I do for the bubble)

 

the first and third nights, you shot at 300s.

the second night you shot at 60s

the other nights were shot at 120s.

 

you will need one set of 300s darks (or one 300s dark master) (drop into first instance of 300s lights)

you will need one set of 60s darks (or one 60s dark master) (drop into first instance of 60s lights)

you will need one set of 120s darks (or one 120s dark master) (drop into first instance of 120s lights)

you will need one set of dark-flats (or one DarkFlat master) (drop into main group)

 

Presuming no changes were made to the optical chain between nights, you can use the same flats for all of the above.

if changes were made, you'll need associated flats(or flat masters) for any/all nights in which the optical chain was disassembled, rotated, or otherwise altered in any way.

 

regarding dissassembly/reassembly, If you have an accurate indexing system for assembly, you *may be able to get away with reusing flats, but it's not ideal. Flats are quick enough to produce anyway, so it's not like taking a series of 30 1200s darks, which can be painfully slow...


Edited by xiando, 24 October 2017 - 10:06 AM.

  • Ken_nneth likes this

#4 Ken_nneth

Ken_nneth

    Vostok 1

  • -----
  • topic starter
  • Posts: 168
  • Joined: 15 Nov 2014
  • Loc: Western Norway

Posted 24 October 2017 - 10:15 AM

Thanks xiando, I am trying to learn how to use PI, but I will try with DSS to see how the "dark.flat" work. Thanks a lot of good info. 


  • xiando likes this

#5 xiando

xiando

    Fly Me to the Moon

  • *****
  • Posts: 6,822
  • Joined: 27 May 2015
  • Loc: Cloudy NEOhio

Posted 24 October 2017 - 10:20 AM

Patience Ken. The PI crew should be in a bit later. If not, since you're using PI, I'd encourage you to search the processing forum for PI related topics and you'll find the folk who can help guide you in your journey. Hopefully it gives you something to chew on in the meantime.



#6 Jon Rista

Jon Rista

    ISS

  • *****
  • Posts: 24,987
  • Joined: 10 Jan 2014
  • Loc: Colorado

Posted 24 October 2017 - 10:31 AM

Trying to get a better grip on calibration files and want to understand “Dark Flats” and how they work. I understand that they should be taken with cap on at the same temp, gain and exposure as normal flats. But what about filters, my flats are taken with different exposures depending on filter choice. Do I take “Dark Flats” for all filters”? How do I use my subs to create a master or masters? How do I then use these masters, where do they fit in the calibration workflow? Does anyone have a good tutorial or instructions to help me with this.

Any "dark" frame is one taken without any light. As such, nothing in the optical train matters, including filters. Dark frames are intended to model all the noise characteristics of the camera at a matched temperature, exposure length, gain and offset (bias offset) that matches whatever "light" frame those darks are to be matched with. If you are matching to a flat, then the dark flats must match the characteristics of the flat, just without the photons. If you are matching a light, then the darks must match the characteristics of the light, just without the photons. 

 

As for creating a master dark flat in PixInsight. That is actually very easy. You do not need to do any preliminary calibration of the dark flat frames. You just need to integrate them into a master. To do that, open up ImageIntegration. Reset it to defaults. Change the following:

 

Image Integration Section:

 

Normalization = No normalization

Weights = Don't care (all weights = 1)

Buffer size = size of your original fits files on disk

Stack size = largest you can manage with the amount of system memory you have (2048, 4096, maybe 8192 if you have lots of ram)

 

Pixel Rejection (1) Section:

 

Rejection algorithm = Sigma Clipping

Normalization = No normalization

 

Pixel Rejection (2) Section:

 

Sigma low = 7

Sigma high = 7

 

Then add all your dark flat frames and apply. This will create a master dark flat for you. 


  • Merc, Ken_nneth, xiando and 4 others like this

#7 Ken_nneth

Ken_nneth

    Vostok 1

  • -----
  • topic starter
  • Posts: 168
  • Joined: 15 Nov 2014
  • Loc: Western Norway

Posted 24 October 2017 - 10:40 AM

Ok thanks Jon, think I got it so far. If I use the Batchpreprocessing script where would I place my dark flats? 



#8 bobzeq25

bobzeq25

    ISS

  • *****
  • Posts: 25,909
  • Joined: 27 Oct 2014

Posted 24 October 2017 - 11:10 AM

Curious.  Why do you need dark flats?  I've always just used bias, mindbogglingly easier.

 

Especially why would you need them on a cooled camera.  Amp glow?



#9 Ken_nneth

Ken_nneth

    Vostok 1

  • -----
  • topic starter
  • Posts: 168
  • Joined: 15 Nov 2014
  • Loc: Western Norway

Posted 24 October 2017 - 11:32 AM

Curious.  Why do you need dark flats?  I've always just used bias, mindbogglingly easier.

 

Especially why would you need them on a cooled camera.  Amp glow?

Mostly for the Amp glow, but I have had some things happening when using bias that I can't figure out, I've been reading and it seems a lot of people are using Flat darks instead.



#10 Jon Rista

Jon Rista

    ISS

  • *****
  • Posts: 24,987
  • Joined: 10 Jan 2014
  • Loc: Colorado

Posted 24 October 2017 - 01:20 PM

 

Curious.  Why do you need dark flats?  I've always just used bias, mindbogglingly easier.

 

Especially why would you need them on a cooled camera.  Amp glow?

Mostly for the Amp glow, but I have had some things happening when using bias that I can't figure out, I've been reading and it seems a lot of people are using Flat darks instead.

 

With exposures under 0.2 seconds, the bias signal on this camera can get wonky. Inconsistent gradients mostly. So it is better to either use 0.2 second "biases", or just use dark flats. 


  • Ken_nneth likes this

#11 Ken_nneth

Ken_nneth

    Vostok 1

  • -----
  • topic starter
  • Posts: 168
  • Joined: 15 Nov 2014
  • Loc: Western Norway

Posted 24 October 2017 - 03:51 PM

Can anyone tell me how I use the dark flats if I use BatchPreprocessing, or how I use it otherwise?



#12 Sean13

Sean13

    Viking 1

  • -----
  • Posts: 513
  • Joined: 17 Nov 2012
  • Loc: North Platte, Nebraska

Posted 24 October 2017 - 04:21 PM

Curious.  Why do you need dark flats?  I've always just used bias, mindbogglingly easier.

 

Especially why would you need them on a cooled camera.  Amp glow?

The point of dark flats, or flat darks, whichever terminology you prefer, is to calibrate out your flats. When you have filters however, sometimes you flats get into 10+ second exposures, and that can see hot/cold pixels, amp glow, bias noise. Dark flats calibrate these out, killing 2 birds with one stone. You calibrate you flats, and the dark flats include the bias noise you are trying to get rid of as well.


  • Ken_nneth and xiando like this

#13 xiando

xiando

    Fly Me to the Moon

  • *****
  • Posts: 6,822
  • Joined: 27 May 2015
  • Loc: Cloudy NEOhio

Posted 24 October 2017 - 04:53 PM

 

Curious.  Why do you need dark flats?  I've always just used bias, mindbogglingly easier.

 

Especially why would you need them on a cooled camera.  Amp glow?

The point of dark flats, or flat darks, whichever terminology you prefer, is to calibrate out your flats. When you have filters however, sometimes you flats get into 10+ second exposures, and that can see hot/cold pixels, amp glow, bias noise. Dark flats calibrate these out, killing 2 birds with one stone. You calibrate you flats, and the dark flats include the bias noise you are trying to get rid of as well.

 

Thank you for a concise, and informative explanation. I hadn't considered the issue of filters upon flat exposure time, but now I will when I get a mono camera and start line filtering.


Edited by xiando, 24 October 2017 - 04:53 PM.

  • Ken_nneth likes this

#14 Ken_nneth

Ken_nneth

    Vostok 1

  • -----
  • topic starter
  • Posts: 168
  • Joined: 15 Nov 2014
  • Loc: Western Norway

Posted 24 October 2017 - 11:28 PM

Thanks all, I now have a good understanding of dark flats and how they work. I have read that some people are doing extreme dithering and not using darks at all and seeing less noise, have any of you tried this?



#15 Sean13

Sean13

    Viking 1

  • -----
  • Posts: 513
  • Joined: 17 Nov 2012
  • Loc: North Platte, Nebraska

Posted 25 October 2017 - 12:09 AM

Extreme Dithering is almost a requirement of this camera, it doesn't eliminate darks however. It's mostly for FPN, fixed pattern noise. When you take as many exposures as you normally would with the ASI1600, you can start to develop some fixed patterns of noise if there is little to no dithering. Aggressive, high dithering is the best way to eliminate it. The camera isn't bad for noise, but I still recommend darks just because they are so easy to shoot with set point cooling.


  • Ken_nneth likes this


CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.


Recent Topics






Cloudy Nights LLC
Cloudy Nights Sponsor: Astronomics