Jump to content

  •  

CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.

Photo

Can we talk seriously about flat/dark/bias frames?

  • Please log in to reply
76 replies to this topic

#1 mostlyemptyspace

mostlyemptyspace

    Messenger

  • *****
  • topic starter
  • Posts: 431
  • Joined: 05 Jan 2014

Posted 13 January 2015 - 11:13 PM

I've been doing AP for a year or so now, and one thing that's always confused me is flat/dark/bias frames. I have many questions.

 

First, how often do I need to take them? Do I need to take them every time I image, or do I just need one master flat/dark/bias for my camera/telescope setup?

 

I've read that your flat/dark/bias frames should match the same ISO as your light frames. What else do I need to consider? What if I take lights at different ISO and exposures to get better dynamic range?

 

I've tried in the past to use flat/dark/bias frames and honestly, I don't notice a difference. Are they really that important? Can I forget about them, or maybe I just did something wrong?

 

Dark and bias frames seem easy enough to take. For bias, you just set the ISO, leave the lens cap on, and take a bunch of quick shots. For dark frames, you set the ISO and exposure, leave the lens cap on, and take some long exposure shots. Flats though, this is where it gets weird. So what is the easiest way to take good flats? I've tried using a white T-shirt on top of my telescope and then shining a flashlight or an iPad into it.

 

Flat/dark/bias frames are really a pain and take a lot of time, valuable time I could be taking light frames, so I've been avoiding them. Is this something that's really an important element to AP?


  • Neil Lefevre likes this

#2 ManicSponge

ManicSponge

    Viking 1

  • *****
  • Posts: 984
  • Joined: 29 Nov 2013
  • Loc: Full time RV wanderer

Posted 14 January 2015 - 12:06 AM

I have been trying aggressive dithering in PHD, rather than taking darks. With no darks, I don't take any bias shots, either. I do take flats with BYEOS, by stretching two layers of a clean Tshirt over the objective, and holding my laptop screen up with a Notepad blank page, which gives a nice even background. I never really saw much benefit from taking darks while dithering. Maybe I just didn't know exactly what to look for. Flats, on the other hand, make a radical difference, from my light polluted back yard.

 Maybe somebody will drop in with a more technically astute explanation, but that's the Joe Sixpack version, that works for me, at the moment. 

 Regards, Kyle


  • Jim Waters likes this

#3 Jon Rista

Jon Rista

    ISS

  • *****
  • Posts: 22896
  • Joined: 10 Jan 2014
  • Loc: Colorado

Posted 14 January 2015 - 06:03 AM

First, I would say that bias and flat frames are fairly critical. Dark frames may be critical they may not be, there are a few factors that play into that. All of these frames are calibration frames, as they allow us to "calibrate" each and every light frame. 

 

Starting with definitions.

 

A bias frame is a frame that represents the bias signal of your camera. As far as I know, all cameras have a bias signal, which is a generally fixed pattern that exists due to the physical and electronic traits of the sensor's design. In general, there is some base voltage applied to the sensor in order for it to operate correctly (you need a voltage applied to each pixel in order to give each pixel the potential to release electrons in response to the energy of incident photons). In DSLR cameras, a bias signal (when stretched enough) is often dominated by some kind of banding structure. In CCD cameras, a bias frame could be very clean, or it may contain some kind of pattern...it depends on the quality of the camera.

 

A dark frame is a frame that represents an exposure done in total darkness. This signal includes the bias signal, but also includes any dark current charge accumulation, and thus any dark current noise that exists within the dark current signal. In both DSLRs and CCD cameras, accumulated dark current tends to reveal another form of fixed pattern noise...hot and cold pixels. Dark current is additional signal, on top of the bias signal and on top of any image signal you would have in a light frame. That additional signal accumulates faster as temperatures rise, revealing more and more hot pixels. 

 

A flat frame is a frame that represents the field flatness. Unlike bias and dark frames, flat frames are not about noise, they are about identifying the "shape" and structure of your field. By field, I mean the area within the frame. A field may be non-flat due to a couple things. First, there is usually some amount of vignetting unless you are using a very small central portion of a very large image circle. In many cases, the image circle of a scope will barely be large enough for sensors of approximately APS-C sized (i.e. a 7D II, or a KAF-8300). In some cases, the image circle of a scope may be huge...as in the case of the Tak FSQ106, which has an 88mm image circle, and is unlikely to produce vignetting on either of those cameras. Flat frames also represent the dust motes on your sensor, which affect the cleanliness of your field.

 

What's Absolutely Necessary

 

Are all of these frames truly necessary? Personally, I believe that, at the very least, you should be correcting your field flatness in every image. Subtracting darks may or may not be beneficial, there can be a number of factors that play into whether they are or not. If you have low dark current (i.e. with a thermally regulated CCD, or DSLR in cold winter months), you may not have much dark signal to worry about in the first place. If you dither and integrate with some kind of sigma clipping algorithm, you might not need dark frames even if you have higher dark signal. So, darks are not necessarily critical...but more on this in a bit. 

 

Back to flat fielding. Vignetting can greatly darken the corners of your image, and as you integrate more and more light frames, that darkening becomes more pronounced (i.e. in the center of the frame, you may be integrating fairly strong signal of say 2000ADU, where as in the corners you may only be integrating very weak signal that might barely be above the read noise floor at 50ADU....add ten frames, your center signal becomes 20,000 ADU, but your corners remain at a mere 500 ADU.) Flattening the field of each individual light frame can greatly mitigate that exaggeration of vignetting when you integrate by brightening the vignetted parts of the frame.

 

So, if only flat fielding is necessary, why did I say biases are also critical? To properly calibrate lights with flats, both the lights and the flats must first have the bias signal removed. This is important because flat frames must usually be scaled first in order to be effectively applied to each light (this is usually called normalization.) Scaling the flats will change the bias signal in the flats...so before that scaling is performed, the bias signal must first be removed. If the bias signal is removed from any calibration frame, it must be removed from every single frame involved in an integration. That includes lights and darks.

 

Flat frames are usually divided out of your light frames, rather than subtracted, which is how vignetting and dust motes are corrected (if you think about the math, it makes sense. If you consider the pixel values in your flat being from 0 to 1.0, and the corners of your flat are 0.025 while the center of your flat is 0.8, dividing the flat out of each light balances out the field: 2000ADU/0.8 = 2500, 50/0.025 = 2000.)

 

So, to properly calibrate light frames with flat frames, you need bias frames. There is an alternative. Instead of using bias frames, you could calibrate each flat with flat dark frames, which subtract dark frames matched to the temperature and exposure duration of the flats. Personally, I find using flat darks to be a lot more work, especially with DSLRs where you may need new flat darks every time you create new flats. Bias signal doesn't change much (it can, but usually very slowly over very long periods of time, months or years), so one master bias is often sufficient to calibrate everything.

 

More on Dark Frames

 

Getting back to dark frames. Do you need them? You may, it depends. For the most part, dark current reveals hot and cold pixels. Some pixels are "hotter" or "colder" than the average due to differences in the response of the silicon in each pixel. Some pixels will accumulate additional charge from both dark current more readily than others. Some pixels will accumulate charge more slowly. Darks, when subtracted from lights, will subtract the additional charge that accumulates in hot pixels. 

 

There is a consequence to subtracting darks like this. Unless you take an exorbitant amount of individual dark frames to integrate into a pristine master dark (and were talking a few hundred frames), dark frames will also contain some random noise as well. Subtracting a master dark from a light will correct the hot pixels, but it will also make cold pixels colder, and it will increase the standard deviation of the signal, increasing the random noise in the light frame. In practice, the increase in random noise may not be enough to be of consequence, particularly not when using thermally regulated CCDs or DSLRs in cold winter temperatures. During hotter weather, the increase in random noise from dark subtraction with DSLRs could become a problem.

 

There is another way to eliminate hot and cold pixels (as well as any statistically unwanted outlier signal, such as cosmic ray hits.) Dithering. With dithering, which is a process of offsetting the stars a little bit in each light frame, special algorithms that will "naturally" reject hot or cold pixels can be used when integrating your calibrated lights. More on dithering a little later.

 

Sometimes sensors accumulate other unwanted signal as well. Glows, for example. Amplifier glow, which is caused by increased dark current due to the uneven heating of the sensor die by other electronics, say a poorly shielded DSP on a motherboard located too near the sensor. This additional signal, often presenting as a hemispheric bubble of increased dark signal, or a brightened sensor edge, can only be removed by subtracting it out with dark frames. There can be other kinds of glow as well, such as the 5-point glow issue with QSI's Sony-based CCD cameras. Dithering will not be sufficient to affect glow signals, so darks are your only option. If you have an issue with some kind of glow, then you should calibrate your light frames with darks as well as flats. Since the flats must be calibrated with biases every individual frame, each light, each dark, each flat, should first be calibrated with a master bias.

 

What are master frames?

 

I've mentioned master frames a few times. When you calibrate each light frame, a "master" bias, "master" flat, and possibly a "master" dark should be used, rather than one single bias, flat, or dark frame. What is a mater calibration frame, and why are they necessary?

 

Every frame you create with your camera will have some amount of random noise in it. Primarily read noise, this random noise can actually be a bit of a problem for your lights. Subtracting a single bias frame, for example, will remove the bias signal, but the random noise in the bias will subtract from the random noise in the light. Random - random = even more random, as tends to be the case. That may be inaccurate...what it really means is subtracting random noise from random noise results in an increase in the standard deviation of pixel values, which means more noise, not really more random. 

 

A master calibration frame is the integration of many individual calibration frames. A master bias, therefor, is an integration of many bias frames. A master dark is an integration of many dark frames. A master flat is the integration of many flat frames. How many frames should be integrated for each master? There are different schools of thought on this. In general, a couple dozen flat frames and dark frames are enough to produce a good master. Personally, I use 30 flat frames for my master flats, and when I was using darks I used 30 dark frames. Some imagers use only 10, others use more. The general idea is to reduce the random noise in your master calibration frames. Noise is reduced as the square root of the number of frames integrated. Integrating 10x results in a reduction in noise by a little over a factor of 3, integrating 30x reduces noise by nearly a factor of 5.5, integrating 50x reduces noise by a factor of 7.

 

How many bias frames to integrate into a master bias is sometimes a more controversial subject. Again, some imagers integrate a mere 10 frames, others maybe 30-50. Some imagers will integrate 200-300 frames. Why so many? Subtract a semi-noisy bias from a dark, that increases the random noise in the dark. Subtract both a bias and the dark from a light, and you increase the random noise of the light even more. There is something to be said for having a truly pristine master bias, one largely devoid of random noise. Tests have shown that integrating ~180 bias frames results in a very clean master bias, and integrating ~360 reaults in a totally clean master bias. If you have the patience, creating a very clean master bias can mitigate any increase in random noise when calibrating your lights with a master dark. It should be noted that there are ways of creating a "superbias" frame, which takes a minimal number of bias frames and agorithmically creates a master bias frame that is as clean as if you integrated thousands of bias frames together. This is the most effective way of creating a super clean bias, without having to expend shutter actuations on your camera or burn cpu cycles creating a massive several-hundred frame master bias. 

 

Calibrating and Integrating

 

When it comes to actually calibrating your images, there are a few things to know. First, for the most part, there are existing tools that will do most of the nitty gritty work of calibrating your light frames for you. DSS, as well as PixInsight's BPP (Batch PreProcessing script) will handle both the creation of master calibration frames, as well as the actual calibration of your lights, for you. All you really have to do is generate the individual calibration frames and load them into DSS or BPP properly. 

 

For integrating your master calibration frames, it's best to use some kind of median replacement sigma clipping algorithm. A sigma clipping algorithm is one which will identify and reject or replace "statistical outliers", or pixels that deviate too far from the expected mean. When generating a master dark, for example, the Median Sigma-Kappa integration in DSS, or Winsorized Sigma Clipping in BPP, will average out random noise, strengthen hot and cold pixels and any other fixed patten, and reject any other statistical outliers...such as a hot pixel or group of pixels that only showed up in one frame (i.e. a cosmic ray hit). This generates a master dark with low random read noise, and a very strong signal for the fixed pattern and/or glow signal that the master dark is intended to remove from each light frame. 

 

The same integration algorithms should be used to create your master bias and master flat frames as well. 

 

Once you have your master calibration frames, you will then need to calibrate your other frames. Generation of a master bias should be first (and usually will be, automatically, with DSS and BPP). Once the master bias is created, it will be used to calibrate each individual dark frame via subtraction. The same goes for each individual flat frame. Once all the dark frames are calibrated, a master dark will be created (if your using darks). Once all the flat frames are calibrated, a master flat frame will be created. You now have all of your master calibration frames.

 

One you have your master calibration frames, you can calibrate your lights. First subtract the bias, then subtract the dark. When subtracting the dark, a procedure called "dark scaling" may be applied. When using dark scaling, the use of biases is a must, since the master dark will be adjusted in brightness to better match the level of identified hot pixels in each light frame. When using DSS or BPP, this scaling will usually be done for you, however you may have the option to override it and manually choose a scaling level.

 

After subtracting out the master bias and dark frames, divide out the master flat. When using DSS or BPP, the master flat will usually first be normalized to each light frame. This simply means the brightness level of the flat is adjusted to more closely match the brightness levels in the light frame. After dividing out the master flat from each light, you should have a set of fully calibrated, flat fielded light frames devoid of vignetting or dust motes.

 

Your light frames are now ready to be integrated. By removing bias signal, hot pixels, fixed pattern noise, and vignetting and dust motes, you preemptively head of the creation of more severe issues in your final integrated image. Integration (the averaging together of the pixel data from each light frame) is a process that reduces noise, but reinforces structure. Fixed patterns, such as the banding of the bias signal, the hot pixels and glows of dark current signal, and the vignetting and dust motes of your field, will all become reinforced and more pronounced during the integration process.

 

If you do NOT calibrate your light frames first, these issues, which may only be mild in each individual light frame, could and likely will end up being much, much more significant in your final integration. A dust mote, for example, that barely shows up in a single light frame might become a pitch black "dark hole" in your final integration. Faint banding in each light frame might become very strong, pronounced banding in the final integration. Calibration of each light frame BEFORE integration minimizes or eliminates these artifacts, these unwanted structures and patterns, before the integration of your light frames...so they never become reinforced.

 


  • Joe Bergeron, NeilG, BobBates and 45 others like this

#4 tmoncmm

tmoncmm

    Vostok 1

  • -----
  • Posts: 120
  • Joined: 05 Nov 2014
  • Loc: Oxford, GA

Posted 14 January 2015 - 06:37 AM

I use calibration frames for stacking, but have wondered about taking proper flats. What is the best method for taking flat frames and how does one properly expose the flat frames?


  • starhunter50 and chdr like this

#5 shawnhar

shawnhar

    Cosmos

  • *****
  • Posts: 9393
  • Joined: 25 Jun 2010
  • Loc: Knoxville, TN

Posted 14 January 2015 - 08:01 AM

AV, middle of the histo for flats is the most common starting place, in BYEOS I just change the type to flats and adjust the exposure time down and take samples til I hit the middle of the histo, then take 27.

 I built a light box out of foamcore and drafting paper for diffusers but I have had good luck with the t-shirt and dawn/dusk, even broad daylight.

http://shawnhar.blog...-light-box.html

 

 I use dithering and flats, that's it, no other calibration.



#6 jhayes_tucson

jhayes_tucson

    Fly Me to the Moon

  • *****
  • Posts: 6773
  • Joined: 26 Aug 2012
  • Loc: Bend, OR

Posted 14 January 2015 - 11:12 AM

A++ Jon!   You explained it perfectly.  Nicely done.

 

John


  • bsavoie and Jon Rista like this

#7 mostlyemptyspace

mostlyemptyspace

    Messenger

  • *****
  • topic starter
  • Posts: 431
  • Joined: 05 Jan 2014

Posted 14 January 2015 - 11:30 AM

Thank you Jon for the thoughtful explanation. I now fully understand how dark/flat/bias frames work and why they are important.

 

Now for a bit of practicum. When do I need to take them? You said the bias noise doesn't change very often, so perhaps I can just make one master bias once and call it a day. Do I have to take dark and flat frames every time I image, or can I do it once and reuse the masters each time I image?

 

One thing you mentioned about flats was that flat frames take away dust. That tells me that the camera needs to be aligned exactly the same as the light frames. If the camera is rotated on the T-threads even a bit, those dust particles will be offset. Does that mean I have to take flats every time I image?

 

For darks, it sounds like it's more about the internal behavior of the camera, which would depend on the ISO and exposure length I assume. So if I create a number of master dark frames at varying ISO and exposure length, would that work, or do I again have to make them every time I image?



#8 Tonk

Tonk

    Cosmos

  • *****
  • Posts: 8829
  • Joined: 19 Aug 2004
  • Loc: Leeds, UK, 54N

Posted 14 January 2015 - 11:49 AM

Flat frames at least should be taken on each imaging session - if you rotate the camera with respect to the OTA you will need a set for each orientation you have used.

Flats are sensitive to - a) the orientation of the camera to the main optics - you are measuring and correcting shadowing in you system that is causing darkening towards the edge of the field (called vignetting) - this changes the position of the shadowing on the sensor as you rotate the camera. Don't assume vignetting is perfectly circular and centred.

b) it is sensitive to the positions of dust on the sensor and optical surfaces. These also cast shadows (motes or dust bunnies) - and again flats correct for this.

Dust moves around over time so as well as changes of camera/scope orientation, this demands the flats are frequently taken (recommend at least 25 in each set on each session/camera reorientation)

Other points - if you have filters in your optics train - leave them in for taking the flats. DONT change focus before taking the flats. DONT change camera orientation before taking the flats

Hope that is of help.

Edited by Tonk, 15 January 2015 - 11:21 AM.


#9 terry59

terry59

    Cosmos

  • *****
  • Posts: 8763
  • Joined: 18 Jul 2011
  • Loc: Colorado, USA

Posted 14 January 2015 - 11:54 AM

Flat frames at least should be taken on each imaging session
 

 

If nothing has changed this isn't necessary, at least in my experience.

 

YMMV



#10 Tonk

Tonk

    Cosmos

  • *****
  • Posts: 8829
  • Joined: 19 Aug 2004
  • Loc: Leeds, UK, 54N

Posted 14 January 2015 - 11:57 AM

For darks, it sounds like it's more about the internal behavior of the camera, which would depend on the ISO and exposure length I assume.


And very critically it depends on temperature - so try and match the temperature too. The camera records temperature in the image EXIF data so you can use software such a DarkLibrary to match stock darks you might have to the lights you wish to calibrate - if not you can at least check what the temp different might be by comparing the temp (can someone remind us what the software is for reading EXIF data?)

#11 Tonk

Tonk

    Cosmos

  • *****
  • Posts: 8829
  • Joined: 19 Aug 2004
  • Loc: Leeds, UK, 54N

Posted 14 January 2015 - 12:02 PM

If nothing has changed this isn't necessary, at least in my experience.


I assume you mean everything has been left on the scope and the scope is still on the mount and hasn't been carried around or knocked?

If you are certain that no new dust has settled or fallen off or moved - then yes you can get a away with it. Otherwsie if you set up each session from scratch its best to assume dust shadows will have moved.

With an appropriate method using a bright light source 25 flats can be taken in < 1 minute - so I see flats as being a non-problem with no value in trying to skip

#12 terry59

terry59

    Cosmos

  • *****
  • Posts: 8763
  • Joined: 18 Jul 2011
  • Loc: Colorado, USA

Posted 14 January 2015 - 12:22 PM

 

If nothing has changed this isn't necessary, at least in my experience.


I assume you mean everything has been left on the scope and the scope is still on the mount and hasn't been carried around or knocked?

If you are certain that no new dust has settled or fallen off or moved - then yes you can get a away with it. Otherwsie if you set up each session from scratch its best to assume dust shadows will have moved.

With an appropriate method using a bright light source 25 flats can be taken in < 1 minute - so I see flats as being a non-problem with no value in trying to skip

 


My mount remains set up but everything else gets taken down at the end of an imaging session. It is extra work to set back up at dawn if I finished imaging at 2 a.m. My setup includes a SX filter wheel. The filter position appears to be identical each time it is used (as advertised). I've tested this on numerous images now with no negative effects that I can see.

 

This is my perspective based on my experience and my equipment. It probably isn't suitable for everyone ;)



#13 tazer

tazer

    Surveyor 1

  • -----
  • Posts: 1882
  • Joined: 22 Dec 2011
  • Loc: The Pitts (NC)

Posted 14 January 2015 - 01:12 PM

I've got some default flats which I use if I'm too lazy to make some out in the field. These can be really handy and take care of a majority of vignetting. It does nothing for dust motes, of course. Now should the "generic" flat have a dust mote in it, then you're actually adding a dust mote to your image. So, be sure any generic flats you have are totally free of dust/debris.

Mark



#14 Jon Rista

Jon Rista

    ISS

  • *****
  • Posts: 22896
  • Joined: 10 Jan 2014
  • Loc: Colorado

Posted 14 January 2015 - 01:25 PM

Thank you Jon for the thoughtful explanation. I now fully understand how dark/flat/bias frames work and why they are important.
 
Now for a bit of practicum. When do I need to take them? You said the bias noise doesn't change very often, so perhaps I can just make one master bias once and call it a day. Do I have to take dark and flat frames every time I image, or can I do it once and reuse the masters each time I image?
 
One thing you mentioned about flats was that flat frames take away dust. That tells me that the camera needs to be aligned exactly the same as the light frames. If the camera is rotated on the T-threads even a bit, those dust particles will be offset. Does that mean I have to take flats every time I image?
 
For darks, it sounds like it's more about the internal behavior of the camera, which would depend on the ISO and exposure length I assume. So if I create a number of master dark frames at varying ISO and exposure length, would that work, or do I again have to make them every time I image?


I agree with the things Terry and Tonk have said. Some things need to be done more frequently...namely, flats and possibly darks. If nothing has changed, then you shouldn't need to re-take flats, and if your setup is very static (i.e. you have it in an observatory so don't take down/set up regularly), you might be able to reuse a set of flats for a while. Personally, my setup is very mobile, and the timing for when I do my imaging to when I take my flats is often offset by a car drive (I don't have any kind of light box, and I usually end my sessions before the sky brightens). That usually results in dust motes moving around, so I try to avoid dust motes entirely. I keep a rocket blower handy, and if I see any motes early on in my imaging session, I'll discard the first few frames, clean off my sensor, and just try to keep things clean. Doing that, I've been able to reuse one set of flats for each of the ISO settings I use, for both of my cameras, for a month or two now. It's more work to keep cleaning off the dust, and I DO still need the flats to correct vignetting...but I don't have to take flats every single session. That's an option.

The other is to simply take flats every session...it isn't difficult, and it's usually pretty quick. It only takes me a few minutes to generate a stack of 30x flats. One thing about flats. Flats should be taken at the ISO setting of the lights. Flats should also be taken with some kind of white covering to flatten out the real-world field and neutralize the tone as much as possible. If taking flats against a morning sky, a thin white t-shirt is necessary to diffuse the sky tones...otherwise you end up with gradients in your flats. Since a t-shirt is going to sit right over the front of your scope (or lens), it's probably going to be closer (likely much closer) than minimum focus distance. That means there isn't going to be any detail, it's literally JUST the field...vignetting, dust motes, any other field quirks. It doesn't matter what your camera orientation is...the sensor is the sensor. If you are using a DSLR, which might include orientation details, you might want to get the general orientation of the camera similar to the orientation you had it on your scope, but in general I don't believe any astro stacking tools use any of the orientation metadata, they simply read bottom up (usually) or top down, so you could probably take flats with the camera upside down...it wouldn't really matter.

With darks, you can create a reusable library if you have predictable temperatures. You may also be able to rely on a library if you are using PixInsight for integration, and have it's dark scaling enabled. If you are using a thermally regulated CCD, then you could probably get away with one master dark for winter, one for spring/fall, and one for summer. You might want to re-take them at the beginning of each season, since some things may change (same goes for biases), but I know that some people have reused their master darks and master biases for years. If you are using dark scaling in PI, the recommendation is to take a set of 30-minute darks (and maybe just 10x) in the general temperature range of your lights. PI's integration tool (or BPP script) should be able to scale those darks to your lights, and removal of dark signal should be fairly effective. It is NOT recommended to use darks of shorter exposure than your lights when using dark scaling, as that could result in an increase in random noise. Darks longer than your light exposures will always be scaled down, which should actually reduce random noise, which is actually more ideal than darks perfectly matched to the exposure times of your lights. I haven't done this yet myself, however I plan to try it soon, as my 5D III has some amp glow to the right side edge that I'd like to get rid of. You might need a more extensive dark library if you use dark scaling, as you do need to be somewhat matched to your sensor temps. I know that my sensor temps can swing as much as 8-10 degrees in an imaging session, and those ranges can jump by 10 degrees from day to day depending on whether the weather is freezing, subfreezing, or above freezing. Come spring/summer, it'll be a whole new range of temperatures, and a whole new set of darks.
  • newman likes this

#15 mostlyemptyspace

mostlyemptyspace

    Messenger

  • *****
  • topic starter
  • Posts: 431
  • Joined: 05 Jan 2014

Posted 14 January 2015 - 01:53 PM

So for dark frames, do I need to take them through the full optical train, or can I just take them with the camera itself and lens cap on? And do ISO and aperture settings matter, or just exposure time? So could I go outside at night, take my camera with the lens cap on, and set it to take 30x1min exposures, while I'm say doing visual observing with my telescope?

 

And this is a slight tangent, but it seems like the biggest problem with reusing flats is dust. How do you guys keep your optics clean and free of dust? Would just some alcohol and cotton balls/swabs work?



#16 Jon Rista

Jon Rista

    ISS

  • *****
  • Posts: 22896
  • Joined: 10 Jan 2014
  • Loc: Colorado

Posted 14 January 2015 - 02:22 PM

Darks can just be taken with the camers itself. Lens cap on, preferably in a very dark room, or covered by something to block out any potential stray light. Personally, at least during winter, I actually stick my 5D III or 7D in the fridge or freezer, after normalizing the temperature closer to the real ambient I had when imaging, and let it rip. ;P It's the only way I can get similar temperatures, and when the refrig/freezer door is closed, the lights go off and it's nice and dark in there. I would let the camera normalize in temperature for about 30 minutes, then do a warmup cycle to get the sensor temperature normalized (take a few dark frames that you'll just throw away)...then take your actual darks.

 

Dust on the optics is usually not a big problem. I sometimes see very, very big, extremely faint motes from particulate on the front of my lens, but that stuff is so mild I don't worry about it. Dust on a filter close to the sensor (i.e. a clip-in from Astronomik or IDAS) might be a problem, so I try to keep my filters clean and devoid of dust. As for the sensor, I just use an air blower, like a rocket blower. Best not to touch the sensor if you can avoid it. Goop on the sensor may require more aggressive means to clean up, but I would find and use a proper sensor cleaning kit, to be on the safe side. You have to be extremely careful that you do not scratch the filters or cover glass over the sensor, as those scratches could then become permanent dust motes.



#17 newman

newman

    Vostok 1

  • -----
  • Posts: 148
  • Joined: 06 Dec 2010
  • Loc: Virginia

Posted 15 January 2015 - 10:25 AM

Wow this is the best thread I have seen on this topic.  Sadly I have another question regarding dark flats.  If I have my camera set to take a sequence of LRGB, and I take a dark after each of the lights in the sequence, how is the flat dark different than a dark?  I am under the impression that I should take my light flats at the same exposure, temperature, focus; through each filter, so what am I missing?  I assume something basic, but it seems to me that if my flats are directly related to my lights and my darks are too than I am not sure of the value of dark flats.  Does this make sense or am I that far off in wonderland?



#18 Tonk

Tonk

    Cosmos

  • *****
  • Posts: 8829
  • Joined: 19 Aug 2004
  • Loc: Leeds, UK, 54N

Posted 15 January 2015 - 11:30 AM

Since a t-shirt is going to sit right over the front of your scope (or lens), it's probably going to be closer (likely much closer) than minimum focus distance. That means there isn't going to be any detail, it's literally JUST the field...vignetting, dust motes, any other field quirks. It doesn't matter what your camera orientation is...the sensor is the sensor.


Re flats this is not right - the shadowing (vignetting) is due to the scope optics too and if you have rear optical elements on a APO or a Wynne corrector etc on a newt, the dust shadows from these are fixed to the scope not sensor.

So lets not confuse this by suddenly countering the advice that camera orientation wrt the scope should not be changed --- please.

#19 Tonk

Tonk

    Cosmos

  • *****
  • Posts: 8829
  • Joined: 19 Aug 2004
  • Loc: Leeds, UK, 54N

Posted 15 January 2015 - 11:33 AM

how is the flat dark different than a dark?


A dark must match the exposure length of each light frame - a flat dark matches the exposure length of each flat - simples :)
 

I am not sure of the value of dark flats


For calibrating out both hot/cold pixels and bias patterns in your master flat. If you don't do this you end up injecting (vignetted!) bias patterns back into the calibrated lights

Edited by Tonk, 15 January 2015 - 11:37 AM.

  • newman likes this

#20 Magellan

Magellan

    Apollo

  • ****-
  • Posts: 1430
  • Joined: 26 Jan 2006
  • Loc: Enfield, NS Canada

Posted 15 January 2015 - 11:58 AM

Ever since I learned that "Clean sensor now" trick I no longer need Darks.  I do the clean sensor now before a session and it takes a mini dark to map hot and cold pixels.  Its called Dark Current Supression I belive.

 

I have been having issue with my Flats.  I have tried multiple methods:

 

1.) Two white t-shirts elasticed tight around the front and using the sky and telling Sequence Generator Pro to map out my flats to 27000ADU.

2.) One White t-shirt at night using an LED headlamp 5' away.

3.) Using http://www.youtube.c...h?v=WKB0JUkksJg (10 hrs of White Screen) on my Mac for lenses.

 

Number 1 works best! but it doesn't seem to subtract all of the dust from the frames have a look:

 

Here is a Master Flat of 20 images set to 27000ADU and Averaged.  SGP reverses and mirrors everything for some reason, the Lights were from BYE because SGP screws too much with the color.  I used PixInsight to reverse and mirror to match the lights.  The Flat is overstretched to show the imperfections:

 

16261245616_0b3687cfe9.jpgMasterFlat_27KADU by AstroJeff, on Flickr

 

Here is my final light of Comet Lovejoy from the 13th of January, it overstretched to show the imperfections:

 

15667304603_85dbe3103f.jpgFlat didn&#x27;t work 100% by AstroJeff, on Flickr

 

I use SGP to take the flats because it maps out based on current conditions a set ADU.  BYE works in AV Flat mode but because the histogram is displayed in JPG I find its not as accurate and differs from in camera histogram.

 

Why aren't my flats being properly subtracted?

 

Thanks,



#21 Tonk

Tonk

    Cosmos

  • *****
  • Posts: 8829
  • Joined: 19 Aug 2004
  • Loc: Leeds, UK, 54N

Posted 15 January 2015 - 01:59 PM

Why aren't my flats being properly subtracted?


Hopefully you are not subtracting them - if you are then that's the problem - flats need to be divided into the lights after adding a pedestal value to normalise the flat to the light.

What's that black border of the left and the thinner one at the top in your final image????

Are you applying the flat after stacking? Or are you applying the master flat to each light frame in turn before alignment and stacking? The latter is the correct processing.

#22 Magellan

Magellan

    Apollo

  • ****-
  • Posts: 1430
  • Joined: 26 Jan 2006
  • Loc: Enfield, NS Canada

Posted 15 January 2015 - 02:51 PM

Artifact of stacking on the comet I suspect.

 

I do whatever DSS does in whatever order DSS does it :)

 

Subtract may have not been the right word.  I was thinking of the flats removing the dust issues and such.  Why hasn't it removed the dust on the top left or properly fix all of the vignetting? 



#23 Jon Rista

Jon Rista

    ISS

  • *****
  • Posts: 22896
  • Joined: 10 Jan 2014
  • Loc: Colorado

Posted 15 January 2015 - 03:02 PM

 

Since a t-shirt is going to sit right over the front of your scope (or lens), it's probably going to be closer (likely much closer) than minimum focus distance. That means there isn't going to be any detail, it's literally JUST the field...vignetting, dust motes, any other field quirks. It doesn't matter what your camera orientation is...the sensor is the sensor.


Re flats this is not right - the shadowing (vignetting) is due to the scope optics too and if you have rear optical elements on a APO or a Wynne corrector etc on a newt, the dust shadows from these are fixed to the scope not sensor.

So lets not confuse this by suddenly countering the advice that camera orientation wrt the scope should not be changed --- please.

 

 

You are indeed correct, however I think you may have misunderstood. I did not mean that the camera orientation should change relative to the scope. Simply that you wouldn't necessarily need to leave the camera and scope on your mount (and thus, oriented the same to the sky) in order to take flats. So long as the whole assembly isn't changed, you could take the scope off the mount, set it down somewhere pointed at a flat, diffuse light source...and if the orientation of the scope and camera together changed, then that wouldn't matter. I whole-heartedly agree, though...don't take the camera off the scope or change it's orientation relative to the scope...that would change vignetting. 



#24 ManicSponge

ManicSponge

    Viking 1

  • *****
  • Posts: 984
  • Joined: 29 Nov 2013
  • Loc: Full time RV wanderer

Posted 15 January 2015 - 06:32 PM

Darks can just be taken with the camers itself. Lens cap on, preferably in a very dark room, or covered by something to block out any potential stray light. Personally, at least during winter, I actually stick my 5D III or 7D in the fridge or freezer, after normalizing the temperature closer to the real ambient I had when imaging, and let it rip. ;P It's the only way I can get similar temperatures, and when the refrig/freezer door is closed, the lights go off and it's nice and dark in there. I would let the camera normalize in temperature for about 30 minutes, then do a warmup cycle to get the sensor temperature normalized (take a few dark frames that you'll just throw away)...then take your actual darks.

Haha, you should see the look on my Wife's face, when she comes home and finds a 30' USB cable running from the fridge into my office. "No honey, dinner will have to wait for awhile. The light in the fridge will ruin everything". On a serious note, thanks for all the info, Jon and others. I did not know that Bias and Flats were related. I ASSumed bias frames were only associated with darks, as they are frequently mentioned in the same breath. Good info! Looks like another thread to be pinned.

Regards, Kyle


Edited by ManicSponge, 15 January 2015 - 06:34 PM.

  • happylimpet likes this

#25 Jon Rista

Jon Rista

    ISS

  • *****
  • Posts: 22896
  • Joined: 10 Jan 2014
  • Loc: Colorado

Posted 16 January 2015 - 12:45 PM

A++ Jon!   You explained it perfectly.  Nicely done.

 

John

 

Thank you, John. :) Always glad to help when I can. If I'd had this explanation when I was starting, things would have been so much easier. :p




CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.


Recent Topics






Cloudy Nights LLC
Cloudy Nights Sponsor: Astronomics