49 replies to this topic

[Jim R]

Okay, I know this topic has been well covered, but I have read contradictory threads on these forums. Some say bias frames are essential for dslr astrophotography, while others say bias frames are not needed at all. I know deep sky stacker is probably superior to sequator, but I do use sequator for stacking, and that software does not even have a provision to use bias frames. Any definitive answer for this? I apologize if I am whipping a dead horse, but I seriously don't know. Thanks in advance.

[galacticinsomnia]

Calibration images are something that you can evolve with, as you grow in your image acquisition and processing skills.
Start with taking none, get the hang of what you are doing.  If you are finding things you don't like, search for ways to fix those things.  It is difficult often times to assess what you need to do, and usually the first thing people say is you need calibration frames and why you need them, which honestly, if I would have done that, I would have given up a long time ago.  There is a place for calibration frames, but good technique in data acquisition as well as post processing will be more important than relying on the mystical seeming calibration voodoo lol

Just shoot, post your shots, identify what you like, and what you don't, when you see characteristics in your images that you need help with, ask specifically if you can, so you don't end up searching for unicorns.

Clear Skies !!

 

[vidrazor]

Okay, I know this topic has been well covered, but I have read contradictory threads on these forums. Some say bias frames are essential for dslr astrophotography, while others say bias frames are not needed at all. I know deep sky stacker is probably superior to sequator, but I do use sequator for stacking, and that software does not even have a provision to use bias frames. Any definitive answer for this? I apologize if I am whipping a dead horse, but I seriously don't know. Thanks in advance.

Sequator is an interesting app, it can sometimes give you great results. It's rather odd that it doesn't use bias, especially when it will use flats, as flats are dependent on bias to process correctly.

 

I would say just roll with what you're getting in Sequator and decide if it's working for you. If you're having difficulties than you may want to start shooting a full set of calibration frames and use an app like DSS or Siril to do your stacking.

[17.5Dob]

Bias frames are absolutely nessesary...you have to use them to calibrate your flats....The fact that Sequator doesn't use them is not all that surprising given how completely stripped down it is. I find it is "adequate" for wide angle nightscapes, but that is all.

[Feldhusen]

Just wondering.

 

Bias, dark and flat frames were used to calibrate CCD image sensors (where uneven sensitivity to pixels is a problem basically for measurenent and scientific use, and read noise is pretty high too). Longer sub exposures compensates for the relatively high read noise (less read noise per sub exposure). With CCDs I took fewer but longer exposures to reach a certain total integration time.

 

CMOS cameras have low read noise (practically of no concern) but shoot noise is more pronounced. Shoot noise is random and can not be calibrated away. Lots of shorter CMOS exposures can be stacked as read noise is no practical problem. Just expose until the background is a bit over the noise floor and that is the most efficient exposure.

 

The most dominant source of noise to most CMOS users (unless having access to a dark location) is usually the sky background. Light pollution and moonlight beeing two typical sources for brightening the background sky.

 

Commercial CMOS cameras are made to take splendid images without calibration. No need for bias (unless shooting flats), or flats (when keeping the image sensor clean), or dark frames (software handles hot pixels).

 

My personal experience with CMOS cameras is that plain shooting without calibration works better. Vignetting is handled by software and dust motes are few and far between when taking care before letting loose.

 

For my CCD cameras I always used bias, dark and flats. Not so with my CMOS cameras.

 

Why cling to calibration routines developed for CCD image sensors when using CMOS image sensors?

 

My impression (right or wrong) is that methods developed for CCD image sensors mainly (not only) deal with problems inherent in one kind of image sensor but not the other one. Questions around this theme always wind up in long discussions where participants have strong opposing opinions - each round seems to only muddle the water more.

[Jim R]

Thanks to all for the responses. I am still a bit confused (I am a little dense) but I have much to ponder for further study. Thanks again.

[Ettu]

Thanks for bringing up this topic Jim R,

The consensus as I read it, seems to be that bias frames are used to make flats their most effective.

And bias frames are easy to take, (and use) so why not take and use them - that is, if you're going to use flats at all.

As it happens I am just starting down the road to see if flats can help me. I have a full frame DSLR (mirrorless actually) and at that size, vignette becomes annoyingly noticeable (a light drop off to ~70% toward the edges). Digital development solutions help but just are not satisfactory for me anymore. So I'm going to give flats a try, and if I go that far, I might as well throw in bias frames as well.

[Michael Covington]

You get perfectly correct calibration with lights, darks, flats, and flat darks, without bias frames.

You get essentially the same thing with lights, darks, flats, and bias frames, without flat darks.  Theoretically the first is a tad more accurate.

 

Bias frames are necessary if your software is going to scale the darks to apply them to a different exposure length than they were taken at.  (This is a dubious practice anyhow.)

The formulas describing how calibration is done are published in several books.  

[Ettu]

Just wondering.

 

Bias, dark and flat frames were used to calibrate CCD image sensors (where uneven sensitivity to pixels is a problem basically for measurenent and scientific use, and read noise is pretty high too). Longer sub exposures compensates for the relatively high read noise (less read noise per sub exposure). With CCDs I took fewer but longer exposures to reach a certain total integration time.

 

CMOS cameras have low read noise (practically of no concern) but shoot noise is more pronounced. Shoot noise is random and can not be calibrated away. Lots of shorter CMOS exposures can be stacked as read noise is no practical problem. Just expose until the background is a bit over the noise floor and that is the most efficient exposure.

 

The most dominant source of noise to most CMOS users (unless having access to a dark location) is usually the sky background. Light pollution and moonlight beeing two typical sources for brightening the background sky.

 

Commercial CMOS cameras are made to take splendid images without calibration. No need for bias (unless shooting flats), or flats (when keeping the image sensor clean), or dark frames (software handles hot pixels).

 

My personal experience with CMOS cameras is that plain shooting without calibration works better. Vignetting is handled by software and dust motes are few and far between when taking care before letting loose.

 

For my CCD cameras I always used bias, dark and flats. Not so with my CMOS cameras.

 

Why cling to calibration routines developed for CCD image sensors when using CMOS image sensors?

 

My impression (right or wrong) is that methods developed for CCD image sensors mainly (not only) deal with problems inherent in one kind of image sensor but not the other one. Questions around this theme always wind up in long discussions where participants have strong opposing opinions - each round seems to only muddle the water more.

In many ways I agree with you.

For me the whole reason for getting a dslr, is that it's relatively easy to get nice, colorful, and usable photos with a minimum of post processing effort. I'd encourage anyone starting in this hobby to begin with that. Maximum return for minimum effort. But forewarned, it's not as easy as taking a picture of a flower in daylight. Never the less, with the equipment, dslr cameras, and free software out there (and free advice in places like this), it is within reach to get results that would be the envy of professionals of just 15 yrs ago.

 

However, at the stage I'm at, I don't want to spend any more money, so I'm becoming interested in getting all I can out of what I already have, with my dslr. If darks flats and bias frames can get me to the next level, for free, I'm going to give it a try. 

Edited by Ettu, 28 December 2021 - 12:32 PM.

[galacticinsomnia]

I've seen plenty of beautiful outcomes without calibration frames.  It is intellectually dishonest to state they are required or necessary for imaging.
I have have many examples where NO calibration frames are used, and really, not so bad for my insomnia... lol

Some of these posts in the past year or so may have calibration frames, but you can look at the images and judge for yourself if you feel you need calibration frames or not.

They are great for myself, family, friends and outreach, not by any means something special, with the exception I am happy I have taken them, and have been enjoying my journey.

https://www.cloudyni...-edph-ii-again/
https://www.cloudyni...wo-frame-panel/
https://www.cloudyni...ield-flattener/
https://www.cloudyni...rica-61edph-ii/
https://www.cloudyni...fid-61-edph-ii/
https://www.cloudyni...ity-in-cepheus/
https://www.cloudyni...822-in-cepheus/
https://www.cloudyni...bula-61-edphii/
https://www.cloudyni...bula-61-edphii/
https://www.cloudyni...bula-happy-4th/
https://www.cloudyni...ebula-ngc-6992/
https://www.cloudyni...us-first-light/
https://www.cloudyni...lephant-nebula/
https://www.cloudyni...ata-5hrs-total/
https://www.cloudyni...vbony-503-80ed/
https://www.cloudyni...1-bodes-galaxy/
https://www.cloudyni...ting-for-bodes/
https://www.cloudyni...nus-star-cloud/
https://www.cloudyni...acked-raw-data/
https://www.cloudyni...d-image-source/

 

Clear Skies !!

[sharkmelley]

I've seen plenty of beautiful outcomes without calibration frames.  It is intellectually dishonest to state they are required or necessary for imaging.
I have have many examples where NO calibration frames are used, and really, not so bad for my insomnia... lol

It's bad advice to skip flats (or lens corrections) because eventually beginners will want to extract more faint detail from their images and in general they will be hampered by the lack of calibration.

 

Mark

[17.5Dob]

I've seen plenty of beautiful outcomes without calibration frames.  It is intellectually dishonest to state they are required or necessary for imaging.
I have have many examples where NO calibration frames are used, and really, not so bad for my insomnia... lol

Some of these posts in the past year or so may have calibration frames, but you can look at the images and judge for yourself if you feel you need calibration frames or not.

They are great for myself, family, friends and outreach, not by any means something special, with the exception I am happy I have taken them, and have been enjoying my journey.

Clear Skies !!

Stacking is not required either, or a dSLR, just  point a cell phone into an eyepiece and take one click......

The point is that calibration frames can improve your photos for an extremely minimal time cost.........

Edited by 17.5Dob, 29 December 2021 - 02:58 AM.

[Ettu]

It's bad advice to skip flats (or lens corrections) because eventually beginners will want to extract more faint detail from their images and in general they will be hampered by the lack of calibration.

 

Mark

Well said!

[galacticinsomnia]

I think it's bad advice to recommend them to newcomers.  My images are not great but they don't suck.  I don't take them..
Learn post work without them, so you can see what you need to improve with them, and things make a lot more sense.

My images are not great, but...No calibration frames.  Could they be improved with calibration frames... Maybe, maybe not. In many of my cases I have found they do not.  However, I will also note, I do take bias and flats, but often find I do not need them, or my images do not benefit from them enough to make it worth my time. 

To say all images will benefit from them is intellectually dishonest because it just isn't true.  I don't want to debate it, I respect your opinions on the matter, I've seen various work and I like them, but if I spent the last few years jerking around trying to get calibration frames to work they way everyone is insisting that they are, instead of working on post processing, tuning my gear, and fixing other image aquisition issues, IE collimation, guiding, dithering, platesolving, etc, I would have most likely given up.  That is just me and that has been my journey.

I am in no way saying that calibration frames are bad, but I am saying they are not absolutely necessary for imaging, especially for a new comer.
I have no interesting in guiding a new comer to taking calibration frames, I am interesting in helping them find joy and reward in what they are doing.
Telling people who may be happy with images like mine or better, that they can't image without calibration frames is like telling a violinist that they synthetic bow will never sound as good and only realy horse hair will work. 

I'm proud of my captures, and I'm glad for my journey, and there are many roads to the same destination. 
I've had some good fortune with flats, and bias, but not enough or frequently enough, to have me telling people they have too have them.

Clear Skies !!

 

[17.5Dob]

In my own experience, (using multiple different lenses and telescopes with multiple different cameras,) the difference between a stack made from calibrated vs uncalibrated subs has been very obvious. Learning to post-process a properly calibrated stack is so much easier when you don't have the compound problems of vignetting on top of gradients.

In many,many cases, which happens on at least a weekly basis,when somebody posts an image and requests help processing it in either this forum or the beginners forum, one of the most common problems that comes up is the lack of flats and vignetting so bad it is unrepairable. Hours of post processing headaches could have been avoided by taking 2 minutes to shoot a set of flats/bias.

I would no more advocate skipping calibration frames than say that good PA is not important because autoguiding can still handle it if you're off.

[BQ Octantis]

Jim,

 

You stirred up a hornets' nest!

 

If you're still reading, for DSLR processing bias is the least critical of all calibration frames. That is likely why there are so many opinions about them.

 

In my experience (producing images that I can scale to 150-200% sensor scale), the order of precedence for calibration frames is

 

1. Flats. Even just one (1) flat will save hours of processing and produce a better result than no flat. Vignetting is a luminosity error that affects all pixels at all image scales, so you can't fix it by simply downsampling. And there is no vignette model in any program that better matches that of your setup's than your own setup's vignette. Even a poorly illuminated flat reduces post-processing time and produces a better result than no flat.
2. Darks. Their biggest benefit is reducing hot pixels, but they also remove thermal noise and bias error.
3. Flat darks (a.k.a. dark flats). If you're going to calibrate your flats you should use flat darks, not bias. But without them, the error introduced by uncalibrated flats is much less than the remaining error after applying darks.
4. Bias. These are only necessary if you're going to use a program that scales your darks to try to match the thermal noise in the lights.

 

In the end, what we're after is pixels with correct luminosity values. Using calibration frames is simply a way to more accurately correct the values by using the setup's own, measured errors as part of the model. Without them I can get a pretty image at 33% sensor scale. With them I can get a pretty image at 150-200% sensor scale—but I've never needed flat darks or bias to do so.

 

BQ

Edited by BQ Octantis, 30 December 2021 - 09:07 AM.

[sharkmelley]

Jim,

 

You stirred up a hornets' nest!

 

If you're still reading, for DSLR processing bias is the least critical of all calibration frames. That is likely why there are so many opinions about them.

 

In my experience (producing images that I can scale to 150-200% sensor scale), the order of precedence for calibration frames is

 

1. Flats. Even just one (1) flat will save hours of processing and produce a better result than no flat. Vignetting is a luminosity error that affects all pixels at all image scales, so you can't fix it by simply downsampling. And there is no vignette model in any program that better matches that of your setup's than your own setup's vignette. Even a poorly illuminated flat reduces post-processing time and produces a better result than no flat.

Your argument is self-contradictory.  If you argue that flats are the most important calibration frames then you have to accept that the bias is equally important, in order for the flats to calibrate properly.

[BQ Octantis]

Your argument is self-contradictory.  If you argue that flats are the most important calibration frames then you have to accept that the bias is equally important, in order for the flats to calibrate properly.

 

Hi Mark,

 

I disagree. And not just from experience, either.

 

The calibration equation for a pixel is

 

 

The relative pixel calibration error from deviations from "truth" in the calibration frames is

 

 

The first term is the residual error from the deviation of the darks from the lights. The second term is the error from the deviation of the flats from the lights. For properly exposed flats ∂f ≈ 0, and the second term will be much smaller than the first—namely because f >>d_f (by more than two orders of magnitude in my data, so <1% if I omit flat darks entirely). And the standard deviation of ∂d_f is much smaller than ∂d—so the contribution of the second term is much more uniform than the first. Lastly, if dithering, sigma-reject stacking, and noise reduction can conquer ∂d, then they can easily vanquish ∂d_f . But in my experience, the error from the second term is imperceptible even without dithering.

 

Still, if your calibration program will calibrate your flats, then do so with flat darks, not bias. Otherwise, in the pursuit of "perfect" calibration, by using bias you're omitting the thermal noise in the flats. But in that pursuit, you probably want to make sure you figured out temperature matching of your darks to your lights first. Indeed, the first term illustrates the greater importance of temperature matching—at hot temperatures, the temperature sensitivity of the standard deviation in ∂d across the image becomes exponential, and in the dim areas, l-d gets pretty bloody small! (If it's not obvious, dividing a wildly varying number by a really small number produces a really big wildly varying number.)

 

I might have to write an article on the matter…

 

Cheers,

 

BQ

Edited by BQ Octantis, 30 December 2021 - 10:32 AM.

[sharkmelley]

Hi Mark,

 

I disagree. And not just from experience, either.

 

The calibration equation for a pixel is

 

calibration.png

 

The relative pixel calibration error from deviations from "truth" in the calibration frames is

 

error.png

 

I really don't follow your argument.  I'm not at all sure your "error" equation is correct but let's give you the benefit of the doubt and assume it is right.  If you decide not to use your dark flat then ∂d_f = d_f.  That means the last term in your error equation is large for a typical Canon camera where d_f = 2048 (the bias level), even if the flat is well exposed so that f = 8000 or so (half the saturation level of 16000).

 

However, putting that aside, here is a better way of looking at it, with a numerical example.

Let's assume a scenario where the vignetting in the corners of the image is 50%.  After calibration, the signal level in the corners should match the signal level in the centre of the image.

 

Let's assume a Canon camera with a bias level of 2000 (instead of 2048, to make the arithmetic easier). 
Assume the signal level in the image centre is 1000, so the signal level in the corners is 500.  Adding the bias gives 3000 and 2500 respectively.
Assume the flats are exposed to almost half histogram, with a signal level of 8000 in the centre and 4000 in the corners. Adding the bias gives 10000 and 6000 respectively.

 

Let's use the calibration equation (l-d)/(f-df), which I agree to be correct.
With proper calibration:

• The calibrated value in the centre is (3000-2000)/(10000-2000)= 0.125
• The calibrated value in the corner is (2500-2000)/(6000-2000) = 0.125

The calibrated values in the centre and corner match, which shows calibration has worked correctly.

 

Now let's "forget" to use the dark flat:

• The calibrated value in the centre is (3000-2000)/10000= 0.1
• The calibrated value in the corner is (2500-2000)/6000 = 0.0833

This calibrated corner value is less than the centre - it's a very obvious undercorrection

 

Now let's "forget" to use both the dark and the dark flat:

• The calibrated value in the centre is 3000/10000= 0.3
• The calibrated value in the corner is 2500/6000 = 0.4167

This calibrated corner value is higher than the center - it's a very obvious overcorrection

 

Mark

Edited by sharkmelley, 30 December 2021 - 11:19 AM.

[Ettu]

Stacking is not required either, or a dSLR, just  point a cell phone into an eyepiece and take one click......

The point is that calibration frames can improve your photos for an extremely minimal time cost.........

Just, perhaps, as an interesting aside on your first point.

This past fall, I helped my local high school teacher host an astronomy night. 

Jupiter, Saturn, the crescent moon, the ring neb, and others, were all "cool' and variously interesting

But by far, the hit of the evening was the cell phone over the eyepiece, trying to get their personal capture of the remarkable mountains of the moon. Once that got started, they all wanted to get their own, and we had to leave the scope up for a good half hour after we were otherwise done, before the last ones either succeeded, or were willing to give up on it.

Just a hint on that kind of capture. Use video mode. Frame #172 of one such capture ended up on the school's web page, with glowing reviews of the fun night.

[BQ Octantis]

I'm not at all sure your "error" equation is correct but let's give you the benefit of the doubt and assume it is right.  If you decide not to use your dark flat then ∂d_f = d_f.  That means the last term in your error equation is large for a typical Canon camera where d_f = 2048 (the bias level), even if the flat is well exposed so that f = 8000 or so.

 

Crikey, Mark, if your flat dark is at 2048 and your flat is only at 8000, then you need to use the dark—or even the bias! Otherwise, the calibration error will be enormous!

 

My wide-open f/2.8 flat (the worst of my vignette) ranges from 0.1104 at the center to 0.0875 at the corner. And my flat dark is at 0.0005. This puts the max of the second term of the calibration error at 0.49-0.61%.

 

Don't take my word for the formula for ∂c/c ; you can derive it for yourself. Just take the total differential of the calibration equation and divide by c ; then set ∂l to 0 (or leave it in for the shot noise in l ). Technically, both terms are negative; I simply took the absolute value for esthetics of the formula.

 

You can even split out the bias error from the thermal error:

 

 

Cheers,

 

BQ

Edited by BQ Octantis, 30 December 2021 - 11:35 AM.

[sharkmelley]

Crikey, Mark, if your flat dark is at 2048 and your flat is only at 8000, then you need to use the dark—or even the bias! 

Go back to my earlier post which I've updated with a worked example.  It might help clarify things.

[BQ Octantis]

Go back to my earlier post which I've updated with a worked example.  It might help clarify things.

 

Never in my processing have I ever had an offset of that magnitude induced by the uncalibrated flat. Indeed, my flat processing is good enough to detect the tiny hysteresis in the vignetting at the very edges of the corners on a uniform target through a zoom lens. This would be masked with an error of that size!

 

The only thing I can reckon is that all the software I use to process the Canon raws automatically subtracts the bias value of 2048. If that's the case, then RTFM for your processing software—bias may indeed make or break the processing.

 

BQ

Edited by BQ Octantis, 30 December 2021 - 12:50 PM.

[sharkmelley]

The only thing I can reckon is that all the software I use to process the Canon raws automatically subtracts the bias value of 2048. If that's the case, then RTFM for your processing software—bias may indeed make or break the processing.

What software are you using?  If it's automatically subtracting the bias level then it confirms just how important the bias level is in astro-image processing.

 

Mark

[BQ Octantis]

What software are you using?  If it's automatically subtracting the bias level then it confirms just how important the bias level is in astro-image processing.

 

Mark

 

Nebulosity, APP, Photoshop, Gimp, and currently RawTherapee. I have Siril installed, but I haven't messed with it yet.

 

Again, my contention is that the bias deviation is the least significant of all calibration error. If the software simply subtracts the known bias value (e.g., 2048 for the Canon 600D/T3i, contained in the EXIF data), then the division of the light by the flat is still correct to first order (consistent with my experience). The residual error from the nonuniformity of the bias (i.e., each pixel's bias deviation from 2048, ∂b_f ) goes as the second term in the normalized differential of the calibration equation ∂c/c, as I already showed.

 

But you are correct that if the software doesn't at least subtract a reasonable guess at a constant bias value from the flat, the flat will be severely misapplied—whether processing AP or a clear blue sky for a 4π steradian panorama. But this would be obvious after calibration.

 

Again, RTFM.

 

BQ

Edited by BQ Octantis, 30 December 2021 - 02:50 PM.