Jump to content

  •  

CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.

Photo

Canon Banding Help

  • Please log in to reply
80 replies to this topic

#26 skywatch

skywatch

    Ranger 4

  • -----
  • Posts: 345
  • Joined: 06 Feb 2008
  • Loc: Ottawa 45.3085N 76.175W

Posted 02 January 2016 - 08:16 PM

Anyone know a good way to remove wide horizontal banding?  I have a Canon 450D which is getting very annoying in that respect.  I have tried various camera settings and cooling and it looks like I'll be stuck with fixing them in post-processing.  I would think that dark frame subtraction would help as they show up in the stretched dark frames but so far that's a no-go (at least in DSS.)

 

I had this issue with my 40D and 5D mark II.  Not only for astrophotos, but also for daylight images in the deep shadow areas.  I suspect that it varies between individual cameras, so some users may never see the problem.

 

Not all frames had the banding, but when present it was mostly in the red channel.  More frames helped.  All ISO settings had the problem.  Try leaving a longer pause between frames.  I ultimately solved the problem by trading in the 5D mark II for a 6D.  No more banding even on individual frames.



#27 17.5Dob

17.5Dob

    Fly Me to the Moon

  • ****-
  • Posts: 7,084
  • Joined: 21 Mar 2013
  • Loc: Colorado,USA

Posted 02 January 2016 - 08:26 PM

THIS is the best solution ;) 



#28 Tonk

Tonk

    Cosmos

  • *****
  • Posts: 9,418
  • Joined: 19 Aug 2004
  • Loc: Leeds, UK, 54N

Posted 03 January 2016 - 04:55 AM

First, stop using DSS. ;) Something I discovered when I switched to using PI to integrate is that I generally stopped having problems with banding. The AHD demosaicing algorithm used by DSS seems to be similar to the AHD algorithm used by Lightroom, and both cause banding.

 

Jon this is heavy handed advise (DSS being tarred with the same brush yet again) - to counter your "evidence" go and look at my Astrobin gallery. All stacked images use DSS, they are not dithered and are properly calibrated with darks, flats, dark flats and not one exhibits banding. So you can't say that DSS is the cause of the banding per se. There is another root cause behind this and its worth investigating what it might be.

 

Looking at HillTop's banding sample it looks like calibration isn't being performed entirely correctly (possibly bad calibration frames) as the sensor bias pattern is still in evidence. Switching applications while using bad calibration frames likely won't improve matters other than adding the application of de-banding algorithms to patch up poor calibration.

 

I'd also consider some local EM interference near the camera as a possible source of banding if it turns out not to be poor calibration.


  • josh smith likes this

#29 Jerry Lodriguss

Jerry Lodriguss

    Vendor

  • ***--
  • Vendors
  • Posts: 7,796
  • Joined: 19 Jul 2008
  • Loc: Voorhees, NJ

Posted 03 January 2016 - 01:57 PM

First, stop using DSS. ;) Something I discovered when I switched to using PI to integrate is that I generally stopped having problems with banding. The AHD demosaicing algorithm used by DSS seems to be similar to the AHD algorithm used by Lightroom, and both cause banding. AHD in general, with Canon raw files, seems to be a little prone to banding, but the DSS algorithm seems to be particularly bad. 

 

That's a bit extreme considering you don't have to use AHD demosaicing in DeepSkyStacker, you can just use bilinear.

 

And more exposure is always a good idea.

 

Jerry



#30 Jon Rista

Jon Rista

    ISS

  • *****
  • Posts: 24,818
  • Joined: 10 Jan 2014
  • Loc: Colorado

Posted 03 January 2016 - 03:39 PM

 

First, stop using DSS. ;) Something I discovered when I switched to using PI to integrate is that I generally stopped having problems with banding. The AHD demosaicing algorithm used by DSS seems to be similar to the AHD algorithm used by Lightroom, and both cause banding. AHD in general, with Canon raw files, seems to be a little prone to banding, but the DSS algorithm seems to be particularly bad. 

 

That's a bit extreme considering you don't have to use AHD demosaicing in DeepSkyStacker, you can just use bilinear.

 

And more exposure is always a good idea.

 

Jerry

 

 

Bilinear isn't really going to be any better, as it is prone to mazing artifacts. 



#31 Jon Rista

Jon Rista

    ISS

  • *****
  • Posts: 24,818
  • Joined: 10 Jan 2014
  • Loc: Colorado

Posted 03 January 2016 - 03:44 PM

 

First, stop using DSS. ;) Something I discovered when I switched to using PI to integrate is that I generally stopped having problems with banding. The AHD demosaicing algorithm used by DSS seems to be similar to the AHD algorithm used by Lightroom, and both cause banding.

 

Jon this is heavy handed advise (DSS being tarred with the same brush yet again) - to counter your "evidence" go and look at my Astrobin gallery. All stacked images use DSS, they are not dithered and are properly calibrated with darks, flats, dark flats and not one exhibits banding. So you can't say that DSS is the cause of the banding per se. There is another root cause behind this and its worth investigating what it might be.

 

Looking at HillTop's banding sample it looks like calibration isn't being performed entirely correctly (possibly bad calibration frames) as the sensor bias pattern is still in evidence. Switching applications while using bad calibration frames likely won't improve matters other than adding the application of de-banding algorithms to patch up poor calibration.

 

I'd also consider some local EM interference near the camera as a possible source of banding if it turns out not to be poor calibration.

 

 

There has been testing done of the AHD (Adaptive Homogeneity-Directed Demosaicing) algorithm, which is used by DSS (as well as Lightroom, and is an option in RawTherapee and DarkTable, and one of the available algorithms in DCRAW), and it has been shown to exacerbate and even introduce banding in the images it demosaics. This is most often the case with Canon files, probably because the existing banding effectively seeds the problem, and is a lot less common with other brands. It is not DSS per-se that is the problem, but the AHD algorithm that is the problem. 

 

I forget if it was the Luminous Landscape forums or Fred Miranda forums or some other forums where testing with multiple RAW converters was done, if I can find it I'll link it. I stand by my comment, though, that AHD can be and often is a SOURCE of banding when demosaicing Canon RAW data.


Edited by Jon Rista, 03 January 2016 - 03:44 PM.


#32 skywatch

skywatch

    Ranger 4

  • -----
  • Posts: 345
  • Joined: 06 Feb 2008
  • Loc: Ottawa 45.3085N 76.175W

Posted 03 January 2016 - 05:35 PM

The bands are most likely in the data.  See below for a particularly bad example of a single frame from my old 40D.  I was able to make improvements by using more frames, dithering, lower ISO, working at colder temperatures, going to where there is less light pollution, suppressing the red base level, using shorter exposures, and switching to a 6D.  No amount of calibration gets rid of them completely because they are only partly random.  They won't show up in dark frames or flats.  Maybe many equally long exposures of a faint uniform background would help.   I think the red pixels have some variation in sensitivity along rows which the camera's gain stage amplifies for the smaller red signals.  The best solution is to get a camera that does not exhibit the problem.  A modded camera is better because the red will have a greater SNR,

Attached Thumbnails

  • IMG_1318.JPG


#33 skywatch

skywatch

    Ranger 4

  • -----
  • Posts: 345
  • Joined: 06 Feb 2008
  • Loc: Ottawa 45.3085N 76.175W

Posted 03 January 2016 - 06:00 PM

I should add that the situation is not totally hopeless. Stacking only 5 frames like the one above, with calibration and adjustments, produced the following.  (obviously even more frames are desirable)

Attached Thumbnails

  • m57_5x300_canon_cropb.JPG

Edited by skywatch, 03 January 2016 - 06:02 PM.


#34 Jon Rista

Jon Rista

    ISS

  • *****
  • Posts: 24,818
  • Joined: 10 Jan 2014
  • Loc: Colorado

Posted 03 January 2016 - 06:42 PM

Oh, one thing I should note. If you sufficiently swamp read noise with signal, then banding ceases to be a problem pretty much regardless. I had lots of problems with banding when I was using DSS, and when using PI with AHD, earlier on as a beginner. Back then, however, I did not expose deeply enough. Background sky was usually totally buried in the read noise, and banding was a huge problem. I moved to PI, and VNG hardly had any issues with banding at all...but my issues with banding disappeared entirely (even with AHD or DSS) once I started exposing long enough to fully swamp read noise with signal. As long as I get to 1/3 histogram with my 5D III, I usually don't have any issues with banding no matter how deep into the signal I dig, regardless of the algorithm.

 

So, my primary recommendation to the OP is to expose more deeply. That can be challenging, as just moving to a higher ISO is not necessarily good enough to avoid banding issues. It really means figuring out a way to expose longer per sub, to the point where your histogram approaches 1/3rd (but not further...any further and your just throwing away dynamic range), and even REDUCING ISO if that is what it takes to get longer subs. I started out using ISO 400 and 120-210 second subs. Then I went to ISO 1600 with 210-270 second subs, and eventually ISO 800-1600 with 300-360 second subs. These days, I use ISO 400-800, with 10-12 minute subs, and I usually aim for 1/4-1/3 histogram. Signal tends to grow faster than noise, and with fewer longer subs, you end up with less total read noise in your stacked images. All of that leads to cleaner results...even with a very noisy Canon DSLR (and trust me, my 5D III is one hell of a noisy sucker!)


  • okiedrifter likes this

#35 17.5Dob

17.5Dob

    Fly Me to the Moon

  • ****-
  • Posts: 7,084
  • Joined: 21 Mar 2013
  • Loc: Colorado,USA

Posted 03 January 2016 - 07:23 PM

 

 

So, my primary recommendation to the OP is to expose more deeply. That can be challenging, as just moving to a higher ISO is not necessarily good enough to avoid banding issues. It really means figuring out a way to expose longer per sub, to the point where your histogram approaches 1/3rd (but not further...any further and your just throwing away dynamic range), and even REDUCING ISO if that is what it takes to get longer subs. I started out using ISO 400 and 120-210 second subs. Then I went to ISO 1600 with 210-270 second subs, and eventually ISO 800-1600 with 300-360 second subs. These days, I use ISO 400-800, with 10-12 minute subs, and I usually aim for 1/4-1/3 histogram. Signal tends to grow faster than noise, and with fewer longer subs, you end up with less total read noise in your stacked images.

Well not according to Roger Clark !! He's been on a roll for the last 6 months. He says that Canons need to shoot at ISO 1600 in order to completely digitize the signal, ISO 800 is OK for Nikons, but dropping your ISO results in not completely "digitizing" your signal and loss of the faintest signals. YMMV ;)


Edited by 17.5Dob, 03 January 2016 - 07:24 PM.


#36 Jerry Lodriguss

Jerry Lodriguss

    Vendor

  • ***--
  • Vendors
  • Posts: 7,796
  • Joined: 19 Jul 2008
  • Loc: Voorhees, NJ

Posted 03 January 2016 - 07:42 PM

 

 

Bilinear isn't really going to be any better, as it is prone to mazing artifacts. 

 

 

Odd that I never have banding or mazing problems.

 

If you are not correctly exposing, and you are getting problems with banding, then that is operator error. :-)

 

Jerry


Edited by Jerry Lodriguss, 03 January 2016 - 07:46 PM.


#37 Jon Rista

Jon Rista

    ISS

  • *****
  • Posts: 24,818
  • Joined: 10 Jan 2014
  • Loc: Colorado

Posted 03 January 2016 - 08:32 PM

So, my primary recommendation to the OP is to expose more deeply. That can be challenging, as just moving to a higher ISO is not necessarily good enough to avoid banding issues. It really means figuring out a way to expose longer per sub, to the point where your histogram approaches 1/3rd (but not further...any further and your just throwing away dynamic range), and even REDUCING ISO if that is what it takes to get longer subs. I started out using ISO 400 and 120-210 second subs. Then I went to ISO 1600 with 210-270 second subs, and eventually ISO 800-1600 with 300-360 second subs. These days, I use ISO 400-800, with 10-12 minute subs, and I usually aim for 1/4-1/3 histogram. Signal tends to grow faster than noise, and with fewer longer subs, you end up with less total read noise in your stacked images.

Well not according to Roger Clark !! He's been on a roll for the last 6 months. He says that Canons need to shoot at ISO 1600 in order to completely digitize the signal, ISO 800 is OK for Nikons, but dropping your ISO results in not completely "digitizing" your signal and loss of the faintest signals. YMMV ;)


I think all that matters is that your gain is such that one single electron results in at least 1 ADU...or more. Most Canon cameras use a gain such that by ISO 400 (pretty much every APS-C camera, and newer high resolution FF cameras) or ISO 800 (FF DSLR cameras with larger pixels), your usually at a gain (e-/ADU) of less than 1.0, which means that you get more than one ADU out for each electron read. That would mean you couldn't be losing any definition for any ISO from 400 and up (although at particularly high ISOs, such as ISO 3200 on my 5D III, a secondary downstream amplifier kicks in which seems to mess with the noise and the dynamic range), you should be fine.

Frank also covered quantization noise in a thread not too long ago here in DSLR (I think...maybe it was BII) and showed that quantization noise is extremely small in DSLR cameras, and it's basically moot to be concerned about it.
  • Miguelo likes this

#38 17.5Dob

17.5Dob

    Fly Me to the Moon

  • ****-
  • Posts: 7,084
  • Joined: 21 Mar 2013
  • Loc: Colorado,USA

Posted 03 January 2016 - 08:55 PM

 

 

So, my primary recommendation to the OP is to expose more deeply. That can be challenging, as just moving to a higher ISO is not necessarily good enough to avoid banding issues. It really means figuring out a way to expose longer per sub, to the point where your histogram approaches 1/3rd (but not further...any further and your just throwing away dynamic range), and even REDUCING ISO if that is what it takes to get longer subs. I started out using ISO 400 and 120-210 second subs. Then I went to ISO 1600 with 210-270 second subs, and eventually ISO 800-1600 with 300-360 second subs. These days, I use ISO 400-800, with 10-12 minute subs, and I usually aim for 1/4-1/3 histogram. Signal tends to grow faster than noise, and with fewer longer subs, you end up with less total read noise in your stacked images.

Well not according to Roger Clark !! He's been on a roll for the last 6 months. He says that Canons need to shoot at ISO 1600 in order to completely digitize the signal, ISO 800 is OK for Nikons, but dropping your ISO results in not completely "digitizing" your signal and loss of the faintest signals. YMMV ;)

 


I think all that matters is that your gain is such that one single electron results in at least 1 ADU...or more. Most Canon cameras use a gain such that by ISO 400 (pretty much every APS-C camera, and newer high resolution FF cameras) or ISO 800 (FF DSLR cameras with larger pixels), your usually at a gain (e-/ADU) of less than 1.0, which means that you get more than one ADU out for each electron read. That would mean you couldn't be losing any definition for any ISO from 400 and up (although at particularly high ISOs, such as ISO 3200 on my 5D III, a secondary downstream amplifier kicks in which seems to mess with the noise and the dynamic range), you should be fine.

Frank also covered quantization noise in a thread not too long ago here in DSLR (I think...maybe it was BII) and showed that quantization noise is extremely small in DSLR cameras, and it's basically moot to be concerned about it.

 

If you have the time, you can wade through THIS thread at DPReview about Unity Gain and low ISO's with another dissertation paper being written by Roger. I may not like his processing methods, or his diatribes about proper star colors in the MW, but he does know a thing or two about digital sensors.



#39 HillTop

HillTop

    Ranger 4

  • -----
  • topic starter
  • Posts: 324
  • Joined: 26 Apr 2014
  • Loc: Belpre, OH

Posted 04 January 2016 - 09:47 AM

     Thanks for the new input but this one has totally stumped me.  Looks like I might just live with it for now.  I've tried all kinds of different combinations of settings and processing and nothing fixes the problem.  It's not too noticeable on images that aren't stretched really hard and/or contain little sky background.  The bands are not like the ones skywatch had in his image, much wider and not evenly sized/spaced.

     Regarding ISO, I haven't done a scientific comparison, but images shot at ISO 400 are much smoother and stretch better than 800, and 1600 is very noisy.  Jerry - operator error is entirely possible.  :lol:



#40 Poochpa

Poochpa

    Vanguard

  • *****
  • Posts: 2,177
  • Joined: 12 Feb 2005
  • Loc: Buffalo Burbs, NY

Posted 04 January 2016 - 02:20 PM

I've been imaging with Canon dslrs for the past 12 years, starting with the original Rebel, and including the XT, XSi, T2i and T4i. Some of them modded, others stock. They all have exhibited horizontal banding from time-to-time, regardless of ISO, exposure duration, whether calibration frames have been applied, or the software used for initial processing (DSS or ImagesPlus).  There is no rhyme or reason as to when it occurs.

Mike



#41 HillTop

HillTop

    Ranger 4

  • -----
  • topic starter
  • Posts: 324
  • Joined: 26 Apr 2014
  • Loc: Belpre, OH

Posted 04 January 2016 - 03:05 PM

Mike, my two 450Ds are pretty consistent with the banding (one modded, one not.)  It's just a lot harder to pick out in some images.  Sometimes I think it looks fine, then a few days later I happen to look at it again and it's there.  Other times I notice it right away in post.  At least for my eyes it depends on image content a lot whether I can see it well or not.



#42 dugpatrick

dugpatrick

    Mariner 2

  • *****
  • Posts: 219
  • Joined: 13 Oct 2010
  • Loc: Round Rock, TX

Posted 04 January 2016 - 05:00 PM

I do not see banding with my Canon 350d or my Canon 450d, when calibrated.  I use DSS and sigma kappa stacking.  You can certainly cause banding by mixing ISO's, or by not calibrating properly.  In my experience dithering is essential when using a DSLR, because the temperatures will never match exactly between light frames and dark frames.

 

Doug



#43 freestar8n

freestar8n

    Vendor - MetaGuide

  • *****
  • Vendors
  • Posts: 10,327
  • Joined: 12 Oct 2007
  • Loc: Melbourne, Australia

Posted 04 January 2016 - 05:18 PM

Hi-

For people interested in Canon banding issues, they should know that a few years ago when 20d's were in use, people on CN complained about the banding and gave advice that results were actually better with *shorter* exposures. To my amazement - this was actually true. All the theory about swamping read noise and so forth went out the window - because it was clear that longer exposures actually showed more banding than short ones.

I spent some time on this and realized something had to be very wrong in the calibration of the exposures - making everything nonlinear - and it turned out it was due to the raw files not being converted in exactly the right way. So the calibration process wasn't linear and the signal was not properly rising above the noise.

I don't do much work with canon's nowadays - but when I see people talking about different degrees of banding I would make sure the conversion from raw is happening in exactly the right way. If the banding is the same in each exposure and if you take all bias, darks, lights at the same ISO then the banding should be normal pattern noise that calibrates away pretty well. If some people are getting good results and others have problems I would look at the raw conversion involved.

As for the issue of ISO and digitization noise - the theory is pretty clear on how big the effect of digitization should be - and it should be small. But if people do see a big effect it may mean there is something else going on - as there often is with dslr's.

Frank
  • skywatch, Miguelo, Jon Rista and 1 other like this

#44 HillTop

HillTop

    Ranger 4

  • -----
  • topic starter
  • Posts: 324
  • Joined: 26 Apr 2014
  • Loc: Belpre, OH

Posted 05 January 2016 - 08:09 AM

In the spirit of testing I did a batch of ISO 400 15 min exposures on M81 and M82 last night and am gathering the darks now.  Next chance I get I'll go for something like ISO 800 @ 8-10 min and compare.  I used to shoot 2-5 min at 800 though, and TBH I think the banding was worse then (although I wasn't dithering at the time.)



#45 dugpatrick

dugpatrick

    Mariner 2

  • *****
  • Posts: 219
  • Joined: 13 Oct 2010
  • Loc: Round Rock, TX

Posted 05 January 2016 - 09:37 AM

One more thought: Horizontal banding is possible if you have power/ground issues.

 

Just last night I was hacking on my old Canon 350d, and I had some of the parts separated while in operation. When I noticed horizontal banding and I was able to fix the issue by unplugging the laptop from AC power, thus separating the usb ground from the other grounds (camera power and shutter cable).

 

My case was not the norm, but it's worth considering ground issues when horizontal banding is not removed with calibration.

 

Doug



#46 HillTop

HillTop

    Ranger 4

  • -----
  • topic starter
  • Posts: 324
  • Joined: 26 Apr 2014
  • Loc: Belpre, OH

Posted 05 January 2016 - 10:07 AM

Doug - I get the same results even running the camera off battery and using an intervalometer.  Thanks for your input, though.

 

Frank - could be a conversion problem as I believe dcraw is used by both DSS and PI, although I'm not too familiar with the details of how it works.  I need to see if the pattern is being recorded the same in all frames, but thinking back I wonder now if it's in the darks but not in the lights - wouldn't make sense but that would be a problem.  I don't remember if I checked for that.  :foreheadslap:



#47 HillTop

HillTop

    Ranger 4

  • -----
  • topic starter
  • Posts: 324
  • Joined: 26 Apr 2014
  • Loc: Belpre, OH

Posted 06 January 2016 - 07:55 AM

May be making some progress on this.  I stacked 32 900s exposures is DSS with only a master bias for calibration and noticed very little, if any, banding (I think I saw some.)  I then stacked the same data adding in darks and flats and the banding really stood out.  Looking over the .CR2 files in Canon's DPP it actually looks like the low signal where the banding lies is being clipped out of the lights.  I need to learn more about what's going on here and check the bias as well.  At this point it looks like I'm subtracting "noise" from the lights that isn't there to begin with.  I'm sure there is a way to clip this info out of the dark frames as well, just leaving hot pixels - or maybe using a defect map instead of darks?  I did get my ISO 800 data on the same subject last night and am building up darks now for the cooler temps we've been having. 



#48 freestar8n

freestar8n

    Vendor - MetaGuide

  • *****
  • Vendors
  • Posts: 10,327
  • Joined: 12 Oct 2007
  • Loc: Melbourne, Australia

Posted 06 January 2016 - 05:41 PM

I was just using the canon 20d as an example where theory and best practice may be completely wrong if something is going on that isn't accounted for. I think there are many different ways to use the canon api to decode the .cr2 to raw data - and it needs to be done correctly. I don't know if that applies in your case but it is something to confirm.

If you can see structure in the cr2 image using canon tools - and you don't see it in 16-bit raw converted files - I think that is worth investigating. I guess the canon tools only show you a debayered and colored version - so it's hard to compare the two - but it is definitely important to get the true, raw data from the sensor in order for the pattern noise to calibrate away as well as possible.

Again - it may not be an issue here at all like it was with the 20d. It may be that the pattern isn't reproducible or something. But if it all behaves as fixed pattern noise and the decoding and calibration processes are linear - it should benefit from long exposure in each sub - and the pattern should calibrate away pretty well.

Frank

#49 Jon Rista

Jon Rista

    ISS

  • *****
  • Posts: 24,818
  • Joined: 10 Jan 2014
  • Loc: Colorado

Posted 06 January 2016 - 10:56 PM

If you have the time, you can wade through THIS thread at DPReview about Unity Gain and low ISO's with another dissertation paper being written by Roger. I may not like his processing methods, or his diatribes about proper star colors in the MW, but he does know a thing or two about digital sensors.


Clark knows CANON sensors. He's never actually tested any other camera, so he doesn't know much about the nature of a Sony Exmor sensor, especially as used in a Nikon DSLR. I really wish he would get off his high Canon horse and test some other brands sensors, because I think he would be quite surprised by the results, plus it would be nice to have another resource for sensor statistics across brands besides DXO/Sensorgen and Bill Claff (Bill's data is sometimes difficult to wade through.)

#50 HillTop

HillTop

    Ranger 4

  • -----
  • topic starter
  • Posts: 324
  • Joined: 26 Apr 2014
  • Loc: Belpre, OH

Posted 11 January 2016 - 10:29 AM

Did some testing over the weekend and found that the "alternate DSLR" integration method for PI helps a lot.  This is using a non-calibrated master dark, a bias-only calibrated master flat, and not using the bias during light calibration.  I need to test some more, but I may be able to dump darks altogether - other than hot pixels the darks don't seem to have any more signal than the bias, regardless of exposure time.   




CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.


Recent Topics






Cloudy Nights LLC
Cloudy Nights Sponsor: Astronomics