Jump to content

  •  

CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.

Photo

OSC Sensor: bayer pattern in darks - Why?

  • Please log in to reply
32 replies to this topic

#1 spokeshave

spokeshave

    Mercury-Atlas

  • *****
  • topic starter
  • Posts: 2,658
  • Joined: 08 Apr 2015

Posted 28 December 2020 - 03:02 PM

I've never given this much thought before until it came up in the lengthy calibration thread. Rather than veer that thread (further) off-topic, I thought I would bring it up here. Someone posted an FFT of a master dark from an OSC camera. The FFT showed obvious regular patterns are present in the dark master. I replied that the FFT is almost certainly showing the bayer pattern. Since I do a lot of OSC imaging, I am used to seeing the familiar checkerboard pattern in undebayered images.

 

Then it occurred to me - darks should be blind to the bayer pattern. After all, the bayer array is an array of color filters over the pixels so there should be no evidence of the bayer pattern in frames exposed to no light. Yet there obviously is. I see it in all of my master darks for my OSC cameras. Darks should only show dark current artifacts and should be agnostic to the CFA, but they're not. So now I'm wondering why. Anyone care to enlighten me?

 

Tim



#2 const

const

    Mariner 2

  • -----
  • Posts: 212
  • Joined: 11 May 2020
  • Loc: Las Vegas, NV

Posted 28 December 2020 - 03:06 PM

Can it be white balance processing somewhere in the pipeline?



#3 Jon Rista

Jon Rista

    ISS

  • *****
  • Posts: 25,366
  • Joined: 10 Jan 2014
  • Loc: Colorado

Posted 28 December 2020 - 03:07 PM

Hmm, I would think the darks should be blind to the bayer pattern, unless somehow the process of actually adding the CFA is changing the response of each pixel to DSNU... (Obviously, it has some kind of impact for PRNU since it is changing which photons are in play.)

 

Do you have a link to the data that was used to create the FFT?


Edited by Jon Rista, 28 December 2020 - 03:07 PM.


#4 spokeshave

spokeshave

    Mercury-Atlas

  • *****
  • topic starter
  • Posts: 2,658
  • Joined: 08 Apr 2015

Posted 28 December 2020 - 03:08 PM

Can it be white balance processing somewhere in the pipeline?

I can see that being a possibility in a DSLR but it doesn't seem likely in a dedicated astro camera. The readout should be true unfiltered raw sensor data.

 

Tim



#5 spokeshave

spokeshave

    Mercury-Atlas

  • *****
  • topic starter
  • Posts: 2,658
  • Joined: 08 Apr 2015

Posted 28 December 2020 - 03:20 PM

Hmm, I would think the darks should be blind to the bayer pattern, unless somehow the process of actually adding the CFA is changing the response of each pixel to DSNU... (Obviously, it has some kind of impact for PRNU since it is changing which photons are in play.)

 

Do you have a link to the data that was used to create the FFT?

It was sharkmelley who posted it - post #315 in this thread. Here is a screenshot of a typical dark with STF applied and zoomed in a bit. 

 

dark.JPG

 

The characteristic checkerboard pattern of the bayer matrix is clearly there.

 

I've been noodling this for the past while and am clueless.

 

Tim


  • CharLakeAstro likes this

#6 rkinnett

rkinnett

    Apollo

  • *****
  • Posts: 1,001
  • Joined: 08 Aug 2018
  • Loc: La Crescenta, CA

Posted 28 December 2020 - 03:26 PM

I'm not sure that's the Bayer pattern.  If it was, and if the matrix did somehow affect the readout then you would see strong correlation in the diagonals, where the green pixels are.  This looks like alternating light/dark values in columns and rows with no strong correlation in the diagonals.


  • wrnchhead likes this

#7 spokeshave

spokeshave

    Mercury-Atlas

  • *****
  • topic starter
  • Posts: 2,658
  • Joined: 08 Apr 2015

Posted 28 December 2020 - 03:39 PM

I'm not sure that's the Bayer pattern.  If it was, and if the matrix did somehow affect the readout then you would see strong correlation in the diagonals, where the green pixels are.  This looks like alternating light/dark values in columns and rows with no strong correlation in the diagonals.

It is typical in an OSC sensor for the red pixels in an undebayered image to appear brighter than the green and blue pixels. That's exactly what I'm seeing even in the darks and what is seen in the snippet I posted.

 

Tim


  • rkinnett likes this

#8 spokeshave

spokeshave

    Mercury-Atlas

  • *****
  • topic starter
  • Posts: 2,658
  • Joined: 08 Apr 2015

Posted 28 December 2020 - 04:33 PM

Well, after over 100 views and no hypotheses, I feel a little bit better. I was thinking that I missed something stupidly obvious.

 

Tim


  • darkstar3d likes this

#9 const

const

    Mariner 2

  • -----
  • Posts: 212
  • Joined: 11 May 2020
  • Loc: Las Vegas, NV

Posted 28 December 2020 - 04:57 PM

I can see that being a possibility in a DSLR but it doesn't seem likely in a dedicated astro camera. The readout should be true unfiltered raw sensor data.

Totally agree. Yet, ZWO has WB knobs in their drivers. Who knows what exactly happens there. The simple guideline is to set them to 50/50 and never touch again. Other settings yield clearly visible tints. I can imagine even "50/50" can leave slight traces visible only after extreme stretching of something like dark noise.

I wish I could get true data our of my sensors (including DSLR), but vendors seem to really want their dirty software in the way at all costs.



#10 sn2006gy

sn2006gy

    Vendor - Rockchuck Summit Observatory

  • ****-
  • Vendors
  • Posts: 1,542
  • Joined: 04 May 2020
  • Loc: Austin, TX

Posted 28 December 2020 - 05:39 PM

Totally agree. Yet, ZWO has WB knobs in their drivers. Who knows what exactly happens there. The simple guideline is to set them to 50/50 and never touch again. Other settings yield clearly visible tints. I can imagine even "50/50" can leave slight traces visible only after extreme stretching of something like dark noise.

I wish I could get true data our of my sensors (including DSLR), but vendors seem to really want their dirty software in the way at all costs.

This was a question I asked in that thread.

 

When I first took darks with my 2600, I used the "native" driver - not the ASCOM driver. The native driver had a weird white balance of like 70/30 or something goofy.   I had problems with dark calibration - but i didn't know if it was the white balance or if it was something else.   I just started NOT dark calibrating and that is what I was asking about in the mentioned thread.  Had i used ascom driver, the white balance is set correctly there. NINA fixed their implementation to set the white balance to fix the native driver instead of assuming defaults (which weren't well documented).

 

To compare not dark calibrated and dark calibrated images again after the calibration thread popped up, I took "fixed" darks (no white balance issue)... compared the end results.. still couldn't see a difference in final image BUT at least darks didn't make it worse smile.gif

 

i swear my bad darks looked like screwed up flats for a while so i went and redid my flats for while and i'm sure they were off until i fixed the WB issue.

 

irks me to no end the white balance seems set in NVRAM by default wrong heh


Edited by sn2006gy, 28 December 2020 - 05:40 PM.

  • jdupton likes this

#11 whwang

whwang

    Skylab

  • *****
  • Posts: 4,243
  • Joined: 20 Mar 2013

Posted 28 December 2020 - 07:55 PM

Take a flat on an object that has white color. If the same pattern shows up, then you can be almost certain that this is some internal WB. To me, this is not something surprising.

#12 Astrojedi

Astrojedi

    Fly Me to the Moon

  • *****
  • Posts: 5,803
  • Joined: 27 May 2015

Posted 28 December 2020 - 07:59 PM

Just FYI... I posted this in the other thread as well. I am not seeing this pattern in my 2600MC neither in the master dark nor in a single dark. In fact I am not seeing the Bayer pattern at all (as it should be). There is something very strange about these darks. Are we sure there is no light leak at play here?



#13 jdupton

jdupton

    Aurora

  • *****
  • Posts: 4,723
  • Joined: 21 Nov 2010
  • Loc: Central Texas, USA

Posted 28 December 2020 - 08:00 PM

Tim,

 

   I may have missed it but which capture software are you using? Are you using the native driver for the camera? Have you had the camera connected to SharpCap or ASICap at any point?

 

   I ask these questions because some software imparts a White Balance setting into the camera. When used with the native driver thereafter, you still get those white balance settings applied. The driver / firmware changes the data read out of the sensor before saving the frame. That means you start out with a modified file that has had adjustments already made to the values the pixels captured. There is no way to recover the original data. 

 

   You would need to connect the camera to one of the (? offending ? ) programs again and set the White Balance settings back to 50 / 50 in order to remedy the issue.

 

   Byron mentioned this indirectly in his Post #10 and it was also mentioned by user const in Post #2 & 9.

 

 

EDIT: Oops, I notice now that you have a QHY Camera rather than ZWO. I am not sure this setting applies to both vendors or is an underlying setting that can be made with either camera brand. See the thread "ZWO 294MC Pro Broken Red/Blue?" for one of many examples here on CN. I don't recall the sensors on which this has been observed to happen.

 

 

John


Edited by jdupton, 28 December 2020 - 08:18 PM.

  • psandelle likes this

#14 Coconuts

Coconuts

    Viking 1

  • *****
  • Posts: 742
  • Joined: 23 Sep 2012

Posted 28 December 2020 - 08:04 PM

Needless white balance knobs that shouldn't exist to begin with makes the most sense, with a small light leak a strong second, but just to get a bit more out there, could cosmic ray or secondary muons interact with camera parts (including the lens cap) so as to create a very faint visible light flux that the camera is recording?  We are really getting down into the few photons region here, and if it were remotely real, professional astronomers would know about it.

 

All the best,

 

Kevin



#15 spokeshave

spokeshave

    Mercury-Atlas

  • *****
  • topic starter
  • Posts: 2,658
  • Joined: 08 Apr 2015

Posted 28 December 2020 - 08:25 PM

Tim,

 

   I may have missed it but which capture software are you using? Are you using the native driver for the camera? Have you had the camera connected to SharpCap or ASICap at any point?

 

   I ask these questions because some software imparts a White Balance setting into the camera. When used with the native driver thereafter, you still get those white balance settings applied. The driver / firmware changes the data read out of the sensor before saving the frame. That means you start out with a modified file that has had adjustments already made to the values the pixels captured. There is no way to recover the original data. 

 

   You would need to connect the camera to one of the (? offending ? ) programs again and set the White Balance settings back to 50 / 50 in order to remedy the issue.

 

   Byron mentioned this indirectly in his Post #10 and it was also mentioned by user const in Post #2 & 9.

 

 

EDIT: Oops, I notice now that you have a QHY Camera rather than ZWO. I am not sure this setting applies to both vendors or is an underlying setting that can be made with either camera brand.

 

 

John

I use SGP and do not use a native driver - ASCOM only. The ASCOM driver should only read out raw data from the camera. I also have a new ASI2400MC Pro that I just got and haven't had a chance to use yet. I just took a dark and the pattern is there for that camera too. The driver interface offers no options for white balance. I have also used several OSC cameras over the years and the characteristic checkerboard pattern has been present in all of them.

 

Edit to add that there is absolutely no potential for light leaks. Darks are taken at night in a closed dark observatory with the lens cap on. Darks calibrate lights perfectly. This is not a light leak artifact. 

 

Tim


Edited by spokeshave, 28 December 2020 - 08:26 PM.


#16 sharkmelley

sharkmelley

    Fly Me to the Moon

  • *****
  • Posts: 5,813
  • Joined: 19 Feb 2013
  • Loc: UK

Posted 28 December 2020 - 08:35 PM

It was sharkmelley who posted it - post #315 in this thread. Here is a screenshot of a typical dark with STF applied and zoomed in a bit. 

 

attachicon.gifdark.JPG

 

The characteristic checkerboard pattern of the bayer matrix is clearly there.

The darks I was using were ASI2600MC PRO darks uploaded by CN member sn2006gy:

https://www.cloudyni...-it/?p=10762295

 

There was no obvious bayer checkerboard  in the master dark with an STF applied:

 

MasterDarkCrop.png

 

However when the raw was split into 4 channels, one of the channels showed a weird diagonal pattern in the FFT:

 

ASI2600MCPRO_DFT.jpg

 

Edit: The explanation for the diagonal stripes was later discovered to be two bright pixels.  See here.

 

Mark


Edited by sharkmelley, 29 December 2020 - 05:36 AM.


#17 spokeshave

spokeshave

    Mercury-Atlas

  • *****
  • topic starter
  • Posts: 2,658
  • Joined: 08 Apr 2015

Posted 28 December 2020 - 08:54 PM

The darks I was using were ASI2600MC PRO darks uploaded by CN member sn2006gy:

https://www.cloudyni...-it/?p=10762295

 

There was no obvious bayer checkerboard  in the master dark with an STF applied:

 

attachicon.gifMasterDarkCrop.png

 

However when the raw was split into 4 channels, one of the channels showed a weird diagonal pattern in the FFT:

 

attachicon.gifASI2600MCPRO_DFT.jpg

 

Mark

Forgive my ignorance, but what do you mean when you say that the image was split into 4 channels?

 

Thanks.

 

Tim



#18 Jon Rista

Jon Rista

    ISS

  • *****
  • Posts: 25,366
  • Joined: 10 Jan 2014
  • Loc: Colorado

Posted 28 December 2020 - 09:18 PM

It was sharkmelley who posted it - post #315 in this thread. Here is a screenshot of a typical dark with STF applied and zoomed in a bit. 

 

attachicon.gifdark.JPG

 

The characteristic checkerboard pattern of the bayer matrix is clearly there.

 

I've been noodling this for the past while and am clueless.

 

Tim

Are we sure that is a dark? Is there any chance there was a small amount of light signal in it? 



#19 spokeshave

spokeshave

    Mercury-Atlas

  • *****
  • topic starter
  • Posts: 2,658
  • Joined: 08 Apr 2015

Posted 28 December 2020 - 09:19 PM

Are we sure that is a dark? Is there any chance there was a small amount of light signal in it? 

Not a chance.

 

Tim



#20 Jon Rista

Jon Rista

    ISS

  • *****
  • Posts: 25,366
  • Joined: 10 Jan 2014
  • Loc: Colorado

Posted 28 December 2020 - 09:20 PM

This was a question I asked in that thread.

 

When I first took darks with my 2600, I used the "native" driver - not the ASCOM driver. The native driver had a weird white balance of like 70/30 or something goofy.   I had problems with dark calibration - but i didn't know if it was the white balance or if it was something else.   I just started NOT dark calibrating and that is what I was asking about in the mentioned thread.  Had i used ascom driver, the white balance is set correctly there. NINA fixed their implementation to set the white balance to fix the native driver instead of assuming defaults (which weren't well documented).

 

To compare not dark calibrated and dark calibrated images again after the calibration thread popped up, I took "fixed" darks (no white balance issue)... compared the end results.. still couldn't see a difference in final image BUT at least darks didn't make it worse smile.gif

 

i swear my bad darks looked like screwed up flats for a while so i went and redid my flats for while and i'm sure they were off until i fixed the WB issue.

 

irks me to no end the white balance seems set in NVRAM by default wrong heh

White balance should play no role in calibration. Calibration, by necessity, must be done on completely RAW data, no processing at all. White balance is something usually performed during demosaicing, so it should have no bearing on the raw data calibration. If there is a white balance setting in the driver, I would really question the design of the camera, its driver, or maybe even the sensor itself...dark signal shouldn't care what color the pixel is. 



#21 Jon Rista

Jon Rista

    ISS

  • *****
  • Posts: 25,366
  • Joined: 10 Jan 2014
  • Loc: Colorado

Posted 28 December 2020 - 09:23 PM

The darks I was using were ASI2600MC PRO darks uploaded by CN member sn2006gy:

https://www.cloudyni...-it/?p=10762295

 

There was no obvious bayer checkerboard  in the master dark with an STF applied:

 

attachicon.gifMasterDarkCrop.png

 

However when the raw was split into 4 channels, one of the channels showed a weird diagonal pattern in the FFT:

 

attachicon.gifASI2600MCPRO_DFT.jpg

 

Mark

So, there is no obvious checkerboard...however, there is DEFINITELY structure in there...I see some kind of horizontal correlation or something going on, often between more than 2 pixels, which is interesting (if it was just 2 pixels, I could possibly understand that if the pixel architecture was a 2-shared (2x1) or 4-shared (2x2 pixels sharing one set of readout circuitry) design). I suspect that is what your FFT is picking up on. This is not the first time I've seen horizontal correlation between pixels, but usually it is just a couple of pixels. 



#22 jdupton

jdupton

    Aurora

  • *****
  • Posts: 4,723
  • Joined: 21 Nov 2010
  • Loc: Central Texas, USA

Posted 28 December 2020 - 09:31 PM

Jon,

 

   For more information, refer to the following two threads.

 

https://www.cloudynights.com/topic/708880-asi533mc-pro-weird-color-balance/

 

https://indilib.org/forum/ccds-dslrs/6368-strange-bias-frames-with-asi294mc-pro-under-astroberry.html#51097

 

   In these cases, it was found that the device driver was, in fact, modifying the data from the sensor before writing the FITs files. That is part of what makes it so insidious. Raw data read out by the images capture application is not really "raw data" any longer.

 

   However, in all the cases I have followed, the ASCOM Driver always sets the parameters equal so that raw data is really raw data. Tim has confirmed that he was only using the ASCOM drivers so there may be something else at play here.

 

 

John



#23 Jon Rista

Jon Rista

    ISS

  • *****
  • Posts: 25,366
  • Joined: 10 Jan 2014
  • Loc: Colorado

Posted 28 December 2020 - 09:47 PM

Jon,

 

   For more information, refer to the following two threads.

 

https://www.cloudynights.com/topic/708880-asi533mc-pro-weird-color-balance/

 

https://indilib.org/forum/ccds-dslrs/6368-strange-bias-frames-with-asi294mc-pro-under-astroberry.html#51097

 

   In these cases, it was found that the device driver was, in fact, modifying the data from the sensor before writing the FITs files. That is part of what makes it so insidious. Raw data read out by the images capture application is not really "raw data" any longer.

 

   However, in all the cases I have followed, the ASCOM Driver always sets the parameters equal so that raw data is really raw data. Tim has confirmed that he was only using the ASCOM drivers so there may be something else at play here.

 

 

John

Wow...that's terrible. The driver should definitely NOT be messing with the raw sensor data... 



#24 RazvanUnderStars

RazvanUnderStars

    Surveyor 1

  • *****
  • Posts: 1,732
  • Joined: 15 Jul 2014
  • Loc: Ontario, Canada

Posted 28 December 2020 - 10:14 PM

Strange, I had to check my darks as well (I have a ZWO ASI2600MC).

 

As I was looking at them, I  noticed that at some zoom levels there is a similar MoirĂ© pattern being displayed, but at 1:1 zoom they go away. What's left is a low-frequency pattern (probably the FPN).

 

For comparison, I'm attaching center crops of the DFT magnitude images at 1:1, with STF applied.

 

 

Red:

 

r.jpg

 

Green:

 

g.jpg

 

Blue:

 

b.jpg

 



#25 sharkmelley

sharkmelley

    Fly Me to the Moon

  • *****
  • Posts: 5,813
  • Joined: 19 Feb 2013
  • Loc: UK

Posted 29 December 2020 - 03:03 AM

Forgive my ignorance, but what do you mean when you say that the image was split into 4 channels?

It's what PixInsight's SplitCFA does.  I would normally label the result Red, Green1, Green2, Blue but I'm not sure which channel is which in the ASI2600MC PRO files.

 

Mark




CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.


Recent Topics






Cloudy Nights LLC
Cloudy Nights Sponsor: Astronomics