32 replies to this topic

[sharkmelley]

So, there is no obvious checkerboard...however, there is DEFINITELY structure in there...I see some kind of horizontal correlation or something going on, often between more than 2 pixels, which is interesting (if it was just 2 pixels, I could possibly understand that if the pixel architecture was a 2-shared (2x1) or 4-shared (2x2 pixels sharing one set of readout circuitry) design). I suspect that is what your FFT is picking up on. This is not the first time I've seen horizontal correlation between pixels, but usually it is just a couple of pixels. 

Yes there is some structure - well spotted! 

 

If I perform a PixInsight SplitCFA on sn2006gy's master dark and measure the standard deviations in each channel I see the following:

 

Channel  stdDev

CFA0      17.8 
CFA1      17.3
CFA2       6.4
CFA3       8.1

 

This clearly shows channel scaling.  But there are no histogram gaps or histogram spikes of the type we see on Nikon cameras with white balance pre-conditioning, so it could indicate that this channel scaling is applied before digitization.

 

CFA1 is the channel with the weird diagonal stripes in the FFT, so given CFA0 and CFA1 have very similar stdDevs we can safely deduce that the "weird" channel is one of the 2 green channels.

 

Mark

Edited by sharkmelley, 29 December 2020 - 03:59 AM.

[sharkmelley]

With much embarrassment I can now confidently reveal the cause of the diagonal pattern in the 2D FFT:

 

 

It was caused by two exceptionally bright pixels in the master dark.  When I capped the value of those two pixels, the diagonal pattern disappeared!

 

It should have been obvious to me because the 2D FFT of two bright dots is a wave pattern.   So two very bright pixels in any image will cause a superimposed wave pattern in the 2D FFT.

 

I will now scurry back to the dark hole from which I emerged!

 

Mark

Edited by sharkmelley, 29 December 2020 - 05:09 AM.

[sn2006gy]

White balance should play no role in calibration. Calibration, by necessity, must be done on completely RAW data, no processing at all. White balance is something usually performed during demosaicing, so it should have no bearing on the raw data calibration. If there is a white balance setting in the driver, I would really question the design of the camera, its driver, or maybe even the sensor itself...dark signal shouldn't care what color the pixel is. 

A few of us noticed our 533 and 2600s were having really red subs or really blue flats and such. We dug in and noticed that the factory WB for these sensors stored in what appears to be NVRAM is incorrect and using ASCOM drivers set it to 50/50 but if the developers implemented native OS drivers, those did not correctly set the white balance.

 

I know NINA rolled a new release to fix this and I believe other tools did the same too.

 

To this day, I don't know why these cameras still ship like this or why the native drive isn't hard coded to correct white balance by default.

 

In our great dark calibration debate, I did ask if screwed up white balance darks were the reason my darks seem to calibrate my images like they were bad flats... i don't really know how that would be possible but when i thought back on what changed/fixed over the time its the only thing that came to mind.

[sn2006gy]

With much embarrassment I can now confidently reveal the cause of the diagonal pattern in the 2D FFT:

 

DiagonalStripeFFT.jpg

 

It was caused by two exceptionally bright pixels in the master dark.  When I capped the value of those two pixels, the diagonal pattern disappeared!

 

It should have been obvious to me because the 2D FFT of two bright dots is a wave pattern.   So two very bright pixels in any image will cause a superimposed wave pattern in the 2D FFT.

 

I will now scurry back to the dark hole from which I emerged!

 

Mark

haha, screw those bright pixels!

 

and thanks for such detailed analysis Mark!

[RazvanUnderStars]

I'm curious, what was the relative orientation of the two bright pixels compared to the orientation of the diagonal stripes in the FFT?

 

With much embarrassment I can now confidently reveal the cause of the diagonal pattern in the 2D FFT:

 

It was caused by two exceptionally bright pixels in the master dark.  When I capped the value of those two pixels, the diagonal pattern disappeared!

 

It should have been obvious to me because the 2D FFT of two bright dots is a wave pattern.   So two very bright pixels in any image will cause a superimposed wave pattern in the 2D FFT.

 

 

[sharkmelley]

I'm curious, what was the relative orientation of the two bright pixels compared to the orientation of the diagonal stripes in the FFT?

The diagonal line joining the two bright pixels is orthogonal to the direction of the diagonal stripes.

 

Mark

[Jon Rista]

A few of us noticed our 533 and 2600s were having really red subs or really blue flats and such. We dug in and noticed that the factory WB for these sensors stored in what appears to be NVRAM is incorrect and using ASCOM drivers set it to 50/50 but if the developers implemented native OS drivers, those did not correctly set the white balance.

 

I know NINA rolled a new release to fix this and I believe other tools did the same too.

 

To this day, I don't know why these cameras still ship like this or why the native drive isn't hard coded to correct white balance by default.

 

In our great dark calibration debate, I did ask if screwed up white balance darks were the reason my darks seem to calibrate my images like they were bad flats... i don't really know how that would be possible but when i thought back on what changed/fixed over the time its the only thing that came to mind.

Its terrible if they are screwing with the raw data for you... These cameras should definitely be fixed to resolve that issue. The raw data should be truly raw and unprocessed.

 

As for darks...I guess it would depend on exactly what this white balancing is doing...if it does exactly the same thing in exactly the same way to every frame, and you are calibrating before demosaicing, then I would expect calibration would work. I would have to actually play with a set of darks, biases, and lights to say for sure though. 

[sn2006gy]

Its terrible if they are screwing with the raw data for you... These cameras should definitely be fixed to resolve that issue. The raw data should be truly raw and unprocessed.

 

As for darks...I guess it would depend on exactly what this white balancing is doing...if it does exactly the same thing in exactly the same way to every frame, and you are calibrating before demosaicing, then I would expect calibration would work. I would have to actually play with a set of darks, biases, and lights to say for sure though. 

Here is the topic:

 

https://www.cloudyni...e#entry10210336

 

The white balance was so off, my images were blood red in HA... Kind of cool at first, but obviously something was wrong.