Here's a brief description of how I began to unlock the spatial filtering algorithm in the D5300 and then to compare it with the Nikon D810A.
Firstly you need to be familiar with the pattern of Red, Green and Blue pixels on a typical DSLR sensor:
Now if we consider the 5x5 block of pixels surrounding a Red pixel we have the following:
The pixels labelled V1 to V8 will all be Red. Let's make the assumption for now that if V0 has an extreme value then its value will be replaced by the spatial filtering algorithm. This is pretty standard stuff - it's how the original star eater algorithm worked. We can test various possible replacement values e.g. the median of the values of the 8 neighbouring red pixels V1-V8 or the maximum of those values. In fact it's the maximum and we can easily prove it - here's how ...
I took a long exposure Nikon D5300 raw dark frame. Then for every single red pixel in the undebayered data, the function V0 - max(V1,V2,V3,V4,V5,V6,V7,V8) was calculated. If the pixel values are truly random (i.e. unaffected by any spatial filtering algorithm) then we should obtain a nice histogram from this. Here it is:
There is a slight spike at the zero point, where V0 - max(V1,V2,V3,V4,V5,V6,V7,V8) = 0, but we can't be certain it's statistically significant, even though we have a full population of 6 million red pixels.
So the next approach is to purposely select the more extreme pixels - I arbitrarily selected all the (unsaturated) red pixels whose value is more than 3 standard deviations above the mean. There were 29500 of them. Here's the histogram of V0 - max(V1,V2,V3,V4,V5,V6,V7,V8) for those pixels:
It's a nice distribution but with a giant spike where V0 - max(V1,V2,V3,V4,V5,V6,V7,V8) = 0. What does this mean? It's impossible for it to arise by chance. Of the 29500 pixels, 5560 of them (i.e. nearly 20%) have a value which matches the maximum of the 8 neighbours. It's the smoking gun - something (i.e. the spatial filtering) has deliberately set those values. What we don't know for certain is exactly what criterion is used to select the offending pixels.
Exactly the same happens for the blue channel - nearly 20% of the 3-sigma blue pixels have a value which matches the maximum of the 8 neighbours.
For the green pixels the story is slightly more complicated. Rather than the G1 and G2 channel being treated separately, the green pixels are treated all together:
Each green pixel is considered to have 12 neighbours: V1 to V12. If the value of a green pixel is considered too extreme by the spatial filtering algorithm then it's value is overwritten by the maximum of those neighbours i.e. max(V1,V2,V3,V4,V5,V6,V7,V8,V9,V10,V11,V12)
So what's different about the D810A? The basic algorithm is the same: extreme red and blue pixels have their value overwritten by the maximum of their 8 neighbours of the same colour and extreme green pixels have their value overwritten by the maximum of their 12 green neighbours.
Just as with the Nikon D5300, I started with a long exposure dark frame. For the full population of 9 million red pixels, here is the histogram of V0 - max(V1,V2,V3,V4,V5,V6,V7,V8):
There is actually the tiniest spike at zero, halfway down the right-hand side of the distribution.
So instead, let's select the (unsaturated) red pixels whose value is more than 3 standard deviations above the mean:
Now the spike is very obvious. There were 13810 red pixels with a value above 3-sigma and only 755 of them have a value which matches the maximum of the 8 neighbours i.e. around 5% of them. It's a similar story in the green and blue channels.
So it appears that for one reason or another, far fewer D810A pixels are being affected by the spatial filtering. The frustrating thing is that we don't yet know the criterion being used to select the offending pixels. It's probably some kind of threshold but how is that threshold set?
In my next post (you must probably wait until this time tomorrow) I will demonstrate that a large number of stars in a D5300 image have their red and blue pixels "zapped" by the algorithm (turning the stars green) but it is apparently not the case in a D810A image.
Mark
Edited by sharkmelley, 10 October 2018 - 03:25 PM.
