168 replies to this topic

[erictheastrojunkie]

Ya there's been quite the discussion on the pdaf striping from the z7 sensor, it seems the culprit appears to be an overcorrection of the pdaf striping that's common to the Sony cameras. It can be corrected in rawtherapy, I'll have to do some testing to see how it'll affect my nightscape imaging. Real world examples are very very inconsistent on their appearance and affect on images, it's dependent on a number of lighting factors and it seems like even some shutter settings. It's a strange phenomenon that Jim and Horschak on DPR have put a lot of testing time into, it seems like it should be correctable by firmware updates, though that remains to be seen with Nikon's historical reputation in the FW update department. They have pushed two FW updates for the Z7 already, however, which is a good sign.

Sent from my Pixel 2 XL using Tapatalk

[erictheastrojunkie]

Yes please, non-bulb for the 30sec exposure. 

 

 

I've added the 30sec ISO 1600 and 2 minute ISO 1600 exposures to the folder, here's the link:

https://drive.google...ee-ZXN3bRS1XAY0

[sharkmelley]

I now have sufficient data for the Z7 

 

It's now pretty obvious that Z7 spatial filtering is not using a threshold based on the 8 or 16 same colour neighbours but it is using either the 16 or 24 neighbours of all colours.  This is really good news because it means there will be virtually no star damage.

 

Here's the plot using 24 neighbours:

 

 

The structure is not as clear as the plots for most of the other cameras.  This is because the sensor is a very clean sensor i.e. the dark current in each pixel behaves quite uniformly with very few of the outlying values that test the boundaries of the algorithm.  There was the same "problem" with the Nikon D850 (believed to be a very similar sensor) which Bernard Delley and myself discuss here:

https://www.dpreview...s/post/61879106

 

Bernard had to use very high ISOs or very long exposures before the structure resulting from the algorithm became clear.  I'm happy to have another go at this if you provide one

 

Mark

Edited by sharkmelley, 28 November 2018 - 03:11 PM.

[erictheastrojunkie]

Good to see, I can certainly provide excessively long exposures or very high ISO exposures, just need to know the settings (10 minutes? 25000+ ISO?). Thanks fir all your work on this stuff!

Sent from my Pixel 2 XL using Tapatalk

[sharkmelley]

Good to see, I can certainly provide excessively long exposures or very high ISO exposures, just need to know the settings (10 minutes? 25000+ ISO?). Thanks fir all your work on this stuff!

Sent from my Pixel 2 XL using Tapatalk

Let's try 5minutes at ISO 1600.  Thanks.

 

Mark

[Jimmy462]

>snip<

 

I've put the original files here if you want to download them:

https://drive.google...EsQ?usp=sharing

 

>snip<

 

Mark

Hi Mark,

 

Fabulous work you've been doing here on this issue, if you're interested I can provide some RAW files from an a6300, a7R and a7sII.

 

In regards to your linked files...for whatever reason my browser is not allowing me access to the Folder level of google drive but I am able to access and download the individually shared image files as posted by SandyHouTex. Any chance you can provide the direct links to the images in your examples?

 

Jimmy G

[sharkmelley]

I have been sent some darks from the Nikon Z6.

 

Similar to the Nikon Z7 it's clear that the Z6 spatial filtering is not using a threshold based on the 8 or 16 same colour neighbours but it is using either the 16 or 24 neighbours of all colours.  This is really good news because it means there will be virtually no star damage.

 

Here's the plot using 24 neighbours:

 

 

The structure in this plot is much clearer than the Z7 plot a few posts ago because the exposure time was longer.  Because of this it is not possible to say for certain at the present time whether or not the Z6 and Z7 are using exactly the same spatial filtering algorithm.

 

Mark

[sharkmelley]

The spatial filtering on the Nikon D750 is very odd.  It's has taken me a while to work out what's going on.  As usual, in the following chart I've plotted every pixel in the raw data against the maximum value found in the 24 neighbouring pixels in the 5x5 surrounding block.  But I've colour coded them according to whether the central pixel is red, green or blue:

 

 

You can immediately see that no green pixel is brighter than its neighbours. On the other hand there are a significant number of red and blue pixels that are significantly brighter than their neighbours. This is purely a result of the spatial filtering algorithm preferentially capping the values of green pixels.

 

In the context of a small tightly focused star this would appear to mean that on average the red and blue pixels will tend to dominate, possibly leading to a magenta/pink star colour. 

 

I would therefore to interested to know if folk have noticed magenta/pink stars on the D750 when the optics are pin sharp.

 

Mark

Edited by sharkmelley, 11 January 2019 - 06:09 PM.

[sharkmelley]

The spatial filtering on the Nikon D750 is very odd.  It's has taken me a while to work out what's going on.  As usual, in the following chart I've plotted every pixel in the raw data against the maximum value found in the 24 neighbouring pixels in the 5x5 surrounding block.  But I've colour coded them according to whether the central pixel is red, green or blue:

 

NikonD750_m24.png

 

You can immediately see that no green pixel is brighter than its neighbours. On the other hand there are a significant number of red and blue pixels that are significantly brighter than their neighbours. This is purely a result of the spatial filtering algorithm preferentially capping the values of green pixels.

 

In the context of a small tightly focused star this would appear to mean that on average the red and blue pixels will tend to dominate, possibly leading to a magenta/pink star colour. 

 

I would therefore to interested to know if folk have noticed magenta/pink stars on the D750 when the optics are pin sharp.

 

Mark

 

It's confirmed.  The Nikon D750 does suffer from "pink star" syndrome. 

 

You can find some details here:

https://www.dpreview...s/post/62164367

 

The same pink star effect on the Nikon D750 can be seen in Alan Dyer's comparison - look at the "De-bayering Star Artifacts" section at the bottom of this article:

 

https://www.diyphoto...trophotography/

 

If you are worried about the colour of small tightly focused stars then the Nikon D750 certainly looks like a camera to avoid.

 

Mark

Edited by sharkmelley, 12 January 2019 - 10:04 AM.

[SandyHouTex]

It's confirmed.  The Nikon D750 does suffer from "pink star" syndrome. 

 

You can find some details here:

https://www.dpreview...s/post/62164367

 

The same pink star effect on the Nikon D750 can be seen in Alan Dyer's comparison - look at the "De-bayering Star Artifacts" section at the bottom of this article:

 

https://www.diyphoto...trophotography/

 

If you are worried about the colour of small tightly focused stars then the Nikon D750 certainly looks like a camera to avoid.

 

Mark

Thanks for looking into that.  I was wondering about it since you did the initial analysis a while back.  It makes me wonder why Nikon would do something like that.  And like you said in the dpreview thread, Nikon seemed to have done a great job with the D7000.

Edited by SandyHouTex, 12 January 2019 - 01:14 PM.

[sharkmelley]

Thanks for looking into that.  I was wondering about it since you did the initial analysis a while back.  It makes me wonder why Nikon would do something like that.

I needed to forget about it for a while!  It was the recent DPReview thread showing pink stars that suddenly enabled me to join the dots. 

 

You wonder why Nikon did that?  You and me both!

 

Mark

[whwang]

One possibility is that the firmwares for different Nikon cameras are maintained/designed by multiple engineers.  So despite the overall similar algorithms (inherited from the earlier cameras), the details vary from camera to camera, depending on the characteristics of the sensors and the taste of each individual engineer.  

 

Also, D750 is supposed to be a low-light camera of its generation. To make images look cleaner and to perform better in lab tests, it is possible for the engineers to "cheat" more.

 

All these are just my speculations.  D750 is an old camera anyway.  Time to move on.  Z6 is the new low-light king.

[SandyHouTex]

One possibility is that the firmwares for different Nikon cameras are maintained/designed by multiple engineers.  So despite the overall similar algorithms (inherited from the earlier cameras), the details vary from camera to camera, depending on the characteristics of the sensors and the taste of each individual engineer.  

 

Also, D750 is supposed to be a low-light camera of its generation. To make images look cleaner and to perform better in lab tests, it is possible for the engineers to "cheat" more.

 

All these are just my speculations.  D750 is an old camera anyway.  Time to move on.  Z6 is the new low-light king.

What you say is true, but if I bought a mirrorless it would have to be the Canon.  That is unless you want to clean your sensor every week or so.  It’s the only one that has a cover that goes over the sensor when not in use.  That’s also probably why I’ll stick with DSLRs.  Unless you’re actually taking a picture, the mirror protects the sensor by being closed, and if you’re not using live view, which I don’t but briefly to focus, the sensor isn’t constantly on increasing the temperature and noise.

[fmeschia]

Unless you’re actually taking a picture, the mirror protects the sensor by being closed, and if you’re not using live view, which I don’t but briefly to focus, the sensor isn’t constantly on increasing the temperature and noise.

Probably off-topic, but since I’m also interested in mirrorless technology, let me ask: don’t mirrorless cameras have focal plane shutters? That would protect the sensor.
Francesco

[whwang]

Probably off-topic, but since I’m also interested in mirrorless technology, let me ask: don’t mirrorless cameras have focal plane shutters? That would protect the sensor.

Most of them do have focal plane shutters.

[17.5Dob]

What you say is true, but if I bought a mirrorless it would have to be the Canon.  That is unless you want to clean your sensor every week or so.  It’s the only one that has a cover that goes over the sensor when not in use.

Do you always take you lens' off , not put a lens cap in it's place and leave your cameras sitting around facing straight up when not in use 
As far as mirrors "protecting" the sensor, you've obviously forgotten the Nikon D600 debacle.......

On any given night when when I'm shooting AP's, my sensor is always wide open and facing "straight up" for hours on end. You wouldn't believe the amount of flocking bits from my APO's that get shaken loose every night by the vibration of a shutter slap, and end up on my sensor.....
 

My camera's sensors get dirty by actually shooting them, and they get filthy, and need cleaning every week.....or less...... Not while storing them on a shelf.

Ever taken a photo trip to Utah .....?
 

Edited by 17.5Dob, 12 January 2019 - 08:34 PM.

[SandyHouTex]

Do you always take you lens' off , not put a lens cap in it's place and leave your cameras sitting around facing straight up when not in use 
As far as mirrors "protecting" the sensor, you've obviously forgotten the Nikon D600 debacle.......

On any given night when when I'm shooting AP's, my sensor is always wide open and facing "straight up" for hours on end. You wouldn't believe the amount of flocking bits from my APO's that get shaken loose every night by the vibration of a shutter slap, and end up on my sensor.....
 

My camera's sensors get dirty by actually shooting them, and they get filthy, and need cleaning every week.....or less...... Not while storing them on a shelf.

Ever taken a photo trip to Utah .....?
 

Actually I have no problem cleaning lenses.  If I put a little scratch or smudge on it, no problem.  Cleaning a sensor with 6.54 um pixels (I have a Canon 6D), is a whole different ball game.  A scratch or smudge is going to wreck a lot of pixels and will be visible in the final image.  And yes I know the sensor is actually covered with a piece of glass which is what you clean, but the pixels are directly beneath it and would be affected.

Edited by SandyHouTex, 12 January 2019 - 10:36 PM.

[SandyHouTex]

Probably off-topic, but since I’m also interested in mirrorless technology, let me ask: don’t mirrorless cameras have focal plane shutters? That would protect the sensor.
Francesco

They do have mechanical shutters, but when the camera is powered on, the shutter is open.  That’s because the sensor is what the Electronic Viewfinder displays and “live view” uses.  I’m not sure exactly when it closes or why.  Probably when you press the button to take a picture, and then it opens again.  If you use the electronic shutter, which some cameras have, it never closes.

[terry59]

Awesome analysis folks....but what does all of this mean to the majority of imagers? Since I see none of that in my calibrated data or processed images with the Nikon D5300/D5500 can you explain the real world ramification?

 

Thanks

 

Edited by terry59, 13 January 2019 - 09:17 AM.

[sharkmelley]

Awesome analysis folks....but what does all of this mean to the majority of imagers? Since I see none of that in my calibrated data or processed images with the Nikon D5300/D5500 can you explain the real world ramification?

 

 

If you don't see it in your own images then don't worry about it.  It's as simple as that!

 

It's probably the case that the majority of astro-imagers won't notice it either.

 

Mark

Edited by sharkmelley, 13 January 2019 - 10:16 AM.

[terry59]

If you don't see it in your own images then don't worry about it.  It's as simple as that!

 

It's probably the case that the majority of astro-imagers won't notice it either.

 

Mark

Hi Mark,

 

  Thanks for the response. So is this mainly an engineering exercise or is this something people should be concerned about when making a camera choice? The lines get blurred on here a lot

[whwang]

It depends on how sharp your optics is, how good the seeing is, and how critical you are.  As Mark has shown, there already exist real astronomical images that show the damage of stars made by the camera.  So this is not just something that only concerns engineers.  It has real effect if one's goal is to pursue the highest image quality.

[terry59]

It depends on how sharp your optics is, how good the seeing is, and how critical you are.  As Mark has shown, there already exist real astronomical images that show the damage of stars made by the camera.  So this is not just something that only concerns engineers.  It has real effect if one's goal is to pursue the highest image quality.

Hi Wei-Hao,

 

  Lol....of course not, but I see the beginning of "Nikon is bad...look at this" already in this thread and the sample size I see at this point is one. I don't wish to be perceived as bashing either...I am looking for more data or more examples of the problem before condemning any of these cameras

 

 

[whwang]

Sorry, Terry.  I constantly skip certain people's posts.  So perhaps I missed the "Nikon is bad" part.  Just based on Mark's reports, I would only reach the conclusion that D750 (and maybe one or two other models from Nikon) is bad.  It appears to me that most other Nikon cameras are just fine, including the D8XX(A) series and the Z series.

 

If there is a brand that's really bad in terms of star-eating, it's Sony, not Nikon.

[sharkmelley]

Hi Wei-Hao,

 

  Lol....of course not, but I see the beginning of "Nikon is bad...look at this" already in this thread and the sample size I see at this point is one. I don't wish to be perceived as bashing either...I am looking for more data or more examples of the problem before condemning any of these cameras

 

My motivation is simply to provide folk with the information I wish I had available when I was making my own camera choice.

 

For me, I would not have chosen the D5300 if I had known about this "green stars" issue in advance.  I would have chosen an alternative Nikon model.

 

Why does this thread concentrates on Nikon?  It's because Sony cameras have a worse spatial filtering issue but it's already well understood and Canon cameras don't appear to have the issue at all unless noise reduction is deliberately switch on.

 

I think most users of this forum are intelligent enough to not interpret my actions as Nikon bashing.

 

Mark

 

P.S. Why did you feel the need to use the "popcorn eating" emoji?

Edited by sharkmelley, 13 January 2019 - 12:53 PM.