Jump to content

  •  

CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.

Photo

AA filter, spatial filter and star colours

  • Please log in to reply
164 replies to this topic

#126 Jawknee

Jawknee

    Lift Off

  • -----
  • Posts: 16
  • Joined: 11 May 2019

Posted 11 May 2019 - 01:12 AM

Hi, I came across this topic while searching for an upgrade to my d5200. I am considering the d5600 and d7500. While I don't think this topic is really of use for an amateur like me, it was a very interesting read non the less. I saw that you don't have any data on the d5200. If ever it still interest you to acquire data on other nikons, here's a 176 seconds/iso1600 dark frame at room temperature. 

 

https://1drv.ms/u/s!...WYFAKMcp4Y9itr_



#127 sharkmelley

sharkmelley

    Aurora

  • *****
  • topic starter
  • Posts: 4,624
  • Joined: 19 Feb 2013
  • Loc: UK

Posted 12 May 2019 - 12:55 PM

Hi, I came across this topic while searching for an upgrade to my d5200. I am considering the d5600 and d7500. While I don't think this topic is really of use for an amateur like me, it was a very interesting read non the less. I saw that you don't have any data on the d5200. If ever it still interest you to acquire data on other nikons, here's a 176 seconds/iso1600 dark frame at room temperature. 

 

https://1drv.ms/u/s!...WYFAKMcp4Y9itr_

Thanks for the D5200 dark frame.  It shows that the D5200 is using a crude version of spatial filtering and also that significant black level clipping at the value of 0 is taking place. 

Here is the graph:

 

NikonD5200.png

 

On this graph the sensor pixels are red or blue are plotted against the max of their 8 neighbours and the sensor green pixels are plotted against the max of their 12 neighbours.  It can immediately be seen that pixels whose values are greater than the maximum of their neighbours (either 8 or 12 neighbours) have their value clipped to that maximum.  Also, because the green pixels are compared to 12 neighbours and the red/blue pixels to 8 neighbours it means that in a tightly focused star the red and blue pixels will be cclipped more often thean the green, leading to greenish stars.

 

Mark



#128 Jawknee

Jawknee

    Lift Off

  • -----
  • Posts: 16
  • Joined: 11 May 2019

Posted 12 May 2019 - 06:16 PM

It's facinating what you can see with only a dark frame!



#129 sharkmelley

sharkmelley

    Aurora

  • *****
  • topic starter
  • Posts: 4,624
  • Joined: 19 Feb 2013
  • Loc: UK

Posted 27 November 2019 - 11:38 AM

Here is the result for the Nikon D3400 with thanks to contributor mralpaca who made the dark frame available:

 

nikon_D3400_5min_iso1600_dark.png

 

On this plot the red and blue sensor pixels are plotted against the max of their 8 neighbours and the green pixels are plotted against the max of their 12 neighbours.  It can immediately be seen that no pixel is allowed to be more than twice the value of its neighbours of the same colour.  Some kind of clipping is taking place.

 

The plot is almost identical to the Nikon D5300 and D5500 analysed in this thread: https://www.cloudyni...atial-filtering

 

Dark subtraction will be ineffective and there will be a tendency to turn tightly focused unsaturated stars to green.

 

Mark



#130 Michael Covington

Michael Covington

    Author

  • *****
  • Posts: 6,790
  • Joined: 13 May 2014
  • Loc: Athens, Georgia, USA

Posted 22 December 2019 - 08:45 PM

One practical point: The Nikon spatial filtering makes the sensor show less read noise when tested in the usual way.  Right?

This may be why Nikon sensors seem to show only half as much read noise as competing Canon sensors.



#131 whwang

whwang

    Soyuz

  • *****
  • Posts: 3,557
  • Joined: 20 Mar 2013

Posted 22 December 2019 - 08:57 PM

I think this filtering mainly works on hot pixels.  It's hard to imagine it has any significant effect on read noise.  Plus, other Sony-inside cameras that are not known to perform this kind of filtering (hacked Nikon, Pentax, Sony without star eater activated, etc) all show very low read noise according to Bill Claff's tests and my own measurements on some hacked Nikon.

 

Cheers,

Wei-Hao


Edited by whwang, 22 December 2019 - 09:13 PM.


#132 fmeschia

fmeschia

    Surveyor 1

  • *****
  • Posts: 1,562
  • Joined: 20 May 2016
  • Loc: Mountain View, CA

Posted 22 December 2019 - 09:11 PM

One practical point: The Nikon spatial filtering makes the sensor show less read noise when tested in the usual way.  Right?

This may be why Nikon sensors seem to show only half as much read noise as competing Canon sensors.

I don’t think this can do anything about read noise. It seems to focus on hot outliers.

Francesco



#133 Michael Covington

Michael Covington

    Author

  • *****
  • Posts: 6,790
  • Joined: 13 May 2014
  • Loc: Athens, Georgia, USA

Posted 22 December 2019 - 10:06 PM

I don’t think this can do anything about read noise. It seems to focus on hot outliers.

Francesco

I am wondering.  I am thinking not about actual read noise in a picture, but readings on tests intended to measure it.  Should we continue believing that the Nikon D5300 has only half as much read noise as the Canon 200D?  Or are the tests thrown off by Nikon's postprocessing?

 

Suppose the Nikon clamps every pixel at 2 x the value of the highest of the 8 nearest pixels of the same color.  Many measurements involve short-exposure dark frames.  If the value of a pixel is very close to zero, it wouldn't take much read noise to elevate it above 2 x the maximum of its neighbors -- would it?  So the postprocessing could cut noise in such a way as to affect measurements.


Edited by Michael Covington, 22 December 2019 - 10:33 PM.


#134 fmeschia

fmeschia

    Surveyor 1

  • *****
  • Posts: 1,562
  • Joined: 20 May 2016
  • Loc: Mountain View, CA

Posted 23 December 2019 - 12:00 AM

I am wondering. I am thinking not about actual read noise in a picture, but readings on tests intended to measure it. Should we continue believing that the Nikon D5300 has only half as much read noise as the Canon 200D? Or are the tests thrown off by Nikon's postprocessing?

Suppose the Nikon clamps every pixel at 2 x the value of the highest of the 8 nearest pixels of the same color. Many measurements involve short-exposure dark frames. If the value of a pixel is very close to zero, it wouldn't take much read noise to elevate it above 2 x the maximum of its neighbors -- would it? So the postprocessing could cut noise in such a way as to affect measurements.


Read noise is measured from bias frames, which are taken at the shortest possible integration (1/4000s if memory serves). As Mark proved, the spatial filtering kicks in at much longer integration time. So it shouldn’t be affected at all.

Francesco
  • sharkmelley and Michael Covington like this

#135 sharkmelley

sharkmelley

    Aurora

  • *****
  • topic starter
  • Posts: 4,624
  • Joined: 19 Feb 2013
  • Loc: UK

Posted 23 December 2019 - 04:55 AM

One practical point: The Nikon spatial filtering makes the sensor show less read noise when tested in the usual way.  Right?

This may be why Nikon sensors seem to show only half as much read noise as competing Canon sensors.

No.  The filtering is not performed on the short exposures typically used for read noise estimates.

 

Read noise is measured from bias frames, which are taken at the shortest possible integration (1/4000s if memory serves). As Mark proved, the spatial filtering kicks in at much longer integration time. So it shouldn’t be affected at all.

That's right.  There's no evidence of any filtering for short exposures (typically 1/5sec or faster).  So it won't affect measurements of read noise.

 

Mark


Edited by sharkmelley, 23 December 2019 - 05:07 AM.

  • Michael Covington likes this

#136 Michael Covington

Michael Covington

    Author

  • *****
  • Posts: 6,790
  • Joined: 13 May 2014
  • Loc: Athens, Georgia, USA

Posted 23 December 2019 - 09:11 AM

OK.  That's what I was unsure of -- whether you had confirmed that none of this filtering happens in short exposures.  Then it appears (from my tests and others) that Canon sensors have about twice as much total read noise as competing Nikon sensors, and this filtering is not responsible for that.  True difference in sensors, or some other as yet unappreciated kind of postprocessing?



#137 erictheastrojunkie

erictheastrojunkie

    Viking 1

  • *****
  • Posts: 997
  • Joined: 15 Jul 2016
  • Loc: Salt Lake City

Posted 23 December 2019 - 10:42 AM

OK.  That's what I was unsure of -- whether you had confirmed that none of this filtering happens in short exposures.  Then it appears (from my tests and others) that Canon sensors have about twice as much total read noise as competing Nikon sensors, and this filtering is not responsible for that.  True difference in sensors, or some other as yet unappreciated kind of postprocessing?

It's a true difference in sensors, Canon has lagged behind Sony (who has produced the vast majority of sensors for Nikon) in the sensor design department for a long time now, your post-processing is only exaggerating the sensor noise characteristics that are already inherent to the sensor. You can easily compare and see the differences between various cameras and their read noise on Bill Claff's website, it becomes much more apparent how much better the Nikon/Sony cameras are compared to equivalent Canon's there:

 

http://www.photonstophotos.net/

 

Long gone are the days of Canon being the most suitable investment in a camera for astrophotography purposes. 


  • 17.5Dob likes this

#138 Michael Covington

Michael Covington

    Author

  • *****
  • Posts: 6,790
  • Joined: 13 May 2014
  • Loc: Athens, Georgia, USA

Posted 23 December 2019 - 11:23 AM

Those Sony sensors are also used in some astrocameras that definitely don't do any postprocessing, aren't they?  That would provide a way to test them "totally raw."  Does anyone know of any test results?

The Canon sensors of course are used only in Canon cameras, but we have every reason to believe that Canon's postprocessing is minimal (just adjustment of bias level, probably).



#139 fmeschia

fmeschia

    Surveyor 1

  • *****
  • Posts: 1,562
  • Joined: 20 May 2016
  • Loc: Mountain View, CA

Posted 23 December 2019 - 12:25 PM

Those Sony sensors are also used in some astrocameras that definitely don't do any postprocessing, aren't they?  That would provide a way to test them "totally raw."  Does anyone know of any test results?

The Canon sensors of course are used only in Canon cameras, but we have every reason to believe that Canon's postprocessing is minimal (just adjustment of bias level, probably).

The QHY247C uses the same Sony IMX193 sensor mounted in the D5300/5500/5600. QHY quotes a read noise figure ranging from 0.8 e- at high gain to 2.6 e- at low gain. Basically the same values one can read from photonstophotos.net for the Nikons.


  • sharkmelley and Michael Covington like this

#140 Michael Covington

Michael Covington

    Author

  • *****
  • Posts: 6,790
  • Joined: 13 May 2014
  • Loc: Athens, Georgia, USA

Posted 23 December 2019 - 02:15 PM

Thanks, exactly what I was looking for, and those numbers match my tests of the D5300 at ISO 6400 and 200 respectively.


Edited by Michael Covington, 23 December 2019 - 04:13 PM.


#141 Michael Covington

Michael Covington

    Author

  • *****
  • Posts: 6,790
  • Joined: 13 May 2014
  • Loc: Athens, Georgia, USA

Posted 23 December 2019 - 04:13 PM

I wonder if Nikon's spatial filtering affects the accuracy of DSLR photometry (e.g., following the AAVSO's procedures).



#142 sharkmelley

sharkmelley

    Aurora

  • *****
  • topic starter
  • Posts: 4,624
  • Joined: 19 Feb 2013
  • Loc: UK

Posted 23 December 2019 - 07:12 PM

I wonder if Nikon's spatial filtering affects the accuracy of DSLR photometry (e.g., following the AAVSO's procedures).

The main effect of the filtering is to cap the value of bright pixels.  So it will potentially attenuate star brightness e.g. the core of a star.  However, one would not be using a DSLR for photometry purposes anyway because of the Bayer matrix.

 

Mark


Edited by sharkmelley, 23 December 2019 - 07:13 PM.


#143 fmeschia

fmeschia

    Surveyor 1

  • *****
  • Posts: 1,562
  • Joined: 20 May 2016
  • Loc: Mountain View, CA

Posted 23 December 2019 - 07:14 PM

The main effect of the filtering is to cap the value of bright pixels.  So it will potentially attenuate star brightness e.g. the core of a star.  However, one would not use a DSLR for photometry purposes anyway because of the Bayer matrix.

 

Mark

I think Michael refers to the AAVSO publication which addresses specifically photometry with a DSLR, so taking the Bayer filters into account.


  • Michael Covington likes this

#144 Michael Covington

Michael Covington

    Author

  • *****
  • Posts: 6,790
  • Joined: 13 May 2014
  • Loc: Athens, Georgia, USA

Posted 23 December 2019 - 07:25 PM

That's right.  Come to think of it, their technique is to put the stars slightly out of focus, which protects them not only from saturation but also from "star eaters."  So I think I've answered my own question. 

Incidentally, they are working on mapping DSLR R G B onto photometric V by some algebraic combination.  I haven't looked at their publications lately to see how far they've come.  I was thinking about Betelgeuse, which is unusually dim right now, and the wide field of view needed to compare it to other stars.



#145 sharkmelley

sharkmelley

    Aurora

  • *****
  • topic starter
  • Posts: 4,624
  • Joined: 19 Feb 2013
  • Loc: UK

Posted 23 December 2019 - 08:02 PM

I think Michael refers to the AAVSO publication which addresses specifically photometry with a DSLR, so taking the Bayer filters into account.

Thanks! 

 

I stupidly attempted to answer the question without knowing exactly what the AAVSO's procedures are.

 

Mark



#146 SandyHouTex

SandyHouTex

    Fly Me to the Moon

  • *****
  • Posts: 5,768
  • Joined: 02 Jun 2009
  • Loc: Houston, Texas, USA

Posted 24 December 2019 - 11:33 AM

It's a true difference in sensors, Canon has lagged behind Sony (who has produced the vast majority of sensors for Nikon) in the sensor design department for a long time now, your post-processing is only exaggerating the sensor noise characteristics that are already inherent to the sensor. You can easily compare and see the differences between various cameras and their read noise on Bill Claff's website, it becomes much more apparent how much better the Nikon/Sony cameras are compared to equivalent Canon's there:

 

http://www.photonstophotos.net/

 

Long gone are the days of Canon being the most suitable investment in a camera for astrophotography purposes. 

I guess those of us who use Canons should just throw them all in the trash then.


  • Alen K likes this

#147 Michael Covington

Michael Covington

    Author

  • *****
  • Posts: 6,790
  • Joined: 13 May 2014
  • Loc: Athens, Georgia, USA

Posted 24 December 2019 - 12:15 PM

The fact is, excellent astrophotos are being taken with Sony (Nikon) sensors, with Canon sensors, and even with older-model sensors that are much worse than current models of either brand.
 

The difference between Nikon D5500 and Canon 200D in the tests on photonstophotos.net seems to be less than in my own tests.  I think it's fair to say it's not large.

 

Remaining Canon selling points include: 
- EFCS (live view shooting, vibrationless) even on low-end cameras (good for lunar and solar work)
- Can use clip-in filters
- Generally more consistency between camera models
- Lack of spatial filtering as far as we can tell

 


  • SandyHouTex, agavephoto and fmeschia like this

#148 Alen K

Alen K

    Surveyor 1

  • *****
  • Posts: 1,669
  • Joined: 25 Nov 2009

Posted 24 December 2019 - 01:42 PM

I guess those of us who use Canons should just throw them all in the trash then.

Sarcasm aside, as has been said many times in this forum, results in photography are more dependant on the photographer than the equipment. As has also been said, equipment (cameras, lenses, telescopes) can influence the degree of difficulty in getting good results. I would say these comments apply to ALL types of photography, not just astrophotography. Generalized, I think they apply to almost everything we do that requires tools or equipment. 

 

Case in point: The best daytime photo I ever got was taken with an early 2Mpixel digital point-and-shoot. But I can't claim any great skill as a photographer. I just got lucky. 


  • SandyHouTex, agavephoto and Michael Covington like this

#149 Michael Covington

Michael Covington

    Author

  • *****
  • Posts: 6,790
  • Joined: 13 May 2014
  • Loc: Athens, Georgia, USA

Posted 24 December 2019 - 02:11 PM

I also think we need to distinguish "can this camera get this image?" from "can this camera get this image a tiny bit better than the other camera?"  

 

We are technology geeks.  We eagerly cheer every small improvement in camera technology.  But, realistically, there is probably not anything I can photograph with one sensor and not the other.  If the pictures are processed properly (which is different for different sensors), you will rarely see any difference.


  • SandyHouTex and agavephoto like this

#150 erictheastrojunkie

erictheastrojunkie

    Viking 1

  • *****
  • Posts: 997
  • Joined: 15 Jul 2016
  • Loc: Salt Lake City

Posted 24 December 2019 - 03:01 PM

I guess those of us who use Canons should just throw them all in the trash then.

Lol, way to take it the extreme! Nobody said you can't get amazing shots with a Canon, but there's no disputing the measured and analysed sensor metrics showing Sony sensors are superior. For the vast majority of astrophotographers the final image quality determination will come from the person sitting behind the camera or computer and not the equipment, but that's not what we're discussing here, from a purely technical and technological point of view the Sony sensors are better. 

 

At the end of the day the combined equipment along with the prowess of the person operating that equipment and doing the processing of the images is what dictates whether the final image is extraordinary or not, but when it comes to equipment and my recommendations I look at the technical aspects and make recommendations. I recommend the cameras with certain Sony sensors and minimal raw manipulation because those sensors are the best from a numbers point of view. And for what it's worth (which isn't much), I've shot with Canon, Nikon, Sony, Fuji, Pentax, and ASI cameras. I'm not married to any one system, I've bought and sold more cameras and lenses and other equipment from every manufacturer than I care to ever remember. I choose what's best based on the many amazing analysis done by people like Mark, Bill Claff, Jim Kasson, Roger Cicala, and the amazing people here on CN, then I go out and test it myself. My personal testing, more often than not, tends to generate results that match those mentioned. 


Edited by erictheastrojunkie, 24 December 2019 - 03:05 PM.



CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.


Recent Topics






Cloudy Nights LLC
Cloudy Nights Sponsor: Astronomics