Jump to content

  •  

CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.

Photo

D750 Non-Linearity Compare D800 W/FW Hack Moded/Unmoded

  • Please log in to reply
28 replies to this topic

#1 skyler

skyler

    Surveyor 1

  • *****
  • topic starter
  • Posts: 1743
  • Joined: 16 Aug 2006
  • Loc: TGPNW

Posted 24 May 2018 - 11:36 AM

Have been looking for advice on a couple of FF cameras for astro-imaging.  I enjoyed using a modified Canon 6D but have been very keen on the Nikon line.  I tried out a temp loaner D750 with a stock 80-200mm F4.5-5.6 Nikon lens and also a few shots prime focus on a Vixen ed103s both with no filters.  As expected I noticed the FF vignetting  in each case which was probably a hair more than expected in comparison to the 6D.  Also I am in an Orange to Yellow sky zone so there was that to contend with.  What I did notice was that when pre processing with PI or DSS and then initially  using STF and ABE, I would see what was seen in Vostock's image noted in the link below. Ironically enough it was in that same region of sky. It could be mostly processed out but it was a real pain.

 

Vostock's image with the circular color artifacts:.
https://www.cloudyni...ts#entry8429369

 

I did some reading and came across more info that seemed very applicable and the root of what I want to know about some particular camera responses.

 

Details provided by Wei Hao in the section discussing  non-linearity response (and images) with the

D800 (no FW hack and D810a:

https://www.cloudyni...n 810a review

 

All that said, I am wondering just what the non-linearity level exists with the D750 and how it would compare to the D800 w/FW hack modified or unmodified and the Canon 6D?  Even an unofficial estimation seen from initial processing of data would be worth knowing.  I hope Wei Hao might chime in on this. I know he said in a different post that he recommended a 6D or (I think) the mod'd D800 w/FW hack, verson for someone but not specifically relating to the non-linearity issue.  And I agree that if you don't have to deal with this from the beginning, it makes for much easier post processing hassles.

 

 

Skylar


Edited by skyler, 24 May 2018 - 11:38 AM.


#2 skyler

skyler

    Surveyor 1

  • *****
  • topic starter
  • Posts: 1743
  • Joined: 16 Aug 2006
  • Loc: TGPNW

Posted 24 May 2018 - 08:42 PM

I am going to add one more thought for the D750.  If what I experienced as a result of the test images with the D750 typical? (which was also described earlier back in 2014 with Wei Hao's pre FW hacked D800) Did any of these issues of non-linearity get resolved other than with what Nikon had done commercially with the D810A? (Altering amount of "black clipping")  Did something change in the D750 after their shutter issues since I don't think I have hardly heard anyone say that they are getting those really difficult to manage vignetting issues that cannot be easily resolved with flats.  Does that mean that people are just processing their way through this as best they can or is it a known issue  and implied that if you buy a D750 or D800/D800E/D810 that you are going to have to deal with this situation?  I may be jumping the gun on this since I am not up on the history or firmware updates or hacks on any or all of these cameras.  I hope someone can set me straight on this since I just haven't had to deal with this situation with the 6D or other cooled cameras.  Is this just particular to these specific Nikon offerings?


Edited by skyler, 24 May 2018 - 09:55 PM.


#3 whwang

whwang

    Vanguard

  • *****
  • Posts: 2085
  • Joined: 20 Mar 2013

Posted 24 May 2018 - 09:29 PM

Hi,

 

I can't tell exactly what's going on in the earlier thread about D5100, although it looks very similar to what I had on my D800 (when not hacked).  I even don't know whether the problem was solved after I read that thread.  This is probably why I didn't participate in that conversation.

 

D800 is based on Expeed 3.  D810, D810A, and your D750 are based on Expeed 4.  Based on my not-so-extensive experience on D810A, the D800 nonlinearity problem does not exit on D810A.  I would naively think that D750 should not have this problem either.  This is also why there is not a firmware hack for this problem on D810, D810A, and D750.  There shouldn't be a problem.  But, of course, I could be wrong, or there could be a new problem that has a different root but cause a similar result.

 

To determine whether there is really a severe problem with D750, I think you need to first examine your workflow in details.  Did you take flat and bias and include them in the calibration?

 

Cheers,

Wei-Hao



#4 ccs_hello

ccs_hello

    Cosmos

  • *****
  • Posts: 9575
  • Joined: 03 Jul 2004

Posted 24 May 2018 - 09:48 PM

re: non-linearity

 

I first saw the title and thought is there something new here?

 

Back to referred old thread, it's not about non-linearity but zero-clip.

Earlier N such as D5100 has sensor clip set too high, this causes incorrect estimate of calibration.  <-- actually it looks nicer since clipping more makes noise (and faint stars) disappear faster smile.gif

 

For that reason, there is the "Nikonhacker firmware mod" which has few options to change the DSP settings (and add back advanced options), on selected camera models, to make it beneficial for AP.

 

I an not familiar with D750 so I don't know if it's already good enough for AP without custom firmware mod.

 

See this thread to understand the D5100/D7000 stories:

https://www.cloudyni...al-noise/page-2

 

Clear Skies!

 

ccs_hello


Edited by ccs_hello, 24 May 2018 - 09:52 PM.


#5 whwang

whwang

    Vanguard

  • *****
  • Posts: 2085
  • Joined: 20 Mar 2013

Posted 24 May 2018 - 10:05 PM

It's not sensor nonlinearity for sure.  Once the black point is incorrectly offset, the "data" become nonlinear, and all the calibrations that depends on the linearity of the data break apart.

 

Anyway, that zero-clipping issue should happen only on Expeed 3 generation of cameras.  This is the first time I hear about this issue on a camera that's based on Expeed 4.



#6 ccs_hello

ccs_hello

    Cosmos

  • *****
  • Posts: 9575
  • Joined: 03 Jul 2004

Posted 24 May 2018 - 10:13 PM

Best way to know the answer is to get an eval version of Rawdigger and use it to read the D750 raw output :) :) .



#7 whwang

whwang

    Vanguard

  • *****
  • Posts: 2085
  • Joined: 20 Mar 2013

Posted 24 May 2018 - 10:46 PM

In the case for D800, it doesn't even require Rawdigger.  Just open an NEF file of bias with PixInsight and look at the pixel histogram, and one can immediately see the data are clipped.  I think this is an easy way to see if D750 suffers from the same problem, or it is something else.



#8 whwang

whwang

    Vanguard

  • *****
  • Posts: 2085
  • Joined: 20 Mar 2013

Posted 24 May 2018 - 10:52 PM

For example, this shows the histogram of a dark exposure taken with my D800 without firmware hack.  The image intensity is increased by 20x to better show the very dark pixels.  One can see that the negative side of the distribution is gone (below zero).

 

If you see the same on D750's bias files, then it is the same problem.

 

Cheers,

Wei-Hao

 

Screen Shot 2018-05-25 at 11.50.10 AM.jpg



#9 skyler

skyler

    Surveyor 1

  • *****
  • topic starter
  • Posts: 1743
  • Joined: 16 Aug 2006
  • Loc: TGPNW

Posted 25 May 2018 - 01:16 AM

Thank you for the links CCS, I look at those posts and Rawdigger.

 

Wei Hao,  Thank you for your input and yes I took many variations of the bias and flats in an attempt eliminate the vignetting but as you noted in one of the prior posts, you end up with even more difficult to deal with artifacts where some corners appear darker than the original vignetting and even areas that show overcorrection where they are now blown out.  I tried both with A mode when taking the flats so the camera came up with the exposure values and also looked at the full well condition of the camera and chose a manual exposure that was closer to 50% seen at the mean values from the Statistics routine in PI.  None of these techniques made any appreciable difference.  I no longer have the camera but may need to get a new one in hand to confirm what you noted about the Exspeed designations. I will take a look at one of the bias images as you suggested to see if there is something similar to what you are describing.

 

BTW, are you still using the D800 that was modified and also running the hacked firmware?  You seemed to be very pleased with the results it has given you in other posts.  Maybe that is the better option since it sounds like it is the equivalent of a D810A in performance?  Did you do the mod yourself and was it a full removal of all filters and replaced with a custom or off the shelf item like Baader?  Also do you have a link to the firmware as well?



#10 whwang

whwang

    Vanguard

  • *****
  • Posts: 2085
  • Joined: 20 Mar 2013

Posted 25 May 2018 - 02:07 AM

Yes, I still have the modified D800 with hacked firmware.  I am very happy with it and it's still one of my workhorse cameras.  With the hacked firmware, the nonlinearity problem is completely gone.  (My latest D800 work is this one: https://www.astrobin.com/340592/)  You can find the hacked firmware on nikonhacker.com.  Just look for a thread called "dark current enabler" or something like that.  I cannot give you the exact link as I cannot connect to nikonhacker.com at this moment.  I asked a local filter manufacture in Taiwan to help me to modify it.  I asked them to replace the stock filter with a UV/IR-cut filter produced by them.

 

I think until D850 becomes affordable or until Sony completely solves the star-eater problem, a used D800 with modification and firmware hack will continue to be the DSLR with best C/P for astrophotography.  D750 is very good too, if it is confirmed to be free from the linearity problem.  You can first check your bias files in PixInsight like what I did above for D800.  This can at least rule out or confirm a D800-like problem.  If this doesn't help to nail down the problem, then it may be a good idea to rent a D750 again (preferentially a different one) to conduct more tests before you actually buy one.

 

If you do take the D800 route, don't get one that's too heavily used.  Pay a bit more and find a lightly used one.  I find D800's resolution is so high that any small play between the lens and lens mount can introduce noticeable distortion on stars in the image corner.  Because of this, you will want a D800 with a lens mount that's still tight and not worn off that much.

 

Cheers,

Wei-Hao


  • tkottary likes this

#11 Jerry Lodriguss

Jerry Lodriguss

    Vendor

  • *****
  • Vendors
  • Posts: 6265
  • Joined: 19 Jul 2008
  • Loc: Voorhees, NJ

Posted 25 May 2018 - 05:45 AM

It's not sensor nonlinearity for sure.  Once the black point is incorrectly offset, the "data" become nonlinear, and all the calibrations that depends on the linearity of the data break apart.

 

Anyway, that zero-clipping issue should happen only on Expeed 3 generation of cameras.  This is the first time I hear about this issue on a camera that's based on Expeed 4.

Hi Wei-Hao,

 

I have a Nikon D5300, which is an EXPEED 4 processor, and it sure looks like it is clipping the blacks to me.

 

Jerry


Edited by Jerry Lodriguss, 25 May 2018 - 05:45 AM.


#12 starbob1

starbob1

    Surveyor 1

  • *****
  • Posts: 1864
  • Joined: 11 Mar 2007
  • Loc: IN

Posted 25 May 2018 - 07:02 AM

Well many of us use the d5300 and it works very well. Low noise

 works great at ISO 400 200.   My biggest problem was with the flats working well. I know this sounds crazy but my flats would only work well if I took a flat and the histogram was 3/4 of the way to the right instead of 1/3 like I use with my Canons. 



#13 whwang

whwang

    Vanguard

  • *****
  • Posts: 2085
  • Joined: 20 Mar 2013

Posted 25 May 2018 - 07:34 AM

Hi Wei-Hao,

I have a Nikon D5300, which is an EXPEED 4 processor, and it sure looks like it is clipping the blacks to me.

Jerry


Then this is very bad. Are you able to get decent flat fielding with D5300?

Edited by whwang, 25 May 2018 - 07:37 AM.


#14 Jerry Lodriguss

Jerry Lodriguss

    Vendor

  • *****
  • Vendors
  • Posts: 6265
  • Joined: 19 Jul 2008
  • Loc: Voorhees, NJ

Posted 25 May 2018 - 09:38 AM

Then this is very bad. Are you able to get decent flat fielding with D5300?

 

I had nightmares trying to flat field a 180mm shot, but I generally don't shoot flats with my refractor with an APS-C sensor.

 

Here are some shots taken with a Canon 7DM2 and a Nikon D5300. Same exposure, ISO, f/stop.  

 

I did these messing around testing for "ISOlessness".

 

Both are 5 stops underexposed for the ISO 100 they were shot at, and "developed" with a +5 push in Adobe Camera Raw.

 

The 7DM2 is very noisy in the blacks because of the higher readout noise, but the D5300 is clipped to 0,0,0. I don't know how well you can see that in the pix posted here, but you can download them and open them in Photoshop and read the black cloth on the wall, and it's 30,12,25 for the 7Dm2 and 0,0,0 for the Nikon.

 

7DM2 is first.

 

7DM2.jpg

 

D5300.jpg

 

Also notice the green cast in the D5300 shot. Both cameras are unmodified, and both were shot with the correct in-camera white balance.

 

Jerry


Edited by Jerry Lodriguss, 25 May 2018 - 09:43 AM.

  • whwang likes this

#15 skyler

skyler

    Surveyor 1

  • *****
  • topic starter
  • Posts: 1743
  • Joined: 16 Aug 2006
  • Loc: TGPNW

Posted 25 May 2018 - 11:22 AM

Very good follow up information and interesting knowing that Jerry is also seeing clipping on his Nikon.

 

 

Here is what I have from a bias frame taken with the D750 with the following information:

 

Timestamp: Sun May 13 23:28:19 2018
Camera: Nikon D750
ISO speed: 800
Shutter: 1/4000.0 sec
Aperture: f/5.6
Focal length: 200.0 mm
Embedded ICC profile: no
Number of raw images: 1
Thumb size:  6016 x 4016
Full size:   6032 x 4032
Image size:  6032 x 4032
Output size: 6032 x 4032
Raw colors: 3
Filter pattern: RG/GB
Daylight multipliers: 2.038880 0.936148 1.143356
Camera multipliers: 1.949219 1.000000 1.343750 0.000000

 

 PI Histogram Bia D750.PNG



#16 whwang

whwang

    Vanguard

  • *****
  • Posts: 2085
  • Joined: 20 Mar 2013

Posted 25 May 2018 - 11:40 AM

Looks like there is not a black clip.  The problem may be something else.  I feel puzzled.



#17 skyler

skyler

    Surveyor 1

  • *****
  • topic starter
  • Posts: 1743
  • Joined: 16 Aug 2006
  • Loc: TGPNW

Posted 25 May 2018 - 03:47 PM

I agree and am at a loss too. I just hope there was not something done with this camera after the third recall on the shutter replacements where some unexpected change was implemented in the firmware.  All just speculation. May be just that this particular camera was a one off issue so I'm very interested now to test another D750.  I would hope other D750 astroimagers would give us their feedback regarding their cameras.  Plus, Jerry's comments were the first I have seen of someone  noting black point clipping with a D5300.

 

Also I wondered, Wei Hao, if you felt the D810A performed identically or even better than your mod'd and firmware hacked D800?  I think I know the answer but in addition I was not sure if you now own a D810A  or if you just had the opportunity to test one but no longer own it?


Edited by skyler, 25 May 2018 - 04:37 PM.


#18 fmeschia

fmeschia

    Messenger

  • *****
  • Posts: 439
  • Joined: 20 May 2016
  • Loc: Mountain View, CA

Posted 25 May 2018 - 04:37 PM

Hi Wei-Hao,

 

I have a Nikon D5300, which is an EXPEED 4 processor, and it sure looks like it is clipping the blacks to me.

 

Jerry

Would you share a NEF frame with clipped blacks? I have examined a few bias frames from a D5300 and it seemed to me to have a non-clipped black level.


Edited by fmeschia, 25 May 2018 - 04:39 PM.


#19 whwang

whwang

    Vanguard

  • *****
  • Posts: 2085
  • Joined: 20 Mar 2013

Posted 25 May 2018 - 09:23 PM

I can’t say D810A is better than a modified/hacked D800. In terms of quantum efficiency, they are equally good. However, D800 has lower read noise. This helps nightscape photography and deep-sky photography under large focal ratios. So the two points together, D800 actually has better raw power than D810A. On the other hand, D810A has better functionality. So it’s a tough call.

I never own a D810A. I was just lucky enough that a friend of my let me use his D810A for a substantial period. So I did some tests and took some images.

#20 17.5Dob

17.5Dob

    Soyuz

  • *****
  • Posts: 3838
  • Joined: 21 Mar 2013
  • Loc: Colorado,USA

Posted 25 May 2018 - 09:49 PM

Hi Wei-Hao,

 

I have a Nikon D5300, which is an EXPEED 4 processor, and it sure looks like it is clipping the blacks to me.

 

Jerry

D5300 bias image analyzed in PI. Not clipped. This has been hashed and rehashed to death.

41453710965_b2b1629b66_o.jpg



#21 Jerry Lodriguss

Jerry Lodriguss

    Vendor

  • *****
  • Vendors
  • Posts: 6265
  • Joined: 19 Jul 2008
  • Loc: Voorhees, NJ

Posted 26 May 2018 - 02:33 AM

Would you share a NEF frame with clipped blacks? I have examined a few bias frames from a D5300 and it seemed to me to have a non-clipped black level.

Sure, do you want a light or a bias?

 

Jerry



#22 Jerry Lodriguss

Jerry Lodriguss

    Vendor

  • *****
  • Vendors
  • Posts: 6265
  • Joined: 19 Jul 2008
  • Loc: Voorhees, NJ

Posted 26 May 2018 - 02:39 AM

Would you share a NEF frame with clipped blacks? I have examined a few bias frames from a D5300 and it seemed to me to have a non-clipped black level.

Sure, here is the D5300 light frame that I posted above.

 

https://www.dropbox....C_0382.NEF?dl=0

 

Maybe it is something that ACR is doing to Nikon NEF files.

 

Jerry



#23 whwang

whwang

    Vanguard

  • *****
  • Posts: 2085
  • Joined: 20 Mar 2013

Posted 26 May 2018 - 04:18 AM

Hi Jerry, 

 

Could you share a bias file too?



#24 fmeschia

fmeschia

    Messenger

  • *****
  • Posts: 439
  • Joined: 20 May 2016
  • Loc: Mountain View, CA

Posted 26 May 2018 - 10:40 AM

Sure, do you want a light or a bias?

 

Jerry

A bias would be perfect.

Thanks!

Francesco



#25 Jerry Lodriguss

Jerry Lodriguss

    Vendor

  • *****
  • Vendors
  • Posts: 6265
  • Joined: 19 Jul 2008
  • Loc: Voorhees, NJ

Posted 26 May 2018 - 05:12 PM

Hi Jerry, 

 

Could you share a bias file too?

Hi Wei-Hao and Francesco,

 

Here's the bias file

 

  https://www.dropbox.com/s/plutm4qkljxian6/DSC_0085.NEF?dl=0

 

Jerry




CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.


Recent Topics






Cloudy Nights LLC
Cloudy Nights Sponsor: Astronomics