873 replies to this topic

[Kenbuddy]

I've used my D5300 for several years for AP, mostly with no problems. Occasionally, though, I have seen these ugly rings. Recently, I shot M33 over two nights using my Orion 8" f/3.9 Newt with the Baader MPCC Mark III along with a Baader Neodymium Moon & Skyglow filter. The first night I only managed about 8 usable 5-minute subs, and the second night, I added about 24 6-minute subs. I made some flats both nights. First night was the t-shirt over the aperture and flashlight method (which I have given up on). Second night, I used a small computer monitor that just barely covers the aperture of the scope. I created a small HTML file that just shows a solid gray background (something like #333333) and then just open the browser in full-screen mode. With the darker gray, I can do flats about 1/8 second and get them about in the middle of the camera histogram, which is what I generally shoot for.  After reading this thread, I know to push it more to the right. I also used some mismatched darks from the previous week where I had shot a bunch of 180 sec darks at about 40 degrees. For M33, the temperature was close to the same - maybe within 5 or 10 degrees, but the exposure length was way off. In PixInsight, I calibrate the short flats with bias only, but I calibrated the lights with the flats and darks (and I used the dark optimization). I can say that the rings were less pronounced the second night, presumably because the flats were brighter.

 

On the calibrated M33 images, I noticed the colored rings, which I could not remove with DBE, so I am very sad and disappointed. I had suspected calibration issues (especially with mismatched darks), but now I see this is a known hardware thing with the D5300. Question is, what triggers it, and how to mitigate it? Seems that ISO 200 is the main culprit, as I have almost always used ISO 200 with this camera because it is supposedly "ISO invariant."  The other culprit seems to be vignetting. I learned that the MPCC Mark III may be causing vignetting with the smaller T-ring. I had to buy a larger T-ring for this camera with my WO GT71, which I think uses T-48. I think there's an option to use that T-ring with the MPCC, so I need to investigate that further. The other case where my photos were ruined, I was using my 300 mm zoom lens, but I had put the 2" Moon & Skyglow filter on the top of the lens, using a couple of filter reducer rings. With this in place, it definitely reduced the aperture of the lens and changed the focal ratio (the aperture on the lens itself was full open). But the reduced aperture caused horrible circular rings in the resulting subs. One guy somewhere said it was an example of "Newton's Rings".  That could have contributed and exacerbated whatever deficiency the camera has.

 

As for my efforts on M33 last weekend, I have made a short video from the PixInsight Blink process that shows all of my calibrated and registered lights (from both nights) that were then stretched inside of the Blink process using the button on top. You can clearly see the colored rings here, but what is interesting is that they seem to expand outward as the night goes on. The first 8 frames were the first night, and the remainder were the second night. Both nights started out west of the meridian, and as the scope got further down (west), the sky also got brighter due to street lights in that general direction (behind the house, though, but still bad). So I think the radius of the rings has something to do with the angle of incidence of the gradient-inducing light source. Perhaps this video showing the motion of the rings is a further clue to what is really going on. I hope this discussion continues and we can eventually know for sure what is causing the problem and how to fix it!

 

https://drive.google...iew?usp=sharing

Edited by Kenbuddy, 23 January 2021 - 05:22 PM.

[Mike in Rancho]

Interesting video, thanks.  Well, aside from the fact that D5300 rainbow rings give me nightmares lol.

 

I don't know if it's a gradient-inducing light source, per say, that results in these rings - in my experience mostly after the application of flats and after a strong stretch.  I have continued testing, of sorts, but without any breakthroughs.  Last weekend I took 7 or 8 different sets of flats and corresponding dark flats, at both an LCD screen and outside (sky and t-shirt), all at various exposures from far left to far right histogram.  I honestly didn't see much difference after stacking.  Except for one - a sky flat at about 15/16 BOC that created a huge circular color cast on the final image that rendered it utterly unprocessable.

 

What was also interesting about that was that Iris showed three ADU peaks: 4.5K, 12.5K, and 14K.  Most of the time I only see two ADU peaks, presumably because two colors are mostly overlaying one another.

 

That said, I may not have chosen appropriate lights for testing, being just an hour or so of the horsehead and flame.

 

I'm also trying out ASTAP stacking to compare against DSS, as well as its blur flats function.  I do wonder if it has anything to do with the ways certain programs may translate the NEF files.

 

Lately I've just become better at handling the rings in processing, using the right amount of wipe and autodev in Startools.  Interestingly, when applying more aggressive gradient wipes I have also noticed the expansion of the color rings during that process - very similar to your video.  Why?  I have no idea lol.

 

But one of your final points is also something I've been musing on this week - whether too much variance in skyglow, for example from tracking before, at, and then through the meridian down into a light pollution dome, causes the flats division to create multicolored ringing as the master is applied to each such light frame with a different overall brightness level.  And then of course they are all stacked together.  Because of that, I've been thinking of sticking to maybe only imaging a couple hours on either side of the meridian.

[Readerp]

Interesting that I have never seen those rings with my D5300 Ha mod camera.

 

I usually only shoot 3 hours on a target on any given night ISO400 typically, although I did shoot a few targets at ISO200.

 

Also most of my work I do at a remote dark site, so don't know if light pollution would affect my experience.

 

For flats I use my laptop, no t-shirt, and shoot to half histogram on the back of the camera (around 1/160").

 

Flats, darks, bias frames in DSS. The resulting stack into StarTools.

 

Mike in Rancho, you could stack just a couple hours of your data from a previous night that produced the rings,

to see if your hypothesis about long duration sessions in light polluted areas might have in influence?

 

-Pete

[Mike in Rancho]

Interesting that I have never seen those rings with my D5300 Ha mod camera.

 

I usually only shoot 3 hours on a target on any given night ISO400 typically, although I did shoot a few targets at ISO200.

 

Also most of my work I do at a remote dark site, so don't know if light pollution would affect my experience.

 

For flats I use my laptop, no t-shirt, and shoot to half histogram on the back of the camera (around 1/160").

 

Flats, darks, bias frames in DSS. The resulting stack into StarTools.

 

Mike in Rancho, you could stack just a couple hours of your data from a previous night that produced the rings,

to see if your hypothesis about long duration sessions in light polluted areas might have in influence?

 

-Pete

Yes I can't figure why some people never have D5300 rainbow rings, and others do.  And one would surmise that your modded camera, letting more of the spectrum in, might be worse?  Perhaps a dark site does help?  Or maybe we should do a survey of who is running what firmware revision lol?  I did update mine to the latest, prior to starting with AP.

 

Half BOC is what I used to do with my flats, almost always a white laptop screen with 2 sheets of paper, until it was recommended that we should shoot for half an ADU histogram.  But as noted, I didn't see much if any difference except with a really far to the right exposure.

 

Where do you expose your lights to on BOC?

 

I applied your idea to some old M81 data I was already playing with.  Not the best, only 2 hours worth and not a huge variation in skyglow pointing north like that.  But the original stack did end up with a single thin yellow/blue ring, out near the area where the vignetting transition is most severe.  I restacked with only half the lights, those with the lowest skyglow rating.  It did make the stack cleaner overall, but only the mildest if any reduction in the ringing.  That said, I will have to keep in mind not taking lights across a wide variation of background light.  I wonder where the point is that you are making things worse as opposed to adding SNR?

 

I then tried a substack with as little variation as possible in Dx and Dy from the reference, on the supposition that the stacking offsets could be smearing around the ringing artifacts.  Unfortunately this data was from when my mount was not tracking well, so I didn't have too many lights.  That said, on a quickie test, it did seem to produce more manageable ringing.  So, nailing my PA and watching my drift and periodic error may help as well.

 

Overall, just some bandaids, still not sure what is behind it all.

 

 

[Kenbuddy]

Well, I spent a LOT of time working on the assumption that this was a calibration or debayering issue, but then I realized that the rings are actually in the raw lights. Here is an example of a single NEF file from one of my lights on the first night. I debayered it every possible way in PI, but kept getting the same result. I read in another discussion that you could convert the NEF in ImagesPlus with the "BayerBasicRaw" option selected, and that was supposed to do the trick. But when I tried it, although the image did look different than opening in PI, after a few more steps, the ringing was present. My test in PI was to take the raw NEF file, debayer it, then apply DBE to the debayered image. After doing the "screen transfer" stretch, the ringing was always there. Key point here is that the problem is in the uncalibrated lights -- i.e. the raw NEF files straight from the camera. So whether it has to do with image scaling or lossy compression or internal reflections due to stray light sources, I don't know.

 

 

 

[sharkmelley]

The question keeps being asked why some people see these rings and others don't. 

 

I will answer this question as best I can.  I've done some more investigation into this, so I'm now in a slightly better position to answer.  At the outset, one very obvious thing to note is that people who don't subtract light pollution and/or stretch their data probably won't see the rings.  Otherwise every terrestrial photographer who used the D5300 would have been complaining to Nikon!

 

Unfortunately the following explanation will get quite technical but you don't need to understand every aspect of it - just jump to the rule of thumb at the end. As part of my explanation I will refer to my earlier D5300 RawDigger histogram of a flat:

 

 

Looking at the green channel you will notice that the left-hand side of the histogram hump has 3 obvious "troughs" in it and the right-hand-side has 3 obvious "peaks".  It is these peaks and troughs in the histogram that cause the rings to appear in the image.  Either your lights or your flats might contain these rings or sometimes both will contain them.

 

Ultimately these large scale peaks and troughs are caused by a side effect of Nikon's lossy data compression.  I will now try to explain this lossy data compression. If you look carefully at the D5300 raw data histogram you will notice that at low data values (600-970) the histogram contains only even values.  When the values reach 970 the histogram contains every third value and when the values reach 1870 the histogram contains every fourth value.  This histogram "combing" is quite obvious when you look carefully with an application like RawDigger.  By the time the maximum value of 16383 (for 14bit data) is reached, the histogram contains only every 11th value!  This sparseness in the compressed data means that the NEF files occupy a much smaller size than they otherwise would be. The data compression scheme is lossy because you can never get the original data back.

 

As previously mentioned, this data compression causes large scale peaks and troughs to appear in the shape of the histogram but I haven't completely worked out why - I'm still trying to reverse engineer exactly what may be happening.  In any case,  there are some ranges of data values where there are no large scale peaks and troughs appearing and if the main part of your histograms for both lights and flats sit in these ranges then your data will not contain rings.  One very obvious range is the range 600-970.  So if your lights are underexposed slightly, with the peak of the back-of-camera histogram slightly less than one quarter from the left then the main part of your red, blue and green data will be just below 970 and the lights will not contain rings.  Also as already shown, if you expose your flats so the peak of the back-of-camera histogram is as far to the right as possible without clipping then your flats will not contain rings. 

 

The above advice is the best rule of thumb I can give at the present time.  It's the same as the advice I gave earlier but now I have a better idea of why it works. If you want to be a bit more sophisticated than check your own histograms for yourself - not the back-of-camera histogram but a raw data histogram (the back-of-camera histogram is not raw)

 

I hope to come up with further advice after I've done more investigation but you should should see from the above explanation that if, by accident or design, your data mainly sits in a range where there are no histogram peaks and troughs then you will not suffer from rings.

 

The other very obvious question that remains to be answered is how many other Nikon cameras are affected by this same issue i.e. where lossy compression causes large scale artefacts in the raw histograms, leading to rings in the data.  Anyone interested to find out should take a series of flats at different exposures and examine the raw histograms, looking for these peaks and troughs.

 

Those who are interested in more technical data, especially which histogram values are used for each different Nikon camera when lossy compression is used, can consult Bill Claff's page on NEF compression:

https://www.photonst...Compression.htm

 

Mark

Edited by sharkmelley, 25 January 2021 - 04:34 AM.

[limeyx]

Thanks Mark,

 

So the conclusion is to expose lights *less* and flats *more* than we might be doing already ? I'm going to look at my histograms

On my latest image with the refractor instead of the DSLR lens, I dont see the rings (sample of 1) BUT I do see a nasty red mottling across the background. 

 

Currently unsure whether its another manifestation of the rings, or just "what you get" in Bortle 6, or if the moon caused this (it was mostly opposite in the sky) OR if my dew heater being on the same USB hub as the camera introduced noise (so many possibilities!)

 

I am much happier with the mottling compared to rings but still all this is irritating as I was about to get this camera modded and now I am not sure I want to

[Kenbuddy]

Great analysis, Mark. Thanks for the detailed explanation. Yes, I did see your previous recommendation and actually added it to my notes for my next AP session. The evidence you present is very convincing. I guess now it just takes more experimentation and more input from other Nikon users with other cameras using lossy compression to add to your already convincing case. As for the lossy compression, it's funny because I've had this camera for several years, and I always just assumed the NEF file was uncompressed. I only realized (to my great disappointment) that it used a lossy compression in the last week while researching this problem!  So now the next thing is to hack the D5300 to turn off lossy compression. I see the hack exists for other cameras on the NikonHacker website, but not for the D5300. Another disappointment!  I'll research that some more. I am a software developer, so maybe I can figure out how to write this hack. Or maybe not.  But I'll look into anyway and see what's involved. Not sure if the reason the hack doesn't already exist is because nobody's tried, or because it's impossible.  

 

Ken

[Alrakis]

I have a D5300 and have this issue and I want to understand this correctly. First off, it sounds like not all D5300s have this issue. I would be very curious if other CNers without this issue would be willing to try Mark's (sharkmelley) experiment to see if it is endemic with the camera or only with certain S/N (maybe a batch issue). Also, does anyone have an idea if anyone on the NikonHacker website is even looking at the D5300? It looks like there was a similar problem with the D5100 which was solved by someone at NikonHacker. 

 

Finally, I was wondering if I understand how to get around this issue:

 

For lights, have the ISO set as low as possible based on the histogram (as far to left as possible) and how long the shutter is open. I am guessing we are looking at ISO 100, 200, and 400 depending? Lower ISOs for longer subs?

 

For darks, same settings as the lights, but with the cover on. With the darks does it matter if the camera is even connected to the lens/scope, or can you just cap off the camera and click away at the same temperature?

 

For flats, same ISO as the lights, but have the exposure of the setup (camera/telescope) pointing toward the daylight sky and/or a monitor (white or grey???) with a sufficient time for the shutter to put the camera's histogram as close as possible to the right without clipping any channel.

 

For dark flats, same as flats, but with cover on.

 

Is this correct?

 

Chris

[bobzeq25]

So if your lights are underexposed slightly, with the peak of the back-of-camera histogram slightly less than one quarter from the left then the main part of your red, blue and green data will be just below 970 and the lights will not contain rings.  Also as already shown, if you expose your flats so the peak of the back-of-camera histogram is as far to the right as possible without clipping then your flats will not contain rings. 

 

Note that these things would have minimal effect on image quality.

 

I'd say, if you do have the problem (I don't) they qualify as a solution.  One with little cost other than paying attention to it.
 

Edited by bobzeq25, 25 January 2021 - 04:11 PM.

[bobzeq25]

Also, does anyone have an idea if anyone on the NikonHacker website is even looking at the D5300? It looks like there was a similar problem with the D5100 which was solved by someone at NikonHacker. 

 

Finally, I was wondering if I understand how to get around this issue:

 

For lights, have the ISO set as low as possible based on the histogram (as far to left as possible) and how long the shutter is open. I am guessing we are looking at ISO 100, 200, and 400 depending? Lower ISOs for longer subs?

 

For darks, same settings as the lights, but with the cover on. With the darks does it matter if the camera is even connected to the lens/scope, or can you just cap off the camera and click away at the same temperature?

 

For flats, same ISO as the lights, but have the exposure of the setup (camera/telescope) pointing toward the daylight sky and/or a monitor (white or grey???) with a sufficient time for the shutter to put the camera's histogram as close as possible to the right without clipping any channel.

 

For dark flats, same as flats, but with cover on.

 

Is this correct?

 

Chris

The problem with the 5100 is entirely different.  The black point is set at zero, which chops half your bias off.  Plays havoc with calibration.  The hack moves it to 600ADU, and the D5300 is already there.

 

ISO 100 seems to have quirks.  I use 200, and do so consistently, it's general purpose.

 

There's no need for dark flats for the D5300/5500/5600.  Bias works fine.  This is also true for many cameras, the ZWO1600 (which does need dark flats) seems to have caused an overreaction spilling onto cameras which do not need dark flats.
 

Edited by bobzeq25, 25 January 2021 - 04:19 PM.

[Alrakis]

Okay, so no dark flats. That is a relief; don't want to clog up the hard drive with data that doesn't help. Typically I have been using ISO 400, but will try ISO 200 as soon as it clears up here. Is everything else that I outlined correct as a possible next step?

 

Chris

[Kenbuddy]

Also, does anyone have an idea if anyone on the NikonHacker website is even looking at the D5300? It looks like there was a similar problem with the D5100 which was solved by someone at NikonHacker. 

It seems like they (he?) are not. I thought maybe if resources were available, I might take a stab at a hack, but it looks to be way over my head. I think it's just the one guy writing stuff. In any case, I did submit a request to Nikon Support to do a firmware update to make lossless NEF compression available on the D5300, but that's probably a longshot. But we'll see what they say...

[limeyx]

Great analysis, Mark. Thanks for the detailed explanation. Yes, I did see your previous recommendation and actually added it to my notes for my next AP session. The evidence you present is very convincing. I guess now it just takes more experimentation and more input from other Nikon users with other cameras using lossy compression to add to your already convincing case. As for the lossy compression, it's funny because I've had this camera for several years, and I always just assumed the NEF file was uncompressed. I only realized (to my great disappointment) that it used a lossy compression in the last week while researching this problem!  So now the next thing is to hack the D5300 to turn off lossy compression. I see the hack exists for other cameras on the NikonHacker website, but not for the D5300. Another disappointment!  I'll research that some more. I am a software developer, so maybe I can figure out how to write this hack. Or maybe not.  But I'll look into anyway and see what's involved. Not sure if the reason the hack doesn't already exist is because nobody's tried, or because it's impossible.  

 

Ken

I am a software developer also ... let me know if I can be of any help here. I've seen the old hacks but the pages are so old I didn't really pay attention.

I'll take a look and see if I can make any sense of it. I don't really want to add a new camera to my list of needs

[limeyx]

Okay, so no dark flats. That is a relief; don't want to clog up the hard drive with data that doesn't help. Typically I have been using ISO 400, but will try ISO 200 as soon as it clears up here. Is everything else that I outlined correct as a possible next step?

 

Chris

hahah I started using ISO200 because I thought we decided 400 was better and maybe even 800

I may try 200 with the new scope. As I said, I have a new issue which I need to diagnose (ISO400) -- I just dont have enough clear nights to experiment outside

 

I am not certain how much use it is to do further indoors experiments w/out a camera firmware change (which may not even be possible) but I am happy to run one here if it helps -- although I definitely had the rings

[sharkmelley]

I can now prove that the peaks and troughs in the histogram are related to the lossy compression.  The Nikon Z6 provides a choice over whether or not to use NEF lossy compression.  Here is the result of 2 identical successive flats taken seconds apart but using different compression:

 

 

It's pretty conclusive - the green histogram on the left shows the peak and trough artefacts.  I'm pretty shocked to discover this on a recent camera such as the Z6, to be honest.  For the avoidance of doubt, I never shoot with lossy compression - this was simply an experiment to show what happens when you do.

 

Mark

Edited by sharkmelley, 25 January 2021 - 06:01 PM.

[Mike in Rancho]

Thank you Mark, heck of a job!  I know you posted that one graph in the past (I think), but I understood the explanation a lot better this time.

 

Is the x-axis on that Rawdigger the ADU?  Thinking maybe it's just scaled over to show your point.  I know you are mentioning shooting from 600-970, but I don't see anything like that on your graphs.

 

Seems to be a commercial program.  Is IRIS a good substitute on the ADU levels?  Alas it does not seem to show the differing colors.

 

When this weather clears I will have to experiment with the lower exposed lights.  Though why some aren't having any issues I have no idea.  I'll also review some of my prior data to see what I was using.  I did quickly check a few already.

 

These were a couple of flats I took at 1/4 and 1/8 BOC histogram.  You can see the main peaks, both of which seem to extend outside of your "good" range on the lossy compression.

 

 

I also looked at a couple lights I was working on recently.  ISO400 (all of this stuff is) and 60 seconds.  Per BOC, or BackyardNikon (which is pretty much the same if not exactly the same histogram), they were exposed with the rightmost blue channel straddling the 1/4 line.  The second one was a little darker.  Seems to me then we might have to go well below 1/4.

 

 

And this was a 60 second dark, which seems to extend from 580 to 620, and some minor counts extended out to 1200 or more.

 

 

That seems to indicate a dark could partially fall in the range of your 600-970 lights data recommendation.  Might our darks then end up subtracting target data off?

 

It may take some exposure experimenting to find the right sweet spot for these lights then, and see how BOC and BYN histograms line up with the RAW histograms.

 

As for firmware hacks by our software gurus - would it only affect the file compression and size of the NEF's?  I believe the BYN guy has said his program won't work right with hacked cameras - but I have no idea what the hacks we was talking about were.

[sharkmelley]

Thank you Mark, heck of a job!  I know you posted that one graph in the past (I think), but I understood the explanation a lot better this time.

 

Is the x-axis on that Rawdigger the ADU?  Thinking maybe it's just scaled over to show your point.  I know you are mentioning shooting from 600-970, but I don't see anything like that on your graphs.

 

Seems to be a commercial program.  Is IRIS a good substitute on the ADU levels?  Alas it does not seem to show the differing colors.

IRIS is a good substitute because it does show the raw histogram.

 

The reason for keeping raw data values below 970 is because of my example in the original post.  The exposure in the middle of the montage (i.e. DSC_0037) had the peak of the back-of-camera histogram at the 1/4 point and raw data values extended up to around 1200 in the green channel.  It's clear that this exposure had a nasty big ring, so it was necessary for me to reduce the exposure further to prevent ring formation. 

 

However, at this stage in the investigation, I can't say for certain that everyone who exposes up to the 1/4 point will have a problem with rings.  Indeed, plenty of people say they have no problem.

 

Mark

Edited by sharkmelley, 25 January 2021 - 07:35 PM.

[Alrakis]

Do you think the differences in D5300 cameras are based in the different firmwares? Version 1.00 vs 1.01 vs 1.02 vs 1.03?

 

If so, is there a way to roll the firmware to a specific version? 

 

Chris 

[Kenbuddy]

Do you think the differences in D5300 cameras are based in the different firmwares? Version 1.00 vs 1.01 vs 1.02 vs 1.03?

No. Differences in firmware only cover lens support and something about cities on the date time menu.

 

https://downloadcent...oad/fw/285.html

 

I believe you can roll back the firmware to any version, as long as you have that version. Not 100% sure if you can download the older versions, but my guess is you can.

[bobzeq25]

I note that the D5500 (which I own) has a different compression algorithm.  It's possible that's why I've only seen this problem once, out of many images.

 

To be clear.  I don't believe the 5100 hack has _any_ relevance here.

Edited by bobzeq25, 26 January 2021 - 02:26 PM.

[sharkmelley]

I note that the D5500 (which I own) has a different compression algorithm.  It's possible that's why I've only seen this problem once, out of many images.

That's right, the D5500 has a lot more room at the lower end where the rings will not cause a problem.  The weird histogram artefacts that cause the rings start when the histogram contains every third value. On the D5300 this happens when values hit 970 whereas on the D550 problems won't begin until values hit 2125.  The bias level is 600 for both cameras. 

 

It's an excellent reason to buy the D5500 instead of the D5300.

 

Mark

[limeyx]

That's right, the D5500 has a lot more room at the lower end where the rings will not cause a problem.  The weird histogram artefacts that cause the rings start when the histogram contains every third value. On the D5300 this happens when values hit 970 whereas on the D550 problems won't begin until values hit 2125.  The bias level is 600 for both cameras. 

 

It's an excellent reason to buy the D5500 instead of the D5300.

 

Mark

Well Cr*p I JUST got the D5300 last year specifically because everyone seemed to be using and recommending it and I really did not want to go buy a new camera thats for all intents and purposes "the same" (minus this issue and some live view potential fixes)

 

This is disappointing.

[Readerp]

Well Cr*p I JUST got the D5300 last year specifically because everyone seemed to be using and recommending it and I really did not want to go buy a new camera thats for all intents and purposes "the same" (minus this issue and some live view potential fixes)

 

This is disappointing.

Don't despair. If you go to a dark site you won't likely expose past 1/4 histogram and will likely never have a problem.

[fmeschia]

That's right, the D5500 has a lot more room at the lower end where the rings will not cause a problem.  The weird histogram artefacts that cause the rings start when the histogram contains every third value. On the D5300 this happens when values hit 970 whereas on the D550 problems won't begin until values hit 2125.  The bias level is 600 for both cameras. 

 

It's an excellent reason to buy the D5500 instead of the D5300.

 

Mark

Thank you Mark for such a detailed analysis, and congratulations for getting to the bottom of it.

Checking my D5500 – it seems the lossy compression algorithm starts “dropping” values at 1552 DN. Still that’s better headroom than the D5300.

TBH I never understood why, even at the bias level, the D5300 drops every other DN. It doesn’t make sense to keep every value only at a DN level that the camera will never use...