678 replies to this topic

[ewave]

That's very useful info!  So it appears that the D850 behaves the same as the D5300 i.e. the hardcoded-correction rings only appear with a lens recognised by the firmware.

 

By the way, this is how the Sony hardcoded correction works - it is only applied for the lenses that the firmware recognises.

Mark, are there any tests that show these rings in the Nikon D780?

I am looking at that camera as a possibility with built in interferometer. 

[sharkmelley]

Mark, are there any tests that show these rings in the Nikon D780?

I am looking at that camera as a possibility with built in interferometer. 

Sorry, I don't have any information on the Nikon D780:

https://www.markshel...ra_summary.html

 

I think it's likely it will have the rings caused by the hardcoded image correction but it's possible they will only occur when a CPU lens is used.

 

Mark

[Michael Covington]

My other camera is a D5300 -- can we confirm that it only has this quirk when a firmware-recognized lens is in place?  Almost all my astrophotography is with non-electronic lenses and I would gladly put the D5300 in service for doing this in situations where a modified camera is not needed.   I'll take test frames with my own D5300 (without a lens) if Mark would like to have them.

My D5500 has rings even with a non-electronic lens.  I know because I bumped into them doing astrophotography before I learned about this thread, and spent a lot of time hunting down reflections -- then realized the reflections were unlikely to be just the same with different optical systems.

[sharkmelley]

My other camera is a D5300 -- can we confirm that it only has this quirk when a firmware-recognized lens is in place?  Almost all my astrophotography is with non-electronic lenses and I would gladly put the D5300 in service for doing this in situations where a modified camera is not needed.   I'll take test frames with my own D5300 (without a lens) if Mark would like to have them.

There was a test here comparing an electronic lens with a telescope:

https://www.cloudyni...4#entry12574874

 

But I'm always happy to look at more files 

 

Mark

[primeshooter]

Reporting on my communication (so far) with Nikon.  I submitted the original request on 3/14 and get a response on 3/17 that my comments and suggestions (their words) was being escalated for review.

 

On the 23rd, they informed me that the suggestions were being forwarded to the offices in Japan for review.  Nothing since, other than a feedback form asking about me recent experience with their support team.  I will respond there and also respond to the ticket asking for follow-up on any decision or action from the Japan office.

 

Bob

I got a response saying they have forwarded my suggestions on for consideration (paraphrasing).

[primeshooter]

Quick thought...from what we have found out so far...I might try one of my nikon afd primes. They have an aperture ring, and can be manually focused. So I can tape over the lens contacts. Do I just use electrical tape? This should mean I should escape and ring problems with those lenses on my d850.

Edited by primeshooter, 13 May 2023 - 11:23 AM.

[FrankieT]

Do I just use electrical tape?

Yep, that worked with my 180mm AF-D (see #616), although it‘s not a very robust solution.

[sharkmelley]

There was a test here comparing an electronic lens with a telescope:

https://www.cloudyni...4#entry12574874

 

But I'm always happy to look at more files 

Thanks for the new Nikon D5300 files. Here is the result of dividing the (bias-subtracted) blue channel by the (bias-subtracted) green channel:

 

 

The top row is the AF Micro-Nikkor 105mm f/2.8D and the bottom row is a non-electronic lens.  Both sets are from the same camera.  It confirms that for the Nikon D5300, the hardcoded image correction only happens for electronic lenses recognised by the firmware.

 

Mark

[Michael Covington]

Thanks for doing that test, Mark, and for publishing the results.  I should add that the rings also appeared with a Sigma lens that follows the AF-D standard and communicates f-stop and exposure with this camera body but does not autofocus on it.  Presumably I could get rid of them by covering the lens contacts (it's a lens on which aperture is easy to set manually).

As I said in the private message, this confirms that I'm in my right mind after all.  When I was doing my book project about 6 years ago, I bought this D5300 and used it extensively.  I knew about the lossy compression >973 ADU but never encountered the hard-coded correction rings -- because, at the time, all the optics I used it on were non-electronic.  Learning about the rings later, I was embarrassed that I hadn't detected them when I was doing my writing.  But the real reason I didn't notice the problem is that it really wasn't there.  Later I got a D5500, thinking it was much the same, but it has rings even with non-electronic optics.

I will continue using this D5300 for astrophotography when I don't need a filter-modified camera, taking care to keep the sky background and the flats just a hair to the left of the first line in the camera histogram, so that only the bright objects are above 973 ADU.   Or if I can't hold to that, I'll simply use your repair tool.  (Which I wonder if it could be made into a PixInsight plug-in.)

 

[Added: Looking back I see that you *did* make a PixInsight script for the D5300.  I'll get it.]

Edited by Michael Covington, 14 May 2023 - 10:48 PM.

[Michael Covington]

Those of you who are in communication with Nikon:  It would be very reasonable for them to make all cameras work like the D5300 and simply turn off the hardcoded corrections when the lens has no electronics.  This would not require putting anything new on the menu and so would be much less work than adding an explicit option.  Would someone like to pass this on?

[vidrazor]

Those of you who are in communication with Nikon:  It would be very reasonable for them to make all cameras work like the D5300 and simply turn off the hardcoded corrections when the lens has no electronics.  This would not require putting anything new on the menu and so would be much less work than adding an explicit option.  Would someone like to pass this on?

Additionally, and more importantly, is to request to have the option to be able turn off RAW data processes.
 

Edited by vidrazor, 14 May 2023 - 03:44 PM.

[FrankieT]

It would be very reasonable for them to make all cameras work like the D5300 and simply turn off the hardcoded corrections when the lens has no electronics.

This might be a workaround for lenses that have a physical aperture ring like AF-D lenses but unfortunately doesn’t help for many modern lenses with an electronic diaphragm and no aperture ring.

Edited by FrankieT, 14 May 2023 - 03:50 PM.

[Michael Covington]

Now let's go back to the very beginning of this thread and ask the question again.  Mark was prompted by anecdotal reports of rings in Nikon D5300 images, and proceeded to discover the lossy compression, and demonstrate it with simple tests on single images.

My question is:  Does the lossy compression have any visible effect in stacked images calibrated against stacked flats?  Or was the original problem actually the hardcoded compression all along?

The answer will depend on whether the D5300 that provoked the original anecdotes was using electronic or non-electronic lenses.

[bobharmony]

Now let's go back to the very beginning of this thread and ask the question again.  Mark was prompted by anecdotal reports of rings in Nikon D5300 images, and proceeded to discover the lossy compression, and demonstrate it with simple tests on single images.

My question is:  Does the lossy compression have any visible effect in stacked images calibrated against stacked flats?  Or was the original problem actually the hardcoded compression all along?

The answer will depend on whether the D5300 that provoked the original anecdotes was using electronic or non-electronic lenses.

Michael - my experience indicates that using fully calibrated data doesn't do anything to reduce the occurrence of rings in the final stacked image.  My example is only impacted by lossy compression as it is a D5300 attached at prime focus to my C6N, no lens electronics involved.  I posted the images of the calibrated and stacked image without the fix applied in this thread in early 2021, and the colored rings are clearly visible.

 

https://www.cloudyni...7#entry10845141

 

After correcting all subs and calibration frames with v1.0 of Mark's program I posted that result a few days later in the thread in post #187.  The difference between the two processes is quite clear.

 

If I misunderstood the intent of your question, let me know and I will delete this post to keep the thread from being sidetracked.

 

Bob
 

[Michael Covington]

That was exactly what I wanted confirmation of.  Thanks!

[bobharmony]

That was exactly what I wanted confirmation of.  Thanks!

You're welcome.  Looking back at the thread entries from 2021, it was interesting to read the discussion around shooting and stacking different length flats to try to mitigate the rings a bit.  At the time I said I would give it a shot, but I never did look into it any more deeply because the results from Mark's program were so effective.  I will look back through some old data and see if I have enough there to give it a go.

 

Bob

[PjBass]

Hi everyone.

I have a full spectrum D5300 and had the issues described here. I did the full spectrum mod myself, so at first i didn't know if i caused the problem myself, or if there was something wrong with the telescope or coma corrector.

I used my D5200 for some time and never had a problem with it, even though everybody says bad things about it. Changing to D5300 was a nightmare until now.

So, when i was about to give up on it i found all this information, the exact same problem i was having.
I gave Mark's software a try, completely corrected the problem of the rainbow rings. Thanks for all your work Mark, it's brilliant and you care about helping people. Faith in humanity restored 😂

I would also like to thank all who is making noise about this with Nikon, even if nothing happens, you are trying to make it happen.

When the clouds go away i will try the recomended histogram spots to shoot the lights and flats, hopefully with better results.

Nikon D5300
C1.00
L2.000

Thank you all

[Michael Covington]

Your filter modification didn't cause it -- it's a software thing.  

[PjBass]

After checking some of my older photos with the D5200 i noticed the same astifacts in some of the pictures, not ao obvious like in the D5300, still, they are there.

So, another camera for the list

Nikon D5200
C1.02
L2.002

Later i will try to process the fotos again, just for curiosity, and practice, wich i need...a LOT 😂

Thanks again

[sharkmelley]

After checking some of my older photos with the D5200 i noticed the same astifacts in some of the pictures, not ao obvious like in the D5300, still, they are there.

So, another camera for the list

Nikon D5200
C1.02
L2.002

Be aware that Nikon cameras have two types of coloured concentric rings:

• Rings caused by the flawed lossy compression algorithm
• Rings caused by some kind of hardcoded image correction - possibly a colour shading correction

If the rings are fixed by my software then they are the lossy type because there is no fix for the rings caused by the hardcoded correction.

[PjBass]

For what i understood from the previous posts, it's the compression issue, since all the pictures i have noticed the problems where shot through a 8" f/4 reflector.

I tested your software with the frames from the D5300, happy ending.
Will try to test with the D5200 frames later, after that i will post about it.

I think the problem was more obvious with the D5300 because of the full spectrum mod. Not so obvious in the stock D5200, but still there.

[Phillip Creed]

Dumb question, maybe, but for Nikon cameras with the ring issue...

 

...what happens if you expose the flats to the same histogram peak as the lights?  That is, if you're seeing rings with a 25% histogram peak, take the flats to a 25% histogram peak?  What happens then?

Clear Skies,

Phil

[Michael Covington]

Dumb question, maybe, but for Nikon cameras with the ring issue...

 

...what happens if you expose the flats to the same histogram peak as the lights?  That is, if you're seeing rings with a 25% histogram peak, take the flats to a 25% histogram peak?  What happens then?

Clear Skies,

Phil

They won't be a perfect match, and the compression happens at a very specific ADU value (like 0-972, then a sudden step when you get to 973-1000 or something like that).  You will not be able to make the transition from 972 ADU to 973 ADU happen in the exact same place in both kinds of frames.

However, I wondered if it might help to dither the exposure for the flats (take each flat with an exposure a few percent different from the others, maybe by very slightly varying the light source).  I think that would hide the compression rings and might explain why they often didn't appear when expected -- the less consistently I hold my flat panel, the better!

[sharkmelley]

Dumb question, maybe, but for Nikon cameras with the ring issue...

 

...what happens if you expose the flats to the same histogram peak as the lights?  That is, if you're seeing rings with a 25% histogram peak, take the flats to a 25% histogram peak?  What happens then?

In theory, if you can match the brightness of the flats perfectly to the brightness of the lights in the affected colour channels (i.e. red and/or green and/or blue), then the rings can be calibrated away.  But, in practice, changing sky conditions during the course of an imaging session means that the brightness of the lights (and therefore the ring positions) will also change.  This makes such a strategy impractical.

[Phillip Creed]

In theory, if you can match the brightness of the flats perfectly to the brightness of the lights in the affected colour channels (i.e. red and/or green and/or blue), then the rings can be calibrated away.  But, in practice, changing sky conditions during the course of an imaging session means that the brightness of the lights (and therefore the ring positions) will also change.  This makes such a strategy impractical.

An exact match...yes.  Impractical.

But what if you're at least close?

Clear Skies,

Phil