304 replies to this topic

[Jerry Lodriguss]

You're probably right but I said it to be extra sure because vignetting control on Sony cameras (shading) definitely affects the raws.

Hi Mark,

 

I didn't realize the Sonys applied that to the raws.

 

My best practice is to turn everything off unless I want the jpegs out of the camera to look as good as possible as a single frame.

 

But the Nikons are only supposed to apply LENR to the raws.

 

Jerry

Edited by Jerry Lodriguss, 23 December 2019 - 11:57 AM.

[lucutes]

I posted this on a RASA thread thinking it have had something to do with the scope. So from what I gather my data is useless and I need to shoot with higher ISO settings to get the histogram in the middle along with the flats? This was taken with the Nikon Z50 on a RASA 8. I guess this is also common on the Z6 sensor as well?

 

ISO800

137X30sec

Note flats might have been underexposed. I added some extra saturation and exposure to show the problem.

Attached Thumbnails

• 

[SandyHouTex]

I posted this on a RASA thread thinking it have had something to do with the scope. So from what I gather my data is useless and I need to shoot with higher ISO settings to get the histogram in the middle along with the flats? This was taken with the Nikon Z50 on a RASA 8. I guess this is also common on the Z6 sensor as well?

 

ISO800

137X30sec

Note flats might have been underexposed. I added some extra saturation and exposure to show the problem.

Looks like a red bullseye.

Edited by SandyHouTex, 01 May 2020 - 09:37 PM.

[sharkmelley]

I posted this on a RASA thread thinking it have had something to do with the scope. So from what I gather my data is useless and I need to shoot with higher ISO settings to get the histogram in the middle along with the flats? This was taken with the Nikon Z50 on a RASA 8. I guess this is also common on the Z6 sensor as well?

 

ISO800

137X30sec

Note flats might have been underexposed. I added some extra saturation and exposure to show the problem.

That's a really good example of the coloured-concentric-banding problem and it's the first I've seen from the Z50.  Thanks for posting it.

 

The workaround is to shoot your lights and flats so the peak back-of-camera histogram is in the middle. 

 

Mark

Edited by sharkmelley, 02 May 2020 - 04:09 AM.

[lucutes]

Would Diffraction Compensation cause this? I noticed this setting was ON.  

[sharkmelley]

Would Diffraction Compensation cause this? I noticed this setting was ON.  

Every type of compensation, correction and noise reduction is switched off on my Z6 and I still see the concentric coloured banding.  I'm certain that diffraction compensation is not the cause.

 

Mark

Edited by sharkmelley, 13 May 2020 - 05:23 PM.

[Jerry Lodriguss]

Every type of compensation, correction and noise reduction is switched off on my Z6 and I still see the concentric coloured banding.  I'm certain that diffraction compensation is not the cause.

Nothing is applied to the raw files except LENR, right Mark?

 

Jerry

[lucutes]

Seems rather unfortunate that these new Nikon cameras have this problem, almost a major flaw. So the only possible way around this is to jack up the ISO and lose DR?

[sharkmelley]

Nothing is applied to the raw files except LENR, right Mark?

The reason we see these rings is precisely because Nikon is applying a hardcoded correction to the raw data. In a post on the DPReview thread ( https://www.dpreview...s/post/63127228 ) I actually derived an approximate functional form of this correction for my own camera (it is unique to every camera - probably factory calibrated).  It would be an impossible task to derive a version accurate enough to cancel the rings in a stretched astro-image.  I think the raw data correction is a crude attempt to prevent a shift in colour towards the corners of an image caused by pixel crosstalk e.g. in the image of a snowy scene it would prevent the snow turning a slight shade of blue towards the corners - an effect known as (peripheral) colour shading.

 

Seems rather unfortunate that these new Nikon cameras have this problem, almost a major flaw. So the only possible way around this is to jack up the ISO and lose DR?

Unfortunately the raw data correction has unintended consequences for astrophotography.  In that sense it is a flaw.  As you say, the only way around it is a mixture of increasing the ISO and increasing the length of exposure, both of which lose DR.

 

Mark

Edited by sharkmelley, 14 May 2020 - 01:54 AM.

[UKalwayscloudy]

These analyses are immensely useful. I'd been going down to ISO 800 to try to exploit the invariance and increase DR and finding all kinds of annular colouring when processing. I'll take it back up in the light of these comments. I had thought good flats were helping though. 

 

lucutes : interesting to see you have the 50 on the RASA8 - I'd been wondering about that as it was a bit over the size limits - seems that it doesn't matter. How do you have it connected? Are you going via the Sony adapter or do you have another route? Ta

[lucutes]

These analyses are immensely useful. I'd been going down to ISO 800 to try to exploit the invariance and increase DR and finding all kinds of annular colouring when processing. I'll take it back up in the light of these comments. I had thought good flats were helping though. 

 

lucutes : interesting to see you have the 50 on the RASA8 - I'd been wondering about that as it was a bit over the size limits - seems that it doesn't matter. How do you have it connected? Are you going via the Sony adapter or do you have another route? Ta

I've been using this: Baader Wide T Ring for Nikon Z (Bayonet) with D52i to T-2 and S52 # TRING-NZW 2408335

Its very very close to the corrector plate. My flats were not good enough so I can't say this is a winner yet. I would be interested to see the results using a modified version and or the higher ISO to counteract that colour banding. Maybe this a BSI Sony sensor issue. IDK.

 

Here is a flat and a YouTube link(s) to the fellow who tired it with a Z7 and Z50. Sorry for the thread drift on the Z50 everyone.
 

https://youtu.be/Cb4cFRY717w

https://youtu.be/ueyEQw9cpg4

Attached Thumbnails

• 

[UKalwayscloudy]

Brilliant thanks. I already have that Baader gear. Do you have room for the HFC filter chamber with that?

[lucutes]

Absolutely not. Be careful not to scratch your corrector plate with your finger when you go for the clicker. 

[Andy Lucy]

Mark,

There is an interesting note in the details of the latest Pixinsight release, as follows.

 

New Separate CFA flat scaling factors parameter.

When this option is enabled and the master flat frame is a single-channel image mosaiced with a Color Filter Array (CFA), such as a Bayer or X-Trans filter pattern, three separate master flat scaling factors are computed for the red, green and blue CFA components, respectively. When this option is disabled, a single scaling factor is computed for the whole master flat frame, ignoring CFA components (which has been the behavior of ImageCalibration until the 1.8.8-6 release).

 

Do you think that this is this relevant to your findings on the Z6 concentric colour bands?

 

Andy

[sharkmelley]

Hi Andy,

 

No this scaling won't have any effect on the concentric colour bands.  They will still be there regardless because they are baked into the raw data..

 

Mark

[otoien]

What about the single scaling factor improvement on the problems discussed in the Color Matched Flats thread?

https://www.cloudyni...se-study/page-2

[sharkmelley]

What about the single scaling factor improvement on the problems discussed in the Color Matched Flats thread?

https://www.cloudyni...se-study/page-2

The problem discussed in that thread is a colour shading in the corners that is not correctable by a flat taken with a different colour illumination.  Scaling the R, G and B channels in the flat does not alter this colour shading in the corners.

 

Mark

[Andy Lucy]

Mark,

 

I understand from your post #88 that the there is a scaling applied to the raw digital data of the red and blue channels.  You might think that this could be correctable through the use of flat calibration frames.  The origin of a deviation in the signal across the sensor – whether optical or caused by software – makes no difference as to how the flat file is used.

My thought was that in versions of Pixinsight up to now the masterflat file from an OSC camera has averaged the contributions of the red, green and blue data.  Thus, if one colour was boosted in a corner but another colour was diminished, Pixinsight would not be able to make a proper correction. 

 

The situation appears to have changed with the latest update of Pixinsight, so that the red green and blue elements of the master flat are used individually for corrections.

 

Andy

[sharkmelley]

My thought was that in versions of Pixinsight up to now the masterflat file from an OSC camera has averaged the contributions of the red, green and blue data. 

No, PixInsight has never done this. It would be the wrong thing to do mathematically because the calibration of a pixel would then be affected by neighbouring pixels.  Also it would mean that calibration would require knowledge of the bayer matrix pattern.

 

Mark

[otoien]

The problem discussed in that thread is a colour shading in the corners that is not correctable by a flat taken with a different colour illumination.  Scaling the R, G and B channels in the flat does not alter this colour shading in the corners.

 

Mark

I am not completely sure I fully understand this argument, but I have made a followup on this in the mismatched flat thread with some tests on other bodies, in order not to get it mixed up with specific problems of color circles on the Z6. 

https://www.cloudyni...2#entry10461645

[UKalwayscloudy]

I’m scratching my head on this issue with coloured flats. A few months ago I discovered that desaturating my flats in LR before feeding them to DSS gave me better results in that the annular banding was less in the output and more easily removed in PS with Gradient Externinator. I told myself that this made no sense as I’d read somewhere the stackers killed the colour in the flats anyway. Maybe there is something to this plan after all perhaps as the raw processing from grey but RGB flats will not contribute an additional piece of banding to that coming from the subs. I’ve only recently installed PI and the same process does not lead to such good results, so I’m left wondering if DSS and PI behave differently in the way they manage colour in flats. Having thrown away my original data for these tests I’m going to get some fresh flat data for my next session and run it through both DSS and PI without and with desaturation first.

Even if I’m not talking nonsense, I dont see how this can affect what’s going on in the subs. Might be a route to halving the impact on the final output by not having another layer of annular crap from the flats though that pops up when you stretch.

[sharkmelley]

If you are seeing coloured annular rings in your data then flats (coloured or otherwise) will not cure the problem.  You need to shoot both your lights and your flats so they have a good level of exposure e.g. the peak of the back-of-camera histogram should be near the centre and not over to the left.

 

Mark

[UKalwayscloudy]

Thanks Mark, but I never said that flats cured the problem that arises from the low level processing in camera. I always look at the histograms to try to get exposure in the right place.

What I did say was that there was some strange behaviour in the processing of flats in either or both of DSS and PI that in the former case lead to less (not zero) annular garbage in the output image. It has lead me to question the way the colour part of flats is managed in both programs.

That’s an observation and a question. My speculation is that managing flats in the right way will not create a second pile of annular garbage has no helpful effect on that in the light frames and possibly adds to the problem. The flats are there to manage stuff like vignetting and crap on the sensor. My concern is that badly managed coloured flats might be creating another layer of annular rubbish along the lines discussed in this thread that because of the way it is generated might add to the problem. Crudely speaking, if the annular garbage from the lights is x, I’m not claiming that flats can reduce x to zero, rather that being careful with the flats might stop x being doubled in the output.

[sharkmelley]

All the results presented by me in this thread use raw files and all calibration (e.g. with flats) is done while the files are raw i.e. before demosaicing.  You spoke about "desaturating flats" in LR before using them.  However, the files produced by LR are non-linear and any attempt to use non-linear flats will never work correctly.  If you are using flats in the wrong way like this then it's very likely they will be creating additional artifacts.

 

Mark

[bokemon]

Dark Current and Sensor Self-Heating

 

The Nikon Z6 has a sensor with IBIS (in body image stabilization).  This means the sensor must be free to be moved, which might make it more difficult to conduct away heat.  To investigate, I took my usual approach of placing the camera in a dark room at 20C ambient temperature and taking successive 5 minute dark frames on a camera that has cooled to ambient.  The dark current is estimated by subtracting each one from its successor (which removes most of the fixed pattern noise).

ummm, so I have some questions about this.  My understanding is that dark current (in each pixel) ramps up linearly with time, and then on top of that there is the sqrt(N) noise.  And your chart shows that things level off after about 1 hour of use.  Fixed pattern noise means that the signal will be the same even if you took a 1/1000th second exposure(?).  So I don't understand because if you subtract one frame from the next after things have "leveled off", that should remove both the fixed pattern noise and also the linearly ramping dark current, leaving only the sqrt(N) part.  Are you actually plotting the RMS value of this sqrt(N)?

Next question: Sooo, what are we supposed to do in the first hour worth of picture taking while the Z6 is still heating up?