13 replies to this topic

[eacron]

Hi Everyone,

I'm interested in being a AAVSO contributor. I'm attempting to take the first step and test the linearity of my Nikon d7500 camera.  Following the DSLR manual I have a uniformly lit light source (an LED tracing panel that has been used successfully to take flats).  I have been taking exposures at iso 100 and f/32 at 2 min intervals.  I used my imaging software Siril to convert the RAW (nef) files to FITS and measure the mean ADU values of a 200 x 200 pixel area in the center of the flat.  Siril doesn't allow measurements of the RGB image only the separate red, blue, green channels.  I have plotted the data points and am not getting a linear graph. When I add a tread line it conforms directly with a moving average tread line.  I've attached the excel file with the data I've accumulated.  Can anyone tell me what I'm doing wrong? I am a MAC user with both apple silicon and intel computers.  The only processing software I have and use is Siril.  I also have AstroimageJ installed but no experience using it.

Thank you so much,

Eli

Attached Files

•  lineartest_02_04_23.xlsx   29.78KB   21 downloads

[munkacsymj]

Your results are not terribly different from the results listed by this study: https://maxmax.com/f...ikon-d700-study

 

DSLR manufacturers have no strong motivation to achieve a linear response; most general photography reviews downgrade truly linear sensors because they saturate more easily and show less detail in the dark areas.

 

And, yes, that will affect the quality of your photometry. There is still valuable contribution that you can make -- this nonlinearity shouldn't affect your ability to do good measurements of time-of-minimum for eclipsing binaries, for example.

 

- Mark M

Edited by munkacsymj, 06 February 2023 - 10:37 AM.

[GaryShaw]

Hi Eli

What makes you think you’re doing something wrong?

 

I use dedicated Astro imaging cameras as you see in my signature and they are nearly linear from a few thousand up to 60,000adu but it may be that the DSLRs just aren’t quite as linear at the low end as your curve shows. The point of the test is to determine where the camera is non-linear and your testing has given you that information. Just keep your exposures well within the linear range and you should be fine.

 

If you plan to pursue photometry via variable star imaging or exoplanet work, be sure to read the ‘Guides’ for each that AAVSO provides and consider taking the AAVSO’s Choice CCD or Exoplanet courses when they come around. The exoplanet course is starting tonight so I’m not sure when it will be offered again. You can find the current course schedules in the Education section of the AAVSO website.

 

Good luck

Gary

[Ed Wiley]

I work only with CCD and CMOS monochrome cameras. There must be about 1000 things I don't know about photometry with DSLR cameras. What I can relate is what I teach my students about finding inflection points given that the linearity test has been conducted correctly.

 

Fit a linear regression line using only that part of the data that shows linearity. Then extend that line backward and forward and look for inflection. I have done that by taking a small sample of your data that looks linear, fitting the line and then extending it. This, of course, is only a guide.

 

From what Mark has said above, the result is not unusual for DSLRs, so I suspect you conducted the test correctly.

 

Ed

 

 lineartest_02_04_23_V2.xlsx   35.93KB   12 downloads

[eacron]

Hi Guys,

Thank you all so much for your replies and encouragement.  I was thinking I was doing something wrong because the canon linear tests I've seen looked very linear and I couldn't believe Nikon would be so different.  Also, I wasn't sure if I was measuring the ADUs correctly.  The only program that I have available to measure ADU is Siril and I'm not sure if that would be accurate enough or if Siril is doing something to the RAW file when converting to FITS.  I don't think there are many alternative software options available for the MAC platform?  So now if I take the linear part of my graph and I get the y-intercept can I come up with the count rate residual or would I not have enough data points?  Maybe I can do this again and vary the shutter speeds to 1 sec intervals to get more data points in the linear part of the graph.

 

Thank you again for all your help!

 

Eli 

Edited by eacron, 06 February 2023 - 05:22 PM.

[robin_astro]

 

And, yes, that will affect the quality of your photometry. 

Years ago I played around with webcam and video cameras for photometry. They were design specifically to be non linear (gamma correction) but I used to apply a reverse gamma correction to linearise the image. These days it would be relatively trivial to have  software correct the pixel values based on  these curves I would have thought?

 

Cheers

Robin

[LauraMS]

It is pretty obvious that there is linearity only at higher signal, i.e. "20" on the x-axis. It appears that you have some systematic (constant) signal contribution which adds to the light signal and becomes more evident at smaller values. Do you have serious noise in the images? Do you have inhomogeneity within your measured 200 x 200 region - i.e. from the lens? Or do you have heavy noise?

 

It would be helpful if you would plot each data point with the standard deviation of raw pixel signal intensities for the data point. Don't know if Siril does that but ImageJ (including AstroimageJ) definitely does it if you read the image and then press ctrl-m (I think this is the key for doing a measurement after you selected your region before).

 

Do you have all postprocessing done within the camera switched off, e.g. low light correction, automatic subtraction of background (whatever they are called at the D7500)? I have that camera as well but didn't look into this camera recently, so might be helpful to also check those settings.

 

Best, Laura

Edited by LauraMS, 07 February 2023 - 05:20 AM.

[robin_astro]

 It appears that you have some systematic (constant) signal contribution which adds to the light signal and becomes more evident at smaller values.

I wondered about that but if this is the cause,  it is not constant but increases with exposure as the extrapolated curve passes through zero at 0 exposure time. I assume  the bias offset been subtracted before measurement. These exposures are quite long so it could be worth checking that the offset does not change significantly with exposure time. 

 

Cheers

Robin

Edited by robin_astro, 07 February 2023 - 06:32 AM.

[robin_astro]

I wondered about that but if this is the cause,  it is not constant but increases with exposure as the extrapolated curve passes through zero at 0 exposure time

EDIT:  The fact that the extrapolated curve runs through zero rules out a constant offset being the cause but to reproduce the curve seen any offset would need to be increasingly negative with exposure time which does not seem likely. Modern cameras do get up to all sorts of tricks to minimise noise for example so perhaps even the RAW image may not remain untouched by these manipulations 

[eacron]

Hi Laura and Robin,

Thank you for all the helpful suggestions.  I believe that I have turned OFF all post processing but I just noticed that I have the picture control set to SD (standard) and they have the sharpening bumped up to 3 and clarity set at +1.  I will set these now to zero and try again.  I also have color space set to Adobe... should it be at sRGB?  Otherwise I have WB set to direct sunlight, and have vignette control, long exp NR, and high ISO NR, all set to OFF.  I know there might be some firmware postprocessing present - nothing to do about that.  I actually did redo the linearity test (before I noticed the SD settings) and this time made sure I went all the way to saturation.  I think this data seems more complete, although still consistent with my previous results, and this time I also calculated the residual (doesn't look pretty).  From the residual data, if I calculated correctly, it seems I'm forced to exposures between 28-32 sec. am I reading that right?  In case you're curios I'm attaching the xls file.

 

Thank you again!

Eli

 

 

Attached Files

•  lineartest_02_06_23.xlsx   21.3KB   11 downloads

[LauraMS]

Hi Eli,

 

since you are using RAW (uncompressed?), WB and color space should be irrelevant. It is important that you have vignette control, long exp NR, and hIgh ISO NR switched off. Which you do. Not shure about sharpening and clarity - I think it should be irrelevant on RAW, but would be better do set them to neutral in case of further measurements.

 

Looking at your data there is basically nothing new here - don't know what happens around 50sec, but anyway: There is linearity at higher signals, but not the expected linearity at low signal/short exposures. To me this looks like some geometric mean such as sqrt( noise*noise + signal*signal), where signal would be linear with exposure time, and noise would be a constant value. Or it may also (but slower) increase with exposure time (as Robin suggested).

 

Did you repeat the measurement under identical conditions but with the LED panel switched off (and completely shielded from any surrounding light source)? That would be a dark image. You need to do it at the same exposure times as your light images.

 

For scientific grade image analysis, you also need to do bias calibration using a very short exposure time (1/25sec or so) image. Did you acquire this as well, and did you use dark and bias calibration for full account of what happens in a digital sensor? If you have the images, Siril should be able to do the calibration for you.

 

One last point: what ISO did you use? 400 would be the optimal value for the D7500. And leave some time (1min or so) between individual exposures so the sensor can cool after it heated up during the exposure. The latter may not be the most important factor, but if you would repeat the measurement I would suggest doing it to be on the safe side. Otherwise you will have increasing levels of dark current because of increasing sensor temperature. This is a small drawback of using an (uncooled) DSLR, although the effect shouldn't be so dramatic.

 

You may consider opening the f-stop of your lens in the linearity measurement from f/32 by one or two steps, and reducing the exposure time to  half of what you used previously, or a quarter. This would help reduce the contribution of dark current, in particular if you are doing this linearity measurement at room temperature.

 

Good luck,

Laura

Edited by LauraMS, 08 February 2023 - 02:38 AM.

[eacron]

Hi Laura,
Thank you for all that helpful information. Yes, using RAW uncompressed. All the linear tests were done at ISO 100 using an A3 tracer led panel at the lowest brightness setting with a neutral density gels filter (3stop loss). I didn’t take any calibration frames. I’ve just been following the AAVSO manual for getting to know my dslr camera. Ok, so I’ll redo the test at iso 400 at f/16. I’ll see at what exposure I saturate and I’ll set exposure accordingly with 1 minute delays between exposures and subtract the calibration frames before measuring the flats. Please let me know if I got that right.

When I take my light of variable stars, if I understand correctly I know my camera saturates at 60400 ADU - flats should be taken at 2/3 saturation when I take my calibration frames so I should shoot for 40000 ADUs flats? Also I calculated my gain to be around .7 at ISO 400 so that supports your recommendation - that’s unity for my camera. I tried to determine my offset by taking a bias frame at 1/8000 and measuring in siril like I did with the linear test but I get weird results. I did it several times and got values 50-300. The results weren’t consistent at all. Could it be there’s no offset added?

Thanks again - I’m really looking forward to this weekend .

Eli

[LauraMS]

Hi Eli,

 

with ISO100 you shouldn't have to much dark current, but as you know ISO 400 is unity gain. Going from ISO 100 to 400 should reduce your exposure time enough, it may be short enough, so stick with f/32 (don't change to many things at a time). 

 

For flats I typically go with  about 50% saturation to be on the safe side regarding linearity, but if AAVSO suggests 2/3 that's fine. You will have less relative noise in the flats.

 

Re offset frames my experience is not good with very short (1/8000 sec) exopsures. Typically it is no good idea to push any parameter to the camera's limit - it may not work perfectly reliable. Typically, I go with 1/250sec for bias with my D7500. try that, or 1/1000. As long as you are at least in the second range with your exposures, that should be perfectly fine.

 

Good luck and have much fun!

 

Laura

[robin_astro]

I have an old Canon 350D DSLR which I don't use for photometry but this thread prompted me to do a quick check of its linearity

 

 

 

The scatter is probably due to my makeshift technique. I just aimed it at a blank wall and took a series of exposures but it looks pretty well behaved with a constant offset and when set at ASA 100 at least, it is roughly  linear all the way up to 4095, presumably limited by the 12 bit A/D converter. (A lower speed setting might allow the full well depth of the sensor to be used) 

 

Cheers

Robin