Jump to content

  •  

CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.

Photo

My 3D-printed DIY spectrograph

  • Please log in to reply
66 replies to this topic

#26 robin_astro

robin_astro

    Apollo

  • -----
  • Posts: 1,224
  • Joined: 18 Dec 2005

Posted 25 January 2021 - 03:49 PM

I would suspect the fine fringes are probably some sort of optical interference fringes or sampling artifacts that are common issues in spectrographs, rather than from the LED source. You can check for this (and potentially remove them) by taking a flat using a halogen lamp.

 

Cheers

Robin



#27 syam

syam

    Vostok 1

  • -----
  • topic starter
  • Posts: 188
  • Joined: 11 Jun 2009
  • Loc: Hamilton, Canada

Posted 25 January 2021 - 05:34 PM

Yeah, I'm sure a spectrum calibration target would be ideal, but I'm pretty happy even with the results I've got.



#28 GeorgeLiv

GeorgeLiv

    Your Light Pollution Info

  • -----
  • Vendors
  • Posts: 130
  • Joined: 04 May 2018
  • Loc: Montreal Canada

Posted 26 January 2021 - 07:12 AM

I much admire of what I assume is your own custom setup. Did you 3D print the housing yourself with your own design and does it have a cover? (i.e., am I seeing half here)?

 

Way back in the 1990s I build my own portable (much simpler) device that I still use today and travel with it. Of course I cannot image spectra of celestial objects but I've had loads of fun with it over the years. A 0.5 nm accuracy is good enough for me.

 

I've had numerous revelations with it from the sky-glow spectra I've acquired over different cities/locations and from filter performance with my very own sky-glow. My 2008 sky-glow with the transmissions for some 3 filters follows (increase the brightness of your screen to see the full extent of transmissions in the red):

 

2008-Sky-glow+Filters.jpg

 

When I travel with my device I typically wait for any cloudy or clear night. With a low & bright overcast I can get the local "up-light" spectrum within about 15 minutes depending on its brightness (with dslr noise reduction feature enabled). Clear nights require exposures ten times longer, oftentimes well over 1 hour, but typically within about 40 minutes (x2 with dslr noise reduction) for big city clear nights. Clear night spectra are more rewarding in that you get the "feel" of the actual sky-glow you're peering or imaging through.


  • syam likes this

#29 syam

syam

    Vostok 1

  • -----
  • topic starter
  • Posts: 188
  • Joined: 11 Jun 2009
  • Loc: Hamilton, Canada

Posted 26 January 2021 - 09:35 AM

^ Thanks, useful data!

 

Correct, this spectrometer is of my design. My first fully-parametric Fusion 360 design.



#30 syam

syam

    Vostok 1

  • -----
  • topic starter
  • Posts: 188
  • Joined: 11 Jun 2009
  • Loc: Hamilton, Canada

Posted 27 January 2021 - 09:42 AM

Here is an idea how to do response curve calibration in a spectrometer.

 

An incandescent bulb produces light close to black body radiation. The problem is that its peak is way into far IR, so one cannot use it for calibration directly because the exact temperature cannot be measured.

 

My idea is to measure spectrum from the same bulb at two different voltages, resulting in somewhat different filament temperatures. Lets assume that both are simple black body radiations with two unknown temperatures, T1 and T2.

 

Each of the two measured spectra "I" is a dot product of the actual brightness curve "B" (black body radiation) and the spectroscope's response curve "R":

 

I1 = R*B1(T1)

I2 = R*B2(T2)

 

By dividing the two spectra (pixel by pixel), we get rid of the unknown response curve R, and get an analytic function of two unknowns - T1 and T2:

 

ratio = I2/I1 = B2(T2) / B1(T1)

 

Using the black body radiation equation, we get:

 

ratio(lambda) = (exp(A/(T1*lambda))-1) / (exp(A/(T2*lambda))-1)

 

(A is a constant: A=1.43877e7 when lambda is in nm.) This is a well defined, non-divergent function of T1 and T2. Using non-linear fitting (in Octave etc) one can derive both temperatures from the observed ratio(lambda)!

 

I did a test: generated two black body curves at T1=3000K and T2=2500K, computed their ratio, then provided initial guesses for both temperatures (2750K for both), and then recovered the original values for T1 and T2 (3000K, 2500K) by means of non-linear fitting of the ratio curve in Octave. Here is the plot for the log(ratio) I recovered:

 

black_body_ratio.png

Now that we know the two temperatures, to get the response curve R, we simply divide one of the measured spectra by the black body radiation for the recovered temperature value:

 

R = I1 / B1(T1)

 

In real world, the issues will be measurement noise, deviation of the incandescent light from black body radiation, and non-linearity of the measurements. But hopefully this will produce useful data.

 

One can use a 12V incandescent bulb (like they use in car turning signals etc) powered via power supply with voltage regulation.

 

I'll give it a try.


Edited by syam, 27 January 2021 - 09:51 AM.

  • robin_astro, JoeVanGeaux and Peter Z like this

#31 robin_astro

robin_astro

    Apollo

  • -----
  • Posts: 1,224
  • Joined: 18 Dec 2005

Posted 27 January 2021 - 10:34 AM

Now that is an interesting idea. Incandescent lamps are routinely used for flat correction in spectrographs (By applying a flat from such a source you are effectively removing the instrument response from the system, replacing it with the flat lamp spectrum) and Christian Buil describes using this method to approximately determine the instrument response  using an incandescent lamp here. (short method)

http://www.astrosurf...nt_response_us/

The limitation is not knowing the precise temperature. Your method could provide a solution

 

Cheers

Robin



#32 syam

syam

    Vostok 1

  • -----
  • topic starter
  • Posts: 188
  • Joined: 11 Jun 2009
  • Loc: Hamilton, Canada

Posted 27 January 2021 - 10:42 AM

I suspect the fundamental issue will be noise.  Dividing two noisy curves is not a great idea. I'll be doing some more calculations.



#33 robin_astro

robin_astro

    Apollo

  • -----
  • Posts: 1,224
  • Joined: 18 Dec 2005

Posted 27 January 2021 - 10:51 AM

Except at the UV end where there is little light there should be plenty of signal and the photon noise can always be beaten down with longer and or multiple exposures.  Another problem though may be the practical one of generating a suitable diffuse source of light without disturbing the black body spectrum. Buil waves the lamp in front of the spectrograph/telescope but that would not work here where reproducibility of the intensity is important. (My spectrographs use  built in diffusers for the calibration lamp but they are known no to be truly "white"

 

Cheers

Robin



#34 robin_astro

robin_astro

    Apollo

  • -----
  • Posts: 1,224
  • Joined: 18 Dec 2005

Posted 27 January 2021 - 11:12 AM

Except at the UV end where there is little light there should be plenty of signal and the photon noise can always be beaten down with longer and or multiple exposures.  

For example here is the incandescent lamp master flat (sum of 30 exposures) from my ALPY spectrograph covering the visible range.  (The Y scale is counts for a 250 pixel high binning zone and the camera gain is ~0.3e-/count) The SNR is  >1000 over much of the range.

 

alpy_flat_counts_250binheight.png

 

You do have to watch for offsets due to scattered light in the spectrograph though and subtract the camera bias signal

 

Cheers

Robin



#35 syam

syam

    Vostok 1

  • -----
  • topic starter
  • Posts: 188
  • Joined: 11 Jun 2009
  • Loc: Hamilton, Canada

Posted 27 January 2021 - 11:13 AM

May be just placing the filament perpendicular to the slit will do? I have one of such 12V bulbs, with a ~10mm long straight filament. Brightness variations from different filament points will cancel out during this procedure. The only thing which is important is the same temperature across the filament. It's probably not bad if you limit yourself to the central part?

 

I actually have my own evidence for the last point - I did focus stacking macro photography of live filaments in these 12v bulbs:

 

Inside the bulb:

 

50003048407_ab5f03330f_w.jpgGlowing bulb filament by SyamAstro (750,000 views - thank you!), on Flickr

 

With the bulb removed:

 

50010629432_cae5f30141_w.jpgLive open bulb filament by SyamAstro (750,000 views - thank you!), on Flickr

 

 

So the temperature does change quite a bit along the filaments. But these are very cold temperatures (barely glowing) - perhaps at higher voltages temperature is more even?


Edited by syam, 27 January 2021 - 11:18 AM.


#36 robin_astro

robin_astro

    Apollo

  • -----
  • Posts: 1,224
  • Joined: 18 Dec 2005

Posted 27 January 2021 - 08:05 PM

I ran a quick test with my ALPY using a pearl tungsten lamp run a 3 different voltages.

 

flat_at different_temperatures.png

 

 

The software that generates the flat images normalises them so I am not seeing the absolute differences but  I can clearly pick up the change in temperature and  max/min temp (black line) looks nice and smooth so it all looks good in principle. I just need to extract the absolute values and wavelength calibrate them.

 

Cheers

Robin


Edited by robin_astro, 27 January 2021 - 08:06 PM.

  • syam likes this

#37 syam

syam

    Vostok 1

  • -----
  • topic starter
  • Posts: 188
  • Joined: 11 Jun 2009
  • Loc: Hamilton, Canada

Posted 28 January 2021 - 08:52 AM

I ran a quick test with my ALPY using a pearl tungsten lamp run a 3 different voltages.

 

attachicon.gifflat_at different_temperatures.png

 

 

The software that generates the flat images normalises them so I am not seeing the absolute differences but  I can clearly pick up the change in temperature and  max/min temp (black line) looks nice and smooth so it all looks good in principle. I just need to extract the absolute values and wavelength calibrate them.

 

Cheers

Robin

Looks good! Definitely disable any normalizations. Let me know if you need my Octave scipt for fitting the curve to get the two temperatures. I still have to play with it - it looks like the initial guess for the two temperatures is important. (It looks like it should be between the two real temperatures.) I'll run with the whole matrix of initial values, to make a sort of "goodness of fit" plot.



#38 syam

syam

    Vostok 1

  • -----
  • topic starter
  • Posts: 188
  • Joined: 11 Jun 2009
  • Loc: Hamilton, Canada

Posted 28 January 2021 - 11:20 AM

I did some more testing, and I am afraid my algorithm is extremely sensitive to the noise in data - to the point that in real life it might not work (to be tested).

 

I am showing a few plots, with a varying amount of noise. In all cases, the actual two temperatures are 2500K and 3000K. My initial guesstimates for both temperatures had the range 2000...3500K, with the step of 50K.

 

A) Zero noise

 

Here is the plot of the ratio of the measured spectra. Obviously, no noise visible. X axis: lambda, Y axis: log(spectra_ratio)

 

noise0_b.png

 

Now I start using nonlinear fitting to recover the two temperatures. Even with zero noise, one has to try many initial guesses (I set both temperature guesses to the same value) to recover the correct values. Here X axis is the std deviation of the recovered temperature values from the actual ones (this is not observable), Y axis is the observed std deviation of the fit from the data (previous plot - ratio of spectra). The idea is to use the observable parameter (axis Y) as a proxy to getting the X axis value as close to zero as possible (zero would mean the temperatures were recovered perfectly). But at least with zero noise case, the situation is straightforward: the lowest Y value corresponds to the cases of the lowest X values, so one can easily recover the two true temperatures.

 

noise_0a.png

 

B) Some small amount of noise

 

Noise is so small that you can't really see it yet in the ratio plot:

 

 

noise_1b.png

 

But it already impacts the fitting process in a big way. Fortunately, lowest Y values still corresponds to lowest X values, so we can still recover the true temperatures accurately:

 

noise_1a.png

 

C) 10x more noise

 

The noise is now observable:

 

 

noise_10b.png

 

And now impact of noise on fitting is quite dramatic - minimum in Y no longer corresponds to zero X. If we use the observed Y axis minimum as our solution, our error in both temperatures is ~150K:

 

noise_10a.png

 

 

D) 100x more noise than in (B)

 

The noise is very obvious now:

 

 

noise_100b.png

 

 

And with this level of noise, the algorithm completely breaks down: minimum Y values result in >1000K errors for both temperatures:

 

noise_100a.png



#39 syam

syam

    Vostok 1

  • -----
  • topic starter
  • Posts: 188
  • Joined: 11 Jun 2009
  • Loc: Hamilton, Canada

Posted 28 January 2021 - 02:50 PM

I just quantified the maximum noise which can be tolerated in my algorithm: if the fractional noise on the blue (left) side of both spectra is larger than ~0.01, the algorithm falls apart. If the fractional noise is <0.001, the algorithm works very well.

 

So the advice here is just integrate enough of frames to drive the left side noise down to 0.001 or so, and then the two temperatures you get should be pretty accurate.


Edited by syam, 28 January 2021 - 03:04 PM.


#40 syam

syam

    Vostok 1

  • -----
  • topic starter
  • Posts: 188
  • Joined: 11 Jun 2009
  • Loc: Hamilton, Canada

Posted 29 January 2021 - 11:24 AM

I ran a test to estimate the accuracy of the recovered temperatures using my algorithm. The core procedure is:

 

  • Given two actual temperatures, T1=2500K and T2=3000K, generate black body radiation spectra
  • Add a certain amount of noise to both (fractional noise at the blue end is say 0.001)
  • Fit my model to the ratio of the two spectra, using 30 different initial values for both temperatures (T_ini=2000, ..., 3500, step 50K)
  • From those 30 models, pick the one which has the smallest std of the data points from the model fit
  • Store the two found temperatures from the best model

Then I repeat the above procedure 100 times, to see the confidence intervals of the measured temperatures.

 

Here is the result when the blue fractional noise is 0.001 (blue points are the results; the red point is the actual temperatures):

 

T1_T2.png

 

Overall accuracy is pretty good - the error for both temperatures is <=40K. What is striking is how degenerative the errors are - instead of a cloud, they form an almost perfect line T2/T1=const. This means that my analysis measures the ratio of the two temperatures to a much higher accuracy than the values of the temperatures. Which was kind of expected, given that in both spectra only a remote blue end is observed, with the bulk of the spectra being in IR. In these tails, things become asymptotically simpler, with the changes mostly explained by the ratio of temperatures.

 

When I increase the blue side noise by a factor of 10 (0.01 fractional noise), I get:

 

T1_T2b.png

 

The degeneracy is still very obvious, so the ratio of temperatures can still be computed quite accurately. But for the purposes of spectrometer calibration the temperature ratio is not useful, we need the temperature values. And with 0.01 fractional blue noise, temperatures cannot be measured accurately - the errors are ~400K, which is too much to be useful for calibration.

 

I also tried an intermediate noise level - 0.003 fractional blue side noise - and got rather large temperature errors up to 200K, not useful for calibrations.

 

Even smaller amount of noise (0.0003) results in ~10K temperature error.

 

So I quantified my earlier observation - with the blue side fractional noise levels ~0.001 or less, we can use my algorithm for spectrometer calibration. Noise 0.003 or higher results in unacceptably high temperature errors (>=200K).


Edited by syam, 30 January 2021 - 09:48 AM.

  • Peter Z likes this

#41 robin_astro

robin_astro

    Apollo

  • -----
  • Posts: 1,224
  • Joined: 18 Dec 2005

Posted 29 January 2021 - 03:26 PM

Does the .003 noise limit imply systematic errors of similar magnitude would generate an unacceptable error ?  eg the deviation of the spectrum from a true black  body curve ? That criterion could be harder to meet

 

Robin



#42 syam

syam

    Vostok 1

  • -----
  • topic starter
  • Posts: 188
  • Joined: 11 Jun 2009
  • Loc: Hamilton, Canada

Posted 30 January 2021 - 09:51 AM

Does the .003 noise limit imply systematic errors of similar magnitude would generate an unacceptable error ?  eg the deviation of the spectrum from a true black  body curve ? That criterion could be harder to meet

 

Robin

This is something harder to test. I'd need examples of real incandescent light spectra, made with a calibrated spectrometer, at different temperatures, to test this. I suspect my algorithm will still work, but there may be systematic shift in recovered temperatures. With calibrated real spectra one could try to quantify this shift.



#43 syam

syam

    Vostok 1

  • -----
  • topic starter
  • Posts: 188
  • Joined: 11 Jun 2009
  • Loc: Hamilton, Canada

Posted 31 January 2021 - 04:47 PM

I am having second thoughts about using tungsten bulbs for calibration. Apparently their spectrum is not as close to blackbody radiation as I hoped. Here I overlapped bulb spectra with corresponding blackbody spectra (they are all normalized to the same maximum brightness):

 

bulbs_blackbody2.png

 

 

 

As you can see, significant deviations, even if restricted to visible part (400-700 nm). And there is apparently enough of variations between bulbs to make creating a bulb model questionable.

 

Here is another, perhaps better, option: LEDs improved quite dramatically in recent years, with the ones used for photography (high CRI index) approaching spectrum of daylight. They all publish there "typical" spectra. I wouldn't be surprised that as long as one uses exactly the same current, individual spectra for such LEDs will not be too far off from the published one.

 

Here is one nice LED: Bridgelux SMD 2835 1W 9V Thrive.

 

The typical spectra:

 

SMD2835.png

 

The 5000K spectrum looks particularly nice. The biggest limitation: not usable below 415nm or so.

 

I am thinking of getting a few of these to experiment (and to see how much variations are between individual LEDs).

 

One can attach an LED to a stepper motor + Arduino so it can move in front of the slit in arc motions, maintaining the same distance to the slit, and always facing the slit. If motions are smooth and accurate, time integration should produce an accurate spectrum, without a need for a diffuser.

 

LED_spectrometer.png


Edited by syam, 31 January 2021 - 05:11 PM.


#44 Mark Lovik

Mark Lovik

    Lift Off

  • -----
  • Posts: 17
  • Joined: 09 Nov 2020

Posted 31 January 2021 - 06:01 PM

On your peak fitting algorithm -- you can notice a peak bias when the curves are asymmetrical.  The peak is consistently skewed when one side of the band is wider -- this is not what you want for emission spectroscopy.  

 

Since you spectra is oversamples (more points than the optics width -- governed by the slit), you have spectral smoothing.

 

One approach that was used in industry and academics for this type of data in the past

 

FFT interpolate - factor of 2 or more

Then quadratic fit of the top 3 points -- accuracy is high with less skew of sidebands 

Take the peak of the fit

 

Alternative

 

Pull off the baseline (fft low frequency removal - then fit) -- the wider tails are at a different frequency level.



#45 robin_astro

robin_astro

    Apollo

  • -----
  • Posts: 1,224
  • Joined: 18 Dec 2005

Posted 31 January 2021 - 06:34 PM

On your peak fitting algorithm -- you can notice a peak bias when the curves are asymmetrical.  The peak is consistently skewed when one side of the band is wider -- this is not what you want for emission spectroscopy.  

 

 

The problem is the poor choice of calibration lamp. This is what a good calibration lamp spectrum looks like.

 

calibration lamp.png

 

The lines are symmetric and are approximately gaussian. Any residual asymmetry is due to the spectrograph optics and would be reproduced on the sky so needs to be included in the calibration so a centroid measurement can be used to determine the line positions.   

 

Cheers

Robin



#46 syam

syam

    Vostok 1

  • -----
  • topic starter
  • Posts: 188
  • Joined: 11 Jun 2009
  • Loc: Hamilton, Canada

Posted 01 February 2021 - 09:33 AM

To be honest, I am amazed I am getting 0.25 nm std line accuracy (less than the pixel size) with my cheapo setup. Perhaps I'll try something better in the future. May be a simple parametrization of a skewed gaussian, or this FFT trick.

 

EDIT: I just tried to use a skewed gaussian fit instead of a symmetric one, and I do see some lines got slightly adjusted, but the std actually went slightly up - from 0.239879 nm (symmetric) to 0.242230 (skewed). I suspect at this point I am limited not by my processing, but by the inaccuracies of the lines measured in this source.


Edited by syam, 01 February 2021 - 12:36 PM.


#47 syam

syam

    Vostok 1

  • -----
  • topic starter
  • Posts: 188
  • Joined: 11 Jun 2009
  • Loc: Hamilton, Canada

Posted 03 February 2021 - 08:55 AM

I ordered 6 of Bridgelux SMD 2835 1W 9V Thrive LEDs, which have spectrum close to sun. I will be testing them with my spectrometer, hopefully they can make a great calibration target. I just got a bench power supply (30V, 5A), which will let me to set precisely the current through these LEDs - a must if trying to use them for  calibration.



#48 robin_astro

robin_astro

    Apollo

  • -----
  • Posts: 1,224
  • Joined: 18 Dec 2005

Posted 03 February 2021 - 12:34 PM

Bridgelux SMD 2835 1W 9V Thrive LEDs, which have spectrum close to sun.

I am very sceptical of this. They may have the same colour temperature as sunlight but that is not at all the same as having the same spectrum. Do you have a published spectrum ?

 

Cheers

Robin



#49 robin_astro

robin_astro

    Apollo

  • -----
  • Posts: 1,224
  • Joined: 18 Dec 2005

Posted 03 February 2021 - 12:41 PM

Also what evidence have you that supports your assumption that there is little spectral variability between individual samples of the same LED ?  A quick googles suggests this may well not be the case eg

https://www.led-prof...-on-performance

 

Cheers

Robin



#50 syam

syam

    Vostok 1

  • -----
  • topic starter
  • Posts: 188
  • Joined: 11 Jun 2009
  • Loc: Hamilton, Canada

Posted 03 February 2021 - 01:41 PM

Also what evidence have you that supports your assumption that there is little spectral variability between individual samples of the same LED ?  A quick googles suggests this may well not be the case eg

https://www.led-prof...-on-performance

 

Cheers

Robin

Definitely, it's all rather likely. I can just go by the claim of the manufacturer, who says

 

To quantify spectral matching, Bridgelux has defined a new term; Average Spectral Difference (ASD). ASD is calculated by measuring the absolute difference
between the LED spectrum and a natural light source spectrum at discrete wavelengths. These values are then averaged across different wavelength ranges
and reported as a percentage. Natural light is defined by the Black Body Curve for sources below 5000K or by the CIE Standard Illuminant D for sources of
5000K or above (D50 for example).

Bridgelux Thrive has an ASD between 3% and 7% for all color points (2700K – 6500K) across the typical LED wavelength range of 440-650nm. This includes a
very close spectral match, with an ASD of only 2% for some color point options, in the blue/cyan color range where most other LED light sources suffer. For
comparison purposes, standard 80, 90 and 98 CRI light sources have an ASD that is 2-6 times larger than Thrive over the same wavelength range and up to 20
times greater in the cyan wavelength range.

 

 

 

Which all sound very promising. And if they say the deviations from the natural light spectrum are small, this should also imply there is not a huge scatter between individual LEDs.

 

Regarding the sun-like spectrum - check my plot above. The yellow curve (5000K LED) looks the most natural, way better than typical (even photography-grade) LEDs, which have a giant blue peak around 450nm, followed by a very deep depression around 490nm. The Bridgelux Thrive LEDs have the fluctuations at a much more modest ~20% level.

 

I will be testing this - I just got 6 Bridgelux Thrive LEDs. I sent an email to Bridgelux asking for the spectrum as a table, and also about the spectral variability. Not likely they will answer, so I also used an excellent online tool https://automeris.io/WebPlotDigitizer/ to automatically read data from the yellow (5000K) curve in the published spectrum. Here it is:

 

SMD2835_spectrum.png

 

If anyone is interested - here is the table version of this spectrum: https://mashchenko.ca/pub/SMD2835.txt


Edited by syam, 03 February 2021 - 01:43 PM.



CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.


Recent Topics






Cloudy Nights LLC
Cloudy Nights Sponsor: Astronomics