13 replies to this topic

[calypsob]

This is not the first time I have had this problem, but the last time it happened I decided to put off processing the data for another day.   Now that it has happened again it is time to crack down.   None of my individual subs contain the red spots that you see below.  Also on this image there are a few blue spots as well.  I am not sure why these are appearing.  If I crank the Linear high fit fader waay down to 1.00, 90% of the red dots go away, but the faint dust becomes damaged pretty badly as well.  I really need to figure out why this is happening.  I have tried windsorized sigma as a rejection algorithm as well and no luck.  I cannot process the image any further until I can eliminate these spots.  Has anyone else come across this scenario? I saw one thread on the Pi forum recommending that the user decrease the clamping threshold in image registration but unfortunately this did nothing for me.  

 

 

 

This is the same image only with ABE applied on the right 

 

Edited by calypsob, 25 June 2017 - 08:21 PM.

[entilza]

Wes, were you dithering and did you calibrate with darks?

[Antonio Spinoza]

Are these done with a mono camera and filters, or a color sensor?

[Jon Rista]

Looks like hot pixels, likely demosaiced hot pixels. You should probably run CosmeticCorrection (check CFA if it is a color sensor) and clean that stuff up before you demosaic, register and integrate.

[Antonio Spinoza]

Wes,

I"ve seen this with my full spectrum 550D.  Don't really know what causes it.

If it's a color camera and you are using the raw CR2 files, then it could be what Jon Rista suggested above:  Hot pixels in the raw data being averaged across pixels during CFA interpolation, causing red or blue spots that are larger than a pixel.  I'm wondering if it could be one of the following two things:

 

1.)  Hot pixels that are coming up consistently that are being spread out over more than four pixels due to the registration of multiple subs (they will be aligned on legitimate stars that spread over multiple pixels and have some variability, which would move the hot pixels around relative to each other from sub to sub).  In this case, cosmetic correction of the raw subs should knock those down.

 

2.) I doubt that this would be the case, but are you registering raw, rather than debayered subs prior to integration?   

[pfile]

changing the clamping threshold is sometimes not enough to get rid of ringing around hot pixels with the default Lancosz-3 algorithm. try switching to bicubic spline and see what happens.

 

rob

[calypsob]

Wes, were you dithering and did you calibrate with darks?

Yes but I was imaging multiple cameras on the same mount and dithering at 135mm scale.  I wonder if that was to small to be of any benefit to the 50mm scale. 

 

Looks like hot pixels, likely demosaiced hot pixels. You should probably run CosmeticCorrection (check CFA if it is a color sensor) and clean that stuff up before you demosaic, register and integrate.

I will give it a try.  My problem is that I cannot see these in the original frames.  I can see some hot and cold pixels, but they are removed correctly.  Maybe these are extremely faint hot pixels.  I am going to equalize one of the .cr2 files in PS and see if I can cross reference the red spots in my pi output integration.

 

Wes,

I"ve seen this with my full spectrum 550D.  Don't really know what causes it.

I'm hoping to find a solution today! I think I remember seeing these in one of your images that you sent me a while ago, now that you mention it.  

 

 

Wes,

I"ve seen this with my full spectrum 550D.  Don't really know what causes it.

If it's a color camera and you are using the raw CR2 files, then it could be what Jon Rista suggested above:  Hot pixels in the raw data being averaged across pixels during CFA interpolation, causing red or blue spots that are larger than a pixel.  I'm wondering if it could be one of the following two things:

 

1.)  Hot pixels that are coming up consistently that are being spread out over more than four pixels due to the registration of multiple subs (they will be aligned on legitimate stars that spread over multiple pixels and have some variability, which would move the hot pixels around relative to each other from sub to sub).  In this case, cosmetic correction of the raw subs should knock those down.

 

2.) I doubt that this would be the case, but are you registering raw, rather than debayered subs prior to integration?   

 

I am using BPP script so I am not exactly sure if it debayers these before integrating.  Is there any way to control this? 

 

changing the clamping threshold is sometimes not enough to get rid of ringing around hot pixels with the default Lancosz-3 algorithm. try switching to bicubic spline and see what happens.

 

rob

Going to give it a try, thanks!

[pfile]

oh - as far as BPP is concerned, the checkbox for "CFA images" is what controls whether or not the debayering step is inserted into the flow.

 

so as long as that's checked, your frames should be debayered. but i think BPP should save all the intermediate files for you to see:

 

_c : calibrated

_c_d : debayered, calibrated frames (what gets registered)

_c_d_r: registered, debayered calibrated frames (what gets stacked)

 

i guess if you really wanted to see if there are residual hot pixels, you could manually stack the _c or _c_d frames and see if you see anything obvious. or maybe blink thru them and see if only some of the _c_d frames have hot pixels and others don't. this is possible because during calibration the dark frame gets scaled by different amounts, so the hot pixel subtraction is more and less effective depending on the scaling factor.

 

rob

[calypsob]

oh - as far as BPP is concerned, the checkbox for "CFA images" is what controls whether or not the debayering step is inserted into the flow.

 

so as long as that's checked, your frames should be debayered. but i think BPP should save all the intermediate files for you to see:

 

_c : calibrated

_c_d : debayered, calibrated frames (what gets registered)

_c_d_r: registered, debayered calibrated frames (what gets stacked)

 

i guess if you really wanted to see if there are residual hot pixels, you could manually stack the _c or _c_d frames and see if you see anything obvious. or maybe blink thru them and see if only some of the _c_d frames have hot pixels and others don't. this is possible because during calibration the dark frame gets scaled by different amounts, so the hot pixel subtraction is more and less effective depending on the scaling factor.

 

rob

Interesting.  I had dark frame scaling turned off and wonder now if that is a negative factor.  

[calypsob]

oh - as far as BPP is concerned, the checkbox for "CFA images" is what controls whether or not the debayering step is inserted into the flow.

 

so as long as that's checked, your frames should be debayered. but i think BPP should save all the intermediate files for you to see:

 

_c : calibrated

_c_d : debayered, calibrated frames (what gets registered)

_c_d_r: registered, debayered calibrated frames (what gets stacked)

 

i guess if you really wanted to see if there are residual hot pixels, you could manually stack the _c or _c_d frames and see if you see anything obvious. or maybe blink thru them and see if only some of the _c_d frames have hot pixels and others don't. this is possible because during calibration the dark frame gets scaled by different amounts, so the hot pixel subtraction is more and less effective depending on the scaling factor.

 

rob

Interesting.  I had dark frame scaling turned off and wonder now if that is a negative factor.  

[pfile]

with a DSLR it might well be better to turn dark scaling off, since the camera firmware plays so many tricks with dark current subtraction. as far as hot pixels go, the best calibration will be had with dark scaling turned off...

 

rob

[calypsob]

with a DSLR it might well be better to turn dark scaling off, since the camera firmware plays so many tricks with dark current subtraction. as far as hot pixels go, the best calibration will be had with dark scaling turned off...

 

rob

ok good.  Im going to leave it alone then.  

[calypsob]

Well I do not have a screen shot right now, but I figured out how to zap the red dots.  I had to adjust the high low linear fits rejection settings and also the same rejection settings in my dark frames to dial out everything. I also switched to bicubic spline in registration per Robs advise, so I need to run everything one more time to figure out exactly what fixed the issue. For now, I have zapped all of the red dots!