Jump to content

  •  

CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.

Photo

Pixinsight Flats Issue

  • Please log in to reply
17 replies to this topic

#1 ruben1999

ruben1999

    Lift Off

  • *****
  • topic starter
  • Posts: 7
  • Joined: 25 Dec 2018

Posted 17 January 2020 - 11:37 AM

Hello everyone. As I was trying to process my recent image of a double star cluster, I came across a really annoying issue with my flats. The master flat frame only shows vignetting and dust motes. However, when I calibrate my light frames, the flats remove the dust motes and add a weird dark streak across my image. Im not sure what this is as it does not appear in any of my light frames. Please let me know if any of you know what this is and how to correct it. 
 
The screenshot attached includes my flat frame (bottom), my stacked subs before calibration (top left) and my stacked subs after calibration (top right). Screen Shot 2020-01-17 at 9.14.26 AM copy.jpg


Edited by ruben1999, 17 January 2020 - 12:22 PM.


#2 BenKolt

BenKolt

    Vanguard

  • *****
  • Posts: 2,074
  • Joined: 13 Mar 2013

Posted 17 January 2020 - 11:57 AM

Ruben:

 

I am unable to see the dark streak to which you are referring.  All the images look like they are unstretched, and that makes it difficult to view your issue.  You will want to first stretch the images in PI, then save to jpeg (or some other permitted format) to upload here.

 

Please post how you are calibrating your images, and don't spare the details!  Information on how you make master darks, master flats and then how you calibrate your light frames will be useful in helping you get to the bottom of this.

 

Thanks!

 

Best Regards,

Ben



#3 ruben1999

ruben1999

    Lift Off

  • *****
  • topic starter
  • Posts: 7
  • Joined: 25 Dec 2018

Posted 17 January 2020 - 12:30 PM

Ruben:

 

I am unable to see the dark streak to which you are referring.  All the images look like they are unstretched, and that makes it difficult to view your issue.  You will want to first stretch the images in PI, then save to jpeg (or some other permitted format) to upload here.

 

Please post how you are calibrating your images, and don't spare the details!  Information on how you make master darks, master flats and then how you calibrate your light frames will be useful in helping you get to the bottom of this.

 

Thanks!

 

Best Regards,

Ben

Hello Ben. I have stretched the images and uploaded the new one. Thanks for pointing it out.

 

The dark and bias frames were taken on the same night and same temperature as the light subs. For flats, I pull up a blank white screen on my iPad and prop it up against the scope. The adu I use for my flats is 13,000. Before calibrating my light frames, I calibrate my flat frames with my master bias and master dark. Then I stack the calibrated flat frames and proceed to calibrate my lights. When calibrating my lights, I uncheck the 'calibrate' box for the dark, bias and flat and set the optimization threshold all the way to zero. After calibration is complete is when the streak is added to the light frames.



#4 BenKolt

BenKolt

    Vanguard

  • *****
  • Posts: 2,074
  • Joined: 13 Mar 2013

Posted 17 January 2020 - 01:00 PM

Ruben:

 

OK, now I see what you're talking about.  Thanks for changing the uploads.

 

I can't tell from your description if you are double subtracting the bias signal from your lights and/or flats.  Please consider doing to following to check whether or not that's what's going on here.

 

I recommend that you do the following as a test:

 

(1) Make a master dark by combining (using ImageIntegration) your uncalibrated darks.  Hopefully your darks are of the same temperature, exposure, etc. as your lights as this step may not work well.  If they are not matched well, this is something that you can easily correct by taking some more dark frames to match the lights.

 

(2) Make a master flat by first subtracting just the master bias from each flat frame (using ImageCalibration).  Make sure you uncheck any calibration or optimization boxes.  Then make the master flat (using ImageIntegration).

 

(3) Now use ImageCalibration, again with all those calibration and optimization boxes unchecked, to calibrate the lights.  Here you include just the master dark and master flat, no bias.  Combine these calibrated lights (using ImageIntegration) and see if that does a better job.

 

Since the bias signal and matching dark current are both included in the master dark that I've described, then you know that you are subtracting both of those just one time from the lights.  And you are just subtracting the bias signal from the flats, and here we are assuming for now that any dark signal in the flats is negligible.  Following these steps will check to see if your original process was double subtracting something or otherwise not doing what you thought it was.  It you are still unsatisfied, there is more to check, and others will likely chime in with more questions and suggestions.

 

Good luck, and please post your results of this.  Feel free to ask any questions if you don't understand my recommendations.

 

Best Regards,

Ben

 

 


  • pfile and elmiko like this

#5 BenKolt

BenKolt

    Vanguard

  • *****
  • Posts: 2,074
  • Joined: 13 Mar 2013

Posted 17 January 2020 - 01:26 PM

I forgot to ask you what camera you are using.  That should have been my first question.  What temperature were you cooling, exposure time, gain settings (if you have variable gain), filter used, etc.?  Thanks.

 

Ben



#6 ruben1999

ruben1999

    Lift Off

  • *****
  • topic starter
  • Posts: 7
  • Joined: 25 Dec 2018

Posted 17 January 2020 - 01:36 PM

I am using the ASI 1600MM Pro. The lights, darks and bias were shot with the sensor at 25 below ambient (20F). My exposures were 30sec x 90 at gain 75 offset 15 for all three RGB filters. I just went reprocessing following your steps and still ended up with the dark streak.Screen Shot 2020-01-17 at 10.33.25 AM.jpg



#7 BenKolt

BenKolt

    Vanguard

  • *****
  • Posts: 2,074
  • Joined: 13 Mar 2013

Posted 17 January 2020 - 01:48 PM

OK, Reuben.  Thanks for following my steps.  It was worth checking as double subtraction is a very common mistake.  I do not have experience with your camera, so I'll step back now and let CMOS users chime in.  I am aware that many of these cameras do not work well with all the automated optimization and calibration options checked in PI.

 

Another possibility is that your filter may have been askew, and that can account for the oddly curved pattern you see.  I can sort of see it in your flat and maybe your uncalibrated light, but I'm not sure.  You can squeeze the contrast even more to see where that pattern shows up.  Is it really there in both the uncalibrated light and flat?  If it's in one but not the other, then that's very telling.  If it's due to some other reason, it may show up in the dark as well?  Anyway, those are quick checks you can do.

 

Good luck!

 

Ben



#8 ruben1999

ruben1999

    Lift Off

  • *****
  • topic starter
  • Posts: 7
  • Joined: 25 Dec 2018

Posted 17 January 2020 - 03:04 PM

I believe it might be the filters. This only happens with my RGB filters and not with my narrowband filters which is why I've just been sticking to narrowband. I tried RGB imaging about a year ago with M101 and got the same result. I believe it might be a light leak somewhere in my imaging train but I'm not sure why it would show up on only the RGB filters and not the narrowband. Another forum said my flats could be over correcting my light frames which I believe might be the case. If this were the case, how could I correct this? Higher / lower adu for flats? What settings can I change in pixinsight during processing to avoid this?



#9 BenKolt

BenKolt

    Vanguard

  • *****
  • Posts: 2,074
  • Joined: 13 Mar 2013

Posted 17 January 2020 - 03:40 PM

Ruben:

 

You ought to be able to check the seating of the filters easily, even if you don't want to take the camera and filter wheel off the scope.  Look down the scope while you change filters and make sure they center properly over the camera sensor.  You can use a flashlight if necessary.  It's odd that you see this pattern only with RGB.

 

The light leak suggestion is a good one.  That's another you can check out with a flashlight.  Take frames while moving the flashlight around, preferably when it's dark.  You should be able to see where the light comes in, and then plug the leak with tape.  I use gaffer's tape.  I also have some aluminum tape, although the brand I got leaves a lot of sticky mess behind.

 

Good luck!

 

Ben



#10 james7ca

james7ca

    Cosmos

  • *****
  • Posts: 7,875
  • Joined: 21 May 2011
  • Loc: San Diego, CA

Posted 17 January 2020 - 08:54 PM

It's probably best NOT to use dark field scaling or optimizations when using a CMOS camera. This means you can't use your master dark to calibrate your flats, unless the flat exposure is the same as your lights and master darks.

 

So, you may need to take flat darks using the same exposure setting for the flat dark as for the flats. You may also want to omit bias files for both your lights and flats since the bias signal is already in your darks and if you don't use dark field scaling or optimizations you really don't need bias files.

 

Beyond that, it's possible that your flats didn't really have a uniformly illuminated field. I also use an iPad for my flats but I take fairly extreme measures to make certain that the field is well centered and I only take flats when the scope is in a dark room (or at night in my car port with extra shielding around the iPad). I also use a piece of flashed opal glass between the scope and the iPad and I rotate the iPad and the opal glass in at least three different positions -- two horizontal and one vertical -- while taking the sequences. Lastly, I very rarely use exposures that are less than one second long to try and average any cycling or flickering in the iPad display (I have a selection of flat grayscale target images on the iPad to vary its output).

 

The other thing to remember is that after using flats you will likely be able to detect variations in the original signal that were largely invisible before (or without) calibration. This is usually a good thing as it may allow you to detect fainter signals, but it also means that any artificial variations that aren't corrected for by the flats will just become that more noticeable.

 

All that said, I've never seen a really great improvement in my images that I could attribute to flats. But, I keep my sensors and filters clean and I use systems that have very little vignetting, so there isn't much for a flat to improve upon. As hard as I've looked I've never seen any improvement from the normalization that a flat is supposed to do on an individual pixel basis, so if it does anything it just corrects for dust and vignetting which I have very little to begin with.

 

However, conditions vary with sky brightness (light pollution) and the length of your integrations/exposures, so while I don't use flats that often you may find them more useful under dark skies or with long integrations and you certainly will want flats if you have lots of dust or vignetting. 


  • richorn likes this

#11 richorn

richorn

    Messenger

  • -----
  • Posts: 417
  • Joined: 17 Apr 2019
  • Loc: Rancho Palos Verdes. CA

Posted 18 January 2020 - 12:47 AM

The only thing I can add to this is the the ASI1600 does NOT like bias files.  Use darkflats to calibrate the flats, not bias

 

So, to continue with the above theme, take darks that match the gain/offset/exposure of your light frames, and take darkflats that match the gain/offset/exposure of the flats.

 

Make masters of each (BenKolt has that methodology down, DON'T use dark scaling) and use the master dark and master flat to calibrate the lights.


Edited by richorn, 18 January 2020 - 12:55 AM.


#12 jhayes_tucson

jhayes_tucson

    Fly Me to the Moon

  • *****
  • Posts: 7,383
  • Joined: 26 Aug 2012
  • Loc: Bend, OR

Posted 18 January 2020 - 12:58 AM

You've gotten some good suggestions here but I'll add my two cents.  The most likely source of the problem is stray light.  Remember that vignetting is a multiplicative modulation of the irradiance in the image plane so it gets removed by dividing the image by the master flat.  Stray light is additive so it won't calibrate out of each image.  It doesn't matter whether the stray light is in the subs or in the flat data.  The effect will be the same.  In your case, it looks like you may have a stray in your flat data.  A lot of astro-cameras are not perfectly light-tight so it's not uncommon to get light leaking into the camera when the flats are taken.  How are you taking your flats?

 

John

 


  • pfile, jdupton and t-ara-fan like this

#13 ruben1999

ruben1999

    Lift Off

  • *****
  • topic starter
  • Posts: 7
  • Joined: 25 Dec 2018

Posted 18 January 2020 - 06:42 PM

Hello everyone. Thank you for all the suggestions. 

 

I have discovered a light leak in my imaging train coming from the Starizona filter drawer system. I have been using this filter drawer in my imaging train for about a year now and I had absolutely no idea. I have removed the filter drawer and swapped in my filter wheel which just came in today. 

 

I will recapture data on this target tonight and redo my flats and darks. As for the dark flats that have been suggested, if I am understanding correctly, I am to NOT use bias at all during calibration and instead use darks with the same exposure time as my flats?


  • PeteM, jhayes_tucson and richorn like this

#14 BenKolt

BenKolt

    Vanguard

  • *****
  • Posts: 2,074
  • Joined: 13 Mar 2013

Posted 18 January 2020 - 07:38 PM

Hello everyone. Thank you for all the suggestions. 

 

I have discovered a light leak in my imaging train coming from the Starizona filter drawer system. I have been using this filter drawer in my imaging train for about a year now and I had absolutely no idea. I have removed the filter drawer and swapped in my filter wheel which just came in today. 

 

I will recapture data on this target tonight and redo my flats and darks. As for the dark flats that have been suggested, if I am understanding correctly, I am to NOT use bias at all during calibration and instead use darks with the same exposure time as my flats?

Good catch there, Ruben!  I hope the change out will help.

 

Yes, you've got it right.  When calibrating flats with a master dark flat of matching exposure, temp, gain, etc., and doing the same with a master dark that matches the lights in exposure, temp, gain, etc., you'll see that there is no bias used at all.  This is my preferred method of calibrating lights from my CCD's. 

 

Best Regards,

Ben



#15 ruben1999

ruben1999

    Lift Off

  • *****
  • topic starter
  • Posts: 7
  • Joined: 25 Dec 2018

Posted 19 January 2020 - 08:49 PM

I recaptured data on this target last night and got the same results. Even with the new darks and flats. Could it be that I have the filters installed in the wrong orientation and it is causing a reflection on the sensor? I use the Astronomik RGB Type 2c 36mm mounted filters. 



#16 jhayes_tucson

jhayes_tucson

    Fly Me to the Moon

  • *****
  • Posts: 7,383
  • Joined: 26 Aug 2012
  • Loc: Bend, OR

Posted 20 January 2020 - 01:16 AM

No, the orientation of the filters won't make any difference.  Turn your OCD way up and be REALLY careful about chasing down any possible stray light.  I'll bet you a beer that you've still got a light leak or a stray reflection somewhere.  One other thing to try is to pull the camera and look backwards through he system with your eye roughly in the same position as the sensor.  Point the scope out a window at a bright scene in the daytime and look for strays from extension tubes or other apertures.  Glancing reflections from inner surfaces that aren't properly baffled or blackened can cause this same problem.

 

John



#17 ruben1999

ruben1999

    Lift Off

  • *****
  • topic starter
  • Posts: 7
  • Joined: 25 Dec 2018

Posted 20 January 2020 - 04:40 PM

I believe I have found the reflection. I removed the camera and one of the filters and noticed a significant amount of light reflecting off my field flattener (William Optics Flat 73A). How can I fix this? I have 3 clear nights coming up and would like to get this done as soon as possible.

Attached Thumbnails

  • IMG_4726.jpg


#18 jhayes_tucson

jhayes_tucson

    Fly Me to the Moon

  • *****
  • Posts: 7,383
  • Joined: 26 Aug 2012
  • Loc: Bend, OR

Posted 20 January 2020 - 06:35 PM

Yep, that could be a problem; though I'm still not sure that's THE problem.  The best approach is to use flat black paint and/or black flocking to reduce those strays.  One other suggestion:  I take my flats in the dead of night to minimize the chances of a light leak turning into a problem.  If you aren't already, take your flats in the dark.

 

John


  • scadvice likes this


CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.


Recent Topics






Cloudy Nights LLC
Cloudy Nights Sponsor: Astronomics