Jump to content

  •  

CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.

Photo

Serious Flat Frame issues - ASI1600MM-C - What am I doing wrong here?!

  • Please log in to reply
59 replies to this topic

#1 cfosterstars

cfosterstars

    Mercury-Atlas

  • *****
  • topic starter
  • Posts: 2,880
  • Joined: 05 Sep 2014
  • Loc: Austin, Texas

Posted 12 October 2017 - 09:19 PM

Hi all,

I am relative new to using a mono camera with a filter wheel. I have been using a Canon 6D for several years and understand the basics for flat frame calibration. For my 6D, flats were very easy and worked (and continue to work when I use that camera). I would either use my home made light boxes or use the tee shirt sky flat method. I would take flats in AV mode in BYEOS - very easy to take, process and the results for the light frames were good – only improved the stacked light frame image as expected.

Well I am not in Kansas anymore. I recently got an ASI1600MM-C with a 7 position 36mm filter wheel. I have LRGB and Ha, OIII and SII filters from ZWO in the slots. The flat frames have been a big problem in my results. I have used SGP flats calibration wizard and it was rather strait forward. I also recently got a FLAT-MAN from optec and it seem to work well. The calibration wizard, with the highest illumination intensity was 0.4 sec for L, 0.9s for R, G, and B, 7s for OIII, 10s for Ha and 12s for SII to get the SGP pixel level to 25000 with the histogram in the middle. I use the same camera temperature, gain, offset and USB settings that I use for the lights, darks and bias frames. I have process the flats using Pixisight using a dark library to create the masterdark with between 30-60 dark frames per temperature, gain and exposure time and a bias master of 200 frames for each gain condition and temperature. I have checked each bias frame, dark frame and flat frame using blink to check for any banding and do not see anything that look suspicious. I stretched the masterdark and masterbias frames and they look fine with just the expected noise signals.

Here is the stretched bias and dark master frames:

 

Master Bias

Master Bias Frame Stretched.jpg

 

Master Dark 180s

Master Dark 180s Stretched.jpg

The calibration and stacking processes of the flat frames in Pixinsight seems fine. I am following the Tutorial on YouTube by Richard Bloch which matches the procedure in Warren Keller's book well. My latest unstretched the flats look good. This is the LUM Masterflat fro PI:

 

MasterFlat_0.09sec_1x1_-12C_Lum_c.jpg

 

At this point you might expect no issues. I wish.



#2 cfosterstars

cfosterstars

    Mercury-Atlas

  • *****
  • topic starter
  • Posts: 2,880
  • Joined: 05 Sep 2014
  • Loc: Austin, Texas

Posted 12 October 2017 - 09:26 PM

My first imaging test with the camera showed that the camera sensor was clearly dirty and the artifacts in the image were due to streaks and dirt on the sensor surface. This would not flat frame out of the image. Here is the examples of the issues with the first sets of flats:

 

This is the L filter:

Uncleaned LUM master flat stretched.jpg

 

This is the Ha filter:

Uncleaned Ha master flat stretched.jpg

 

You can see the streaks that I could not remove by flat frames in the final image of NGC7000: You may need to click on the image to see the streaks in the upper right hand quadrant. 

NGC7000 LHaSIIOIII SM

 

I cleaned the sensor as I have done with my DSLR previously and this cleaned up the flats considerably.

 



#3 cfosterstars

cfosterstars

    Mercury-Atlas

  • *****
  • topic starter
  • Posts: 2,880
  • Joined: 05 Sep 2014
  • Loc: Austin, Texas

Posted 12 October 2017 - 09:42 PM

After cleaning, this is the L filter:

cleaned LUM master flat stretched.jpg

And this is the cleaned Ha filter:

cleaned Ha master flat stretched.jpg

 

However, even with these better flats, the results are very much less than stellar (pun intended). This is NCG7380, the Wizard Nebula. I had to crop severely due to flat fielding artifacts.

 



#4 rockstarbill

rockstarbill

    Cosmos

  • *****
  • Posts: 9,401
  • Joined: 16 Jul 2013
  • Loc: Snohomish, WA

Posted 12 October 2017 - 09:46 PM

Junk on the sensor, I would contact ZWO. 



#5 cfosterstars

cfosterstars

    Mercury-Atlas

  • *****
  • topic starter
  • Posts: 2,880
  • Joined: 05 Sep 2014
  • Loc: Austin, Texas

Posted 12 October 2017 - 10:07 PM

It is clear that the flat correction and artifacts from the flats are still in my stacked light frames. This is different for each filter and resembles the structure of the highly stretched flat for that filter. 

 

This is the stacked Ha and SII light frames:

 

SII:

NGC7380 Master SII STR
 

Ha

NGC7380 Master Ha STR

 

Outside of the expected stacking artifacts, these stacked lights from the NB filters show signs of what looks like vignetting:

 

I have a very hard time believing that this is vignetting since I don’t see it in the LRGB filter flats:
This is the L flats:

 

cleaned LUM master flat stretched.jpg

 

1)    Could the issues with the SII and Ha filter due to have the filter mounted in the filter wheel in the wrong orientation?
I used the information of the ZMO web site to determine what I the sensor side of the filter and what is the telescope side, but I admit that it was very hard to tell which way was correct. It was not nearly as easy to tell as the web site seemed to indicate. However, this does not explain why the flats don’t calibrate out the optical train from the light frames.

 

I have some theories as to what is wrong, but would really like some suggestions:
1)    How do you determine the focus for each filter for taking flats? I have taken flats the next day. I have not touched to focus, but that was from the last filter that I used. If the filters each have different focus points and you are doing this during the day, how do you reset the focus position for each filter? I have not used the focus position setting in SGP for these individual filters. There is a small focus difference between filters but it is not big. I have a hard time believing that this is the root cause but I have not had experience with how sensitive the flat frame calibration should be to focus.

 

2)    If the focus offsets between filters is needed, how do you do this in practice? I hear that many do there flats against the dawn sky. Do you reset the focus between filters? If so, how do you do this without stars to focus on. Do the focus positions in SGP take this into account and give you at least a better “focus” for each filter?

 

3)    Even if it is focus, the “vignetting” for the SII and Ha filters should not be that focus dependent. The dust circles should be more sensitive, but they don’t seem to be contaminating the images. Why are these “vignetting” artifacts not flat frame calibrating out?

 

4)    Based on lost of reading of posts on this forum, I am using a of 76 with an offset of 40 at -15C with USB set to 50. With a 180s exposure, the histogram is still very tight against the left edge and not at all like I would expect from my DSLR experience. The stretched images show good structure and do not look under exposed. Does this look right?

 

Any suggestions on where to go from here would be very welcome.


Edited by cfosterstars, 12 October 2017 - 11:18 PM.


#6 Sean13

Sean13

    Viking 1

  • -----
  • Posts: 513
  • Joined: 17 Nov 2012
  • Loc: North Platte, Nebraska

Posted 12 October 2017 - 11:13 PM

Interesting. This makes me want to check my individual frames in blink now. I got the same camera 2 weeks ago with the 1.25" filters and wheel instead. I did have to change my flat process from how I did with my DSLR, but not by much. I am using a white EL panel similar to your flat fielding device but homemade. With the DSLR I did not use anything over the end of the scope, just put the panel on, set to AV-flat, and went to town with perfect results. On the ASI1600 I noticed some similar, but not the same, results as your flats with the same method. It was also hard to turn the brightness down to a level to be able to shoot a minimum 0.2 second exposure as I've herd these camera's do not do well under 0.2 seconds. To accomplish this on the LRGB set I had to add a tshirt to the end of the scope in front of my EL panel. This removed the artifacts I was seeing, and gave me the length of time needed for exposures. I would try added a tshirt between your flat device and scope, just as a test to see exactly where the artifacts are coming from. Then I would rotate the camera and filterwheel to the scope and see what moves, again to determine where exactly they are coming from. This won't solve you issue, but should give some more insight.

 

Is this the same scope you photograph(ed) with your DSLR on? I'm assuming you saw none of these issues with the other camera on the same scope, correct?

 

As far as I know, focus point is not extremely critical. You want your flats to be near the same focus you took your lights at, but the small offsets for different filters should be insignificant to flats. You don't focus flats.

 

Are you using dark flats (or flat darks depending on how you want to say it) as an alternative to bias frames now that you are taking 10+ second flats? I started doing this with much improved results in vignetting and amp glow removal in the calibration process.

 

I do notice slight vignetting in my flats with the 1.25" mounted filters, but it calibrates out no problem. I would think the 36mm should have no vignetting issues. It may be amp glow causing this, just going out on a limb here?


Edited by Sean13, 12 October 2017 - 11:17 PM.

  • RedLionNJ likes this

#7 wenjha

wenjha

    Vendor

  • *****
  • Vendors
  • Posts: 1,089
  • Joined: 24 Sep 2008
  • Loc: SuZhou China

Posted 13 October 2017 - 09:01 AM

this seems some light link from the edge of the filter

a simple way to fix:

you can exchange the order of connection with the EFW to avoid this

I mean use the camera to connect with the opposite side of the EFW 


Edited by wenjha, 13 October 2017 - 09:03 AM.

  • kingjamez likes this

#8 cfosterstars

cfosterstars

    Mercury-Atlas

  • *****
  • topic starter
  • Posts: 2,880
  • Joined: 05 Sep 2014
  • Loc: Austin, Texas

Posted 13 October 2017 - 11:07 AM

this seems some light link from the edge of the filter

a simple way to fix:

you can exchange the order of connection with the EFW to avoid this

I mean use the camera to connect with the opposite side of the EFW 

I am sorry, but I dont understand what you mean: "you can exchange the order of connection with the EFW to avoid this. I mean use the camera to connect with the opposite side of the EFW" Are you taking the physical side of the filter wheel or the order of connection electrically to the filter wheel? Do you think that the filter position is not centered over the camera sensor and light is leaking through the filter wheel around the filters or from the outside into the camera?



#9 cfosterstars

cfosterstars

    Mercury-Atlas

  • *****
  • topic starter
  • Posts: 2,880
  • Joined: 05 Sep 2014
  • Loc: Austin, Texas

Posted 13 October 2017 - 11:33 AM

Interesting. This makes me want to check my individual frames in blink now. I got the same camera 2 weeks ago with the 1.25" filters and wheel instead. I did have to change my flat process from how I did with my DSLR, but not by much. I am using a white EL panel similar to your flat fielding device but homemade. With the DSLR I did not use anything over the end of the scope, just put the panel on, set to AV-flat, and went to town with perfect results. On the ASI1600 I noticed some similar, but not the same, results as your flats with the same method. It was also hard to turn the brightness down to a level to be able to shoot a minimum 0.2 second exposure as I've herd these camera's do not do well under 0.2 seconds. To accomplish this on the LRGB set I had to add a tshirt to the end of the scope in front of my EL panel. This removed the artifacts I was seeing, and gave me the length of time needed for exposures. I would try added a tshirt between your flat device and scope, just as a test to see exactly where the artifacts are coming from. Then I would rotate the camera and filterwheel to the scope and see what moves, again to determine where exactly they are coming from. This won't solve you issue, but should give some more insight.

 

Is this the same scope you photograph(ed) with your DSLR on? I'm assuming you saw none of these issues with the other camera on the same scope, correct?

 

As far as I know, focus point is not extremely critical. You want your flats to be near the same focus you took your lights at, but the small offsets for different filters should be insignificant to flats. You don't focus flats.

 

Are you using dark flats (or flat darks depending on how you want to say it) as an alternative to bias frames now that you are taking 10+ second flats? I started doing this with much improved results in vignetting and amp glow removal in the calibration process.

 

I do notice slight vignetting in my flats with the 1.25" mounted filters, but it calibrates out no problem. I would think the 36mm should have no vignetting issues. It may be amp glow causing this, just going out on a limb here?

I check every frame with blink just to be sure. It was a pain, but it rules some stuff out. I can set the brightness lower for my LUM and RGB filters to get them longer into the 1-2 second level I think. I will give it a try and add the t-shirt. I dont think the artifiacts are due to the way I am doing the flats, but it is east to check. I can also try the camera rotation since that is not hard either. My issue is not with the artifacts accept for the "vignetting" for lack of a better term.

 

I have run the OTA with my Canon 6D many times and did not see any problems with flat corrections. 

 

I hope that the subtle delta in focus point between filters would not make any real difference. If so, then it would be very hard and tedious to take flats and you could not do them during the day without an absolute encoder with no slip or backlash in the focuser.

 

I have been using flats with bias and dark frame calibration. For flat darks, I can try that if I can get the exposure time up a bit or uses scaled darks. I have read in other threads that you should not use bias frames for the ASI1600MM camera due to the issues with very short exposures through the ASCOM driver. I think that this will be fixed when SGP upgrades to use the native driver for the camera.

 

I am pretty sure a that the "vignetting" is not due to AMP glow. It is in the wrong place and you can see it in the master dark above. Maybe the filter wheel is not calibrated and can be reset or the filters are in upside down. That was my theory. Since I see it only on some filters and not all filters, that says it isolated to the filter wheel and is not part of the optical system. Since the LUM does not show the issue and only some filter do, I suspect the light leakage or calibration issue. At least I have some things to try.



#10 kingjamez

kingjamez

    Vanguard

  • *****
  • Posts: 2,310
  • Joined: 03 Oct 2006
  • Loc: Fairfax, VA

Posted 13 October 2017 - 11:45 AM

 

this seems some light link from the edge of the filter

a simple way to fix:

you can exchange the order of connection with the EFW to avoid this

I mean use the camera to connect with the opposite side of the EFW 

I am sorry, but I dont understand what you mean: "you can exchange the order of connection with the EFW to avoid this. I mean use the camera to connect with the opposite side of the EFW" Are you taking the physical side of the filter wheel or the order of connection electrically to the filter wheel? Do you think that the filter position is not centered over the camera sensor and light is leaking through the filter wheel around the filters or from the outside into the camera?

 

He means screwing the camera to the opposite side of the filter wheel... easy way to check for reversed filters and would put the metal baffle of the filter wheel on the other side.

 

I have a similar, but not as pronounced issue with my OIII filter, but not my LRGB, HA, and S2.. I'm wondering if it's a reversed filter issue and I'll try Sam's test as soon as I get home.

 

-Jim



#11 cfosterstars

cfosterstars

    Mercury-Atlas

  • *****
  • topic starter
  • Posts: 2,880
  • Joined: 05 Sep 2014
  • Loc: Austin, Texas

Posted 13 October 2017 - 01:08 PM

 

 

this seems some light link from the edge of the filter

a simple way to fix:

you can exchange the order of connection with the EFW to avoid this

I mean use the camera to connect with the opposite side of the EFW 

I am sorry, but I dont understand what you mean: "you can exchange the order of connection with the EFW to avoid this. I mean use the camera to connect with the opposite side of the EFW" Are you taking the physical side of the filter wheel or the order of connection electrically to the filter wheel? Do you think that the filter position is not centered over the camera sensor and light is leaking through the filter wheel around the filters or from the outside into the camera?

 

He means screwing the camera to the opposite side of the filter wheel... easy way to check for reversed filters and would put the metal baffle of the filter wheel on the other side.

 

I have a similar, but not as pronounced issue with my OIII filter, but not my LRGB, HA, and S2.. I'm wondering if it's a reversed filter issue and I'll try Sam's test as soon as I get home.

 

-Jim

Thanks Jim,

 

I will give it a go. I think that reversed filters is the only reasonable explanation for differences in the flat for the different NB filters. They should basically look the same and they dont. 



#12 cfosterstars

cfosterstars

    Mercury-Atlas

  • *****
  • topic starter
  • Posts: 2,880
  • Joined: 05 Sep 2014
  • Loc: Austin, Texas

Posted 13 October 2017 - 01:11 PM

I also realize the I did not post the NGC7380 final image with the new cleaned sensor. This is my best image so far with the ASI1600MM-C:

 

NGC7380 WIZARD NEBULA HSO Narrow band

  • happylimpet and Mryak like this

#13 Sean13

Sean13

    Viking 1

  • -----
  • Posts: 513
  • Joined: 17 Nov 2012
  • Loc: North Platte, Nebraska

Posted 13 October 2017 - 01:23 PM

You should use bias frames, however you should take them at 0.2 seconds exposure. Since I started using dark flats, I quit using bias as my bias signal is in the dark flats as well and thus removed at the same stage a bias would be. Much happier results with dark flats instead. I did not see the issues the same as you are seeing however.


  • happylimpet likes this

#14 Jared

Jared

    Fly Me to the Moon

  • *****
  • Moderators
  • Posts: 6,846
  • Joined: 11 Oct 2005
  • Loc: Piedmont, California, U.S.

Posted 13 October 2017 - 07:02 PM

1) As mentioned, others have found uneven performance in bias frames if the exposure duration is below 0.2s, so use that value for your flats

2) Others have also found issues when trying to scale dark frames, so avoid doing that

3) This means that the best approach is probably to either:

    a) Build a master bias based on 0.2s exposures; bias subtract flats; use a master dark taken at your exposure duration with the same

        cooling and gain settings as the lights, and do not bias subtract the darks (no need to if you aren't trying to scale them)

 

        or 

 

    b) Don't bother with bias frames at all; use a master dark taken at your exposure duration with the same cooling and gain settings

        as the lights; rather than bias subtracting your flats, apply flat darks them (which will work like a bias frame, but with the added

        potential benefit of addressing some hot pixels from long-ish flat frame exposures.

4) You do have a couple spots on the sensor itself (or, more likely, on the cover glass on the outside of the CCD chamber) which appear to have been introduced when you did your cleaning.  Did you clean just the filters, or did you clean the CCD chamber?  Frankly, this is pretty minor and should calibrate out just fine.

5) Maybe someone knows something I don't, but I'm not aware of filter orientation being relevant for any of the issues you are describing.  Some filters behave better or worse with regard to reflections depending which side is facing the camera.  Also, it is possible to orient the filter wheel itself in different positions which might be useful for balancing an uneven declination load such as that imposed by a finder scope or a focuser.  But I'm unaware of there being any issues with how the filters are rotated with regard to the camera.  As long as you don't change anything between your flat frames and lights you should be good to go.

6) Focus should be at infinity for both imaging and for making flats.  It doesn't need to be exact, though.  Minor differences between infinity focus for different filters won't matter at all, and even setting an approximate value on your draw tube with a piece of tape or the lake is sufficient for taking flats prior to taking lights.  If your draw tube is within a millimeter or so of correct focus you should be fine. 

7) It is normal for different narrow band filters to generate flats with a different appearance.  Often the thickness of coatings varies a bit over the surface of the filter, for example.  HOWEVER, I have not seen narrow band filters generating the concentric circles you are getting, and it seems odd to me that both your luminance flat and your Ha flat seem to have very similar large, concentric circles.  You may, in fact, have some sort of light leak.  That would explain those hot spots on the right side of the Ha flat.  

 

There are a few things I would try.  First, make sure you don't have any light leaks by taking dark frames with a flashlight pointed in various places to see if you can induce any strange results.  I don't THINK you have a light leak, but it's worth ruling out.  A dark frame taken under brightly lit conditions with the camera mounted on the telescope should let you know.  Make sure you shine the flashlight around gaps in the focuser as well as the camera and filter wheels.

 

Next, make sure your flats and darks are taken with the same offset, gain, and temperature as your lights.  I assume they are, but I wanted to confirm.  Make sure your bias frames or your flat dark frames (whichever you decide to use, but not both) are also taken with the same settings.  

 

Next, I'd start to suspect the EL panel not providing as even illumination as you thought.  Try a 'T' shirt flat at dawn or dusk.  

 

After that, I'd want to look at your process overall in PixInsight.  It's easy enough to make an error in calibration settings such that you get strange results.  

 

One other thing that is really odd, when I look at your calibrated and stacked Ha image, is the variation in results with alignment of the different frames.  When everything is done properly, you will often see noise increasing in areas of the frame where you don't have coverage from all your subs--where the image rotated or shifted.  However, it is very odd that your flat correction would be different for the different frames.  Is there any chance your registered the images prior to calibrating them rather than the other way around?  Calibration has to come first in the workflow.

 

Last thing I will mention is that in a lot of places the flat seems to be overcorrecting, not under correcting.  I didn't measure it to be certain, but the top right corner of your Ha frame, for example, is actually lighter than areas near the middle of the frame (if you don't choose an area with nebulosity).  Knowing that you have overcorrection rather than under correction might be useful.  Usually this happens if you don't subtract out the bias / flat dark from the flat frame.  Double check your settings.  It can also happen if your flat frames are taken at an extreme ADU--something outside the linear range for your camera.  This is pretty unusual, though, but with a 12 bit camera you might have a slightly narrower range of values that both swamp read noise and are nowhere near the anti-blooming protection.  

 

That's all I've got for ideas.


Edited by Jared, 13 October 2017 - 07:03 PM.


#15 cfosterstars

cfosterstars

    Mercury-Atlas

  • *****
  • topic starter
  • Posts: 2,880
  • Joined: 05 Sep 2014
  • Loc: Austin, Texas

Posted 13 October 2017 - 07:23 PM

Jared,

 

Wow. Thank you so much. THAT is a lot to chew on. I will work on it. A lot of thing you have suggested I have actually done correctly, but not all. I will take some time to go through them all along with the other things that were suggested. I will definitely try using the dark flat method. The dark flat at least is something I can do once since I have now set all my flat parameters on my EL panel to get a decent exposure time. The shortest now is the L of course at 0.5s but everything else is 2-20s depending on the filter. I have run t-shirt flats previously with basically the same results so I think that the EL panel is good



#16 kingjamez

kingjamez

    Vanguard

  • *****
  • Posts: 2,310
  • Joined: 03 Oct 2006
  • Loc: Fairfax, VA

Posted 13 October 2017 - 07:23 PM

I know this thread isn’t for me. But my “blooming” flats went away after I discovered that my OIII filter was in backwards thanks to Sams suggestion to reverse the filter wheel. Give that a shot and see if it works for you.

-Jim

#17 Sean13

Sean13

    Viking 1

  • -----
  • Posts: 513
  • Joined: 17 Nov 2012
  • Loc: North Platte, Nebraska

Posted 13 October 2017 - 08:54 PM

My next suggestion was going to be try your stacking in a simple program like Deep Sky Stacker as opposed to the pixinsight as its more automated with generic settings and could see if a workflow problem is the issue.



#18 RedLionNJ

RedLionNJ

    Aurora

  • *****
  • Posts: 4,689
  • Joined: 29 Dec 2009
  • Loc: Red Lion, NJ, USA

Posted 14 October 2017 - 11:36 AM

You should use bias frames, however you should take them at 0.2 seconds exposure. Since I started using dark flats, I quit using bias as my bias signal is in the dark flats as well and thus removed at the same stage a bias would be. Much happier results with dark flats instead. I did not see the issues the same as you are seeing however.

If you're exposing as long as 0.2 seconds, that's not a bias frame, then.

 

With a 10-12 second flat exposure, you *need* to compensate with a dark flat. A simple bias will have nowhere near the noise structure needed to calibrate the flat.

 

And depending on how fast your optical system is, focus may indeed be critical for flats. Try very, very, very hard to keep the same focus as you used for your lights.


  • happylimpet likes this

#19 cfosterstars

cfosterstars

    Mercury-Atlas

  • *****
  • topic starter
  • Posts: 2,880
  • Joined: 05 Sep 2014
  • Loc: Austin, Texas

Posted 14 October 2017 - 12:31 PM

 

You should use bias frames, however you should take them at 0.2 seconds exposure. Since I started using dark flats, I quit using bias as my bias signal is in the dark flats as well and thus removed at the same stage a bias would be. Much happier results with dark flats instead. I did not see the issues the same as you are seeing however.

If you're exposing as long as 0.2 seconds, that's not a bias frame, then.

 

With a 10-12 second flat exposure, you *need* to compensate with a dark flat. A simple bias will have nowhere near the noise structure needed to calibrate the flat.

 

And depending on how fast your optical system is, focus may indeed be critical for flats. Try very, very, very hard to keep the same focus as you used for your lights.

 

My system is the ORION EON 115mm with a Williams optic flattener IV so it is not that fast at F/5.6. I re-ran flats last night with the EL panel set to only 40 for the LRGM and 200 for the Ha, OIII and SII.

 

This gave these exposure times

L - 0.5s

R - 2.26s

G - 1.37s

B - 1.8s

 

OIII - 3.60s

Ha - 10.79s

SII - 15.46s

 

I took 40 flats for each filter and then ran darks with exactly the same times. THe gain was 76, offset 40 and USB 50 at -15c for all flats and flat darks. I have a nice data set of NGC7000 that I will be trying to process with these new flats as see where is goes with dark flats and no bias frames. The fltats and flat darks are at the same gain, offset and temperature as the light frames.

 

I also check autofocus numbers of the filters as they ran the light frames.


Edited by cfosterstars, 14 October 2017 - 12:33 PM.


#20 cfosterstars

cfosterstars

    Mercury-Atlas

  • *****
  • topic starter
  • Posts: 2,880
  • Joined: 05 Sep 2014
  • Loc: Austin, Texas

Posted 14 October 2017 - 06:14 PM

Jason,

 

Again thanks for all the time and information you put in your response. I took your post and addressed each point to the extent that I can to shed more light on this issue - pun intended. You had so much in you post, that I drug it to MSWord to edit. Your input is in bold:

 

1) As mentioned, others have found uneven performance in bias frames if the exposure duration is below 0.2s, so use that value for your flats

 

Only my LUM flats were short at 0.09s. The NB flats were 2.44s for OIII, 10.1s for Ha and 13.86s for SII seconds. For my next set of data, I re-ran flats last night with the EL panel set to only 40 for the LRGM and 200 for the Ha, OIII and SII. This gave these exposure times: L - 0.5s, R - 2.26s, G - 1.37s, B - 1.8s – with EL power setting of 40, and OIII - 3.60s, Ha - 10.79s, SII - 15.46s for EL power of 200. I took 40 flats for each filter and then ran darks with exactly the same times. The gain was 76, offset 40 and USB 50 at -15c for all flats and flat darks. I have a nice data set of NGC7000 that I will be trying to process with these new flats as see where is goes with dark flats and no bias frames. The flats and flat darks are at the same gain, offset and temperature as the light frames.

 

2) Others have also found issues when trying to scale dark frames, so avoid doing that

 

With my first set of processing in Pixinsight, I only used the dark that was appropriate for the lights which was obviously much longer than the flat frame times. For the images and flats above, I used a master dark for a 10s exposure to be much closer to the flat frame exposure times. The next attempt will use flat darks at the same exposure time as the flats.

 

3) This means that the best approach is probably to either:

  • a. Build a master bias based on 0.2s exposures; bias subtract flats; use a master dark taken at your exposure duration with the same cooling and gain settings as the lights, and do not bias subtract the darks (no need to if you aren't trying to scale them) or
  • b. Don't bother with bias frames at all; use a master dark taken at your exposure duration with the same cooling and gain settings as the lights; rather than bias subtracting your flats, apply flat darks them (which will work like a bias frame, but with the added potential benefit of addressing some hot pixels from long-ish flat frame exposures.

 

I plan to try to use option b for my NGC7000 third attempt. I have all the data collected, but I need time to do the image processing.

 

4) You do have a couple spots on the sensor itself (or, more likely, on the cover glass on the outside of the CCD chamber) which appear to have been introduced when you did your cleaning.  Did you clean just the filters, or did you clean the CCD chamber?  Frankly, this is pretty minor and should calibrate out just fine.

 

There is one spot on the sensor itself. I can see it with a magnifying glass. There are also a couple of spots on the window of the sensor chamber. From the first set of data that showed the streaks, I cleaned only the sensor itself since that was where the streaks clearly were. I did not clean the filters. I would have expected that nothing in any of these flats would not calibrate out. That IS my issue – I don’t know why they are not calibrating out.

 

5) Maybe someone knows something I don't, but I'm not aware of filter orientation being relevant for any of the issues you are describing.  Some filters behave better or worse with regard to reflections depending which side is facing the camera.  Also, it is possible to orient the filter wheel itself in different positions which might be useful for balancing an uneven declination load such as that imposed by a finder scope or a focuser.  But I'm unaware of there being any issues with how the filters are rotated with regard to the camera.  As long as you don't change anything between your flat frames and lights you should be good to go.

 

The issue I am trying to explain but maybe badly, is not rotation of the filters, but which side of the filter faces the telescope and which faces the sensor. There have been other replies to this post and other that suggest that the Ha and SII filter ARE facing the wrong direction and the OIII is in the correct orientation. It is an easy issue to test and I will test it after I finish some image collection on another target. I don’t want to change the optical path just yet. I have a new adapter to directly thread my Williams Optics Flattener IV to the focuser of my EON. That way my image train will be fully threaded.

 

6) Focus should be at infinity for both imaging and for making flats.  It doesn't need to be exact, though.  Minor differences between infinity focus for different filters won't matter at all, and even setting an approximate value on your draw tube with a piece of tape or the lake is sufficient for taking flats prior to taking lights.  If your draw tube is within a millimeter or so of correct focus you should be fine.

 

I have measured the focus position of several but not all of my filters as they have run autofocus in SGP. The positions are rather close with a small offset. Once I change my optics train to the new adapter, I was going to setup the focus points in SGP. There is just no point right now. I spent some time understanding how this actually works with SGP and it really seem to be the right track. I honestly don’t believe that the focus offset between filters is the issue anymore. I believe that it is the combination of having the Ha and SII filters facing the wrong direction and how I am doing the actually calibrations in pixinsight.

 

7) It is normal for different narrow band filters to generate flats with a different appearance.  Often the thickness of coatings varies a bit over the surface of the filter, for example.  HOWEVER, I have not seen narrow band filters generating the concentric circles you are getting, and it seems odd to me that both your luminance flat and your Ha flat seem to have very similar large, concentric circles.  You may, in fact, have some sort of light leak.  That would explain those hot spots on the right side of the Ha flat. 

 

I believe that the hot spot on the Ha flat is actually an artifact from using a 180s masterdark scaled for my short flat exposures. If you look at the master dark, you can see the amp glow in the corners of the right side of the dark master. I think that this is being scaled incorrectly and creating a artifact in the calibration and stacking process for the flats. If this is correct, this should be addressed with the flat darks.

 

  • There are a few things I would try.  First, make sure you don't have any light leaks by taking dark frames with a flashlight pointed in various places to see if you can induce any strange results.  I don't THINK you have a light leak, but it's worth ruling out.  A dark frame taken under brightly lit conditions with the camera mounted on the telescope should let you know.  Make sure you shine the flashlight around gaps in the focuser as well as the camera and filter wheels.

I have verified that I do not have any light leaks. I did this with a dark frame in daylight. No issues.

 

  • Next, make sure your flats and darks are taken with the same offset, gain, and temperature as your lights.  I assume they are, but I wanted to confirm.  Make sure your bias frames or your flat dark frames (whichever you decide to use, but not both) are also taken with the same settings. 

 

Yes, my darks and flats were and are taken at the same offset, gain, USB and temperature as the lights. The bias frames that have used previously were also taken similarly, but with a very short exposure. This may also have been an issue.

 

  • Next, I'd start to suspect the EL panel not providing as even illumination as you thought.  Try a 'T' shirt flat at dawn or dusk. 

I tested the EL flatman panel on my other OTA with my DSLR and it performed flawlessly. The flats were better than sky flats and had very good uniformity. I have used tee-shirt flats many time previously and they work also just fine with my other OTA and Canon 6D. I am pretty sure the the EL panel is working correctly.

 

  • After that, I'd want to look at your process overall in PixInsight.  It's easy enough to make an error in calibration settings such that you get strange results. 

I followed the method for Youtube from Richard Bloch which is identical to what is in Warren Keller’s book. I have checked several times to be sure.

 

  • One other thing that is really odd, when I look at your calibrated and stacked Ha image, is the variation in results with alignment of the different frames.  When everything is done properly, you will often see noise increasing in areas of the frame where you don't have coverage from all your subs--where the image rotated or shifted.  However, it is very odd that your flat correction would be different for the different frames.  Is there any chance your registered the images prior to calibrating them rather than the other way around?  Calibration has to come first in the workflow.

 

Definitely not registering frames prior to calibration.

 

  • Last thing I will mention is that in a lot of places the flat seems to be overcorrecting, not under correcting.  I didn't measure it to be certain, but the top right corner of your Ha frame, for example, is actually lighter than areas near the middle of the frame (if you don't choose an area with nebulosity).  Knowing that you have overcorrection rather than under correction might be useful.  Usually this happens if you don't subtract out the bias / flat dark from the flat frame.  Double check your settings.  It can also happen if your flat frames are taken at an extreme ADU--something outside the linear range for your camera.  This is pretty unusual, though, but with a 12 bit camera you might have a slightly narrower range of values that both swamp read noise and are nowhere near the anti-blooming protection. 

 

I am not really sure what to say about this. I have been doing the bias and dark subtraction from the flat frame using image calibration in Pixinsight.
Jason, thanks for all the help and hints. If you have anything to add based on my input to your question/suggestion please respond to the post. I will keep following up as I get more data. I definitely want to get this issue solved and understood.



#21 cfosterstars

cfosterstars

    Mercury-Atlas

  • *****
  • topic starter
  • Posts: 2,880
  • Joined: 05 Sep 2014
  • Loc: Austin, Texas

Posted 16 October 2017 - 01:33 PM

I have made a bit of progress here. I am processing data from my more recent runs for reproduce-ability testing and will be processing the data through both pixinsight and ImagesPlus to compare. I have both DSLR and ASI1600 data taken at the same time on the same object.



#22 cfosterstars

cfosterstars

    Mercury-Atlas

  • *****
  • topic starter
  • Posts: 2,880
  • Joined: 05 Sep 2014
  • Loc: Austin, Texas

Posted 19 October 2017 - 09:03 PM

I am starting to make some progress on these issues and I will be updating this thread with the answers and data as I get it. I really want to leave this as a reference with enough data and explanations as I get them so that others can learn from my debug of this issue. I am also taking all the input from the members who have responded to this thread very seriously and will be trying to run down their suggestions. This is going to take a while, but I hope to get to it all.

 

One of the first things I have figured out relates to the question of dealing with the differences in focus between the filters in my filter wheel. I do see differences in optimal focus position between the filters as determined by the v-curve optimization that is done under Sequence Generator Pro's autofocus routine. I have not fully implemented all this learning yet since I want to complete the data I am taking on the pelican nebula before I really start ripping and tearing into my hardware. I will need to do that soon but I don't want to disturb things until I am finished with the data run. I need only 1-2 more good night's worth - famous last words.. A lot of what I am going to say may be old new for a lot of members, but I hope it will be useful for someone.

 

On the hardware side, I have been working with my FLATMAN EL plate to get better flats in general and to get the exposure time longer. I am doing this to deal with the issue with short exposure times issue on the ASI1600MM-C. At first, I was not putting a tee shirt between the FLATMAN and the OTA. But now, I bought a brand new tee shirt 3$ that will be strictly for my flats. I am also using different power settings on the panel between my LRGB filters (Power setting of 40 out of 255) verses my Ha, OIII and SII.filters (200 out of 255). I first tried this without a tee shirt. It worked well and that got my flat exposure times to these:

 

L - 0.5s

R - 2.26s
G - 1.37s
B - 1.8s

OIII - 3.60s
Ha - 10.79s
SII - 15.46s

 

I added a tee shirt (single layer) and that change the times only sightly +20%:

 

L - 0.6s
R - 2.71s
G - 1.65s
B - 2.16s

OIII - 4.32s
Ha - 12.95s
SII - 18.55s

 

The flat themselves showed basically no real visible difference that I could see when stretched to the flats without the tee shirt. I then went to two layers of tee shirt and was able to get the LRGB flat exposures to:

 

L - 1.07s
R - 4.98s
G - 2.94s
B - 3.93s

 

These times are now long enough that I should use DARK FLATs and not BIAS calibration for the flat frames. Again the double-tee-shirt flats showed no visible difference when stretched that either of the previous flats that I posted earlier in the thread. Since they look the same I will not post more images - but it proved to me that the FLATMAN was not the issue. Also from the above data, I will use two tee-shirt layers for LRGB, but I will use only one layer of tee shirt for the Ha, OIII and SII.filters since the times are already long enough to get away from BIAS frames. 

 

So how to use this information? I have found that SGP has some really nifty features for both filter focus positions and for flats that I am now just really starting to understand. First the flats calibration wizard is a very good tool.  The data above was actually taken with that wizard and it collect this information and optimized the time to hit 30K for the mean pixel value automatically for each binning mode and each filter. I had used this wizard before, but then created a separate "FLATS" sequence to take my flat frames. This is not the best uses of this wizard. It actually associates this calibration information with each equipment profile combination of OTA, camera, filter, etc. that you set up and uses it in other clever ways. What I now learned is what the the Flats Wizard does and how that addresses the focus position issues. 

 

First some more calibration information that you should collect and setting to enter. In SGP you should run the autofocus routine sequentially on each of your filters in a short period of time while the temperature is stable and record the optimized focuser positions. Then you enter these "focus positions" into the filter wheel setup dialog, along with your filter types, their positions in the wheel and how many flat frames for each filter you want to take when you take flats. SGP uses the focus positions that you enter as starting point for focus on a new sequence and once you have run your first autofocus on a sequence it uses the difference between these focus positions to determine the focus offsets between filters. This means that as you switch filters, the offset should position the focuser for the new filter very close to optimal focus and if you autofocus you should have a very symmetric and centered V-curve.

 

As you run a sequence, SGP averages the optimal focus positions determined by the autofocus routine that you get while you are running the sequence and remember them. This is where the Flats Wizard comes in. It is used to add a set of flat events to existing targets in your sequences. If you have run the Flats calibration wizard and have the focus offsets in your filter wheel set up and have been running autofocus while collecting your target data, then the Flats wizard will create flat events for each filter that you have used for your light events with the number of exposures that you specified in your filter wheel setup and at the exposures that you have set with the flat calibration wizard. When you then run those flat events - preferably after you have collected all your lights - it can even be after more than a single night of imaging - it will use the autofocus data from the sequence and the focus offsets you have specified to reset the focus position of each filter for each flat frame event. THIS IS SO COOL.. This should eliminate any question about the focus position for each filter for your flats. SGP takes care of that for you if you set up your filter wheel with the proper information and run the wizards correctly.

 

I am still in the process of getting this all working and I can get everything done until I complete my pelican nebula data, but this should close the book on at least one factor for my flats issues. 


Edited by cfosterstars, 19 October 2017 - 09:16 PM.

  • Jon Rista and astronate like this

#23 cfosterstars

cfosterstars

    Mercury-Atlas

  • *****
  • topic starter
  • Posts: 2,880
  • Joined: 05 Sep 2014
  • Loc: Austin, Texas

Posted 20 October 2017 - 11:48 AM

I am also going to be investigating the differences in processing methodology between ImagesPlus 6.0 (calibration/stacking) with Photoshop CC (processing) vs. Pixinsight and between Canon 6D vs. ASI1600 data on the same object to see if that may be having in impact. I have four good datasets to work with but I have settled these tests:

 

NCF7000 WF (DSLR), NGC7000 (NB), M45 WF (DSLR) and M45 (LRGB)

 

I want to compare processing with:

  • LIGHTS, DARKS, FLATS, BIAS - IP/PS
  • LIGHTS, DARKS, FLATS, BIAS - PI
  • LIGHTS, DARKS, FLATS, DARK FLATS - PI/PS
  • LIGHTS, DARKS, FLATS, DARK FLATS - PI

I am being methodical here since I really dont know what could be the issue. I am hoping that this massive tome of data will help me pinpoint the issue if its is from my calibration methodology. Right now I am inspecting every frame both LIGHT, DARK, FLAT, BIAS and DARK FLAT with Blink in PI and grading the images in PI. I am also going to be looking at the statistics of me BIAS, DARKS and FLAT DARKS to make sure that I am not getting black pixels due to too low an offset on the ASI1600. I dont think this is an issue, but since I am going through all this, I might as well see and verify that I am not clipping the low end.



#24 Ruthlessaz

Ruthlessaz

    Vostok 1

  • -----
  • Posts: 129
  • Joined: 09 Jun 2017
  • Loc: Phoenix, AZ

Posted 14 December 2017 - 09:13 PM

I am having the same issue with my asi 1600 mm-c... Looks like a circle in the calibration frames stack...any luck on diagnosing it?



#25 jhayes_tucson

jhayes_tucson

    Fly Me to the Moon

  • *****
  • Posts: 7,388
  • Joined: 26 Aug 2012
  • Loc: Bend, OR

Posted 15 December 2017 - 12:22 AM

You've received some good advice but I have a few things to add.

 

1)  First of all, you shouldn't have to stretch an image of your flat to display it.  You should be exposing the frame so that roughly 50-80% of the full well depth is filled.  That will be quite a bright image and will not require any stretching.  You are way underexposed if you have to stretch the image to see it.

 

2)  Here is an annotated image of what you are seeing in your flat.  The blotchy/rectilinear background that you see is something called "Fixed Pattern Noise," which is a property of the sensor itself.  It is caused by variations in gain between pixels (it can actually be caused by variations in amplifier gain, small variations in the size of the pixels, or in responsivity due to variations in the material.)  Most systems have enough optical vignetting to make FPN hard to see but you've done a nice job of bringing it out in this image (well done!)  FPN is not really a noise source since it is fixed in time but it's called "noise" because it can be treated like a noise source when evaluating temporal noise by analyzing ensembles of pixels across the sensor.  FPN is proportional to the signal strength and it will be the dominant factor causing variations in the signal across the sensor with a bright source.  CMOS sensors typically have larger FPN values than CCD sensors but neither will have much less than 1% variation due to FPN.  The reason that you shoot flats is to correct for optical vignetting, dust motes (a special case of vignetting,) and to correct for FPN.  Since FPN is proportional to the signal, it divides out of the image just like vignetting.  The bottom line is you want to record all this stuff in your flats so you are on the right track!

 

John

Attached Thumbnails

  • ASI-1600 Flat Annotated - cropped 12-14-17.jpg

Edited by jhayes_tucson, 15 December 2017 - 12:55 AM.



CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.


Recent Topics






Cloudy Nights LLC
Cloudy Nights Sponsor: Astronomics