Jump to content

  •  

CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.

Photo

Fixed Pattern Noise getting worse and worse

  • Please log in to reply
145 replies to this topic

#126 fmeschia

fmeschia

    Surveyor 1

  • *****
  • Posts: 1,944
  • Joined: 20 May 2016
  • Loc: Mountain View, CA

Posted 01 January 2019 - 12:29 AM

Yeah, I didn’t think it could be that.
Google is refusing to let me download the whoke folder you shared. It says it prepares a zip file, which never downloads. I will try again later.

#127 joelin

joelin

    Mercury-Atlas

  • *****
  • topic starter
  • Posts: 2,873
  • Joined: 14 Jan 2008
  • Loc: Saratoga, CA

Posted 01 January 2019 - 04:13 AM

some very careful use of the levels/curves tool in PS (instead of a STF) can bury the noise a bit better.

 

 

vkUkGuI.jpg

 

 

although getting rid of that blue glow on the left is hard ...i blame my flats being off for some reason

 

at least with this i feel like the image is somewhat presentable...even though the sploches are all still there

 

does anyone know the issue with the flats?


Edited by joelin, 01 January 2019 - 04:23 AM.


#128 joelin

joelin

    Mercury-Atlas

  • *****
  • topic starter
  • Posts: 2,873
  • Joined: 14 Jan 2008
  • Loc: Saratoga, CA

Posted 01 January 2019 - 04:46 AM

I reprocessed it without flats and the result looks better

Nl3y6uG.jpg

 

I reduced saturation/luminence on red colors so the sky looks darker and less blotchy


  • Scott Mitchell, 42itous1 and Cfreerksen like this

#129 fmeschia

fmeschia

    Surveyor 1

  • *****
  • Posts: 1,944
  • Joined: 20 May 2016
  • Loc: Mountain View, CA

Posted 01 January 2019 - 01:09 PM

I’m assuming you stretched your image, so we moved from the realm of proper calibration to the realm of post-processing. If so, my suggestion is to use a mask to protect the bright areas, and aggressively reduce chrominance noise in the rest of the image.



#130 fmeschia

fmeschia

    Surveyor 1

  • *****
  • Posts: 1,944
  • Joined: 20 May 2016
  • Loc: Mountain View, CA

Posted 01 January 2019 - 01:19 PM

Finally, Google decided to prepare the zip file and let me download it.

The first thing I noticed is: only one single bias frame (which doesn’t look like a master bias), and only three flat frames. Is that right? If so, you’re actually adding the noise of the single bias frame to each of your light frames, and dividing each light by the still considerable noise from just three flats.



#131 FlankerOneTwo

FlankerOneTwo

    Apollo

  • *****
  • Posts: 1,087
  • Joined: 30 Aug 2017
  • Loc: Vegas, baby!

Posted 01 January 2019 - 03:21 PM

It does take a while to prepare the zip files, for me it downloaded three of them with the entire complement of files.

There are far more experienced folks on here, but my $0.00002:

  The walking noise is definitely gone so the dithering is working for you.

  I'm sure you've noticed there appears to be tilt at the right side of the frame.

  I do see the same pattern after integration (LinearFit outlier rejection) and DBE, there is a component of both chrominance and luminance noise in the blotchy pattern.
 

  I do have a few questions / observations:

    1. I've always heard that one should use flat darks with the ASI1600 instead of bias frames, because the bias "floats" with exposure length.

    2. The flats were acquired at a different temperature (-6) from the rest of the frames (-20). This wouldn't matter if you could use normal bias frames, but obviously flat darks will have to match the flats in temp and exposure.

    3. Not sure why the FITS header reports the EGAIN as 4.96? At the settings you're using (unity gain with offset of 50), shouldn't the EGAIN be nearly 1? That's what SGP puts in my FITS files, I don't know that this makes any difference, just wondering.

 

My guess is that you need more integration time and have a component of residual bias noise as well. I'll be curious to see your results with more imaging time and flat darks instead of bias frames - you can easily grab some flat darks and redo the processing, just make sure they're at your flat temp (-6).


Edited by FlankerOneTwo, 01 January 2019 - 03:23 PM.

  • joelin and 42itous1 like this

#132 joelin

joelin

    Mercury-Atlas

  • *****
  • topic starter
  • Posts: 2,873
  • Joined: 14 Jan 2008
  • Loc: Saratoga, CA

Posted 01 January 2019 - 09:51 PM

Finally, Google decided to prepare the zip file and let me download it.

The first thing I noticed is: only one single bias frame (which doesn’t look like a master bias), and only three flat frames. Is that right? If so, you’re actually adding the noise of the single bias frame to each of your light frames, and dividing each light by the still considerable noise from just three flats.

Strange there should be about 30 bias frames, 20 flats...I think something got lost in the zip file...can you try again. 


Edited by joelin, 01 January 2019 - 09:52 PM.


#133 17.5Dob

17.5Dob

    Voyager 1

  • *****
  • Posts: 10,133
  • Joined: 21 Mar 2013
  • Loc: Colorado,USA

Posted 01 January 2019 - 09:56 PM

Strange there should be about 30 bias frames, 20 flats...I think something got lost in the zip file...can you try again. 

I  know nothing about the 1600 series, but even with my dSLR I shoot 500 bias frames minimum, to build a superbias master.....



#134 joelin

joelin

    Mercury-Atlas

  • *****
  • topic starter
  • Posts: 2,873
  • Joined: 14 Jan 2008
  • Loc: Saratoga, CA

Posted 02 January 2019 - 12:23 PM

It does take a while to prepare the zip files, for me it downloaded three of them with the entire complement of files.

There are far more experienced folks on here, but my $0.00002:

  The walking noise is definitely gone so the dithering is working for you.

  I'm sure you've noticed there appears to be tilt at the right side of the frame.

  I do see the same pattern after integration (LinearFit outlier rejection) and DBE, there is a component of both chrominance and luminance noise in the blotchy pattern.
 

  I do have a few questions / observations:

    1. I've always heard that one should use flat darks with the ASI1600 instead of bias frames, because the bias "floats" with exposure length.

    2. The flats were acquired at a different temperature (-6) from the rest of the frames (-20). This wouldn't matter if you could use normal bias frames, but obviously flat darks will have to match the flats in temp and exposure.

    3. Not sure why the FITS header reports the EGAIN as 4.96? At the settings you're using (unity gain with offset of 50), shouldn't the EGAIN be nearly 1? That's what SGP puts in my FITS files, I don't know that this makes any difference, just wondering.

 

My guess is that you need more integration time and have a component of residual bias noise as well. I'll be curious to see your results with more imaging time and flat darks instead of bias frames - you can easily grab some flat darks and redo the processing, just make sure they're at your flat temp (-6).

very helpful post

 

i do wonder about the tilt... what could cause that? i screwed my hyperstar in tightly into the secondary mirror holder... its collimated in the sense that the stars have a perfectly shaped donut while in the center of the frame...i dont know how else to make that more ideal

 

could the light tracer pad not be perfectly flat on the dew shield? if part of it is a few mm higher than the rest...could that cause an uneven surface?

 

looking at the following capture..i can see that the right side has slighly elongated stars and the left side has a glow

 

v48s8vL.jpg



#135 FlankerOneTwo

FlankerOneTwo

    Apollo

  • *****
  • Posts: 1,087
  • Joined: 30 Aug 2017
  • Loc: Vegas, baby!

Posted 02 January 2019 - 02:12 PM

very helpful post

 

i do wonder about the tilt... what could cause that? i screwed my hyperstar in tightly into the secondary mirror holder... its collimated in the sense that the stars have a perfectly shaped donut while in the center of the frame...i dont know how else to make that more ideal

 

could the light tracer pad not be perfectly flat on the dew shield? if part of it is a few mm higher than the rest...could that cause an uneven surface?

 

looking at the following capture..i can see that the right side has slighly elongated stars and the left side has a glow

 

I might be jumping the gun in saying tilt, more accurate to say that the stars are comatic especially at the right side of the field - to my eye they appear to point directly to the left at the bottom-right, and left and downward at the top right (crop attached). They may be a little pointing downwards at top left. The bottom left corner looks best to me.

 

upper right corner:

Untitled.jpg

 

You can have tilt that results from (fixed) sensor tilt within the camera, not to mention effects from primary mirror flop both of which would be more visible at f/2. I would also of course double check collimation very carefully. The other thing I wonder about is whether your corrector plate is decentered relative to the primary, as I understand that Hyperstar is quite a bit more sensitive to decentering than when used in standard SCT mode. Have you looked at jhayes' guide here: https://www.cloudyni...mize-perf-r3013

 

I'm pretty new to Hyperstar myself so I'm not going to pretend to have a fount of knowledge on it, I'm following your progress with interest. I do plan to center my C11 Edge corrector before doing any serious imaging with Hyperstar.


Edited by FlankerOneTwo, 02 January 2019 - 02:13 PM.


#136 joelin

joelin

    Mercury-Atlas

  • *****
  • topic starter
  • Posts: 2,873
  • Joined: 14 Jan 2008
  • Loc: Saratoga, CA

Posted 02 January 2019 - 04:30 PM

I've seen the jhayes article and to the best of my visual acuity the corrector is centered.

 

As for the tilt in the sensor, I've used hyperstar on two different cameras the ASI1600 mono and color. I will examine the frames on the other sensor more closely to see if the comatic stars are in the same place. My recollection is I had some issues similar between the color and mono cameras. 



#137 Jon Rista

Jon Rista

    ISS

  • *****
  • Posts: 25,616
  • Joined: 10 Jan 2014
  • Loc: Colorado

Posted 02 January 2019 - 04:55 PM

@joelin: I've been trying to download your data, but Google seems to be having some trouble zipping it all up. I'm downloading some individual files here, as I would like to see what each individual sub looks like as well as what the calibration masters look like. Someone mentioned it looked like you might have been imaging at Gain 0, which would be why EGAIN showed 4.96 (that would be e-/ADU). If that is the case, then you may find that imaging at Gain 76 helps a lot. You will have the same effective dynamic range, however keep an eye on clipping and see how it goes, you may need shorter exposures.

 

I still think, for whatever reason, your object signal is just too shallow. If DFPN is problematic, since it is FPN then integrating more subs may not actually help. A higher gain may help, longer exposures to get more object signal in each sub may help. When I look at your flat calibrated examples above, the background signal areas look very thin and translucent over the red/green color blotch. If the blotch is indeed coming from FPN, then you need more object signal. If the blotch is indeed FPN, then it will grow in time with stacking more subs, and you could quadruple your current sub count, and it may not help (this is the problem with FPN!) So, I would try some longer exposures and see what happens.

 

If all of that still does not work, then you may want to try a different camera. I would be curious to see how the ASI1600 color fares. You may also want to try the ASI071 (or the QHY counterpart), which are also OSC but with a full sized APS-C sensor and basically no amp glow.



#138 FlankerOneTwo

FlankerOneTwo

    Apollo

  • *****
  • Posts: 1,087
  • Joined: 30 Aug 2017
  • Loc: Vegas, baby!

Posted 02 January 2019 - 11:13 PM

Hi Jon,

 

I was able to pull the complete collection of files, the FITS header says these were taken with the ASI1600MC, T -20C, gain 139, offset 50, but EGAIN 4.96. I don't understand how that can happen?



#139 Jon Rista

Jon Rista

    ISS

  • *****
  • Posts: 25,616
  • Joined: 10 Jan 2014
  • Loc: Colorado

Posted 02 January 2019 - 11:32 PM

Hi Jon,

 

I was able to pull the complete collection of files, the FITS header says these were taken with the ASI1600MC, T -20C, gain 139, offset 50, but EGAIN 4.96. I don't understand how that can happen?

If the EGAIN in the file is 4.96, then I suspect the gain was 0, not 139.



#140 phenrotay

phenrotay

    Explorer 1

  • *****
  • Posts: 82
  • Joined: 24 Jun 2007
  • Loc: Belgium

Posted 03 January 2019 - 02:58 AM

If you are sure that the gain was set to 139 but discover that EGAIN does not match (should be 1 but is not), and are using ZWO ASCOM drivers...

 

I am in the same club : it seems that after ZWO ASCOM 1.0.3.18, EGAIN is sometimes not updated on a change in gain.

1.0.3.18 is fine.

 

The original post on the ZWO support forum is:

 

https://bbs.astronom...php?f=21&t=8604

 

Pierre 



#141 joelin

joelin

    Mercury-Atlas

  • *****
  • topic starter
  • Posts: 2,873
  • Joined: 14 Jan 2008
  • Loc: Saratoga, CA

Posted 23 January 2019 - 03:45 AM

I reprocessed my data, this time including some data that was very slightly out of focus...Overall it seems ok. I now have close to 2 hours of data in 20.5 mag skies. I also severely cropped it. 

 

I think the ugly fixed pattern noise is gone and the image is usable.

 

xsA65Ru.jpg


  • Scott Mitchell and Cfreerksen like this

#142 Cfreerksen

Cfreerksen

    Surveyor 1

  • *****
  • Posts: 1,705
  • Joined: 18 Aug 2018
  • Loc: Tooele, Ut

Posted 23 January 2019 - 08:58 AM

I reprocessed my data, this time including some data that was very slightly out of focus...Overall it seems ok. I now have close to 2 hours of data in 20.5 mag skies. I also severely cropped it. 

 

I think the ugly fixed pattern noise is gone and the image is usable.

 

xsA65Ru.jpg

What resolved the issue?

 

Chris



#143 joelin

joelin

    Mercury-Atlas

  • *****
  • topic starter
  • Posts: 2,873
  • Joined: 14 Jan 2008
  • Loc: Saratoga, CA

Posted 25 January 2019 - 12:37 PM

mostly dithering. and a lot of background extraction and cropping



#144 joelin

joelin

    Mercury-Atlas

  • *****
  • topic starter
  • Posts: 2,873
  • Joined: 14 Jan 2008
  • Loc: Saratoga, CA

Posted 25 January 2019 - 12:40 PM

Here is another noisy image...the area around the christmas tree cluster. 

 

It was done with an AT60ED, AS1600MC and 60x60sec at Gain139. Skies were mag 20.0. All calibration files were applied. It seems excessively noisy. 

 

6uwenzw.jpg

 

Here is a closeup

auU0i3d.png

 

Am I a victim of some kind of removable noise or is this just a product of light polluted skies and lack of signal?



#145 Jon Rista

Jon Rista

    ISS

  • *****
  • Posts: 25,616
  • Joined: 10 Jan 2014
  • Loc: Colorado

Posted 25 January 2019 - 03:16 PM

I still think you just have a lack of signal. Your signal barely seems to be rising above the read noise level. You need longer subs to solve that. I am not sure why this is the case...given you are using hyperstar. But when you have strong signal, it is clean and smooth, and all the patterns like this fade into the background.

 

This may be one of the problems with hyperstar. Stellar intensity is proportional to both the f-ratio and the aperture. A hyperstar aperture gathers a huge amount of light for each star, and the falloff of that light over the short focal distance is very small, so all that intensity very, very rapidly saturates the wells. So rapidly that, if you expose to avoid clipping too much, then you could find you are under-exposing your background signal, despite the extremely fast f-ratio. I know it sounds counter-intuitive, but it is one of the reasons I've tried to stick to around f/4, and with apertures that are not too large.

 

Just to test...I would say try this. Ignore stellar clipping. Expose twice as long as you are exposing now to test the same targets. Then expose four times as long as you are exposing now to test the same targets. See if you still have problems. If you DO still have problems, then the patterns are unlikely to actually be coming from the dark signal. FPN can also come from photon signal, due to PRNU (photo response non-uniformity). Flats correct for this, but you need to make sure your flats are very well matched (identical gain, offset and temp) and well exposed, as well as fairly deeply stacked to eliminate as much random noise as possible (i.e. 50 frames). Make absolutely certain you are not scaling the darks either...scaled darks will not correct DFPN properly, and leave you with a remnant.

 

But try longer exposures, twice as long and four times as long. Continue dithering. Also, I would integrate no less than 20 subs. For dithering to be most effective, more subs is better, but 20 subs will at least get the job done.


  • ChrisWhite and xthestreams like this

#146 xthestreams

xthestreams

    Messenger

  • -----
  • Posts: 430
  • Joined: 18 Feb 2020
  • Loc: Melbourne, Australia

Posted 30 March 2021 - 11:18 PM

Just discovered this thread - answers/addresses a lot of the challenges I've been having. Who knew that dithering was so important!?

 

With the "end" of the ASI1600 in sight I fear that this kind of trial knowledge will never be complied into a single place and I'll be forever hunting the internet looking for the definitive guide to getting the most out of my venerable 1600.




CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.


Recent Topics






Cloudy Nights LLC
Cloudy Nights Sponsor: Astronomics