Jump to content

  •  

CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.

Photo

How often do you dither with ASI1600MM?

  • Please log in to reply
45 replies to this topic

#1 georgian82

georgian82

    Messenger

  • *****
  • topic starter
  • Posts: 446
  • Joined: 16 Jan 2016
  • Loc: Healdsburg, California

Posted 14 August 2019 - 12:47 AM

Hi guys,

 

How often do you dither with ASI1600MM? I know it's all relative and a lot has to do with the amount of frames but are there any general rules?

 

Right now for Ha filter I have settled to take 200 subs at gain 139 and I wonder if I should dither every frame or every two or three frames...

 

Thanks in advance,

 

Sebastian



#2 Jon Rista

Jon Rista

    ISS

  • *****
  • Posts: 23349
  • Joined: 10 Jan 2014
  • Loc: Colorado

Posted 14 August 2019 - 01:18 AM

Every two or three frames is right. Unless you will be stacking thousands, less often than every 3 is too infrequent. 


  • numchuck, georgian82 and OldManSky like this

#3 Der_Pit

Der_Pit

    Viking 1

  • -----
  • Posts: 724
  • Joined: 07 Jul 2018
  • Loc: La Palma

Posted 14 August 2019 - 04:43 AM

Whoo, looks I should re-evaluate my setup.  I'm not doing narrow-band, but so far I'm dithering every 6-10 frames (given a target of around 200 subs at unity gain).  So far I had the impression this is 'enough'.  Even more as a dither can take as much time as one full exposure, so I'd lose a lot of sky time.

 

So is the 1 in 3 a general recommendation, or tailored for (longer exposure) NB data?



#4 Mert

Mert

    Fly Me to the Moon

  • *****
  • Posts: 6175
  • Joined: 31 Aug 2005
  • Loc: Spain, Pamplona

Posted 14 August 2019 - 08:04 AM

I use the color OSC version and dither every 3 frames nearly always

and for a "small" quantity of subs like 60 ( of 240 seconds each ).

The background in the stack looks smooth, uniform.


Edited by Mert, 14 August 2019 - 12:28 PM.

  • georgian82 likes this

#5 TimN

TimN

    Gemini

  • *****
  • Posts: 3127
  • Joined: 20 Apr 2008
  • Loc: Ontario, Canada

Posted 14 August 2019 - 08:28 AM

I use 2-3 frames for LRGB and every frame for my 5 minute NB. However, I have mixed LRGB and NB in one sequence and every second frame has worked.


  • Stelios and georgian82 like this

#6 entilza

entilza

    Soyuz

  • *****
  • Posts: 3716
  • Joined: 06 Oct 2014
  • Loc: Oakville, ON, Canada

Posted 14 August 2019 - 08:28 AM

For me something like this:

 

>= 180", every frame.

 

<= 120",  every 2nd

 

<= 90", every 3rd


  • Stelios, georgian82 and NuclearRoy like this

#7 Jon Rista

Jon Rista

    ISS

  • *****
  • Posts: 23349
  • Joined: 10 Jan 2014
  • Loc: Colorado

Posted 14 August 2019 - 07:31 PM

Whoo, looks I should re-evaluate my setup.  I'm not doing narrow-band, but so far I'm dithering every 6-10 frames (given a target of around 200 subs at unity gain).  So far I had the impression this is 'enough'.  Even more as a dither can take as much time as one full exposure, so I'd lose a lot of sky time.

 

So is the 1 in 3 a general recommendation, or tailored for (longer exposure) NB data?

Not even remotely close to enough. No more than every 3 frames. If you are stacking 100 or less, every 2 frames. With exposures 5 minutes or longer, you might as well dither every frame, the overhead is not that high and the more  you dither the better your results will be. 

 

BTW, if dithering is taking as long as an exposure, you have something misconfigured. Dithering should take 5-10 seconds. It will vary a bit, as each dither is different, a big one takes longer to settle than a small one. But it should never take as long as an exposure (unless your exposures are just 10 seconds long). You should not be trying to settle below your guide RMS. If you are, that is an exercise in true futility, and you are  in effect just timing out and the settling operation will just give up after a while. If your guide RMS is say 1.4 pixels, and you are trying to settle at 0.25 pixels, you are never gonna settle properly. You need to settle at 1.4 pixels in this example. 


Edited by Jon Rista, 14 August 2019 - 07:34 PM.

  • H-Alfa, dmdouglass, numchuck and 3 others like this

#8 georgian82

georgian82

    Messenger

  • *****
  • topic starter
  • Posts: 446
  • Joined: 16 Jan 2016
  • Loc: Healdsburg, California

Posted 15 August 2019 - 01:32 AM

Not even remotely close to enough. No more than every 3 frames. If you are stacking 100 or less, every 2 frames. With exposures 5 minutes or longer, you might as well dither every frame, the overhead is not that high and the more  you dither the better your results will be. 

 

BTW, if dithering is taking as long as an exposure, you have something misconfigured. Dithering should take 5-10 seconds. It will vary a bit, as each dither is different, a big one takes longer to settle than a small one. But it should never take as long as an exposure (unless your exposures are just 10 seconds long). You should not be trying to settle below your guide RMS. If you are, that is an exercise in true futility, and you are  in effect just timing out and the settling operation will just give up after a while. If your guide RMS is say 1.4 pixels, and you are trying to settle at 0.25 pixels, you are never gonna settle properly. You need to settle at 1.4 pixels in this example. 

Jon,

 

To your point but not to drift off topic although this is all related, my mount is actually having difficulties settling (it failed to settle in most instances las imaging session) with the following settings:

 

- settle pixel tolerance: 1.5

 

- minimum settle time: 10 sec

 

My RMS for that oscillated between 0.7 and 1.6"

 

Do you have any recommendations? 

 

Thanks



#9 ks__observer

ks__observer

    Viking 1

  • *****
  • Posts: 981
  • Joined: 28 Sep 2016
  • Loc: Long Island, New York

Posted 15 August 2019 - 05:38 AM

I have same issue: i always get settlement failed.

I increased PHD2 pixel region but thar didn't help.

I use 20 sec settle time based on visually looking at my guide graph.



#10 happylimpet

happylimpet

    Soyuz

  • *****
  • Posts: 3814
  • Joined: 29 Sep 2013
  • Loc: Southampton, UK

Posted 15 August 2019 - 05:48 AM

Folks,

 

Do we have quantitative testing or theory to base these frequencies on?

 

To use Jon's example of stacking 100 frames and dithering every 2, that will give 50 dither positions, which to my mind is certainly more than adequate to spot a statistical outlier. I would have thought far fewer (perhaps 20) would also be more than enough, ie dithering every 5, however this isnt based on anything empirical, just a feeling for how many samples it should take to establish what is an outlier.

 

Of course if theres no real penalty to dithering (Quick) then makes sense to do it a lot and I would support that.

 

I would have thought a rule of thumb like 50 dither positions throughout the stack would be a better way of stating the required frequency, as sub exposure times and number vary greatly between projects and individuals.

 

Cheers.

 

EDIT: of course, what I have been neglecting above, is that this is done millions of times for every image, so while 20 might work a lot of the time, there will still be many pixels in the stack where flukes occur and having a much better range of samples will help, so in fact 50 does sound like a much more sensible number than 20.


Edited by happylimpet, 15 August 2019 - 05:50 AM.

  • Der_Pit likes this

#11 Der_Pit

Der_Pit

    Viking 1

  • -----
  • Posts: 724
  • Joined: 07 Jul 2018
  • Loc: La Palma

Posted 15 August 2019 - 06:14 AM


Not even remotely close to enough. No more than every 3 frames. If you are stacking 100 or less, every 2 frames. With exposures 5 minutes or longer, you might as well dither every frame, the overhead is not that high and the more  you dither the better your results will be.

I'll definitely have a try. But to be fair, one would have to compare results that use the same observing time, something like either 120 images dithered evey third or 150-170 dithered every 6th....

 


BTW, if dithering is taking as long as an exposure, you have something misconfigured. Dithering should take 5-10 seconds. It will vary a bit, as each dither is different, a big one takes longer to settle than a small one. But it should never take as long as an exposure (unless your exposures are just 10 seconds long). You should not be trying to settle below your guide RMS. If you are, that is an exercise in true futility, and you are  in effect just timing out and the settling operation will just give up after a while. If your guide RMS is say 1.4 pixels, and you are trying to settle at 0.25 pixels, you are never gonna settle properly. You need to settle at 1.4 pixels in this example.

Not necessarily. This will also strongly depend on guiding parameters.  If you run with long exposures for guiding, and a lower aggressiveness, it can easily take 20 seconds (3-4 guide camera exposures) for PHD2 to settle, and I always add 5s additional security in my observing software.  Haven't really measured it, but I think my median 'dead time' during dither is around 20s.  Typical exposure time is 30s.  So it is a substantial fraction of the exposure time.

I'm aware that things are (very) different for long exposure narrow band imaging, but with RGB, already a 60s  frame (for me) will saturate all stars down to 9-10mag or so (rough guess), definitely too many to accept for most targets.



#12 lucam

lucam

    Viking 1

  • -----
  • Posts: 502
  • Joined: 09 Oct 2017
  • Loc: Upstate NY, USA

Posted 15 August 2019 - 06:35 AM

Folks,

 

Do we have quantitative testing or theory to base these frequencies on?

 

To use Jon's example of stacking 100 frames and dithering every 2, that will give 50 dither positions, which to my mind is certainly more than adequate to spot a statistical outlier. I would have thought far fewer (perhaps 20) would also be more than enough, ie dithering every 5, however this isnt based on anything empirical, just a feeling for how many samples it should take to establish what is an outlier.

 

Of course if theres no real penalty to dithering (Quick) then makes sense to do it a lot and I would support that.

 

I would have thought a rule of thumb like 50 dither positions throughout the stack would be a better way of stating the required frequency, as sub exposure times and number vary greatly between projects and individuals.

 

Cheers.

 

EDIT: of course, what I have been neglecting above, is that this is done millions of times for every image, so while 20 might work a lot of the time, there will still be many pixels in the stack where flukes occur and having a much better range of samples will help, so in fact 50 does sound like a much more sensible number than 20.

I suspect the answer will depend on the ratio between fixed pattern noise power and statistical noise power. When you dither every five subs, you are effectively averaging five frames with aligned pattern noise: statistical noise (Poisson and Gaussian) is reduced by sqrt(5), which makes pattern noise stand out more. Now the twenty positions have to recognize much stronger outliers and chances of butchering the image increase. However, if pattern noise is still buried in statistical noise after the five averages, then perhaps twenty random positions have a chance of smoothing out the fixed pattern while still being dominated by statistical noise, which can be handled via noise reduction. 

 

Jon's rule of thumb comes from his experience with the ASI1600 and I don't think it's meant to be a general statement that applies to all sensors and read-noise swamp factors. 



#13 Der_Pit

Der_Pit

    Viking 1

  • -----
  • Posts: 724
  • Joined: 07 Jul 2018
  • Loc: La Palma

Posted 15 August 2019 - 06:37 AM

Do we have quantitative testing or theory to base these frequencies on?

Hmm, I'd say the proper question to ask in that context is 'Is there a quantitative way to specify fixed pattern noise'.  And many other things go in, too.  Like is there a systematic drift/field rotation.  The eye/brain is utterly sensitive to repetitive patterns, and a noise that it would probably not recognize in a single frame pops out when repeated with a prefered direction (-> 'rain', 'walking noise').  

My expectation would be that the less systematic variation you have in image position, the less dither realizations are needed to bury the FPN in randomness.

So far I've used around 15-30 dither positions.  And while those with 30 do look better, it's (IMHO) mostly because they contain twice as many subs wink.gif

But I agree that for statistical matters, around 20 is probably the lower limit.  I'll definitely have a go at (at least) doubling my dither frequency.


  • happylimpet likes this

#14 Der_Pit

Der_Pit

    Viking 1

  • -----
  • Posts: 724
  • Joined: 07 Jul 2018
  • Loc: La Palma

Posted 15 August 2019 - 06:41 AM

I suspect the answer will depend on the ratio between fixed pattern noise power and statistical noise power. When you dither every five subs, you are effectively averaging five frames with aligned pattern noise: statistical noise (Poisson and Gaussian) is reduced by sqrt(5), which makes pattern noise stand out more. Now the twenty positions have to recognize much stronger outliers and chances of butchering the image increase. However, if pattern noise is still buried in statistical noise after the five averages, then perhaps twenty random positions have a chance of smoothing out the fixed pattern while still being dominated by statistical noise, which can be handled via noise reduction. 

Interesting question that just popped up when reading this:

What's (in FPN respect) the difference between a single 5min exposure and the sum/average of 10 (undithered) 30s exposures?


  • ks__observer likes this

#15 lucam

lucam

    Viking 1

  • -----
  • Posts: 502
  • Joined: 09 Oct 2017
  • Loc: Upstate NY, USA

Posted 15 August 2019 - 07:22 AM

Interesting question that just popped up when reading this:

What's (in FPN respect) the difference between a single 5min exposure and the sum/average of 10 (undithered) 30s exposures?

If you model FPN as a small difference in gain between two neighboring pixels, you can ask the question what is the contrast between the two pixels as a function of total exposure time and number of subs. In the limit of no read noise, there is no difference between the two cases you have highlighted. However, for finite read noise (hence the dependence on swamp factor), both pixel SNR and contrast between the two pixels (CNR) are lower if you acquire more subs (usual argument that each sub contributes to read noise total variance), than if you acquire a single long sub.


Edited by lucam, 15 August 2019 - 07:22 AM.

  • Der_Pit and NuclearRoy like this

#16 NuclearRoy

NuclearRoy

    Vostok 1

  • -----
  • Posts: 135
  • Joined: 11 Apr 2019
  • Loc: Pittsburgh, PA

Posted 15 August 2019 - 07:32 AM

CEM60, ASI1600, NB, 3-6 minute subs, gain 139, usually only 1 or 2 hours per filter, 5 pixel random dither limit in RA and Dec.

I dither every sub with a timeout of 20 seconds. Maybe 5%-10% reach the timeout.

Average + LN MAD Winsor clip in APP and I see no issues.

But I'm new, so...


Edited by NuclearRoy, 15 August 2019 - 07:33 AM.


#17 ks__observer

ks__observer

    Viking 1

  • *****
  • Posts: 981
  • Joined: 28 Sep 2016
  • Loc: Long Island, New York

Posted 15 August 2019 - 08:58 AM

If you model FPN as a small difference in gain between two neighboring pixels, you can ask the question what is the contrast between the two pixels as a function of total exposure time and number of subs. In the limit of no read noise, there is no difference between the two cases you have highlighted. However, for finite read noise (hence the dependence on swamp factor), both pixel SNR and contrast between the two pixels (CNR) are lower if you acquire more subs (usual argument that each sub contributes to read noise total variance), than if you acquire a single long sub.

Does shot noise bury FPN?

Do you need to dither more often at dark sites and for NB where there is less background sky photons creating unwanted shot noise?

I generally dither every 8 minutes -- every 4 exposures -- I don't notice any banding -- but maybe I am not looking hard enough.



#18 Jon Rista

Jon Rista

    ISS

  • *****
  • Posts: 23349
  • Joined: 10 Jan 2014
  • Loc: Colorado

Posted 15 August 2019 - 04:06 PM

I'll definitely have a try. But to be fair, one would have to compare results that use the same observing time, something like either 120 images dithered evey third or 150-170 dithered every 6th....

 


Not necessarily. This will also strongly depend on guiding parameters.  If you run with long exposures for guiding, and a lower aggressiveness, it can easily take 20 seconds (3-4 guide camera exposures) for PHD2 to settle, and I always add 5s additional security in my observing software.  Haven't really measured it, but I think my median 'dead time' during dither is around 20s.  Typical exposure time is 30s.  So it is a substantial fraction of the exposure time.

I'm aware that things are (very) different for long exposure narrow band imaging, but with RGB, already a 60s  frame (for me) will saturate all stars down to 9-10mag or so (rough guess), definitely too many to accept for most targets.

What gain are you using?

 

BTW, way way back when I was musing about the possibility of very sparse dithering (which in the intervening years apparently became law with CMOS :''(  ), I actually DID test. There were huge problems dithering even every 5 frames with as many as 500-600 subs. FPN (not just DFPN, but FPN from PRNU) became a problem (definitely limited SNR) within just a few hundred frames when sparse dithering like that. For shallower signal areas DFPN was a problem as well. 

 

After my original experimentation I never dithered less often than every 3 frames. That is usually with LRGB, and my exposures are never less than 60s (usually 90-120s) (and, I've mostly dropped L, so 90-120s RGB). I dither every 2 frames with exposures of 90-120s. With narrow band, I expose 5 minutes or longer and dither every frame. 


Edited by Jon Rista, 15 August 2019 - 04:06 PM.


#19 roofkid

roofkid

    Viking 1

  • -----
  • Posts: 633
  • Joined: 13 Jan 2016

Posted 15 August 2019 - 04:42 PM

Very interesting thread and experiences here. I can only tell you my experience with an experiment I did. I once shot a galaxy (forgot which it was, think M96) back before I had filter offsets. So I shot each filter to completion before switching. I tried dithering every 4 frames as an experiment. I then built groups from the ~400 L raw frames:

- dithering every frame (removing 3/4 frames here)

- dithering every 2nd frame (removing 2/4 frames here)

...

you get the idea.

 

I had walking noise in all of the stacks except in the one where I dithered every frame. Admittedly it got better the more often the dither occurred but I just couldn't stand it or edit it out. So I dither every frame since then and did not look back.

 

Today I have filter offsets and shoot LRGB<dither>LRGB<dither>... (or SHO<dither>SHO<dither>...)

So while I technically dither only every 4th/3rd frame I actually STILL do it every frame of each channel. It's just a hell of a lot more time efficient this way.

 

That's my experience with it


Edited by roofkid, 15 August 2019 - 04:42 PM.

  • Stelios and Mark233 like this

#20 freestar8n

freestar8n

    Vendor - MetaGuide

  • *****
  • Vendors
  • Posts: 8681
  • Joined: 12 Oct 2007

Posted 15 August 2019 - 05:19 PM

It's very hard to know how much dithering helps because it is hard to know how big the fpn is in the first place, as a noise term.  It could be 0, in which case you should never dither.  But as I described in an earlier thread - whatever its noise is would theoretically go down as sqrt(NDither) in a sequence of exposures.  If you don't dither then the fpn stacks on top of itself and will fundamentally limit the SNR you can obtain for a given signal value - because it adds linearly in the stack.

 

In my case with 1600 I dither every frame because the overhead is fairly small at about 15 seconds per 5 minute frame.  And if I change filters then I would need a number of seconds to do that anyway.

 

I use MetaGuide for guiding and dithering - and I wait for the guide error settle before the next exposure.  Sometimes it is only a few seconds and other times it is longer - mainly limited by dec. backlash.  My guide exposures are effectively 1 second long - so I have a nearly realtime view of the settling.  If your guide exposures are 10 seconds long or something, I expect that would make it harder to know when you have settled.

 

A high end mount should dither much more quickly.

 

Dithering is another reason that longer exposures are good - even if shorter ones "swamp" the read noise.

 

Frank


  • happylimpet and NuclearRoy like this

#21 jerahian

jerahian

    Messenger

  • *****
  • Posts: 483
  • Joined: 02 Aug 2018
  • Loc: Maine

Posted 15 August 2019 - 10:07 PM

I switched to dithering between every frame a few months back and I haven't looked back.  I used to do every 3-5, and I don't shoot much less than 180" anymore (except a few RGB frames of 60" here and there for star color replacement).



#22 Der_Pit

Der_Pit

    Viking 1

  • -----
  • Posts: 724
  • Joined: 07 Jul 2018
  • Loc: La Palma

Posted 16 August 2019 - 10:08 AM

What gain are you using?

Unity, 139.

 

BTW, way way back when I was musing about the possibility of very sparse dithering (which in the intervening years apparently became law with CMOS :''(  ), I actually DID test. There were huge problems dithering even every 5 frames with as many as 500-600 subs. FPN (not just DFPN, but FPN from PRNU) became a problem (definitely limited SNR) within just a few hundred frames when sparse dithering like that. For shallower signal areas DFPN was a problem as well. 
 
After my original experimentation I never dithered less often than every 3 frames. That is usually with LRGB, and my exposures are never less than 60s (usually 90-120s) (and, I've mostly dropped L, so 90-120s RGB). I dither every 2 frames with exposures of 90-120s. With narrow band, I expose 5 minutes or longer and dither every frame.

As I wrote, I'm not doing any narrow band, only RGB (also not taking any L).  And after Lucas comment above I think you wouldn't see too much difference between 100 90s frames dithered every frame and 300 30s frames dithered every 3rd (regarding FPN)

 

Then again, my demands most likely are (still?) much lower than yours, I've only started AP seriously a year ago wink.gif



#23 Jon Rista

Jon Rista

    ISS

  • *****
  • Posts: 23349
  • Joined: 10 Jan 2014
  • Loc: Colorado

Posted 16 August 2019 - 04:41 PM

Unity, 139.

 

As I wrote, I'm not doing any narrow band, only RGB (also not taking any L).  And after Lucas comment above I think you wouldn't see too much difference between 100 90s frames dithered every frame and 300 30s frames dithered every 3rd (regarding FPN)

 

Then again, my demands most likely are (still?) much lower than yours, I've only started AP seriously a year ago wink.gif

Yeah, unity is freakin high for LRGB  man. You should use 76 at most, but I would try Gain 0. You just don't have the dynamic range you need for LRGB. Unity is fine for NB, but you are indeed going to saturate FAST with LRGB filters.

 

I've used Gain 0, and swamped the read noise squared 50, 70, in some cases 100x. Swamp factor, assuming you at least get enough, becomes largely meaningless with LRGB even under moderate LP. You just need long-enough exposures that you aren't having to get a ton of them. If you are getting 30-60 second subs right now at unity, you should be at 120-240 seconds at Gain 0. 


  • Jim Cauthen likes this

#24 RazvanUnderStars

RazvanUnderStars

    Viking 1

  • -----
  • Posts: 636
  • Joined: 15 Jul 2014
  • Loc: Toronto, Canada

Posted 16 August 2019 - 09:41 PM

So that I learn: what DR is needed for LRGB? According to the manual (pg 9), the camera has a DR of 12.5 stops at gain 0 and little over 11 at gain 139. The chart is below. 

 

gain.png

 

Thank you.

 

 

Yeah, unity is freakin high for LRGB  man. You should use 76 at most, but I would try Gain 0. You just don't have the dynamic range you need for LRGB. Unity is fine for NB, but you are indeed going to saturate FAST with LRGB filters.

 

I've used Gain 0, and swamped the read noise squared 50, 70, in some cases 100x. Swamp factor, assuming you at least get enough, becomes largely meaningless with LRGB even under moderate LP. You just need long-enough exposures that you aren't having to get a ton of them. If you are getting 30-60 second subs right now at unity, you should be at 120-240 seconds at Gain 0. 

 



#25 Jon Rista

Jon Rista

    ISS

  • *****
  • Posts: 23349
  • Joined: 10 Jan 2014
  • Loc: Colorado

Posted 16 August 2019 - 11:28 PM

So that I learn: what DR is needed for LRGB? According to the manual (pg 9), the camera has a DR of 12.5 stops at gain 0 and little over 11 at gain 139. The chart is below. 

 

attachicon.gif gain.png

 

Thank you.

As much as you can get, generally speaking. There is no specific number here. It is simply that the broader the filter, the faster stars will saturate.




CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.


Recent Topics






Cloudy Nights LLC
Cloudy Nights Sponsor: Astronomics