32 replies to this topic

[arrowspace90]

Recently I had 2 image sessions that appeared to show "pattern noise".  Which I thought I had solved back when I upped my dithering from 1 pixel to 10.  I dither every 3 frames.  This had been working, but suddenly it seems to not be working, at least at times.

 

So I upped my dithering to every 2 frames.  Obviously, pattern noise cannot be tolerated and one must take whatever steps are necessary to prevent it.

The drawback is, you can quickly be dithering the night away.

Last night I had the scope in the driveway for 7 hours.  Of that, actual imaging took up just over 5 hours.

So 25% of the time, the scope/camera are dithering.

Would you say that is pretty much in line with what many or most people get, or does it seem too much?  Or too little?  I know that some people dither every frame.  How many people do that?

 

My ASIAIR software is also apparently suffering dithering issues in its latest update.  But I don't know if or how much difference that makes.  I upped my settling time to 40 seconds.

[Ken Sturrock]

Yeah, it can add up.

 

I'm ignorant of the ASIAir, sorry. Which camera are you using? If the mono camera, it may (or may not...) be possible to set your work-flow and software to dither only after your filter changes. For example, instead of shooting:

 

red <dither> blue <dither> green <dither> red <dither> blue <dither> green

 

you can shoot:

 

red blue green <dither> red blue green

 

so long as all the subs that will be combined into a single master are dithered from each other then you're golden. I've lost track, this may not be a commonly available technique and, of course, it won't work for OSC.

 

If you have a lot of frames, you can kind of mimic this by dithering every X frames and then integrate to sets of masters: one master with the odd sub frames and one master based on the even sub frames (or three masters each made from a third of the frames that are dithered against each other). You'd then average the master stacks. Obviously, this will work with OSC. It is a lot of work, though, and reduces some of the statistical benefits of stacking up a large number of subs - so you'd need even more.

 

Another thing may be to try to tweak settings depending on how the dither is done. For example, many systems guide their way to a new (post-dither) location. If your max-move is set really low then it can take a while. As you say, some software offers a settle value that you could relax. Again, though, this is software specific.

 

Finally, although unpopular with the "take a lot of short exposures" crowd, you could run the exposure a little longer such that less total time is spent doing overhead.

 

All the best.

 

-Ken

[Jon Rista]

I would say that dithering every 3 frames is the upper limit...however, I would only dither that "infrequently" if I was planning to integrate well beyond 100 subs. If you are planning to integrate fewer than 100 subs, then I would dither every 2 frames, and 50 or less every frame. Given your post, it sounds like you are probably using fairly short exposures. So I'm assuming you are probably integrating around 100 subs or so. 

 

Dithering more frequently could help, but you may also need to increase the aggression of your dithers. If you are dithering in small enough amounts that you aren't offsetting the stars in each frame enough from each other to randomize the FPN enough, then the patterns could still align once those subs are registered and stacked. Instead of being obvious streaking, it might be softer streaking, fatter streaks, or a more "wandering walk" where the pattern meanders just a little along its path. 

 

You need to make sure that you are dithering by enough each frame, that the patterns, when the frames are registered, just become random noise again. I recommend dithering by 5-10 pixels, which should be around 2-3x the diameter of the average size star, assuming you are sampling somewhere between 2-3 pixels across the FWHM (if you are oversampling more, or undersampling severely, then you may need to adjust this, but 5-10 pixels will usually work for most cases). That is usually enough.

 

Now, there is a balancing act here...you want to dither aggressively enough that you can randomize the FPN in the stack, but not so much that it becomes harder or takes too long to settle after each dither. So you should experiment a bit, and find the settings that work for you. You SHOULD be able to fully settle and get the next frame started within 10 seconds after each frame. If it takes 30, 40, 60 seconds or longer to settle, then you need to adjust your dithering settings (and specifically, what you are settling at and for how long) to bring that down to around 10 seconds. Note that, there is no reason to try and settle below your guide RMS. Usually you settle to some ratio of a guider pixel. So if your guide RMS, in guider pixels, is say 1.4 pixels...then there is no reason to try and settle below 1.4. If you try to settle below that, then it becomes merely chance that you settle, because in effect, you are trying to settle within the "tracking noise". If  your guide RMS was 0.7 pixels, then you could settle at 0.7, and if it was 2.5 pixels, you wouldn't be able to settle much below 2.5. You may need to fiddle with this a bit...if it takes too long to settle right at your RMS, try slightly above...1.5, 0.8, 2.6 etc. The other factor here is, how long you expect the settling factor to be maintained...1, 2, 3, seconds etc. I usually settle for 3 seconds with my setups, however you may find you need to settle a bit longer to fully settle, and this will generally be more important when oversampling than when undersampling.

 

 

Now, dithering and how you  do it is a factor here, for sure. That said, you should also try and balance out the length of exposures and number of total exposures as well as the actual frequency of dithers. If you are using very, very short subs, if you can find a way to use longer subs, that will improve the efficiency of your imaging sessions. You may also be able to improve efficiency in other ways.

 

First off, if you are using very short exposures, consider the gain you are using. If you are using a high or even moderate gain and are stuck with short exposures (i.e. a minute or less), then reduce your gain and increase your exposures. For LRGB, you may in fact be able to use the minimum gain, even on a 12-bit camera...and this would especially be the case for light polluted skies (its a no brainer with Bortle 7 or brighter). You may be able to double or triple your exposures, which can reduce the number of dithers you have to perform each night.

 

Beyond this, if reducing exposure length isn't that much of an option, you can also configure an interleaved imaging session. Instead of something like this:

 

LL <dither> LL <dither> LL <dither> ... RR <dither> RR <dither> RR <dither> ... GG <dither> ... BB <dither> ...

 

Where you dither every two frames. If you can configure automatic focusing, set up filter focus offsets (to avoid having to re-focus every time you change filters), and then interleave imaging across filters...you could greatly reduce the number of dithers you have to perform each night:

 

LLRGB <dither> LLRGB <dither> LLRGB <dither> ...

 

This would be 2.5 times more efficient than imaging one filter at a time and dithering every 2. Now, this dithers every 2 L and every 1 RGB. You could potentially extend that even more, if you plan to get enough RGB data to support dithering every 2 frames:

 

LLRRGGBBLL <dither> LLRRGGBBLL <dither> LLRRGGBBLL <dither> ...

 

This would be FIVE TIMES more efficient, than dithering every 2 frames!! Further, this can also distribute differences in seeing across all channels, as well as guarantee you have a full constituent of data across channels every night you image.

Edited by Jon Rista, 26 November 2020 - 12:30 PM.

[arrowspace90]

I would say that dithering every 3 frames is the upper limit...however, I would only dither that "infrequently" if I was planning to integrate well beyond 100 subs. If you are planning to integrate fewer than 100 subs, then I would dither every 2 frames, and 50 or less every frame. Given your post, it sounds like you are probably using fairly short exposures. So I'm assuming you are probably integrating around 100 subs or so. 

 

Dithering more frequently could help, but you may also need to increase the aggression of your dithers. If you are dithering in small enough amounts that you aren't offsetting the stars in each frame enough from each other to randomize the FPN enough, then the patterns could still align once those subs are registered and stacked. Instead of being obvious streaking, it might be softer streaking, fatter streaks, or a more "wandering walk" where the pattern meanders just a little along its path. 

 

You need to make sure that you are dithering by enough each frame, that the patterns, when the frames are registered, just become random noise again. I recommend dithering by 5-10 pixels, which should be around 2-3x the diameter of the average size star, assuming you are sampling somewhere between 2-3 pixels across the FWHM (if you are oversampling more, or undersampling severely, then you may need to adjust this, but 5-10 pixels will usually work for most cases). That is usually enough.

 

Now, there is a balancing act here...you want to dither aggressively enough that you can randomize the FPN in the stack, but not so much that it becomes harder or takes too long to settle after each dither. So you should experiment a bit, and find the settings that work for you. You SHOULD be able to fully settle and get the next frame started within 10 seconds after each frame. If it takes 30, 40, 60 seconds or longer to settle, then you need to adjust your dithering settings (and specifically, what you are settling at and for how long) to bring that down to around 10 seconds. Note that, there is no reason to try and settle below your guide RMS. Usually you settle to some ratio of a guider pixel. So if your guide RMS, in guider pixels, is say 1.4 pixels...then there is no reason to try and settle below 1.4. If you try to settle below that, then it becomes merely chance that you settle, because in effect, you are trying to settle within the "tracking noise". If  your guide RMS was 0.7 pixels, then you could settle at 0.7, and if it was 2.5 pixels, you wouldn't be able to settle much below 2.5. You may need to fiddle with this a bit...if it takes too long to settle right at your RMS, try slightly above...1.5, 0.8, 2.6 etc. The other factor here is, how long you expect the settling factor to be maintained...1, 2, 3, seconds etc. I usually settle for 3 seconds with my setups, however you may find you need to settle a bit longer to fully settle, and this will generally be more important when oversampling than when undersampling.

 

 

Now, dithering and how you  do it is a factor here, for sure. That said, you should also try and balance out the length of exposures and number of total exposures as well as the actual frequency of dithers. If you are using very, very short subs, if you can find a way to use longer subs, that will improve the efficiency of your imaging sessions. You may also be able to improve efficiency in other ways.

 

First off, if you are using very short exposures, consider the gain you are using. If you are using a high or even moderate gain and are stuck with short exposures (i.e. a minute or less), then reduce your gain and increase your exposures. For LRGB, you may in fact be able to use the minimum gain, even on a 12-bit camera...and this would especially be the case for light polluted skies (its a no brainer with Bortle 7 or brighter). You may be able to double or triple your exposures, which can reduce the number of dithers you have to perform each night.

 

Beyond this, if reducing exposure length isn't that much of an option, you can also configure an interleaved imaging session. Instead of something like this:

 

LL <dither> LL <dither> LL <dither> ... RR <dither> RR <dither> RR <dither> ... GG <dither> ... BB <dither> ...

 

Where you dither every two frames. If you can configure automatic focusing, set up filter focus offsets (to avoid having to re-focus every time you change filters), and then interleave imaging across filters...you could greatly reduce the number of dithers you have to perform each night:

 

LLRGB <dither> LLRGB <dither> LLRGB <dither> ...

 

This would be 2.5 times more efficient than imaging one filter at a time and dithering every 2. Now, this dithers every 2 L and every 1 RGB. You could potentially extend that even more, if you plan to get enough RGB data to support dithering every 2 frames:

 

LLRRGGBBLL <dither> LLRRGGBBLL <dither> LLRRGGBBLL <dither> ...

 

This would be FIVE TIMES more efficient, than dithering every 2 frames!! Further, this can also distribute differences in seeing across all channels, as well as guarantee you have a full constituent of data across channels every night you image.

As I said in my post, I have been dithering at 10 pixels.  I am also taking more like, 200 frames or more, or well over 100.  In the session with the bad pattern noise, my subs were 90 seconds at gain 0.  This was with a OSC ASI533 and a fast RASA 8 at F/2.

So really, I don't understand the dynamics of what is happening.

[Rasfahan]

If the noise suddenly shows up, double check if flat calibration was right. Try and renew your dark library. I had serious problems with noise and overcorrecting flats with my ASI183 - it all went away with new darks/darkflats. Beware of firmware updates. Lastly, try and contact ZWO to see what pattern they use for dithering in the ASIAIR (or look into the guiding settings). If it is not random, there is a problem. 

[Jon Rista]

As I said in my post, I have been dithering at 10 pixels.  I am also taking more like, 200 frames or more, or well over 100.  In the session with the bad pattern noise, my subs were 90 seconds at gain 0.  This was with a OSC ASI533 and a fast RASA 8 at F/2.

So really, I don't understand the dynamics of what is happening.

If you are still having problems, then you may simply need to dither more often. With an interleave, you can dither LRGB <dither> and keep things efficient, while still dithering every frame. The fact that it takes 40 seconds to settle, indicates you don't have the right settings for your settle...you should be settling FAR quicker than 40 seconds. 

 

Also, make sure you are getting enough signal in each frame. This may require a lower gain. If you are not getting enough signal in each frame, then your fainter signals are going to suffer more from DFPN (hot/cold pixels, banding, etc.). Burying the DFPN more can also help. Lower gain, longer subs. 

[whwang]

Perhaps some pictures showing the pattern noise would help us to see what's really going on.  It looks to me you had already sufficiently dithered.  If you still see patterns, then the probably may be somewhere else.

[michael8554]

Dithering has to be a certain number of pixels on the imaging camera - is ASIAIR aware of the imaging pixel scale, or is the Dither Setting only related to the guidecam pixel scale ?

 

You note how much imaging time is lost due to Dithering - how long does your mount take to Settle after Dithers, and have you set the Settle Time to high ?

Edited by michael8554, 27 November 2020 - 04:46 AM.

[happylimpet]

What with dithering and periodic refocusing, I get 3000 seconds of exposure per hour (3600seconds). Im happy with that.

 

Many years ago my PhD supervisor told me that Hubble only spends about 8% of its time actually integrating images. This was in about 1993 so may be outdated, or just wrong!

[whwang]

8% sounds way too little to me. Typically, Hubble can spend about 50 minutes on target during its roughly 90-minute orbit, because its view would be occulted by earth in the other 40 minute or so. And during that 50 minutes, it has to spend some time reading the images and setting up the camera or spectrograph. But still, 8% is too little. It must be some very special observing mode.

On ground based telescopes in imaging mode, an 80% efficiency is quite typical in the optical. In the near-IR, because exposure time is much shorter (sky much brighter), an efficiency of 60% is very common, and 50% is not something unheard of.

[arrowspace90]

Currently the ASIAIR dithering software is messed up.  It frequently reports "signal error" during dithering and "settling time out" even when I have it settling time allowed at 40 seconds.  So some of the problem is with them, but I don't know how much.  I'm going to leave the dither at every 2 frames but move the settling time back to 30 seconds, which as a poster said, "should be" plenty of time.

I'm between a rock and a hard place, because a moderator at APP said the "roughness" of the edges of my raw image stack and image overlap (causing wide, ugly vertical bands on both left and right edges (meridian flip) is "caused by aggressive dithering".  Ha.  **** if you do and **** if you don't.  Those vertical bands have to be cropped out (he agreed) and if you are trying to image a big nebula, there goes a chunk of it.  

Trying to do the right things can be very difficult.  I have about a year of experience now, and its amazing how many things still go wrong.  From the above poster saying "60% efficiency is quite typical", I suppose my 75% really isn't bad.  Good to know.

[happylimpet]

Looks like I was wrong:

 

 

During a typical ~96 minute orbit, it will spend most of its time in pointed astronomical observations. This efficiency of ~50% or better was reached during the first 6 years of science operations as improvements were made in how HST was utilized and operated. Efficiency values predicted at launch were 35%. As a target is observed it will be occulted by the earth anywhere between zero and ~50% of an orbit, at which point HST can halt and resume an exposure after the occultation.

 

https://arxiv.org/ft...3/1203.0002.pdf

[arrowspace90]

Ok, so here is a raw image that caused me headaches.

 

Yes, some of this could be flat field error.  I took what "should have been" good flats for this image, but 80% of problems turn out to be flat field errors right?  I was actually able to end up with a pleasing image of these pretty galaxies.

 

But, what I am talking about is the "fingers" of repeating artifacts that slant diagonally from right to left.  That is unlike my typical flat errors, at least the ones I have seen so far.  Again, I make every effort to calibrate my flats, every time.  Their histogram peak is parked right in the middle of the dynamic range.

 

If you don't think this is a pattern noise, feel free PLEASE to voice your opinion.  I have to be able to correct errors.

Attached Thumbnails

• 

[arrowspace90]

And not to be outdone, here is a raw stack showing the edge problems that APP says are caused by aggressive dithering.  Look at the uneven lighting on the two sides of the photo.  This all has to be cropped out.

I upped the multiband-blending from it's default 5 up to 20!  I redid this stack, but the artifacts remained, somewhat less conspicuous but not sufficient to leave them uncropped.  

Attached Thumbnails

• 

Edited by arrowspace90, 27 November 2020 - 10:03 AM.

[whwang]

Hi,

 

I can't say for sure what the "fingers" are, but they are not pattern noise, at least not the kind of pattern noise that can be removed with standard dithering.  Dithering is not likely to help you here.  You need to figure out what causes it.

 

The edge issue in the California nebula shot is caused by dithering (perhaps coupled with strong sky gradient), indeed. It can be reduced by using smaller dither offsets, but it will always be there.  

[endless-sky]

Like it has been suggest above, dithering frequency depends on how many images you plan to stack. I dither every 2 or 3 frames, when I plan multiple nights worth integrations (100 or more subs to integrate).

 

I am still experimenting with dither distance, as I haven't been auto-guiding for long. I started with 3-5 pixels on the main camera. Now I am trying 10-15 and see if results improve. I guess each camera is different, so experimenting is needed, here.

 

And not to be outdone, here is a raw stack showing the edge problems that APP says are caused by aggressive dithering.  Look at the uneven lighting on the two sides of the photo.  This all has to be cropped out.

I upped the multiband-blending from it's default 5 up to 20!  I redid this stack, but the artifacts remained, somewhat less conspicuous but not sufficient to leave them uncropped.  

This is a dithering side effect, and is perfectly normal. The area in which all your subs overlap will show less noise and will have the highest SNR, while the areas that do not overlap completely will be increasingly darker, have more noise and less SNR. The edges/corners will have to be cropped. The amount depends on how aggressive the dithering was.

[Alex McConahay]

I don't buy that the problem is caused by dithering. At least for the California.

 

 

Notice there is a rotation to it. It is most obvious at the right side, where the margin gets wider as it goes. It is also noticeable at the top, where the black area at the left side is shorter and the right side is taller. 

 

Dithering does not cause rotation. Polar alignment does. There may be other causes. But I don't think it is dithering. 

 

Alex

Edited by Alex McConahay, 27 November 2020 - 10:58 AM.

[Kevin_A]

Not sure why you would image a 90 second sub at gain 0 with a ASI 533MC pro when your noise would be greatly reduced at just above unity gain.

I set mine at gain 101 for 120 second subs and the lower noise may help give you a better starting point for dither noise reduction.

[sn2006gy]

As I said in my post, I have been dithering at 10 pixels.  I am also taking more like, 200 frames or more, or well over 100.  In the session with the bad pattern noise, my subs were 90 seconds at gain 0.  This was with a OSC ASI533 and a fast RASA 8 at F/2.

So really, I don't understand the dynamics of what is happening.

The RASA 8 has a lot of momentum during dithering on the EQ6R - i had a similar issue with my fully loaded Esprit 120. 

 

Here is what I would do.

 

In EQMOD - slow your sideareal speed to say - .5x to .6x  

 

In your acquisition software, set dither to 3-5 pixels or whatever it is for default

 

In PHD2, open advanced settings, on global tab set dither settings to random and scale to 1.5 (a multiplier instead of fixed dither pixel size).  Then click on Guiding tab and on there click the advanced button then click reset so it re-calculates based off slower sideareal speed.  Click OK

 

unpark mount, start sideareal tracking

 

Click tools -> drift align -> slew to 0 0 (just click slew) and cancel out.  Then re-calibrate PHD2 from here and re-run tuning and maybe re-calibrate after tuning.

 

I find my dithering works MUCH better and i spend a LOT less time recovering with the slower sideareal 

[sn2006gy]

Not sure why you would image a 90 second sub at gain 0 with a ASI 533MC pro when your noise would be greatly reduced at just above unity gain.

I set mine at gain 101 for 120 second subs and the lower noise may help give you a better starting point for dither noise reduction.

He's on a RASA, this would be way over exposed and clipping everything too high.

[Kevin_A]

He's on a RASA, this would be way over exposed and clipping everything too high.

He could drop exposures down to 30-60 seconds..... and have the best of both worlds... and it would lessen mount tracking needs too.

But the caveat then is the dithering time uses up 30-50% of your imaging time if you dither every 2 or so frames.

That is the biggest downside to very fast systems and balancing gain noise levels vs sub length vs dithering % of time... its much easier to balance all the variables with a slower f5.6 setup for sure.

My comments are just to get the base images down to the best noise level for most imagers first and doesn't take into account that if the mount wont settle down in a normal amount of time... as that will screw up shorter imaging sub routines for fast glass.  

Edited by Kevin_A, 27 November 2020 - 12:13 PM.

[arrowspace90]

He could drop exposures down to 30-60 seconds..... and have the best of both worlds... and it would lessen mount tracking needs too.

But the caveat then is the dithering time uses up 30-50% of your imaging time if you dither every 2 or so frames.

That is the biggest downside to very fast systems and balancing gain noise levels vs sub length vs dithering % of time... its much easier to balance all the variables with a slower f5.6 setup for sure.

My comments are just to get the base images down to the best noise level for most imagers first and doesn't take into account that if the mount wont settle down in a normal amount of time... as that will screw up shorter imaging sub routines for fast glass.  

Exactly.  Dithering half of the time was what I was trying to avoid.  My driveway, the only place on my property that has a clear view of Polaris, is in a light canyon between houses and trees.  Some targets only afford a few hours of available observation.  That's why I went RASA in the first place, and I enjoy the scope.  But obviously, I am still not a seasoned professional who has seen it all.  So these suggestions are useful.  I have typically shot 60 second subs with the OSC.  With the ASI1600 mono, it's very difficult to get to 60 seconds.

I will obviously keep working on my problem areas.

[arrowspace90]

Hi,

 

 You need to figure out what causes it.

 

 

Yes, that's why I am letting people see my dirty laundry here.  I don't know what is causing it.  I have now seen it a couple of times.

[arrowspace90]

I don't buy that the problem is caused by dithering. At least for the California.

 

 

Notice there is a rotation to it. It is most obvious at the right side, where the margin gets wider as it goes. It is also noticeable at the top, where the black area at the left side is shorter and the right side is taller. 

 

Dithering does not cause rotation. Polar alignment does. There may be other causes. But I don't think it is dithering. 

 

Alex

I am a decent PA'er at this point.  The PA is always less than an arc minute (said to be excellent) and usually below 30 arc seconds.  I got some increased advice from the creator of APP this morning.  He says to stack with "advanced" normalization instead of default, and also to use first degree LNC with a couple of iterations (which I usually do for multi-session only). And the Multi Band Blending at 15.

There WAS lots of moonlight, which normally could make one side brighter than the other.  However, I was shooting with the L-extreme, which is only supposed to let through Ha and OIII.  So if there is LP, I don't quite get it.  The California was shot with 60second subs, 300 of them, back up at unity gain on the ASI533.  Dithering every 2 subs at 10 pixels.

I am trying his suggestions as I write this post.

[ryanha]

Not sure why you would image a 90 second sub at gain 0 with a ASI 533MC pro when your noise would be greatly reduced at just above unity gain.

I set mine at gain 101 for 120 second subs and the lower noise may help give you a better starting point for dither noise reduction.

 

To give a bit of data behind this, if you look at the camera noise profile you can see there is a good drop when you get to 100 gain (see chart below).

 

This read noise is significant I think.

 

As for dither, here is what my current thinking is (based on some error and trials):

 

I think there are two factors

- # of dithers in your sequence

- Frequency of dither

 

The trade off of dithering is:

- Time (more subs)

- Guiding (unless you use a really long settle guiding is marginally impacted by dithering frequency)

 

The one time that I forgot to dither I had a bunch of hot pixel artifacts I had to remove (1.5 hours on the target of 3 min subs) so I know dither works. 

 

I am honing in on an unsubstantiated rule of thumb for me

- Dither enough to get 5 - 10 dither events in the sequence

- Dither at least every 15 - 20 minutes

 

 

Lastly, I think for DSO (vs planetary) it is better to do long sequences on the same filter for effeciency. So if I have a full night on one target Ill do 2h straight on each RGB and then 4 h on L (or something like that).  Saves on filter settle time, etc.  The down side is if the conditions are going to vary throughout the night in which case there is a case for interleaving, but I would shoot to minimize it.

 

For multi night targets I will do one filter the whole night.

 

--Ryan

Attached Thumbnails

• 

Edited by ryanha, 27 November 2020 - 02:01 PM.