Jump to content

  •  

CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.

Photo

Strange pattern in stacked images (not walking noise)

  • Please log in to reply
21 replies to this topic

#1 mistateo

mistateo

    Apollo

  • *****
  • topic starter
  • Posts: 1,310
  • Joined: 02 Feb 2017
  • Loc: San Diego, CA

Posted 23 January 2020 - 06:25 PM

So I am getting a strange artifact in my stacked files that I can't explain.  It is a strange looking pattern in the background that prevents me from stretching my processed images as much as I like, and making me over darken the background (like a total rookie) to hide the defect.  The stacked image should show exactly what I am referring to.  The issue is MUCH more pronounced on the stacked image, which to me indicates it is signal (however unwanted it may be), but I can't even see it in a calibrated single. Any ideas on what it may be?

 

Full disclosure: the astronomik L2 is 50mm unmounted, placed under the QHY rotator only a few mm from the AR coated optical window, about 20mm or so from sensor.  I suppose I can do another integration without the filter to see if the problem goes away

Equipment:

Orion Eon 120mm w/.85x reducer

QHY247C with Astronomik L2 filter "DSO preset, 2126 Gain, 76 offset"

 

Integration:

30 darks

30 dark flats (2 sec)

30 flats (2 sec, t-shirt plus ipad, 21k adu mean)

 

30 lights (3 min) for stacked image (dithered every other frame)

1 light (3 min) for single calibrated image

 

Processing on both images:

Crop overscan (24 pixels on each axis)
DBE

Background Neutralization

Color Calibration

Histogram Transformation (over stretched to exaggerate issue)

 

Stacked Image:

 
QHY247C Astronomik L2 m101 artifact stacked

Single Image:

 

QHY247C Astronomik L2 m101 artifact single


Edited by mistateo, 23 January 2020 - 06:52 PM.


#2 scadvice

scadvice

    Surveyor 1

  • *****
  • Vendors
  • Posts: 1,776
  • Joined: 20 Feb 2018
  • Loc: Lodi, California

Posted 23 January 2020 - 06:43 PM

You didn't say whether you dithered or not. But assuming you did because of you title...maybe a problem with the dark flats? I'm just guessing here... 


Edited by scadvice, 23 January 2020 - 07:22 PM.


#3 Jim Waters

Jim Waters

    Soyuz

  • *****
  • Posts: 3,626
  • Joined: 21 Oct 2007
  • Loc: Phoenix, AZ USA

Posted 23 January 2020 - 06:53 PM

I would eliminate a filter issue.  Image without the Astronomik L2 filter.  Was there dew when imaging?



#4 sharkmelley

sharkmelley

    Mercury-Atlas

  • *****
  • Posts: 2,840
  • Joined: 19 Feb 2013

Posted 23 January 2020 - 07:18 PM

You have a severe posterization issue.

 

The first thing to check is whether or not the problem exists in the stacked data - try cropping a central region and performing a screen transfer function without doing DBE or anything similar.

 

If it's not in the stacked data then something in the postprocessing has caused it - eliminate them one by one..

If it is in the stacked data then check your master dark, master flat, master bias for problems.

Also check you are not doing anything daft during ImageIntegration e.g. "Combination" set to Median.

 

The other possibility is that the data is fine but it is being displayed weirdly.  Try examining the image at 100% scale.  Try stretching the background even further with HistogramTransformation - does the posterization disappear or move position?

 

Mark


Edited by sharkmelley, 23 January 2020 - 07:24 PM.


#5 pfile

pfile

    Fly Me to the Moon

  • -----
  • Posts: 5,426
  • Joined: 14 Jun 2009

Posted 23 January 2020 - 07:38 PM

also another thing to try is to turn on 24-bit STF and see if the posterization goes away.

 

even if it does, something is weird per mark's advice.

 

rob


  • sharkmelley and Jon Rista like this

#6 sharkmelley

sharkmelley

    Mercury-Atlas

  • *****
  • Posts: 2,840
  • Joined: 19 Feb 2013

Posted 24 January 2020 - 05:23 AM

also another thing to try is to turn on 24-bit STF and see if the posterization goes away.

 

even if it does, something is weird per mark's advice.

 

rob

Here's how to set 24-bit LUT:

Edit -> GlobalPreferences -> MiscellaneousWindowSettings -> Use24-bitScreenTransferFunctionLUTs

 

Mark


  • Jim Waters and Jon Rista like this

#7 gatsbyiv

gatsbyiv

    Viking 1

  • *****
  • Posts: 703
  • Joined: 29 Mar 2009
  • Loc: Doylestown, PA

Posted 24 January 2020 - 08:01 AM

Look at the DBE background model image.  Does the pattern you see there match the background issue in the final image? 

 

Is the spacing between each of these bands about 3 minutes of motion across the sky for the objects declination?  It could just be moonglow (was the moon out?) or a light pollution gradient on one side of the sky.



#8 happylimpet

happylimpet

    Skylab

  • *****
  • Posts: 4,353
  • Joined: 29 Sep 2013
  • Loc: Southampton, UK

Posted 24 January 2020 - 10:04 AM

What stacking algorithm are you using? Not median is it? Just a thought. If so try average or kappa-sigma.

 

Note this problem could arise from your flats or darks also. Try stacking without each in turn and see if the posterization disappears. My guess would be its in your flats.



#9 mistateo

mistateo

    Apollo

  • *****
  • topic starter
  • Posts: 1,310
  • Joined: 02 Feb 2017
  • Loc: San Diego, CA

Posted 24 January 2020 - 02:04 PM

Thanks everyone for the replies.  I am using Kappa-Sigma for stacking, not median.  There was no moon out, the sky background in DSS (yes, I need to learn how to integrate images with PI) was 0.6 %, so reasonably dark for my bortle 5 sky.  The sequence was dithered every other frame with "very high dither" setting in SGP, so I am confident it isn't walking noise.

 

What I have done since writing the post:

I tried reintegrating without dark flats (also not using bias) and the end result looked exactly the same!  I wasn't expecting that... Should I try a brighter screen, shorter flats and bias frames instead?  I used to do this with my ASI1600 but it produced fixed pattern noise, so I switched to dark flats and longer flats with a dimmer setting on the ipad and the issue went away.

 

I tried reintegrating with only lights and darks.  As expected, it shows a little vignetting in the corners.  When I ran DBE using division for this attempt, I ended up with a "bullseye" looking gradient.  Plus I don't want to image without flats, so this was just a test.  What I find interesting about this approach is that although I saw the expected dimming in the corners, I saw essentially no "LP" gradient running from left to right like I do when I integrate with flats/flat darks.

 

My observations:

I find it odd that I see no trace of this when calibrating a single image and then running DBE.  I also find it odd that I see no trace of the pattern before running DBE on the stack.  It starts as a "bright on left side" gradually dimming towards the other side, standard LP looking gradient.  Once I run DBE, the brightness of the gradient is pretty uniform across the image, but I get those weird color bands shown in the photo for the original post.

 

DBE Process:

My DBE process has been to click generate, then to bump up "tolerance" until the bright gradient areas are all covered, then bump down minimum sample weight until dark areas were covered.  I would then inspect every sample to ensure it wasn't placed on an area with desirable signal, or have any stars in the samples.  When shooting small galaxies, not in IFN areas, with relatively few stars, this should be an easy process...

 

Next tests:

I have been reading Jon Rista's blog on DBE and I found something I have been doing that is probably VERY wrong.... Instead of manipulating "shadows relaxation" to include more dark areas, I was lowering the "minimum sample weight" to get those areas included when generating samples.  I will try again now with the new found knowledge.


Edited by mistateo, 24 January 2020 - 02:07 PM.


#10 pfile

pfile

    Fly Me to the Moon

  • -----
  • Posts: 5,426
  • Joined: 14 Jun 2009

Posted 24 January 2020 - 03:15 PM

Here's how to set 24-bit LUT:

Edit -> GlobalPreferences -> MiscellaneousWindowSettings -> Use24-bitScreenTransferFunctionLUTs

 

Mark

 

be careful with this though - the amount of memory consumed by every view will triple because of the 24-bit LUT. you don't need to turn this on globally - you can just click the little monitory with the "24" in the 'intensity transformations" toolbar (or choose Image > Screen Transfer Functions > Use 24-Bit LUTS) to turn it on and off on a per-view basis.

 

rob


  • sharkmelley likes this

#11 sharkmelley

sharkmelley

    Mercury-Atlas

  • *****
  • Posts: 2,840
  • Joined: 19 Feb 2013

Posted 24 January 2020 - 03:20 PM

be careful with this though - the amount of memory consumed by every view will triple because of the 24-bit LUT. you don't need to turn this on globally - you can just click the little monitory with the "24" in the 'intensity transformations" toolbar (or choose Image > Screen Transfer Functions > Use 24-Bit LUTS) to turn it on and off on a per-view basis.

Thanks - as always there are so many slightly different ways of doing things!

 

Mark


  • pfile likes this

#12 freestar8n

freestar8n

    Vendor - MetaGuide

  • *****
  • Vendors
  • Posts: 9,304
  • Joined: 12 Oct 2007

Posted 24 January 2020 - 04:48 PM

This looks to me like a modulation of noise caused by a combination of warping the image for alignment in the stack, and interpolation.

 

It looks like you are using a deBayering algorithm rather than drizzle deBayer - and I would try drizzle deBayer with nearest neighbor interpolation, if possible.

 

Does the single image have any warping or shifting for alignment - or is it calibrated as a pure single image?  The bands in it are harder to explain if it wasn't warped somehow.

 

I don't think it's simply posterization because the stacked image has an overall flat background with color bands across it.  If it had a steady linear gradient then posterization could explain it - but this is overall flat.  Hence I think it is the noise in each color channel that is modulated and that gives the appearance of bright bands of color - and interpolation combined with warping can cause that.

 

Frank


Edited by freestar8n, 24 January 2020 - 04:51 PM.


#13 mistateo

mistateo

    Apollo

  • *****
  • topic starter
  • Posts: 1,310
  • Joined: 02 Feb 2017
  • Loc: San Diego, CA

Posted 24 January 2020 - 09:49 PM

Nothing I did with DBE settings seemed to matter.  BUT I did something different.  I saved the output image in DSS with 32 bit Rational TIFF instead of 16 bit TIFF and POOF, no more garbage gradients.  This image was just loaded into Pixinsight, cropped out the black overscan area, ran the exact same dbe settings as before, and overstretched.

 

FINALLY.jpg


  • pfile and ks__observer like this

#14 mistateo

mistateo

    Apollo

  • *****
  • topic starter
  • Posts: 1,310
  • Joined: 02 Feb 2017
  • Loc: San Diego, CA

Posted 24 January 2020 - 09:50 PM

My apologies if this was a total facepalm moment for anyone.  I just figured 32 bit vs 16 bit wouldn't matter much since I was shooting with a 14 bit camera.  Lesson learned (hopefully).



#15 jhayes_tucson

jhayes_tucson

    Fly Me to the Moon

  • *****
  • Posts: 7,388
  • Joined: 26 Aug 2012
  • Loc: Bend, OR

Posted 24 January 2020 - 11:14 PM

That raises the question:  Why are you stacking in DSS if you have PixInsight?

 

John


  • Jon Rista likes this

#16 sharkmelley

sharkmelley

    Mercury-Atlas

  • *****
  • Posts: 2,840
  • Joined: 19 Feb 2013

Posted 24 January 2020 - 11:26 PM

My apologies if this was a total facepalm moment for anyone.  I just figured 32 bit vs 16 bit wouldn't matter much since I was shooting with a 14 bit camera.  Lesson learned (hopefully).

I'm glad you found the cause.  Posterization issues can be very difficult to track down.

 

Mark



#17 mistateo

mistateo

    Apollo

  • *****
  • topic starter
  • Posts: 1,310
  • Joined: 02 Feb 2017
  • Loc: San Diego, CA

Posted 24 January 2020 - 11:35 PM

That raises the question:  Why are you stacking in DSS if you have PixInsight?

 

John

I don't have a good answer for that one.  Haven't learned how yet.  I previously tried to find a workflow example for batch preprocessing that uses dark flats instead of bias frames but was unsuccessful.  I really do need to stop procrastinating on that though.  DSS may be fast and easy, but PI integration probably yields a better result.



#18 mistateo

mistateo

    Apollo

  • *****
  • topic starter
  • Posts: 1,310
  • Joined: 02 Feb 2017
  • Loc: San Diego, CA

Posted 25 January 2020 - 02:07 AM

Here is the end result.  The imaging conditions were pretty lousy and the FWHM values are higher than on any photo I have processed to date.  Since the targets are really small and I am soon to have an Edge HD9.25 setup, I am not going to bother getting any more data on this with my current setup.  So here is 90 minutes, overly dark background to hide some noise, but not bad considering where I started with this data...

 

49438042812_1e84d1abca_b.jpg


  • jhayes_tucson, Lead_Weight and ks__observer like this

#19 mrpizza

mrpizza

    Vostok 1

  • *****
  • Posts: 118
  • Joined: 30 Apr 2010
  • Loc: Palm Coast, FL

Posted 25 January 2020 - 10:50 AM

I don't have a good answer for that one.  Haven't learned how yet.  I previously tried to find a workflow example for batch preprocessing that uses dark flats instead of bias frames but was unsuccessful.  I really do need to stop procrastinating on that though.  DSS may be fast and easy, but PI integration probably yields a better result.

I would be interested if you find something because that is where I am at right now.  I do all my preprocessing manually in PI and looking to automate it using dark flats.  Nice image by the way.

 

Chris



#20 Tayson82

Tayson82

    Vostok 1

  • -----
  • Posts: 100
  • Joined: 17 May 2018
  • Loc: Poland / Wolomin

Posted 29 January 2020 - 06:04 AM

Sometimes after 2 or 3 times used DBE/ABE I have similar background.

Solution im my case is 1times DBE/ABE on linear, and another 1-2times DBE/ABE on non-linear photo.



#21 Jon Rista

Jon Rista

    ISS

  • *****
  • Posts: 24,085
  • Joined: 10 Jan 2014
  • Loc: Colorado

Posted 30 January 2020 - 11:38 AM

I don't have a good answer for that one.  Haven't learned how yet.  I previously tried to find a workflow example for batch preprocessing that uses dark flats instead of bias frames but was unsuccessful.  I really do need to stop procrastinating on that though.  DSS may be fast and easy, but PI integration probably yields a better result.

Definitely take the time to learn how to integrate with PI. And not just BPP, which has its own issues. Learn the manual integration process. It is not as bad as it is often made out to be, and when you learn how to control the things that matter, your results can definitely be superior. 


  • jhayes_tucson likes this

#22 jhayes_tucson

jhayes_tucson

    Fly Me to the Moon

  • *****
  • Posts: 7,388
  • Joined: 26 Aug 2012
  • Loc: Bend, OR

Posted 30 January 2020 - 09:45 PM

I completely agree with Jon.  Manual calibrating and stacking in PI is not hard and it provides an opportunity to check your results after each step.  It does take a bit more time but I never mind working my way through the data to get to the end result.  That's a big part of what processing an image is all about.

 

John


  • Jon Rista and mistateo like this


CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.


Recent Topics






Cloudy Nights LLC
Cloudy Nights Sponsor: Astronomics