Jump to content

  •  

CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.

Photo

Will this crazy processing idea work on my background mottling?

  • Please log in to reply
41 replies to this topic

#1 pbkoden

pbkoden

    Vostok 1

  • *****
  • topic starter
  • Posts: 188
  • Joined: 21 Aug 2015

Posted 26 April 2017 - 09:58 AM

I have an LRGB image that I am working on processing (Pixinsight FYI). The source data had some very bad gradients due to light pollution spread over ~270 subs. I was able to use dynamic background extraction in Pixinsight to get a "nearly" uniform background, but there is still some very minor mottling. As long as I don't overstretch, it shouldn't be a problem. But... there are some faint background details that I want to bring out. The background details are ever so slightly above the background, but still get easily lost in the background mottling.

 

Would it be possible to add a noise floor to my image to fill in the "valleys" of the background mottling and allow me to stretch and show the faint background detail without accentuating the mottling itself. In my mind I can create a copy of the image with a gaussian noise distribution that matches the values of the high points of my background mottling. I can then do a max( ) operation with pixel math to basically fill in the dark areas of my background mottling. What I should be left with is a nearly uniform background where my faint signal is an outlier.

 

Any chance this will work?

 

 



#2 xiando

xiando

    Aurora

  • *****
  • Posts: 4560
  • Joined: 27 May 2015
  • Loc: Cloudy NEOhio

Posted 26 April 2017 - 10:50 AM

I sometime add baseline to my images so I can hide the CO noise and stretch whatever is there a bit more. I use a levels adjustment layer in PS to do it. Not sure how one would do that in PI, so I'll leave that to those who use it.



#3 pbkoden

pbkoden

    Vostok 1

  • *****
  • topic starter
  • Posts: 188
  • Joined: 21 Aug 2015

Posted 26 April 2017 - 07:11 PM

Okay, the answer is that yes, it works extremely well. The faint signal that I want to show is still too faint to pull out (you can only see it if you are looking for it), but this process will be very useful to me in the future.

 

Here's my process:

  • create an image with matching dimensions (I copied and used CurvesTranformation to turn it blank grey)
  • fill image with noise using NoiseGenerator
  • use CurvesTransformation to stretch it to the same rough background level and noise strength as your luminance. This is tricky. I used a matching ScreenTransferFunction to get close, and statistics probe for fine tuning.
  • Use PixelMath with the Max() operator to add the two together. You can fine tune the strength of the noise image in the expression: Max(lum,fill*1.05)

Here's my before and after (autostretched). The before is on top, and the after is on the bottom.

 

8nqmbxW.jpg

 

The difference in intentionally subtle, but that is what I wanted to do. Now I can stretch the background a little more without being scared of showing the splotchiness. 


  • gunny01, xiando and *Axel* like this

#4 ismosi

ismosi

    Viking 1

  • -----
  • Posts: 569
  • Joined: 19 Apr 2014
  • Loc: New London, Pennsylvania, USA

Posted 26 April 2017 - 07:57 PM

I wondered why there are tools in PI to add noise. Now I know at least one reason. Nice!



#5 xiando

xiando

    Aurora

  • *****
  • Posts: 4560
  • Joined: 27 May 2015
  • Loc: Cloudy NEOhio

Posted 26 April 2017 - 08:20 PM

I wondered why there are tools in PI to add noise. Now I know at least one reason. Nice!

Yes, it's been standard fare in image processing tools for decades. :)



#6 Thirteen

Thirteen

    Mercury-Atlas

  • *****
  • Posts: 2865
  • Joined: 12 Jul 2013
  • Loc: Milford, Michigan

Posted 26 April 2017 - 08:51 PM

Interesting method. I assume you are trying to pull out those tidal loops. Good luck with that, I just captured 16.25 hours of Lum on this and I haven't seen squat. Some hints here and there but that stuff is FAINT!
  • FiremanDan and roofkid like this

#7 Jon Rista

Jon Rista

    Hubble

  • *****
  • Posts: 18089
  • Joined: 10 Jan 2014
  • Loc: Colorado

Posted 26 April 2017 - 08:55 PM

Phil, just a side note. Are you sure that background structure is actually an artifact...and not actually some real structure? Any chance it is IFN (which as time goes on and more and more sensitive cameras are used, seems to be quite prolific throughout the galaxy season skies...)?


  • calypsob likes this

#8 schmeah

schmeah

    Soyuz

  • *****
  • Posts: 3673
  • Joined: 26 Jul 2005
  • Loc: Morristown, NJ

Posted 26 April 2017 - 10:12 PM

I've been doing this for a long time in photoshop, using a generic flat background that I adjust according to the background "noise" level of my target image. It works a charm. In fact with my skies I could never get a decent looking deep shot without it. As Jon mentioned, however, there is always that risk of blending away something real that is very faint.

 

Derek


Edited by schmeah, 26 April 2017 - 10:12 PM.

  • Jon Rista likes this

#9 pbkoden

pbkoden

    Vostok 1

  • *****
  • topic starter
  • Posts: 188
  • Joined: 21 Aug 2015

Posted 27 April 2017 - 06:02 AM

Phil, just a side note. Are you sure that background structure is actually an artifact...and not actually some real structure? Any chance it is IFN (which as time goes on and more and more sensitive cameras are used, seems to be quite prolific throughout the galaxy season skies...)?

 

I compared my image with others online as a reference. I think I can make out some of the stronger tidal loop structure but only because I know where to look. But as far as I can tell, there is no other faint nebulosity in this region. Plus, my gradients were all over the place. Here is image pre background extraction.

 

A9PXkzp.jpg



#10 pbkoden

pbkoden

    Vostok 1

  • *****
  • topic starter
  • Posts: 188
  • Joined: 21 Aug 2015

Posted 27 April 2017 - 06:04 AM

Interesting method. I assume you are trying to pull out those tidal loops. Good luck with that, I just captured 16.25 hours of Lum on this and I haven't seen squat. Some hints here and there but that stuff is FAINT!

 

What is your sky quality like? I'm about 18.5 or so SQM. I will be moving in a little over a year to something more like 20.5, and will have to try this again.

 

Even in my current skies, I think I've started to see some. Here is a close up of the galaxy. It's just not bright enough to pull up without also blowing up some of my less-ideal background areas.

 

jBKbSKc.jpg


Edited by pbkoden, 27 April 2017 - 06:06 AM.


#11 HunterofPhotons

HunterofPhotons

    Apollo

  • -----
  • Posts: 1294
  • Joined: 26 Apr 2008
  • Loc: Rhode Island, USA

Posted 27 April 2017 - 09:47 AM

Phil, just a side note. Are you sure that background structure is actually an artifact...and not actually some real structure? Any chance it is IFN (which as time goes on and more and more sensitive cameras are used, seems to be quite prolific throughout the galaxy season skies...)?

 

Phil,

A gentle note of caution: you may want to reevaluate your additive technique in order to get a result that you prefer.  As they say, it's a slippery slope.

You say that you didn't subtract out any IFN because you didn't see it in other's images.  What's to say that they didn't do what you're doing, painting in a background that they wanted to see rather than the one that was actually captured?

As a second step, why not add in a tidal loop where you know there is one?  It's easy enough to do in most processing programs, a trivial exercise.

As Dave Chappell was wont to point out, there's nothing wrong with 'keeping it real'.

 

dan k.



#12 pbkoden

pbkoden

    Vostok 1

  • *****
  • topic starter
  • Posts: 188
  • Joined: 21 Aug 2015

Posted 27 April 2017 - 10:51 AM

Dan,

 

My addition of a flat background noise level is a global operation, which I consider completely acceptable in processing. It won't work in all situations, but I think it did in this one.

 

I don't consider manual manipulation of data to get an expected result is acceptable though. If I didn't capture it with my camera and telescope, then I can't in good conscience present a finished product as if I did. To each his own, but I'm not willing to falsify my data, whether it's realistic or not.


  • schmeah, Jon Rista and xiando like this

#13 xiando

xiando

    Aurora

  • *****
  • Posts: 4560
  • Joined: 27 May 2015
  • Loc: Cloudy NEOhio

Posted 27 April 2017 - 11:22 AM

Yeah, looking at your two images, I see the difference. The background in the second is more uniform. And the auto-stretch is not as affected by discontinuities, so it's able to hit a slightly darker, yet, still ~uniform tone for the background.

 

While it obviously won't add any data that's not already there, it's a good mask for not-quite-enough-leftover-integration-time after noise reduction, to save what otherwise might not be saveable. smile.gif

 

Does PI include a version of despeckle? I generally use that after applying bias to the background to further smooth residual discontinuities into the new baseline. 

 

rant on

I was afraid someone would snark about 'real vs unreal'. It seems to be an automatic comeback by certain members of the community to prove they're superior, or perhaps more accurately, to suggest others are inferior to them. Adding a little bias to the background level after encountering sporadic background levels due to underwhelming integration time and/or following noise reduction is not by even a leap equivalent to painting in substantive features, and commenting as though it does, or suggesting they're committing artistic fraud by adding a bit of back ground bias,  does little to prove anyone's superiority. Good grief. Maybe drizzle should be outlawed, or dithering, or gradient treatments, or for that matter any processing of any kind.
rant off

 


Edited by xiando, 27 April 2017 - 12:24 PM.


#14 Thirteen

Thirteen

    Mercury-Atlas

  • *****
  • Posts: 2865
  • Joined: 12 Jul 2013
  • Loc: Milford, Michigan

Posted 27 April 2017 - 12:20 PM


Interesting method. I assume you are trying to pull out those tidal loops. Good luck with that, I just captured 16.25 hours of Lum on this and I haven't seen squat. Some hints here and there but that stuff is FAINT!


What is your sky quality like? I'm about 18.5 or so SQM. I will be moving in a little over a year to something more like 20.5, and will have to try this again.

Even in my current skies, I think I've started to see some. Here is a close up of the galaxy. It's just not bright enough to pull up without also blowing up some of my less-ideal background areas.

jBKbSKc.jpg

Yeah, that's about what I saw in my stack, maybe a bit on the other side too. Turns out I had a little problem with flat correction and my configuration had changed a bit, so I need to build a synthetic flat now. It's too much data to toss!

I've never measured my SQM, but it's a bit darker than where you are. I'm afraid of how much data I'd need to actually pull this out.

#15 FiremanDan

FiremanDan

    Soyuz

  • -----
  • Posts: 3990
  • Joined: 11 Apr 2014
  • Loc: Virginia

Posted 27 April 2017 - 01:55 PM

This sounds interesting. Is there a good tutorial for this anyone can suggest? 



#16 Jon Rista

Jon Rista

    Hubble

  • *****
  • Posts: 18089
  • Joined: 10 Jan 2014
  • Loc: Colorado

Posted 27 April 2017 - 02:02 PM

 

Interesting method. I assume you are trying to pull out those tidal loops. Good luck with that, I just captured 16.25 hours of Lum on this and I haven't seen squat. Some hints here and there but that stuff is FAINT!

 

What is your sky quality like? I'm about 18.5 or so SQM. I will be moving in a little over a year to something more like 20.5, and will have to try this again.

 

Even in my current skies, I think I've started to see some. Here is a close up of the galaxy. It's just not bright enough to pull up without also blowing up some of my less-ideal background areas.

 

jBKbSKc.jpg

 

Based on these images, it does look like there is some faint dust in there...in addition to the tidal stream. Cool stuff!



#17 Thirteen

Thirteen

    Mercury-Atlas

  • *****
  • Posts: 2865
  • Joined: 12 Jul 2013
  • Loc: Milford, Michigan

Posted 27 April 2017 - 03:55 PM

It's not just one tidal stream, there are actually two complete loops around the galaxy. Well, maybe it is one just really long one? It almost looks helical in some images. The images that capture it are very rare, with only a few really nice examples. It's outrageously faint for light polluted skies, I can attest to that.

If you want to see, Google "NGC5907". There are some deeply stretched examples for your perusal.

Edited by Thirteen, 27 April 2017 - 03:56 PM.


#18 gunny01

gunny01

    Viking 1

  • *****
  • Posts: 706
  • Joined: 02 Jun 2014

Posted 28 April 2017 - 07:14 AM

Yeah, looking at your two images, I see the difference. The background in the second is more uniform. And the auto-stretch is not as affected by discontinuities, so it's able to hit a slightly darker, yet, still ~uniform tone for the background.

 

While it obviously won't add any data that's not already there, it's a good mask for not-quite-enough-leftover-integration-time after noise reduction, to save what otherwise might not be saveable. smile.gif

 

Does PI include a version of despeckle? I generally use that after applying bias to the background to further smooth residual discontinuities into the new baseline. 

 

rant on

I was afraid someone would snark about 'real vs unreal'. It seems to be an automatic comeback by certain members of the community to prove they're superior, or perhaps more accurately, to suggest others are inferior to them. Adding a little bias to the background level after encountering sporadic background levels due to underwhelming integration time and/or following noise reduction is not by even a leap equivalent to painting in substantive features, and commenting as though it does, or suggesting they're committing artistic fraud by adding a bit of back ground bias,  does little to prove anyone's superiority. Good grief. Maybe drizzle should be outlawed, or dithering, or gradient treatments, or for that matter any processing of any kind.
rant off

 

  Interesting tool to add in PI.  Thanks for this one Phil.

 

   Xiando has a point about the criticism that sometimes gets dealt out.  I one had a nice nb image, or so I thought, that I posted on the FLI website.  Because I had gotten rid of the magenta rings around stars  that are common with nb, I was criticized for not following processing ethics.  If someone could direct me to the book for processing ethics, let me know.  Keller's book has been my guide for a year now and I don't have enough smarts to try too much outside of what is shown in that publication.

 

  I'm always trying to learn new processing techniques and am really impressed with the new processes' that PI has introduced, along with the ways you guys utilize this tool.  I'll always be my harshest critic, but as long as I can turn out a decent image, that's all I'm concerned about. 

 

  The help I've received here has been great, but I do have to agree that some get a little too critical and esoteric with the philosophical points.   I especially don't like it when beginners are looking for simple answers and are barraged with highly technical information.  This seems to turn the aspiring imager into a forget this hobby type.  Haven't seen a lot of that here, but I have seen it displayed on other astroimaging websites.


  • scopenitout and xiando like this

#19 calypsob

calypsob

    Mercury-Atlas

  • *****
  • Posts: 2658
  • Joined: 20 Apr 2013
  • Loc: Roanoke, Virginia

Posted 28 April 2017 - 10:52 AM

Phil, just a side note. Are you sure that background structure is actually an artifact...and not actually some real structure? Any chance it is IFN (which as time goes on and more and more sensitive cameras are used, seems to be quite prolific throughout the galaxy season skies...)?

Exactly what I was thinking, that is not all mottling.  



#20 calypsob

calypsob

    Mercury-Atlas

  • *****
  • Posts: 2658
  • Joined: 20 Apr 2013
  • Loc: Roanoke, Virginia

Posted 28 April 2017 - 10:56 AM

That whole region is one giant dust cloud, NGC 4244 right?  Take your image after DBE and throw it in PS and do a star removal, kill all of the bright stars with a  brush and run equalize.  You will see all of the dust then. It will also give you an idea how far your signal is off of the noise floor. 


Edited by calypsob, 28 April 2017 - 10:57 AM.


#21 pbkoden

pbkoden

    Vostok 1

  • *****
  • topic starter
  • Posts: 188
  • Joined: 21 Aug 2015

Posted 28 April 2017 - 11:35 AM

There may be dust in that area, but it doesn't make sense that I've been able to pull it out. Going by much higher quality images than mine of the area around this galaxy, any dust that is out there is fainter than the tidal streams. I know I don't have enough integration to pull out the tidal streams, so therefore I don't have enough integration to pull out the faint dust.



#22 Jon Rista

Jon Rista

    Hubble

  • *****
  • Posts: 18089
  • Joined: 10 Jan 2014
  • Loc: Colorado

Posted 28 April 2017 - 09:06 PM

There may be dust in that area, but it doesn't make sense that I've been able to pull it out. Going by much higher quality images than mine of the area around this galaxy, any dust that is out there is fainter than the tidal streams. I know I don't have enough integration to pull out the tidal streams, so therefore I don't have enough integration to pull out the faint dust.

I dunno. Would be hard to know without knowing exactly how the few images that show the streams clearly were processed. I did find this, which seems to indicate that there is more out there besides the streams, but, it would be about the same brightness or fainter:

 

http://www.cosmotogr...C5907invert.jpg

 

I do see some pockets of information in your image that I think could be either part of the streams or maybe some dust. But yes, would need deeper integration to know for sure. ;)



#23 schmeah

schmeah

    Soyuz

  • *****
  • Posts: 3673
  • Joined: 26 Jul 2005
  • Loc: Morristown, NJ

Posted 29 April 2017 - 11:35 AM

For you Luddites still using Photoshop. Attached is a non descript corner crop  of a recent LRGB image showing the typical splotchy uneven background than I am left with after stretching an image, followed by the same crop immediately after applying my flat background. My experience is that if I adjust the K value carefully enough it will leave fine detail so long as it is even slightly above the background noise level. I'm sure I could find better examples of this. This is just a quickie.

 

My flat background was created from an old LRGB image from which I removed all stars and DSO, applying blurring, dust and scratches filters similar to JPMs tone mapping technique. I level adjust the K value of the background just below (lower is brighter) the most uniform background level of the object image. Then I combine it with a lighten blend and fade according to taste.

 

Derek

 

BackgroundTestCrop.jpg

 

BackgroundTestCropBG.jpg

 

 

 

 


  • pbkoden likes this

#24 pfile

pfile

    Skylab

  • -----
  • Posts: 4201
  • Joined: 14 Jun 2009

Posted 29 April 2017 - 02:16 PM

we did a pretty deep image of this galaxy from SRO with a C300 @ 2700mm and if there is any IFN there, it's very, very, very faint; as speculated upthread it would be on the order of the brightness of the tidal loop itself if it exists. in the end i never processed the image as the tidal loop was maybe 1-2 ADU above the noise floor and i couldn't get the results i was looking for. before we could put more time on the object we switched focal reducers to 1400mm and abandoned the project.

 

rob

 

edit: as for adding noise to the BG, it is something that i've done from time to time and while there are risks usually the situation that you're using it in (high LP) you're not going to be picking up too many faint details anyway. if you go for 50 hours or something to counteract that, then you probably don't need to be adding noise to the background... i don't really see an 'ethics' problem here :)


Edited by pfile, 29 April 2017 - 02:18 PM.


#25 dkeller_nc

dkeller_nc

    Viking 1

  • *****
  • Posts: 921
  • Joined: 10 Jul 2016
  • Loc: Central NC

Posted 29 April 2017 - 02:30 PM

Nevermind the presence/absence of faint details or the "processing ethics" debate (which is absurd, if you ask me.  As long as you're not painting in structures, who cares?), I'm interested in why this worked at all.  One would think that if you're essentially adding a "pedestal" to all of the pixel values in the image, then the uneven background would still be there.  Unless, of course, the original, processed image has some of the background pixels clipped, but still preserves the numerical information so that when an offset is added, it brings those pixels away from dead black, and our eyes see this as less of a contrast difference?




CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.







Cloudy Nights LLC
Cloudy Nights Sponsor: Astronomics