Jump to content

  •  

CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.

Photo

Need some help on an experiment...

  • Please log in to reply
23 replies to this topic

#1 sbharrat

sbharrat

    Vanguard

  • *****
  • topic starter
  • Posts: 2,084
  • Joined: 28 Nov 2020
  • Loc: NJ, USA

Posted 03 February 2024 - 02:51 PM

So the imaging conditions here have been maddening bangbang.gif  In the downtime, I started work on an experiment. With the various recent discussions about deconvolution, I wanted to understand a bit better how much detail can be recovered from classical deconvolution. Unfortunately, my skills at RL are pretty bad and hence not representative. But I know that many of you on this forum are experienced at this so I am soliciting your help. fingerscrossed.gif

 

In https://drive.google...?usp=drive_link you can find an image of M101 in both fits and xisf format.

 

Here are the details about the starting image:

- Obtained from the Insight Observatory starbase database: https://starbase.ins...om/imageset/218

- Taken with PlaneWave CDK 12.5 f/8 scope and ZWO ASI6200MM Pro in the Utah Desert

- This is luminance only, 60m aggregate exposure

- Image scale is 0.307"/px and worst FWHM region according to PI script is 2.8"

 

Some details about the image provided:

- My image scale is about 0.85"/px. I resampled* one of my images to make it approximately the same image scale

- Used PSFImage on that image to obtain PSF. 

- Convolved the IO image with my PSF to obtain the final image

- Worst FWHM region according to PI script is now 4.1"

 

* I have some concerns about this resample followed by extracting the PSF. But the resulting PSF looked the same in profile to obtaining it on the non-resampled image, just wider in pixels now. 

 

What I am hoping to get is a few deconvolved versions of this image. The directory above should have write privileges for you to upload. Hopefully I can entice a couple of you to try your hand at it, if only for curiosity sake. Or sense of competition. Or sense of altrusim. Whatever works for you grin.gif

 

I haven't quite figured out how to quantify "goodness" and will probably have questions about that. But my initial pass of this is just trying to see at a qualitative level what "good" deconvolution looks like compared to what I did. 

 

Appreciate your effort and thanks in advance

 

Update: if you use PI, would be very interested in the deconv settings you used


Edited by sbharrat, 03 February 2024 - 02:53 PM.


#2 Jon Rista

Jon Rista

    ISS

  • *****
  • Posts: 26,034
  • Joined: 10 Jan 2014
  • Loc: Colorado

Posted 03 February 2024 - 04:02 PM

So, the key thing with PI decovolution is regularization. Its kind of odd describing what regularization is, but the best way I've come up with is to say that regularization protects the algorithm from the noise. Without proper regularization, the noise will cause an error in the iterations of the deconvolution process, which leads to artifacts (i.e. things like ringing). Getting a good PSF is important, for sure...but I think a lot of people forget that they also need to properly regularize, and when they do not, they run the high risk of artifacts being generated as part of the process. Ideally, if you configure the regularization settings properly for a given image, you should NOT need to use a separate mask!!

 

Its been a while since I've done any deconvolution, so I'll have to remember how I approached configuring the regularization settings in the past. I'll see how it goes and share my results. 


  • sbharrat likes this

#3 sbharrat

sbharrat

    Vanguard

  • *****
  • topic starter
  • Posts: 2,084
  • Joined: 28 Nov 2020
  • Loc: NJ, USA

Posted 05 February 2024 - 08:44 AM

So I spent a good chunk of yesterday working on this. Basic approach was from Adam Block's Fundamental videos but I intentionally did not use a mask over the object (given my goal here of seeing what "real" deconvolution would do overall). I used a star mask as a local support created using the StarMask process. For the PSF, I used Harmut's PSFImage script.

 

For the LR deconvolution itself, there are so many degrees of freedom that it was seemingly random (though informed by AB and other YT videos). I played with the global dark setting starting from the default and then lowering it until there was no divergence warnings in 10 iterations. I kept lowering until I started noticing the ringing and then raised it again. Then I started increasing the iterations until ringing appeared again. For the regularization settings, I used the defaults and didn't spend any time playing with that. In the end, I ended up with:

- external PSF from PSFImage

- global dark 0.05

- iterations 12

- local deringing enabled with local support from star mask

- 2 wavelet regularization layers: 1) noise threshold 3.0, reduction 1.0; 2) noise threshold 2.00, reduction 0.7

 

I uploaded my deconvolved image and the PSF from PSFImage to the same directory.

 

Definitely some detail is recovered. So even with my non-experienced hands, if there isn't that much noise it seems worth it to spend some time on it. I don't know if it will show in the following gif but both the convolved and then convolved-deconvolved images are in https://drive.google...usp=drive_link 

 

m101-conv-vs-deconv.gif

 

update: you might need to click on image for gif to play. Better resolution at link below

 

https://drive.google...?usp=drive_link


Edited by sbharrat, 05 February 2024 - 08:47 AM.


#4 sbharrat

sbharrat

    Vanguard

  • *****
  • topic starter
  • Posts: 2,084
  • Joined: 28 Nov 2020
  • Loc: NJ, USA

Posted 05 February 2024 - 09:20 AM

Now to the comparison with the reference. Below is the comparison of the same crop of the above but this time comparing my deconvolved image against the original reference. It is too big to include inline here so you can see the gif at https://drive.google...?usp=drive_link

 

Important differences I see

- the additional detail within the galaxy and the small stars within the galaxy (well, in front of the galaxy but you know what I mean) is very close at least visually. 

- Original larger stars are much smaller than the deconvolved versions. I don't know if this has something to do with my StarMask but none of these stars were included in the mask. And these stars were shrunk, but just nowhere as small as the original

 

I would really like to understand which of these remaining differences are due to my lack of skill and which are inherent in the (ill-formed) deconvolution problem. For that, I ASK AGAIN FOR SOME OF YOUR EXPERTISE praying.gif . I would greatly appreciate it if you can try your hand at deconvolving the image and uploading your result to the directory. 

 

Thanks in advance. 

 

PS: It would be nice to be able to quantify the differences between the reference and the attempt. Any suggestions on imaging metrics I should be researching?  

 

 



#5 Jon Rista

Jon Rista

    ISS

  • *****
  • Posts: 26,034
  • Joined: 10 Jan 2014
  • Loc: Colorado

Posted 05 February 2024 - 12:10 PM

Shaun, any chance you could integrate with the proper rejection settings to eliminate those airplane/satellite trails? 



#6 Jon Rista

Jon Rista

    ISS

  • *****
  • Posts: 26,034
  • Joined: 10 Jan 2014
  • Loc: Colorado

Posted 05 February 2024 - 12:52 PM

Well, I've been working on this for a bit here. The noise profile is....unusual, and I'm having a hard time handling it with the regularization settings.

 

You mentioned rescaling something...which channel was that? I am wondering if I'm trying to work with the rescaled channel here. For regularization to work properly, you do need to have a fairly normal noise profile...and the profile of the image I am working with is extremely soft, blurred. I normally work on deconvolution with images where the noise profile is basically pixel level, and regularization on such images seems to work great. On this image, I'm having a real tough time dialing in settings that properly model the noise in this image.

 

The images you shared, say "conv" and "deconv" as well. Do you by chance, just have the raw, strait out of integration data, without any processing or rescaling or anything else done to them? 



#7 sbharrat

sbharrat

    Vanguard

  • *****
  • topic starter
  • Posts: 2,084
  • Joined: 28 Nov 2020
  • Loc: NJ, USA

Posted 05 February 2024 - 01:41 PM

Shaun, any chance you could integrate with the proper rejection settings to eliminate those airplane/satellite trails? 

Unfortunately this is not my data. I wanted to start with some "good," low FWHM data so I purchased this integrated stack from the Insight Observatory database. This was taken with a Planewave PlaneWave CDK 12.5 f/8 scope and ZWO ASI6200MM Pro in the Utah Desert. Unfortunately, their integration does leave in the satellite trail you see.

 

Well, I've been working on this for a bit here. The noise profile is....unusual, and I'm having a hard time handling it with the regularization settings.

 

You mentioned rescaling something...which channel was that? I am wondering if I'm trying to work with the rescaled channel here. For regularization to work properly, you do need to have a fairly normal noise profile...and the profile of the image I am working with is extremely soft, blurred. I normally work on deconvolution with images where the noise profile is basically pixel level, and regularization on such images seems to work great. On this image, I'm having a real tough time dialing in settings that properly model the noise in this image.

 

The images you shared, say "conv" and "deconv" as well. Do you by chance, just have the raw, strait out of integration data, without any processing or rescaling or anything else done to them? 

Yeah, this is flaw in my experiment frown.gif . To understand how close I was getting to the original, I started with A (the purchased stacked image). I then convolved that with a PSF to create image B (the one named _conv). I then tried to deconvolve B to create my best C (the one named _conv_deconv). The goal was to then compare C to A. 

 

Of course, as you are seeing that A->B step messes up the noise profile. I put the original image in the directory (I originally left this out because I wanted to hide the base truth until the end) and you can see the expected pixel level noise in the original.  This of course is not present in the convolved image B or in the deconvolved image C. 

 

Suggestions on a better posed experiment welcomed... 

 

PS: the reason I wanted to pursue this was to not only understand how real convolution (by multiple people) does at this, but to also compare to results from BXT. 


Edited by sbharrat, 05 February 2024 - 01:53 PM.

  • Jon Rista likes this

#8 Jon Rista

Jon Rista

    ISS

  • *****
  • Posts: 26,034
  • Joined: 10 Jan 2014
  • Loc: Colorado

Posted 06 February 2024 - 01:15 AM

Thanks for the additional image. IMO, having the raw noise at a pixel level, without any kind of preliminary processing, is actually an important factor for proper deconvolution. This is where regularization comes in, and with a true noise profile (gaussian or as is more the case with astrophotography, poisson), regardless of how high the noise is, regularization can work to allow the regularized richardson-lucy deconvolution routine to operate on the details without being affected by, or affecting, the raw noise. Once you have properly deconvolved, then you can denoise, and often get significant noise reduction (especially with some of the newer tools available today, as I'm learning here).

 

The key is that, you need to do the noise reduction AFTER deconvolution. In fact, I wonder if my PSF is even accurate, given the processing that was done on the data (the convolution). I thought the data looked overly smooth and in some cases maybe slightly distorted, and I wonder if that may have had an impact on my PSF generation. It's quite important to do deconvolution, and all the various steps involved, on your image right after integration, before any other processing. A real, natural noise profile, even if there is a lot of noise, is an important factor (as strange as that may sound.) I think now, that my regularization probably needed to have a low to zero noise level and low to zero noise reduction on the first two, maybe even first three layers. PI's implementation of regularization only supports up to I think 5 or 6 noise layers, meaning that given the convolution that was applied, I probably would have only been able to do two layers of regularization. In fact, I've never even tried to "disable" the first few layers, and I don't even know if PI's algorithm can even handle that (if you get repeated pink warning notices every iteration or other iteration during regularized deconvolution, then the algorithm is working against itself with the chosen settings, and I ran into that frequently...with zero noise level and zero NR for the first two or three layers, I suspect PI would have had an even harder time.) 



#9 sbharrat

sbharrat

    Vanguard

  • *****
  • topic starter
  • Posts: 2,084
  • Joined: 28 Nov 2020
  • Loc: NJ, USA

Posted 06 February 2024 - 09:19 AM

Thanks for the additional image. IMO, having the raw noise at a pixel level, without any kind of preliminary processing, is actually an important factor for proper deconvolution. This is where regularization comes in, and with a true noise profile (gaussian or as is more the case with astrophotography, poisson), regardless of how high the noise is, regularization can work to allow the regularized richardson-lucy deconvolution routine to operate on the details without being affected by, or affecting, the raw noise. Once you have properly deconvolved, then you can denoise, and often get significant noise reduction (especially with some of the newer tools available today, as I'm learning here).

 

The key is that, you need to do the noise reduction AFTER deconvolution. In fact, I wonder if my PSF is even accurate, given the processing that was done on the data (the convolution). I thought the data looked overly smooth and in some cases maybe slightly distorted, and I wonder if that may have had an impact on my PSF generation. It's quite important to do deconvolution, and all the various steps involved, on your image right after integration, before any other processing. A real, natural noise profile, even if there is a lot of noise, is an important factor (as strange as that may sound.) I think now, that my regularization probably needed to have a low to zero noise level and low to zero noise reduction on the first two, maybe even first three layers. PI's implementation of regularization only supports up to I think 5 or 6 noise layers, meaning that given the convolution that was applied, I probably would have only been able to do two layers of regularization. In fact, I've never even tried to "disable" the first few layers, and I don't even know if PI's algorithm can even handle that (if you get repeated pink warning notices every iteration or other iteration during regularized deconvolution, then the algorithm is working against itself with the chosen settings, and I ran into that frequently...with zero noise level and zero NR for the first two or three layers, I suspect PI would have had an even harder time.) 

Thanks for the time on this... given that it is just an experiment.

 

Yes, this is a bit flawed. And I did have to use significantly different global dark level than I have typically used in deconvolution on my images. With the values I generally use, the divergence was quite regular. 

 

That said, the PI deconvolution process did "work" even on this smoothed image. Ignoring the pixel level noise, if you compare the deconvolved image to the reference, the structure within the galaxy is visually very close. The only big differences really are the bigger bright stars against the background which are only shrunk down partially. 



#10 Jon Rista

Jon Rista

    ISS

  • *****
  • Posts: 26,034
  • Joined: 10 Jan 2014
  • Loc: Colorado

Posted 06 February 2024 - 12:31 PM

Well, I am kind of surprised at the quality of this data, given you paid for it. They did not do any pixel rejection when integrating, and they don't seem to have a truly flat field. I'm also not even sure they are well focused!! I would be wary of buying more data from anyone who isn't taking every measure to make sure their system is performing at peak. I feel this image could have been more optimal in many ways...
 
 
The noise profile actually looks fine to me. This is not high noise, really, based on my experience. The exposure is pretty deep so it supports a deep screen stretch, and if you don't stretch as much, the noise gets much cleaner. It is good, clean noise, though. I would NEVER try to do anything with a noise profile like this, except simply minimize its amplitude. I don't think most people actually appreciate enough the benefits of a good, clean, poisson noise profile like this...this is excellent (so, whatever camera was used, it has very good characteristics IMO.)
 
As for the deconvolution. I have made some progress now that I have a proper noise profile. This data is SO soft and so radically oversampled (I don't know if they just didn't spend the time to focus properly, or if the seeing is so horrid that their scope is radically oversized), that its taking quite a lot of iterations, and as I push the iteration count farther and farther, its harder and harder to manage the deringing. Anyway, here is what I've got so far:
 
kcAUFlX.gif

 

There is a reasonable increase in detail after 120 iterations of deconvolution. Here are the settings used:

 

w2DgFYN.png

 

I did create a star mask for local support (couldn't eliminate all ringing any other way), and here were the settings for that:

 

UTuhFTv.png

 

I'm going to see how far I can push the iteration count before things fall apart. There might end up being some slight dark ringing around the larger stars, but to extract even more detail from data that is so soft, there might not be any other way. FWIW, saving as a GIF is costing some of the detail. When comparing the preview before and after in PI, the differences are more notable, and deconvolution appears to be providing a reasonable improvement in the details.

 

NOW, one other thing. I am testing on a preview here... You have to be careful about how small a preview you use. Too small a preview, and you end up using settings for the preview that don't work for the full image. I am not yet sure, how the full image will respond to these settings...and they may need to be tweaked. Some weird quirk about how this algorithm was implemented, I think... Something I remember from my past forays into the best way to use PI's deconvolution tool.


Edited by Jon Rista, 06 February 2024 - 12:34 PM.

  • sbharrat likes this

#11 sbharrat

sbharrat

    Vanguard

  • *****
  • topic starter
  • Posts: 2,084
  • Joined: 28 Nov 2020
  • Loc: NJ, USA

Posted 06 February 2024 - 12:36 PM

Well, I am kind of surprised at the quality of this data, given you paid for it. They did not do any pixel rejection when integrating, and they don't seem to have a truly flat field. I'm also not even sure they are well focused!! I would be wary of buying more data from anyone who isn't taking every measure to make sure their system is performing at peak. I feel this image could have been more optimal in many ways...
 
 
The noise profile actually looks fine to me. This is not high noise, really, based on my experience. The exposure is pretty deep so it supports a deep screen stretch, and if you don't stretch as much, the noise gets much cleaner. It is good, clean noise, though. I would NEVER try to do anything with a noise profile like this, except simply minimize its amplitude. I don't think most people actually appreciate enough the benefits of a good, clean, poisson noise profile like this...this is excellent (so, whatever camera was used, it has very good characteristics IMO.)
 
As for the deconvolution. I have made some progress now that I have a proper noise profile. This data is SO soft and so radically oversampled (I don't know if they just didn't spend the time to focus properly, or if the seeing is so horrid that their scope is radically oversized), that its taking quite a lot of iterations, and as I push the iteration count farther and farther, its harder and harder to manage the deringing. Anyway, here is what I've got so far:
 

 

There is a reasonable increase in detail after 120 iterations of deconvolution. Here are the settings used:

 

 

...

Yes, I was NOT happy with the data once I got it. Fortunately, it wasn't expensive so not much wasted on it. 

 

Thanks for the details on your process.

 

Regarding your result, this is a deconv of the initial (from the observatory) image right? 



#12 Jon Rista

Jon Rista

    ISS

  • *****
  • Posts: 26,034
  • Joined: 10 Jan 2014
  • Loc: Colorado

Posted 06 February 2024 - 03:51 PM

Yes, I was NOT happy with the data once I got it. Fortunately, it wasn't expensive so not much wasted on it. 

 

Thanks for the details on your process.

 

Regarding your result, this is a deconv of the initial (from the observatory) image right? 

Yes, from the original initial observatory data. 

 

I could not get PI's regularization to handle the convoluted versions. I tried to disable the first few wavelet layers in the regularization settings, but I kept ending up with artifacts or ringing. 

 

A proper noise profile is actually quite important for the regularization to work. FWIW, the noise profile was untouched by deconvolution. Here is a normal screen stretched image to demonstrate...NO image mask, only regularization:

 

ANVlfLL.png

 

A lot of people use masks with their deconvolution to prevent the noise from ending up looking like worms. You should never use a mask. You should use the regularization settings. 

 

I pushed this version farther...180 iterations. I may be able to push it even more. There is some slight ringing around brighter stars...not sure if I can adjust global dark enough to eliminate that or not yet. 


  • sbharrat likes this

#13 Jon Rista

Jon Rista

    ISS

  • *****
  • Posts: 26,034
  • Joined: 10 Jan 2014
  • Loc: Colorado

Posted 06 February 2024 - 11:51 PM

Well, I mentioned that the PI deconvolution seems to be parameterized relative to the size of the image being deconvolved and that that can lead to problems applying a deconv config to the full size image when it was configured on a smaller preview. That seems to be the case here... The process is of course taking a lot longer to run, but I'm getting fairly regular warnings, so I doubt the process will succeed.

 

I do not know why PI's deconv seems to function relative to the dimensions of the image. It doesn't seem like that should matter, but I've run into this problem many times before. The sad thing is, with as long as it takes to run deconv on a full size image, it makes it practically impossible to properly tune the settings for that image. I will usually tune "in the blind", canceling the process once it starts demonstrating a feedback loop of divergence, and keep tuning until that stops happening. Sadly, a lot of the time, I have to adjust some settings so much that there is no meaningful improvement in the end.

 

Seems like a rather severe issue with this particular process. 


  • sbharrat likes this

#14 italic

italic

    Apollo

  • *****
  • Posts: 1,386
  • Joined: 05 Apr 2022
  • Loc: San Francisco, CA

Posted 06 February 2024 - 11:57 PM

Well, I mentioned that the PI deconvolution seems to be parameterized relative to the size of the image being deconvolved and that that can lead to problems applying a deconv config to the full size image when it was configured on a smaller preview. That seems to be the case here... The process is of course taking a lot longer to run, but I'm getting fairly regular warnings, so I doubt the process will succeed.

 

I do not know why PI's deconv seems to function relative to the dimensions of the image. It doesn't seem like that should matter, but I've run into this problem many times before. The sad thing is, with as long as it takes to run deconv on a full size image, it makes it practically impossible to properly tune the settings for that image. I will usually tune "in the blind", canceling the process once it starts demonstrating a feedback loop of divergence, and keep tuning until that stops happening. Sadly, a lot of the time, I have to adjust some settings so much that there is no meaningful improvement in the end.

 

Seems like a rather severe issue with this particular process. 

I don't think it's related to the image dimensions because I have more of these failures with a preview with very low dynamic range or generally very low values, like dark clouds against a dark background and no bright dust or stars. If I run that same process on the full image or on a different patch with brighter clouds, it doesn't fail (as much). I haven't rigorously tested this, but I've noticed a trend among the images I've processed for others (in the PMD thread in BDSI).



#15 Jon Rista

Jon Rista

    ISS

  • *****
  • Posts: 26,034
  • Joined: 10 Jan 2014
  • Loc: Colorado

Posted 07 February 2024 - 02:55 AM

I don't think it's related to the image dimensions because I have more of these failures with a preview with very low dynamic range or generally very low values, like dark clouds against a dark background and no bright dust or stars. If I run that same process on the full image or on a different patch with brighter clouds, it doesn't fail (as much). I haven't rigorously tested this, but I've noticed a trend among the images I've processed for others (in the PMD thread in BDSI).

I've been fiddling, and I think it may be due to too many pixels clipped to black. I am pretty sure I knew this before, as I religiously cropped my images before deconvolution in the past...its been a couple of years since I did much with PI and I've forgotten stuff. With this current image, it had some solid black pixels in the corners due to registration, and some edge artifacts (probably also due to registration and maybe dithering). Once I cropped those poor quality regions out, I was able to run the same deconv settings without issues.

 

I you have some dark images where too many pixels might clip to black, that might be the cause. It might be worth using a larger offset on such images. 

 

Anyway, I was able to get the full image to deconvolve properly, for the most part. I can't seem to get the global dark setting high enough to prevent some ringing around the larger and brighter stars. I am usually able to push both global dark and global bright around a lot more, but there is something about this data that it is just not handling deconvolution well. Its extremely oversampled/soft, probably more than any image I've processed before, stars are not round, and I suspect there is a combination of both poor seeing and some defocus. I'll fiddle with it some more tomorrow, maybe I can push the regularization settings themselves around more and get it to work without ringing. 


  • sbharrat likes this

#16 sbharrat

sbharrat

    Vanguard

  • *****
  • topic starter
  • Posts: 2,084
  • Joined: 28 Nov 2020
  • Loc: NJ, USA

Posted 07 February 2024 - 08:32 AM

Well, I mentioned that the PI deconvolution seems to be parameterized relative to the size of the image being deconvolved and that that can lead to problems applying a deconv config to the full size image when it was configured on a smaller preview. That seems to be the case here... The process is of course taking a lot longer to run, but I'm getting fairly regular warnings, so I doubt the process will succeed.

 

...

 

Appreciate the work on this. And will be interesting to see the result of the deconv on the original image for sure. But that result unfortunately won't help with my (flawed) experiment. For that, I want to be able to compare the result with a known ground truth, hence my attempt to run deconvolution on the convolved image versus the original. Unfortunately, this approach has the tragic issue you pointed out with the noise profile getting messed up. 

 

I think what I end up with effectively is analyzing how well deconvolution works when an image was (incorrectly) noise-reduced before deconvolution. From that experiment, I think i see the results I summarized earlier:

- the detail within the galaxy is visually very close/indistinguishable to the original

- medium size and larger stars in the background are shrunk but not all the way back to original size

- pixel level noise is (obviously) not recovered 

 

My next step is to compare what BXT comes up with relative to the original. I don't know how to quantitatively assess these efforts so this initial pass will be visual comparison



#17 sbharrat

sbharrat

    Vanguard

  • *****
  • topic starter
  • Posts: 2,084
  • Joined: 28 Nov 2020
  • Loc: NJ, USA

Posted 07 February 2024 - 08:38 AM

...

 

Anyway, I was able to get the full image to deconvolve properly, for the most part. I can't seem to get the global dark setting high enough to prevent some ringing around the larger and brighter stars. I am usually able to push both global dark and global bright around a lot more, but there is something about this data that it is just not handling deconvolution well. Its extremely oversampled/soft, probably more than any image I've processed before, stars are not round, and I suspect there is a combination of both poor seeing and some defocus. I'll fiddle with it some more tomorrow, maybe I can push the regularization settings themselves around more and get it to work without ringing. 

Nice. Can you upload the finished image to https://drive.google...?usp=drive_link

 

And can you post a screenshot with your final deconvolution settings? 

 

Thanks again....



#18 Jon Rista

Jon Rista

    ISS

  • *****
  • Posts: 26,034
  • Joined: 10 Jan 2014
  • Loc: Colorado

Posted 07 February 2024 - 12:44 PM

Nice. Can you upload the finished image to https://drive.google...?usp=drive_link

 

And can you post a screenshot with your final deconvolution settings? 

 

Thanks again....

Still working on it. This data is too soft, I think, to really get a good result for the full image. I am unable to eliminate the ringing around the brighter stars, which will usually become problematic down the road with other processing. 

 

I've managed to reduce the strength of the ringing, but its still present and visible. 


  • sbharrat likes this

#19 Jon Rista

Jon Rista

    ISS

  • *****
  • Posts: 26,034
  • Joined: 10 Jan 2014
  • Loc: Colorado

Posted 07 February 2024 - 01:11 PM

This may be the best I can do. I nee to fiddle with the regularization settings to see if I can tone that down a bit, and maybe allow me to push global dark and bright more to correct that last ring. 

 

I ran this on a crop of the full image that eliminated edge artifacts. That resolved most, but not all, of the warnings about divergence. 

 

Here is the before:

 

6th7KMV.png

 

The after:

 

oVgrB6L.png

 

And the settings (same star mask):

 

v36h2NJ.png


  • sbharrat likes this

#20 sbharrat

sbharrat

    Vanguard

  • *****
  • topic starter
  • Posts: 2,084
  • Joined: 28 Nov 2020
  • Loc: NJ, USA

Posted 10 February 2024 - 06:41 PM

Redid the experiment using some better data. Posted the result, including a BXT version, over on the Beginners forum 

https://www.cloudyni...on-experiments/



#21 Higgsfield

Higgsfield

    Viking 1

  • -----
  • Vendors
  • Posts: 743
  • Joined: 10 Sep 2020

Posted 10 February 2024 - 07:01 PM

So the imaging conditions here have been maddening bangbang.gif  In the downtime, I started work on an experiment. With the various recent discussions about deconvolution, I wanted to understand a bit better how much detail can be recovered from classical deconvolution. Unfortunately, my skills at RL are pretty bad and hence not representative. But I know that many of you on this forum are experienced at this so I am soliciting your help. fingerscrossed.gif

 

In https://drive.google...?usp=drive_link you can find an image of M101 in both fits and xisf format.

 

Here are the details about the starting image:

- Obtained from the Insight Observatory starbase database: https://starbase.ins...om/imageset/218

- Taken with PlaneWave CDK 12.5 f/8 scope and ZWO ASI6200MM Pro in the Utah Desert

- This is luminance only, 60m aggregate exposure

- Image scale is 0.307"/px and worst FWHM region according to PI script is 2.8"

 

Some details about the image provided:

- My image scale is about 0.85"/px. I resampled* one of my images to make it approximately the same image scale

- Used PSFImage on that image to obtain PSF. 

- Convolved the IO image with my PSF to obtain the final image

- Worst FWHM region according to PI script is now 4.1"

 

* I have some concerns about this resample followed by extracting the PSF. But the resulting PSF looked the same in profile to obtaining it on the non-resampled image, just wider in pixels now. 

 

What I am hoping to get is a few deconvolved versions of this image. The directory above should have write privileges for you to upload. Hopefully I can entice a couple of you to try your hand at it, if only for curiosity sake. Or sense of competition. Or sense of altrusim. Whatever works for you grin.gif

 

I haven't quite figured out how to quantify "goodness" and will probably have questions about that. But my initial pass of this is just trying to see at a qualitative level what "good" deconvolution looks like compared to what I did. 

 

Appreciate your effort and thanks in advance

 

Update: if you use PI, would be very interested in the deconv settings you used

Why are you limiting yourself to so called classical decovolution? Are you not interest in revovering as much detail as possible? It's a bit like sticking with a slide  rule when calculators are available!



#22 sbharrat

sbharrat

    Vanguard

  • *****
  • topic starter
  • Posts: 2,084
  • Joined: 28 Nov 2020
  • Loc: NJ, USA

Posted 10 February 2024 - 08:53 PM

Why are you limiting yourself to so called classical decovolution? Are you not interest in revovering as much detail as possible? It's a bit like sticking with a slide  rule when calculators are available!

You mean versus using BXT or something like that? I do in fact use BXT. I was more following through on trying to verify that what is being recovered is what is in the baseline. Since many of us have been arguing ad nauseum about whether what BXT is doing is deconvolution, that is what I have been comparing it to. 



#23 Higgsfield

Higgsfield

    Viking 1

  • -----
  • Vendors
  • Posts: 743
  • Joined: 10 Sep 2020

Posted 11 February 2024 - 09:12 PM

You mean versus using BXT or something like that? I do in fact use BXT. I was more following through on trying to verify that what is being recovered is what is in the baseline. Since many of us have been arguing ad nauseum about whether what BXT is doing is deconvolution, that is what I have been comparing it to. 

Here is an example of classical deconvolution out performing BXT. I found this to be the case quite often for very small targets like PNs. Classical deconvolution is hard to do quite frankly. But if you removed the stars, and use the psf from the stars good results can be gotten. I likely did this on the stretched data, I know, heresy right! shocked.gif

 

https://www.cloudyni...or/?p=13155463 

 

OKlooking at the image again, I likely keep the stars in place and applied it to the stretched image. The reason one might do this it to better manage ringing as you have more dynamic range to work with. 


Edited by Higgsfield, 11 February 2024 - 09:15 PM.


#24 Jon Rista

Jon Rista

    ISS

  • *****
  • Posts: 26,034
  • Joined: 10 Jan 2014
  • Loc: Colorado

Posted 11 February 2024 - 10:12 PM

Here is an example of classical deconvolution out performing BXT. I found this to be the case quite often for very small targets like PNs. Classical deconvolution is hard to do quite frankly. But if you removed the stars, and use the psf from the stars good results can be gotten. I likely did this on the stretched data, I know, heresy right! shocked.gif

 

https://www.cloudyni...or/?p=13155463 

 

OKlooking at the image again, I likely keep the stars in place and applied it to the stretched image. The reason one might do this it to better manage ringing as you have more dynamic range to work with. 

I commented in the other thread...but I'll comment here as well.

 

You need to tune your regularization settings to prevent the noise from being affected by deconvolution. Once you do that, then you won't see any changes in the noise profile.

 

Fundamentally, I don't think deconvolution works well on stretched data. The PI tool is designed to be run on linear data. You say that the stretched data would have more dynamic range...but, technically, it would have less? Since stretching moves the faint signals up and compresses the bright signals...you are actually losing dynamic range, right?




CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.


Recent Topics






Cloudy Nights LLC
Cloudy Nights Sponsor: Astronomics