103 replies to this topic

[sharkmelley]

In my effort to completely migrate off Iris and onto PixInsight I have written a PixInsight PCL module to perform my favourite stretch, the ArcSinh stretch (a.k.a. Asinh, Hyperbolic Arcsine).

 

For those unfamiliar with the ArcSinh stretch, this is a method of stretching the data whilst preserving the correct colour (i.e. the original R,G,B ratios).  Colour is retained from the very faintest object to the very brightest unsaturated star.  This is unlike a traditional "curves" stretch where the brighter objects become desaturated (i.e. bleached) during the stretch.

 

The module has a standard preview window so you can interactively judge your results while playing with the sliders. I have written it within the PI PCL development environment and have built it for the 64bit Windows platform (PixInsight "Ripley").

 

Hopefully it is intuitive to use (there are tooltips on every control) but I'm happy to hear suggestions.  It works best if the image is already background extracted and white balanced.  One of the sliders will remove any residual background pedestal in the data.

 

In the example below you can see how the preview window is able to highlight pixels becoming negative if too much background subtraction is applied (these values will be truncated to zero when applied to the main image).  Obviously I hadn't performed a proper background extraction first.

 

The zip file can be downloaded here:  https://drive.google...R1NVcUhhY1NPbDA

 

Mark

 

Edited by sharkmelley, 10 March 2017 - 01:59 AM.

[termhn]

Very cool!

[Gucky]

Thanks for sharing!

[bmhjr]

A new tool! Thanks

[starbob1]

I like  arcsinh stretch also. I use Images Plus though and it does a good job of keeping star colors. When you say background extraction is that the same as flatten the background to set the level using multi points ' which makes the BG the same all over .Thanks

[sharkmelley]

Yes, background extraction is PixInsight terminology for background flattening.
Mark

[johnpane]

Mark, 

 

Would it be straightforward to build this for use on other systems besides Windows? I am on a Mac.

 

John

[sharkmelley]

Mark, 

 

Would it be straightforward to build this for use on other systems besides Windows? I am on a Mac.

 

John

In principle, yes.  The development framework provided by PixInsight means that the same code can be compiled for all PixInsight platforms.  Personally I don't have the ability to do this for the Mac or Linux but I'm happy to share the code with someone who can.  In the medium term I hope this module can be added to the code repository and become part of the standard distribution but that would depend on the view of the PixInsight dev team.

 

Let's get some feedback first - on whether or not folk think it is a useful module.

 

Mark

Edited by sharkmelley, 10 March 2017 - 01:34 PM.

[johnpane]

I have PCL installed on my computer and the Mac development tools. I would be happy to try building it.

[balu01]

I have PCL installed on my computer and the Mac development tools. I would be happy to try building it.

Please do! Mac here too

[rockstarbill]

I can build it on Linux if needed.

[sharkmelley]

I have PCL installed on my computer and the Mac development tools. I would be happy to try building it.

 

I can build it on Linux if needed.

 

My development process was to borrow heavily from existing modules - that seems to be the PixInsight recommended approach.  So I started by using Sandbox as a base and later added the preview functionality I found in UnsharpMask.

 

Attached is the zipped source code. I'm interested to know how you get on.

 

I did try using the Makefile Generator script from within PixInsight to generate the makefiles and project files but all I got was an error message.  So for Windows I took a copy of the relevant Sandbox project file and performed a global search/replace within it. 

 

If the Makefile Generator works on your system then it ought to make things much easier.

 

Mark

 

 Arcsinh00.00.01.0100.zip   21.87KB   105 downloads

Edited by sharkmelley, 11 March 2017 - 05:04 AM.

[sharkmelley]

I managed to make the Makefile Generator script run by unticking the "All Architectures" option.  So here's a new zip including the makefiles for each platform.

 

Unzip it into your PCL\src\modules\processes folder

 

You'll need to set up your environmental variables as described here: 

https://github.com/PixInsight/PCL

 

Mark

 

 Arcsinh_with_makefiles.zip   31.58KB   96 downloads

[IshanAstronomer]

Mark, Kudos !!! A really good stretching process. I tried it with one of my old images. The colours on the star were maintained.  At the very least its better than MaskedStretch. Clearly much better colour retention than HT or a manual CurvesTransformation.  I don't think I'm good enough to comment on technicalities.

And, takes hardly any time (about 5s on my sluggish laptop) to execute.

One more thing, somewhere on Pixinsight Forum Juan (PI Developer) stated a long time ago that Curves Transformation (or curves in PS) is powerful enough to do all sorts of stretches possible.

He's right. But that doesn't undermine the utility of a simple, intuitive and "previewable"  (for lack of better term) process developed by you Mark. Thanks for this. Again, well done !!  

[sharkmelley]

Mark, Kudos !!! A really good stretching process. I tried it with one of my old images. The colours on the star were maintained.  At the very least its better than MaskedStretch. Clearly much better colour retention than HT or a manual CurvesTransformation.  I don't think I'm good enough to comment on technicalities.

And, takes hardly any time (about 5s on my sluggish laptop) to execute.
 

Thanks for your comments.  At present it's using just a single processing core when it applies the stretch to the image.  It should be possible to make a multithreaded version to take advantage of multi-core processors - it would then run much faster.  I just need to find an example PI module

 

Mark

Edited by sharkmelley, 11 March 2017 - 05:52 AM.

[johnpane]

The attached zip file contains the Mac build for this process module.

 

Mark, in a quick glance at your code (ArcsinhInstance.cpp), two questions arose:

 

1. Is this calculation incorrect? It produces 2^24, not 2^32.

 

127      case 32: MaxPossibleVal = 256.0*256.0*256.0; break;

 

2. Here it looks like you are calculating luminance by giving equal weight to each color channel. By default PixInsight does not weight the channels equally when extracting luminance, as discussed in documentation of RGB Working Spaces (such as https://pixinsight.c.../why_rgbws.html). Would you get better results by using the coefficients defined in the RGB Working Space?

 

170       for (int ich = 0; ich < numChannels; ich++)
171       {
172        double val = image.Pixel(point,ich); 
173        sum += val;
174       }

 

 

 

Attached Files

•  Arcsinh-pxm.dylib.zip   206.58KB   55 downloads

Edited by johnpane, 11 March 2017 - 08:38 AM.

[russ.carpenter]

I would like to add my thanks to johnpane for working on a Mac version.

[sharkmelley]

The attached zip file contains the Mac build for this process module.

 

Mark, in a quick glance at your code (ArcsinhInstance.cpp), two questions arose:

 

1. Is this calculation incorrect? It produces 2^24, not 2^32.

 

127      case 32: MaxPossibleVal = 256.0*256.0*256.0; break;

 

2. Here it looks like you are calculating luminance by giving equal weight to each color channel. By default PixInsight does not weight the channels equally when extracting luminance, as discussed in documentation of RGB Working Spaces (such as https://pixinsight.c.../why_rgbws.html). Would you get better results by using the coefficients defined in the RGB Working Space?

 

170       for (int ich = 0; ich < numChannels; ich++)
171       {
172        double val = image.Pixel(point,ich); 
173        sum += val;
174       }

Thanks for building the Mac version!

 

You are of course correct about the case of the 32 bit integer.  I'm wrong by a factor of 256.  I'll fix this.  You can see I didn't test the code on a 32bit integer image

 

The weighting of the R,G and B channels when calculating luminance is an interesting one.  I simply followed the simplistic version in the original Lupton et al paper: https://arxiv.org/ab...ro-ph/0312483v1   However they do mention that there are other possibilities.  How much difference it would actually make in practice I don't know.  I'll give it a bit more thought.

 

Thanks again,

 

Mark

Edited by sharkmelley, 11 March 2017 - 01:57 PM.

[calypsob]

Thanks for sharing mark this will save me from having to go into photoshop to check for faint ifn using equalize command 

[sharkmelley]

I have fixed the bug with 64bit integer files noticed by JohnPane.

 

The Windows version of the fixed executable (version 00.00.01.0101) can be picked up here: 

https://drive.google...Wi1tS0lHWkJIX00

 

The updated source code and makefile tree is in the attached zip file.

 

Mark

 

 ArcsinhMake00.00.01.0101.zip   31.58KB   83 downloads

[entilza]

Mark, thanks for your work on this.  

 

I am having trouble figuring out how to use this.  Could you provide a sample image and maybe some steps to demo it or explain how I would use this?

[sharkmelley]

Mark, thanks for your work on this.  

 

I am having trouble figuring out how to use this.  Could you provide a sample image and maybe some steps to demo it or explain how I would use this?

I've put a demo image here:  https://drive.google...dzVGeVEzQ0lhRjA

It's a DSLR image of the North American Nebula in 32bit floating format linear data which has been white balanced it and  background flattened (flattened but not zero).

 

Open the image in PixInsight and examine it - you'll notice the background level is approx. 0.1 (on a scale of 0.0-1.0) so this will need to be subtracted.  Arcsinh has a slider to do this.   

 

The Arcsinh preview functionality is essential but don't apply a Screen Transfer Function because that will upset the preview.  The upper slider is the stretch factor.  Good data can be stretched by 100x or more but only after the background level has been subtracted.

 

The image below shows a stretch of 100 but the background is not sufficiently subtracted so the image lacks contrast:

 

 

 

On the other hand, if too high a value is subtracted then negative values will be produced - this is highlighted in the preview window if "Highlight negative values" is enabled:

 

 

 

The image below shows a good subtraction and the stretch factor has even been increased to 251:

 

 

The noise level in your image will be the limiting factor to the amount of stretch you can apply.  Broadly speaking, the noise will also be increased by the stretch factor.

 

Arcsinh is not always the optimum way of stretching data but at the very least it does give a very quick view of what can be achieved in your image.  In many cases I find that Arcsinh provides exactly what I want.  In other cases further work is required in the brighter areas of the image, where the intensities are compressed together, losing contrast.  Even if you only use Arcsinh as a diagnostic tool, I think it is a worthwhile tool to have.

 

I hope this makes sense and I'm happy to answer further questions.

 

Mark

Edited by sharkmelley, 12 March 2017 - 04:12 PM.

[russ.carpenter]

Thanks to sharkmelly and johnpane for this promising module. I'm a Mac user and therefore downloaded johnpane's version.

I've only tried it twice, but the behavior is somewhat different from what I expected. No doubt, I'm doing something silly.

 

Here is a faint galaxy, which I'll use as a test object. I used a plain vanilla series of processing steps: ChannelCombination, DynamicCrop, AutomaticBackground Extraction, BackgroundNeutralization, ColorCalibration and SCNR. The image below is the galaxy with an Auto STF.

 

 

I needed to move the Stretch Factor all the way to 500 to get a useable image.

 

 

Even a little nudge to the slider for Background Subtract filled the preview with the highlight for negative values.

 

 

Any words of wisdom?

[johnpane]

Here is a newer version for the Mac, with the bug fix Mark announced on March 12. This change will only affect users whose image is in 32-bit integer format.

 

 

Attached Files

•  Arcsinh-pxm.dylib.zip   206.58KB   36 downloads

[sharkmelley]

 

I needed to move the Stretch Factor all the way to 500 to get a useable image.

 

Even a little nudge to the slider for Background Subtract filled the preview with the highlight for negative values.

 

Any words of wisdom?

When you are stretching by a factor of 100 or more, the very tiniest adjustment of the background value makes a very noticeable difference because it is magnified by a factor of 100 or more!  So you need to get the background value accurate to all 4 decimal places.  The up and down arrows on the keyboard are supposed to nudge the value but it's not working for me - I think it's probably something I have overlooked in the code.  I'll take a look into this - it's the first module I have written

 

However, I do find the mouse wheel works fine for nudging the value.  Failing that, just type the background number in the box - it's easy to overtype the final digit as required.

 

Once you've done it once or twice you'll get the hang of it.

 

Another possibility, though a bit more complicated would be for me to include a fine adjustment slider immediately below the original slider.

 

Mark

Edited by sharkmelley, 14 March 2017 - 05:23 PM.