Return to the Cloudy Nights Telescope Reviews home pageAstronomics discounts for Cloudy Nights members
· Get a Cloudy Nights T-Shirt · Submit a Review / Article

Click here if you are having trouble logging into the forums

Privacy Policy | Please read our Terms of Service | Signup and Troubleshooting FAQ | Problems? PM a Red or a Green Gu… uh, User

Astrophotography and Sketching >> Beginning and Intermediate Imaging

Pages: 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | (show all)
Madratter
Postmaster


Reged: 01/14/13

Processing Messier 33 in PixInsight - A Tutorial
      #6069579 - 09/08/13 01:59 PM

First, let me state flat out, I am NOT an expert in either PixInsight or Image Acquisition. This data we will be using is not perfect; it has significant warts. But in a way, that is a good thing since it will require techniques and such that better data might not. Likewise, the choices I make when processing this may not be optimum. But I'm pretty sure we will end up with a usable result. If you have an alternative you think is better, by all means speak up. I'm open to learning something new.

This is not an ideal format for this for at least two reasons. First, there is a limit of one picture per post. And secondly, the max size for a picture is 800x800. So I'll have to be reducing the size of some of the results/and or cropping them for display here.

The data itself was taken on the night of September 4th, 2013. A Celestron Powerstar III 8" SCT with a f/6.3 reducer flattener was used so the focal length is close to 1260mm. All subs were 5 minutes in length. Luminosity was unbinned (1x1). R, G, and B were all binned 2x2. Flats were taken for each of those 4 channels individually. Total exposure time for luminosity was 1 hour 55 minutes. Each color channel had 45 minutes of time. I can get by with less than that and sometimes go with just 30 minutes color. However, I like using 45 minutes when I can get it because Winsorized sigma clipping works better.

This tutorial is going to start with the stacked, calibrated images (darks, flats, flat darks). I happen to really like doing the calibration and stacking within PixInsight but I know others still use other tools like Deep Sky Stacker.

The channel stacks are available on DropBox here:

Messier 33 Channel Stacks on Dropbox

I'm a strong believer that you learn better when you actually follow along and do the procedure. Still, it is your choice.

It is also possible I will have false starts and will back up in the processing. That is fine if it occurs. That will show how I think about the processing.

To a certain extent my workflow depends on what happens with the image, but here is a skeleton outline of some of the likely steps:

Luminosity:

Crop out stacking artifacts
Dynamic Background Extraction (or Automatic Background Extraction)
Create Point Spread Function for Deconvolution
Create Star Mask for Deconvolution
Create Overall Mask for Deconvolution
Deconvolution
Histogram Transformation
HDRMultiscaleTransform (Wavelets)
TGV Denoise
Histogram Transformation

RGB

Channel Combine the separate color stacks
Crop
Dynamic Background Extraction or Automatic Background Extraction to get rid of gradients
Histogram Transformation
TGV Denoise
Histogram Transformation
Saturation done using the Curves Tool

L and RGB
Register (Align) the separate L and RGB images
LRGB Combine
Crop
Curves (on the luminosity portion) to adjust contrast


Edited by Madratter (09/08/13 04:19 PM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6069603 - 09/08/13 02:12 PM Attachment (37 downloads)

Luminosity: Cropping Out Stacking Artifacts.

If you bring up the luminosity stack and do a [ctrl]a (which does an auto-stretch, you will notice that there are black borders on some of the sides. They are caused by when we stack, we are aligning the stars. Unless you got things absolutely perfect when guiding, you have no flexure, and you did not dither, the FOV will be slightly different between the different subs. That mismatch of FOVs will show up as darker borders. In this case, it is most obvious on the left side of the image. This particular image only has minor amounts of it most obvious on the left edge.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Jeff2011
Carpal Tunnel
*****

Reged: 01/01/13

Loc: Sugar Land, TX
Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6069611 - 09/08/13 02:22 PM

That is a great tip. I have been doing the same thing in StarTools using the AutoDev module. It is important to remove the stacking artifacts since it can degrade the performance of gradient removal operations.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6069613 - 09/08/13 02:23 PM Attachment (28 downloads)

The dynamic crop tool is used to get rid of those edges. Here is what the image looks like while applying the tool.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6069616 - 09/08/13 02:23 PM Attachment (25 downloads)

And here is what the Dynamic Crop settings were:

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6069623 - 09/08/13 02:26 PM

Jeff is exactly right about why the cropping is important. If you don't crop those edge it will mess up gradient removal, and it will prevent you from stretching the image as much as is desirable.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6069638 - 09/08/13 02:36 PM Attachment (19 downloads)

To actually apply the Dynamic Crop, you click the green check mark at the bottom of the Dynamic Crop controls.

This might be a good time to mention another tool that is useful. While [ctrl]a is great for quick looks at the data, sometimes you want more control. The ScreenTransferFunction is used for this.

Edited by Madratter (09/08/13 02:37 PM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6069661 - 09/08/13 02:45 PM Attachment (22 downloads)

When using the ScreenTransferFunction, (and [ctrl]a, you are not actually changing the underlying data. What you are doing is changing how that data is presented on the screen. When using this, I almost always have the blue check mark on the bottom right clicked. This tells the screen transfer function to take on the settings of whatever image is selected. If you click the yellow and black icon that looks something like a radition symbol, this will do on auto-strech of the data similar to hitting [ctrl]a. Moving sliders around will affect the results. The only other thing I want to mention here is that the symbol that looks like to links of chain is currently selected. This ties all channels together, when doing the stretching. This is important for RGB data. Sometimes you want to do the best stretch for each channel independently. To do that you would deselect that link icon.

Here are the autostretched results of the Dynamic Crop.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6069679 - 09/08/13 02:55 PM Attachment (23 downloads)

One of the clearly bad things about this data is there is an obvious gradient across it. Even worse, that gradient is a little tougher than average to remove because Messier 33 fills so much of the frame. I often will use automatic background extraction to get rid of gradients but it is hopeless here.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6069684 - 09/08/13 02:57 PM Attachment (23 downloads)

Here are the settings in ABE for that miserable failure:

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6069691 - 09/08/13 03:00 PM

As can be seen, the results are actually worse than what we started with. All kinds of dark artifacts have been introduced around M33, and the upper right and lower left corners are still bad. Various other settings can be tried. For example you might want to try reducing the Deviation setting on Global Rejection (often a handy thing to do), but nothing will work well here.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6069722 - 09/08/13 03:11 PM Attachment (21 downloads)

When automatic fails, it is time to bring out the big guns. In this case, that is DynamicBackgroundExtraction (DBE).

Unfortunately, the automatic settings for DBE will also fail miserably. Bring up DBE. Then click on the image. Open up sample generation, and the click on the generate button.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6069725 - 09/08/13 03:13 PM Attachment (27 downloads)

Your image will look something like this:

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6069730 - 09/08/13 03:14 PM Attachment (17 downloads)

If you actually tried to apply this the results would be even more disastrous than ABE.

The trick is to actually place those little samples where we actually need them.

Edited by Madratter (09/08/13 03:26 PM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6069769 - 09/08/13 03:33 PM Attachment (16 downloads)

Then this actually needs to be applied to the image. Here is the settings box.

Note that I change the tolerance setting under model parameters so that all samples would be good (not red). I also had to change the correction setting under target image correction to subtraction. Otherwise, it won't actually do anything, just show you the background it would subtract.

Edited by Madratter (09/08/13 03:35 PM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6069776 - 09/08/13 03:39 PM Attachment (28 downloads)

Here is the result. Those unsightly gradients are gone without the terrible artifacts introduced by ABE. By the way, to apply this, you drag the little triangle on the bottom left of the settings box over the image.

Edited by Madratter (09/08/13 04:05 PM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6069779 - 09/08/13 03:42 PM

Next up will be deconvolution. This should be done while the data is still linear (unstretched). This is a good place to pause for a while.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Raginar
Postmaster
*****

Reged: 10/19/10

Loc: Rapid CIty, SD
Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6069802 - 09/08/13 03:52 PM

I'd make this on a website... You'll find it flows easier than the constant threads

Thanks for sharing your techniques. One request, could you post at the beginning your 'work flow'? I often don't need the individual frames... but seeing how you tackle going from mono to RGB and linear to non-linear is nice.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Raginar]
      #6069845 - 09/08/13 04:17 PM

Hi Chris:

I've done many a website before, and it is possible this could end up as one or on one if enough people find it useful.

I have added an overall workflow that may or may not need to be tweaked as we go along.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Raginar
Postmaster
*****

Reged: 10/19/10

Loc: Rapid CIty, SD
Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6069883 - 09/08/13 04:39 PM

Thanks MR

Post Extras: Print Post   Remind Me!   Notify Moderator  
joelimite
Carpal Tunnel
*****

Reged: 09/01/08

Loc: Fayetteville, AR
Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Raginar]
      #6069886 - 09/08/13 04:42 PM

Excellent tutorial, Madratter! I'm extremely new to deep-sky astrophotography and have only used DeepSkyStacker and Photoshop. I'm intrigued by your tutorial, though, and now want to try PixInsight.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Jeff2011
Carpal Tunnel
*****

Reged: 01/01/13

Loc: Sugar Land, TX
Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Raginar]
      #6069997 - 09/08/13 05:48 PM

So far it all makes sense to me. I can can correlate the PI operations to what I do in StarTools. However, I am beginning to see some differences. Good stuff. Looking forward to the combining process.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Jeff2011]
      #6070197 - 09/08/13 08:27 PM Attachment (17 downloads)

Hi Joel. Thanks for the kind words. Even if you don't end up using PixInsight the concepts will still apply with other processing techniques. That goes hand in hand with what Jeff is saying about StarTools (which I also own).

Deconvolution is a way of tightening up the detail in your image. It can make stars smaller, and it can bring out detail in your images.

Perhaps the way to start is to show what happens if you actually use the defaults on an image. It is usually pretty dreadful, and will illustrate why we go to the trouble of all the steps that are about to occur.

If you have followed along, you might want to save your current results. I called mine L_dbe. I save as a 32 bit fit file. I actually save after every operation. There is actually a history explorer, but that won't help if the electric goes out.

Bring up that image and use either [ctrl]a or STF (ScreenTransferFunction) to stretch it so you can see what is happening. Next we want to define a preview. This can be done by pressing [alt]n. Move the cursor to where you want to start. Then press and hold the left mouse button and drag to where you want the preview to end.

Previews are very useful in PixInsight and we will be using them from time to time for a number of purposes.

If successful, you will end up with something like this:

Edited by Madratter (09/08/13 08:42 PM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6070205 - 09/08/13 08:30 PM Attachment (10 downloads)

If you left click on the tab down the left side of the image that says Preview01, you should now see this:

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6070210 - 09/08/13 08:35 PM Attachment (12 downloads)

Notice I made my preview large enough that it does not all fit, when showing it at scale 1:1. (You can change the scale by using your middle mouse scroll wheel). However, we really want this at 1:1. We can either live with it this way, or using common windows methodology, we can grab the end of the frames and enlarge the window.

Now one of the things that makes previews useful, is that because they are not as big as the full image, computationally expensive processing can be tried out before committing the settings to the entire image.

Bring up the deconvolution tool. I have opened mine up to also show the deringing parameters. We aren't currently using them but we will.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6070217 - 09/08/13 08:38 PM Attachment (13 downloads)

Now with the preview tab showing, drag the triangle in the lower left of the deconvolution parameters box over the preview image. This will apply those settings to the preview.

The result is this:


Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6070221 - 09/08/13 08:40 PM Attachment (21 downloads)

I think it is pretty obvious the result of using the defaults is horrid. There are now black rings around the stars. The bright stars have a bright ring around a brighter middle, and the noise in the image has been intensified.

You can get rid of these bad results by using [crtl][shift]z. Actually that will toggle the results so you can look at the before and after. Get rid of the bad results.

Do NOT try and get rid of the by hitting the back button in PixInsight. That will NOT accomplish what you want.

We can avoid some of the ringing artifacts by playing with the deringing parameters. Try these settings:



Edited by Madratter (09/08/13 08:51 PM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6070237 - 09/08/13 08:53 PM Attachment (14 downloads)

If you apply those parameters, you will get something like this:

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6070245 - 09/08/13 08:57 PM

Again, this is no good but at least we have gotten rid of the dark rings around the stars. But now there are all the unsightly bright lumps around stars and other places in the image. You can try to control them with the Global Bright Deringing parameter (try .01) but the results will still be worse than the original.

OK, that should give you some insight into why we are going to go through the steps that follow. To do Deconvolution properly, we are going to need three things first:

1) A Point Spread Function for stars in the image.
2) A star mask to help protect the brightest star from ringing artifacts.
3) A mask to protect the background areas in the image from being touched and made more noisy.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6070252 - 09/08/13 09:02 PM Attachment (11 downloads)

First we need to get a Point Spread Function (PSF) for the stars in the image. Bring up the DynamicPSF tool.

Use of this tool is anything but obvious.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6070265 - 09/08/13 09:10 PM Attachment (9 downloads)

Go back to your image (click the tab on the left on the window). Mine is named L_dbe. Depending on what you have been doing, yours may be named differently.

Now click on stars within the image. They need to be distributed across the image. Take care to avoid ones that are real bright or real faint. It is very important to avoid stars that are saturated (overexposed). Also avoid stars that are real close together. As you click, the results will be added to the DynamicPSF window.

You'll end up with something that looks like this:


Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6070268 - 09/08/13 09:13 PM Attachment (10 downloads)

Here is a view of the DynamicPSF window with parameters filled in:

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6070272 - 09/08/13 09:16 PM Attachment (8 downloads)

Scroll through the entries in the DynamicPSF window and look for any that say something other than Moffat.

Any such entries are not good for modeling the PSF we need for deconvolution.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6070275 - 09/08/13 09:18 PM Attachment (8 downloads)

Those entries can and should be eliminated by selecting them and then clicking the red "-" icon in the DynamicPSF window.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6070283 - 09/08/13 09:24 PM Attachment (7 downloads)

It is now time to export the PSF as an image to be used when doing the deconvolution. Select all the entries in the window [ctrl]a (yes that is confusing since it is the same key sequence used to auto-stretch the data). It is very important to select them all or the PSF you export will be for only the star or stars selected. This is a easy mistake to make.

Once all the entries are selected, click the icon that looks a bit like a camera.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6070286 - 09/08/13 09:26 PM Attachment (6 downloads)

If done correctly, you should now have an image that is your average PSF of those stars. I save that image as a fit file (I called it PSF). After you have the PSF image, get rid of the DynamicPSF window (click the X). Keeping it around will just confuse things later.

Edited by Madratter (09/08/13 09:35 PM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6070295 - 09/08/13 09:32 PM Attachment (8 downloads)

Next up is creating a star mask. Fortunately there is a handy tool for this purpose called, wait for it, StarMask (those clever developers). If they were really Unix gurus, they would have called it something like Spot because that was the name of their dog (or biff). OK, enough of the deep inside jokes.

Bring up the StarMask Tool.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6070312 - 09/08/13 09:44 PM Attachment (9 downloads)

The defaults are reasonable, but I like changing the Noise Threshold parameter. On this image setting it to .05 will pick up a few more stars.

Change that parameter if you wish and then drag the triangle at the bottom of the StarMask window over the image of M33. A new window will come up containing the star mask that was created. Don't worry that it does not show every star in the image. We are only using it to protect the brightest stars and it has found those.

(You can do a screen transfer function on it or use [ctrl]a. You will see that it really did pick up only the bright stars).

I save this image as a 32 bit fit (it will want to default to 16 bits). I called mine star_mask.

Edited by Madratter (09/08/13 09:44 PM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6070325 - 09/08/13 09:52 PM Attachment (7 downloads)

The last step of preparation for deconvolution is creating a protective mask. For this I'm going to star by using the Screen Tranfer Function. Click the radiation symbol to get the default stretch. Make sure you have selected the M33 image or else you will be stretching something like your star mask or your PSF image.

Now bring up the HistogramTransformation process. HistogramTransformation is used to stretch the data in a permanent way. Once you do this your data is no longer linear (and no good for doing deconvolution).

Since we want to create a mask, we need a copy of the M33 image. We can get a copy by clicking the clone button up on the PixInsight Toolbar. (It is about 4 icons over from the left and looks like two windows with a green plus sign over them). Alternatively, we can right click in the M33 image window and select the duplicate entry.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6070331 - 09/08/13 09:59 PM Attachment (16 downloads)

Ok, now is we play Felix and reach into our bag of tricks. If you followed the instructions and used the screen transfer function to stretch the image, we can apply the settings of the screen tranfer function to the histogram transformation process. You do this by dragging the triangle on bottom left of the STF window up to the bottom of the histogram transformation window.

Unfortunately, this is one of those things that is better demonstrated in video.

Once you have it positioned as shown, then release the left mouse button, and the settings from STF will be applied to HistogramTransformation.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6070337 - 09/08/13 10:04 PM Attachment (13 downloads)

Now make sure your cloned image of M33 is selected. Then click the blue check mark on the Histogram Transformation window (if you mouse over it, it will say Track View). You should now have something that looks like this:

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6070342 - 09/08/13 10:09 PM Attachment (10 downloads)

Now reset the STF screen to the defaults (no stretch). You do that with the Reset button on the bottom right of the ScreenTransferFunction window (it is next to the blue checkmark and looks like a cross). Make sure you do this on the STF window and not the HistogramTransformation window since it has those icons too.

Then get a preview of your histogram transformation by clicking the blue circle icon on the bottom left of the HistogramTranformation Window.

You should now get a real-time preview image:


Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6070349 - 09/08/13 10:14 PM Attachment (11 downloads)

What we want to accomplish is we want to only do deconvolution on the bright parts of the image and not on any of the darker background.

This is one place where clipping the darks is a GOOD thing.

Go ahead and apply the histogram transformation to the cloned image of M33 that we will be using as a mask. Delete the real-time preview and then drag the bottom left triangle of the HistogramTransformation over the the cloned image.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6070364 - 09/08/13 10:21 PM Attachment (12 downloads)

We have not clipped the darks nearly as much as I want.

Again using the real-time preview, drag the left slider on the HistogramTransformation over until things look more like this.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6070365 - 09/08/13 10:23 PM Attachment (12 downloads)

Here is a view of the HistogramTranformation window.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6070370 - 09/08/13 10:26 PM Attachment (14 downloads)

Now dismiss the real-time preview window and apply these settings to your clone of the M33 image. This will now serve as your mask. I save this as deconv_mask.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6070371 - 09/08/13 10:28 PM

All the bits are now ready for doing the deconvolution properly. I realize there are a lot of steps in doing this, but with practice, it becomes easier and more natural.

This is a good place to leave off for the night.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Jeff2011
Carpal Tunnel
*****

Reged: 01/01/13

Loc: Sugar Land, TX
Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6070915 - 09/09/13 09:48 AM

That is a bit more to the decon than what I am used to. With StarTools, I only set a mask for the stars and not the background. I have looked at the system requirements for PI and I will need replace my old clunker PC. It is time anyways. I have just been putting it off. In the mean time I have been processing your data with StarTools. The challenge is doing the scaling and the registration of the processed L and RGB images. DSS does not support scaling and I don't have PS, so I found out I could do it in good old IRIS. Look forward to seeing your complete process.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Jeff2011]
      #6072929 - 09/10/13 10:36 AM

Yeah, deconvolution in StarTools is MUCH simpler. For those who have both, the temptation to transfer over into StarTools, and just do the deconvolution there definitely exists.

However, the results are very nice and controllable in PixInsight, if a bit (OK a lot) more work.

Edited by Madratter (09/10/13 10:37 AM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
GRB
member


Reged: 08/24/10

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6073012 - 09/10/13 11:14 AM

Wow I need to read this through a few times and let things soak in. Since I am new to this, but this really is a great thread for learning thank you Madratter.

I need to go the free or inexpensive software route for now, equipment startup bills need to be paid.

What would you recommend? I am using DSS and CS2 right now.

Haven't done any gradient stuff yet. The Tutorial I saw used a purchased plugin for CS5.

Lots to learn but is should be fun.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Jeff2011
Carpal Tunnel
*****

Reged: 01/01/13

Loc: Sugar Land, TX
Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6073051 - 09/10/13 11:33 AM

It does look like you have more control over the decon in PixInsight. At this stage in the processing, I don't think you can flip back and forth between StarTools since ST keeps track of the noise until the final noise reduction is done. If you flip back and forth, that noise tracking is lost, I think.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Jeff2011]
      #6073088 - 09/10/13 11:55 AM

GRB, there are a bunch of free tools. FitsLiberator gets mentioned fairly often, although I haven't used it.

FitsLibererator web page

StarTools is inexpensive and is a great value. It is only 60$ Australian.

It can be tried for free. You just cannot save the files.

StarTools web page

Although I primarily use PixInsight, I bought StarTools after I already had PixInsight, so that tells you I was pretty impressed with it. I liked some of the tools enough I thought it was a worthwhile addition to my tool chest.

DSS you already have and I have played with that. It is a good tool.

Jeff: It is no trouble going back and forth between PixInsight And StarTools at this stage. Once things are no longer linear, that is a different story.

You would simply save the PixInsight file as Tiff 16 bit with no compression. You then bring it into StarTools, do the deconvolution, and save as Tiff. You do lose some precision in the process because you have down converted from 32 bit floating point to 16 bit integer. But still it works well.

What IS true is that once you have stretched the data permanently (not just a Screen Transfer Function or [ctrl]a), StarTools will not be able to optimize its noise reduction. And processes like deconvolution that depend on linear data to work correctly will not work as well. You could still try importing the data and doing a reverse stretch which StarTools will try.

Edited by Madratter (09/10/13 12:01 PM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
Jeff2011
Carpal Tunnel
*****

Reged: 01/01/13

Loc: Sugar Land, TX
Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6073106 - 09/10/13 12:06 PM

Hmmm. Perhaps my understanding is wrong. I thought that ST kept track of the noise after each operation while still in a linear state and that noise tracking was used when tracking was ended and the denoise was applied taking the image out of a linear state. Perhaps Ivo can clarify this. To me any operation done in PI can't be tracked by ST. Plus if you do some operations in ST and then leave ST, I think the noise tracking is lost for any operations done so far. I thought I saw Ivo mention this before either on his support site or a thread on CN. Again I could be completely wrong here. Also I am using his latest production version and not the Beta.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Jeff2011]
      #6073113 - 09/10/13 12:12 PM

What you are saying is technically correct. That said, if you were to transfer over to StarTools and then continue on in StarTools after having done the cropping, and DBE in PixInsight, StarTools would still be able to track additional changes and would still optimize noise reduction better than if it got the image in a non-linear state.

BTW, the key word here is optimize. Even if you bring over a non-linear image, the noise reduction in StarTools will still work. In fact, I have done it.

Edited by Madratter (09/10/13 12:13 PM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
Jeff2011
Carpal Tunnel
*****

Reged: 01/01/13

Loc: Sugar Land, TX
Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6073144 - 09/10/13 12:27 PM

Ok, that makes sense. To take full advantage of the noise tracking in ST you would do all of the PI operations first and then finish in ST. By finish I mean take it to a non-linear state. I realize there can be more processing after that in either PI or ST.

In ST I usually do the initial noise reduction that ends tracking on the whole image with nothing masked off. Then if the background is still noisy, I do another noise reduction (now in a non-linear state) with the DSO masked off, so I can do a more aggressive noise reduction on the background.

This has been a great discussion. Thanks.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Jeff2011]
      #6074008 - 09/10/13 09:57 PM Attachment (9 downloads)

If you have saved your 4 files needed for deconvolution and are coming back to this like I am, it is time to load them again. You need the following:

1) The image of M33 we have been working on
2) The PSF image.
3) The star mask
4) The mask for protecting the background (deconv_mask)

Now you want to select the little tab that has the name of your deconv_mask (whatever you called it) and holding down the left mouse button, you want to drag it over to just below the tab with the name of your main image (in my case L_dbe). Then release the mouse button. This should apply the mask to your main image.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6074020 - 09/10/13 10:03 PM Attachment (8 downloads)

Alternatively, right click on your main image. This should bring up a menu. One of the items is Mask and it has a submenu. From there you can "Select Mask". Then you will get a little window that will let you choose from your various images of the right size for your mask. If you happen to love shortcuts, you can bypass all of that and get right to the mask selection window with [ctrl]m. Personally, I use the first method of dragging the little tab.

If you have done this correctly, your main image will now have a red overlay that gives you an idea of what is masked and what isn't.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6074026 - 09/10/13 10:09 PM Attachment (8 downloads)

That is great to know one is applied, but fortunately, we can turn off its display. To do this you can either use the mask menu or you can just use [ctrl]k. I use the later.

You can tell a mask is still applied because the tab of the image name will still be colored brown.

Now bring up the deconvolution process. Click on the tab saying external PSF. Then click the little box next to view identifier and select your PSF image.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6074030 - 09/10/13 10:12 PM Attachment (8 downloads)

Finally, expand the deringing settings and click on the checkbox to enable deringing. Set Global Dark to 0.0. Check the local deringing box. Click the box next to Local Support and select your star mask.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6074038 - 09/10/13 10:15 PM Attachment (9 downloads)

Now under the Algorithm section, change the iterations to something more useful like 50. This is one parameter you might want to play with.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6074043 - 09/10/13 10:21 PM Attachment (6 downloads)

Next we need to define a preview. We do this exactly like we did before ([alt]n left click and drag mouse to define opposite corners). Here is my preview area. You definitely want to preview this because these calculations take a long time on the full image and we may need to adjust deringing to get something reasonable.

Remember to do a screen transfer function on the main image so we can see what we are doing.

Here is my preview defined. That preview area is a good one because:

1) It has some really bright stars (which are easy to mess up)
2) Some regular stars.
3) Detail in the galaxy to be enhanced.
4) Background we don't want messed up.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6074049 - 09/10/13 10:25 PM Attachment (6 downloads)

Select the Preview tab and we are ready to actually try this out. You might want to expand the window some if your preview area is as large as mine so you can see all 4 of those features.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6074059 - 09/10/13 10:29 PM Attachment (9 downloads)

Now drag the little triangle at the bottom left of the deconvolution window over your preview and let the calculations run. My result is this:

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6074064 - 09/10/13 10:33 PM

This is already a much better result than when we tried deconvolution with the default settings. Star sizes have shunk nicely, detail in the galaxy is enhanced (especially in that HA region), the background has been protected, and the big stars don't have big holes or anything.

You can toggle back and forth looking at the original and new versions using [ctrl][shift]z.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6074075 - 09/10/13 10:38 PM Attachment (7 downloads)

That said, I'm not entirely happy with it. I tried setting the global dark reringing parameter to .01 and I like the result much better.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6074084 - 09/10/13 10:47 PM Attachment (8 downloads)

I'm now happy with what I have and I'm going to apply these settings to the image as a whole. I select the tab for the regular image (L_dbe in my case) and then drag the little triangle at the bottom left of deconvolution to my image. This will take a longer time to calculate. It should now be obvious why I used a preview in making my adjustments. It takes a couple minutes to calculate the whole image on my computer.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6074097 - 09/10/13 10:52 PM Attachment (9 downloads)

As soon as it finishes, I save my image (I used L_deconv) as a 32 bit fit.

You can compare the before and after by using the Undo and Redo buttons at the top right of PixInsight.

You might want to zoom in some using your middle mouse wheel when doing these comparisons. Don't forget to make sure you Redo as your last step. Otherwise, you might continue processing on the image before you did the deconvolution and that would waste all this work.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6074103 - 09/10/13 10:55 PM

Next up, we are finally ready to stretch our image. But this is a good stopping point for this evening. Besides I'm collecting data on another project tonight. Automation software is wonderful.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6077619 - 09/12/13 07:31 PM Attachment (5 downloads)

It is now time to stretch the luminosity data. Bring up the HistogramTransform process. You want to blue check mark on the bottom right. That enables track view. What it buys you in practice is that changes you make will now be displayed in the upper histogram (i.e. what the histogram will now look like). You also want to press the blue circle in the bottom left of the window. That will bring up a preview window. Resize that window so you can see what you are doing.

Edited by Madratter (09/12/13 07:35 PM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6077628 - 09/12/13 07:41 PM Attachment (6 downloads)

Make sure you have reset the ScreenTransferFunction. Now move the middle slider for the Histogram Transformation over towards the left. You'll want to be quite aggressive with it.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6077629 - 09/12/13 07:42 PM Attachment (7 downloads)

Here is what the real-time preview looks like.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6077639 - 09/12/13 07:49 PM Attachment (6 downloads)

Now close the real-time preview and drag the blue triangle on the bottom left of the HistogramTranformation window over the image. Alternatively, click the blue box on bottom left of the HistogramTransformation window to apply it. My image now looks like this.

Edited by Madratter (09/12/13 07:49 PM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6077642 - 09/12/13 07:53 PM Attachment (4 downloads)

The settings in the HistogramTransformation window now need to be reset. Do that with the blue cross like icon on the bottom left.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6077644 - 09/12/13 07:55 PM Attachment (7 downloads)

Give things another stretch.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6077647 - 09/12/13 07:57 PM Attachment (12 downloads)

This is the result once applied.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6077689 - 09/12/13 08:41 PM Attachment (7 downloads)

Although it looks like we might be able to move the black point in, that is not the case. It would result in immediate clipping. It is time to run a denoise routine. My favorite and a relatively recent addition to PixInsight is the TGVDenoise Process. This process can take even longer than deconvolution so definitely define a preview using [alt]n. I'm going to define two of them: one where there is background as well as some interesting detail, and one over the center of the galaxy.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6077699 - 09/12/13 08:47 PM Attachment (5 downloads)

Now bring up the TGVDenoise process. I don't particularly like the defaults although they work at least acceptably well with this image.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6077701 - 09/12/13 08:49 PM Attachment (7 downloads)

Here is the center preview with the defaults.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6077705 - 09/12/13 08:52 PM Attachment (6 downloads)

A good starting point for many images is to up the number of iterations considerably to 500. Then bring the strength way down to between 1.2 and 1.4.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6077717 - 09/12/13 08:58 PM Attachment (10 downloads)

Fiddle around with the settings until you find something you like. The most important of the sliders is actually the middle one, edge protection. After trying a number of settings, I ended up with these.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6077723 - 09/12/13 09:00 PM Attachment (11 downloads)

Here is the background preview after applying it.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6077726 - 09/12/13 09:03 PM Attachment (8 downloads)

This was the preview before it was applied.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6077736 - 09/12/13 09:12 PM Attachment (7 downloads)

Now apply the settings to the image as a whole. Go ahead and get rid of the previews. (Under the Preview Menu, there is an option for delete all.) Once you do this, the histogram will look markedly different. Notice how flat things are to the left of the hump.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6077751 - 09/12/13 09:20 PM Attachment (12 downloads)

In the HistogramTransformation window, there are a number of handy tools. One of these is the "Auto zero shadows button". I have circled it in red to make it easy to find. Press it. Your histogram will now look something like this.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6077757 - 09/12/13 09:23 PM Attachment (6 downloads)

Now using the middle slider and the real-time preview, adjust the image to your taste.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6077761 - 09/12/13 09:25 PM Attachment (11 downloads)

The real-time preview:

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6077774 - 09/12/13 09:28 PM Attachment (6 downloads)

Again, delete the preview and apply the HistogramTransformation to the image. This concludes the processing on L portion of the image. Make sure you save this version of the image since you will need it as a file in a later step. I called mine L_final.

Next up are the color channels.

Edited by Madratter (09/12/13 09:35 PM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6077794 - 09/12/13 09:39 PM Attachment (7 downloads)

Bring up the R, G, and B stacks as well as the channel combination tool. Assign the stacks appropriately.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6077802 - 09/12/13 09:45 PM Attachment (7 downloads)

Now press the blue circle on the bottom left to actually combine the channels into one color image. Then do a ScreenTransferFunction to see the results. Make sure you unlink the channels (top right in STF window, the icon that looks like two chain links). Otherwise your result with be a rather ugly yellow-green.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6077816 - 09/12/13 09:50 PM Attachment (8 downloads)

Crop out the bad edges with the DynamicCrop tool just like you did earlier with the luminosity channel. If you redo the STF, the result will now reveal some rather unfortunate things.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6077827 - 09/12/13 10:02 PM Attachment (7 downloads)

Do another DynamicBackgroundExtraction. You'll need to be sure to close the DynamicCrop tool first. I auto-generated a bunch of points but then needed to move many of them and create more.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6077830 - 09/12/13 10:03 PM Attachment (9 downloads)

My samples:

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6077832 - 09/12/13 10:05 PM Attachment (9 downloads)

This is beginning to look reasonably good, even without the luminosity applied. There is 2 1/4 hours of data here, so that is not completely surprising.

Edited by Madratter (09/12/13 10:08 PM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6077859 - 09/12/13 10:17 PM Attachment (10 downloads)

The next step is color calibration. Frankly this is an area where I am not completely happy with PixInsight. I won't go into long explanations of why. Suffice it to say that I often now use other tools like Regim to color calibrate my images. However, this is a PixInsight Tutorial. So bring up the ColorCalibration tool. Also define some previews.

What we are going to do is define one preview around a good portion of the galaxy. Another will be defined in an area that is primarily background. These preview areas will be used to color balance the image. What PixInsight will do is try to make the balance of the galaxy neutral overall. This will result in a rather yellow middle and somewhat blue arms.

Here are my previews.

Edited by Madratter (09/12/13 10:22 PM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6077867 - 09/12/13 10:21 PM Attachment (8 downloads)

In the ColorCalibration settings, set the reference image to the preview around the galaxy. Make sure you turn structure detection off (if you leave it on, it will balance using the stars rather than the galaxy). Set the background reference to the preview around the background.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6077876 - 09/12/13 10:26 PM Attachment (12 downloads)

After applying the color calibration, you can now redo the STF with the channels linked. The image will no longer have the horrible yellow-green cast to it.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6077882 - 09/12/13 10:29 PM

It is now time to stretch the RGB data using HistogramTransformation. You can delete the previews used in ColorCalibration.

One thing I want to point out before I forget is the lack of dust donuts here in the RGB image. The reason is that flats were taken for each color separately. If you take just an L flat, you will often be dealing with them here.

By the way, I forgot to put in the Color Calibration step in the initial post with the summary of the workflow. Unfortunately, I can no longer edit that post.

That's all for tonight.

Edited by Madratter (09/12/13 10:41 PM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
pfile
Post Laureate


Reged: 06/14/09

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6077942 - 09/12/13 11:01 PM

Quote:

that was the name of their dog (or biff). OK, enough of the deep inside jokes.





% biff y


Post Extras: Print Post   Remind Me!   Notify Moderator  
pfile
Post Laureate


Reged: 06/14/09

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: pfile]
      #6077977 - 09/12/13 11:16 PM

all jokes aside... this is a great step-by-step... i would add the following to what you have now:

1) sometimes the deconvolution can jack up the cores of the brightest stars. one way to deal with this is to use a starmask as a "local support" image (found in the deconvolution process under the deringing area.) in fact, it's usually a good idea to always use a local support image. another way is to further modify the luminance mask for the deconvolution by subtracting a star mask from it, thus protecting the cores of the bright stars.

2) any process's triangle may be dragged to the desktop to save it. a good one to save is the dynamic crop of the L, so that you can apply it to the RGB with no ambiguity. this of course assumes you've registered all your L,R,G,B subs to a common frame. you can also load the history explorer from any one of your images and drag any step out to the desktop or onto another image. finally you can open the history explorer and drag the triangle to the desktop, thus making a copy of the processing steps you've done. you can then edit that 'process container' to remove steps (if you like) and apply that whole mess to another image. the ability to save and repeat processes is one of the most important features of PI and can save you boatloads of time.

3) along those lines, if you save the DBE from the L you can then double click the process icon which re-loads the tool with those settings. you can then tweak it for the RGB without having to re-place all your samples.

4) also related to this - in the color calibration dialogs you will notice the "region of interest" boxes for the white and background references. if instead of giving the preview as the reference, you load the region of interest from the preview, the color calibration process becomes "re-entrant", that is, you can use it on a clone of your image without first defining the exact same previews. otherwise, the saved icon for CC is specific to the image, since the name of the image and the name of the preview is called out.

rob


Post Extras: Print Post   Remind Me!   Notify Moderator  
jerryaxe
member


Reged: 10/07/08

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: pfile]
      #6078513 - 09/13/13 10:49 AM

Good morning to all,

I guess the best thing to do is sharing the process icons of your pre-processing and post-processing routine.
Organized the proper way, it allows to have a logical organization between the linear and non-linear processing.

Cheers and clear sky

Jerry


Post Extras: Print Post   Remind Me!   Notify Moderator  
pfile
Post Laureate


Reged: 06/14/09

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: jerryaxe]
      #6078586 - 09/13/13 11:44 AM

yes - good idea. the thought had popped into my mind but i forgot to write it above.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: pfile]
      #6078894 - 09/13/13 03:04 PM Attachment (7 downloads)

Thanks Rob. Your tips are on target.

In the second deconvolution where I do it right, I actually do use a star mask as the local support.

I do use process icons myself and they can be very handy with certain repeated actions.

Unfortunately, since this is being posted here, there is no good way of distributing a set for this processing. If it later gets moved to a website, that is a different kettle of fish.

So I'm off for the day. Let's see if I can wrap this up.

As mentioned when I left off, the next step is to start stretching the RGB data. Again, we will use a HistogramTransformation, I have moved the middle slider quite aggressively. I am going to assume you know the drill by now of using the real-time preview and then applying the transformation to the actual image.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6078901 - 09/13/13 03:08 PM Attachment (11 downloads)

The result...

By the way, might notice that this result is more washed out than the STF we did earlier. The reason for this is that the STF when used in auto mode will happily clip the blacks.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6078908 - 09/13/13 03:13 PM Attachment (6 downloads)

Now for TGVDenoise on the RGB image. Create some previews around an area with some background but some interesting detail ([alt]n). Also create a preview around the center of the galaxy.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6078932 - 09/13/13 03:25 PM Attachment (9 downloads)

With an RGB image, we can do denoise for both the luminance (lightness) and the color (chrominance) portions of the image. Again, the key to good TGV denoise is lower the strength way down and up the iterations to something like 500.

If I recall correctly from the excellent (and free) video tutorial by Warren on TGVDenoise:

http://www.ip4ap.com/IP$AP_Flash/Pixinsight_2/PI-13_Nonlinear-7/PI-13_Nonline...

You should use RGB on linear images. On non-linear images, you should use CIE Lab.

Here are the settings I ended up with:


Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6078938 - 09/13/13 03:28 PM Attachment (8 downloads)

Make sure you have the local support checked.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6078960 - 09/13/13 03:37 PM Attachment (7 downloads)

Once applied, the image looks like this. You will notice it pretty much looks the same. And that is the point. At a macro level it should look the same. It is only when looking closely that the differences should become real apparent.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6078971 - 09/13/13 03:43 PM Attachment (6 downloads)

Now delete the previews. We don't need them. It is time for another histogram stretch.

NOTE: DO NOT use the auto zero shadows. It will do each channel individually and mess up the color calibration.

When moving the left slider, go as far to the right as you can without clipping the image (the pixel count for shadows should stay at zero).


Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6078980 - 09/13/13 03:45 PM Attachment (5 downloads)

Things are starting to look a bit better.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6078987 - 09/13/13 03:55 PM Attachment (4 downloads)

While this is starting to look good, it definitely could use a little color boost. To do this the saturation of the image needs to be increased. However, if you boost the saturation everywhere, you can get ugly colors in the shadows that you don't want. The way to prevent this is to create a mask. Clone the image as before (or you can right click on the image and select, "Duplicate". Then create a histogram. Again, here you probably WANT to clip the shadows.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6078991 - 09/13/13 03:57 PM Attachment (11 downloads)

Here is what the mask looks like. If this was football, we would definitely have a penalty.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6078994 - 09/13/13 03:59 PM Attachment (4 downloads)

Apply the mask to the image (drag the tab on the right side with the mask name over below the tab for the image). Again, turn off the display of the mask ([ctrl]k).

Edited by Madratter (09/13/13 04:00 PM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6079005 - 09/13/13 04:06 PM Attachment (4 downloads)

Despite what you might initially think, one of the best ways to increase saturation in an image is to use the CurvesTransformation process. (Logically, you might go looking for a ColorSaturation process and one exists. However CurvesTransformation is the tool we want right now).

To the far right on the CurvesTransformation window, there is an icon with an S beside it. When selected, that puts CurvesTransformation into Saturation mode. Then using real-time preview, adjust up the Saturation. I prefer doing this in two stages for more control. Here is the first take at it.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6079010 - 09/13/13 04:08 PM Attachment (4 downloads)

Apply it.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6079014 - 09/13/13 04:11 PM Attachment (3 downloads)

Now a second go around is done to fine tune where I want this. How far to boost the saturation is definitely a matter of taste. Some people like a lot; some like very little.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6079017 - 09/13/13 04:12 PM Attachment (7 downloads)

Applied:

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6079025 - 09/13/13 04:18 PM Attachment (4 downloads)

The image must now be saved because it is time to align it with the luminosity channel that was done earlier. I saved mine as RGB_final (with 32 bits Fits of course). Once this is done, bring up the StarAlignment process. Exactly what the files will be called is going to depend on what you named them. The same is true with the folder where you want the result saved.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6079032 - 09/13/13 04:23 PM Attachment (3 downloads)

Once this is run, you should now have an RGB file in the specified folder that is aligned precisely with the luminosity file.

NOTE: If as in this case, your data for RGB is binned but your luminosity data is not, the RGB data will now be upsampled to match the luminosity data.

Here is the aligned RGB file. If you compare this image with the previous one, you can see things have been shifted slightly. Note the _r appended to the name.

Edited by Madratter (09/13/13 04:24 PM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6079038 - 09/13/13 04:29 PM Attachment (4 downloads)

Load this and the final luminosity file. Then bring up the LRGBCombination process. We have already combined the RGB together so we only need to check the L box and fill that in. Also make sure to check the Chrominance Noise Reduction. With some images you may find that it is necessary to move the lightness and saturation sliders to get a pleasant looking image after the combine. That is not the case with this image.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6079044 - 09/13/13 04:32 PM Attachment (6 downloads)

Drag the blue triangle at the bottom left over the registered (aligned) RGB image. This will combine them.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6079055 - 09/13/13 04:38 PM Attachment (9 downloads)

Carefully inspect the edges of the image. Because we have aligned things, it may be necessary to do another crop. That does not happen to be the case here.

There was a time in the not to distant past I would have been ecstatic with the image as is. However, by adjusting the contrast using CurvesTransformation, we can get the image to pop a little better. Your CurvesTransformation process is probably still set for Saturation, so make sure you set it to RGB/K. Using real-time preview adjust the curve to get more contrast. A kind of S shaped curve will do that and mimic a response more similar to film.

Edited by Madratter (09/13/13 05:08 PM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6079059 - 09/13/13 04:41 PM Attachment (7 downloads)

Apply it.

As a free tip, this trick works great on regular photographs as well!

Edited by Madratter (09/13/13 04:41 PM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6079064 - 09/13/13 04:45 PM Attachment (4 downloads)

There is now a touch of green in the dim parts of M33 in the left portion of the photo. There are very few things in Astrophotography that are green. Fortunately, PixInsight gives us a tool we need to clean this up, SCNR (Subtractive Chrominance Noise Reduction if you must know). The default settings work fine for our purpose.

Edited by Madratter (09/14/13 10:19 AM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6079069 - 09/13/13 04:47 PM Attachment (6 downloads)

applied...

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6079080 - 09/13/13 04:53 PM Attachment (12 downloads)

Some people might want to quit right here. Again, there is a certain amount of discretion in image processing, which, aside from skill, is why two people processing the same data will come up with different versions. Personally, I want to boost the brightness of the arms. To do this, I take a final whack at it with CurvesTransformation. To figure out where on the curve I need to really adjust, I run my cursor over the image. The arms have RGB values of around 15 to 20% so that is the area I need to lift. As usual real-time preview is used to fine tune the adjustment.

Edited by Madratter (09/13/13 04:54 PM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6079084 - 09/13/13 04:58 PM Attachment (8 downloads)

We apply the curve to the image and at long last, the image has been completed.

At this point let me encourage you to post your own versions if you have run through the tutorial. If you have questions, by all means post them.

Also, go ahead and post versions that have been produced using other software. Those will certainly be of interest as well.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6079100 - 09/13/13 05:12 PM Attachment (9 downloads)

Here is a version that has been resampled to 800 pixels maximum in PixInsight and then slightly sharpened for display here on Cloudy Nights.

Messier 33 on Astrobin

Edited by Madratter (09/13/13 05:28 PM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
pfile
Post Laureate


Reged: 06/14/09

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6079103 - 09/13/13 05:12 PM

looks great, good tutorial.

one minor point - you don't have to save any images to do registration. the reference image can be a view, so if your final L is still open, you can select it as a view. and then you can just drag the triangle of the StarAlignment process to the RGB, and a new image will be created which is registered to the reference.

as i mentioned above this alignment step is not necessary if you register all your different filtered subs to the same reference. that way you avoid another registration, which always degrades the data.

rob


Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: pfile]
      #6079106 - 09/13/13 05:19 PM

Rob, the key point that makes your method of aligning the luminosity with the RGB channels early work, is that you can save the DynamicCrop settings as a Process Icon and then use the same settings when cropping the Luminosity and RGB images. I haven't been doing that, and indeed, it is a worthwhile addition to my workflow. Thanks for bringing it up.

Post Extras: Print Post   Remind Me!   Notify Moderator  
pfile
Post Laureate


Reged: 06/14/09

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6079182 - 09/13/13 06:19 PM

one weird thing is, and i don't know why it is this way, that you need to save the dynamic crop process icon *before* you apply it to the image. there's something about applying a dynamic process to an image that changes it so that it's specific to the target image.

but i think you can still drag a dynamic crop step out of a process history onto another image and it works okay. so even if you didn't save it explicitly you can always go back and find it, provided you didn't close the image. the process history does not get saved when you save an image to disk (but it does get saved when you save a project from within PI)


Post Extras: Print Post   Remind Me!   Notify Moderator  
pfile
Post Laureate


Reged: 06/14/09

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: pfile]
      #6079223 - 09/13/13 06:52 PM

ah - i just re-read this and regarding the color calibration step... you need to first do BackgroundNeutralization before doing CC. that might be why you are having problems with the color calibration.

so you define the preview on the background just like you did, then open BackgroundNeutralization, do region of interest from the preview, run your mouse over the background, looking at the RGB values. set the upper limit high enough to exceed the highest background level, but just so. you don't want to pick up any stars or bright structures in the preview area.

after applying BN, then move to the ColorCalibration and forward from there.

after the BN, the colors will look funny since the STF was trying to neutralize the background for you. at this point you can link the channels (if you want) and recompute the STF and it should look okay again.


Edited by pfile (09/13/13 06:54 PM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
gramaglia
super member


Reged: 04/02/12

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: pfile]
      #6079643 - 09/13/13 11:52 PM

An additional process that seems to enhance contrast quite a bit and removes some of the "flatness" from HDR wavelets is the local equalization histogram function. It's a process that I found seems to improve a lot of images. It might be that my images are pretty bad therefore easily made better!!! Anyways just a suggestion on a additional step to process with pixinsight. Other wise you tutorial is virtually identical to the steps I follow that I learned pixinsight watching the videos from harry's astronomy shed.
your tutorial is a great written down version and step by step guide.


Post Extras: Print Post   Remind Me!   Notify Moderator  
mattw
sage


Reged: 07/19/06

Loc: Arkansas
Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: gramaglia]
      #6079825 - 09/14/13 05:00 AM

I routinely use PI, but still picked up numerous excellent tips from your
thread. Thanks for all the hard work, it is highly appreciated!


Post Extras: Print Post   Remind Me!   Notify Moderator  
terry59
Post Laureate
*****

Reged: 07/18/11

Loc: Colorado, USA
Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6079950 - 09/14/13 08:22 AM

Thank you for taking the time to post all of this information. It is the best beginning to end processing method for PI that I've seen (and it is free too).

Hopefully it will get more people to take the plunge.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: terry59]
      #6081749 - 09/15/13 09:42 AM

Thanks for the kind words.

Pfile, thanks for the hints on the color calibration step. I went back and listened to Harry's video tutorial (which is how I learned some of this stuff) and he does mention it.

BTW, for those who are not aware, Harry has excellent video tutorials on PixInsight. They don't cover everything here (such as the deconvolution workflow or TGVDenoise), but they are great and he covers things I do not.

As Gramaglia mentions, my workflow closely follows the one laid out there. (Thanks for the tip on Histogram Equalization - I'll need to experiment with that).

Harry's tutorials can be found here:

Harry's PixInsight Tutorials

I should also say that you should not be slave to the workflow here (or anywhere else). Alternatives are possible and experimenting will sometimes lead to a better result. For example, I was recently working on an image where I found I got better results doing a denoise before I did the HDRMultiscale transform.

Edited by Madratter (09/15/13 09:44 AM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
harry page 1
super member


Reged: 07/25/09

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6082498 - 09/15/13 05:42 PM

Hi

well looks like I will have to do decon and TGV then

Regards

Harry


Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: harry page 1]
      #6082515 - 09/15/13 05:48 PM

Quote:

Hi

well looks like I will have to do decon and TGV then

Regards

Harry




I would love to see you do that. Your tutorials are fabulous and without them, PixInsight would have been a total bust for me.


Post Extras: Print Post   Remind Me!   Notify Moderator  
gramaglia
super member


Reged: 04/02/12

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6082885 - 09/15/13 09:50 PM

will looking forward to you new videos on decon and TVG harry, your previous videos have been a priceless. Saved me months of learning. Also thank you madratter for you tutorial on decon. Always easier to learn from people that already done it instead of trying to reinvent the wheel. Without the help of you guys I would still be driving with a pentagonal/hexagonal wheel!!!

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: gramaglia]
      #6086134 - 09/17/13 02:26 PM

Gramaglia, you're welcome. I'm glad the deconvolution section was useful.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Jeff2011
Carpal Tunnel
*****

Reged: 01/01/13

Loc: Sugar Land, TX
Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6086513 - 09/17/13 06:09 PM

IMHO this thread should be included in the Best Of sticky thread. Good job MR. This has been most interesting and helpful in understanding PI and processing in general.

Post Extras: Print Post   Remind Me!   Notify Moderator  
SergeC
scholastic sledgehammer
*****

Reged: 03/23/12

Loc: Gainesville, FL
Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Jeff2011]
      #6086551 - 09/17/13 06:28 PM

Even though I don't use PI I want yo thank you for all your hard work and selfless effort in putting this together. Regardless of the specific tools one might use for working on images, there's a wealth of information here that anyone can apply to their processing

Edited by SergeC (09/17/13 06:32 PM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: SergeC]
      #6091383 - 09/20/13 09:47 AM

Thanks Jeff and Serge. While I would love to see this put into the best of thread, I have added it to my Sig so as long as I am posting frequently, people should be able to find it even if it has fallen way down the list.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Jeff2011
Carpal Tunnel
*****

Reged: 01/01/13

Loc: Sugar Land, TX
Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6105273 - 09/28/13 03:05 AM Attachment (7 downloads)

Here is my attempt at processing this data in the StarTools beta 1.3.5.246. All processing was done in StarTools except the scaling of the resulting RGB to match the Luminance and the alignment of the Luminance and RGB images. Both done in IRIS. Final scaling and conversion to JPEG to post on CN was in GIMP. This represents my 4th attempt at processing this data and I really went aggressive on the stretching on this last attempt. Perhaps a bit too much on the color saturation, but the emission nebulas in this galaxy really stand out. I also encountered some black speckling throughout the image for both the luminance and RGB image which I tried to clean up with StarTool's denoise. This has been fun to process as it is the first time I have attempted an LRGB image.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Jeff2011]
      #6105717 - 09/28/13 11:03 AM

Psychedelic. it does really bring out the Ha regions though!

Post Extras: Print Post   Remind Me!   Notify Moderator  
Jeff2011
Carpal Tunnel
*****

Reged: 01/01/13

Loc: Sugar Land, TX
Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6105765 - 09/28/13 11:32 AM

I over saturated the RGB because I thought combining it with the luminance would dilute it. I also blew out the brighter stars and the core with my aggressive stretching. Perhaps I should redo the RGB.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Ivo Jager
Vendor ( Star Tools )


Reged: 03/19/11

Loc: Melbourne, Australia
Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Jeff2011]
      #6106411 - 09/28/13 06:16 PM

This is an awesome tutorial Madratter! I'm sure a lot of folks getting started with PI would find this extremely helpful!
Would love to take your data and do an ST tutorial as well...


Post Extras: Print Post   Remind Me!   Notify Moderator  
Jeff2011
Carpal Tunnel
*****

Reged: 01/01/13

Loc: Sugar Land, TX
Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Ivo Jager]
      #6106434 - 09/28/13 06:34 PM

Ivo,

I would love to see that. I hope MR grants you permission to do that. Or at a minimum post your results and steps on this thread. He did ask for people to post results using other processing programs as well.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Jeff2011]
      #6106506 - 09/28/13 07:44 PM

Ivo, I think that would be terrific! Besides, I would benefit from it since I have StarTools too.

As for where to put it, I'm sure you would want your own thread, but the final result here and a link to that thread would be great. But that is completely up to you.

Edited by Madratter (09/28/13 07:44 PM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
Jeff2011
Carpal Tunnel
*****

Reged: 01/01/13

Loc: Sugar Land, TX
Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6106580 - 09/28/13 08:58 PM

That is great. Ivo you can find a link to his files in his first post in this thread. That is if you have not found them already.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Jeff2011]
      #6107699 - 09/29/13 03:04 PM

I realized that the L channel is not aligned with the R, G, and B channels and that is going to cause problems in StarTools unless the user aligns the file. And of course, that is not what the StarTools tutorial should be about, and people use different tools for that. So I have created an aligned version of the L file called L_forStarTools and uploaded it to the dropbox link.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Ivo Jager
Vendor ( Star Tools )


Reged: 03/19/11

Loc: Melbourne, Australia
Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6107991 - 09/29/13 06:27 PM

Quote:

I realized that the L channel is not aligned with the R, G, and B channels and that is going to cause problems in StarTools unless the user aligns the file. And of course, that is not what the StarTools tutorial should be about, and people use different tools for that. So I have created an aligned version of the L file called L_forStarTools and uploaded it to the dropbox link.




Thanks! It is actually possible to do this in StarTools, but indeed, pre-processing isn't what StarTools is about.
What format would you like me to do this in? I was thinking an annotated YouTube video would be nice as well (full credits for the data to you of course), so that people can see what was clicked in what sequence (and why). It'll also allow me a bit more freedom to compare the sequence and operation to what one would expect in other 'linear' software (as you know ST can be distinctly different due to Tracking) - I've run into trouble with an overzealous moderator before, for even mentioning the 'name(s)-that-will-not-be-mentioned', just because I'm apparently classified as a 'vendor' here...

Edited by Ivo Jager (09/29/13 06:30 PM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Ivo Jager]
      #6108077 - 09/29/13 07:41 PM

Ivo, annotated YouTube video would be great if that is what you feel comfortable doing.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Ivo Jager
Vendor ( Star Tools )


Reged: 03/19/11

Loc: Melbourne, Australia
Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6108459 - 09/29/13 11:54 PM

I have a small problem with the 'L_forStarTools' file. It doesn't appear to match the other FITS file. In particular, it appears it has no signal that corresponds with blue-dominant pixels in the RGB file (making any composite yellow). May I asked how you created the new L file?

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Ivo Jager]
      #6108980 - 09/30/13 10:02 AM

Taking this to PM.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6111881 - 10/01/13 08:05 PM Attachment (5 downloads)

Ivo Jager, author of StarTools has now produced a Video tutorial for StarTools using this data. I would like to thank him for the time he put into this. It is much appreciated, and I certainly benefit since I own StarTools as well (in fact I purchased it after I bought PixInsight because of some of the things it does so well).

Here is a pointer to that tutorial:

StarTools Tutorial using M33 Data

The fullsize final result

And here is a resized version to meet Cloudy Night Requirements:

Edited by Madratter (10/01/13 08:07 PM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
pfile
Post Laureate


Reged: 06/14/09

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6112366 - 10/02/13 01:28 AM Attachment (9 downloads)

saw this thread had been bumped and realized i forgot to post my processed image (pixinsight 1.8RC7)

Edited by pfile (10/02/13 01:29 AM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: pfile]
      #6112855 - 10/02/13 10:46 AM

Excellent Pfile. You managed to get more of the faint stuff on the left than I did.

Post Extras: Print Post   Remind Me!   Notify Moderator  
torsinadoc
sage


Reged: 02/04/13

Loc: Fayetteville AR
Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6149886 - 10/21/13 01:21 PM

Quote:

First, let me state flat out, I am NOT an expert in either PixInsight or Image Acquisition. This data we will be using is not perfect; it has significant warts. But in a way, that is a good thing since it will require techniques and such that better data might not. Likewise, the choices I make when processing this may not be optimum. But I'm pretty sure we will end up with a usable result. If you have an alternative you think is better, by all means speak up. I'm open to learning something new.

This is not an ideal format for this for at least two reasons. First, there is a limit of one picture per post. And secondly, the max size for a picture is 800x800. So I'll have to be reducing the size of some of the results/and or cropping them for display here.

The data itself was taken on the night of September 4th, 2013. A Celestron Powerstar III 8" SCT with a f/6.3 reducer flattener was used so the focal length is close to 1260mm. All subs were 5 minutes in length. Luminosity was unbinned (1x1). R, G, and B were all binned 2x2. Flats were taken for each of those 4 channels individually. Total exposure time for luminosity was 1 hour 55 minutes. Each color channel had 45 minutes of time. I can get by with less than that and sometimes go with just 30 minutes color. However, I like using 45 minutes when I can get it because Winsorized sigma clipping works better.

This tutorial is going to start with the stacked, calibrated images (darks, flats, flat darks). I happen to really like doing the calibration and stacking within PixInsight but I know others still use other tools like Deep Sky Stacker.

The channel stacks are available on DropBox here:

Messier 33 Channel Stacks on Dropbox

I'm a strong believer that you learn better when you actually follow along and do the procedure. Still, it is your choice.

It is also possible I will have false starts and will back up in the processing. That is fine if it occurs. That will show how I think about the processing.

To a certain extent my workflow depends on what happens with the image, but here is a skeleton outline of some of the likely steps:

Luminosity:

Crop out stacking artifacts
Dynamic Background Extraction (or Automatic Background Extraction)
Create Point Spread Function for Deconvolution
Create Star Mask for Deconvolution
Create Overall Mask for Deconvolution
Deconvolution
Histogram Transformation
HDRMultiscaleTransform (Wavelets)
TGV Denoise
Histogram Transformation

RGB

Channel Combine the separate color stacks
Crop
Dynamic Background Extraction or Automatic Background Extraction to get rid of gradients
Histogram Transformation
TGV Denoise
Histogram Transformation
Saturation done using the Curves Tool

L and RGB
Register (Align) the separate L and RGB images
LRGB Combine
Crop
Curves (on the luminosity portion) to adjust contrast





I have been confused with DSLR imaging and most workflows

How would a DSLR debayered image modify the above workflow?


Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: torsinadoc]
      #6149894 - 10/21/13 01:29 PM

First I would crop out the stacking artifacts. Then what I would do is create a luminosity channel from the debayered RGB image. I would then use that for the luminosity steps above except no cropping is necessary. Since the RGB is already combined, the first step in the RGB processing is not necessary. The rest is pretty much the same except since the L came directly from the cropped RGB layer, it is already aligned so no alignment is necessary.

Also make sure the image out of your camera is linear or else the deconvolution portions will not work properly.

All in all, the work flow is very similar.

Edited by Madratter (10/21/13 01:31 PM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
torsinadoc
sage


Reged: 02/04/13

Loc: Fayetteville AR
Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6149957 - 10/21/13 02:19 PM

thanks. its a raw image so assume it is linear. I stack and debayer it. I have been using the batch preprocessing with CFA. I also use it to integrate but get a message suggesting using imageintegration instead of the batchpreprocess. Is there really a difference? I like using the batchpreprocess for calibration, registration and integration. If stacking would be better using imageintegration I could use it.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: torsinadoc]
      #6149962 - 10/21/13 02:22 PM

I haven't played with the batch processing script so I can't really comment on it vs the image calibration, alignment, image integration route that I use.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6271790 - 12/25/13 07:47 AM

The instructions on this page show how to correctly setup the calibration frames in PixInsight. Since I was matching my darks to my lights temperature wise, I was skipping the Bias frames. I am now using them. Also, I wasn't using the flux equalization setting under pixel rejection for normalization of my flats. I am now getting better results:

http://pixinsight.com/tutorials/master-frames/en.html


Post Extras: Print Post   Remind Me!   Notify Moderator  
terry59
Post Laureate
*****

Reged: 07/18/11

Loc: Colorado, USA
Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6271801 - 12/25/13 07:57 AM

Thanks for doing this. Added to my list of things to experiment with

Post Extras: Print Post   Remind Me!   Notify Moderator  
torsinadoc
sage


Reged: 02/04/13

Loc: Fayetteville AR
Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: terry59]
      #6477346 - 04/20/14 01:04 PM

Other than decon, how elese do you incorporate Startools into the above workflow?

Post Extras: Print Post   Remind Me!   Notify Moderator  
torsinadoc
sage


Reged: 02/04/13

Loc: Fayetteville AR
Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: terry59]
      #6477347 - 04/20/14 01:04 PM

Other than decon, how elese do you incorporate Startools into the above workflow?

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: torsinadoc]
      #6477419 - 04/20/14 01:53 PM

My post processing technique is in a constant state of flux depending on what I have learned at any particular time. I was using StarTools a lot to process the L channel in general. The color data I still preferred to process in PI.

Recently, I have switched back to doing L processing in PI. However, I am now using masked stretch to control star bloat. Actual combining of the color and L data I do in Photoshop CS6. (any version of Photoshop CS2 and on can do what I need).

All of that is more advanced technique. The foundation are the things found here. I would probably rewrite some of this, but it won't take you too far wrong.


Post Extras: Print Post   Remind Me!   Notify Moderator  
terry59
Post Laureate
*****

Reged: 07/18/11

Loc: Colorado, USA
Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6477434 - 04/20/14 02:04 PM

Quote:

However, I am now using masked stretch to control star bloat.




I tried that for the first time yesterday and really liked the result.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: terry59]
      #6477456 - 04/20/14 02:19 PM

One downside to masked stretch is it can do very weird things to stars if your collimation is off. It can also mess up really bright saturated stars. The former is best fixed by fixing the collimation. The later can be easy touched up in Photoshop or the like.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Madratter
Postmaster


Reged: 01/14/13

Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6478154 - 04/20/14 10:05 PM

A suggestion made early on in this thread was gathering this information onto a website. I have now started that process:

http://astroimages.weebly.com/

Click on the PixInsight Tutorial button.

I plan to add information from some of my other threads to the site as well.


Post Extras: Print Post   Remind Me!   Notify Moderator  
terry59
Post Laureate
*****

Reged: 07/18/11

Loc: Colorado, USA
Re: Processing Messier 33 in PixInsight - A Tutorial new [Re: Madratter]
      #6478160 - 04/20/14 10:07 PM

Quote:

A suggestion made early on in this thread was gathering this information onto a website. I have now started that process:

http://astroimages.weebly.com/

Click on the PixInsight Tutorial button.

I plan to add information from some of my other threads to the site as well.




Excellent....bookmarked


Post Extras: Print Post   Remind Me!   Notify Moderator  
Pages: 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | (show all)


Extra information
7 registered and 10 anonymous users are browsing this forum.

Moderator:  Knuklhdastrnmr, fishonkevin, WOBentley, rflinn68 

Print Thread

Forum Permissions
      You cannot start new topics
      You cannot reply to topics
      HTML is disabled
      UBBCode is enabled


Thread views: 4137

Jump to

CN Forums Home


Cloudy Nights LLC
Cloudy Nights Sponsor: Astronomics