Return to the Cloudy Nights Telescope Reviews home pageAstronomics discounts for Cloudy Nights members
· Get a Cloudy Nights T-Shirt · Submit a Review / Article

Click here if you are having trouble logging into the forums

Privacy Policy | Please read our Terms of Service | Signup and Troubleshooting FAQ | Problems? PM a Red or a Green Gu… uh, User

Astrophotography and Sketching >> DSLR & Digital Camera Astro Imaging & Processing

Pages: 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | (show all)
jsines
sage
*****

Reged: 09/06/11

Loc: Berkley. Michigan
Re: Learning PixInsight new [Re: mmalik]
      #5748114 - 03/21/13 05:34 PM

Quote:


Thing I am not so sure about if DBE is the "right" way to fix the green cast of 'conversion/alignment/integration' processes or is this just masking the problem? Ideally, wouldn't one want to have a normal looking integration.fit upon screen Auto Stretch like in other programs? At least I would like to see an 'OK' looking conclusion to a core step in the process instead of hoping the next will take care of the previous.

Would it be a correct presumption that 'conversion/alignment/integration' processes are NOT optimized in PixInsight? NOT a critique of PixInsight in any way, just would like an honest assessment of the 'conversion/alignment/integration' problem if there is one and if one should skip these processes in PixInsight if they may be deficient?


If true, this complicates matters further. I am NOT able to process CR2s converted/aligned/combined in ImagesPlus for e.g., DBE processing in PixInsight?






It seems to me like you're trying to open a fit file created in ImagesPlus in PixInsight, or trying to open a fit file created in PixInsight in ImagesPlus, finding that the files are not 100% compatible, then concluding that PixInsight has a problem. I don't think this is the fault of ImagesPlus or PixInsight, I think it's just the way the 2 programs define the fit file.

The solution is to do everything in one program, and since this thread is about PixInsight, lets do it all in PixInsight. Not sure why you would want to calibrate/align/register in ImagesPlus and then move over to PI anyway, since the PI process is better.

Was the PixInsight integration.fit file you uploaded created solely in PixInsight using your CR2 files?


Post Extras: Print Post   Remind Me!   Notify Moderator  
Falcon-
Post Laureate
*****

Reged: 09/11/09

Loc: Gambier Island, BC, Canada
Re: Learning PixInsight new [Re: mmalik]
      #5748136 - 03/21/13 05:40 PM

Quote:

Thing I am not so sure about if DBE is the "right" way to fix the green cast of 'conversion/alignment/integration' processes or is this just masking the problem? Ideally, wouldn't one want to have a normal looking integration.fit upon screen Auto Stretch like in other programs? At least I would like to see an 'OK' looking conclusion to a core step in the process instead of hoping the next will take care of the previous.




DBE, BackgroundNeutralization, and ColorCalibration are the way to fix the green cast. (not necessarily all three, but nearly always at least one of the three).

Keep in mind that other apps either do a rough-guess initial version of BackgroundNeutralization or ColorCalibration or try and automatically do it. In other words the "problem" is always there with any program, PixInsight just chooses not to hide it from you

As to if DBE specifically is the right way - I would say that for *THIS* instance where the field of view is narrow and the "background" is all desired target I would not use it. DBE is going to look at the gradient of brightness in the HA curtain wall behind the horse-head and try and remove it as if it was a light pollution gradient. MOST of the time DBE would be desirable, for this image good flats and no DBE is what I would use for best results.


Quote:

Would it be a correct presumption that 'conversion/alignment/integration' processes are NOT optimized in PixInsight? NOT a critique of PixInsight in any way, just would like an honest assessment of the 'conversion/alignment/integration' problem if there is one and if one should skip these processes in PixInsight if they may be deficient?




Basic alignment may be only equal with many apps, but the mosaic tools are superior. Calibration and especially integration are superior to my experiences with Nebulosity and DeepSkyStacker.

Quote:

I am NOT able to process CR2s converted/aligned/combined in ImagesPlus for e.g., DBE processing in PixInsight?




Check ImagesPlus for FITS format options. It may be that a slight change in the way ImagesPlus saves FITS files will let PixInsight work with them.

Unfortunately FITS files in general are just not as standard as one might hope. It is such a open/permissive "standard" that each program has decided to implement it in different ways. PixInsight can save and open various types of FITS (16 and 32bit integer, 32 and 64 bit floating point, a few others) as well as a few of the ways to arrange the coordinates in the FITS files themselves. I also know Nebulosity has preferences for saving it's FITS files either in the "ImagesPlus" way or the "Maxim" way just to try and work around such problems (but it still can not open the PixInsight 32bit float types). Perhaps ImagesPlus can output to a 16bit or 32bit TIFF?


Post Extras: Print Post   Remind Me!   Notify Moderator  
bluedandelion
Carpal Tunnel
*****

Reged: 08/17/07

Loc: Hazy Hollow, Western WA
Re: Learning PixInsight new [Re: Falcon-]
      #5748233 - 03/21/13 06:24 PM

Mike, you raise some valid points:

About the color cast. I like control of my data so I do not do auto white balance. Nebulosity always gave me a yellowish tinged stack and DSS a blue tinged one. The point is that all images have to be color balanced at some point as Harry and Rob (pfile) have pointed out. Sometimes this happens in camera as with "daylight" photography. No modded DSLR is not going to give a color balanced image. People use grey cards and other techniques to know the correct balance. I had to do this with Nebulosity.

The cast you see with the screen stretch (STF for Screen Transfer Function) is because the colors are equally weighted by default. Unlink them and you should see something better. Also remember what you see with STF hasn't been applied to your data. Its just a way of previewing the image. Only after a color calibration and a proper histogram stretch will you see an accurate representation on the screen. At that point you should not use STF.

About image formats: Your difficulty with ImagesPlus files in PI is exactly what I experienced with Compressed Fits from Nebulosity. Fits is supposed to be a standard, but often the output of one program cannot be directly used in another. This is an old story. The PI forums may have some solution to your problem. The developer reads posts and responds.

Hope this helps.

Edit: Mostly what Sean said

Ajay

Edited by bluedandelion (03/21/13 06:32 PM)


Post Extras: Print Post   Remind Me!   Notify Moderator  
mmalik
Post Laureate
*****

Reged: 01/13/12

Loc: USA
Re: Learning PixInsight new [Re: jsines]
      #5748259 - 03/21/13 06:38 PM

Quote:

The solution is to do everything in one program, and since this thread is about PixInsight, lets do it all in PixInsight. Not sure why you would want to calibrate/align/register in ImagesPlus and then move over to PI anyway, since the PI process is better.

Was the PixInsight integration.fit file you uploaded created solely in PixInsight using your CR2 files?




Fully agree with you; let's keep all things PixInsight. I was just trying to validate/investigate the green cast, that's all. But yes, I would like to stick to PixInsight; broke my own rule I advocated above.


Yes, the PixInsight integration.fit file I uploaded was created solely in PixInsight using my CR2 files; all that was done to get to integration.fit was:

1. Batch DeBayer
2. Star Alignment
3. Image Integration


Will see if I could upload all CR2 for all us to try integrating and/or to reproduce the same scenarios. Thx


Post Extras: Print Post   Remind Me!   Notify Moderator  
Peter in Reno
Postmaster
*****

Reged: 07/15/08

Loc: Reno, NV
Re: Learning PixInsight new [Re: mmalik]
      #5748274 - 03/21/13 06:46 PM

If there's too much nebulosity, galaxy or IFN all over the image, I suggest you try ABE instead of DBE. ABE does a wonderful job to automatically build background model. It's very difficult to pick DBE points if you can't find a good background.

Peter


Post Extras: Print Post   Remind Me!   Notify Moderator  
mmalik
Post Laureate
*****

Reged: 01/13/12

Loc: USA
Re: Learning PixInsight new [Re: Peter in Reno]
      #5748394 - 03/21/13 07:44 PM

OK, let's all start over and together; I have uploaded 11 CR2 here....
Note: CR2s are IN-Camera noise reduced so no calibration is needed.

Process I would like to follow is what Ajay laid out:

1. Stack (calibrate+integrate) -> crop boundaries
2. Gradient removal and color correction (DBE, Color calibration, SCNR)
3. Noise removal via Atrous Wavelets
---
4. Stretch - Histogram transfer (levels)
5. HDR wavelets to increase dynamic range
6. LHE Local contrast enhancement
7. ACDNR, Nose reduction
8. Saturation Boost via curves

Note: The dashed line separates the first three steps you do on linear data.


What I would like is we move forward with one step/task at a time and also share some granular details how it was done. I am completely new to PixInsight interface, so if you tell someone to do something, please also explain how to actually do it in somewhat follow-able detail and/or share screen clips.


Let's do #1 which consists of following granular steps; at the end of it let's confirm/reproduce the green cast problem being discussed and move forward from there.


-Batch DeBayer (Script menu-Batch Processing, take defaults)
-Star Alignment (Process menu-All Processes, take defaults, define a reference image)
-Image Integration (Process menu-All Processes, take defaults)


Version I have is 1.8 RC4. Thx


Post Extras: Print Post   Remind Me!   Notify Moderator  
jsines
sage
*****

Reged: 09/06/11

Loc: Berkley. Michigan
Re: Learning PixInsight new [Re: mmalik]
      #5748895 - 03/22/13 12:54 AM Attachment (16 downloads)

I like your plan. Let's go 1 step at a time on this.

1. Stack (calibrate and integrate). Let's stop there. You've done the Debayering, Star Alignment, and now you're on the Image Integration. But you need to set the parameters of Image Integration so that you're maximizing the SNR increase. How do you do that?

First, you need a baseline to see what is basically the maximum SNR increase you can get out of it. So you set it to No Rejection and integrate. This gives you a stack of pictures without any rejected pixels. I do this with my images and I see the hot pixels trailing across the screen because my alignment isn't the best right now, or I see a satellite trailing through the middle of my image. I want to get rid of these.

I took your 11 CR2 files and debayered and aligned them. Then I set the Image Integration to No Rejection. This is also where you can set your reference frame, the picture everything else aligns to. I usually use Blink to look at them and pick the one with the best star, but for this I picked one at random, the middle.

Run it, then hover over "Process Control" on the left, and you'll see the history. At the bottom, you see the average SNR increases for each channel. For No Rejection, I got Red 3.097, Green 2.7100, and Blue 2.5144. You want to get as close to these numbers as possible, so you're rejection as few pixels as possible but keeping the SNR high as possible.

Open Image Integration and go to Pixel Rejection (1). Hover over the words "Rejection Algorithm" and it'll explain each one of the options. That's one plus to PI - hovering over things will bring up the explanation of what they do. You've got several options - Harry's video suggests Windsorized Sigma Clipping is best for large stacks, and I agree although you need to adjust the low and high clipping to suit your needs. I ran Windsorized Sigma Clipping (low 4.8, high 3.0) and got Red 2.9215, Green 2.5194, and Blue 2.3299. I should see if I can get closer to the No Rejection SNR levels.

I ran linear fit clipping (low 5, high 2.5) and got Red 2.9906, Green 2.6129, and Blue 2.4174. Better. I'm going to stick with this integrated file.

Next - Dynamic Crop - I cropped out the edges of the picture to remove any possible inconsistent stacking and the black edges.

Next - Uncheck aligned channels on STF and do auto-STF. If I leave it checked and have the channels aligned, I do see the green overcast. I never leave that checked. It's the first thing I do when I open STF. I unchecked it to un-align the channels and did auto-STF. The reason I don't have a green overcast to it now is because of the un-aligned channels in the auto-STF, and the black level is moved higher to compensate. see photo. The integrated file at the top left is the No Rejection stack that I can compare. I usually have stuff that gets removed this way (hot pixel trails, satellites, etc.)

I'm going to bed, I have to work in the AM, but I'll continue tomorrow on this or continue wherever we are at.


Post Extras: Print Post   Remind Me!   Notify Moderator  
mmalik
Post Laureate
*****

Reged: 01/13/12

Loc: USA
Re: Learning PixInsight new [Re: jsines]
      #5749004 - 03/22/13 02:56 AM Attachment (6 downloads)

Quote:

Uncheck aligned channels on STF and do auto-STF. If I leave it checked and have the channels aligned, I do see the green overcast. I never leave that checked. It's the first thing I do when I open STF. I unchecked it to un-align the channels and did auto-STF. The reason I don't have a green overcast to it now is because of the un-aligned channels in the auto-STF, and the black level is moved higher to compensate.




Thanks jsines; that was it!


Folks, jsines has found the root cause of the green cast and it was 'Link RGB Channels' setting STF (Screen Transfer Function).


To sync up with you and everyone else, I did following:

-Batch DeBayer (defaults)
-Star Alignment (defaults)
-Image Integration (Linear Fit Clipping)
-Unlinked RGB Channels in STF
-Auto stretched in STF to verify image (no green cast, etc.)
-Reset STF
-Saved the new file as integration.fit (have also uploaded the same here...)


Once everyone catches up to this point we all can move onto #2 (Gradient removal and color correction (DBE, Color calibration, SCNR). Even though DBE might not be well advised for this particular image, let's do that any way for learning sake. I'll look forward to some granular instructions for #2 items, whoever may want to volunteer. Regards


Post Extras: Print Post   Remind Me!   Notify Moderator  
terry59
Post Laureate
*****

Reged: 07/18/11

Loc: Colorado, USA
Re: Learning PixInsight new [Re: mmalik]
      #5749161 - 03/22/13 07:51 AM

Quote:



Folks, jsines has found the root cause of the green cast and it was 'Link RGB Channels' setting STF (Screen Transfer Function).





You were already told that


Post Extras: Print Post   Remind Me!   Notify Moderator  
pfile
Post Laureate


Reged: 06/14/09

Re: Learning PixInsight new [Re: terry59]
      #5749466 - 03/22/13 11:09 AM

this thread is hilarious.

Post Extras: Print Post   Remind Me!   Notify Moderator  
mmalik
Post Laureate
*****

Reged: 01/13/12

Loc: USA
Re: Learning PixInsight new [Re: pfile]
      #5749944 - 03/22/13 03:23 PM Attachment (15 downloads)

Here is DBE sample I tried, along with settings; suggestions welcome.

(Process Menu-<All Processes>-Dynamic Background Extraction)


Post Extras: Print Post   Remind Me!   Notify Moderator  
mmalik
Post Laureate
*****

Reged: 01/13/12

Loc: USA
Re: Learning PixInsight new [Re: mmalik]
      #5749950 - 03/22/13 03:25 PM Attachment (12 downloads)

For comparison, I also tried ABE with these settings:

(Process Menu-<All Processes>-Automatic Background Extraction)


Post Extras: Print Post   Remind Me!   Notify Moderator  
harry page 1
super member


Reged: 07/25/09

Re: Learning PixInsight new [Re: mmalik]
      #5749963 - 03/22/13 03:28 PM

Hi
Put at least one sample in each corner and lower the tollerance as low as possible without turning the samples red.
DBE used with care will work well with this image
and don't forget to relink the channels ( on stf) after running dbe to get a real view of the data

Harry


Post Extras: Print Post   Remind Me!   Notify Moderator  
mmalik
Post Laureate
*****

Reged: 01/13/12

Loc: USA
Re: Learning PixInsight new [Re: mmalik]
      #5749969 - 03/22/13 03:30 PM Attachment (18 downloads)

DBE and ABE comparison side by side; DBE on top (with greenish tint), ABE at the bottom (no greenish tint).

Post Extras: Print Post   Remind Me!   Notify Moderator  
mmalik
Post Laureate
*****

Reged: 01/13/12

Loc: USA
Re: Learning PixInsight new [Re: harry page 1]
      #5750038 - 03/22/13 04:02 PM Attachment (5 downloads)

Quote:

Put at least one sample in each corner and lower the tollerance as low as possible without turning the samples red.
DBE used with care will work well with this image




Thanks Harry; below are DBE corner samples and low tolerance I re-tired; results in the next post.


Post Extras: Print Post   Remind Me!   Notify Moderator  
mmalik
Post Laureate
*****

Reged: 01/13/12

Loc: USA
Re: Learning PixInsight [Re: harry page 1]
      #5750039 - 03/22/13 04:02 PM Attachment (14 downloads)

Quote:

don't forget to relink the channels ( on stf) after running dbe to get a real view of the data




Here is DBE output (integration_DBE.fit) with channels linked. (Note: DBE settings used are in the previous post)


Post Extras: Print Post   Remind Me!   Notify Moderator  
harry page 1
super member


Reged: 07/25/09

Re: Learning PixInsight [Re: mmalik]
      #5750080 - 03/22/13 04:17 PM

Hi
Maybe you might try a sample in the middle to remove the slight tint or just use scnr
Harry


Post Extras: Print Post   Remind Me!   Notify Moderator  
mmalik
Post Laureate
*****

Reged: 01/13/12

Loc: USA
Re: Learning PixInsight [Re: harry page 1]
      #5750108 - 03/22/13 04:28 PM

Quote:

Maybe you might try a sample in the middle to remove the slight tint or just use scnr





Sounds good Harry; I might wait to see what SCNR can do. Not too concerned with the tint as HLVG kind of tools can help as well, if you agree?

Continuing on with Ajay's list...
2. Gradient removal and color correction (DBE, Color calibration, SCNR)


What is the next logical step (after DBE); color calibration or SCNR? Does it matter which one is done before the other?

By the way, I did forget to crop before I did DBE. Might try again after cropping. Thx


Post Extras: Print Post   Remind Me!   Notify Moderator  
Peter in Reno
Postmaster
*****

Reged: 07/15/08

Loc: Reno, NV
Re: Learning PixInsight [Re: mmalik]
      #5750126 - 03/22/13 04:36 PM

I use Background Neutralization followed by Color Calibration after DBE/ABE. Sometimes after BN, CC may not make much difference and that's okay. It looks like your ABE did a better job than DBE, so why not use ABE instead?

I typically use SCNR after Histogram Transformation. I've never tried SCNR while data is still linear.

Yes, it's important to crop after Image Integration and before any kind of processing including DBE/ABE.

Peter


Post Extras: Print Post   Remind Me!   Notify Moderator  
jsines
sage
*****

Reged: 09/06/11

Loc: Berkley. Michigan
Re: Learning PixInsight [Re: harry page 1]
      #5750394 - 03/22/13 06:59 PM

As a matter of preference, I always have "Discard background model" unchecked on both DBE and ABE, because I want to see what it is removing. The background model will be a linear image, but if you do an unaligned auto-STF on it, you can see what you're subtracting. It should look like a gradient. I think if it looks like a bunch of mismatched, circular colors, it wasn't done right, but someone can correct me if I'm wrong.

Quote:

Hi
Maybe you might try a sample in the middle to remove the slight tint or just use scnr
Harry




I would do this, but I'd do it after stretching, which is what I think Harry is suggesting. I always do SCNR after stretching and it completely removes any green tint for me.

I may continue processing your image tonight, but it's also looking like it is my first clear night in about 2 weeks. I have also received a new Astronomik CLS, so I may be outside all night.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Pages: 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | (show all)


Extra information
9 registered and 20 anonymous users are browsing this forum.

Moderator:  Dave M, fishonkevin, WOBentley, tecmage 

Print Thread

Forum Permissions
      You cannot start new topics
      You cannot reply to topics
      HTML is disabled
      UBBCode is enabled


Thread views: 10452

Jump to

CN Forums Home


Cloudy Nights LLC
Cloudy Nights Sponsor: Astronomics