43 replies to this topic

[sharkmelley]

Adam Block has a new YouTube video and, according to him, SPCC is the most accurate color calibration to date.

I would express it differently.  SPCC is the most accurate white balancing tool to date.  White balancing is a necessary but not sufficient condition for displaying accurate colour.

[freestar8n]

OK so I have looked at the Adam Block videos on SPCC and I will be trying it soon - but I'm not crazy about using terms like "accurate" or "true".  I guess the main thing is - if that is what's important to people - are you not going to adjust the colors at all after it finishes?  In other words - if you are going after an aesthetic final result - won't you be prone to alter the colors for max impact in final processing?  Any changes to saturation would lose accuracy.  So this isn't so much a question about the tool itself - but how people intend to use it - and if "accuracy" is their main concern.  If it is - you better not touch those dials.

 

A second point is that there is always a disconnect between the filter passbands and how the final image is presented.  I use Sloan filters for R, G, B - and even though I haven't tried SPCC yet - I believe it does support Sloan filters and would make an "accurate" view of a galaxy.  But there is no getting around that the Ha emission will show as green rather than red - because the Ha lands in the r' filter, which maps to green.  This is an extreme case - but in general the passbands of the filters are very different, and the calibration process is just doing a sensible integration over the passbands with appropriate weighting.  But that only achieves so much in terms of "accuracy"

 

One reason I have avoided PI over the years is that its usage was prone to making highly processed images, and galaxies were consistently shown with exaggerated colors that were disconnected from reality - such as bright purple in the outer stars.  So if people use this tool and do stick with the colors - and galaxies to start to show more subtle color variations - I think that would be great.  But the images are likely going to be a lot more subtle and less jazzy.

 

Much of what is being touted here is based on principles I have been promoting for 7 years, and before PCC was implemented in 2017.  I think this is the first example I described publicly: 

 

https://www.cloudyni...ibrated-colors/

 

It was based on using APASS stars as references, integrating over passbands, and showing plots of the resulting linear fits.  But I described it as "calibrated" colors rather than accurate.  The colors of galaxies with Sloan filters mapped this way don't look right - but they are data driven - and only a stretch was applied to compress the dynamic range.  For a globular cluster I show that calibrated colors help reveal the underlying astrophysics that normally gets processed away - the example here being the horizontal branch.  It also allows good H-R diagrams.  The goal for me has been data-driven, calibrated colors that were not chosen or altered by the imager.

 

If you look at my astrobin, most of the RGB images are calibrated in this way, so they can all be seen in the same color context.  This is one thing Adam Block promotes - and I fully support it.  But it requires you not to mess with the colors after calibrating.

 

The APASS star measurements are limited in accuracy so they aren't ideal for this procedure and the GAIA data are clearly superior.  But I don't expect a major change in colors due to this improved accuracy and spectral detail because a lot of the variation averages out - and the elephant in the room is the disconnect of the filter passbands with how the colors get displayed.

 

I wouldn't make a big deal about white balancing based on average galaxies - particularly when imaging a nebula.  All you can really do is calibrate the image and say what you used for white balancing.  But "accuracy" is a little strange to use there.

 

Finally - how will people use this tool when imaging the Fireworks galaxy?  Will it rebalance the whole scene to remove reddening from the galaxy - while making the foreground stars overall bluer?  Or will the whole galaxy be fairly orange?  I'll be interested to see how people show that one - and if they adjust colors.

 

Frank

[italic]

Finally - how will people use this tool when imaging the Fireworks galaxy? Will it rebalance the whole scene to remove reddening from the galaxy - while making the foreground stars overall bluer? Or will the whole galaxy be fairly orange? I'll be interested to see how people show that one - and if they adjust colors.

Frank

You're in luck https://www.cloudyni.../#entry12334543

[freestar8n]

You're in luck https://www.cloudyni.../#entry12334543

That's certainly timely - it wasn't long after my posting.  If you look at astrobin images of that galaxy you'll see the colors are all over the map.  So things like "true" or "accurate" don't really apply here.  But "calibrated" applies - along with a corresponding de-reddening operation.

 

And "accurate" definitely doesn't apply if people end up altering the colors anyway.  I see people comparing different images as calibrated by different routines and asking which they prefer - but if "accuracy" is the goal, what people prefer shouldn't matter.

 

If accuracy isn't the goal - then routines like these just give you a starting point from which you enhance the colors.  That's fine - but it does miss the point of all the line plots and precise white point claims.

 

Frank

[Loopster]

I just wish image solving was baked into SPCC. Clearly they're trying to get everyone to use WBPP because astrometric solution is part of the script, but given that not everyone uses WBPP for various reasons they shouldn't have taken it out of PCC and SPCC. Seems like a wrong choice.

[james7ca]

I just wish image solving was baked into SPCC. Clearly they're trying to get everyone to use WBPP because astrometric solution is part of the script, but given that not everyone uses WBPP for various reasons they shouldn't have taken it out of PCC and SPCC. Seems like a wrong choice.

Shouldn't have taken what out?

 

You can do astrometric solutions on any image using PI's ImageSolver script. In fact, I suspect that is what is used by WBPP.

[ngc1535]

That's certainly timely - it wasn't long after my posting.  If you look at astrobin images of that galaxy you'll see the colors are all over the map.  So things like "true" or "accurate" don't really apply here.  But "calibrated" applies - along with a corresponding de-reddening operation.

 

And "accurate" definitely doesn't apply if people end up altering the colors anyway.  I see people comparing different images as calibrated by different routines and asking which they prefer - but if "accuracy" is the goal, what people prefer shouldn't matter.

 

If accuracy isn't the goal - then routines like these just give you a starting point from which you enhance the colors.  That's fine - but it does miss the point of all the line plots and precise white point claims.

 

Frank

I thought I was pretty careful in general with my usage of words. 

There is a degree of accuracy/precision in the calculation of ratios and fluxes. This is what I was referring to. I didn't relisten to everything- I believe I used the word "truth" once. I said the GAIA data was a kind of truth/reference. It is fair in the sense it is the best set of measurements currently available of its type. 

 

It could be i was more lax with language at some point- but it seems a little petty given my genuine attempt to explain things. 

 

What happens after the image is color calibrated is beyond the scope of the video. Of course you are correct- no one is arguing this. PixInsight developers have a "documentary philosophy." That argument is one for them. Everyone is aware "non-documentary" image processing is done. I have not found that PixInsight software users are generating more crazy images- Photoshop still wins this little award by several orders of magnitude  (mostly because it has been around longer). 

 

Regarding the white reference- the accuracy/precision again I refer to is with respect to the specificity of the definition and the calculation of the relative flux ratios given filters and sensors. 

This is why I belabored showing all of the curves and multiplying them together. 

 

Regarding the NGC 6946 or IC342- I suspect there are enough stars of various temperatures that are not reddened or only reddened somewhat. The linear fits will probably result in the expected correction for the system and end up showing a very reddened galaxy given the average spiral galaxy WB. 

A real challenge would be NGC 6369. I do not believe there are many stars in the foreground of thick dust for several degrees here. 

Maybe it makes sense to take some short exposures of stars in a nearby (less dusty) patch of sky?

 

 

-adam

[Scott Badger]

Near the bottom of the lengthy documentation for SPCC, there's a section on "The meaning of Color" defined as, "the artwork of astrophotography is based on the unification of the inner pictorial content of the image with the documentary content of the scene. This defines the aesthetics of astrophotography, which lies at the point where beauty meets authenticity."  (quoted from a book by Vincent Perez). I don't generally disagree with the list of color breaking practices following that statement, but it's interesting that while manipulating any color channel individually is bad, there's no mention of overall saturation adjustment. Maybe because some processing tools collaterally effect saturation, so some (re)adjustment to bring the image back to 'true' is necessary? Also interesting that several examples are given of how the hue of a target can be dramatically effected by (arbitrarily?) changing the white reference. Also note the last bad practice listed, using SCNR to remove green. If we accept the premise that green is not a 'natural' color of stars or most DSO's (prevailing belief and SCNR is part of most tutorials), then isn't removing it from our images when it appears making the image more true to reality? And if it breaks the true to your data rule, is overall color calibration, then, fundamentally different? The line between true-to-reality and true-to-data gets a bit blurry, for me at least....

 

Anyhow, while I may not always completely understand the PI team's principles of true/accurate/aesthetic image processing, and sometimes, probably, not fully comply, I do appreciate their philosophical ambition.

 

Cheers,
Scott

Edited by Scott Badger, 07 December 2022 - 08:04 AM.

[Jose_Salcedo]

Dan:

 

I have also been testing out the new SPCC tool on M33 data.  Choosing white reference G2V also results in my image being a bit redder than expected.  I'll go back and use the spiral galaxy reference as that seems to be what PI is recommending.

 

Ben

Ben,

 

I second your opinion on using the tool and looking at M33 color calibration.

[Loopster]

Shouldn't have taken what out?

 

You can do astrometric solutions on any image using PI's ImageSolver script. In fact, I suspect that is what is used by WBPP.

Taken it out of the script. Prior to SPCC release, PCC would do all-in-one: solve the image and then apply colour correction. Release of SPCC not only didn't include the solve, but also took it out from PCC.

 

There was no need to do that. Why go to 2 different tools if everyone can be done in one.

[pfile]

Taken it out of the script. Prior to SPCC release, PCC would do all-in-one: solve the image and then apply colour correction. Release of SPCC not only didn't include the solve, but also took it out from PCC.

 

There was no need to do that. Why go to 2 different tools if everyone can be done in one.

juan answered this in the PI forum. he strongly believes in object oriented paradigms, meaning that one should chain together existing tools rather than merging them together. in other words, going to 2 different tools is the superior solution in general. he now feels that putting image solving into PCC was a mistake, because ImageSolver already existed. so he took it out while he was working on SPCC.

 

honestly it was 2 places where he had to maintain the code, so it makes sense from a developer standpoint as well.

 

for my part i never used the PCC solver as i had better luck solving my images with ImageSolver, so i didn't really notice this change.

[idclimber]

Taken it out of the script. Prior to SPCC release, PCC would do all-in-one: solve the image and then apply colour correction. Release of SPCC not only didn't include the solve, but also took it out from PCC.

 

There was no need to do that. Why go to 2 different tools if everyone can be done in one.

I suggest you write a script that combines both of these processes. 

[freestar8n]

I thought I was pretty careful in general with my usage of words. 

There is a degree of accuracy/precision in the calculation of ratios and fluxes. This is what I was referring to. I didn't relisten to everything- I believe I used the word "truth" once. I said the GAIA data was a kind of truth/reference. It is fair in the sense it is the best set of measurements currently available of its type. 

 

It could be i was more lax with language at some point- but it seems a little petty given my genuine attempt to explain things. 

 

What happens after the image is color calibrated is beyond the scope of the video. Of course you are correct- no one is arguing this. PixInsight developers have a "documentary philosophy." That argument is one for them. Everyone is aware "non-documentary" image processing is done. I have not found that PixInsight software users are generating more crazy images- Photoshop still wins this little award by several orders of magnitude  (mostly because it has been around longer). 

 

Regarding the white reference- the accuracy/precision again I refer to is with respect to the specificity of the definition and the calculation of the relative flux ratios given filters and sensors. 

This is why I belabored showing all of the curves and multiplying them together. 

 

Regarding the NGC 6946 or IC342- I suspect there are enough stars of various temperatures that are not reddened or only reddened somewhat. The linear fits will probably result in the expected correction for the system and end up showing a very reddened galaxy given the average spiral galaxy WB. 

A real challenge would be NGC 6369. I do not believe there are many stars in the foreground of thick dust for several degrees here. 

Maybe it makes sense to take some short exposures of stars in a nearby (less dusty) patch of sky?

 

 

-adam

Hi Adam - thanks for commenting here.  And as a new learner of PI I certainly appreciate your videos on youtube and other write ups.

 

Regarding accuracy I was mainly referring to your SPCC part 2 video.  I guess it is only on the thumbnail for the video and not in the video, but it says, "Your colors are not accurate (unless you are using PixInsight)"  I realize this meant somewhat for attention grabbing - but I did want to clarify that I have been using APASS calibrated colors with Sloan filters for over 7 years now and have been touting its merits in terms of things like consistent representation of objects here and on astrobin.  But my results look odd precisely because they are calibrated and use Sloan filters - without later altering the colors (unless I boost saturation slightly and indicate it in the description).  The filters used will always be different from what is used to display the image, so there are limits to what can be gained by improving the calibration with more detailed spectral info.

 

At the same time, unless people using SPCC resist altering the colors after the calibration finishes - the whole point of the calibration - and consistency of representation - is lost.  I would not want people using the tool, then masking and altering colors here and there, and saying "these colors are great because they are accurate" or something.  As always - I'm fine with people doing whatever they want to an image - but if they describe what they did or didn't do I hope it captures everything.

 

I think most PI users are out to make their images as compelling as possible, so they would not hold back additionally tweaking the colors.  So the calibration really just serves to set a starting point for aesthetic color tweaks.  Which, again, is fine if people don't value the data-driven aspect of the colors.

 

But I really appreciate your point about consistent representation of objects - in one of your videos.  The coloring of galaxies is usually based on conventions that have been adopted, including bright blue or violet outer stars.  The results you show are much more subtle - so if SPCC does help drive more subtle and consistent colors in objects - that would be great.  It's fine if people want to express themselves by artistically altering the colors in ways they find pleasing - but some of us prefer to have the objects speak for themselves, and in a consistent manner that allows comparison.  It's not so much a rigid "documentary" goal - just one of seeing the object itself and not a personal interpretation of it.

 

Thanks again,
Frank

[Loopster]

juan answered this in the PI forum. he strongly believes in object oriented paradigms, meaning that one should chain together existing tools rather than merging them together. in other words, going to 2 different tools is the superior solution in general. he now feels that putting image solving into PCC was a mistake, because ImageSolver already existed. so he took it out while he was working on SPCC.

 

honestly it was 2 places where he had to maintain the code, so it makes sense from a developer standpoint as well.

 

for my part i never used the PCC solver as i had better luck solving my images with ImageSolver, so i didn't really notice this change.

That makes sense, and i can appreciate that explanation. I always thought of PCC as a chained script that combined two functions, as opposed to a standalone function that also integrated redundant solving capabilities, but that's clearly not how it was intended and i understand that now.

 

I suggest you write a script that combines both of these processes. 

Unfortunately i'm not skilled enough to script a solution, so i'll wait patiently for smarter people than me to do that.

All i'm trying to convey is that from a user experience perspective, having an all in one tool for both solving and colour calibrating an image was a superior solution to splitting them up, especially for folks who are trying to optimize and streamline their processing workflows.

[ngc1535]

Hi Adam - thanks for commenting here.  And as a new learner of PI I certainly appreciate your videos on youtube and other write ups.

 

Regarding accuracy I was mainly referring to your SPCC part 2 video.  I guess it is only on the thumbnail for the video and not in the video, but it says, "Your colors are not accurate (unless you are using PixInsight)"  I realize this meant somewhat for attention grabbing -

Thanks again,
Frank

Thank you Frank.

Indeed.. yes, the thumbnail did apparently do its job. Clickbaity .... Yes, the thumbnail does communicate what you mention- but its intent was different.

 

Concerning the colors- yes I agree with respect to achieving some fiducial only to later modify it. However, the initial result is really a wonderful place to begin from whether additional adjustments are made or not. I can imagine that having this kind of reproducible outcome will actually constrain some color modification since there is good confidence in what we initially see is less prone to variables we might not know to take into account (such as transforms and previous uncertainties in other catalogs...etc).

 

-adam

[arbit]

That makes sense, and i can appreciate that explanation. I always thought of PCC as a chained script that combined two functions, as opposed to a standalone function that also integrated redundant solving capabilities, but that's clearly not how it was intended and i understand that now.

Unfortunately i'm not skilled enough to script a solution, so i'll wait patiently for smarter people than me to do that.
All i'm trying to convey is that from a user experience perspective, having an all in one tool for both solving and colour calibrating an image was a superior solution to splitting them up, especially for folks who are trying to optimize and streamline their processing workflows.

Personally I see modular code and modular UI as two different things.

If input A is essential for process B, then personally, I'd expect a good UI to keep the two together.

PCC or SPCC could just take the inputs and pass them onto the IS script, avoiding any redundancy in code.

WBPP is a great example - an integrated UI but modular execution.

Anyway, finally it's the developer's call.

Sent from my SM-S908E using Tapatalk

[WadeH237]

WBPP is a great example - an integrated UI but modular execution.

Agreed that WBPP is a great example.  Specifically, WBPP is a script that automates the operation of a bunch of processes.

 

SPCC and PCC are not scripts.  They are processes.  Personally, I agree with Juan's decision to keep processes focused on their specific purpose.  In particular, there are multiple processes that depend on an astrometric solution, and I believe that we'll see more in the future.  It would be nonsensical for some of them to invoke image solving and some not to.  For consistency, all processes that require a solution would need to invoke it.  Otherwise, you'd end up with a hodge podge of different behaviors.  To me, it's better to make creating the solution an isolated step, which is what they've done here.

 

I suspect that if PCC hadn't included it in the first place, we wouldn't be missing it.  In other words, it's the change that folks are having an issue with, particularly since it breaks older workflows.  If it bothers folks enough, someone could write a script to automate color calibration that - like WBPP - presents a single UI that automates multiple steps.

[arbit]

I don't know.

From the user perspective, processes and scripts are largely just a categorization- both have a bunch of input fields you fill in to get whatever you want. There's no hard and fast distinction. You just move between the two as needed.

Even in color calibration, you first need to use a script (Image solver) then a process.

WBPP is so popular (imo) not just because it automates, but also because it hides a lot of the complexity in the underlying processes and gives access in one spot to all the inputs needed across the multiple processes and scripts it calls.

It's true that a lot of the reactions are probably to the change per se, but that doesn't necessarily mean the initial design was flawed.

Having said all that, these are individual preferences, and the decision is upto the developer. Luckiky, this is a relatively minor inconvenience :-)

Sent from my SM-S908E using Tapatalk

[Dan_I]

   Hi,

 

I did some more tests with the SPCC tool. Sometimes there's a color cast after color calibration. I get more neutral results if I do first a standard color calibration (neither PCC nor SPCC), and then SPCC.