582 replies to this topic

[galacticinsomnia]

I tried another all-Siril run without any calibration frames. The problem is in the demosaicing implementation in Siril. The color it assigns seems to be luminosity-scaled. This results in less color noise in the background, for sure. But the result is highly inaccurate. So I can no longer in good conscience recommend Siril for processing raws from an unmodded DSLR.

 

Below is what I was able to pull from a single 30 sec sub at 50% scale with

 

1. RawTherapee demosaicing and calibration
2. Siril color calibration and stretching
3. RawTherapee luminosity-scaled noise control

 

Final tweaks were in Photoshop; banding control was with Astronomy Tools v1.6 (which is more effective than Siril's strongest). Even the Running Chicken is visible in such scant data.

 

Preview (click for full size @ 50% sensor scale):

Uncropped version in the Google

 

Can someone with DSS give the data a go and post the stack? I'd love to compare it to RawTherapee…

 

BQ

It may Seem that way..  But it does just seem that way.  It is not being scaled by luminosity.
You can't draw those kinds of conclusions in one test aquisition.  You need more data.  I have two years plus using Siril almost exclusively for stacking and I have paid for software, and I donate to Siril regularly, I don't do these things all willy nilly.  I do them to support the Devs, and Support the evolution, which has been really phenominal in the past year.

Bandwidth removal.. Yummy...  Works a treat. 

Clear Skies !!

[BQ Octantis]

I tink I just showed that it can reveal color, maybe not as much, but like I said before for 40 minutes of aquisition time using 30 second subs, on a wide field target, with unmodified camera, you are getting an honest display of color.

I can easily run your stack in DSS, Siril, APP, etc.  The results in APP are no better, and I'm sure in Pi they are no better.

Yeah, but you killed my Chicken. And I showed that there's some Chicken even in 30 seconds of f/3.5 data. So something is amiss.

 

Can you run the subs through DSS? That bloke I referenced in Post #1 showed a clear difference between DSS and Siril, and I'd love to see what DSS spits out.

 

BQ

[galacticinsomnia]

Yeah, but you killed my Chicken. And I showed that there's some Chicken even in 30 seconds of f/3.5 data. So something is amiss.

 

Can you run the subs through DSS? That bloke I referenced in Post #1 showed a clear difference between DSS and Siril, and I'd love to see what DSS spits out.

 

BQ

Sure, issue is, that no matter what my images come out different from DSS, so I am not sure exactly you are looking for, but I can stack both, but I won't use calibration or offsets.  I will also stack without the background removal and upload 16 bit tiff ?  or 32bit Fits ?

Clear Skies !!

[BQ Octantis]

Just to see how it's different. Preferably 32bit FITS, please.

[galacticinsomnia]

Okay.. Here is both..

Siril Fit.
https://drive.google...iew?usp=sharing

DSS Autosave fts
https://drive.google...iew?usp=sharing

 

Just Did a quick color correction background neutralization and auto stretch of both of these in Siril and they look almost exactly

the same.

These posted links however are just straight up stack.

Clear Skies !!

Edited by galacticinsomnia, 17 September 2022 - 04:02 PM.

[vidrazor]

Well this is what I got messing around. Here is just lights stacked in Siril, followed by post in a combo of Starnet and Photoshop...

Attached Thumbnails

• 

[vidrazor]

...then I tried DSS with all the calibration frames with some GHT and Asinh in Siril...

Attached Thumbnails

• 

[vidrazor]

...and finally out of curiosity I tried stacking in Sequator, followed by a little Photoshop.

 

So, not exactly what you hoped for on any of these LOL.

Attached Thumbnails

• 

[galacticinsomnia]

...and finally out of curiosity I tried stacking in Sequator, followed by a little Photoshop.

 

So, not exactly what you hoped for on any of these LOL.

From a Stacking standpoint, most do the same work, with the exception of Siril which stacks everything for me in 2min or less.  

Post work wise, the Siril gets me a lot closer to where I want to be before going into Affinity.

Clear Skies !!

[BQ Octantis]

...and finally out of curiosity I tried stacking in Sequator, followed by a little Photoshop.

 

So, not exactly what you hoped for on any of these LOL.

Not at all—thanks for giving the data a go! This last one is what I would expect—you even got some strong Ha from the whole Statue of Liberty Nebula region.

[sharkmelley]

It should be noted that the workflow using RawTherapee is very different to what is done by Siril, DSS, PixInsight etc.  RawTherapee is designed for normal photography using a DSLR/Mirrorless camera.  So RawTherapee performs all the necessary processing steps to display colours correctly in the chosen target colour space (e.g. sRGB, AdobeRGB etc.)  These processing steps include the colour correction matrix to transform data into the colour space and then to apply the gamma tone curve relevant for that colour space.  Siril, DSS and PixInsight do not perform these necessary steps and therefore their colours are muted.

 

It is certainly possible to take the (linear) raw stacked output from Siril, DSS and PixInsight and apply the necessary colour space transformations so the colours match RawTherapee but it is not straightforward.

 

Mark

[BQ Octantis]

From a Stacking standpoint, most do the same work, with the exception of Siril which stacks everything for me in 2min or less.  

Post work wise, the Siril gets me a lot closer to where I want to be before going into Affinity.

Stacking's not my issue—it's demosaicing. The difference between Siril and RawTherapee is obvious in the autostretch prior to stacking. Siril also likes to sum both G's together to make a very green image—a real PITA for anything from 15mm or less (when it gives up on colorimetric calibration, presumably because of the star bloat).

[BQ Octantis]

It should be noted that the workflow using RawTherapee is very different to what is done by Siril, DSS, PixInsight etc.  RawTherapee is designed for normal photography using a DSLR/Mirrorless camera.  So RawTherapee performs all the necessary processing steps to display colours correctly in the chosen target colour space (e.g. sRGB, AdobeRGB etc.)  These processing steps include the colour correction matrix to transform data into the colour space and then to apply the gamma tone curve relevant for that colour space.  Siril, DSS and PixInsight do not perform these necessary steps and therefore their colours are muted.

 

It is certainly possible to take the (linear) raw stacked output from Siril, DSS and PixInsight and apply the necessary colour space transformations so the colours match RawTherapee but it is not straightforward.

 

Mark

 

Thanks for chiming in, Mark!

 

So you're saying there's no hope for color in any AP software from my unmodded 600D/T3i?

Edited by BQ Octantis, 17 September 2022 - 04:55 PM.

[BQ Octantis]

Which also begs…

 

What is reality?

 

Isn't my goal a picture colorful to my non-colorblind human eyeballs in either print or an sRGB monitor? With capture done in a time-efficient manner, given my finite existence?

Edited by BQ Octantis, 17 September 2022 - 05:46 PM.

[sharkmelley]

So you're saying there's no hope for color in any AP software from my unmodded 600D/T3i?

No, I'm simply saying that RawTherapee,  LightRoom etc. make it very easy to obtain good colour.   However, the absence of red emissions in the Siril result in your original post looks very strange to me and is probably caused by a number of factors.

 

 

Which also begs…

 

What is reality?

RawTherapee, LightRoom etc. are designed to give pretty good colour reproduction because that's what photographers want.  Of course, it will never be perfect but it's close enough to reality for most practical purposes.

 

Mark

[cissou8]

Hi,

 

got a message from @Bluemoon asking if I could drop by, so here I am Just to be very transparent, I'm more into the star detection/registration/stacking part of Siril but I'll give it a go.

 

Just a few points here and there I've noted throughout the thread that could do with some clarifications on what Siril does and does not:

- Siril does not use debayered flats because debayering should occur as the last step of pre-processing. While still undebayered, each light pixel is corrected by a matching dark pixel and flat pixel so that having them debayered beforehand makes no sense. It would just mean introducing more interpolations (=more noise) in the process.

- For the same reason (which is we are still in CFA state while making the division), Siril does not blur the masterflat. So I would advise you take a few flats and stack them. Whether you want to correct the flats with bias, darkflats or just 2048 is up to you. My stance on the matter is pretty obvious having written the article you've linked to

- Siril does not sum the G's. Nor average them. And that's not the reason for the image green cast. It's the sensitivity of the sensor which is more efficient in the green part of the spectrum. And the fact that we are then using the data raw without any balance correction.

- As you have rightly pointed out, Siril uses debayering code from RT (I think it's one of RT's devs who did the integration but I was not yet in the project back then). So debayering is done pretty much in the same way in both codes.

 

I would second what @sharkmelley said. There is probably some sort of of tweaking of the colors, first of all to apply the WB coefficients of the camera and then some further stuff being done to map them to some given colour space (and probably a bit of stretching while we're at it). As I'm not at all familiar with RT or regular daylight photography, could you kindly add a single RT-calibrated light to your drive so I could have a look at the histogram and compare with why I get from Siril?

I already dl'ed a few of your frames but both my Internet connection and my available disk space are giving me a hard time. I just stacked 5 of them. I can pull out some red provided I:

- remove the stars so that I'm not blowing them off

- heavily strech the red layer, but it brings out significant noise.

  

 

The fact that there's very little red even though there's plenty of Ha in the nebula really comes down to the fact that there is both less sensitivity in the red band of the spectrum and a filter on unmodded DSLRs. Don't know on the 600D but I used to shoot with a 700D and clearly, going to a camera was a game changer in the Ha part of the spectrum.

Now if you like the colors you get from RT calibration, I believe you have gotten yourself a perfectly functional workflow to make the most of your setup.

 

Clear skies,

 

Cecile

Edited by cissou8, 17 September 2022 - 06:23 PM.

[BlueMoon]

Ah, thanks folks for the info and insights and for having the time to clarify a bit Cecile. 

 

I'm still fairly new at this but the gist of it, as I understand, is that the issue comes down to 2 things primarily; an absence of red due to sensor sensitivity and filtering, and some programmatic conversion or mapping changes in the color space. Is this correct? So, to compensate one would probably go for an Ha mod to provide more red in the image data?

[BQ Octantis]

… could you kindly add a single RT-calibrated light to your drive so I could have a look at the histogram and compare with why I get from Siril?

 

The fact that there's very little red even though there's plenty of Ha in the nebula really comes down to the fact that there is both less sensitivity in the red band of the spectrum and a filter on unmodded cameras. Don't know on the 600D but I used to shoot with a 700D and clearly, going to a camera was a game changer in the Ha part of the spectrum.

Now if you like the colors you get from RT calibration, I believe you have gotten yourself a perfectly functional workflow to make the most of your setup.

Hi Cecile,

 

Thank you so much for taking the time to help investigate. I have indeed gotten eye-watering results from my unmodded 600D with my existing workflows (RawTherapee + Lynkeos + Photoshop + Starnet++), but for me it is Siril that has been a game changer—particularly for registration, stacking, and color calibration. This last one is ironic, given that my difficulty now is with color—but only when I use Siril for demosaicing instead of RawTherapee.

 

Per your request, I uploaded a set of 16 RawTherapee calibrated lights to a new folder called "RT_Calibrated". My fundamental question is, is an all-Siril workflow a red herring for my unmodded 600D data?

 

BQ

Edited by BQ Octantis, 17 September 2022 - 06:57 PM.

[galacticinsomnia]

It should be noted that the workflow using RawTherapee is very different to what is done by Siril, DSS, PixInsight etc.  RawTherapee is designed for normal photography using a DSLR/Mirrorless camera.  So RawTherapee performs all the necessary processing steps to display colours correctly in the chosen target colour space (e.g. sRGB, AdobeRGB etc.)  These processing steps include the colour correction matrix to transform data into the colour space and then to apply the gamma tone curve relevant for that colour space.  Siril, DSS and PixInsight do not perform these necessary steps and therefore their colours are muted.

 

It is certainly possible to take the (linear) raw stacked output from Siril, DSS and PixInsight and apply the necessary colour space transformations so the colours match RawTherapee but it is not straightforward.

 

Mark

I totally agree !  Thank you for articulating that. !!

 

Clear Skies !!

[Alen K]

...and finally out of curiosity I tried stacking in Sequator, followed by a little Photoshop.

 

So, not exactly what you hoped for on any of these LOL.

Interesting that Sequator got more color than the others. What would be your (or anyone’s) theory as to why? 

[sharkmelley]

Interesting that Sequator got more color than the others. What would be your (or anyone’s) theory as to why? 

I can easily answer that because I investigated it a while back.  Sequator is applying the proper colour space conversions, so essentially it behaves more like RawTherapee and LightRoom.  Under the hood, Sequator is using LibRaw to perform the demosaicing and also LibRaw's CCM (colour correction matrix) for the colour space transformations. I believe the CCM ultimately comes from Adobe, just like RawTherapee's CCM.

 

Mark

[BQ Octantis]

See, this is what's hurting my head.

 

If the RGB output of a stock DSLR requires a color correction matrix, isn't that a critical part of the process? And doesn't that mean the standard Siril color workflow isn't applicable to a stock DSLR?

 

You can clearly see the colorful Ha and OIII in Cecil's crack at my Carina. But he says he had to stretch the dickens out of the red. I have no qualms with manually manipulating histograms (I've done it for decades), but the whole point of heading down the all-Siril color path was the accuracy of all-raw combined with colorimetric calibration and background extraction. I thought those would handle the proper slopes between the RGB curves, but instead they render the output colorless.

 

Occam's razor sez, the correction matrix is necessary. Otherwise, I'm back to eyeballing histograms and outputs.

 

BQ

Edited by BQ Octantis, 18 September 2022 - 09:35 AM.

[BlueMoon]

 

but the whole point of heading down the all-Siril color path was the accuracy of all-raw combined with colorimetric calibration and background extraction. I thought those would handle the proper slopes between the RGB curves, but instead they render the output colorless.

Perhaps this thread will spark some changes to Siril to address the issue as needed. That's my hope.

[BQ Octantis]

Perhaps this thread will spark some changes to Siril to address the issue as needed. That's my hope.

It's either that or use RawTherapee for calibration and demosaicing. Given that both are open source (and they already trade code), maybe putting in the CCM option is straightforward enough to tackle…

[BlueMoon]

It's either that or use RawTherapee for calibration and demosaicing. Given that both are open source (and they already trade code), maybe putting in the CCM option is straightforward enough to tackle…

That would be nice to see. I'm in the "one program to rule them all" mode these days for image stacking/processing. I could learn RT but as it's already part of Siril, as you say perhaps working the CCM option in would be doable.