23 replies to this topic

[BQ Octantis]

Now I'm not one to rock the boat or anything…

 

and I'm definitely not one to challenge my elders, seniors, superiors, dogma, or the status quo…

 

but I seriously violated the 25% Histogram Rule the other night. I shot at 50% histogram!

 

 

And literally nothing happened. Or rather, I got very satisfactory results—at least for shooting for just 70 minutes through thin clouds in a Bortle 10 sky:

 

 

Nothing in the stack was anywhere close to clipped. In fact, my stack had 3:1 margin against clipping—or stated differently, I only used 33% of the available dynamic range:

 

 

So what am I missing here? I'm not seeing the technical merit of the rule.

 

I ask because I see other OPs limiting their exposures to the 25% Histogram Rule.

 

BQ

 

[jml79]

In your skies, the histogram is dominated by skyglow. Your histogram is measuring the light pollution and not the stars or target. If you tried a 50% histogram at a dark site you would have very long exposures and likely have a lot of stars blowing out. Like all rules of thumb, they are meant to cover something as broadly as possible and shooting in downtown New York is not the norm. It is possible that your signal to noise is so bad that you could see improvements by exposing as long as possible or as close to a typical exposure time under dark skies as your light pollution will allow. PS, not advice, I don't know. I am lucky enough to live in darker skies than that.

[Michael Covington]

The way I understand it, 25% is a recommended minimum rather than maximum.  You do lose dynamic range (have more of the brightest regions blown out as pure white) when you go substantially higher.  But I have shot at 50% histogram quite a bit in a light-polluted area.

[Michael Covington]

BTW, in that picture, none of the nebulae are all that bright, so you had no real risk of clipping them.

[galacticinsomnia]

I don't follow all the rules, because YMMV and your experiences may be different for one reason or another.
Generally speaking, keeping the top of the bell curve between 30 and 50 to work very well for me the only caveot being that there must be no clipping on the right.  
As was pointed out, setting your exposure for the Avg value of the sky, verses, exposing for your brightest star in the field, would be completely different.  Your histogram, depending on the camera, and exposure settings, and is it read from stright raw, a read off of a jpg, so knowing which type of histogram you are viewing, as well as the premise exposure setting was chosen, may give you a more rigid meaning behind the why it seems okay.  It is no suprise to see an image now work exposed in the manner you did.
 

[bobzeq25]

I'm guessing the histogram was being dominated by light pollution (what is Bortle 10, the scale only goes to 9?) reflecting off the thin clouds.  While light from the target was reduced by the clouds, hence, no clipping.   The (approximate) "rule" simply doesn't apply to that situation.

Edited by bobzeq25, 26 February 2023 - 12:54 AM.

[vidrazor]

The potential danger is washing out dim data. You signal is swamped by noise and "lost in the sauce". That's why you want to keep the subs on the 1/4 mark and shoot four billion subs in Bortle 9 skies.

 

I'm curious however if you applied the per-sub background extraction in Siril. I wonder how effectively that process can extract noise while somehow maintaining at least some of the signal. Also, were you using any narrowband filtration?

Edited by vidrazor, 26 February 2023 - 01:59 AM.

[sharkmelley]

The idea behind the 25% back-of-camera histogram guidance for DSLRs is that you will be "swamping" the camera's read noise.  If your exposures are too short then the final stack will contain a large noise contribution from the camera's read noise, which is clearly undesirable if it can be avoided.

 

It is important to emphasize that we talking about the back-of-camera histogram which is the histogram of the JPG that the camera would produce i.e. it is the histogram of non-linear data.  Don't use a histogram of linear data when using this guidance otherwise you will be unnecessarily over-exposed.

 

Also note that many modern DSLR and mirrorless cameras have much lower read noise than was previously the case and so the read noise can be swamped with the back-of-camera histogram further to the left.

 

Mark

[17.5Dob]

"The Rule of 25%" is the recommended minimum exposure length to swamp your read noise, and as Mark pointed out, is actually too high for any modern camera. I like to keep mine more towards 1/8 to 3/16. Way back when, when everyone was using an old Canon Rebel "The Rule" was always 1/3 histogram minimum.

Edited by 17.5Dob, 26 February 2023 - 04:38 AM.

[BQ Octantis]

Thanks to all for the thoughtful responses. I never know if I'm poking the bear or flogging a dead horse. Clearly, this horse isn't completely dead.

 

Jerry's over-exposure example clearly shows clipping of the highlights as a big no-no. But I discovered recently in processing daylight photos in Siril that many of my auto-exposed camera JPEGs that the histogram showed being overexposed (and the highlights are indeed clipped) have almost 2x margin in the raw version. So the histogram isn't being truthful about the raw. I was well aware that my on-camera histogram was for the JPEG (slope + gamma + tone curve). But I didn't appreciate just how much margin was left in the raw. I mean, sure, the highlights recovery in ACR and RT hinted at it. But Siril just gives the numbers as I showed above.

 

For my purposes, I have a standard capture workflow, akin to f/8 and be there: polar align and 30 second subs. I've kept to ISO800 for years because of the 25% histogram rule, but going to ISO1600 would buy me a quarter stop of improvement in read noise SNR. It would also put me much farther into "overexposed" territory for my current sky, at least per the back of the camera histogram. But if I've got the margin (for the sample I gave, I'd still have no clipping), what is the harm in putting my histogram hump further to the right? As Michael points out, most of the stuff I care about is in the hump—and I know how to recover clipped stars. Is my trade-off just potentially some clipped stars for more SNR? In Bortle 10 skies, every little bit of SNR helps—so what is the drawback if I put the histogram as far to the right without clipping the raw?

 

 

what is Bortle 10, the scale only goes to 9?

Really? Mine goes to 11:

 

[Source]

 

To me, Bortle 10 is just an irritatingly bright sky. And Bortle 0…well, you'll know Bortle 0 when you experience it.

 

On the actual Bortle scale as defined, I'm currently probably between Class 6 and Class 7. It's a major adjustment, having first learned in Class 3 skies (with occasional treks to Bortle 0 sites).

 

BQ

Edited by BQ Octantis, 26 February 2023 - 08:40 AM.

[FrankieT]

So the histogram is lying about the raw.

No it isn't, just like the 25% histogram guideline is not, and never was, a rule and 10 doesn't exist on the Bortle scale.

 

Now I'm not one to rock the boat or anything…

I'm on to you BQ
 

[BQ Octantis]

No it isn't, just like the 25% histogram guideline is not, and never was, a rule and 10 doesn't exist on the Bortle scale.

 

I'm on to you BQ
 

 

 

Shhhhh! Don't tell anyone!

Edited by BQ Octantis, 26 February 2023 - 08:41 AM.

[sharkmelley]

So the histogram isn't being truthful about the raw.

No it's not lying.  It's the genuine histogram of the JPG that the camera would produce.  But it's also true that there is plenty more "headroom" in the raw file to allow photographers some highlight recovery.

[BQ Octantis]

No it's not lying.  It's the genuine histogram of the JPG that the camera would produce.  But it's also true that there is plenty more "headroom" in the raw file to allow photographers some highlight recovery.

Sure, I get that—I use it often because I use the JPEGs out of my 600D/T3i and other cameras pretty prolifically. Indeed, that screenshot of the camera screen is my default screen for shooting anything, day or night.

 

I guess my question is, there any strangeness that occurs when the histogram is up, say, in the 75% range? Some sort of mathematical or electronic compression that harms the data? Again, I'm ok with clipping Trapezium and a few star cores.

 

Jerry just seemed insistent on "proper" exposure. And I see references to data being "overexposed" in the forums all the time that are far from clipping.

Edited by BQ Octantis, 26 February 2023 - 08:52 AM.

[timelapser]

I guess my question is, there any strangeness that occurs when the histogram is up, say, in the 75% range? Some sort of mathematical or electronic compression that harms the data?

Apart from clipping stars as already mentioned, as you approach saturation you potentially could see some nonlinearity in the sensor.  Sensors typically gradually start to loose sensitivity (relative to what you'd expect for a linear sensor) as you approach saturation.  That could make it tricky to stack since stacking assumes you've got linear data.

 

But in practice I think you'd have to be very close to saturation to see that with modern sensors.  Maybe someone here can chime in with saturation percentages for current sensors where you start to see significant nonlinear response.

[BQ Octantis]

Aha! That's what I'm talking about!

 

How does one test this?

[Nick Dangerr]

I'm guessing the histogram was being dominated by light pollution (what is Bortle 10, the scale only goes to 9?) reflecting off the thin clouds. 

I suspect "Bortle 10" was a joke - kind of like the Spinal Tap amps that went to "11" instead of "10" because "11" is more, right?

[BQ Octantis]

I suspect "Bortle 10" was a joke - kind of like the Spinal Tap amps that went to "11" instead of "10" because "11" is more, right?

Sheer hyperbole.

 

See post #10…

 

BQ

[BQ Octantis]

Here's an post on "optimum" exposure relative to read noise:

 

https://www.cloudyni...e-is-8-seconds/

 

Not sure how to translate it into DSLR speak…

Edited by BQ Octantis, 26 February 2023 - 04:54 PM.

[SteveInNZ]

I suspect "Bortle 10" was a joke - kind of like the Spinal Tap amps that went to "11" instead of "10" because "11" is more, right?

Many years ago, a friend wanted me to build him a stereo amp. One of the requirements was no volume knob, but a volume switch. Because either the wife is home, or she's not.

 

Steve.

[vidrazor]

On the actual Bortle scale as defined, I'm currently probably between Class 6 and Class 7.

That's not Bortle 10, this is Bortle 10.

Attached Thumbnails

• 

Edited by vidrazor, 26 February 2023 - 09:11 PM.

[Michael Covington]

Here's an post on "optimum" exposure relative to read noise:

 

https://www.cloudyni...e-is-8-seconds/

 

Not sure how to translate it into DSLR speak…

There has been extensive work on this by Robin Glover (which I was recently working through for other purposes).  The upshot of it is that 25% to 50% histogram is generally best; newer, lower-noise DSLRs can go somewhat lower.

[BQ Octantis]

Many years ago, a friend wanted me to build him a stereo amp. One of the requirements was no volume knob, but a volume switch. Because either the wife is home, or she's not.

 

Sigh. I can relate. I own a trumpet I never play for pretty much the same reason: it's too loud to play when anyone's home.

 

 

That's not Bortle 10, this is Bortle 10.

Bloody oath, mate!

[BQ Octantis]

There has been extensive work on this by Robin Glover (which I was recently working through for other purposes).  The upshot of it is that 25% to 50% histogram is generally best; newer, lower-noise DSLRs can go somewhat lower.

Thanks Mike. I plan to give ISO1600 a go, assuming the sky is ever clear without a big bloody moon blowing out the Bortle 10! But with Daylight Savings about to start and the vernal equinox almost upon us, I'm having my doubts for the season…