Return to the Cloudy Nights Telescope Reviews home page


Astrophotography and Sketching >> CCD Imaging & Processing

Pages: 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | (show all)
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: Mike7Mak]
      #5654905 - 01/31/13 07:08 PM

Quote:

I have to agree. It seems quite unscientific to broaden the meaning of noise to the point that 'noise' has noise. IMO use of the term 'noise' as a catch-all for unwanted signal approaches the level of slang.




I think everything is signal, it just may not be the signal you "want". But if we go by what you "want" how the heck do we know what you want? We can't read minds.... We can rigorously define a parameter of interest and come up with a practical way to measure it, and try to do so with the least "error" possible. As mentioned in stats we don't really talk about "noise" usually. We have error terms, which really simplify to variance and bias.

Take for example darts. If the goal is to hit the bullseye, we throw the dart a lot and it goes all over. Then we have high variance. If it always hits the same place, we have, very low variance. However, even with very low variance, your darts may always land in the same place, but miss the dartboard completely. Then you have large bias. If they land very close to the bullseye, you have low bias and if they hit the bullseye, you have no bias. So, together you have a measure of " accuracy". The deviation from the bullseye is the measurement.

If your target is well defined, such as a bullseye, then It's theoretically possible to determine your accuracy by measuring your deviation over multiple dart throws (I.e. your taking "samples" of your dart throwing ability) However, in reality, the true value of most things we study, are probably not known, it's sort of like measuring your dart throws while blindfolded. While you can't do this, you may be able to measure your dart throws, under different circumstances and use this to "calibrate" your aim latter, when you put the blindfold back on. You will still have random variation to contend with, but on average, if you throw enough darts, your deviation from the bullseye, if calibrated correctly, will average out to zero, as there is no longer any fixed variation, all the deviation is random and looking at the average of your measurements, you will now look like a perfect player!

This may seem like a slight of hand, but if our goal is to figure out where the bullseye really is, we now have a perfect estimate.


Post Extras: Print Post   Remind Me!   Notify Moderator  
pfile
Post Laureate


Reged: 06/14/09

Re: 5 or 30 minute exposures for narrowband? new [Re: Inverted]
      #5655081 - 01/31/13 09:08 PM

"bias" - isn't this meant in the EE sense, like forward biasing a diode?

my crude understanding of how a CCD works is that before you expose the frame, the capacitive sites are biased with a positive voltage.

there's some uncertainty in this bias voltage from pixel to pixel, hence, "bias noise".

as photons fall onto the detector, they cause electrons in the silicon to move to a higher valence, thus progressively making the voltage more negative. but even if the number of photons that landed on each site was the same, the end voltage is different because of difference in the bias voltages. (and of course, dark current).


Post Extras: Print Post   Remind Me!   Notify Moderator  
pfile
Post Laureate


Reged: 06/14/09

Re: 5 or 30 minute exposures for narrowband? new [Re: pfile]
      #5655091 - 01/31/13 09:11 PM

inverted, you are simply describing photon shot 'noise' which really has nothing to do with detectors. it's just part of how this universe is put together. bias has nothing to do with it. bias and bias noise are artifacts of the photon detectors we have invented.

Post Extras: Print Post   Remind Me!   Notify Moderator  
vpcirc
Post Laureate


Reged: 12/09/09

Loc: Merced CA
Re: 5 or 30 minute exposures for narrowband? new [Re: Inverted]
      #5655101 - 01/31/13 09:18 PM

I'd like to see some examples of the great images the experts here to see how they apply your obvious extensive knowledge?
The guys "who make pretty pics" must know what they're talking about the evidence to me lies in what they produce. Many are pioneers in this field and have years of extensive experiance. Not all are mathematicians or engineers, but I'll trust experiance and advice from someone producing what I hope too, any day over someone trying to show how they know more than anyone else. People are critizing others for sharing links for advice, but those folks are producing amazing images and being acknowledged with APODS. The question is how many do you have?
I'm from Missouri "Show Me"


Post Extras: Print Post   Remind Me!   Notify Moderator  
shams42
Pooh-Bah
*****

Reged: 01/05/09

Loc: Kingsport, TN
Re: 5 or 30 minute exposures for narrowband? new [Re: vpcirc]
      #5655114 - 01/31/13 09:29 PM

Mike, please show us something from one of these top imagers that you keep referring to that does not agree with what everyone here is saying.

What is this mysterious advice?

You keep insinuating that we all have some profound ignorance of this topic and yet you do not expose your own view for discussion and criticism. So out with it!


Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: pfile]
      #5655166 - 01/31/13 10:05 PM

Quote:

"bias" - isn't this meant in the EE sense, like forward biasing a diode?

my crude understanding of how a CCD works is that before you expose the frame, the capacitive sites are biased with a positive voltage.

there's some uncertainty in this bias voltage from pixel to pixel, hence, "bias noise".

as photons fall onto the detector, they cause electrons in the silicon to move to a higher valence, thus progressively making the voltage more negative. but even if the number of photons that landed on each site was the same, the end voltage is different because of difference in the bias voltages. (and of course, dark current).




That's a confusion I've been having. "Bias" in an EE sense, is apparently used in one way, however, "bias frames"', are actually apparently named for the EE definition, but function in a statistical way. To really make things confusing, mathematically, they function statistically to remove something else called "bias" in stats, although this actually differs from the definition of bias used in EE


Post Extras: Print Post   Remind Me!   Notify Moderator  
Mike7Mak
Pooh-Bah
*****

Reged: 12/07/11

Loc: New York
Re: 5 or 30 minute exposures for narrowband? new [Re: Inverted]
      #5655208 - 01/31/13 10:27 PM

Quote:

Quote:

I have to agree. It seems quite unscientific to broaden the meaning of noise to the point that 'noise' has noise. IMO use of the term 'noise' as a catch-all for unwanted signal approaches the level of slang.




Take for example darts.



I'm not quite sure why what I said prompted that explanation but thanks.

I might have used the analogy of sighting in a rifle, where the offset of the group from the bullseye represents unwanted signal (subtracted with clicks of the scope crosshairs) and the individual hits within the group represent noise (the variance/uncertainty type that must be averaged out).

My point however was if 'noise' is going to mean everything from hot pixels to measurement uncertainty then without further qualification the term 'noise' is profoundly imprecise. Saying an image is noisey becomes only slightly more descriptive than saying it sucks.

In the grand scheme of this discussion this is just a nit pick, so please feel free to ignore it and carry on.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: Mike7Mak]
      #5655236 - 01/31/13 10:42 PM

Quote:

inverted, you are simply describing photon shot 'noise' which really has nothing to do with detectors. it's just part of how this universe is put together. bias has nothing to do with it. bias and bias noise are artifacts of the photon detectors we have invented.




I wouldn't claim to understand all of the noise sourses in the hobby, but I do know enough to understand that you either do not understand what i am saying, and/or do not understand modern statistical theory. I've tried to give a very general overview of statistical theory, as relating to sampling theories, several times, I'm not sure what else I can say without getting into more technical nuts and bolts.... I think the results are fairly intuitive, so, I'm apparently not doing a very good job anyways perhaps your a technical person and would prefer a more technical explanation?. I think a good introduction would be Sampling Design and Analysis by Shannon Lohr, most of the other sampling texts people seem to use really get complex quick, and get into fairly hardcore math and especially matrix algebra though. Really, most of the ideas seem to come from the central limit theorems and various theorems referred to as the "law" of large numbers (but I think these are probably more results of the central limit theorems, the proofs go over my head, but my graduate school stats professors, who specialized in sampling theories, seemed to think so). Anyways, I can't really add anything to the conversation, other than try to put some of the statistical concepts in perspective. The models used are very statistical, but attempts to use them, seem to try to treat them in more deterministic perspective. I think that is really only necessary if your under-sampled though (and this could be sampling with regards to calibration, not just image acquisition)).


Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: vpcirc]
      #5655253 - 01/31/13 10:49 PM

Quote:

I'd like to see some examples of the great images the experts here to see how they apply your obvious extensive knowledge?




The hobby can be an art or a science or both. A scientist who creates the paints for an artist need not be a proficient artist and the artist need not be a proficient scientist. When people of different backgrounds and perspectives work together, though, great things can happen.


Post Extras: Print Post   Remind Me!   Notify Moderator  
David PavlichAdministrator
Transmographied
*****

Reged: 05/18/05

Loc: Mandeville, LA USA
Re: 5 or 30 minute exposures for narrowband? new [Re: Inverted]
      #5655267 - 01/31/13 11:00 PM

Sometimes we all tend to overthink this stuff. If you have a chip like the 8300 class and you're shooting narrowband, take as long as you can stand and your equipment can handle. Neil Flemming recommends 30 minutes. Then take darks and flats and process the stuff.

If you have a camera with a Sony chip like the QHY8 or the Starlight Express MC25, then you don't really need the darks, just a bunch of bias and flats and process.

At the AIC, I learned a couple of things of note. For LRGB, there's a point of diminishing returns beyond 10 minutes. I believe Craig Stark was the presenter that had a chart showing how the curve was very steep until it got to around 10 minutes. The curve tapers to near flat after 10 minutes. Of course, that can change with the scope. A Powernewt is not going to need the full 10 minutes where as an f10 SC may need more time.

Tony Hallas stated that 16 lights seemed to be a good number. In order to get big returns, you needed to go way past 16. In other words, doing 20 lights won't make much difference.

The one thing from every presenter was for a given time, longer exposures ARE best as long as the equipment can handle it and skyglow doesn't take over. Of course, NB isn't affected by skyglow...well not like LRGB.

Anyway, take your images, process them and enjoy them. This stuff will give you a headache if you overdo it.

David


Post Extras: Print Post   Remind Me!   Notify Moderator  
pfile
Post Laureate


Reged: 06/14/09

Re: 5 or 30 minute exposures for narrowband? new [Re: Inverted]
      #5655278 - 01/31/13 11:06 PM

there is nothing wrong with the statistics you are talking about but you are using "bias" in a statistical sense while in the process of calibrating ccd images, bias means something very different. that's all.

Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: Mike7Mak]
      #5655279 - 01/31/13 11:06 PM

Quote:


I'm not quite sure why what I said prompted that explanation but thanks.





It was in part an attempt to explain why the definition of noise does not necessarily need to be precise, if "target" is precisely defined. Also, I agreed with Chris's statements about authority, I'm not big on authority either. However, I think a lot of the best results do not need to rely on authority or complicated technical definitions, but simple logic and though experiments. I though it better to provide a starting point for such, rather than just state a bunch of technical jargon. I may or may not have done so successfully, but in the end, when methods produce results people like, they will be used, when they don't they won't. In the meantime though, there are no new results without new perspective.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: pfile]
      #5655301 - 01/31/13 11:24 PM

Quote:

there is nothing wrong with the statistics you are talking about but you are using "bias" in a statistical sense while in the process of calibrating ccd images, bias means something very different. that's all.




But it is also statistical sense. There is random noise in the bias frame that converges to a null value, and there is noise that doesn't. If you take enough bias frames and average them, you are mostly left with the stuff that doesn't. This seems important if read noise truly is the dominant source of noise, as many claim. There may be some left that could theoretically if more bias frames were stacked, but this does at least become more Gaussian. Then, when we calibrate the light frames the statistical "bias" from the "reads" are eliminated to an extent (minus some now mostly Normal random error, provided n is large enough). Much of the remaining "noise" and other random noise, from other sources, then still tends further towards the null as more calibrated light frames are stacked. These frames may be intended to linearize the data, as it sounds, however, they along with other calibration frames and sampling techniques should also help remove anything things that converge to a non-null value, independent of the sampled parameter(s) of interest. More or less so, depending on how they are used. More so for example if I were to stack say 400, less so, if I were to stack a few. If you take enough bias frames though (or any other frames that contain the asymptotically fixed and random components of read noise) then it seems you can reduce read noise pretty significantly, especially if read noise is relatively low to begin with. Then when you plug the numbers into the equations, the implications may change substantially.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Inverted
sage
*****

Reged: 01/19/13

Loc: LP Land
Re: 5 or 30 minute exposures for narrowband? new [Re: David Pavlich]
      #5655326 - 01/31/13 11:37 PM

For sure and the general rules are tried and true and for the most part surly good to follow. It is a technical hobby though.... It's good to exercise the noggin once and a while and try to understand the technical aspects I think. Otherwise, I could save a lot of money and just do basket weaving or something

Post Extras: Print Post   Remind Me!   Notify Moderator  
Mike7Mak
Pooh-Bah
*****

Reged: 12/07/11

Loc: New York
Re: 5 or 30 minute exposures for narrowband? new [Re: Inverted]
      #5655339 - 01/31/13 11:42 PM

Quote:

Quote:


I'm not quite sure why what I said prompted that explanation but thanks.





It was in part an attempt to explain why the definition of noise does not necessarily need to be precise, if "target" is precisely defined.




Gotcha. Yes, I'm all for simple logic and though experiments. My interest in topics like this is in understanding the principles and applying the techniques, to the extant that is possible without knowing advanced statistical theory and higher math. I don't have the educational background to go there nor the motivation to start learning such things now.


Post Extras: Print Post   Remind Me!   Notify Moderator  
cn register 5
scholastic sledgehammer


Reged: 12/26/12

Re: 5 or 30 minute exposures for narrowband? new [Re: vpcirc]
      #5655541 - 02/01/13 03:39 AM

Quote:

I'd like to see some examples of the great images the experts here to see how they apply your obvious extensive knowledge?
The guys "who make pretty pics" must know what they're talking about the evidence to me lies in what they produce. Many are pioneers in this field and have years of extensive experiance. Not all are mathematicians or engineers, but I'll trust experiance and advice from someone producing what I hope too, any day over someone trying to show how they know more than anyone else. People are critizing others for sharing links for advice, but those folks are producing amazing images and being acknowledged with APODS. The question is how many do you have?
I'm from Missouri "Show Me"



Mike, this is an excellent example of the argument from authority, with a side order of ad hominem.

You don't need to do this, after all you gave us both of the links to web pages with good technical information.

Chris


Post Extras: Print Post   Remind Me!   Notify Moderator  
freestar8n
Post Laureate
*****

Reged: 10/12/07

Re: 5 or 30 minute exposures for narrowband? new [Re: cn register 5]
      #5655569 - 02/01/13 04:16 AM

Well I took a look at the numbers and I can't find good specs on the 383L ccd. The ad says 7e read noise, but I don't see a spec. on the gain. Various users on the web measure higher read noise - 8-11e, and gain around 0.45 e/adu.

If we assume 7e for the read noise, and aim for that to be 10% of the sky glow shot noise - and remember that 10% is just a number from a 12-year old discussion on the web - then you want the sky shot noise to be 70e. Assuming Poisson statistics, the sky signal would then be 4900e. Dividing 4900e by the gain, 0.45e/adu, we get a target ADU for the sky glow in the 'black' regions of the image to be 11000 adu.

If a calibrated sub-exposure gave about 1200 adu sky glow in a 5m exposure, then the target sub-exposure to bury the read noise to 10% sky noise would be about 45 minutes.

Note that in this calculation, which is along the line of the web discussion Chris has chosen to use as a reference, there is no mention of the nebulosity signal itself - or anything involving "exposing until the faint stuff shows." It is all about making the read noise small compared to other camera noise, because read noise is the only thing that can have reduced impact through longer sub-exposures. It also assumes a cold camera so dark current can be ignored - which may not be the case in general - and sky noise is all that matters.

The problem is - that web discussion is somewhat stream of consciousness and includes errors that are corrected later in the thread. It says right there:

"A simple dictum would be to expose long enough so that camera noise is only about 10% of the background noise."

- but that is a mistake that is later retracted in the same thread. This is why I prefer the more scientific approach of using textbooks and journal references that themselves have references.

Ultimately, the target they "decide" on is that sky noise should be 3x read noise - or a threshold of 33% instead of 10%. This is much easier to achieve.

TSub(33%) = (7/0.33)^2/0.45/1200*5 = 4m

So one of the biggest factors here is the choice of that 10-33% threshold value - and there is nothing magic about that choice.

I don't know the f/number used here, but assuming it is f/8 - if we switched to f/2 hyperstar, the sky background would be reached 16 times as fast, and the original 45 minutes, assuming 10% target, would be reached in about 3 minutes. Assuming 33% target, about 15 seconds. This would still be narrowband with the same filter - just a faster system - so narrowband does not absolutely require long exposures - even with this 10% threshold and 7e read noise.

So the target numbers depend on many system parameters, plus a user-specified threshold. They require good estimates of the gain and read noise, plus a personal choice for the threshold. Since those numbers vary, I think it's good to measure your own camera - and it isn't hard - and to decide what threshold value works best for you.

I could have made a mistake above so I welcome corrections.

Note that if the original 1200 adu background value came from a raw sub that included a pedestal of, say, 600 - then all estimates above would be doubled because it would take twice as long to gather the needed sky background electrons.

Frank


Post Extras: Print Post   Remind Me!   Notify Moderator  
vpcirc
Post Laureate


Reged: 12/09/09

Loc: Merced CA
Re: 5 or 30 minute exposures for narrowband? new [Re: shams42]
      #5655636 - 02/01/13 06:27 AM

Quote:

Mike, please show us something from one of these top imagers that you keep referring to that does not agree with what everyone here is saying.

What is this mysterious advice?

You keep insinuating that we all have some profound ignorance of this topic and yet you do not expose your own view for discussion and criticism. So out with it!




I am in no way saying that all these posts are wrong or incorrect. But I have heard repeatedly "you don't need to go longer" or several short exposures stacked with equal time are the same, My philopshy mirrors what David has posted. Hallas says there's a limit to how many exposures can be stacked before there's no benefit, therefore shoot as long of exposures as you can. In my case I would trust someone who's been shooting AP since the film days, produces top notch images, has several APODS, is used by NASA and Apple any day over someone who can quote the statistical reasoning of how noise accumulates, but can't produce a quality image themselves.


But when I see statements like the following;

Well I certainly agree with that - which is why I provide direct links to professional sources - which themselves contain references. Many of links provided in these discussions are just to web pages that themselves have no references. An argument from authority should be distinguished from a "reference to the literature" - and primary sources. On this topic, the amateur sources, and arbitrary web pages, can depart from professional literature - and cause confusion or state incorrect conclusions.

Or

Noise remains a mystical evil spirit that we ward off with long exposure and image stacking on faith alone...cuz the guys who make pretty pics say so.

That appears to me as a direct slap at any of the people whom have posted links to information sources to try and help people. Most imagers could care less about the science behind noise they just want help in understanding what works best.


Post Extras: Print Post   Remind Me!   Notify Moderator  
dan17
super member


Reged: 03/30/09

Re: 5 or 30 minute exposures for narrowband? new [Re: vpcirc]
      #5655675 - 02/01/13 07:14 AM

Many things will be clear and easy to understand if You all have a look at this:

http://www.cloudynights.com/item.php?item_id=1966

http://www.cloudynights.com/item.php?item_id=1973

http://www.cloudynights.com/item.php?item_id=2001

http://www.cloudynights.com/item.php?item_id=2042

http://www.cloudynights.com/item.php?item_id=2394


Post Extras: Print Post   Remind Me!   Notify Moderator  
Mike7Mak
Pooh-Bah
*****

Reged: 12/07/11

Loc: New York
Re: 5 or 30 minute exposures for narrowband? [Re: vpcirc]
      #5655979 - 02/01/13 09:59 AM

Quote:

Or

Noise remains a mystical evil spirit that we ward off with long exposure and image stacking on faith alone...cuz the guys who make pretty pics say so.

That appears to me as a direct slap at any of the people whom have posted links to information sources to try and help people. Most imagers could care less about the science behind noise they just want help in understanding what works best.



And again... good grief, passive aggressive much?

Dude, that's completely out of context, it meant nothing of the sort. And to the degree that last bit was a poke in the ribs, it was at you, not "any of the people" who post links.

So let's pretend you know what 'most imagers' couldn't care less about. 'Most imagers' aren't participating in this thread. The ones that are obviously DO care about the science. In addition to understanding 'what works best' some of us also want to know why.

Both those issues, 'don't need to go longer' and the diminishing returns of stacking, were thoroughly explored and apparently you didn't understand a word of it. If you had you'd realise no one blasphemed the imaging gods.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Pages: 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | (show all)


Extra information
10 registered and 21 anonymous users are browsing this forum.

Moderator:  Oldfield, droid, bilgebay, WOBentley, dr.who 

Print Thread

Forum Permissions
      You cannot start new topics
      You cannot reply to topics
      HTML is disabled
      UBBCode is enabled


Thread views: 7611

Jump to

CN Forums Home




Cloudy Nights Sponsor: Astronomics