Jump to content

  •  

CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.

Photo

Long Exposure Noise Reduction Vs. Dark Frame Subtraction

  • Please log in to reply
50 replies to this topic

#26 jgraham

jgraham

    ISS

  • *****
  • Posts: 22,493
  • Joined: 02 Dec 2004
  • Loc: Miami Valley Astronomical Society

Posted 03 October 2018 - 06:36 AM

I have used both libraries of dark frames and LENR. I even spent about a year taking a close look at the LENR function and decided that for me it was the way to go. I have used it ever since and I am delighted with how well it works. It has now been some 5 years and I still use it when I use my DSLRs. I have also found that it has had zero impact on my productivity since my imaging is driven by when an object climbs above the trees and when it reaches the meridian, not by how many subs that I take.

LENR, I use it, love it, won't image without it, taken a lot of flack over it, don't care. It is my secret sauce.

One quick example...

 

M27 LENR Example (v1).jpg

 

Telescope: Meade SN10, Orion Atlas EQ-G
Camera: Baader Modified Canon 550D (Rebel T2i)
Filter: Orion Imaging Skyglow Filter
Guide scope: Williams Optics 50mm, Meade DSI Pro II, PHD
Exposure: 64x30sec, ISO 1600 saved as RAW
Darks: Internal (Long Exposure Noise Reduction On)
Flat: 32x1/20sec, ISO 400 taken with an LED tracing tablet
Average Light Pollution: Red zone, good transparency
White Balance: Deep Sky Stacker Automatic
Software: Backyard EOS, Deep Sky Stacker, Nebulosity, Photoshop

 

Whatever process that you use to take images that you like is a good thing.

 

Enjoy!


  • Alen K, WarmWeatherGuy, Jon Rista and 3 others like this

#27 freestar8n

freestar8n

    Vendor - MetaGuide

  • *****
  • Vendors
  • Posts: 10,984
  • Joined: 12 Oct 2007
  • Loc: Melbourne, Australia

Posted 03 October 2018 - 07:14 AM

Usually, astronomical post processing softwares match the darks to the temperature (in fact noise) of the lights so that a slight temperature deviation is not detrimental to the image quality. This is possible because the thermal noise is a function of the temperature.

 

Therefore, it is not a problem if you shot your darks after you have shot your lights. During warm nights it may be worth shooting intermediate batches of darks (for example take 20 lights, then take 10 darks, and so on).

 

However, if you shoot with LENR ON, never substract bias ! The bias will be contained with thermal noise inside each individual images. Just apply the flatfield correction. When you will stack your pile of LENR-lights, you will average the noises all together. 

I know that it is fairly standard to do scaling of the dark to match the light for optimal noise reduction in the subtraction - but that won't work well if the temperature is changing at different rates across the sensor - unless they do the optimization locally across the sensor somehow.

 

So - again - if the dark current noise is very low in the first place then LENR would not make sense.

 

But in the real world with a real DSLR in a changing thermal environment - there is good reason to expect dark matching not to work well.

 

And it does appear that people have tried LENR with an open mind and concluded it does work better for them.

 

It's not just about maximizing signal, but keeping noise low.  And although dark matching should work perfectly in theory - in practice other factors may make LENR work better.

 

Frank



#28 Alen K

Alen K

    Surveyor 1

  • *****
  • Posts: 1,745
  • Joined: 25 Nov 2009

Posted 03 October 2018 - 11:27 AM

I have used both libraries of dark frames and LENR. I even spent about a year taking a close look at the LENR function and decided that for me it was the way to go. I have used it ever since and I am delighted with how well it works. It has now been some 5 years and I still use it when I use my DSLRs. I have also found that it has had zero impact on my productivity since my imaging is driven by when an object climbs above the trees and when it reaches the meridian, not by how many subs that I take.

LENR, I use it, love it, won't image without it, taken a lot of flack over it, don't care. It is my secret sauce.

One quick example...
 
attachicon.gif M27 LENR Example (v1).jpg

Seeing your excellent image reminded me that I used LENR in the debut prime-focus outing of my DSLR a few weeks ago. I had intended to do multi-minute subs with an autoguider but couldn't get it to work, so I just did 30-second unguided subs instead. As an experiment I decided to enable LENR. It was shirt-sleeve temperatures. Coincidentally, one of my targets through my Meade LXD55 SN6 OTA, the younger brother of your SN10 (you have or had two SN6's, right?), was M27. I took only 32 subs @ ISO 1600 however, half as many as you did. And my camera, a Pentax K-3II, is not modified, but I'm not sure it makes much of a difference for this object. I'm reasonably pleased with how it turned out given the short integration time. This is about a 50% crop of the original frame.

Attached Thumbnails

  • IMG_2756.JPG

  • James Paulson likes this

#29 Jerry Lodriguss

Jerry Lodriguss

    Vendor

  • ****-
  • Vendors
  • Posts: 7,797
  • Joined: 19 Jul 2008
  • Loc: Voorhees, NJ

Posted 03 October 2018 - 11:39 AM

I know that it is fairly standard to do scaling of the dark to match the light for optimal noise reduction in the subtraction - but that won't work well if the temperature is changing at different rates across the sensor - unless they do the optimization locally across the sensor somehow.

 

So - again - if the dark current noise is very low in the first place then LENR would not make sense.

 

But in the real world with a real DSLR in a changing thermal environment - there is good reason to expect dark matching not to work well.

 

And it does appear that people have tried LENR with an open mind and concluded it does work better for them.

 

It's not just about maximizing signal, but keeping noise low.  And although dark matching should work perfectly in theory - in practice other factors may make LENR work better.

Hi Frank,

 

I don't see any DSLRs in your sig, do you have a lot of experience with them for long-exposure deep-sky astrophotography?

 

Have you used LENR for long-exposure deep-sky astrophotography?

 

Have you used a dark library for long-exposure deep-sky astrophotography?

 

Have you quantified the difference for long-exposure deep-sky astrophotography?  (I have not)

 

I see your comments about temps changing at different rates across the sensor. Do you know of any documentation that this is actually a problem in the real world with DSLRs?

 

How would you even measure it on a DSLR?

 

Thanks,

 

Jerry


  • Jim Waters likes this

#30 freestar8n

freestar8n

    Vendor - MetaGuide

  • *****
  • Vendors
  • Posts: 10,984
  • Joined: 12 Oct 2007
  • Loc: Melbourne, Australia

Posted 03 October 2018 - 04:55 PM

Hi Frank,

 

I don't see any DSLRs in your sig, do you have a lot of experience with them for long-exposure deep-sky astrophotography?

 

Have you used LENR for long-exposure deep-sky astrophotography?

 

Have you used a dark library for long-exposure deep-sky astrophotography?

 

Have you quantified the difference for long-exposure deep-sky astrophotography?  (I have not)

 

I see your comments about temps changing at different rates across the sensor. Do you know of any documentation that this is actually a problem in the real world with DSLRs?

 

How would you even measure it on a DSLR?

 

Thanks,

 

Jerry

Canon 20d long ago - and Canon 5dii right now - but I don't use it for astro imaging due to all the issues I cite.  The only value it has for me is the larger sensor.  I will add the 5dii to my signature.

 

With the canon 20d I was using ImagesPlus and there was a banding problem that caused trouble for a lot of people.  People blamed the sensor and bought new cameras to fix the problem - but I recognized the noise behavior they described could not be a problem in the sensor and instead had to do with how the raws were being converted.

 

It is described here from I guess 2009 where I am credited:

 

http://www.mlunsold..../darkriver.html

 

In cloudynights threads from back then I was mainly focused on getting the most from dslr's - but when I switched to astro cameras there was no reason to go back.

 

There is an image of m95 and m96 I did with the 20d and EdgeHD8 at f/10 - and that was quite painful and required the fixes to ImagesPlus - but it came out pretty well and is in some of the Celestron documents on EdgeHD since it was a very early dslr image with EdgeHD.

 

Much of what I'm describing here can be seen by inspection in a series of images with any dslr.  If you see things changing - as shown above - you know a single dark master won't work without some kind of scaling or matching.  And if you see gradients also changing - then you know a single dark master combined with a global scaling won't be effective. 

 

I don't know why these concepts wouldn't be obvious - and I have no idea why people would ignore the role of noise and assume more signal is always more better.

 

Frank


Edited by freestar8n, 03 October 2018 - 04:56 PM.

  • Jon Rista and 42itous1 like this

#31 Jerry Lodriguss

Jerry Lodriguss

    Vendor

  • ****-
  • Vendors
  • Posts: 7,797
  • Joined: 19 Jul 2008
  • Loc: Voorhees, NJ

Posted 03 October 2018 - 09:46 PM

Canon 20d long ago - and Canon 5dii right now - but I don't use it for astro imaging due to all the issues I cite.  The only value it has for me is the larger sensor.  I will add the 5dii to my signature.

 

With the canon 20d I was using ImagesPlus and there was a banding problem that caused trouble for a lot of people.  People blamed the sensor and bought new cameras to fix the problem - but I recognized the noise behavior they described could not be a problem in the sensor and instead had to do with how the raws were being converted.

 

It is described here from I guess 2009 where I am credited:

 

http://www.mlunsold..../darkriver.html

 

In cloudynights threads from back then I was mainly focused on getting the most from dslr's - but when I switched to astro cameras there was no reason to go back.

 

There is an image of m95 and m96 I did with the 20d and EdgeHD8 at f/10 - and that was quite painful and required the fixes to ImagesPlus - but it came out pretty well and is in some of the Celestron documents on EdgeHD since it was a very early dslr image with EdgeHD.

 

Much of what I'm describing here can be seen by inspection in a series of images with any dslr.  If you see things changing - as shown above - you know a single dark master won't work without some kind of scaling or matching.  And if you see gradients also changing - then you know a single dark master combined with a global scaling won't be effective. 

 

I don't know why these concepts wouldn't be obvious - and I have no idea why people would ignore the role of noise and assume more signal is always more better.

You think these concepts are obvious?

 

The original poster asked if there was any consensus on the question of LENR and latest research from the ap community.

 

I don't see any quotations on scientific research from either side here, neither your LENR pro side, nor my anti-LENR side.  I see anecdotal testimonials. Mine included. Yours included. Jon's included.

 

It should be pretty obvious from the posts in this thread that there isn't any consensus.

 

This reminds me of a zen parable:

 

"The student Doko came to a zen master and said: "I am seeking the truth. In what state of mind should I train myself, so as to find it?"

 

Said the master: "There is no mind, so you cannot put it in any state. There is no truth, so you cannot train yourself for it."

 

"If there is no mind to train, and no truth to find, why do you have these monks gather before you everyday to study zen and train themselves for this study?"

 

"But I haven't an inch of room here," said the master, "so how could the monks gather? I have no tongue, so how could I call them together to teach them?"

 

"Oh, how can you lie like this?" asked Doko.

 

"But if I have no tongue to talk to others, how can I lie to you?" said the master.

 

Then Doko said sadly, "I cannot follow you. I cannot understand you."

 

"I cannot understand myself" said the master.

 

I guess the best we can do is tell the original poster to try it both ways for himself and see what works with his individual camera.

 

Jerry


Edited by Jerry Lodriguss, 03 October 2018 - 11:57 PM.

  • Jim Waters likes this

#32 Jon Rista

Jon Rista

    ISS

  • *****
  • Posts: 24,995
  • Joined: 10 Jan 2014
  • Loc: Colorado

Posted 04 October 2018 - 12:10 AM

Jerry, sadly, there really hasn't been good enough testing of LENR in an AP context. 

 

I was starting to test, then the weather got cold, and the benefits that LENR was starting to show for me disappeared. Then the weather got bad...and, outside of maybe two to three relatively short periods per year where I get a few clear nights in a row or within a week or so...I really haven't had the kind of clear night counts I had a couple of years ago. I did some more tests, with darks only, and I seem to have pretty consistent higher temp results with my 5D III. 

 

But it would be nice to have some more examples of what LENR does across a wider range of DSLRs. I think there can indeed be some solid IQ improvements with LENR, when used at the right times. I don't think anyone, even myself these days since I have so few clear nights, is willing to spend the time testing, though. :p 

 

I may be able to try it again at some point here, though. I now have two mounts, and most of my for-keeps imaging is done with cooled astro cameras. I might be able to set up the other mount with a second imaging rig on some warm nights, acquire some data with the DSLR and mirrorless cameras I own, with and without LENR, and do some more scientific tests. (I'll have to get my other mount rebuilt, though...)



#33 Jerry Lodriguss

Jerry Lodriguss

    Vendor

  • ****-
  • Vendors
  • Posts: 7,797
  • Joined: 19 Jul 2008
  • Loc: Voorhees, NJ

Posted 04 October 2018 - 03:27 AM

Jon, these tests could easily be done in a controlled studio environment (on a cloudy night).

 

For example, here is a 240-second shot in my office with the windows blacked out (and at night) and a controlled light source.

 

DSC_0181.jpg

Jerry


Edited by Jerry Lodriguss, 04 October 2018 - 03:35 AM.


#34 freestar8n

freestar8n

    Vendor - MetaGuide

  • *****
  • Vendors
  • Posts: 10,984
  • Joined: 12 Oct 2007
  • Loc: Melbourne, Australia

Posted 04 October 2018 - 08:43 AM

You think these concepts are obvious?

 

The original poster asked if there was any consensus on the question of LENR and latest research from the ap community.

 

I don't see any quotations on scientific research from either side here, neither your LENR pro side, nor my anti-LENR side.  I see anecdotal testimonials. Mine included. Yours included. Jon's included.

 

It should be pretty obvious from the posts in this thread that there isn't any consensus.

 

This reminds me of a zen parable:

 

"The student Doko came to a zen master and said: "I am seeking the truth. In what state of mind should I train myself, so as to find it?"

 

Said the master: "There is no mind, so you cannot put it in any state. There is no truth, so you cannot train yourself for it."

 

"If there is no mind to train, and no truth to find, why do you have these monks gather before you everyday to study zen and train themselves for this study?"

 

"But I haven't an inch of room here," said the master, "so how could the monks gather? I have no tongue, so how could I call them together to teach them?"

 

"Oh, how can you lie like this?" asked Doko.

 

"But if I have no tongue to talk to others, how can I lie to you?" said the master.

 

Then Doko said sadly, "I cannot follow you. I cannot understand you."

 

"I cannot understand myself" said the master.

 

I guess the best we can do is tell the original poster to try it both ways for himself and see what works with his individual camera.

 

Jerry

?

 

Hoo boy.  Yes I think it is obvious that only focusing on signal will miss the role of noise.  Is that not obvious?  If you have one room with 2 pies in it, and another with 3 pies in it - will you get more pie in the room that has 3?  What if the first room has 2 people in it and the second has 200?

 

There are people who say that LENR is definitely worse because it will collect less signal.  And I simply respond - if the noise is also reduced then it can be a win.  And there is good reason it can be reduced significantly.  And I consider that all obvious.  If it isn't obvious to you that is perfectly ok with me.

 

At no point have I said one way is definitely better or worse than the other.  But I'm confident that the blind claim that more signal means a better result - is missing important factors.

 

The other thing I hope obvious is that using a dslr on a hot summer night is very different from using one in mid winter.  With the 20d the first frame would be somewhat acceptable - and the second would be terrible - and there was no way any kind of dark matching would help.  But in winter it was much more tolerable.  Other factors will play a strong role - with temperature, temperature change, software, and camera type being the main ones.  And I consider that obvious.  That is why - umm - controlled room temperature conditions are exactly what you would not want for a meaningful test.

 

That is why my answer to the OP is - it will depend on many factors and it is worth exploring with an open mind.  And I hope that my description of the fundamental trade off's will be helpful.  It is clear that over the years more people have become open to the possible wins of LENR and they have switched - and they have stated so in this thread.  And I think that's great.  The worst answer in a case like this is a confident flat out statement that ignores fundamental variables and oversimplifies a complex situation.  The best is to describe trade off's and to encourage basic testing with an open mind - and sharing of the results in a forum like this.  And I think for the most part this thread is a good step forward.

 

Frank


  • B 26354 and sbharrat like this

#35 WarmWeatherGuy

WarmWeatherGuy

    Aurora

  • *****
  • Posts: 4,709
  • Joined: 27 Aug 2011
  • Loc: Las Cruces, NM 32° N, 107° W

Posted 04 October 2018 - 09:00 AM

If taking pictures heats up the sensor and the temperature slowly ramps up over time then if you take your darks first and then your lights then your darks will all be at a cooler temperature than the lights. Every dark will be at a cooler temperature than every light. The warmest dark will be cooler than the coolest light. The problem is reversed if you take the lights first and then the darks.

 

If you use LENR then every dark will be matched temperature-wise with a light. Yes, each dark will be at a higher temperature than the light it is matched with but only slightly.

 

I used the time-lapse feature to take my darks. For grins I had the camera make a .mp4 movie file. When playing the movie file I see the darks getting brighter over time. I posted the first and last frame in comment #18. It seems that taking pictures heats up the sensor.

 

I guess the best way (if not using LENR) is to take your darks last, while you are tearing down your equipment, so the colder night temperature will offset the sensor heating up due to use.


  • smokindok likes this

#36 Jon Rista

Jon Rista

    ISS

  • *****
  • Posts: 24,995
  • Joined: 10 Jan 2014
  • Loc: Colorado

Posted 04 October 2018 - 11:22 AM

Jon, these tests could easily be done in a controlled studio environment (on a cloudy night).

 

For example, here is a 240-second shot in my office with the windows blacked out (and at night) and a controlled light source.

 

attachicon.gif DSC_0181.jpg

Jerry

You should try again without the light source....looks like you have a huge signal there compared to what you get per sub from the night sky. 

 

Also, what is the ambient temp? The cases where LENR is useful is when ambient is like 80F or so, which would lead to sensor temps higher than that.



#37 Jerry Lodriguss

Jerry Lodriguss

    Vendor

  • ****-
  • Vendors
  • Posts: 7,797
  • Joined: 19 Jul 2008
  • Loc: Voorhees, NJ

Posted 04 October 2018 - 01:32 PM

Other factors will play a strong role - with temperature, temperature change, software, and camera type being the main ones.  And I consider that obvious.  That is why - umm - controlled room temperature conditions are exactly what you would not want for a meaningful test.

My office stays at about 75 to 76 degrees. That would be comparable to the warmest temps I encounter in the summer here where I shoot at night.

 

My software and camera type are not variables for me in a single test.

 

Ambient temps at night would be dropping, but the temperature inside the camera would be rising until it levels off.

 

I don't see any reason why any of this would be objectionable for a meaningful test of LENR versus a cloudy-nights master dark.

 

If the ambient temp dropped and that did in fact cool the camera somewhat, you would still be testing a worst case scenario which would be exactly the same as a summer night where the temperature stays relatively constant.

 

BTW, just as a note for anyone who wants to try this... when I make my master dark, I let the camera run all night, I shoot continuously with just a 10 second pause between frames. The temp increases for the first couple of hours. I throw out those frames. Then I have 6 hours or so worth of darks that are at the same temperature.

 

Since I use Live View to focus after acquiring a field, it heats the camera up, so when I start my lights run, the camera is usually already close to maximum temp, and the lights are usually at a reasonably consistent temp.

 

Then I use automatic dark frame matching and I've never had any artifacts from incorrect temperature matching.

 

But this is anecdotal evidence.

 

Conducting an indoor test to compare LENR to a cloudy-night master dark should be perfectly fine to tell if your individual camera works better with one or the other.

 

And how would YOU recommend setting up a test light this?  Outside at night where the temp is changing?  If I suggested that, you would object that the temp might be changing at a different rate on different nights and that would invalidate the test.  It seems to me an indoor test would actually eliminate one variable which would be a good thing in a test like this.

 

Jerry


Edited by Jerry Lodriguss, 04 October 2018 - 01:54 PM.


#38 Jerry Lodriguss

Jerry Lodriguss

    Vendor

  • ****-
  • Vendors
  • Posts: 7,797
  • Joined: 19 Jul 2008
  • Loc: Voorhees, NJ

Posted 04 October 2018 - 01:43 PM

You should try again without the light source....looks like you have a huge signal there compared to what you get per sub from the night sky. 

 

Also, what is the ambient temp? The cases where LENR is useful is when ambient is like 80F or so, which would lead to sensor temps higher than that.

I have areas in the frame with little or no signal and it's a 240-second exposure.

 

Why do you need the entire frame?  To see banding?

 

If you think you need the entire frame blank like the night sky, then you could just as easily shoot a blank wall and fill the frame.

 

Controlling the light to get a low level of illumination is not that hard.

 

Ambient temp was 75-76 degrees F.

 

Where do people shoot with an ambient temp of 80F at night on a regular basis? 

 

I think 75 degrees is a perfectly acceptable temp for a test like this.

 

If you are shooting with temps higher than this at night, you need a regulated cooled CCD or CMOS astro camera.

 

Jerry


Edited by Jerry Lodriguss, 04 October 2018 - 02:02 PM.


#39 Jon Rista

Jon Rista

    ISS

  • *****
  • Posts: 24,995
  • Joined: 10 Jan 2014
  • Loc: Colorado

Posted 04 October 2018 - 02:15 PM

I have areas in the frame with little or no signal and it's a 240-second exposure.

 

Why do you need the entire frame?  To see banding?

 

If you think you need the entire frame blank like the night sky, then you could just as easily shoot a blank wall and fill the frame.

 

Controlling the light to get a low level of illumination is not that hard.

 

Ambient temp was 75-76 degrees F.

 

Where do people shoot with an ambient temp of 80F at night on a regular basis? 

 

I think 75 degrees is a perfectly acceptable temp for a test like this.

 

If you are shooting with temps higher than this at night, you need a regulated cooled CCD or CMOS astro camera.

 

Jerry

Well, I was thinking more globally... There are various factors that affect the noise profile in an AP sub exposure. If we want to get really scientific, I think using actual night sky subs would be best. I also want to actually integrate a bunch of LENR subs vs. some non-LENR subs calibrated with darks taken after the fact. That will give us a directly comparable example of what the differences might be in a real astrophotography context. Comparing individual subs is insufficient, IMO, as a lot of the time FPN doesn't show up much in a single sub...however stack 50, 80, 100 subs...and FPN (even a residual remnant after calibration) can become a real problem. 

 

I think I am going to give it a try. I have a tracker and a Samuang 135mm lens, and I may just attach the 5D III to that, toss it on my Star Adventurer and do some LENR testing. Early evening it is still warm here, 77-82 F for a while after the sun sets, so it should make for some good tests.



#40 Dwight J

Dwight J

    Soyuz

  • *****
  • Posts: 3,945
  • Joined: 14 May 2009
  • Loc: Lethbridge, Alberta, Canada

Posted 04 October 2018 - 02:21 PM

A number of years ago, like about 10, I read that turning off the camera for about the same length of time that your exposure was would make a difference in noise.  At the time I had a Canon 300 D which has amp glow from that warm place.  I tried it and compared to subs taken successively and it did make a big difference.  I did not continue that practise as it was a bit of PIA as I was using the Canon software to run the camera and would have to restart it everytime. I see the same effect now with the T3i - each successive exposure (with 5 sec interval) is noisier than the previous.  The effect is obvious when you view all the frames at once in something like Canon’s Digital Photo or ZoomBrowser.  That being said, LENR would seem to be a better match than a series of darks taken before or after a series of exposures.  

A dark library taken in the same fashion as the lights (temp, duration, interval) with a master dark could approach this method in effectiveness but I would need to see a comparison test.  Does the sensor generate heat when not collecting data?  If not, extending the interval could be useful without having to turn off the camera.  In the interests if full disclisure, I started the thread about LENR on the DSLR forum.  And yes, both James and I were attendees at an Alan Dyer workshop where Alan did state that he and Jerry have disagreed on LENR vs more data.  



#41 Jim Waters

Jim Waters

    Fly Me to the Moon

  • *****
  • Posts: 6,324
  • Joined: 21 Oct 2007
  • Loc: Phoenix, AZ USA

Posted 04 October 2018 - 03:17 PM

So if I wanted to try LENR what would the process be to stack and align the images in PixInsight?  I usually subtract my Master BIAS from my Master Dark... 

 

If LENS is used you will not have a Master Dark. 



#42 freestar8n

freestar8n

    Vendor - MetaGuide

  • *****
  • Vendors
  • Posts: 10,984
  • Joined: 12 Oct 2007
  • Loc: Melbourne, Australia

Posted 04 October 2018 - 04:55 PM

I have areas in the frame with little or no signal and it's a 240-second exposure.

 

Why do you need the entire frame?  To see banding?

 

If you think you need the entire frame blank like the night sky, then you could just as easily shoot a blank wall and fill the frame.

 

Controlling the light to get a low level of illumination is not that hard.

 

Ambient temp was 75-76 degrees F.

 

Where do people shoot with an ambient temp of 80F at night on a regular basis? 

 

I think 75 degrees is a perfectly acceptable temp for a test like this.

 

If you are shooting with temps higher than this at night, you need a regulated cooled CCD or CMOS astro camera.

 

Jerry

75F is about 24C.  Melbourne can be 40C when the sun is setting.  My 20d work in summer in southern NY it was in the 90's.

 

For other people it may never get above 10C and temperature is less of a factor - unless their camera heats up significantly.  Is that behavior camera dependent?  Does it depend on whether the images are stored on the camera - or captured on a laptop?  If you continued taking images for several hours - would they get worse?

 

I think your test is good as a basic sanity test - but the main things missing are a nonlinear stretch to reveal the noise - along with actual details on what the test represents.  

 

A simple way I would test this is first to see if LENR is doing anything under the covers.  So let the camera sit there taking darks of about 120s for a long time so things equilibrate.  Then switch to taking LENR frames and see if the difference in pairs of darks is any different from the LENR.  If they are different, then LENR is doing something funny and you won't be able to model the noise properly.  It's the equivalent of Sony star eating - and all you can do is say I like the result with LENR more than using a master dark.  Or worse if you find it worse.  But it has more to do with the firmware and its aesthetic benefit as opposed to noise.

 

But if you do find that LENR appears to do a true dark subtract then go ahead and continue to take perhaps 20 darks.  Use the final 10 darks to make a master dark and subtract that master from frames 1, 3, 5, 7, 9 and measure the noise in each result.  Then take frame 1 and subtract 2 and measure the noise - and repeat for 3, 5, 7, 9.

 

In measuring the noise you should probably reject the truly hot or cold pixels.

 

If you do all this at a given temperature - that should tell you something about behavior at that single temperature.

 

Now repeat under controlled conditions where the temp. is being ramped down steadily from perhaps 30C to 20C.  And repeat outdoors on a typical clear night.  It's important that it's clear because the temp. will drop more - but sadly you will just be capturing darks for this test.

 

If you definitely find that the manually subtracted pairs are no better than using a master dark, then you have strong reason to think LENR is a waste of signal and should not be used.  But if you find realistic conditions where the subtracted pairs do have less noise after subtraction - then you would need to quantify that benefit more carefully and do longer stacks with dithering both ways and measure the noise in the results - along with deep stretching to see what the noise looks like - in the dithered stack.  But again this can all be done just with darks.

 

That should all give a good idea of what is going on.  It's what I would do if I wanted a less anecdotal and more scientific assessment.

 

Frank


  • Susanna144 likes this

#43 Jerry Lodriguss

Jerry Lodriguss

    Vendor

  • ****-
  • Vendors
  • Posts: 7,797
  • Joined: 19 Jul 2008
  • Loc: Voorhees, NJ

Posted 04 October 2018 - 06:28 PM

Hi Frank, I think that information should be very helpful for anyone who actually wants to go to the not inconsiderable trouble of actually doing the tests.

Thank you for contributing that in the spirit of science.

I wish I had energy these days to undertake such a project.

Jerrt

#44 James Paulson

James Paulson

    Vanguard

  • -----
  • topic starter
  • Posts: 2,014
  • Joined: 09 Mar 2014

Posted 15 October 2018 - 07:59 AM

A number of years ago, like about 10, I read that turning off the camera for about the same length of time that your exposure was would make a difference in noise.  At the time I had a Canon 300 D which has amp glow from that warm place.  I tried it and compared to subs taken successively and it did make a big difference.  I did not continue that practise as it was a bit of PIA as I was using the Canon software to run the camera and would have to restart it everytime. I see the same effect now with the T3i - each successive exposure (with 5 sec interval) is noisier than the previous.  The effect is obvious when you view all the frames at once in something like Canon’s Digital Photo or ZoomBrowser.  That being said, LENR would seem to be a better match than a series of darks taken before or after a series of exposures.  

A dark library taken in the same fashion as the lights (temp, duration, interval) with a master dark could approach this method in effectiveness but I would need to see a comparison test.  Does the sensor generate heat when not collecting data?  If not, extending the interval could be useful without having to turn off the camera.  In the interests if full disclisure, I started the thread about LENR on the DSLR forum.  And yes, both James and I were attendees at an Alan Dyer workshop where Alan did state that he and Jerry have disagreed on LENR vs more data.  

I was doing the same thing Dwight, that is, providing an interval of the same duration as the exposure to allow the camera sensor to cool prior to shooting the next frame, doing darks the same way later in the evening and then subtracting the darks and integrating it all with DSS. The workshop showed a whole new process of using LENR, not worrying about the interval so much because you are subtracting the noise on a frame by frame basis and even stacking with Photoshop instead of DSS.

 

Both methods will work. The answer to which works better lies in testing by scientific methods. It would make an interesting test to do this with identical gear, identical targets, identical exposures, and control the variables.

 

I see that John is using LENR with success in his imaging. I am willing to try this for my own purposes, although my camera is a bit older (1000d) and just stack the images with DSS like I always have and see the results.

 

Someone needs to do a field test like I proposed above. That is the only way we can truly know.



#45 900SL

900SL

    Vostok 1

  • -----
  • Posts: 117
  • Joined: 11 Apr 2021

Posted 14 July 2021 - 05:00 AM

This is of particular interest to me being currently resident in Saudi

 

I'm heading out this weekend and the evening temperatures will be 95F falling to 90F by midnight

 

D5600 unmodded on a Fornax LT2, SY 135mm f2

 

I usually shoot a bunch of lights only, typically 45s-90s. I might try LENR to see if it results in cleaner subs for a few test shots beginning and end.   

 

I'll also try for some wide angle shots of the MW D610 20mm f1.8 @ f2.8  ISO 800  128s  

 

I guess the effectiveness will depend on a bunch of factors including the amount of thermal noise and hot pixels, how that varies over time and probably most importantly how the camera firmware processes the dark frame and how this is subtracted from the light. 


Edited by 900SL, 14 July 2021 - 05:11 AM.


#46 jonnybravo0311

jonnybravo0311

    Mercury-Atlas

  • *****
  • Posts: 2,951
  • Joined: 05 Nov 2020
  • Loc: NJ, US

Posted 14 July 2021 - 09:32 AM

Will be interesting to see the results. I'm of the opinion that I can take my darks pretty much anytime, anywhere, so why waste the clear night sky? If you assume that you can image uninterrupted for 6 hours and you take 60 second exposures, you'll either get 360 light frames or 180 light frames if you use the LENR - assuming no time lost between captures. With signal increasing linearly and noise increasing as a square root, it just seems to me that I'm better off gathering as many lights as I can to get the best SNR possible.

 

Jon Rista wrote an extremely good article on signal and noise here: https://jonrista.com...phy-basics/snr/



#47 klaussius

klaussius

    Apollo

  • *****
  • Posts: 1,423
  • Joined: 02 Apr 2019
  • Loc: Buenos Aires

Posted 14 July 2021 - 10:37 AM

All this discussion makes me think that a better way to deal with all this, is to improve the cooling on the camera.

 

Some people add active cooling to DSLRs, but that's a tricky mod. Maybe a less invasive mod that simply improves airflow to avoid uneven heat gradients, a heat spreader here and there, may bring a bigger benefit than anything else.

 

I remember from my DSLR days, that I discovered that by merely turning off the screen I could cool the camera down by several degrees and extend battery life, and it made a bigger impact on my images than tuning subexposure length or ISO did.

 

So, perhaps, drilling a few holes in the back cover might make a big impact as well. I'm not going to drill my DSLR, I found a better solution in the form of a cooled CMOS, but worth giving it some thought, especially if you're one of those willing to mod their camera.


Edited by klaussius, 14 July 2021 - 05:57 PM.


#48 freestar8n

freestar8n

    Vendor - MetaGuide

  • *****
  • Vendors
  • Posts: 10,984
  • Joined: 12 Oct 2007
  • Loc: Melbourne, Australia

Posted 14 July 2021 - 05:50 PM

This is of particular interest to me being currently resident in Saudi

 

I'm heading out this weekend and the evening temperatures will be 95F falling to 90F by midnight

 

D5600 unmodded on a Fornax LT2, SY 135mm f2

 

I usually shoot a bunch of lights only, typically 45s-90s. I might try LENR to see if it results in cleaner subs for a few test shots beginning and end.   

 

I'll also try for some wide angle shots of the MW D610 20mm f1.8 @ f2.8  ISO 800  128s  

 

I guess the effectiveness will depend on a bunch of factors including the amount of thermal noise and hot pixels, how that varies over time and probably most importantly how the camera firmware processes the dark frame and how this is subtracted from the light. 

I think a key factor is how uniform and repeatable the temperature profile across the sensor is over time.  If it isn't repeatable and is session and exposure dependent then there is no reason to expect dark matching to work better compared to LENR.

 

At the same time, losing exposure time isn't definitively worse if in fact the net SNR in an exposure with LENR is better than two exposures that were less perfectly calibrated with scaled darks, rather than fresh darks.

 

There are many factors involved, and camera thermal behavior is different - so it really needs experiment to tell which is better.  But the simple fact of losing signal by taking darks instead of lights isn't necessarily bad if the SNR ends up better in a given amount of imaging time using LENR.

 

Frank


  • B 26354 likes this

#49 TelescopeGreg

TelescopeGreg

    Soyuz

  • -----
  • Posts: 3,745
  • Joined: 16 Jul 2018
  • Loc: Auburn, California, USA

Posted 14 July 2021 - 09:57 PM

What Frank said.

 

If you can't match up the lights and darks because of wild temperature swings, or have lots of extra time on your hands, I think LENR can be an option.  I tried using it when I was first starting out (before I knew the "right" way to do things), and did notice that the LENR subs seemed to have better color.  But getting only half as many did have its cost, and once I understood how to do things, LENR got turned off for good.

 

But thanks for the question.  Some times we need to go back and revisit these things, as there can be situations were doing things "wrong" can be the right answer.


  • freestar8n likes this

#50 900SL

900SL

    Vostok 1

  • -----
  • Posts: 117
  • Joined: 11 Apr 2021

Posted 15 July 2021 - 12:51 PM

First up, Nikon D610, ISO 800 120s exposure

 

I left the camera outside for an hour to get up to ambient (about 38C)

 

Took 5 exposures with no NR

 

Then 1 exposure with NR

 

Took the last two frames for comparison (the last no NR, then the NR)

 

Bumped up to 9 exposure in Affinity photo

 

See if you can guess which is which

 

 

Attached Thumbnails

  • LENR.jpg
  • No LENR.jpg

  • B 26354 likes this


CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.


Recent Topics






Cloudy Nights LLC
Cloudy Nights Sponsor: Astronomics