13 replies to this topic

[nathanm]

Focus stacking software is used by macro and microscope photographers (sort the opposite of astrophotography) as well as some landscape photographers, to create an apparently infinite depth of field.   In use, one takes a series of photos - up to 1000 or more - each with slightly different focal point.  The software finds the in-focus parts of the picture and combines them.    

 

Here are some leading examples https://www.helicons.../helicon-focus/   and http://zerenesystems.com/cms/stacker.

 

I am wondering if anybody has tried this for planetary imaging?   The problem is similar in some ways - focus stacking software does registration - but it has the additional aspect of looking for sharp regions.   There are several algorithms that are used.

 

This would seem to be a possible candidate for putting together the best planetary shots to do a sort of lucky imaging.   In that case partial depth of field  is not the problem - it is blurring of portions of the image by atmosphere effects.  But the effect should be similar.

 

To be clear - I am not suggesting that the focus be changed between shots - just that one takes a sequence of planetary photos and find out if the software does a better job than conventional planetary image stacking.  

 

However, if you did rack focus and combine photos at slightly different focus points, it would allow correcting field flatness, or tilted field issues.

 

I will hasten to point out that this is speculative on my part.  But I am not the only one.  While looking for examples, I found this thread on DPReview https://www.dpreview.../thread/4250969 , which shows a lunar image.

 

Now, it may well be the case that this does not work, but I if I had a bunch of planetary images to stack, I would try this.   

 

Since I don't, I wonder if anybody else here has tried it, and if not, then I am making the suggestion that it might be worth trying.

 

 

 

 

 

 

[Tom Glenn]

There is no equivalent to focus stacking as it exists for macro photography.  However, a recording for planetary imaging usually last several minutes in duration.  During this time, you can absolutely adjust the focus if you wish.  This is generally only done when conditions are very intermittent, and so you aren't entirely sure of the focus when you begin the recording, but then conditions seem a bit calmer after it begins.  In this case, you could quickly rack through focus and try to reacquire best focus.  Some frames will be completely lost during this procedure due to blurring, but only a minority of frames will be stacked in any case.  The stacking software (usually Autostakkert) selects frames based upon contrast, so it will always select the ones with best focus if you attempted this method.  In fact, when stacking planetary or lunar images, Autostakkert analyzes parts of individual frames (alignment points) independently, and so the final result is already a selection of the best parts of the entire video.  In practice, most people don't adjust the focus during the capture, but instead take a series of captures, each of several minutes duration, and reacquire focus before each new recording.  The final results will almost never be equivalent, with some images looking sharper than others.  Some of this is due to seeing conditions improving between captures, and some of it could be caused by achieving a better focus.  In essence, your question is already the basis of "lucky imaging", and so this is already done, although it's not focus stacking as the term is used in other areas of photography.  

[nathanm]

I understand that it is different, but my speculation - and the result of the one thread on DPReview, are that the focus stacking algorithms might have some value here.     

 

In general I think that the focus stacking algorithms will be different than a program like AutoStakkert.   Yet the problem is close enough that i wonder how the two compare to each other.   There might be cases where the focus stacking algorithms pull out some detail in a better way than AS.

 

Indeed, the DPReview thread I found claims to have gotten better results with Zerene than AutoStakkert. 

 

Note that I am not making that claim generally, and from the small image in the older thread I couldn't conclude that.    But it is what that one guy found and he isn't a planetary imaging expert like the people here on CN.

 

There is no way of telling unless one tries it, which I will at some point.  However, given the DPReview thread I thought there was a possibility somebody at CN would have tried it.

[Ron359]

I  used the focus stacking routine in Affinity Photo to combine about 11 images (taken w/ 7D DLSR) of an ISS transit of the full moon. It worked well, but all the images were 'in-focus' and so brief an interval of about 1 sec. there was no blurring from the seeing.  It was a very quick way to stack multiple frames showing the ISS crossing the moon.   Sure beat making multiple layers as you might do in PS.   But I don't think it would work for the 'normal' planetary imaging where the image details and focus is constantly shifting due to the seeing.   Sorting, ranking of the best frames then aligning and stacking at the pixel level is beyond the function of focus stacking  which assumes the imaged point of pixels do not move or shift at all in 2 dimensions.   

[Tom Glenn]

I understand that it is different, but my speculation - and the result of the one thread on DPReview, are that the focus stacking algorithms might have some value here.     

 

In general I think that the focus stacking algorithms will be different than a program like AutoStakkert.   Yet the problem is close enough that i wonder how the two compare to each other.   There might be cases where the focus stacking algorithms pull out some detail in a better way than AS.

 

Indeed, the DPReview thread I found claims to have gotten better results with Zerene than AutoStakkert. 

 

Note that I am not making that claim generally, and from the small image in the older thread I couldn't conclude that.    But it is what that one guy found and he isn't a planetary imaging expert like the people here on CN.

 

There is no way of telling unless one tries it, which I will at some point.  However, given the DPReview thread I thought there was a possibility somebody at CN would have tried it.

The DPReview thread you linked is a post showing a small image of the Moon, generated by someone who is not a lunar imager.  23 frames were stacked, and the version that he claims is "better" with Zerene is actually worse, appearing over processed.  The version with Autostakkert is actually better, and more natural looking.  But these are very small images of the Moon, and only 23 frames were stacked.  Typically, planetary and lunar images are generated by first collecting many thousands of frames, and programs like Autostakkert are quite sophisticated at handling this data, far beyond what most people realize.  Thousands of frames are analyzed, and in fact each frame is broken down into a subset of APs, which are analyzed, aligned, and stacked independently, and then the final image is reconstituted with pixel warping algorithms.  The program is state of the art for amateur planetary and lunar imaging, and is not trivial in its operations.  In fact, there is a member of this forum (Rolf) who has a longstanding project of creating an open source stacking software with comparable function to Autostakkert.  He has made excellent progress thus far, but it hasn't been easy, and he is an expert in this field.  There exist several potential methods to stack frames for image processing, but because of the many unique challenges in solar system imaging, these are all highly specialized programs, that are outside the scope of image processing that you will encounter in normal photography.

[Tom Glenn]

 But I don't think it would work for the 'normal' planetary imaging where the image details and focus is constantly shifting due to the seeing.   Sorting, ranking of the best frames then aligning and stacking at the pixel level is beyond the function of focus stacking  which assumes the imaged point of pixels do not move or shift at all in 2 dimensions.   

This pretty much sums it up.  In focus stacking, such as for macro photography, the image is completely static, but the focal plane is narrow and moving across a deep target as you change focus.  In solar system imaging, the image is actually moving because of turbulence, and also the entire image is contained in one focal plane.  There is no depth to the image, but there are a series of challenges related to turbulence that require specialized methods.  And as I mentioned above, the stacking itself is not even simple frame averaging, because each frame is broken apart to account for differential sharpness across the frame, and the best individual pieces are then recomposed using local pixel warping methods.  

[nathanm]

I'm going to stop pushing this, but just will point out some facts.

 

Focus stacking is nothing like you think - the algorithms are very complicated - much more so than at least some planetary stacking programs.  I don't know AutoStakkert, but the open source PSS approach discussed by Rolf another CN thread is very simplistic, and it is apparently still useful.   

 

That isn't meant to be negative comment at all- if it works great!   And Rolf's thread says he wants to do more but he is writing in Python so he is performance limited.  

 

Focus stacking is a set of very sophisticated image processing steps, and is not at all the simple thing you think it is.  

 

The pixels in focus stacking are not static - they move with focus.  You can see this if you look through a camera and rack the focus.   Indeed each focus point is actually a different focal length, and therefore (at macro scales) is at a different magnification.  

 

It is true that in planetary imaging the image is in the same plane of focus, but some parts are - in effect - defocused because of the lensing effect of atmospheric turbulence can obscure all or parts of the image.  If it is all, then there is not much point, but if it is only some parts of an image then that is where having a sophisticated algorithm for determining what parts are sharp (high contrast) could be useful.   

[Ron359]

I'm going to stop pushing this, but just will point out some facts.

 

Focus stacking is nothing like you think - the algorithms are very complicated - much more so than at least some planetary stacking programs.  I don't know AutoStakkert, but the open source PSS approach discussed by Rolf another CN thread is very simplistic, and it is apparently still useful.   

 

That isn't meant to be negative comment at all- if it works great!   And Rolf's thread says he wants to do more but he is writing in Python so he is performance limited.  

 

Focus stacking is a set of very sophisticated image processing steps, and is not at all the simple thing you think it is.  

 

The answer is pretty simple.   If you think focus stacking will or should work, go ahead and use it.  Let us know what you find.     

[Tom Glenn]

The answer is pretty simple.   If you think focus stacking will or should work, go ahead and use it.  Let us know what you find.     

+1 to this.  Imaging is an empirical exercise.  If something works well, people use it.  I can understand why the OP might think we are being dismissive of a method here, and I didn't mean to imply that focus stacking is "simplistic".  However, we get a number of threads around here in which people try to reinvent the wheel, or make claims about what techniques might work well in planetary imaging when they don't have any clue what they are talking about.  So this also gets frustrating.  Planetary imaging has almost nothing in common with terrestrial photography.  Focus stacking is meant to recover a deep depth of field from a series of images that have narrow depth of field.  In planetary and lunar imaging, it is actually quite simple: There is no depth of field, and a frame is either in focus or it isn't.  If it isn't, or if it is distorted from turbulence, then it should be discarded.  Only the sharp frames are chosen, and then stacked.  By all means if anyone has any ideas on how to improve results, then by all means try and report back.  

[nathanm]

If you think focus stacking will or should work, go ahead and use it.  Let us know what you find.

 

Well, my first step was to see whether it had already been tried.  

 

Step two is to try it myself.   I would love to try it but at the moment I am equipped to take decent planetary shots.   

 

Sometimes people put a set of raw images up for download - for people to practice software processing on, or part of a tutorial on how to use software.   

 

If there is some sequence of planetary shots like that you can point me at, I would be glad to give it a try with a couple different focus stacking algorithms.

[Tom Glenn]

In order to better understand our skepticism here, I think you need to understand how planetary images are processed, if you are not already familiar.  A raw video is acquired, typically corresponding to between 20,000 and 100,000 frames.  Dimmer planets such as Saturn may only have 20,000 frames (or less), due to shutter speed and frame rate requirements, but brighter planets such as Mars can exceed 100,000 frames in a single video.  The stacking software, such as Autostakkert, then takes this raw video files, analyzes each frame, stabilizes them for motion, and ranks them according to contrast, which effectively serves as a quality score.  The frames are then broken apart further into alignment points, ranked again, stacked, and spliced together to form the final image stack.  So, the final image may represent a stack of 5000 out of  30,000 frames, or similar numbers.  

 

So my question is this....is the focus stacking software prepared to handle tens of thousands of frames of raw video, extract the individual frames, and then perform this operation? 

Edited by Tom Glenn, 29 January 2021 - 07:34 PM.

[Tom Glenn]

There have been several examples of raw data posted here for people to practice processing.  One that comes to mind is some excellent Jupiter data from Darren (DMach).  See link below.  As you can see, the raw video files are many GB in size each.  

 

https://www.cloudyni...to-a-good-home/

[nathanm]

OK, here is a quick result.

 

I downloaded the SER files.  They each have an info file that starts like this:

 

FireCapture v2.6  Settings
------------------------------------
Camera=ZWO ASI290MC
Filter=L
Profile=Jupiter

 

There are 3 of them, so maybe they are R, G, B, but they each say L, which I take to be Luminance.   The ZWO ASI290MC is a monochrome camera.

 

But without knowing which is R,G or B, my results are in monochrome as a result - I didn't know what else to do frankly.  Sorry if I am missing something elementary.

 

I took the first SER file,   Jup_012639.ser , and exported the images as TIFF with SIRIL.  There are 7202 images in that file.

 

But to be simplistic I took the first 100  (i.e. Jup_00000.tif  to Jup_00099.tif) and I loaded them into Helicon Focus, using algorithm C - which uses image pyramids.

 

Here is a comparison.   The DMach image integrated with AS!3 is the color image on the left, the Helicon Focus image is on the right.

 

The DMach image has been reduced by a factor of 3X in photoshop - i.e. to its original resolution, since it is a 3X drizzle.  I should have increased the resolution of the TIFF files before stacking with Helicon but that's time consuming so I skipped it.   The Helicon result might well be improved by scaling before combining.   

 

 

 

Helicon did this very fast - like 20 seconds.  It is heavily GPU optimized, and that is probably why it is so fast.

 

I think that the result is not too bad under the circumstances.   The DMach image is color, of course, and uses the best 30% of the frames as I recall.  I think that is across all three SER files.   So that is about 6300 frames  across 21,000.  Meanwhile Helicon is the just the first 100 frames.

 

Now, it's clear that the DMach image is in color, and I think it is also much better.   But, for a quick draft, Helicon seems to do a decent job.

 

I tried putting all 7200 files into Helicon and that did not work, perhaps because of rotation of Jupiter.   So using a focus stacking program might be limited to short sequences.

 

Anyway, there is the test.   Since I am not a planetary imager, and made this post just out of curiosity, I am not planning on pursuing this further.  Thanks for indulging my crazy idea.

[Tom Glenn]

The problems you are encountering are all completely expected, but I understand your interest and curiosity.  The ASI290MC is a color camera, but the raw frames need to be debayered.  The bayer matrix is still visible (in parts of) your result on the right, and must be removed from the frames before any type of stacking.  Autostakkert debayers the frames before stacking, and it does so in a superior way to standard debayering algorithms, because it makes use of all of the available frames using a drizzle algorithm, taking advantage of the slight movement between frames.  The other issues related to limitations on frame number are also problematic.  One needs to be able to sort the frames by quality, and then select a percentage of the best to stack.  You could likely devise a way to use the focus stacking software to do a better job than what is represented above, although given the limitations it is unlikely to serve as a viable alternative to the currently used methods.  However, your post has been intriguing, and does make me want to learn more about how the focus stacking software works, simply out of general interest, although I don't see much applicability to planetary imaging.  

Edited by Tom Glenn, 30 January 2021 - 04:17 PM.