31 replies to this topic

[RedLionNJ]

Hi Grant,

 

I'm sorry that you wasted your time in getting my code to do something meaningful on your computer. uage with good parallel support.

 

Btw.: This weekend I wrote a detailed documentation of the algorithmic details of my code and put it on Github. If you want to see how the stacking program works (without having to read the source code), perhaps this PDF is interesting.

 

All the best,

 Rolf

No worries, Rolf - it was hardly a waste - I learned a few things along the way. Always a good experience. Probably better for the rest of the day's chores I did fail, actually. Else I'd likely have spent the entire Sunday sitting here at my computer, experimenting with it!

 

I just read the documentation - nicely-done.

 

One question, though - when handling RGB image data, is the stacking done in a similar way to Emil's method, where you take advantage of the Bayer matrix to build up a full-resolution image, only adding in data of one specific color for each pixel per frame?

 

Cheers,

 

Grant

[John Duchek]

After installing several of these I am getting the same error as the other user.

Even then, after all that, I get an error about (rightly) not being able to locate a DLL (mkl_rt.dll).

Since I am in linux, I don't think I will be installing a dll soon. 

 

As to my comment about siril having the stack button greyed out, next time I tried the program it worked fine.  go figure.

 

John

[Rolf]

Hi John,

 

After installing several of these I am getting the same error as the other user.

Even then, after all that, I get an error about (rightly) not being able to locate a DLL (mkl_rt.dll).

Since I am in linux, I don't think I will be installing a dll soon. 

 

As to my comment about siril having the stack button greyed out, next time I tried the program it worked fine.  go figure.

 

John

I really cannot help you in getting the Python libraries installed correctly on your Linux system. I have installed all the libraries I listed in my previous note on an Ubuntu 16.04 LTS, and everything worked without any problems.

 

Best wishes,

 Rolf

[Rolf]

Hi Grant,

 

One question, though - when handling RGB image data, is the stacking done in a similar way to Emil's method, where you take advantage of the Bayer matrix to build up a full-resolution image, only adding in data of one specific color for each pixel per frame?

 

Cheers,

 

Grant

Thank you very much for this suggestion! At the moment stacking is done pixelwise for all three colors. That's also how all three colors of an image are stored. Of course one could do as you suggested. This could save operations at the cost of a more complicated data access pattern. If this really helps in terms of performance would have to be tried out. (This part of the algorithm is not performance-critical, anyway.)  It could help to reduce RAM usage, though, because the color images would only occupy a third of the memory compared to the current scheme. We will think about it!

 

Best wishes,

 Rolf

[RedLionNJ]

Hi John - I found the Intel MKL functionality is available via an "add on" download here:

 

https://software.see...mance-libraries

 

 

It looks like there's a version for Linux as well as one for Windows.

 

Grant

[TorstenEdelmann]

 For performance reasons I certainly would not choose Java. Stacking is really performance-critical, and an improvement over Python + Numpy seems to me possible only if I use a compiled language with good parallel support.

 

 

Hi Rolf,

 

First of all I'm quite impressed how far your project has evolved since the last time we emailed.

 

I can't contribute much but just a note regarding performance: when I started with FC I did a lot of performance testing between Java and C++ just to figure out when and which I should run parts of the code on native side using JNI. On those tests it was obvious that the difference in performance was very little.

 

When it comes to performance you really should consider using GPU acceleration as this IMO offers by far the most potential, much much more than just the decision to choose C++ over Java or whatever language. So one idea whould be making your stacking tool CUDA compatible offering NVIDIA users an option to run the code on the GPU. There are a couple of tools available for reimplementing algorithms on GPU (e.g. OpenCV CUDA). At the end it needs to be tested how good the stacking code could be adapted to running efficiently on the GPU but in case it works speed improvement is astounding.

 

Torsten

Edited by TorstenEdelmann, 17 January 2019 - 01:40 AM.

[Rolf]

Hi Torsten,

 

Hi Rolf,

 

First of all I'm quite impressed how far your project has evolved since the last time we emailed.

 

I can't contribute much but just a note regarding performance: when I started with FC I did a lot of performance testing between Java and C++ just to figure out when and which I should run parts of the code on native side using JNI. On those tests it was obvious that the difference in performance was very little.

 

When it comes to performance you really should consider using GPU acceleration as this IMO offers by far the most potential, much much more than just the decision to choose C++ over Java or whatever language. So one idea whould be making your stacking tool CUDA compatible offering NVIDIA users an option to run the code on the GPU. There are a couple of tools available for reimplementing algorithms on GPU (e.g. OpenCV CUDA). At the end it needs to be tested how good the stacking code could be adapted to running efficiently on the GPU but in case it works speed improvement is astounding.

 

Torsten

Thank you very much for your comments!

 

I think that with Python and the underlying Numpy framework I can get quite close to C performance inside those library functions for array operations. I see the main performance problem in unneccessary memory copies and temporary objects kept by Python between such operations. Since Python (as well as Java) does not have an explicit memory management, at the application programming level I can hardly avoid that. Similarly, multi-core control is of course provided by Numpy, but possibly one could do it more efficiently outside the innermost operation blocks. Again, this is sonething I would like to look at in a language like C or Fortran.

 

I already considered exploiting the power of graphics cards. Here at my institute we have several projects where that has been done successfully. (So I have the experts at hand who can tell me how to do it.) On the negative side, however, it makes code deployment and portability more complicated. So, this topic is not of highest priority right now, but I will keep an eye on it.

 

All the best,

 Rolf