10 replies to this topic

[Phisci]

I have a Lenovo Legion y530 i7 8750H laptop; 512GB NVME SSD and 2TB HDD; Running Windows 10. 

 

I started at 16GB ram and got the following with a 4GB ramdisk using PI Benchmark

Overall 10400; CPU 9535 and swap of 16883 (4GB ramdisk)

 

At 32GB RAM I got

overall: 11163; CPU 9957 and swap of 22540 (16GB ramdisk)

 

I expected more. Am I missing something? I got more improvement simply by shutting down Chrome. 

 

In both cases I used 4 instances of SSD and 4 instances of ramdisk in swap storage directories.

 

Running EZ denoise gave same results (15min) on 16GB (4GB swapdisk) and 32GB RAM (16GB swapdisk)

 

Thanks for any insights!

Ryan 

[AstroBrett]

Chrome is a memory hog, so I shut it down before I process using PI.

 

I can't comment on your other issues as I use a 120 GB SSD for my system.

 

Brett

[PirateMike]

It is true that PI requires a lot of computer power to run quickly. No fault of the program, it just needs to do a lot of calculating.

 

To see what level of performance a similar computer to yours should deliver... https://pixinsight.com/benchmark/

 

It's quite possible that your computer is running PI as well as can be expected.

 

 

 

Miguel   8-)

Edited by PirateMike, 03 March 2021 - 03:30 PM.

[rgsalinger]

Try taking a look at the performance monitor that comes with Win10 and see if you can identify the bottleneck in your system.

 

I think that you'll find that the PC bus is the limiting factor once you make sure that you have a really fast CPU. I have 64gb of memory and PI never gets above 32 or so which tells me that I'm running as disk or bus speed. 

 

Rgrds-Ross

[jdupton]

Ryan,

 

   Your results on running EZ_Denoise are as I would expect. The PI Benchmark results are not indicative of real world processing speed, especially when using a RAMDisk. In your EZ_Denoise example, the RAMDisk is likely only accessed one time for a single partial frame write to each defined swap directory. It is not used more than once for a single process so it has little effect on timed results.

 

   The place to gauge your improvement from adding more RAM to your system is the Image_Integration process. You can try running it with different numbers of subs to integrate. For a few subs, you will not see an increase in performance going from 16 GB to 32 GB. If you try running the integration with more and more subs, you will find that all of a sudden, you start to notice the effects of the extra RAM.

 

   To run this experiment, create a 16 GB RAMDisk. (Image_Integration does not use it. It only takes away 32 16 GB of otherwise usable RAM.) After testing something like 30, 100, 300, and 600 subs in Image_Integration, destroy the RAMDisk so that the RAM is freed up and repeat the experiment using the full 32 GB. You will likely see a sizable improvement in the speed of the larger integrations once the RAMDisk has been destroyed.

 

   Running a RAMDisk doesn't help but rather hurts the speed of preprocessing steps that are RAM and CPU intensive like Image_Integration. Preprocessing steps that create lots of changed files, like Image_Calibration, are most affected by the storage subsystem -- the type of drives used for your images. RAMDisk is only used by PI at the beginning or end of a processing step that changes an image in place -- thinks like histogram stretches, curves, denoise processes, and such. The RAMDisk is only used when PI needs to be able to Undo some operation you just performed.

 

   More details are available in a thread I started a few months ago. Check out the following thread if you want to better understand how PI uses a RAMDisk and why it is great for the Benchmark bragging numbers but may not help your day-to-day image processing activities. The same thread also explains when and why adding more more RAM can help processing and also touches on how to speed up PI by changing (or adding) different types of disk storage.

 

https://www.cloudynights.com/topic/747617-thoughts-on-pixinsight-use-of-ramdisk/

 

 

John

Edited by jdupton, 03 March 2021 - 06:46 PM.

[joeytroy]

Ryan,

 

Check out my post I made on speeding up PixInsight. If you check the last thread I posted #14 your best bet is moving to Linux I show the speed increases from OSX to Ubuntu. I can't believe the speed difference just by changing OS's?

 

https://www.cloudyni...-up-pixinsight/

[dhaval]

PixInsight eats up CPU cores, RAM and I/O. The more  you can throw at it, the better off you will be. Linux helps. The difference between a Windows and Linux machine is around 5-10%. The offset can be lowered (on Windows side) by adding more RAMDisk, but there is a cost associated with that as John points out. 

 

I am building a system right now with the following specs - 

 

CPU - AMD Threadripper 3960x (24-cores with 48 threads). I won't be over-clocking the CPU. I haven't found a real need for that.

RAM - 128GB @ 2933MHz, once again, not going to overclock it.

GPU - I am going to be using my old GPU (GeForce 1030). I don't use Starnet, and PI does not use CUDA cores for anything else. So, there will be no real loss in compute power for me.

Hard Drive - 2TB M2.2880 SSD. 

OS - Kubuntu 20.10 (I might fall back on Ubuntu, but I want to give KDE Plasma desktop a shot). 

 

Now, to peel the onion a bit more - 

 

As you can imagine - I/O speed helps when you read/write files. Especially if you have a camera that creates 120MB files - this is really important. For cores/threads to be useful, the underlying process needs to use multi-threading. ImageIntegration uses multi-thread processing. If you have multiple files, each file will be given its own thread for processing. ImageIntegration also uses a lot of RAM just because each thread, after processing each file, has to pass back that information back to the "mother-thread" - if the mother-thread is busy, that information then has to be cached in the RAM. If you have a camera with a large number of small pixels, the RAM gets used pretty fast - the more you have, the better off you will be.

 

CS!

[sn2006gy]

I run PixInsight on a Ryzen 1700 32gb, Ryzen 3900x 64gb and an EPYC with 64 cores and 512gb of ram.  Outside of benchmarks i notice 0 difference in windows/linux so i advocate use what you're familiar with.

 

For the most par the 3900x 64gb is a sweet spot price/perf but i find myself using the epyc since i've been doing longer integrations where even 64gb of ram isn't enough.

 

As for GPU, i hope it comes soon

[jdupton]

Byron,

 

   I agree, although I think the sweet point has moved recently. I would say the Ryzen 5900X or 5950X with 64 GB of DDR4 3200 DRAM paired with an X570 motherboard and PCIe 4.0 NVMe drives is now the sweet spot for price / performance. If there is no budget involved, I do like the latest generation Epyc systems, though.

 

 

John

[sn2006gy]

Byron,

 

   I agree, although I think the sweet point has moved recently. I would say the Ryzen 5900X or 5950X with 64 GB of DDR4 3200 DRAM paired with an X570 motherboard and PCIe 4.0 NVMe drives is now the sweet spot for price / performance. If there is no budget involved, I do like the latest generation Epyc systems, though.

 

 

John

Those 5900/5950x's are so rare - good for AMD, but man I can't wait for supply chain to catch up!

[Phisci]

thanks for all the valuable points, will go have a read of the links mentioned.

Ryan