Jump to content

  •  

CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.

Photo

Looking Forward to my Dedicated PI Box

  • Please log in to reply
28 replies to this topic

#1 Peregrinatum

Peregrinatum

    Viking 1

  • *****
  • topic starter
  • Posts: 510
  • Joined: 27 Dec 2018

Posted 19 November 2019 - 03:21 PM

Can't wait to run PI on my new (refurb) box, got tired of "out of memory" fails during image integration and LN taking 90' to process! Not the fastest out there but it fit my budget and will be a significant upgrade to my i5 processor, spinning HD, and 8GB DDR3 ram

 

I'll install PI on the SSD along with the current imaging project, everything else to go on the 1TB drive.

 

Any tips to make this faster with specific settings?

 

Lenovo IdeaCentre 720 Desktop Computer

8-Core AMD Ryzen 7 1700 (16 threads)

up to 3.7GHz

16GB DDR4

256GB SSD (OS) + 1TB 7200RPM HDD (data)


Edited by Peregrinatum, 19 November 2019 - 03:24 PM.


#2 SilverLitz

SilverLitz

    Ranger 4

  • -----
  • Posts: 304
  • Joined: 17 Feb 2018
  • Loc: Louisville, KY

Posted 19 November 2019 - 03:28 PM

2 Suggestions:

* More RAM to feed the 8-cores

* M.2 MVMe PCIe SSD for PI's swap files


  • Stelios, tcchittyjr, Monkeybird747 and 2 others like this

#3 Hawkdl2

Hawkdl2

    Viking 1

  • *****
  • Posts: 689
  • Joined: 22 Feb 2015
  • Loc: Claremont, CA

Posted 19 November 2019 - 03:38 PM

I agree more RAM would be beneficial.  I would also agree with the NVMe.  You can pick up a 1 TB NVMe for bit more than $125, and they are amazingly faster than a standard SSD and way faster than a HDD.  If you don't want to go that route,  you might consider 32Gb RAM and use 16gm as a RAM drive. 


  • Peregrinatum and Swordfishy like this

#4 Dynan

Dynan

    Gemini

  • *****
  • Posts: 3,221
  • Joined: 11 Mar 2018
  • Loc: NOLA

Posted 19 November 2019 - 05:01 PM

A RAMDISK sped my computer up by 63%. Easy to do.

 

https://sourceforge....imdisk-toolkit/

 

Also, see this thread:

 

https://www.cloudyni...low-can-you-go/


  • Peregrinatum likes this

#5 SilverLitz

SilverLitz

    Ranger 4

  • -----
  • Posts: 304
  • Joined: 17 Feb 2018
  • Loc: Louisville, KY

Posted 21 November 2019 - 04:56 PM

Here is some color on NMVe PCIe SSD compared to SATA III:

 

On my HP Z840 workstation that I have been putting together over the past several weeks to be my PI beast (2x Xeon 2680v3, 24-cores, 48 threads, 160GB DDR4 ECC RAM):

The Z840 came with 2 SATA III SSDs, which I kept one for boot and one for normal documents.  I added two 1TB HP EX920 NMVe PCIe SSDs, one for RAW data, and the 2nd for PI files including PI Swap.

 

Before adding the NMVe PCIe SSDs, PI's swap was going to the SATA SSD and PI Swap Benchmark score was 2,227.  Moving the PI Swap to the NMVe PCIe SSD improved the PI Swap Benchmark score to 5,364.  The move from a normal spinning HD would be even greater.

 

Also, make sure the NMVe PCIe SSD is connected to PCIe 3 slots, not PCIe 2.  On my other computer, HP Z440 had 3 NMVe PCIe SSDs (boot, documents, PI) and a Quadro Graphics Card in another PCIe 3 slot.  The Z440 does not have enough PCIe 3 slots to accommodate all of this, some my PI NMVe PCIe SSD had to go into a PCIe 2 slot.  After removing a NMVe PCIe SSD to my new Z840, I was able to have the PI Swap SSD on a PCIe 3 slot, and the PI Swap Benchmark score improved from 4,720 to 6,480.


  • bsavoie and Peregrinatum like this

#6 Monkeybird747

Monkeybird747

    Viking 1

  • *****
  • Posts: 668
  • Joined: 31 Aug 2017
  • Loc: Louisville, KY

Posted 21 November 2019 - 11:10 PM

If running Linux is an option for you that will significantly boost your processing speed. Peruse the benchmarks at the link below (the one linked is my own machine). All the fastest machines are running Linux. They are grouped by processor, and you will quickly notice that the same processor running windows takes almost twice the time as the Linux builds.

 

Also the number and location of your swap folders makes a difference, so you might research that. In the descriptions of each machine on the benchmark site you can find sometimes find details about how people are configuring their swap directories.

 

http://pixinsight.co...3IF2CWXUJF589W0

 

I opted for the 1950x 16 core running at base clock, but the 12 core 3900x seems to outperform it with its faster processors.


  • Peregrinatum likes this

#7 Peregrinatum

Peregrinatum

    Viking 1

  • *****
  • topic starter
  • Posts: 510
  • Joined: 27 Dec 2018

Posted 22 November 2019 - 11:54 AM

Thanks for the tips, but I am not sure just how to go about setting up the Ram Disk and adding the swap directories.

 

I will have 16MB of Ram, one thread suggests setting up a 4MB Ram Disk and adding two swap directories to it.

 

Another thread suggests adding a swap directory for each core on the SSD, so in my case I should add 8 more swap directories that point to the SSD.

 

Not sure.



#8 Dynan

Dynan

    Gemini

  • *****
  • Posts: 3,221
  • Joined: 11 Mar 2018
  • Loc: NOLA

Posted 22 November 2019 - 01:37 PM

I was leery of adding a RAM drive also, but it turned out to be very simple...and helpful.

 

After you install Ram Disk from the provided link, go to the thread link. It explains it quite well.

 

I added 5 swap directories and got a 63% improvement. I have 32Gb of RAM so I made a 16Gb RAM Disk. (I have NEVER seen my computer use over ≈ 12 Gb of RAM at one time, no matter what I was running...even multiple intense programs simultaneously...so I gave it half my RAM. You might have to figure out how much you can give it with only 8GB RAM...so more RAM would help your computer if you do this.) You can add as many swap directories as you like. But there will be an optimum number, over which, it won't speed anything up any more.

 

In PI choose Edit > Global Preferences > Directories and Network. Add your Ramdrive (Root Directory R:) to the list multiple times. For best results you need to determine how many times is best.


  • Peregrinatum likes this

#9 Cfreerksen

Cfreerksen

    Viking 1

  • *****
  • Posts: 703
  • Joined: 18 Aug 2018
  • Loc: Tooele, Ut

Posted 22 November 2019 - 03:42 PM

Here is some color on NMVe PCIe SSD compared to SATA III:

 

On my HP Z840 workstation that I have been putting together over the past several weeks to be my PI beast (2x Xeon 2680v3, 24-cores, 48 threads, 160GB DDR4 ECC RAM):

The Z840 came with 2 SATA III SSDs, which I kept one for boot and one for normal documents.  I added two 1TB HP EX920 NMVe PCIe SSDs, one for RAW data, and the 2nd for PI files including PI Swap.

 

Before adding the NMVe PCIe SSDs, PI's swap was going to the SATA SSD and PI Swap Benchmark score was 2,227.  Moving the PI Swap to the NMVe PCIe SSD improved the PI Swap Benchmark score to 5,364.  The move from a normal spinning HD would be even greater.

 

Also, make sure the NMVe PCIe SSD is connected to PCIe 3 slots, not PCIe 2.  On my other computer, HP Z440 had 3 NMVe PCIe SSDs (boot, documents, PI) and a Quadro Graphics Card in another PCIe 3 slot.  The Z440 does not have enough PCIe 3 slots to accommodate all of this, some my PI NMVe PCIe SSD had to go into a PCIe 2 slot.  After removing a NMVe PCIe SSD to my new Z840, I was able to have the PI Swap SSD on a PCIe 3 slot, and the PI Swap Benchmark score improved from 4,720 to 6,480.

Is there a performance improvement with xeon processors? Or is it just throw more cores at it?

 

Chris



#10 Peregrinatum

Peregrinatum

    Viking 1

  • *****
  • topic starter
  • Posts: 510
  • Joined: 27 Dec 2018

Posted 22 November 2019 - 05:10 PM

I was leery of adding a RAM drive also, but it turned out to be very simple...and helpful.

 

After you install Ram Disk from the provided link, go to the thread link. It explains it quite well.

 

I added 5 swap directories and got a 63% improvement. I have 32Gb of RAM so I made a 16Gb RAM Disk. (I have NEVER seen my computer use over ≈ 12 Gb of RAM at one time, no matter what I was running...even multiple intense programs simultaneously...so I gave it half my RAM. You might have to figure out how much you can give it with only 8GB RAM...so more RAM would help your computer if you do this.) You can add as many swap directories as you like. But there will be an optimum number, over which, it won't speed anything up any more.

 

In PI choose Edit > Global Preferences > Directories and Network. Add your Ramdrive (Root Directory R:) to the list multiple times. For best results you need to determine how many times is best.

 

OK sounds good, I purchased another 16MB or Ram and I'll use that as a Ramdrive... seems like its a trial and error process with the Benchmark to optimize how many directories to add.



#11 Synon

Synon

    Mariner 2

  • *****
  • Posts: 261
  • Joined: 02 Jun 2012
  • Loc: Flagstaff AZ

Posted 23 November 2019 - 01:14 AM

Is there a performance improvement with xeon processors? Or is it just throw more cores at it?

 

Chris

I run a Xeon currently, not very impressed with it though. High core counts are helpful for processes that can utilize many threads... like calibration, registration, etc where it can work on lots of individual images simultaneously. Once you have a fully integrated image higher single core clock speeds come into play more often. Either way, the new AMD CPU's seem to excel at both and are miles ahead of the Intel Xeon's and i9's for Pixinsight work. 



#12 Cfreerksen

Cfreerksen

    Viking 1

  • *****
  • Posts: 703
  • Joined: 18 Aug 2018
  • Loc: Tooele, Ut

Posted 23 November 2019 - 01:42 AM

 I run a Xeon currently, not very impressed with it though. High core counts are helpful for processes that can utilize many threads... like calibration, registration, etc where it can work on lots of individual images simultaneously. Once you have a fully integrated image higher single core clock speeds come into play more often. Either way, the new AMD CPU's seem to excel at both and are miles ahead of the Intel Xeon's and i9's for Pixinsight work. 

PI really just needs to tap GPU's. Cuda cores would be a real "game changer" for processing. Then just go buy a good graphics card, plug in. Done. Well as well as Ram, SSD aka Rabbit hole.

 

It's never fast enough if I have to watch the spinning circle/tumbling hourglass or .............  now I have "dated" myself. gramps.gif  I want all that time back!

 

Chris


Edited by Cfreerksen, 23 November 2019 - 01:46 AM.

  • Dynan likes this

#13 SilverLitz

SilverLitz

    Ranger 4

  • -----
  • Posts: 304
  • Joined: 17 Feb 2018
  • Loc: Louisville, KY

Posted 23 November 2019 - 04:48 PM

Is there a performance improvement with xeon processors? Or is it just throw more cores at it?

 

Chris

HP Z's are more for mission critical type of applications that are designed for 24/7/365 use.  They use Xeons for a couple of reasons: use of ECC (error correcting) RAM for greater data integrity and the ability to address massive amounts of RAM (my Z840 can use up to 2TB of RAM; newer Z8's can address 3TB).  These machines are VERY reliable and very expandable.  I like Z's because you can get a lot of high performance machine for relatively low price when buying refurbs.

 

It does NOT seem that PI is actually using both of my CPUs, as my Benchmark report was only using 24 threads.  When I 1st added the 2nd CPU, PI Benchmark (1.8.8) showed about 32% improvement (disappointed, hoping closer to 100%), but after updating to 1.8.8.1 the CPU score fell back to the range before I installed the 2nd CPU.  Hopefully, PI will make changes to use both CPUs.

 

This my everyday PC (Win10) and I normally have PI with a very large project (5-6 different targets with 20Mpx), DSS, Photoshop, 4-5 very large Excel models, JRiver, Chrome w/ 50-90 tabs open, etc... open at once.  I have not yet run PI on the Z840 (other than Benchmark) yet, as I have just gotten it setup.  My prior PC is no slouch, as it is a HP Z440 with a 6-core Xeon E5-1650v4 with 64GB RAM and M.2 MVMe SSDs.



#14 Cfreerksen

Cfreerksen

    Viking 1

  • *****
  • Posts: 703
  • Joined: 18 Aug 2018
  • Loc: Tooele, Ut

Posted 23 November 2019 - 07:31 PM

HP Z's are more for mission critical type of applications that are designed for 24/7/365 use.  They use Xeons for a couple of reasons: use of ECC (error correcting) RAM for greater data integrity and the ability to address massive amounts of RAM (my Z840 can use up to 2TB of RAM; newer Z8's can address 3TB).  These machines are VERY reliable and very expandable.  I like Z's because you can get a lot of high performance machine for relatively low price when buying refurbs.

 

It does NOT seem that PI is actually using both of my CPUs, as my Benchmark report was only using 24 threads.  When I 1st added the 2nd CPU, PI Benchmark (1.8.8) showed about 32% improvement (disappointed, hoping closer to 100%), but after updating to 1.8.8.1 the CPU score fell back to the range before I installed the 2nd CPU.  Hopefully, PI will make changes to use both CPUs.

 

This my everyday PC (Win10) and I normally have PI with a very large project (5-6 different targets with 20Mpx), DSS, Photoshop, 4-5 very large Excel models, JRiver, Chrome w/ 50-90 tabs open, etc... open at once.  I have not yet run PI on the Z840 (other than Benchmark) yet, as I have just gotten it setup.  My prior PC is no slouch, as it is a HP Z440 with a 6-core Xeon E5-1650v4 with 64GB RAM and M.2 MVMe SSDs.

Yes, I am familiar with the HP Zseries. We use them in my industry. Medical imaging. Very nice hardware. We also now use dedicated GPU's for doing heavy computations during image reconstruction, FFT (fast fourier transforms), and image enhancement. GPU's are much more robust for these types of computations.

 

Chris


Edited by Cfreerksen, 23 November 2019 - 07:33 PM.


#15 Peregrinatum

Peregrinatum

    Viking 1

  • *****
  • topic starter
  • Posts: 510
  • Joined: 27 Dec 2018

Posted 29 November 2019 - 11:15 AM

Update: I sent the first box back, it was a refurb and now I know why!  New box came in, Dell Inspiron 5676 Ryzen7v2, 8 cores 16 threads, 16GB Ram, 256GB SSD NVMe boot drive, 1TB spinner data drive, very happy with it the speed increase in PI versus my old box is incredible. I have not yet optimized it yet with PI in terms of a Ramdisk or number of PI threads but going to do that soon, just having fun now being able to process 1200 subs at one time Lol  will play with the opt later.

 

I have this graphics card: AMD Radeon RX 580 4GB Discrete Graphics, it's supposed to be for gaming but I don't do that.  This card has some RAM and a "GPU" whatever that is....

 

Question:  is there a way for PI to use the graphics card RAM and GPU along with the system RAM/CPU at the same time?


Edited by Peregrinatum, 29 November 2019 - 11:22 AM.


#16 tcchittyjr

tcchittyjr

    Vostok 1

  • *****
  • Posts: 106
  • Joined: 10 Dec 2012
  • Loc: Orlando, FL

Posted 29 November 2019 - 11:30 AM

Update: I sent the first box back, it was a refurb and now I know why!  New box came in, Dell Inspiron 5676 Ryzen7v2, 8 cores 16 threads, 16GB Ram, 256GB SSD NVMe boot drive, 1TB spinner data drive, very happy with it the speed increase in PI versus my old box is incredible. I have not yet optimized it yet with PI in terms of a Ramdisk or number of PI threads but going to do that soon, just having fun now being able to process 1200 subs at one time Lol  will play with the opt later.

 

I have this graphics card: AMD Radeon RX 580 4GB Discrete Graphics, it's supposed to be for gaming but I don't do that.  This card has some RAM and a "GPU" whatever that is....

 

Question:  is there a way for PI to use the graphics card RAM and GPU along with the system RAM/CPU at the same time?

At this point, PI doesn't use GPU for anything other than display. There was talk a while back about using nVidia cards, but that hasn't happened.

 

TomC



#17 Cfreerksen

Cfreerksen

    Viking 1

  • *****
  • Posts: 703
  • Joined: 18 Aug 2018
  • Loc: Tooele, Ut

Posted 29 November 2019 - 12:51 PM

HP Z's are more for mission critical type of applications that are designed for 24/7/365 use.  They use Xeons for a couple of reasons: use of ECC (error correcting) RAM for greater data integrity and the ability to address massive amounts of RAM (my Z840 can use up to 2TB of RAM; newer Z8's can address 3TB).  These machines are VERY reliable and very expandable.  I like Z's because you can get a lot of high performance machine for relatively low price when buying refurbs.

 

It does NOT seem that PI is actually using both of my CPUs, as my Benchmark report was only using 24 threads.  When I 1st added the 2nd CPU, PI Benchmark (1.8.8) showed about 32% improvement (disappointed, hoping closer to 100%), but after updating to 1.8.8.1 the CPU score fell back to the range before I installed the 2nd CPU.  Hopefully, PI will make changes to use both CPUs.

 

This my everyday PC (Win10) and I normally have PI with a very large project (5-6 different targets with 20Mpx), DSS, Photoshop, 4-5 very large Excel models, JRiver, Chrome w/ 50-90 tabs open, etc... open at once.  I have not yet run PI on the Z840 (other than Benchmark) yet, as I have just gotten it setup.  My prior PC is no slouch, as it is a HP Z440 with a 6-core Xeon E5-1650v4 with 64GB RAM and M.2 MVMe SSDs.

I took a closer look at the PI benchmarks. The Xeon based PC's don't really do much better than my I7-5820@4GHz OC'd (not bad for an old processor) and they have more cores and dual processors. I decided not to dump money into a new box. Watching how my resources are utilized during the processing script I see most of the hold up is my system using swapfiles, much slower than ram. So with Black Friday I ordered 64GB (max for my hardware) of ram to replace the 32GB that is installed now (couldn't find a match to what was current). I guess my only other avenue would be a M.2 for swapfiles instead of my Samsung EVO SATA.

 

I have given some thought to Ramdisk. If your system runs low on memory it goes to swapfiles. People use a Ramdisk for the the swapfiles. But you use memory for the swapfiles and that reduces your total memory available for the system. Isn't this robbing Peter to pay Paul? I can see using a Ramdisk if you install like an Optane or similar ram extension board but not using ram. I would like to see "real world" testing Ramdisk and not just the benchmark number. If you have enough ram and never have to swapfiles isn't that faster? Thoughts?

 

Chris


Edited by Cfreerksen, 29 November 2019 - 01:02 PM.


#18 Peregrinatum

Peregrinatum

    Viking 1

  • *****
  • topic starter
  • Posts: 510
  • Joined: 27 Dec 2018

Posted 29 November 2019 - 01:08 PM

I took a closer look at the PI benchmarks. The Xeon based PC's don't really do much better than my I7-5820@4GHz OC'd (not bad for an old processor) and they have more cores and dual processors. I decided not to dump money into a new box. Watching how my resources are utilized during the processing script I see most of the hold up is my system using swapfiles, much slower than ram. So with Black Friday I ordered 64GB (max for my hardware) of ram to replace the 32GB that is installed now (couldn't find a match to what was current). I guess my only other avenue would be a M.2 for swapfiles instead of my Samsung EVO SATA.

 

I have given some thought to Ramdisk. If you system runs low on memory it goes to swapfiles. People use a Ramdisk for the the swapfiles. But you use memory for the swapfiles and that reduces you total memory available for the system. Isn't this robbing Peter to pay Paul? I can see using a Ramdisk if you install like an Optane or similar ram extension board but not using ram. I would like to see "real world" testing Ramdisk and not just the benchmark number. If you have enough ram and never have to swapfiles isn't that faster? Thoughts?

 

Chris

I messed around some with the Ramdisk yesterday.  I had 16GB or Ram so I setup a 4GB Ramdisk and all was fine, then I increased it to 8GB and my computer started to freak out, which let me to believe the system needs more than 8GB to be happy.

 

I only have 2 Ram slots, both 8GB.  This morning I took one stick out and put in a new 16GB stick for a total of 24GB of Ram (don't ask about the odd number, but my computer didn't barf over it, I was sure to match the DDR4 sticks in everyway but size, Lol)...  I setup a 12GB Ramdisk leaving 12GB system Ram, and I am having no issues with this setup, and it's running faster than before.



#19 bobzeq25

bobzeq25

    Hubble

  • *****
  • Posts: 17,404
  • Joined: 27 Oct 2014

Posted 29 November 2019 - 02:25 PM

Thanks for the tips, but I am not sure just how to go about setting up the Ram Disk and adding the swap directories.

 

I will have 16MB of Ram, one thread suggests setting up a 4MB Ram Disk and adding two swap directories to it.

 

Another thread suggests adding a swap directory for each core on the SSD, so in my case I should add 8 more swap directories that point to the SSD.

 

Not sure.

Here's the easiest path, and very effective.  What I've done.

 

Add an additional SSD.

 

Put 8 (identical) swap files for the 8 cores.  PI, as always, makes this somewhat unintuitive, but it's easy enough when you know how.  Here's how.

 

Main menu - EDIT.  Dropdown box - Global Preferences.  In the box - Directories and Network.  On the right - Swap Directories.

 

ADD.  Should look like F:\swap.  Do that 8 times (yeah, I know)  Click blue circle lower left (Apply global)

 

Done.

 

And you haven't used any more of your main memory, which is a plus.  A good SSD (I use Samsung Pro) is about as fast as Ramdisk.


Edited by bobzeq25, 29 November 2019 - 02:27 PM.

  • SilverLitz and Cfreerksen like this

#20 Cfreerksen

Cfreerksen

    Viking 1

  • *****
  • Posts: 703
  • Joined: 18 Aug 2018
  • Loc: Tooele, Ut

Posted 29 November 2019 - 03:17 PM

I messed around some with the Ramdisk yesterday.  I had 16GB or Ram so I setup a 4GB Ramdisk and all was fine, then I increased it to 8GB and my computer started to freak out, which let me to believe the system needs more than 8GB to be happy.

 

I only have 2 Ram slots, both 8GB.  This morning I took one stick out and put in a new 16GB stick for a total of 24GB of Ram (don't ask about the odd number, but my computer didn't barf over it, I was sure to match the DDR4 sticks in everyway but size, Lol)...  I setup a 12GB Ramdisk leaving 12GB system Ram, and I am having no issues with this setup, and it's running faster than before.

Did you ever benchmark? Before and after?

 

My current.

 

*******************************************************************************
The Official PixInsight Benchmark version 1.0
Copyright © 2014-2015 Pleiades Astrophoto. All Rights Reserved.
*******************************************************************************

Benchmark version ...... 1.00.08
Input checksum ......... 2cd72b67e12fff2812ef5b5da054ab2a70a25e23
Serial number .......... XXK18M7KG857DXE6JZX7Q328ESREP56F

CPU Identification
CPU vendor ............. GenuineIntel
CPU model .............. Intel® Core™ i7-5820K CPU @ 3.30GHz

System Information
Platform ............... Windows
Operating system ....... Microsoft Windows 10 Pro
Core version ........... PixInsight Core 1.8.8-2 (x64)
Logical processors ..... 12
Total memory size ...... 31.927 GiB

Execution Times
Total time ............. 00:44.33
CPU time ............... 00:37.27
Swap time .............. 00:07.03
Swap transfer rate ..... 2356.472 MiB/s

Performance Indices
Total performance ...... 10613
CPU performance ........ 10157
Swap performance ....... 13051

*******************************************************************************

Benchmark with S/N XXK18M7KG857DXE6JZX7Q328ESREP56F has been recorded on 2019/11/29 05:00:03 UTC

 

 

I am going to see if this changes much with the more memory.

 

 

Here's the easiest path, and very effective.  What I've done.

 

Add an additional SSD.

 

Put 8 (identical) swap files for the 8 cores.  PI, as always, makes this somewhat unintuitive, but it's easy enough when you know how.  Here's how.

 

Main menu - EDIT.  Dropdown box - Global Preferences.  In the box - Directories and Network.  On the right - Swap Directories.

 

ADD.  Should look like F:\swap.  Do that 8 times (yeah, I know)  Click blue circle lower left (Apply global)

 

Done.

 

And you haven't used any more of your main memory, which is a plus.  A good SSD (I use Samsung Pro) is about as fast as Ramdisk.

As well as trying Bobzeq's guidance. Right now I have 3 swapfiles but have 6 physical cores. Bobzeq, is that 1 swap for each physical core or logical core?

 

Chris


Edited by Cfreerksen, 29 November 2019 - 03:17 PM.


#21 Peregrinatum

Peregrinatum

    Viking 1

  • *****
  • topic starter
  • Posts: 510
  • Joined: 27 Dec 2018

Posted 29 November 2019 - 09:58 PM

Did you ever benchmark? Before and after?

 

My current.

 

*******************************************************************************
The Official PixInsight Benchmark version 1.0
Copyright © 2014-2015 Pleiades Astrophoto. All Rights Reserved.
*******************************************************************************

Benchmark version ...... 1.00.08
Input checksum ......... 2cd72b67e12fff2812ef5b5da054ab2a70a25e23
Serial number .......... XXK18M7KG857DXE6JZX7Q328ESREP56F

CPU Identification
CPU vendor ............. GenuineIntel
CPU model .............. Intel® Core™ i7-5820K CPU @ 3.30GHz

System Information
Platform ............... Windows
Operating system ....... Microsoft Windows 10 Pro
Core version ........... PixInsight Core 1.8.8-2 (x64)
Logical processors ..... 12
Total memory size ...... 31.927 GiB

Execution Times
Total time ............. 00:44.33
CPU time ............... 00:37.27
Swap time .............. 00:07.03
Swap transfer rate ..... 2356.472 MiB/s

Performance Indices
Total performance ...... 10613
CPU performance ........ 10157
Swap performance ....... 13051

*******************************************************************************

Benchmark with S/N XXK18M7KG857DXE6JZX7Q328ESREP56F has been recorded on 2019/11/29 05:00:03 UTC

 

 

I am going to see if this changes much with the more memory.

 

 

As well as trying Bobzeq's guidance. Right now I have 3 swapfiles but have 6 physical cores. Bobzeq, is that 1 swap for each physical core or logical core?

 

Chris

 

I got around to optimizing today, I might post graphs of the results if I get time...

 

Tested swap files to Ramdisk from 2 to 16

Tested swap files to SSD from 2 to 16

Tested Ramdisk from 2GB to 16GB

 

This was the best combination and corresponding Benchmark Totals:

 

Old Computer (1 swap file to SSD) = 2,108 (i5 4 core 4 thread, 8GB Ram, 1TB spinner HD)

New Computer, PI as installed (1 swap file to SSD) = 8,619 (8 core Ryzen7v2 16 threads, 24GB Ram, 256GB system drive, 1TB spinner HD)

New Computer, Optimized = 12,589

 

Optimization = 12GB Ramdisk, 8 swap files to Ramdisk, 8 swap files to SSD

 

Testing for optimization is time well spent, gave me ~ 50% increase in performance.

 

Increase in Benchmark Total Old to New computers = 12589/2108 ~ 6x


Edited by Peregrinatum, 29 November 2019 - 10:02 PM.


#22 telfish

telfish

    Vanguard

  • *****
  • Posts: 2,004
  • Joined: 17 Nov 2010
  • Loc: Adirondack Mountains NY

Posted 29 November 2019 - 10:29 PM

I decided to buy an older server computer. Twin Xeon 8 core processors so 16 cores 32 threads. 128 gb ram. Fitted a good graphics card in it. Total cost less than $350 and it really is fast . Not as fast as a threadripper but the whole kit costs less than half of a current I7 chip alone.

 

Pi benchmark 14142 which is really good for the money I spent.

 

There are a lot of these rack mount servers around right now as many big companies upgrade to newer more power efficient designs. Thing is these are really robust servers that are designed to run for years with little maintenance, have worked in a clean environment and have lots of life left. You can get newer more powerful ex servers at closer to $800 that would be better but the Twin xeon 8 cores giving 16 total gave me the best bang for the buck.

 

Lastly because there are lots of these coming off of service there are plenty of cheap spare parts around. They use DDR3 1600 memory which is pretty cheap as well.


Edited by telfish, 29 November 2019 - 10:31 PM.

  • dswtan, bsavoie, bobzeq25 and 1 other like this

#23 bobzeq25

bobzeq25

    Hubble

  • *****
  • Posts: 17,404
  • Joined: 27 Oct 2014

Posted 29 November 2019 - 11:13 PM

Did you ever benchmark? Before and after?

 

My current.

 

*******************************************************************************
The Official PixInsight Benchmark version 1.0
Copyright © 2014-2015 Pleiades Astrophoto. All Rights Reserved.
*******************************************************************************

Benchmark version ...... 1.00.08
Input checksum ......... 2cd72b67e12fff2812ef5b5da054ab2a70a25e23
Serial number .......... XXK18M7KG857DXE6JZX7Q328ESREP56F

CPU Identification
CPU vendor ............. GenuineIntel
CPU model .............. Intel® Core™ i7-5820K CPU @ 3.30GHz

System Information
Platform ............... Windows
Operating system ....... Microsoft Windows 10 Pro
Core version ........... PixInsight Core 1.8.8-2 (x64)
Logical processors ..... 12
Total memory size ...... 31.927 GiB

Execution Times
Total time ............. 00:44.33
CPU time ............... 00:37.27
Swap time .............. 00:07.03
Swap transfer rate ..... 2356.472 MiB/s

Performance Indices
Total performance ...... 10613
CPU performance ........ 10157
Swap performance ....... 13051

*******************************************************************************

Benchmark with S/N XXK18M7KG857DXE6JZX7Q328ESREP56F has been recorded on 2019/11/29 05:00:03 UTC

 

 

I am going to see if this changes much with the more memory.

 

 

As well as trying Bobzeq's guidance. Right now I have 3 swapfiles but have 6 physical cores. Bobzeq, is that 1 swap for each physical core or logical core?

 

Chris

Logical, but it's hardly carved in stone.  <smile>



#24 Cfreerksen

Cfreerksen

    Viking 1

  • *****
  • Posts: 703
  • Joined: 18 Aug 2018
  • Loc: Tooele, Ut

Posted 29 November 2019 - 11:44 PM

Logical, but it's hardly carved in stone.  <smile>

I will try 6 and then give 12 swapfiles a shot. We'll see will see how that goes.

 

Thanks,

 

Chris


  • bobzeq25 likes this

#25 dswtan

dswtan

    Explorer 1

  • *****
  • Posts: 79
  • Joined: 29 Oct 2006
  • Loc: Morgan Hill, CA

Posted 30 November 2019 - 03:07 PM

I decided to buy an older server computer. Twin Xeon 8 core processors so 16 cores 32 threads. 128 gb ram. Fitted a good graphics card in it. Total cost less than $350 and it really is fast . Not as fast as a threadripper but the whole kit costs less than half of a current I7 chip alone.

This seems like a good idea. Do you have an example link to start researching these things? Ebay? 




CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.


Recent Topics






Cloudy Nights LLC
Cloudy Nights Sponsor: Astronomics