Jump to content

  •  

CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.

Photo

ryzen 3700x on image prosessing.

  • Please log in to reply
37 replies to this topic

#26 Synon

Synon

    Mariner 2

  • *****
  • Posts: 257
  • Joined: 02 Jun 2012
  • Loc: Flagstaff AZ

Posted 21 July 2019 - 07:33 PM

HOw much improvement should I expect by doubling my DDR4 RAM from 16Gb to 32Gb and adding 4TB SSD RAID PCIe X16 card for swap folders instead of a 1TB X2 SSD drive?

Double your RAM and then use a portion of that as a RAM disk, that will be better than an SSD. I have a modern NVME drive, but using the RAM disk is still much faster. My swap time is around 4.5 seconds, score of 20k. Linux and overclocking would also give some noticeable increases looking at the benchmarks, you do have the k chip after all, so it's unlocked.


  • John O'Grady likes this

#27 cfosterstars

cfosterstars

    Mercury-Atlas

  • *****
  • Posts: 2523
  • Joined: 05 Sep 2014
  • Loc: Austin, Texas

Posted 21 July 2019 - 10:48 PM

Double your RAM and then use a portion of that as a RAM disk, that will be better than an SSD. I have a modern NVME drive, but using the RAM disk is still much faster. My swap time is around 4.5 seconds, score of 20k. Linux and overclocking would also give some noticeable increases looking at the benchmarks, you do have the k chip after all, so it's unlocked.

I am doubling my RAM to 32Gb. How big of a RAM disc would I need? 4Gb?



#28 Synon

Synon

    Mariner 2

  • *****
  • Posts: 257
  • Joined: 02 Jun 2012
  • Loc: Flagstaff AZ

Posted 22 July 2019 - 12:02 AM

I am doubling my RAM to 32Gb. How big of a RAM disc would I need? 4Gb?

I'm actually doing a little more experimenting with the RAM disks today, I found if I created 2 disks at 4GB I can get a swap time under 4 seconds (score of 23300), about a 55% performance increase from using the NVME SSD. Now, whether or not the size is going to be too small I'm not actually sure. Definitely something to do some experimenting with. 


Edited by Synon, 22 July 2019 - 12:04 AM.


#29 pfile

pfile

    Fly Me to the Moon

  • -----
  • Posts: 5125
  • Joined: 14 Jun 2009

Posted 22 July 2019 - 12:39 AM

it's worth testing - i run some truly ridiculously large PI instances and my swap is always huge. i can't remember what happens with the parallel swapping when one partition fills up. does it keep writing to the swap directories that still have space, or does it die when the first one fills up?

 

imo the swap isn't really that important to tweak the hell out of - obviously it shows up in the benchmark as a huge improvement, but every process only ever does one swap write and possibly one swap read (if there is a mask applied). most processes in PI are cpu-bound, not io-bound, so making the swap super fast has its limits on overall performance.

 

the way PI projects work though is to copy all the files from inside the project directory into your swap area(s) when you first load the project, so having a ramdisk as swap will definitely speed up project loads and saves (as long as the entire project fits on the ramdisk, that is.)

 

rob


Edited by pfile, 22 July 2019 - 12:40 AM.

  • Synon likes this

#30 Herra Kuulapaa

Herra Kuulapaa

    Viking 1

  • -----
  • Posts: 511
  • Joined: 10 Dec 2014
  • Loc: Vantaa, Finland

Posted 22 July 2019 - 10:33 AM

Wow, that really puts my 8th-gen core i7/16GB RAM/500G m.2 ssd  to shame.

Tell it to my i5-750 ;) 10 years anniversary next year and still it handles my +800GB solar/lunar mosaics just fine. 

It takes full day of computing and a certain amount of patence is required though..


  • calypsob and cfosterstars like this

#31 cfosterstars

cfosterstars

    Mercury-Atlas

  • *****
  • Posts: 2523
  • Joined: 05 Sep 2014
  • Loc: Austin, Texas

Posted 22 July 2019 - 10:57 AM

Tell it to my i5-750 wink.gif 10 years anniversary next year and still it handles my +800GB solar/lunar mosaics just fine. 

It takes full day of computing and a certain amount of patence is required though..

I feel your pain brother...



#32 cfosterstars

cfosterstars

    Mercury-Atlas

  • *****
  • Posts: 2523
  • Joined: 05 Sep 2014
  • Loc: Austin, Texas

Posted 23 July 2019 - 09:48 PM

I got a HUGE boost. I installed an additional 32Gb of 3200 DDR4 and used 10Gb to make a RAM disk for my swap folders. Here is the before and after benchmarks.

 

PI performance.png


  • John O'Grady and jdupton like this

#33 Synon

Synon

    Mariner 2

  • *****
  • Posts: 257
  • Joined: 02 Jun 2012
  • Loc: Flagstaff AZ

Posted 10 August 2019 - 06:20 PM

AMD just released their EPYC 2 lineup including a 64 core server processor. Can anyone with data center experience speak to whether something like this would work/excel for Pixinsight? 

 

I'm thinking I'll wait for the Ryzen 3950X to come out in September, should be a monster!



#34 TareqPhoto

TareqPhoto

    Skylab

  • -----
  • Posts: 4490
  • Joined: 20 Feb 2017
  • Loc: Ajman - UAE

Posted 11 August 2019 - 07:25 AM

I hope one day i get into that myself, i did build 3 rigs in the past [not by myself] and ended up using 2 of them very rare as i gave my least build to my daughter, i spent too much for them, but that time i was working, so i had budget, now my budget is lower than before and i won't risk build one that will be outdated in next minute i finish my build, but sounds i got into this astro with big files to play with then i have to have one.

 

For now i am doing fine with my current rig, the main one i built it for gaming actually that time, so it is fast, still with 32GB, but memory isn't everything as i can see, in PC groups on Facebook they told me it is about the processor, and recommended me to move to Ryzen, so i won't upgrade my current PC to 64GB as it won't make a big difference, i didn't play much games on it to be honest, and i don't use it for anything else, so it doesn't bottle-nick, and i tested it with one large file of video of planetary and it did the job just fine, i mean i easily left it doing the job and i got busy in my gaming laptop, it takes several minutes to finish stacking, so i feel that if i have to upgrade to newer system then i must reach to limits of my current rig first, i save my money on another things for astro, maybe next year i can start thinking/planning for a new rig if necessary.



#35 Seanem44

Seanem44

    Vanguard

  • *****
  • Posts: 2054
  • Joined: 22 Sep 2011
  • Loc: Stafford, VA

Posted 12 August 2019 - 06:09 AM

Nice...  I am building a 3900x gaming rig.  This looks promising for editing as well.


  • Synon likes this

#36 calypsob

calypsob

    Aurora

  • *****
  • Posts: 4683
  • Joined: 20 Apr 2013

Posted 12 August 2019 - 11:20 AM

AMD just released their EPYC 2 lineup including a 64 core server processor. Can anyone with data center experience speak to whether something like this would work/excel for Pixinsight? 

 

I'm thinking I'll wait for the Ryzen 3950X to come out in September, should be a monster!

Can you use a server chip like a regular cpu?
I guess Linux might be an option for that, honestly I have never used Linux but the performance increases I am seeing over windows make it tempting. Overall Im pretty happy with my process times, I only have issues when registering more than 600 osc dslr images, where I typically run out of 32gb ram. I would be curious to see how one of these threadrippers handle big data sets 



#37 ChrisMoses

ChrisMoses

    Viking 1

  • -----
  • Posts: 987
  • Joined: 22 Oct 2014
  • Loc: Fort Wayne, IN, USA

Posted 12 August 2019 - 12:33 PM

Hi,

That is good improvement with a RAM drive.  Do you know if it is possible to set-up one under Win10?



#38 cfosterstars

cfosterstars

    Mercury-Atlas

  • *****
  • Posts: 2523
  • Joined: 05 Sep 2014
  • Loc: Austin, Texas

Posted 12 August 2019 - 12:51 PM

Hi,

That is good improvement with a RAM drive.  Do you know if it is possible to set-up one under Win10?

 

 

Yes you can do it in Window 10. That is what I am using. I am just using a 3rd party RAM disc utility and it works great. It also has the advantage of purging your swap folders every time you reboot. When PI crashes, the swap folders do not clear. After a few crashes, they can fill up your hard drive and you not know it. 




CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.


Recent Topics






Cloudy Nights LLC
Cloudy Nights Sponsor: Astronomics