Jump to content

  •  

CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.

Photo

Seeking Pixinsight system performance Masters

  • Please log in to reply
24 replies to this topic

#1 PhotonHunter1

PhotonHunter1

    Viking 1

  • *****
  • topic starter
  • Posts: 630
  • Joined: 19 Mar 2015
  • Loc: Northwest IL (for now)

Posted 19 August 2019 - 09:13 PM

I'm seeking help from the PI performance masters out there. I installed AMD Radeon Ramdisk and have run multiple scenarios. Using the PI Benchmark script, system performance with and without Ramdisk is crazy good - a major increase in performance:

Total Performance without (5787) and with (12304)

CPU Performance without (10576) and with (11261)

Swap Performance without (2019) and with (20017)

 

However, when processing 50 darks with Image Integration, the Process Console shows run times of 1:30 without Ramdisk (Memory in Use was 5GB per Task Manager) and 1:23 with Ramdisk (Memory in Use was 16.8GB per Task Manager) which I would expect since I've allocated 12GB to Ramdisk. Why would I only see a process time improvement of 7 seconds?? Is Image Integration not the best measure of Ramdisk performance? 

 

Per the PI Benchmark script the best all around configuration seems to be 12GB Ramdisk and 6 swap directories pointed to my Ramdisk (drive P:/) with an additional swap directory pointed to C:/...temp (which resides on my 256GB M.2 SSD drive). Below is a screenshot of PI configuration.

pi directories and network.png

 

Do I need to configure my Ramdisk or PI Directories and Network differently? Any help would be appreciated!

 

PS: The balance of my desktop configuration is ASRock AB350 Pro4 mother board. AMD Ryzen 1700x CPU 3400 mhz. 2-16GB sticks of Ram, overclocked to 3266 mhz.

Thanks.


Edited by mike8888, 19 August 2019 - 09:15 PM.


#2 Jim Waters

Jim Waters

    Gemini

  • *****
  • Posts: 3007
  • Joined: 21 Oct 2007
  • Loc: Phoenix, AZ USA

Posted 19 August 2019 - 10:15 PM

The biggest performance gain I got was replacing my HDD with a SSD.  I also added two extra M.2 RAID 0 drives.  My performance increased over 12X.  Adding a RAM Disk is OK but you will be better off with a SSD.


  • bsavoie likes this

#3 bobzeq25

bobzeq25

    Hubble

  • *****
  • Posts: 16931
  • Joined: 27 Oct 2014

Posted 19 August 2019 - 10:15 PM

Why do you have one swap directory on the c drive, and all the others on the ramdisk?



#4 PhotonHunter1

PhotonHunter1

    Viking 1

  • *****
  • topic starter
  • Posts: 630
  • Joined: 19 Mar 2015
  • Loc: Northwest IL (for now)

Posted 19 August 2019 - 11:00 PM

Why do you have one swap directory on the c drive, and all the others on the ramdisk?

C:/ was set up in PI by default. In reading through some of the links from a previous post, the best performance was with multiple Ramdisk directories and a single directory on the SSD. I ran performance tests with 12GB of Ramdisk space with and without the C:/ directory. PI Benchmark performance was minimally impacted:

Total Performance with C (12304 - 38.23) and without (11670 - 40.31) +

CPU Performance with C (11261 - 33.61) and without (10537 - 35.92) +

Swap Performance with C (20017 - 04.59) and without (21094 - 04.35) -

 

Image Integration run time remained 1:23 seconds. Would a different Ramdisk product produce better results or would a dedicated SSD drive, as Jim states, be better?



#5 PhotonHunter1

PhotonHunter1

    Viking 1

  • *****
  • topic starter
  • Posts: 630
  • Joined: 19 Mar 2015
  • Loc: Northwest IL (for now)

Posted 19 August 2019 - 11:02 PM

The biggest performance gain I got was replacing my HDD with a SSD.  I also added two extra M.2 RAID 0 drives.  My performance increased over 12X.  Adding a RAM Disk is OK but you will be better off with a SSD.

Jim, was the 12x increase in run time performance per the Process Control window or the PI Benchmark script? How many times do you point to the dedicated SSD in PI Directories and Network?



#6 PhotonHunter1

PhotonHunter1

    Viking 1

  • *****
  • topic starter
  • Posts: 630
  • Joined: 19 Mar 2015
  • Loc: Northwest IL (for now)

Posted 19 August 2019 - 11:03 PM

I also forgot to mention that I am running Windows 10 Pro.


Edited by mike8888, 19 August 2019 - 11:03 PM.


#7 bobzeq25

bobzeq25

    Hubble

  • *****
  • Posts: 16931
  • Joined: 27 Oct 2014

Posted 19 August 2019 - 11:10 PM

C:/ was set up in PI by default. In reading through some of the links from a previous post, the best performance was with multiple Ramdisk directories and a single directory on the SSD. I ran performance tests with 12GB of Ramdisk space with and without the C:/ directory. PI Benchmark performance was minimally impacted:

Total Performance with C (12304 - 38.23) and without (11670 - 40.31) +

CPU Performance with C (11261 - 33.61) and without (10537 - 35.92) +

Swap Performance with C (20017 - 04.59) and without (21094 - 04.35) -

 

Image Integration run time remained 1:23 seconds. Would a different Ramdisk product produce better results or would a dedicated SSD drive, as Jim states, be better?

In my experience a RAMdisk and a state of the art SSD drive gave pretty similar results.  In neither case was the system swap disk limited, it was CPU limited.  Yours appears to be that, also.

 

Bottom line.  It's unlikely to matter much.


Edited by bobzeq25, 19 August 2019 - 11:10 PM.


#8 Jim Waters

Jim Waters

    Gemini

  • *****
  • Posts: 3007
  • Joined: 21 Oct 2007
  • Loc: Phoenix, AZ USA

Posted 19 August 2019 - 11:57 PM

Jim, was the 12x increase in run time performance per the Process Control window or the PI Benchmark script? How many times do you point to the dedicated SSD in PI Directories and Network?

My own Benchmark Script on M42.  My tower PC Core i7-5820K had a mix of 5400 and 7200 RPM hard disks.  I replaced the primary C disk with a SSD and replaced the other 2 disks with M.2 SSD 's configured for RAID 0.  I did add more DDR4 memory.

 

On my Lenovo E440 laptop I had a 5400 RPM HDD and I replace it with a SSD and one M.2 drive.  Before the mod it took me over 2 hours to run BPP on a very large number of subs.  After the mod it took under 30 mins.

 

IMHO I would convert everything to SSD.



#9 Jim Waters

Jim Waters

    Gemini

  • *****
  • Posts: 3007
  • Joined: 21 Oct 2007
  • Loc: Phoenix, AZ USA

Posted 19 August 2019 - 11:58 PM

I also forgot to mention that I am running Windows 10 Pro.

I would kill all unneeded services / processes to free up CPU cycles and memory IO.



#10 Chris Ryan

Chris Ryan

    Vostok 1

  • -----
  • Posts: 146
  • Joined: 08 Mar 2014
  • Loc: Brisbane

Posted 23 August 2019 - 02:42 AM

I also forgot to mention that I am running Windows 10 Pro.

I suspect it might not be an option for you, but you'll get a substantial performance improvement by switching to a Linux system instead. 


  • Jim Waters and bsavoie like this

#11 bobzeq25

bobzeq25

    Hubble

  • *****
  • Posts: 16931
  • Joined: 27 Oct 2014

Posted 23 August 2019 - 09:02 AM

I suspect it might not be an option for you, but you'll get a substantial performance improvement by switching to a Linux system instead. 

How "substantial?  I've seen numbers like 10%.



#12 PhotonHunter1

PhotonHunter1

    Viking 1

  • *****
  • topic starter
  • Posts: 630
  • Joined: 19 Mar 2015
  • Loc: Northwest IL (for now)

Posted 23 August 2019 - 02:42 PM

I suspect it might not be an option for you, but you'll get a substantial performance improvement by switching to a Linux system instead. 

Hey Chris, thanks. I had Ubuntu, then moved to Mint Cinnamon 19 (Ubuntu at the end of the day). In both cases I had major stability issues - not sure why. The system would freeze up and I would have to reboot. At one point, the system locked up on me. I don't recall the error message but after 6 hours of Google searches and fix attempts I tried to reinstall Ubuntu with no success. For whatever reason Ubuntu does not play well with my Nvidia GTX1050 video card. I may give it another try down the road but unwillingly made the move to win 10 pro.


Edited by mike8888, 23 August 2019 - 02:48 PM.


#13 Jim Waters

Jim Waters

    Gemini

  • *****
  • Posts: 3007
  • Joined: 21 Oct 2007
  • Loc: Phoenix, AZ USA

Posted 23 August 2019 - 02:51 PM

 For whatever reason Ubuntu does not play well with my Nvidia GTX1050 video card. I may give it another try down the road but unwillingly made the move to win 10 pro.

Linux and Nvidia have had long standing issues with driver development / support.  Nvidia Linux drivers are VERY problematic.  

 

https://www.youtube....h?v=IVpOyKCNZYw


Edited by Jim Waters, 23 August 2019 - 02:52 PM.


#14 PhotonHunter1

PhotonHunter1

    Viking 1

  • *****
  • topic starter
  • Posts: 630
  • Joined: 19 Mar 2015
  • Loc: Northwest IL (for now)

Posted 23 August 2019 - 05:42 PM

Linux and Nvidia have had long standing issues with driver development / support.  Nvidia Linux drivers are VERY problematic.  

 

https://www.youtube....h?v=IVpOyKCNZYw

Exactly Jim. I ran Ubuntu from the disk image, selected the Nvidia drivers from the default Linux drivers and it still bombed. I swapped out my GTX 1050 (lower end graphics card) to an even LOWER end Pegatron graphics card and still no luck. It was so frustrating...



#15 bobzeq25

bobzeq25

    Hubble

  • *****
  • Posts: 16931
  • Joined: 27 Oct 2014

Posted 23 August 2019 - 05:49 PM

Just another data point.  I've had no problems.  Ubuntu, Nvidia 1030.  My recollection is that I didn't "select" a driver, one was loaded automatically.

 

https://www.amazon.c.../dp/B072HRD3CY/


Edited by bobzeq25, 23 August 2019 - 05:50 PM.


#16 Jim Waters

Jim Waters

    Gemini

  • *****
  • Posts: 3007
  • Joined: 21 Oct 2007
  • Loc: Phoenix, AZ USA

Posted 23 August 2019 - 06:53 PM

Just another data point.  I've had no problems.  Ubuntu, Nvidia 1030.  My recollection is that I didn't "select" a driver, one was loaded automatically.

 

https://www.amazon.c.../dp/B072HRD3CY/

What Ubuntu build?



#17 bobzeq25

bobzeq25

    Hubble

  • *****
  • Posts: 16931
  • Joined: 27 Oct 2014

Posted 23 August 2019 - 07:03 PM

What Ubuntu build?

18.04.3 LTS



#18 PhotonHunter1

PhotonHunter1

    Viking 1

  • *****
  • topic starter
  • Posts: 630
  • Joined: 19 Mar 2015
  • Loc: Northwest IL (for now)

Posted 23 August 2019 - 07:51 PM

Initially I had 16.04 LTS, then Mint 19, then recently tried 18.04 LTS. Maybe I need to look at the 1030 card?



#19 Jim Waters

Jim Waters

    Gemini

  • *****
  • Posts: 3007
  • Joined: 21 Oct 2007
  • Loc: Phoenix, AZ USA

Posted 23 August 2019 - 09:48 PM

This is the reason I haven't migrated to Linux - MX Linux, Manjaro or Mint.  I can't find reliable information on what 'high-end' video cards works.  I want something significantly better than SVGA.  SXGA is my entry point.



#20 Chris Ryan

Chris Ryan

    Vostok 1

  • -----
  • Posts: 146
  • Joined: 08 Mar 2014
  • Loc: Brisbane

Posted 24 August 2019 - 02:10 AM

How "substantial?  I've seen numbers like 10%.

I can only report based on perf benchmark numbers (so as to compare similar set ups since I don't use Windows on this system), and it was a while ago when I looked, but it was much more than that - up to 40 to 50% more.  However, that's the benchmark and how well that translates to Real World Usage is another thing.  Still, at least it gives some baselines.



#21 Chris Ryan

Chris Ryan

    Vostok 1

  • -----
  • Posts: 146
  • Joined: 08 Mar 2014
  • Loc: Brisbane

Posted 24 August 2019 - 02:16 AM

This is the reason I haven't migrated to Linux - MX Linux, Manjaro or Mint.  I can't find reliable information on what 'high-end' video cards works.  I want something significantly better than SVGA.  SXGA is my entry point.

I've got an nVidia GeForce RTC 2070 installed for my Acer X34 P monitor.  It's near flawless.  I also run a game through wine and the 3D performance is great.  Before I got the new monitor and card, I was using a GTX 1030 card with no problems either.

 

The linux distro shouldn't really matter so long as it has a modern kernel.  I run Gentoo right now, but also Kubuntu on other machines. 


  • Jim Waters and PhotonHunter1 like this

#22 Jim Waters

Jim Waters

    Gemini

  • *****
  • Posts: 3007
  • Joined: 21 Oct 2007
  • Loc: Phoenix, AZ USA

Posted 24 August 2019 - 02:39 AM

I've got an nVidia GeForce RTC 2070 installed for my Acer X34 P monitor.  It's near flawless.  I also run a game through wine and the 3D performance is great.  Before I got the new monitor and card, I was using a GTX 1030 card with no problems either.

 

The linux distro shouldn't really matter so long as it has a modern kernel.  I run Gentoo right now, but also Kubuntu on other machines. 

Thanks Chris.



#23 Chris Ryan

Chris Ryan

    Vostok 1

  • -----
  • Posts: 146
  • Joined: 08 Mar 2014
  • Loc: Brisbane

Posted 24 August 2019 - 09:15 AM

It should say RTX 2070, not RTC 2070 - typo.



#24 mikefulb

mikefulb

    Surveyor 1

  • *****
  • Posts: 1697
  • Joined: 17 Apr 2006

Posted 25 August 2019 - 11:18 AM

Like bobzeq25 I had no problems with my 1050Ti and Ubuntu 18.0.4.1 LTS.  I opted to installed the Nvidia drivers (396.54 - a tad old but I don't like changing things when they are working) which gave me good support for games (via Steam/Proton my Windows games work fine) and 3 headed support.  I used my Spyder 2 and an open source software (ArgyllCMS) to calibrate my monitor.  APP and PI work great.  I do startup vmware to use Photoshop CS3 under an image of my old Win7 box this system replaced.   It has been a great experience.



#25 brave_ulysses

brave_ulysses

    Apollo

  • -----
  • Posts: 1133
  • Joined: 19 Apr 2009
  • Loc: far outside the wire

Posted 10 September 2019 - 09:21 PM

you can get the latest nvidia drivers on ubuntu-ish distros with

 

sudo add-apt-repository ppa:graphics-drivers

sudo apt update

 

390 is the latest in the base 18.04 distro. 435 is currently available in the ppa

 

the nouveau driver is likely installed by default

 

apt search nvidia.*435
Sorting... Done
Full Text Search... Done
libnvidia-cfg1-435/bionic 435.21-0ubuntu0~18.04.2 amd64
  NVIDIA binary OpenGL/GLX configuration library

libnvidia-common-435/bionic,bionic 435.21-0ubuntu0~18.04.2 all
  Shared files used by the NVIDIA libraries

libnvidia-compute-435/bionic 435.21-0ubuntu0~18.04.2 amd64
  NVIDIA libcompute package

libnvidia-decode-435/bionic 435.21-0ubuntu0~18.04.2 amd64
  NVIDIA Video Decoding runtime libraries

libnvidia-encode-435/bionic 435.21-0ubuntu0~18.04.2 amd64
  NVENC Video Encoding runtime library

libnvidia-fbc1-435/bionic 435.21-0ubuntu0~18.04.2 amd64
  NVIDIA OpenGL-based Framebuffer Capture runtime library

libnvidia-gl-435/bionic 435.21-0ubuntu0~18.04.2 amd64
  NVIDIA OpenGL/GLX/EGL/GLES GLVND libraries and Vulkan ICD

libnvidia-ifr1-435/bionic 435.21-0ubuntu0~18.04.2 amd64
  NVIDIA OpenGL-based Inband Frame Readback runtime library

nvidia-compute-utils-435/bionic 435.21-0ubuntu0~18.04.2 amd64
  NVIDIA compute utilities

nvidia-dkms-435/bionic 435.21-0ubuntu0~18.04.2 amd64
  NVIDIA DKMS package

nvidia-driver-435/bionic 435.21-0ubuntu0~18.04.2 amd64
  NVIDIA driver metapackage

nvidia-headless-435/bionic 435.21-0ubuntu0~18.04.2 amd64
  NVIDIA headless metapackage

nvidia-headless-no-dkms-435/bionic 435.21-0ubuntu0~18.04.2 amd64
  NVIDIA headless metapackage - no DKMS

nvidia-kernel-common-435/bionic 435.21-0ubuntu0~18.04.2 amd64
  Shared files used with the kernel module

nvidia-kernel-source-435/bionic 435.21-0ubuntu0~18.04.2 amd64
  NVIDIA kernel source package

nvidia-utils-435/bionic 435.21-0ubuntu0~18.04.2 amd64
  NVIDIA driver support binaries

xserver-xorg-video-nvidia-435/bionic 435.21-0ubuntu0~18.04.2 amd64
  NVIDIA binary Xorg driver

 

 

lshw -c display
  *-display                
       description: VGA compatible controller
       product: GP106 [GeForce GTX 1060 3GB]
       vendor: NVIDIA Corporation
       physical id: 0
       bus info: pci@0000:81:00.0
       version: a1
       width: 64 bits
       clock: 33MHz
       capabilities: pm msi pciexpress vga_controller bus_master cap_list rom
       configuration: driver=nvidia latency=0
       resources: irq:87 memory:d8000000-d8ffffff memory:e0000000-efffffff memory:f0000000-f1ffffff ioport:c000(size=128) memory:c0000-dffff


  • bsavoie and PhotonHunter1 like this


CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.


Recent Topics






Cloudy Nights LLC
Cloudy Nights Sponsor: Astronomics