Jump to content

  •  

CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.

Photo

PixInsight Benchmark - How low can you go?

  • Please log in to reply
103 replies to this topic

#26 pfile

pfile

    Fly Me to the Moon

  • -----
  • Posts: 6,080
  • Joined: 14 Jun 2009

Posted 21 December 2019 - 10:19 PM

juan develops PI under linux and i'm sure if it were economically viable he'd only produce a linux version of PI. the thing that enables PI to run on windows and OSX is the Qt gui toolkit which is cross-platform. in the latest version of PI, there have been lots of problems with openGL drivers vs. Qt on windows which has really been a chore for juan to resolve. the linux PI userbase is small but the main problem for most linux users seems to be getting it going on different distros than juan uses, and once they get it started it seems to work OK. i don't see many support requests beyond missing dynamic libraries/dependencies that prevent PI from starting.

 

i'm not sure that PI is particularly optimized for linux, but it is of course heavily multithreaded and can really punish an I/O system when doing preprocessing tasks. i think in some instances the windows performance is than both OSX and linux, IIRC on I/O tasks. the differences between PI performance on different OSs probably comes down to thread models and scheduler policy.

 

rob


Edited by pfile, 22 December 2019 - 07:33 PM.

  • zakry3323 likes this

#27 pbkoden

pbkoden

    Mariner 2

  • *****
  • Posts: 284
  • Joined: 21 Aug 2015

Posted 22 December 2019 - 11:01 AM

I've always been intrigued by Linux and even installed it a few times. Every time I gave up on it as "too difficult" and moved back to Windows. But I built a new machine this year and have moved to Ubuntu as my primary OS with the intent of upping my Pixinsight speed and dropping the mess that is Windows. The learning curve was kind of steep, but so is the learning curve for Pixinsight. I keep Windows available in a dual-boot configuration for some specific gaming needs, but boot it up as little as possible. It has really grown on me now and I liked it so much I've switched my wife's computer and my home file server / Plex server over to Ubuntu as well.

 

As far as Pixinsight performance, I'm currently rocking the Ryzen 3600 with a 1TB NVME drive. Performance has been really good and these are very affordable components. You can build a very good Pixinsight machine for much less than a grand using these and similar parts. Especially if you save $100 by ditching Windows and going Linux.



#28 pfile

pfile

    Fly Me to the Moon

  • -----
  • Posts: 6,080
  • Joined: 14 Jun 2009

Posted 22 December 2019 - 07:35 PM

i think if you are used to windows or osx then linux is very fiddly. i guess this is why i love osx - it's a well-supported GUI-based OS with unix underneath. being an old fart CS guy i was raised on BSD and ultrix and so enjoy being able to fiddle under the hood but i also just want to run random desktop applications that don't run on linux.



#29 kanefsky

kanefsky

    Explorer 1

  • -----
  • Posts: 76
  • Joined: 02 Oct 2019

Posted 22 December 2019 - 07:52 PM

i think if you are used to windows or osx then linux is very fiddly. i guess this is why i love osx - it's a well-supported GUI-based OS with unix underneath. being an old fart CS guy i was raised on BSD and ultrix and so enjoy being able to fiddle under the hood but i also just want to run random desktop applications that don't run on linux.

Agreed. MacOS has always been my favorite for running GUI apps and most day-to-day stuff. I still prefer unix/linux for doing software development and for running servers or other mission-critical code. I have linux-based systems that have literally run for 5+ years (including updates) without needing to be rebooted.

 

In a couple days I'll be able to post some PixInsight benchmark numbers from one of the new Mac Pros which should be interesting :)


  • pfile and sheepofblue like this

#30 kanefsky

kanefsky

    Explorer 1

  • -----
  • Posts: 76
  • Joined: 02 Oct 2019

Posted 24 December 2019 - 03:05 AM

In a couple days I'll be able to post some PixInsight benchmark numbers from one of the new Mac Pros which should be interesting smile.gif

 

26106 :)

 

http://pixinsight.co...9CE2NAOSJ8N7372


  • zakry3323 and Cfreerksen like this

#31 james7ca

james7ca

    Voyager 1

  • *****
  • Posts: 11,382
  • Joined: 21 May 2011
  • Loc: San Diego, CA

Posted 24 December 2019 - 08:07 AM

Yes, number two on the current list of total performance (all operating systems, including Linux and Windows). It looks like the CPU score for the new Mac Pro is the highest by far, while the top score using Linux and the AMD Ryzen just has a much better swap score. Of course, the Mac Pro probably cost about ten times (or more) as much as the Ryzen, but I wonder how high the Mac Pro benchmark could go if you tweaked the setup for the swap space, it might go higher on total performance (potentially making it the number one system, but it will cost ya). 

 

[UPDATE]

I just checked on a new Mac Pro configured similarly to the benchmark given by kanefsky and it comes to $14000. You can go higher with more memory and graphics and other options, but I wonder how this compares to the current AMD Ryzen system that is at the top of the benchmark results. Probably isn't as cheap as the 1/10th figure I gave earlier, which would mean a $1400 configuration.

 

But, hey, if you are rocking a $100,000 observatory and telescope (which some are, and then some) then $14000 is almost chump change.

[/UPDATE]


Edited by james7ca, 24 December 2019 - 08:38 AM.

  • zakry3323 likes this

#32 kanefsky

kanefsky

    Explorer 1

  • -----
  • Posts: 76
  • Joined: 02 Oct 2019

Posted 24 December 2019 - 12:00 PM

I wonder how high the Mac Pro benchmark could go if you tweaked the setup for the swap space, it might go higher on total performance (potentially making it the number one system, but it will cost ya). 

 

I spent some time tweaking the swap performance but that's about the best I could get (RAM disk copied eight times). I tried a few different sizes for the RAM disk, different filesystems (HFS+/APFS/ExFAT), different numbers of copies, two separate RAM disks, and even enabling the compression option. I also tried combining RAM disk with the internal SSD. The only thing left I can think of is adding more NVMe SSD on the PCIe bus using something like this: https://eshop.macsal...c-accelsior-4m2



#33 james7ca

james7ca

    Voyager 1

  • *****
  • Posts: 11,382
  • Joined: 21 May 2011
  • Loc: San Diego, CA

Posted 24 December 2019 - 12:22 PM

I spent some time tweaking the swap performance but that's about the best I could get (RAM disk copied eight times). I tried a few different sizes for the RAM disk, different filesystems (HFS+/APFS/ExFAT), different numbers of copies, two separate RAM disks, and even enabling the compression option. I also tried combining RAM disk with the internal SSD. The only thing left I can think of is adding more NVMe SSD on the PCIe bus using something like this: https://eshop.macsal...c-accelsior-4m2

Well, that Ryzen system that is currently at number one (largely because of the swap performance) appears to have been using swap space on four different SSDs (with some that could have been NVMe drives).

 

In any case, I think the fastest storage would be one of Intel's Optane systems that run in a PCIe slot.

 

However, I don't think that the swap performance is really that critical in the day-to-day performance of PixInsight. Swap performance can be a significant factor for some tasks, but probably not nearly as important as indicated (or weighted) in the PI benchmarks.

 

CPU speed is what really matters and I wouldn't be surprised if the Mac Pro was already the fastest PixInsight machine that you can get from a widely situated retail store.


Edited by james7ca, 24 December 2019 - 12:32 PM.


#34 kanefsky

kanefsky

    Explorer 1

  • -----
  • Posts: 76
  • Joined: 02 Oct 2019

Posted 24 December 2019 - 12:44 PM

Well, that Ryzen system that is currently at number one (largely because of the swap performance) appears to have been using swap space on four different SSDs (with some that could have been NVMe drives).

There's an almost identical swap score (and a higher total score) on a Ryzen 9 3900X system with PixInsight 1.8.8-2 which just says "16G Ram Disk" for swap: http://pixinsight.co...60KWM3632VVGS08

 

On the Mac Pro the RAM disk doesn't perform all that much better than the SSD on the BlackMagic Disk Speed Test (~4800 vs ~2900 MB/sec), so it seems reasonable that the swap could be improved using multiple SSD drives on separate PCIe lanes.

 

I'm not sure how much the swap really matters in real life so when I get a chance I'm going to do something more real-world like run Batch Preprocessing on a large number of subs and see how it compares to my 2012 Macbook Air :)



#35 pbkoden

pbkoden

    Mariner 2

  • *****
  • Posts: 284
  • Joined: 21 Aug 2015

Posted 24 December 2019 - 01:59 PM

CPU speed is what really matters

 

The W-3275M in the Mac costs $4,500 with a CPU score of 28275. That's ~6.3 points per dollar.

 

The second place CPU is a 3900X costing $500 with a score of 26343. That's ~52.7 points per dollar. 

 

Ouch... 


  • jdupton and zakry3323 like this

#36 zakry3323

zakry3323

    Gemini

  • *****
  • topic starter
  • Posts: 3,306
  • Joined: 11 Apr 2016
  • Loc: Fairfield, Westmoreland County, PA

Posted 24 December 2019 - 02:48 PM

The W-3275M in the Mac costs $4,500 with a CPU score of 28275. That's ~6.3 points per dollar.

 

The second place CPU is a 3900X costing $500 with a score of 26343. That's ~52.7 points per dollar. 

 

Ouch... 

That's exactly what I'm looking for - a range of "best value" per performance hardware configurations. 



#37 kanefsky

kanefsky

    Explorer 1

  • -----
  • Posts: 76
  • Joined: 02 Oct 2019

Posted 24 December 2019 - 04:05 PM

It makes more sense to look at the whole system cost and not just the CPU. Points per dollar isn't necessarily the best metric since the Ryzen 2700X is 1/3 of the price of the 3900X with twice the points per dollar and it probably gets better and better until you're down to CPUs that cost $20 or less.

 

It also makes sense to consider what else the system is going to be used for, resale value, etc. The resale value of a Mac is huge even as a percentage of the initial cost compared to a home-built or even most name-brand PCs.


  • zakry3323 likes this

#38 Peregrinatum

Peregrinatum

    Surveyor 1

  • *****
  • Posts: 1,820
  • Joined: 27 Dec 2018
  • Loc: South Central Valley, Ca

Posted 25 December 2019 - 09:51 AM

Just got a new box for PI, old box had a score of 2,000, new box has a score of 13,000... makes processing in PI so much more enjoyable as you get to experiment so much more.  New box made processing 1,000 subs possible.

 

It's a Windoze10 box, Ryzen7v2, SSD swap drive, 16GB Ramdisk, 24GB total, just an average gamer box that I got a decent deal on but very happy with.



#39 zakry3323

zakry3323

    Gemini

  • *****
  • topic starter
  • Posts: 3,306
  • Joined: 11 Apr 2016
  • Loc: Fairfield, Westmoreland County, PA

Posted 25 December 2019 - 10:06 AM

It makes more sense to look at the whole system cost and not just the CPU. Points per dollar isn't necessarily the best metric since the Ryzen 2700X is 1/3 of the price of the 3900X with twice the points per dollar and it probably gets better and better until you're down to CPUs that cost $20 or less.

 

It also makes sense to consider what else the system is going to be used for, resale value, etc. The resale value of a Mac is huge even as a percentage of the initial cost compared to a home-built or even most name-brand PCs.

That's an interesting insight. Personally PixInsight is easily the most intensive program I'm bound to run. PS and Lightroom can also put my current system through its paces, but  not as much.  I don't use my computer for gaming much anymore, but in my experience  games tend to be much more graphics card dependant rather than bottlenecking with the CPU/RAM. 

 

Resale though, has never been a consideration for me. I run them until they die, or an upgrade turns them into cheap emulator boxes :)



#40 kanefsky

kanefsky

    Explorer 1

  • -----
  • Posts: 76
  • Joined: 02 Oct 2019

Posted 25 December 2019 - 04:29 PM

That's an interesting insight. Personally PixInsight is easily the most intensive program I'm bound to run. PS and Lightroom can also put my current system through its paces, but  not as much.  I don't use my computer for gaming much anymore, but in my experience  games tend to be much more graphics card dependant rather than bottlenecking with the CPU/RAM. 

 

Resale though, has never been a consideration for me. I run them until they die, or an upgrade turns them into cheap emulator boxes smile.gif

 

Lightroom and Photoshop are pretty intensive when you get back home and want to import hundreds or thousands of photos and generate 1:1 previews so you can go through them, pick out the good ones and process them.  Stitching together large panoramas and whatnot can be very intensive as well.

 

Video processing is where things really get nuts. Processing RAW 4K/6K/8K video files, applying noise reduction, image stabilization, etc. and then encoding it into an output format can take hours to process just a few minutes of video. You also want the ability to apply effects and see the results in real-time as you're editing. I think that's what the new Mac Pro is mostly targeted for. It doesn't just let you have lots of CPU cores but you can have up to four very high-end video cards plus the ProRes accelerator card.

 

Not that I'm arguing it's the most cost-effective. Prices always tend to increase exponentially at the very high end. For a business that's paying six-figure salaries to the people who use and maintain the machines the cost of the hardware isn't that big of a factor. I don't think Apple figured they'd sell too many of the new Mac Pros to home hobbyists.

 

I used to build PCs and not worry about getting rid of them. I probably had a dozen or so at one point as I kept upgrading them and making new machines out of the leftover parts. Similarly I've collected many other types of esoteric gear but the older I get the more I realize that it's much more of a hassle to get rid of things in a legal/responsible way than to acquire them in the first place.  It's really nice that if I want to replace an older Mac with a newer one that I can put it on eBay for three days with a $0.99 starting bid and there will be a large market for it and I'll get a very good price (plus it's kind of fun to watch :) )



#41 kanefsky

kanefsky

    Explorer 1

  • -----
  • Posts: 76
  • Joined: 02 Oct 2019

Posted 25 December 2019 - 05:56 PM

Here's some results of comparing the Mac Pro with a 2018 Mac Mini on both the PI benchmark and on on a Batch Preprocessing script (around 450 total subs including LRGB lights, darks, flats, dark flats, and bias). The script included integration to generate master lights. Obviously there are too many variables to compare it to anyone else's results but the script and the data were identical in all my tests.  I also ran it with both the default swap config (single instance on the internal SSD) and with an optimized swap config (multiple instances on a RAM disk).

 

The swap appears to make absolutely no difference whatsoever in the batch preprocessing time even when it makes a tremendous difference in the PI benchmark scores. Also the total preprocessing time didn't scale very well with the CPU score in the benchmark (2.8x increase in CPU score gave less than 2.0x increase in speed). Faster internal storage with multiple SSDs spread across more PCIe lanes using RAID 0 might help realize more of the CPU advantage.

 

2019 Mac Pro, 28-core 2.5GHz Xeon W, with RAM disk, 8 swap instances
PI Benchmark Total: 26106
PI Benchmark CPU: 28275
PI Benchmark Swap: 20044
Batch Preprocess: 12:23.42

 

2019 Mac Pro, 28-core 2.5GHz Xeon W, no RAM disk, 1 swap instance
PI Benchmark Total: 14353
PI Benchmark CPU: 27657
PI Benchmark Swap: 4823
Batch Preprocess: 12:19.00

 

2018 Mac Mini, 6-core 3.2GHz i7, with RAM disk, 5 swap instances
PI Benchmark Total: 11761
PI Benchmark CPU: 10648
PI Benchmark Swap: 21039
Batch Preprocess: 24:00.40

 

2018 Mac Mini, 6-core 3.2GHz i7, no RAM disk, 1 swap instance
PI Benchmark Total: 8537
PI Benchmark CPU: 10484
PI Benchmark Swap: 4846
Batch Preprocess: 23:51.86


Edited by kanefsky, 25 December 2019 - 05:57 PM.


#42 Chris Ryan

Chris Ryan

    Mariner 2

  • -----
  • Posts: 234
  • Joined: 08 Mar 2014
  • Loc: Flaxton, Qld, Australia

Posted 25 December 2019 - 07:39 PM

You're right that the swap setup makes a big difference to the benchmark, but in reality for most operations, it's not worth a RAM disk over a fast SSD.  Especially since with a RAM disk you will "lose" the benefits over multiple sessions (unless you back up the RAM to storage - and who's going to mess around with that? :) ).

 

Btw, the best Benchmark I got was this: http://pixinsight.co...OZ31DD6F44US3L 

 

26548

 

It's for 1.8.7, but I doubt 1.8.8 is much different.

 

I did tweak the system a lot to do this though (RAM timing adjustments, CPU overclocking, RAM disks, etc).

 

My daily usage setup though, the system sits around the 22-24K for the benchmark (just ran it before to recheck).



#43 kanefsky

kanefsky

    Explorer 1

  • -----
  • Posts: 76
  • Joined: 02 Oct 2019

Posted 27 December 2019 - 01:51 AM

OK just for fun I spun up a Google Compute Engine instance with 60 Xeon Cascade Lake 3.1GHz CPUs running Kubuntu and blew away every score ever recorded :)

http://pixinsight.co...P576RRPOUCDG8SQ

 

At about $3/hour it could actually be somewhat cost effective, especially if you used a smaller configuration whenever you didn't need all that power (4 CPUs for $0.21 per hour or 16 CPUs for $0.84 per hour for example). It only takes a moment to stop and start different virtual machine configurations and they can share the same boot disk and data.

 

The main issue would be getting all your data on it, which would offset any time savings unless you're uploading your subs in real time as you're taking them. Also you obviously have to do everything through a remote desktop (vnc or equivalent) type connection which isn't as nice as a monitor directly connected to your PC or Mac.


  • james7ca, Chris Ryan and Cfreerksen like this

#44 ajaxuk

ajaxuk

    Explorer 1

  • -----
  • Posts: 81
  • Joined: 15 Nov 2019
  • Loc: Cyprus

Posted 27 December 2019 - 02:36 AM

Since the topic title is how low:

 

Total:2558

CPU:2957

Swap:1644

 

on a Surface Pro 4.

 

Very useable on the the laptop, if a little slow smile.gif


  • zakry3323 likes this

#45 james7ca

james7ca

    Voyager 1

  • *****
  • Posts: 11,382
  • Joined: 21 May 2011
  • Loc: San Diego, CA

Posted 27 December 2019 - 03:53 AM

OK just for fun I spun up a Google Compute Engine instance with 60 Xeon Cascade Lake 3.1GHz CPUs running Kubuntu and blew away every score ever recorded smile.gif

http://pixinsight.co...P576RRPOUCDG8SQ

 

At about $3/hour it could actually be somewhat cost effective, especially if you used a smaller configuration whenever you didn't need all that power (4 CPUs for $0.21 per hour or 16 CPUs for $0.84 per hour for example). It only takes a moment to stop and start different virtual machine configurations and they can share the same boot disk and data.

 

The main issue would be getting all your data on it, which would offset any time savings unless you're uploading your subs in real time as you're taking them. Also you obviously have to do everything through a remote desktop (vnc or equivalent) type connection which isn't as nice as a monitor directly connected to your PC or Mac.

Yes, but as you suggested the time to upload your data would take away all of the advantage (and then some).

 

That said, if you had a small data set (like running noise reduction on a single master) then that would probably work (but, still a bit of a hassle and it's going to cost you to save a few minutes of time now and then).

 

At $3 per hour that means you could run $14000/$3 or about 4666 hours of computation to equal the price of one relatively high-end Mac Pro. At 8 hours per day that would come out to 583 days. Then, at the end of that time you'd have nothing (except the finished work) while a Mac Pro could continue to be used or even be sold to recoup some of your money.



#46 kanefsky

kanefsky

    Explorer 1

  • -----
  • Posts: 76
  • Joined: 02 Oct 2019

Posted 27 December 2019 - 04:33 AM

Yes, but as you suggested the time to upload your data would take away all of the advantage (and then some).

 

That said, if you had a small data set (like running noise reduction on a single master) then that would probably work (but, still a bit of a hassle and it's going to cost you to save a few minutes of time now and then).

 

At $3 per hour that means you could run $14000/$3 or about 4666 hours of computation to equal the price of one relatively high-end Mac Pro. At 8 hours per day that would come out to 583 days. Then, at the end of that time you'd have nothing (except the finished work) while a Mac Pro could continue to be used or even be sold to recoup some of your money.

I mentioned that if you were able to upload subs as they are captured (i.e. you're imaging somewhere with a good Internet connection) then you might not have any penalty for the upload time.

 

I doubt many people require 8 hours per day of hard-core processing on such a high-end system. Probably 1-2 hours a day on average would be more like it, and if you only used the high-end configuration when you had a lot of data to process and used a mere 16 CPUs the rest of the time then it would only be costing you $0.84 per hour for that time.

 

It's also quite likely that the price will decrease over time, and you should account for the time value of the $14K that you could invest instead of spending it up front on the Mac Pro. If you earned 10% (the average annual return of the S&P 500) that would be enough to cover an hour a day of 60-CPU time plus another hour of 16-CPU time (or two hours of 30-CPU time if you prefer) forever without ever spending a dime.



#47 pbkoden

pbkoden

    Mariner 2

  • *****
  • Posts: 284
  • Joined: 21 Aug 2015

Posted 27 December 2019 - 02:22 PM

I've been trying to get the Ryzen 3950x 16-core chip since release. I was finally able to snag one and tested out the Pixinsight benchmark today. CPU score of 29824 and Swap of 53257 (Ram Disk) for a total score of 32603. That's enough to take the top crown on the benchmark for CPU and overall scores for now. This kind of power at this price point was unthinkable a year ago.

 

Now I'm waiting for someone to come along with one of the new Threadripper chips and destroy this score.



#48 kanefsky

kanefsky

    Explorer 1

  • -----
  • Posts: 76
  • Joined: 02 Oct 2019

Posted 27 December 2019 - 05:56 PM

Nicely done :)

 

I tweaked the swap settings some more and managed to get the Mac Pro back ahead of the 2950X and the Google Compute Engine VM back to #1 ahead of the 3950X.

 

I have to say that in actual usage I'm hardly even noticing whether I'm using the 28-core Mac Pro versus a 6-core Mac Mini.  I still have to go away and do something else while batch preprocessing runs and most of the routine interactive stuff doesn't really feel that much different.



#49 lakeorion

lakeorion

    Surveyor 1

  • -----
  • Posts: 1,983
  • Joined: 03 Aug 2010
  • Loc: Lake Orion MI

Posted 30 December 2019 - 09:47 AM

On the other end of the spectrum...

 

Just for fun I tried out my new imaging laptop I'm setting up.  It's the AMD A6-7310 APU with AMD Radeon R4 Graphics (2) at the bottom of the list.  Interesting observation.  With the exact same hardware (dual boot Win 10 / Kubuntu) the Windows swap speed is seven times slower resulting in a total score about half that of Kubuntu. Windows 1109 / Kubuntu 1983.

 

It's an old HP laptop given up for dead by my son that I pried open and put a SATA SSD and some RAM in - boom it's gooder.

 

I don't plan on processing with this machine, I only install PI to pixel peep at subs on the fly.


Edited by lakeorion, 30 December 2019 - 09:48 AM.

  • Cfreerksen likes this

#50 ribuck

ribuck

    Viking 1

  • -----
  • Posts: 889
  • Joined: 01 Nov 2005
  • Loc: Newcastle, England

Posted 01 January 2020 - 01:34 PM

16 Cores Rock!

Technically 32 cores as it's a 16 core chip with HyperThreading or whatever Amds name is for it, so 32 hyper threaded cores.




CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.


Recent Topics






Cloudy Nights LLC
Cloudy Nights Sponsor: Astronomics