Jump to content

  •  

CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.

Photo

Let's Speed up PixInsight!

  • Please log in to reply
20 replies to this topic

#1 joeytroy

joeytroy

    Viking 1

  • *****
  • topic starter
  • Posts: 715
  • Joined: 14 Aug 2020
  • Loc: Belen, NM, USA

Posted 17 February 2021 - 05:09 PM

So I have been doing some tweaking in PI on my Mac Mini but this will work on Windows/Linux/Mac. I did two simple tweaks

 

1. I checked to see how many procs / threads I have

 

Mac Terminal Command. Remember the hw.logicalcpu number

$ sysctl hw.physicalcpu hw.logicalcpu
hw.physicalcpu: 6
hw.logicalcpu: 12

Windows PowerShell: Remember the Number of LogicalProcessors

PS> Get-WmiObject -Class Win32_Processor | Select-Object -Property Name, Number*
Name                                    NumberOfCores NumberOfLogicalProcessors
----                                    ------------- -------------------------
Intel(R) Core(TM) i7-26...                          2                         4

Linux Terminal:  Remember the Thread Count

$ sudo dmidecode -t 4 | grep -E 'Socket Designation|Count'
    Socket Designation: CPU1
    Core Count: 8
    Thread Count: 16
    Socket Designation: CPU2
    Core Count: 8
    Thread Count: 16

Now that we know our threads hop into PixInsight and go to Edit < Global Preferences < Parallel Processing and Threads and change the "Maximum number of read and write threads" to your thread count. Mine was 12 so I have 12 in both boxes

Screen-Shot-2021-02-17-at-2-58-15-PM.png

 

2. Next go to Directories and Networks and setup some more directories for swap anywhere you want. I added 3 after testing 4 and 5 and not making a big difference so 3 was my magic number but I read some other posts people have made as many as 11 based on their beefy hardware.

Screen-Shot-2021-02-17-at-2-59-25-PM.png

 

After making the changes I ran Scripts < Benchmarks < Pixinsight Benchmark and saw the following difference in bench marks after making the changes. The lower numbers are pre changes and then the faster number was after the thread and swap folder changes.

Screen-Shot-2021-02-17-at-2-59-47-PM.png

 

As usual YMMV, best of luck!

 


  • D_talley, Ken Sturrock, LauraMS and 4 others like this

#2 joeytroy

joeytroy

    Viking 1

  • *****
  • topic starter
  • Posts: 715
  • Joined: 14 Aug 2020
  • Loc: Belen, NM, USA

Posted 17 February 2021 - 05:58 PM

One more thing I would like to share for Windows users that have an NVIDIA video card and use StarNet++. This will also get you some serious RPM's from what I am reading. I need to verify my Xeon Laptop NVIDIA card model so not sure it's going to work for me but may work for others.

 

https://darkskies.sp...t-starnet-cuda/


Edited by joeytroy, 17 February 2021 - 06:00 PM.


#3 MikiSJ

MikiSJ

    Surveyor 1

  • *****
  • Posts: 1,890
  • Joined: 24 Feb 2006
  • Loc: San Jose, CA

Posted 17 February 2021 - 06:25 PM

I tried your suggestion and here is the increase in my system. I increased the file threads to 8 from 1

 

Current Performance Indices
Total performance ......  5880 CPU performance ........ 6643 Swap performance ....... 4009

 

Prior Performance Indices

Total performance ......  5503 CPU performance ........ 6055  Swap performance ...... 4022

 

Not a significant improvement, but worth a try.

 

Here is my system information:

 

CPU model .............. AMD Ryzen 7 3700U with Radeon Vega Mobile Gfx

System Information
Operating system .... Microsoft Windows 10 Home
Core version ............ PixInsight Core 1.8.8-7 (x64)
Logical processors .. 8
Total memory size ... 9.879 GiB



#4 joeytroy

joeytroy

    Viking 1

  • *****
  • topic starter
  • Posts: 715
  • Joined: 14 Aug 2020
  • Loc: Belen, NM, USA

Posted 17 February 2021 - 06:30 PM

I tried your suggestion and here is the increase in my system. I increased the file threads to 8 from 1

 

Current Performance Indices
Total performance ......  5880 CPU performance ........ 6643 Swap performance ....... 4009

 

Prior Performance Indices

Total performance ......  5503 CPU performance ........ 6055  Swap performance ...... 4022

 

Not a significant improvement, but worth a try.

 

Here is my system information:

 

CPU model .............. AMD Ryzen 7 3700U with Radeon Vega Mobile Gfx

System Information
Operating system .... Microsoft Windows 10 Home
Core version ............ PixInsight Core 1.8.8-7 (x64)
Logical processors .. 8
Total memory size ... 9.879 GiB

Did you also increase the swap directories as well?



#5 MikiSJ

MikiSJ

    Surveyor 1

  • *****
  • Posts: 1,890
  • Joined: 24 Feb 2006
  • Loc: San Jose, CA

Posted 17 February 2021 - 06:35 PM

Yes, I added 3 new swap directories.



#6 joeytroy

joeytroy

    Viking 1

  • *****
  • topic starter
  • Posts: 715
  • Joined: 14 Aug 2020
  • Loc: Belen, NM, USA

Posted 17 February 2021 - 06:37 PM

Yes, I added 3 new swap directories.

Not bad at all then!



#7 imtl

imtl

    Fly Me to the Moon

  • *****
  • Posts: 7,277
  • Joined: 07 Jun 2016
  • Loc: Down in a hole

Posted 17 February 2021 - 07:23 PM

It's weird. I don't have those options when I go to Parallel processing. I don't have the last 4 parameters on the right! Is this only in the newest PI? My options end at the Allow using all available processing tickmark. 



#8 joeytroy

joeytroy

    Viking 1

  • *****
  • topic starter
  • Posts: 715
  • Joined: 14 Aug 2020
  • Loc: Belen, NM, USA

Posted 17 February 2021 - 07:53 PM

I’m currently running the newest version right now and all current updates including the newest WBP 2.0

#9 jonnybravo0311

jonnybravo0311

    Aurora

  • *****
  • Moderators
  • Posts: 4,833
  • Joined: 05 Nov 2020
  • Loc: NJ, US

Posted 17 February 2021 - 09:56 PM

Thanks for the tip on editing the global preferences, Joey. I was running with PI defaults - never even bothered looking at these settings previously. I ended up running a whole bunch of different configurations. My best score ended up being 4 swap directories (all on my MBP internal drive) and 8 each read/write threads. Final score:

 

med_gallery_347158_15661_345722.png

 

Not too shabby. Before the change, I was getting a considerably lower benchmark.


  • joeytroy likes this

#10 airscottdenning

airscottdenning

    Apollo

  • *****
  • Posts: 1,095
  • Joined: 22 Aug 2008
  • Loc: Colorado

Posted 18 February 2021 - 12:23 AM

Thanks for these suggestions. I run latest PI on a 2017 Mac Mini (6 cores/12 threads, 64 GB RAM).

 

Before your recommended changes I got 

Total =7460 CPU=9039 SWAP=4357

 

After I got

Total=8101 CPU=9116 SWAP=5592

 

For a tad less than 10% improvement.


  • joeytroy likes this

#11 airscottdenning

airscottdenning

    Apollo

  • *****
  • Posts: 1,095
  • Joined: 22 Aug 2008
  • Loc: Colorado

Posted 18 February 2021 - 08:55 AM

Another data point, on my 2016 MacBook Pro:

 

Speedup 3% total,

slight SLOWDOWN CPU,

43% improvement on swap


Edited by airscottdenning, 18 February 2021 - 08:55 AM.

  • joeytroy likes this

#12 corpusjonsey

corpusjonsey

    Sputnik

  • -----
  • Posts: 42
  • Joined: 11 Dec 2020
  • Loc: Northern Colorado

Posted 18 February 2021 - 09:18 AM

I tried this. Haha. my total performance is less than your improvements. 2010 i3. Guess I need an upgrade.


Edited by corpusjonsey, 18 February 2021 - 09:18 AM.


#13 joeytroy

joeytroy

    Viking 1

  • *****
  • topic starter
  • Posts: 715
  • Joined: 14 Aug 2020
  • Loc: Belen, NM, USA

Posted 18 February 2021 - 09:30 AM

Thanks for the tip on editing the global preferences, Joey. I was running with PI defaults - never even bothered looking at these settings previously. I ended up running a whole bunch of different configurations. My best score ended up being 4 swap directories (all on my MBP internal drive) and 8 each read/write threads. Final score:

 

Nice Jonny! Glad it's improving speed!

 

Thanks for these suggestions. I run latest PI on a 2017 Mac Mini (6 cores/12 threads, 64 GB RAM).

 

You bet Scott!

 

I tried this. Haha. my total performance is less than your improvements. 2010 i3. Guess I need an upgrade.

Yeah never fails, there seems to always be something we need to upgrade in AP lol.gif



#14 joeytroy

joeytroy

    Viking 1

  • *****
  • topic starter
  • Posts: 715
  • Joined: 14 Aug 2020
  • Loc: Belen, NM, USA

Posted 23 February 2021 - 03:27 PM

So I wanted to post an update. I recently decided to give OSX the old heave ho and installed Ubuntu 20.04.2 on my 2018 Mac Mini. To say I am impressed by the speed increase is an simply an understatement!

 

OSX Big Sur Speeds first run with out changes, second with 12 proc and 3 swaps

Screen-Shot-2021-02-17-at-2-59-47-PM.png

 

Ubuntu 20.04.2 Speeds first run with out changes, second with 12 proc and 3 swaps

afterchanges.png

 

If I did my math correct that is a huge increase over the second runs from OSX to Ubuntu. So glad I made the switch!

 

TOTAL= 42.9078% CPU=17.2724% SWAP=158.499%


Edited by joeytroy, 23 February 2021 - 03:28 PM.

  • airscottdenning, RedLionNJ and Snousi like this

#15 RolkaLT

RolkaLT

    Explorer 1

  • -----
  • Posts: 62
  • Joined: 17 Oct 2016
  • Loc: Woolwich, London, UK

Posted 22 October 2022 - 10:16 AM

Changes done: total 8 swapts, increased threads from 8 to 12 (11th Gen Intel® Core™ i7-11800H @ 2.30GHz   + 16GB RAM: Laptop)

 

before:

Performance Indices

Total performance ...... 12473

CPU performance ........ 14936

Swap performance ....... 7432

 

After:

Performance Indices

Total performance ...... 15163

CPU performance ........ 14649

Swap performance ....... 17782



#16 Snousi

Snousi

    Lift Off

  • -----
  • Posts: 1
  • Joined: 26 Mar 2023

Posted 26 March 2023 - 01:26 PM

One great way to amp up the swap, make some of your ram into ram disk/disks and assign them as swap folder with few additional folders from your HDD/SSD.

 

Also play with the read/write threat count, mine works faster with a larger number than i actually have.

 

Hopefully that cuda-support for everything is coming soon!

 

 

In that file i added is some of my penching results.

 

 

On the bottom, out of the box performance, thou i think i left hyperthreading thingie on for this test.

 

In the middle, threadcount is pumped up to 32/32 (thou i only have 20, but this seemed to work nicely, according to benchmarks), assigned 4 ram disks as swap disks (4gb each, it's a overkill, you don't need that much, it's more about the number than the size, i think).

 

In the top one, i added 3 more swap folders from my SSD. After 3, it started to actually get worse.

 

So, a nice and useful way to spend few hours with Pixinsight and find a combination that works for your PC.

 

Mine did get a lot faster with these settings, so, it was a good way to spend some time while waiting for data to process laugh.gif I could squeeze CPU past 20.000 Pixinsight benchmark points but it doesn't affect the overall speed a lot, so, saving my hardware Pixi-benches.jpg laugh.gif


Edited by Snousi, 27 March 2023 - 03:34 AM.


#17 smiller

smiller

    Vanguard

  • *****
  • Posts: 2,294
  • Joined: 27 Oct 2018
  • Loc: Vancouver Washington (not BC!)

Posted 27 March 2023 - 08:54 AM

Page 4 of this thread has some stacker performance comparisons and PI speed up hints and pointers to further hints:

 

https://www.cloudyni...and-lrgb/page-4


Edited by smiller, 27 March 2023 - 09:05 AM.

  • joeytroy likes this

#18 joeytroy

joeytroy

    Viking 1

  • *****
  • topic starter
  • Posts: 715
  • Joined: 14 Aug 2020
  • Loc: Belen, NM, USA

Posted 27 March 2023 - 09:08 AM

Page 4 of this thread has some stacker performance comparisons and PI speed up hint pointers:

 

https://www.cloudyni...and-lrgb/page-4

 

Steven,

 

Based on your data capture you may also want to show the speed differences with a RAM drive for swap as well 

 

https://joeytroy.com...ith-pixinsight/

 

Finally there is the tweaks for NVIDIA CUDA for using starnet++

 

https://joeytroy.com...s-your-starnet/


  • smiller likes this

#19 smiller

smiller

    Vanguard

  • *****
  • Posts: 2,294
  • Joined: 27 Oct 2018
  • Loc: Vancouver Washington (not BC!)

Posted 27 March 2023 - 09:11 AM

Steven,

 

Based on your data capture you may also want to show the speed differences with a RAM drive for swap as well 

 

https://joeytroy.com...ith-pixinsight/

 

Finally there is the tweaks for NVIDIA CUDA for using starnet++

 

https://joeytroy.com...s-your-starnet/

With my gazillion subs due to short exposures, I can’t use this, but rely on fast SSDs.



#20 urbanMark

urbanMark

    Viking 1

  • -----
  • Posts: 619
  • Joined: 08 Jul 2006
  • Loc: San Jose, CA

Posted 01 August 2023 - 04:24 AM

Great tips, thanks!

 

Don't forget things like closing other apps (including your web browser), adjusting your power profile, or even plugging in vs battery (for laptops). All these had significant impacts on my performance.

 

My ThinkPad X1 Extreme Gen 4 (i7-11800H @2.3GHz, 32GB RAM, dual M2 SSD's, Win 11 Pro, 16 threads) went from

 

Total = 8540  CPU= 13288 SWAP = 3456 (default settings, 8 threads, single swap file)

to

Total = 15548 CPU= 14503 SWAP=22257 (16 threads, 4 swap files on each SSD for a total of 8)

 

As you can see, most of the gain was in the swap. Selecting "enable thread CPU affinity" actually decreased my performance, so I switched it back off. Also, as you may have noticed, running the benchmark multiple times (with no changes) can yield surprising variation in results, most likely due to background OS activity or thermal throttling (esp with laptops).


  • Bob Denny likes this

#21 Jeff Morgan

Jeff Morgan

    James Webb Space Telescope

  • *****
  • Posts: 15,534
  • Joined: 28 Sep 2003
  • Loc: Prescott, AZ

Posted 01 August 2023 - 06:15 PM

So I wanted to post an update. I recently decided to give OSX the old heave ho and installed Ubuntu 20.04.2 on my 2018 Mac Mini. To say I am impressed by the speed increase is an simply an understatement!

 

OSX Big Sur Speeds first run with out changes, second with 12 proc and 3 swaps

Screen-Shot-2021-02-17-at-2-59-47-PM.png

 

Ubuntu 20.04.2 Speeds first run with out changes, second with 12 proc and 3 swaps

afterchanges.png

 

If I did my math correct that is a huge increase over the second runs from OSX to Ubuntu. So glad I made the switch!

 

TOTAL= 42.9078% CPU=17.2724% SWAP=158.499%

 

The FAQ page on the PixInsight website says that fastest performance will always be with a Linux system.

 

But I think I will trade my M1 iMac for a M2 Max Studio before I start beating my head against a Linux wall wink.gif




CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.


Recent Topics






Cloudy Nights LLC
Cloudy Nights Sponsor: Astronomics