Jump to content

  •  

CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.

Photo

What Hardware to Make PI Really Fast?

  • Please log in to reply
103 replies to this topic

#26 jerahian

jerahian

    Viking 1

  • *****
  • Posts: 619
  • Joined: 02 Aug 2018
  • Loc: Maine

Posted 18 September 2019 - 02:25 AM

Ramdisks will be faster than an SSD, but they are not worth the effort to use (except for benchmarking whistling.gif grin.gif) for day to day use.  You lose that memory for other tasks, and you have to set them up each time you boot up.  Also, you lose the contents when powering off.  For PI, those swap disks are what it uses for caching.  So if you do repeat processing (say after tweaking a variable), PI might be able to draw on what's in the swap.  The Ramdisk would be ok within a session, but if you come back the next day, etc, you won't get the benefit.

Actually, depending on the utility, RAM disks can be configured to be saved to an image file upon shutdown, and reloaded upon boot up, automatically, thereby acting more like a real disk but powered by RAM.


  • PirateMike and Chris Ryan like this

#27 Jim Waters

Jim Waters

    Gemini

  • *****
  • Posts: 3000
  • Joined: 21 Oct 2007
  • Loc: Phoenix, AZ USA

Posted 18 September 2019 - 02:44 AM

Since you have a lot of RAM, you can definitely try a RAM disk and see it's impact right now (as an exercise).  Although, I have to say, I kinda agree with Jon Rista's comments above.

 

Anyhoo, you can use ImDisk to easily create a RAM disk.  Just follow these instructions:  

https://www.maketech...m-disk-windows/

 

Just as a test, I just took my Acer Aspire 7 laptop, which has an Intel® Core™ i7-8750H CPU @ 2.20GHz with 16 GB of RAM and created a 4 GB RAM disk on it.  This took about 5 minutes to do, again, just as a test.

 

My benchmark before and after were:

Before:  http://pixinsight.co...CWF3WH27M4O4Y95

After:  http://pixinsight.co...25QWS9D42LM991U

 

-Ara

It was my understanding that the SW overhead for a RamDisk is high and outweighs any benefit when you are using a fast NVMe drive or SSD.  The last benchmarking I did was many many years ago.  If true moving the PI's Swap space to a RamDisk would be a benefit.  I will have to do some testing this week.  This would mean having a PC with larger memory - 64GBytes? 

 

Since RamDisks are a fixed size it would mean having 2 Swap Disk locations or a VERY large RamDisk.  You may need an overflow Swap Disk too. 

 

EDIT - looks like I have been away from technology too long...  I need to reconfigure my PC's and do some testing / benchmarking.  Thanks Chris....

https://www.geckoand...al-disk-memory/


Edited by Jim Waters, 18 September 2019 - 02:52 AM.

  • PirateMike and Chris Ryan like this

#28 PirateMike

PirateMike

    Mercury-Atlas

  • -----
  • topic starter
  • Posts: 2652
  • Joined: 27 Sep 2013
  • Loc: A Green Dot On A Blue Sea

Posted 18 September 2019 - 02:47 AM

Something to keep in mind before you spend a lot of money. This is the total time it takes to perform a completely automated, start to finish post-process of an image. So, your computer took 90 seconds to perform all of that processing, one process after another, back to back. A minute and a half.

 

How long does it take you to process an image? A minute and a half? If you say yes to that, my head might literally explode! tongue2.gif I'm going to assume it takes you...at least 30 minutes? Longer? An hour...two hours? With the kind of processing involved in the benchmark, I would say a lot of imagers, including myself, would probably take an hour to do it all.

 

SO...take that 90 seconds, and distribute all that processing time out over 30, 60, 90 MINUTES of total time. Do you think you are going to notice the difference in how long it takes to, say, do a sample pass of noise reduction on a small preview with TGV, between the system that gets the benchmark job done in 53 seconds vs. your system which gets it done in 90? How much difference in time to run, say, MMT do you think you would see with the faster build? Maybe a second? Two?

 

Do you think you would even notice a 5 second difference in how long it took TGV, or MMT, or HDRMT, or any one of those other longer running processes, when you were actually sitting there fiddling around with settings and figuring out how best to handle your data?

 

I use an "ancient" i7 4930k CPU, overclocked a bit, with 32gb of "ancient" DDR3 overclocked a bit, on an "old" SSD (not an M2) for the OS, with a similarly "old" SSD for data storage. Since PI got multi-threaded capabilities in its pre-processing tools, I have never felt that PI was slow. By far the slowest part of PI was the pre-processing, which when it was all single threaded took FOR-E-VER!!!! But now it blitzes right through all the pre-processing stuff, and lately I'm down to integrating 40, 50, 60 frames per channel, for a total of less than 200 most of the time, which has made it all go even faster. Once I am into the actual post-processing? The single largest bottleneck...is me. When running HEFTY processes like noise reduction, deconvolution, etc. I always do it on smaller previews to start. So the time to process is just seconds anyway. I then will fiddle around with the settings and re-apply to these previews over, and over, and over and over and over...until I have it all "just right". Then I'll apply it to the full image, which might take about a minute to process. I'll easily spend 10, 20, 30 times as much time fiddling around and getting things just right, than the amount of time PI actually takes to compute. Then I'm onto the next phase of processing, where I'll do the same thing...fiddle around until things are just right, using previews to speed things up, then apply to the full image just once.

 

I haven't upgraded my computer in some five years. PI does not feel slow at all. It isn't "instantaneous" for everything, but...look at the benchmarks. You've got about 40-50 seconds for high end modern windows rigs (expensive rigs!), vs. 90 seconds for your rig. I mean, at best, spending a lot of money on a new computer for PI might speed up the things PI itself does by a factor of...two? If you go the Linux route, and get one of the higher end Ryzens with lots of cores, you MIGHT get down into the 20 second range. That would be about 4x faster than your rig...but again...for the things PI itself does. That...minute or two of compute time. Out of the entire post-processing workflow that will usually take...30, 40, 60 minutes, if not longer.

 

Just to shed some light on what it is you would actually be spending your hard earned money on. wink.gif

You are right John. The pre-processing stuff does take a lot of time, adding, subtracting, dividing all those subs and putting them together into one image file. After that we are basically processing a single image file and most likely small files (previews) to boot. I get all that, you are 100% correct, no doubt about it.

 

 

 

But sometimes our own human nature gets in the way of efficiency and expediency or totally neglects those concepts when making decisions. Somebody once said "Necessity is the mother of all invention" but there is a follow on, a lowly whispered second part to the statement, one that not many people speak of, and that is... Inventions are the mother of all perceived necessities.

 

I admit that upgrading my computer is a perceived necessity, something that is not completely based on unyielding logic or need. It is something I want, not for logical reasons but for self satisfying reasons, and that I do understand and realize.

 

So now you make me think. Your logical statement has switched on my logical side of my brain. What will a human logic machine like me do in such a predicament, stuck between unbending cold reasoning and the very temporary warm sensation of having a very fast computer?

 

There is a beauty in having a finely tuned machine, turning at an almost unbelievable rate with nearly a vibration or a sound...

 

 

 

Well so much for poetry. Now I'm conflicted and you may have just pulled me out of the abyss. flowerred.gif

 

 

Miguel   8-)

 

.


Edited by PirateMike, 18 September 2019 - 02:49 AM.


#29 PirateMike

PirateMike

    Mercury-Atlas

  • -----
  • topic starter
  • Posts: 2652
  • Joined: 27 Sep 2013
  • Loc: A Green Dot On A Blue Sea

Posted 18 September 2019 - 02:57 AM

It was my understanding that the SW overhead for a RamDisk is high and outweighs any benefit when you are using a fast NVMe drive or SSD.  The last benchmarking I did was many many years ago.  If true moving the PI's Swap space to a RamDisk would be a benefit.  I will have to do some testing this week.  This would mean having a PC with larger memory - 64GBytes? 

 

Since RamDisks are a fixed size it would mean having 2 Swap Disk locations or a VERY large RamDisk.  You may need an overflow Swap Disk too. 

 

EDIT - looks like I have been away from technology too long...  I need to reconfigure my PC's and do some testing / benchmarking.  Thanks Chris....

https://www.geckoand...al-disk-memory/

And don't forget to add the cost side of things. One system may be better at shuffling around data, but is it's cost justifiable. How much more money will it cost to move a 64 bit of data around in a giga-fraction of a second?

 

Miguel   8-)

 

.



#30 Jon Rista

Jon Rista

    ISS

  • *****
  • Posts: 23721
  • Joined: 10 Jan 2014
  • Loc: Colorado

Posted 18 September 2019 - 02:59 AM

You are right John. The pre-processing stuff does take a lot of time, adding, subtracting, dividing all those subs and putting them together into one image file. After that we are basically processing a single image file and most likely small files (previews) to boot. I get all that, you are 100% correct, no doubt about it.

 

 

 

But sometimes our own human nature gets in the way of efficiency and expediency or totally neglects those concepts when making decisions. Somebody once said "Necessity is the mother of all invention" but there is a follow on, a lowly whispered second part to the statement, one that not many people speak of, and that is... Inventions are the mother of all perceived necessities.

 

I admit that upgrading my computer is a perceived necessity, something that is not completely based on unyielding logic or need. It is something I want, not for logical reasons but for self satisfying reasons, and that I do understand and realize.

 

So now you make me think. Your logical statement has switched on my logical side of my brain. What will a human logic machine like me do in such a predicament, stuck between unbending cold reasoning and the very temporary warm sensation of having a very fast computer?

 

There is a beauty in having a finely tuned machine, turning at an almost unbelievable rate with nearly a vibration or a sound...

 

 

 

Well so much for poetry. Now I'm conflicted and you may have just pulled me out of the abyss. flowerred.gif

 

 

Miguel   8-)

 

.

You can buy the rig that costs half as much for most of the gain. wink.gif Just as with AP gear, where you eventually get into the realm of diminishing returns, it is mostly the same with computers. You could spend a heck of a lot of money chasing down 20-30 second benchmark times, or, spend a LOT less for 45-50 seconds (or maybe even better, some of those high benchmarks are actually with relatively modest modern gear.)

 

Truly, by the time the benchmark takes less than a minute, there really isn't much point in continuing to spend money. To that end, you will probably notice that a LOT of that performance in the benchmark comes from swap time. Swap is really a scratch disk, where data used for the image history and undo/redo is saved. The amount of time it takes to save that data introduces a lag time before the process can actually start. Your CPU actually performs in line with the top performer in the list of 11 benchmarks you shared. Your bottleneck is really the swap performance.

 

Now...you could completely turn that around without upgrading anything. All you have to do is sacrifice some RAM for a ram disk, and put all your swap on that. A ram disk is going to be significantly faster than anything but an NVMe SSD, and even then to get the fastest possible NVMe speeds you need to spend quite a bit of money, and it still won't be as fast as a ram disk. The average NVMe still ain't going to compare to a ram disk. Ram is just so much faster. So if you really want to crank up those numbers, you might not even have to spend a penny (unless you don't have much RAM). Just configure some of your RAM as a ram disk and use that for storing your swap files. I'd bet your computer jumps to within the top three on your list.


Edited by Jon Rista, 18 September 2019 - 03:05 AM.

  • PirateMike likes this

#31 PirateMike

PirateMike

    Mercury-Atlas

  • -----
  • topic starter
  • Posts: 2652
  • Joined: 27 Sep 2013
  • Loc: A Green Dot On A Blue Sea

Posted 18 September 2019 - 03:51 AM

You can buy the rig that costs half as much for most of the gain. wink.gif Just as with AP gear, where you eventually get into the realm of diminishing returns, it is mostly the same with computers. You could spend a heck of a lot of money chasing down 20-30 second benchmark times, or, spend a LOT less for 45-50 seconds (or maybe even better, some of those high benchmarks are actually with relatively modest modern gear.)

 

Truly, by the time the benchmark takes less than a minute, there really isn't much point in continuing to spend money. To that end, you will probably notice that a LOT of that performance in the benchmark comes from swap time. Swap is really a scratch disk, where data used for the image history and undo/redo is saved. The amount of time it takes to save that data introduces a lag time before the process can actually start. Your CPU actually performs in line with the top performer in the list of 11 benchmarks you shared. Your bottleneck is really the swap performance.

 

Now...you could completely turn that around without upgrading anything. All you have to do is sacrifice some RAM for a ram disk, and put all your swap on that. A ram disk is going to be significantly faster than anything but an NVMe SSD, and even then to get the fastest possible NVMe speeds you need to spend quite a bit of money, and it still won't be as fast as a ram disk. The average NVMe still ain't going to compare to a ram disk. Ram is just so much faster. So if you really want to crank up those numbers, you might not even have to spend a penny (unless you don't have much RAM). Just configure some of your RAM as a ram disk and use that for storing your swap files. I'd bet your computer jumps to within the top three on your list.

It is still true that the law of diminishing return is a law!

 

I'm going to give the RamDisk a try, I just need to find out how to create one. So off to Google, be back soon.

 

Another thing, the acquisition computer in the garage is actually newer and faster than this PI computer. I think it would be a smart thing on my part to bring that one in the house (at some point) and "upgrade" that one for PI and use this one to control my AP setup.

 

 

Miguel   8-)

 

.


Edited by PirateMike, 18 September 2019 - 04:57 AM.


#32 PirateMike

PirateMike

    Mercury-Atlas

  • -----
  • topic starter
  • Posts: 2652
  • Joined: 27 Sep 2013
  • Loc: A Green Dot On A Blue Sea

Posted 18 September 2019 - 04:53 AM

Hey John,

 

I bought a RamDisk program for $9.99 and installed it to my computer. Here are the surprising (to me, probably not to you) results...

 

 

Before RamDisk

 

Benchmark - Baseline.png

 

 

After installing Ramdisk

 

Benchmark - With 12gig RamDisk.png

 

 

And here is the new updated report from the PI website

 

PixInsight Benchmark Reports - After RamDisk.png

 

 

 

I went from number 11 (now number 12 on this updated report) to number 2.

 

 

Ramdisk really is quite effective and very monetarily efficient. lol.gif

 

 

 

 

I don't know how to thank you enough for your (as always) excellent advice.

 

Miguel   8-)

 

,


Edited by PirateMike, 18 September 2019 - 04:55 AM.

  • Jim Waters, Jon Rista and Adun like this

#33 PirateMike

PirateMike

    Mercury-Atlas

  • -----
  • topic starter
  • Posts: 2652
  • Joined: 27 Sep 2013
  • Loc: A Green Dot On A Blue Sea

Posted 18 September 2019 - 08:15 AM

So now I am thinking of changing my upgraded machine to a Dual Boot machine. Linux for PI and Windows for everything else.

 

Are there any technical reasons that such a Dual Boot setup would somehow slow down PI as compared to a straight Single Boot Linux machine?

 

 

Miguel   8-)

 

.



#34 Kaydubbed

Kaydubbed

    Vostok 1

  • *****
  • Posts: 178
  • Joined: 29 Sep 2018
  • Loc: Kansas City, Missouri

Posted 18 September 2019 - 08:20 AM

I'm building a xeon E-2176M system with an m.2 drive and 32gb of RAM.

 

Really that is excessive though. People have been using PI for years with slower RAM and processors, and less cores. It just makes it a LOT easier to use when you don't have to wait 5-10 seconds after each change you make to your image. 


  • PirateMike likes this

#35 dhaval

dhaval

    Vendor

  • *****
  • Vendors
  • Posts: 1737
  • Joined: 21 Jul 2008
  • Loc: Round Rock, TX

Posted 18 September 2019 - 08:32 AM

So now I am thinking of changing my upgraded machine to a Dual Boot machine. Linux for PI and Windows for everything else.

 

Are there any technical reasons that such a Dual Boot setup would somehow slow down PI as compared to a straight Single Boot Linux machine?

 

 

Miguel   8-)

 

.

No, there is nothing technical that will hamper running PI on a dual boot platform as compared to a straight up Linux machine other than the fact that you are going to be working with smaller hard disk space - but that does not affect performance.

 

The other thing that you can think of is, just get a refurbished server that has dual CPU (Xeon 8-core per CPU) with server grade RAM and put Linux on it. Now you should be able to remote in to that machine and run PI there. Those cost about $850-$950 depending on how you build them. They are mostly HDDs (I haven't seen those NVMe type SSDs on them). You gain processing speed, loose some IO speed - personally, I would love to have more processing speed than IO speed - but that's me.

 

CS!

 

CS! 


  • PirateMike likes this

#36 xb39

xb39

    Vostok 1

  • -----
  • Posts: 124
  • Joined: 24 Nov 2013
  • Loc: Germany

Posted 18 September 2019 - 10:30 AM

Hi,

even running Linux in Windows using Hyper-V I have better results compared to running it in Win10: 

http://pixinsight.co...01X7M1J681UTORF


  • Jim Waters likes this

#37 bobzeq25

bobzeq25

    Hubble

  • *****
  • Posts: 16931
  • Joined: 27 Oct 2014

Posted 18 September 2019 - 12:00 PM

Re RAMDisk versus SSD.  When I built my PI machine, earlier this year, RAM prices were ridiculous, so I went SSD, using the best Samsung drive available.  Was pleased by the results.

 

The PI benchmark in 23 seconds (4 seconds are the swap speed) is quite decent, the record is maybe 17.

 

I've watched other people since.  The very fastest, bleeding edge people, like RAMDisks.  They tend to have very large amounts of RAM, like 64 or 128.  The SSD people get really good speed, also.  Even a top of the line 512GB SSD is a whole lot cheaper than 512 of RAM.  <grin>  You don't need _that_ much swap space, you can also use the SSD for other things.

 

It's a personal choice, not right/wrong.

 

My PI machine is strictly for AP (of course I have to do some browsing/downloading), so I have Linux (only) on it.  Makes life simpler.


Edited by bobzeq25, 18 September 2019 - 12:02 PM.

  • Chris Ryan likes this

#38 Jim Waters

Jim Waters

    Gemini

  • *****
  • Posts: 3000
  • Joined: 21 Oct 2007
  • Loc: Phoenix, AZ USA

Posted 18 September 2019 - 12:05 PM

Hey John,

 

I bought a RamDisk program for $9.99 and installed it to my computer. Here are the surprising (to me, probably not to you) results...

 

 

Before RamDisk

 

attachicon.gif Benchmark - Baseline.png

 

 

After installing Ramdisk

 

attachicon.gif Benchmark - With 12gig RamDisk.png

 

 

And here is the new updated report from the PI website

 

attachicon.gif PixInsight Benchmark Reports - After RamDisk.png

 

 

 

I went from number 11 (now number 12 on this updated report) to number 2.

 

 

Ramdisk really is quite effective and very monetarily efficient. lol.gif

 

 

 

 

I don't know how to thank you enough for your (as always) excellent advice.

 

Miguel   8-)

 

,

Which RamDisk did you buy?



#39 Jim Waters

Jim Waters

    Gemini

  • *****
  • Posts: 3000
  • Joined: 21 Oct 2007
  • Loc: Phoenix, AZ USA

Posted 18 September 2019 - 12:13 PM

Hi,

even running Linux in Windows using Hyper-V I have better results compared to running it in Win10: 

http://pixinsight.co...01X7M1J681UTORF

Which Linux / kernel and what was the configuration?



#40 james7ca

james7ca

    Fly Me to the Moon

  • *****
  • Posts: 7412
  • Joined: 21 May 2011
  • Loc: San Diego, CA

Posted 18 September 2019 - 12:15 PM

Has anyone tried using an Intel Optane AIC/PCIe drive with PixInsight? This is a relatively new type of persistent memory technology that sits somewhere between DRAM and NAND flash in terms of speed. It supposedly is just as fast in sequential read/writes as the best NAND flash NVMe drives but has random access that is about five times better. It also seems to have about ten times the endurance of NAND flash and when it is used as a direct substitute for DRAM it seems to benchmark only about 20% slower than DRAM (tentative findings, since processors that can use the so-called Optane DC memory chips are just starting to become available). The upside (against DRAM) is that the Optane memory technology is about ten times more dense that DRAM. So, it's possible to offer systems with hundreds of gigabytes of main memory for less cost than a system based upon DRAM.

 

I believe that Optane DC memory is only being tested for enterprise-level server applications right now, but you can purchase Optane-based drives for both M.2 and PCIe slots that start at under $300 (for a 280GB drive). In fact, if you had a computer with two open PCIe x4 slots you should be able to run two of these drives in a RAID configuration to produce a very large and fast PixInsight swap space with plenty of room remaining for persistent data storage. This would also be a really good system for software like AutoStakkert! since I know that SSDs speed up that application rather significantly (since you can easily have hundreds of gigabytes of lunar images to process in Autostakkert!).


Edited by james7ca, 18 September 2019 - 12:18 PM.

  • jdupton and Jon Rista like this

#41 Kaydubbed

Kaydubbed

    Vostok 1

  • *****
  • Posts: 178
  • Joined: 29 Sep 2018
  • Loc: Kansas City, Missouri

Posted 18 September 2019 - 01:48 PM

Re RAMDisk versus SSD.  When I built my PI machine, earlier this year, RAM prices were ridiculous, so I went SSD, using the best Samsung drive available.  Was pleased by the results.

 

The PI benchmark in 23 seconds (4 seconds are the swap speed) is quite decent, the record is maybe 17.

 

I've watched other people since.  The very fastest, bleeding edge people, like RAMDisks.  They tend to have very large amounts of RAM, like 64 or 128.  The SSD people get really good speed, also.  Even a top of the line 512GB SSD is a whole lot cheaper than 512 of RAM.  <grin>  You don't need _that_ much swap space, you can also use the SSD for other things.

 

It's a personal choice, not right/wrong.

 

My PI machine is strictly for AP (of course I have to do some browsing/downloading), so I have Linux (only) on it.  Makes life simpler.

RAM prices are not ridiculous. RAM is cheaper than ever. I remember when it was $200 a gig in 2003 for DDR2.Now you can pick up 32GB of blazingly-fast DDR4 for $120. Perhaps you mean m.2 nvme drives, which are still pretty expensive.



#42 TelescopeGreg

TelescopeGreg

    Apollo

  • -----
  • Posts: 1067
  • Joined: 16 Jul 2018
  • Loc: Auburn, California, USA

Posted 18 September 2019 - 01:59 PM

Just a note, if you use any sort of flash-based storage (SSD, et al), it is best to treat it as long-term volatile and not store anything on them that you really really want to keep.  They are inherently not permanent, as the act of writing to them causes permanent wear.  Modern drives are much better in this regard, but you can't cheat physics.



#43 View2

View2

    Vanguard

  • *****
  • Posts: 2049
  • Joined: 20 May 2016
  • Loc: Vancouver, WA USA

Posted 18 September 2019 - 02:01 PM

CRAY

#44 Kaydubbed

Kaydubbed

    Vostok 1

  • *****
  • Posts: 178
  • Joined: 29 Sep 2018
  • Loc: Kansas City, Missouri

Posted 18 September 2019 - 04:16 PM

Just a note, if you use any sort of flash-based storage (SSD, et al), it is best to treat it as long-term volatile and not store anything on them that you really really want to keep.  They are inherently not permanent, as the act of writing to them causes permanent wear.  Modern drives are much better in this regard, but you can't cheat physics.

Yes..but modern NVME and SSD drives will always last far beyond their warranties, which is usually capacity x 50. Anyone who has run a RAID array, or just used a mechanical hard drive over time knows they will not get close to that before they get a drive failure; usually mechanically-related. A  terabyte NVME m,2 drive that has a quarter of its memory partitioned for durability will last for years and years and years.

 

So yes, quarterly backups of your frames to either cloud, or NAS storage is smart, but no, don't lose sleep thinking your new NUC's NVME drive will fail because you put on 20GB a week on it a week in light and flat frames. 


  • TelescopeGreg likes this

#45 xb39

xb39

    Vostok 1

  • -----
  • Posts: 124
  • Joined: 24 Nov 2013
  • Loc: Germany

Posted 18 September 2019 - 05:23 PM

Which Linux / kernel and what was the configuration?

Hi Jim,

what excately do you want to know what is not listed in the System information of the link?

 

I used 32 of my 64 GB for the Ubuntu Linux in the Hyper-V. From this 32 GB I created a 16GB Ramdisk in the Ubuntu sub system. In addition, I created in one of my 2 1TB Samsung SSD 970 EVO´s 4 swap folders.

 

I tested many different configurations and with the PixInsight Core 01.08.06.1457 (x64), running PI under Linux has had the biggest impact:

 

Hope this make sense.

 

Best regards

Stefan

Attached Thumbnails

  • 2950x_PI_Benchmarks.JPG


#46 Adun

Adun

    Mercury-Atlas

  • *****
  • Posts: 2611
  • Joined: 02 Dec 2016

Posted 18 September 2019 - 06:04 PM

RAM prices are not ridiculous. RAM is cheaper than ever. I remember when it was $200 a gig in 2003 for DDR2.Now you can pick up 32GB of blazingly-fast DDR4 for $120. Perhaps you mean m.2 nvme drives, which are still pretty expensive.

 
DDR4 prices did rise enormously through 2016 and 2017. Several manufacturers were sued for apparent collusion.
 
I had to postpone my long overdue upgrade because of this, until 2018Q3

Edited by Adun, 18 September 2019 - 06:05 PM.

  • PirateMike likes this

#47 Scott1244

Scott1244

    Sputnik

  • -----
  • Posts: 26
  • Joined: 30 Dec 2017

Posted 18 September 2019 - 06:23 PM

Jeez, I bought a  6 year old Lenovo I5 with 8 gigs of RAM for $120.00 off Craigslist and it runs PI blazingly fast. Compared to CAD/CAM and video processing, PI does not need very much horsepower to run well. At least that's been my experience


Edited by Scott1244, 18 September 2019 - 06:23 PM.

  • PirateMike likes this

#48 bobzeq25

bobzeq25

    Hubble

  • *****
  • Posts: 16931
  • Joined: 27 Oct 2014

Posted 18 September 2019 - 06:41 PM

RAM prices are not ridiculous. RAM is cheaper than ever. I remember when it was $200 a gig in 2003 for DDR2.Now you can pick up 32GB of blazingly-fast DDR4 for $120. Perhaps you mean m.2 nvme drives, which are still pretty expensive.

No, I meant RAM, and this was in January 2019.  That may have been the high, RAM has gone down in price significantly. 

 

https://www.extremet...-roof-stuck-way

 

32GB of (excellent) RAM cost me about $400. (!)  The state of the art m.2 nvme 512GB drive (which is what I use for swap, and more) was about $150.


Edited by bobzeq25, 18 September 2019 - 06:45 PM.


#49 Jon Rista

Jon Rista

    ISS

  • *****
  • Posts: 23721
  • Joined: 10 Jan 2014
  • Loc: Colorado

Posted 18 September 2019 - 06:47 PM

Has anyone tried using an Intel Optane AIC/PCIe drive with PixInsight? This is a relatively new type of persistent memory technology that sits somewhere between DRAM and NAND flash in terms of speed. It supposedly is just as fast in sequential read/writes as the best NAND flash NVMe drives but has random access that is about five times better. It also seems to have about ten times the endurance of NAND flash and when it is used as a direct substitute for DRAM it seems to benchmark only about 20% slower than DRAM (tentative findings, since processors that can use the so-called Optane DC memory chips are just starting to become available). The upside (against DRAM) is that the Optane memory technology is about ten times more dense that DRAM. So, it's possible to offer systems with hundreds of gigabytes of main memory for less cost than a system based upon DRAM.

 

I believe that Optane DC memory is only being tested for enterprise-level server applications right now, but you can purchase Optane-based drives for both M.2 and PCIe slots that start at under $300 (for a 280GB drive). In fact, if you had a computer with two open PCIe x4 slots you should be able to run two of these drives in a RAID configuration to produce a very large and fast PixInsight swap space with plenty of room remaining for persistent data storage. This would also be a really good system for software like AutoStakkert! since I know that SSDs speed up that application rather significantly (since you can easily have hundreds of gigabytes of lunar images to process in Autostakkert!).

So this is really interesting to me... I hadn't heard of Optane DC memory yet, but if it really has these advantages, in a general sense, that is very impressive.

 

I generally upgrade my computers these days about once every 6 years. :p I used to upgrade all the time, a part here and there a  couple times a year and a new motherboard, CPU and RAM every year or two. Spent a lot of money on computer junk in my younger years... My current computer is pretty long in the tooth, and it doesn't even seem to support NVMe PCI-e type drives, which I found out when I went to get a new SSD for a new Win 10 install (trying to keep my Windows 8 around for AP so I don't have to worry about forced reboots, and for a couple other things) so I can start using new DirectX 12 raytracing functionality with games (still a big gamer).

 

In a year or two here, I'm hoping this Optane memory will be more cost effective and more mainstream, though. Sounds like we could finally have multi-TB drives with it, at more reasonable costs than with current SSD.


  • PirateMike likes this

#50 Jim Waters

Jim Waters

    Gemini

  • *****
  • Posts: 3000
  • Joined: 21 Oct 2007
  • Loc: Phoenix, AZ USA

Posted 18 September 2019 - 07:18 PM

Hi Jim,

what excately do you want to know what is not listed in the System information of the link?

 

I used 32 of my 64 GB for the Ubuntu Linux in the Hyper-V. From this 32 GB I created a 16GB Ramdisk in the Ubuntu sub system. In addition, I created in one of my 2 1TB Samsung SSD 970 EVO´s 4 swap folders.

 

I tested many different configurations and with the PixInsight Core 01.08.06.1457 (x64), running PI under Linux has had the biggest impact:

 

Hope this make sense.

 

Best regards

Stefan

Sorry - I should have been clearer.  I was wondering if it was just Ubuntu 18.04.2 release that's certified to run PI.  Also what about Mint and MX Linux?  I think I am going to stick with the current build of Ubuntu.


Edited by Jim Waters, 18 September 2019 - 07:19 PM.

  • PirateMike likes this


CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.


Recent Topics






Cloudy Nights LLC
Cloudy Nights Sponsor: Astronomics