Jump to content

  •  

CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.

Photo

Pixinsight using all processor and almost no RAM?

  • Please log in to reply
8 replies to this topic

#1 mewmartigan

mewmartigan

    Apollo

  • *****
  • topic starter
  • Posts: 1,103
  • Joined: 02 Jul 2008
  • Loc: outside of Pittsburgh, PA

Posted 24 February 2020 - 08:40 PM

So I was thinking about upgrading my 32GB of RAM to 64GB to help with bigger file sizes when my new camera arrives.

 

Here is a screenshot of my computer task manager while doing a CFA Drizzle Integration on 73 frames. I noticed that while the processor was maxed out, it was barely using any RAM. Is this normal? I want to say the task manager looked similar during BPP as well.  Maybe 32GB is more than enough?

Pixinsight, Windows 10 and the files were all running off an SSD.

 

 https://photos.app.g...UbeujDQ3NTYbQj6


Edited by mewmartigan, 24 February 2020 - 09:09 PM.


#2 calypsob

calypsob

    Fly Me to the Moon

  • *****
  • Posts: 5,075
  • Joined: 20 Apr 2013

Posted 24 February 2020 - 09:10 PM

Registration is usually the RAM hog, I did not realize drizzle was a cpu heavy process.  I always thought it was a RAM process as well


  • psandelle and mewmartigan like this

#3 Dynan

Dynan

    Soyuz

  • *****
  • Posts: 3,649
  • Joined: 11 Mar 2018
  • Loc: NOLA

Posted 24 February 2020 - 09:11 PM

You might look at these threads to use that RAM to help the CPU:

 

https://www.cloudyni...-2#entry9855355

 

https://www.cloudyni...s/#entry9990034


  • mewmartigan likes this

#4 jdupton

jdupton

    Vanguard

  • *****
  • Posts: 2,274
  • Joined: 21 Nov 2010
  • Loc: Central Texas, USA

Posted 24 February 2020 - 09:35 PM

Marcus,

 

   I think your observation is completely within reason. I have also noted that Drizzle Integration is not very demanding of RAM. (I have only done CFA Drizzle Integration but also note that it uses CPU much more heavily that RAM.) However, regular integration is very RAM intensive. I have reached over 90 GB of virtual memory being allocated on my 64 GB RAM system while integrating ~750 22 MB (Raw, 132 MB CFA XISF) frames. I think this particular light use of RAM is because Drizzle Integration uses the Drizzle files generated by Image Integration so all of the heavy work of pixel rejection has been taken care of previously.

 

 

John


Edited by jdupton, 24 February 2020 - 09:36 PM.

  • mewmartigan likes this

#5 mewmartigan

mewmartigan

    Apollo

  • *****
  • topic starter
  • Posts: 1,103
  • Joined: 02 Jul 2008
  • Loc: outside of Pittsburgh, PA

Posted 24 February 2020 - 09:44 PM

You might look at these threads to use that RAM to help the CPU:

 

https://www.cloudyni...-2#entry9855355

 

https://www.cloudyni...s/#entry9990034

Dynan, thanks for the links. I had read before about setting up a RAM disk but I don't want to take away any of that 32GB until I know its being under-utilized.

 

Marcus,

 

   I think your observation is completely within reason. I have also noted that Drizzle Integration is not very demanding of RAM. (I have only done CFA Drizzle Integration but also note that it uses CPU much more heavily that RAM.) However, regular integration is very RAM intensive. I have reached over 90 GB of virtual memory being allocated on my 64 GB RAM system while integrating ~750 22 MB (Raw, 132 MB CFA XISF) frames. I think this particular light use of RAM is because Drizzle Integration uses the Drizzle files generated by Image Integration so all of the heavy work of pixel rejection has been taken care of previously.

 

 

John

John,

 

Thanks for the info. I will take a look at the task manager the next time I run BPP. I will still upgrade to the 64GB of RAM either way.

 

My subframes are currently also about 22MB but its amazing how big they get. The ASI2600MC Pro I have on order generates 52MB files so I guess I may be looking at 150-180MB debayered files?



#6 jdupton

jdupton

    Vanguard

  • *****
  • Posts: 2,274
  • Joined: 21 Nov 2010
  • Loc: Central Texas, USA

Posted 24 February 2020 - 09:54 PM

Marcus,

 

   Your ASI2600MC Frames will be about 312 MB each once DeBayered. When you go to integrate more than a hundred or so, you will eat up every last bit of the physical RAM very quickly and then Windows will start allocating virtual memory (on the system drive) and use a lot of that also. Be sure to keep about 100 GB free space on your drive which contains the Windows Virtual Memory Swap File. If it runs out of drive space, you will get the PI "Out Of Memory" error / abort and have to start the Image Integration run again after clearing drive space.

 

 

John


Edited by jdupton, 24 February 2020 - 09:54 PM.

  • psandelle, mewmartigan and Dynan like this

#7 mewmartigan

mewmartigan

    Apollo

  • *****
  • topic starter
  • Posts: 1,103
  • Joined: 02 Jul 2008
  • Loc: outside of Pittsburgh, PA

Posted 25 February 2020 - 08:04 AM

John,

 

Thanks for the warning. Guess those debayered files are bigger than I assumed. I do have a dedicated drive for doing the processing. Actual storage of files that aren't being worked on are on a separate HDD.

 

Thanks!

 

Marcus



#8 mewmartigan

mewmartigan

    Apollo

  • *****
  • topic starter
  • Posts: 1,103
  • Joined: 02 Jul 2008
  • Loc: outside of Pittsburgh, PA

Posted 27 February 2020 - 01:43 PM

Piggybacking off this question. I found a good deal on some RAM and my question is....how much RAM is enough before my processor itself becomes the bottleneck?

 

I upgraded to 64GB. I can get to 128GB (or 96GB) for a pretty good price but will Pixinsight be able to take advantage of it? Or is my 6 (7? 8?) year old 3.50GHz Intel Xeon with 6 cores/12 logical processors going to slow me down before the software takes full advantage of 128GB anyway?

 

Thanks!



#9 OldManSky

OldManSky

    Gemini

  • *****
  • Posts: 3,085
  • Joined: 03 Jan 2019
  • Loc: Valley Center, CA USA

Posted 27 February 2020 - 03:39 PM

Looking at your screenshot, it appears the CPU usage (pegged out at 100% most of the time) drops down at exactly the same time your disk usage goes up.

Which would lead me to conclude that PI is paging bits of an image from disk at a time, throwing all the cores/threads it can at that image bit, writing that part back out, and repeating.

So in this case (drizzle), PI's being smart and only working on part of an image at a time, which is very efficient use of RAM.

That may not be quite the case when you're stacking 50 big images...:)


  • mewmartigan likes this


CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.


Recent Topics






Cloudy Nights LLC
Cloudy Nights Sponsor: Astronomics