Return to the Cloudy Nights Telescope Reviews home page


Astrophotography and Sketching >> CCD Imaging & Processing

Pages: 1 | 2 | (show all)
jerryyyyy
Pooh-Bah


Reged: 10/06/11

Loc: Stanford, California
Re: 32-bit PixInsight? DOA? new [Re: pfile]
      #5691733 - 02/20/13 08:48 PM

Sorry to have set off a bit of a computing firestorm today.

So, bottom line is that I ran PI in 32-bit all day and I seem to have survived. It does peg the dual processors at 100%, but I never ran out of memory or used virtual memory. It does not seem to me to be worse than some processes I have had to run in Maxim.

However, on the Dynamic Background Extraction (DBE), there must be a lot of secret sauce involved here. I have some vignette images and it was very hard to get anywhere with them, so I cropped them and ran as TIFF files after cropping in CS5. This worked better, but would be nice to have some best practices for DBE like:

1. How many points are optimal?
2. Is there an optimal pattern?
3. Do you target noisy areas?
4. Can you run it multiple times without a problem?

Anyway, I am learning and probably will buy the software as I need all the help I can get.

PS I have an SSD on my computer so if I run into virtual memory it is hardly noticed...


Post Extras: Print Post   Remind Me!   Notify Moderator  
pfile
Post Laureate


Reged: 06/14/09

Re: 32-bit PixInsight? DOA? [Re: jerryyyyy]
      #5691756 - 02/20/13 08:59 PM

see my reply on the PI forum.

re: SSD that makes paging fast but you still have this hard 2GB/3GB limit that effectively forms a 'jail' for PI. it just can not allocate more memory past that point and memory intensive processes (like imageintegration) can fail.


Post Extras: Print Post   Remind Me!   Notify Moderator  
neptun2
scholastic sledgehammer


Reged: 03/04/07

Loc: Bulgaria
Re: 32-bit PixInsight? DOA? new [Re: pfile]
      #5692151 - 02/21/13 02:59 AM

One last reply to Wade's post - 64-bit windows uses emulation layer to run 32-bit applications. Here are more details:

http://en.wikipedia.org/wiki/WoW64

The CPU can execute both 32-bit and 64-bit code but not simultaneously so when you use 64-bit windows the core of the OS and drivers are 64-bit and CPU works in 64-bit mode so the WoW emulation layer is needed for 32-bit programs to run.


Post Extras: Print Post   Remind Me!   Notify Moderator  
WadeH237
Pooh-Bah


Reged: 02/24/07

Loc: Snohomish, WA
Re: 32-bit PixInsight? DOA? new [Re: neptun2]
      #5692471 - 02/21/13 10:20 AM

Wow64 is a set of 32 bit Windows libraries. Most of them are the same dlls that ship on the 32 bit only version of Windows. Wow64 also provides thunks for calls into the 64 bit kernel. These thunks are a translation layer to provide interoperability between user mode 32 bit code and 64 bit kernel code. The term "emulation" technically could describe this thunking layer because it makes 32 bit calls from user mode look like 64 bit calls to the kernel, and vice versa. It is not software emulation of the x86 instruction set, which is what was implied at the start of this thread.

On Itanium, Windows includes IA32Exec, which actually does software emulation of 32 bit x86. On this platform, 32 bit code does run slower than 64 bit code because of this emulation.

On 64 bit Intel CPUs other than Itanium, and on all 64 bit AMD CPUs, 32 bit code is executed natively. I have done substantial work with benchmarking code compiled for both 32 and 64 bits. In cases where there is no memory address advantage, 32 bit code frequently runs a few percentage points faster than 64 bit code on the same hardware for these CPUs.

My direct experience is with the previous generation of CPUs and compilers My expectation is that, at some point, we'll see parity in execution performance between 32 and 64 bits, or maybe a slight advantage to 64 bits, due to optimizations for 64 bit.

At the end of the day, though, the advantage to 64 bit is in memory addressing. Actual execution speed will remain very close between them, because they both run natively on the CPU.

And to tie this in some way to the topic, PixInsight benefits greatly from the 64 bit memory address space. It makes perfect sense to me that they would want to focus on that environment. Similar software, like CCDStack and Photoshop also run much better on 64 bit.


Post Extras: Print Post   Remind Me!   Notify Moderator  
jerryyyyy
Pooh-Bah


Reged: 10/06/11

Loc: Stanford, California
Re: 32-bit PixInsight? DOA? new [Re: WadeH237]
      #5692523 - 02/21/13 10:59 AM Attachment (10 downloads)

I agree with what Wade says and use the 64-bit machines in my lab for the severe data crunching that is memory intensive.

To return to image processing for a bit, I am having some problems determining if the crud I am seeing in my images recently is due to poor flats or skyglow. PixInsight has led me to try to make that distinction.

This shot is of M80 from early this AM. It is stacked in maxim using flats from Sky Flats Assistant. When I use these flats with narrow band images there is no crud at all. But with LRGB you see this and I think it is skyglow because I am in a Red Zone just south of San Francisco.

The screen shot is of my attempt to do DBE. I am pretty sure this is skyglow because it is not exactly centered on the image (the flats are) and that probably reflects some sort of gradiant across the sky.

Anyway, opinions cheerfully received and appreciated. I still need to clean up these images one way or another.

Finally, to reduce skuglow I am trying to use CCDComander to just take images as high as possible in the sky, i.e. only when obejcts is above 45 or 55 DEG. I had to take M80 at above 25 DEG since it is low in the sky... was the last object on my Messier list.


Post Extras: Print Post   Remind Me!   Notify Moderator  
pfile
Post Laureate


Reged: 06/14/09

Re: 32-bit PixInsight? DOA? new [Re: jerryyyyy]
      #5692735 - 02/21/13 01:11 PM

have the flat subs been calibrated before making the master flat?

Post Extras: Print Post   Remind Me!   Notify Moderator  
Mike Wiles
scholastic sledgehammer


Reged: 02/04/09

Loc: Goodyear, AZ
Re: 32-bit PixInsight? DOA? new [Re: jerryyyyy]
      #5692762 - 02/21/13 01:25 PM

Quote:

To return to image processing for a bit, I am having some problems determining if the crud I am seeing in my images recently is due to poor flats or skyglow. PixInsight has led me to try to make that distinction.

This shot is of M80 from early this AM. It is stacked in maxim using flats from Sky Flats Assistant. When I use these flats with narrow band images there is no crud at all. But with LRGB you see this and I think it is skyglow because I am in a Red Zone just south of San Francisco.

The screen shot is of my attempt to do DBE. I am pretty sure this is skyglow because it is not exactly centered on the image (the flats are) and that probably reflects some sort of gradiant across the sky.




When you apply the DBE process what correction method are you using? Division? or Subtraction? The gradient looks more like skyglow from light pollution - which would benefit from subtraction.

With DBE if you're trying to correct skyglow gradients, use subtraction.

If you're trying to correct vignetting or flat field issues - use division.

Mike


Post Extras: Print Post   Remind Me!   Notify Moderator  
jerryyyyy
Pooh-Bah


Reged: 10/06/11

Loc: Stanford, California
Re: 32-bit PixInsight? DOA? new [Re: pfile]
      #5693320 - 02/21/13 06:46 PM

Quote:

have the flat subs been calibrated before making the master flat?




Thanks for the comment but I do not know how to do this. I just use the standard settings in maxim for Callibration Settings. It does create a master flat. I am going to collect a fresh set tonight and start from scratch to make sure it is not a flats issue.


Post Extras: Print Post   Remind Me!   Notify Moderator  
jerryyyyy
Pooh-Bah


Reged: 10/06/11

Loc: Stanford, California
Re: 32-bit PixInsight? DOA? new [Re: Mike Wiles]
      #5693325 - 02/21/13 06:50 PM

Quote:

Quote:

To return to image processing for a bit, I am having some problems determining if the crud I am seeing in my images recently is due to poor flats or skyglow. PixInsight has led me to try to make that distinction.

This shot is of M80 from early this AM. It is stacked in maxim using flats from Sky Flats Assistant. When I use these flats with narrow band images there is no crud at all. But with LRGB you see this and I think it is skyglow because I am in a Red Zone just south of San Francisco.

The screen shot is of my attempt to do DBE. I am pretty sure this is skyglow because it is not exactly centered on the image (the flats are) and that probably reflects some sort of gradiant across the sky.

When you apply the DBE process what correction method are you using? Division? or Subtraction? The gradient looks more like skyglow from light pollution - which would benefit from subtraction.

With DBE if you're trying to correct skyglow gradients, use subtraction.

If you're trying to correct vignetting or flat field issues - use division.




I believe it is skyglow and am using substraction. The image does improve with substraction, but is not perfect. I seem to get as good if not better results in CS5 with Gradient Eliminator plug-in. I first tried division and that did little if anything I could see.

Mike




Post Extras: Print Post   Remind Me!   Notify Moderator  
pfile
Post Laureate


Reged: 06/14/09

Re: 32-bit PixInsight? DOA? new [Re: jerryyyyy]
      #5693376 - 02/21/13 07:24 PM

Quote:

Quote:

have the flat subs been calibrated before making the master flat?




Thanks for the comment but I do not know how to do this. I just use the standard settings in maxim for Callibration Settings. It does create a master flat. I am going to collect a fresh set tonight and start from scratch to make sure it is not a flats issue.




i have never used maxim so i just don't know what those standard settings do.

flats have to be calibrated with at least bias frames. fixed offsets have to be removed from the flat since the flat will be multiplied by some factor 0..1 in order to properly remove the vignetting. if the bias is left in the flats this can mess up the computations.

you can also make dark flats which are darks that have the same exposure length as your flats. those will also contain the bias signal. with cooled CCDs the amount of thermal noise in a typical flat (2-10s or so) is so low that calibration with a master bias will suffice.


while DBE has extracted something that looks like a sky gradient, the original images really look to me like incorrect flat subtraction was done. you've really got to get your flats in order before trying to subtract sky gradients. you've got an artificial gradient in there that should be removable with good flats.


Post Extras: Print Post   Remind Me!   Notify Moderator  
jerryyyyy
Pooh-Bah


Reged: 10/06/11

Loc: Stanford, California
Re: 32-bit PixInsight? DOA? new [Re: pfile]
      #5695259 - 02/22/13 05:44 PM

Thanks for all the information, which I will process carefully. I am making progress. I am running the files also through Deep Sky Stacker to compare the results... have to say they look better than Maxim with the default settings. I am pretty sure it is skyglow and have various countermeasures for that... mostly careful cropping to minimize the gradients.

BTW, I am on travel for a week. Does anyone know if PixInsight works offline? Or do you have to be on the internet. Tried to process some files while on my lastr flight and got an error about connecting.

Does this go away with the paid copy?


Post Extras: Print Post   Remind Me!   Notify Moderator  
pfile
Post Laureate


Reged: 06/14/09

Re: 32-bit PixInsight? DOA? new [Re: jerryyyyy]
      #5695368 - 02/22/13 06:42 PM

that message goes away with the paid copy for sure. i think the trial just wants to phone home to keep the trial from somehow going on forever...

Post Extras: Print Post   Remind Me!   Notify Moderator  
Pages: 1 | 2 | (show all)


Extra information
13 registered and 16 anonymous users are browsing this forum.

Moderator:  Oldfield, droid, Scott in NC, bilgebay, WOBentley 

Print Thread

Forum Permissions
      You cannot start new topics
      You cannot reply to topics
      HTML is disabled
      UBBCode is enabled


Thread views: 1175

Jump to

CN Forums Home




Cloudy Nights Sponsor: Astronomics