22 replies to this topic

[moxican]

I am trying to put together a lightweight grab & go set up and want to buy a mirrorless camera. There are a few options Sony, Panasonic, Samsung, Pentax, Olympus. The problem is that they all have their own RAW file and I do not know if stacking software are compatible with them. I use Nebulosity4 now, and I know it works with Canon and Nikon Raws.

Anyone using any cameras from the above mentioned list? If yes, how do you register and calibrate your exposures? Images of examples are also welcome.

[james7ca]

I stack my Sony NEX images using PixInsight. Note that many of the stacking packages use the open-source decoder dcraw, which handles many different cameras.

 

  https://en.wikipedia.org/wiki/Dcraw

[moxican]

Thanks James, but the issue with Dcraw that it "develops" the raw files into Tif format, meaning that it is no longer raw file. If I remember right I've read it somewhere on this forum that it is not a good method. People complained of outcome images. 

Once the raw file is turned into Tif format it is a color image. So, how does any stacking software would subtract Bias, Darks and Flats?

I do not have Pixinsight, does it also use dcraw?

Edited by moxican, 08 July 2018 - 09:49 PM.

[james7ca]

dcraw has options that allow different formats for its output, it doesn't have to be TIFF.

 

Here is a brief summary on the dcraw commands to output to different formats:

 

In versions 8.25 and later, "dcraw -T" writes TIFF output with metadata. To write other formats:

dcraw -c crw_0001.crw | pnmtofits > crw_0001.fits
dcraw -c crw_0001.crw | pnmtopng > crw_0001.png
dcraw -c crw_0001.crw | ppmtobmp > crw_0001.bmp
dcraw -c crw_0001.crw | cjpeg > crw_0001.jpeg

 

However, you'll have to check on what options are allowed by various dcraw front-ends. The actual command-line options may be different.

 

Here is the command line that PixInsight uses to open a Sony NEX RAW file:

 

  dcraw -4 -o 0 -r 1 1 1 1 -D -t 0 -k 0 -H 1

Edited by james7ca, 09 July 2018 - 06:05 AM.

[whwang]

I believe you can use DSS to stack Sony and Pentax raw files, and it's free.  I used DSS for Sony A7R and Pentax 645z files.  It works with no problems.

 

DCRaw convert raw files into TIFF, but this doesn't mean the TIFF are not raw.  It's just a different format.  You can ask DCRaw to maintain all the original information (data linearity etc) in the output TIFF file, like what James said above.  As far as I can tell, Canon's CR2 file itself is also based on the TIFF format, but packed with multiple layers of resized images for quick preview, and with compression.  So, stop thinking that TIFF cannot be raw.  As long as the pixel values are unaltered, it's raw.

 

Cheers,

Wei-Hao

[bobzeq25]

To emphasize the above.  Your "dcraw mangles the data" would apply to Canons and Nikons as well.  Yet a great many very serious imagers use PiXinsight to stack their data from those.

 

I don't think they're either foolish or incompetent.

 

By the way, deviations from "pure raw" often occur inside the camera, perhaps on the sensor.  Noise reduction, compression, etc.

Edited by bobzeq25, 08 July 2018 - 11:22 PM.

[SKYGZR]

To confirm, DSS stacks Pentax .dng files just fine

[bulrichl]

There is a new RAW module for PixInsight (since february 2018) that is based on libraw (instead of dcraw):

 

https://pixinsight.c...p?topic=12126.0

 

This RAW module is not yet part of the current PI version, but can be installed as a replacement of the DSLR_RAW module.

 

Bernd

[Tonk]

Most good stacking software does the calibration steps on each light frame *before* stacking - and the debayer step is only done on the final output image. This is certainly the case with DSS (and the debayer is a linear conversion).

 

A common mistake is when people pre treat light, dark, flat and bias RAW files and debayer them to full colour images with colour balance and gamma corrections applied (often the automatic default) before feeding them to a stacking program. Under these circumstances calibration does not work as expected and the rssults are compromised.

Edited by Tonk, 09 July 2018 - 05:07 AM.

[james7ca]

Most good stacking software does the calibration steps on each light frame *before* stacking - and the debayer step is only done on the final output image. This is certainly the case with DSS (and the debayer is a linear conversion).

 

A common mistake is when people pre treat light, dark, flat and bias RAW files and debayer them to full colour images with colour balance and gamma corrections applied (often the automatic default) before feeding them to a stacking program. Under these circumstances calibration does not work as expected and the rssults are compromised.

This is true, but dcraw doesn't have to debayer the image before it is output to the file container.

 

In fact, for complete coverage here is the dcraw command to perform a pure RAW conversion from a Sony NEX file (.ARW):

 

  dcraw -4 -o 0 -r 1 1 1 1 -D -t 0 -k 0 -H 1

 

and here is the output to debayer that same file:

 

  dcraw -4 -o 0 -w -q

 

And, here is a link to the command-line options for dcraw:

 

  https://www.cybercom...aw/dcraw.1.html

[Tonk]

I wasn't disputing what dcraw can do (I didnt even mention it) instead I was highlighting a common pre stacking mistake that people can make and that modern stacking programs are designed to avoid this mistake if you stick to feeding in raw data (in contrast some now well respected stacking programs in their early incarnations circa 2004 did debayer before calibration - dcraw was not involved - and yes you got strange artifacts as a result)

Edited by Tonk, 09 July 2018 - 08:16 AM.

[gbeaton]

Just have your camera output is DNG. Having said that, unfortunately Nebulosity still doesn’t handle DNGs from the Pentax. I just discovered Siril. It’s pretty good at stacking DNG, including Pentax but be warned it’s fits file may be a mirror image compared to what you use now.

[lock042]

 it’s fits file may be a mirror image compared to what you use now.

As discussed on the Siril forum, I do not agree with this statement.

Edited by lock042, 11 July 2018 - 02:53 AM.

[ImNewHere]

Save the Pentax files in .dng format and use the latest 64 bit version of DSS.

[lock042]

Siril uses libraw that is able to natively read pentax files.

 

I've tested this file: http://www.rawsample...X_K10D_SRGB.DNG

Here the result

Edited by lock042, 11 July 2018 - 05:54 AM.

[17.5Dob]

Most good stacking software does the calibration steps on each light frame *before* stacking - and the debayer step is only done on the final output image. This is certainly the case with DSS (and the debayer is a linear conversion).

 

A common mistake is when people pre treat light, dark, flat and bias RAW files and debayer them to full color images with color balance and gamma corrections applied (often the automatic default) before feeding them to a stacking program. Under these circumstances calibration does not work as expected and the results are compromised.

+1

DSS, PI, Nebulosity, Sequator, etc. etc.  all work exactly same way and use the same RAW conversion engine (outside of the new module for PI).

There are only two very vocal people in the AP community that advocate pre processing your RAWS in ACR or LR and then stacking the TIF's.

The other several 10,000' s ++++  just stack RAWS using any of the above programs.

 

[Tonk]

Hallas is an unfortunate distraction in this regard. Who is the other one (a Hallas desciple?)?

[sharkmelley]

Hallas is an unfortunate distraction in this regard. Who is the other one (a Hallas desciple?)?

ClarkVision?

[17.5Dob]

ClarkVision?

Bingo

[gbeaton]

Bingo

Really? I have to check that out (about Clarkvision).

 

One thing I'm beginning to wonder about is this green cast in the output of stackers, including PI - as I've seen on a youtube tutorial. It didnt make sense to me that it would be like that. Now I'm beginning to believe that they just aren't converting properly. I dont think they are applying the colour conversion matrix. It seems they are just applying a simple 1:1:1 conversion of the RAW pixel value to RGB. This will not produce a faithful preproduction of the scene since there is overlapping light from the green spectrum into the red, for example. Am I right in this conclusion - i.e. the stackers like DSS (and PI it seems) are using a 1:1:1 colour conversion?

Edited by gbeaton, 13 July 2018 - 05:41 AM.

[james7ca]

There are options in the PixInsight format converter to use a white balance from the DSLR camera. Otherwise, if you do a pure RAW conversion there is no color adjustment made to the pixel values. Look under PI's DSLR_RAW format converter (go to the Format Converter tab and double click on the DSLR_RAW entry to view the options). Generally, however, you probably don't want to apply any color correction with the format conversion debayering on files that have had bias/DF calibrations and you can always change the color balance during the post processing.

 

Here is a simple shot of the blue sky that I took using a Sony NEX camera. The image in the background was opened using the camera white balance setting in DSLR_RAW while the foreground image was opened with no white balance (they are the same source file). After each file was opened I applied an auto screen transfer stretch to the image. You can see that the version that received no white balance is somewhat green or cyan, while the camera color balance produced a deep blue (maybe too blue, but a greater stretch on the linear source would likely make it appear closer to what you might consider normal).

 

And, the DSLR_RAW options dialog in PI. Note, if you change any of these setting make certain that you go back to "Pure RAW" before you do any processing of your astro subs.

Attached Thumbnails

• 
• 

Edited by james7ca, 13 July 2018 - 10:56 AM.

[gbeaton]

Thanks. I figured PI had a setting. But my point is that colour balance is only part of colour conversion. Without the colour matrix applied (pure raw it seems to be called), the colour is not correct because each pixel has to be adjusted for slight green and slight blue components in the red cell, for example.

[greasemonkey]

To confirm, DSS stacks Pentax .dng files just fine

 

There is a new RAW module for PixInsight (since february 2018) that is based on libraw (instead of dcraw):

 

https://pixinsight.c...p?topic=12126.0

 

This RAW module is not yet part of the current PI version, but can be installed as a replacement of the DSLR_RAW module.

 

Bernd

 

I concur,  I have used DSS for my pentax k70 and seems to work fine

 

As far as PI it seems to process my pentax raw files just fine, but just downloaded the trial today and still exploring it. Spent my entire Sunday just playing around with it   but looks promising