Jump to content

  •  

CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.

Photo

Sony A7S - Split sensor and banding

  • Please log in to reply
138 replies to this topic

#26 sharkmelley

sharkmelley

    Mercury-Atlas

  • *****
  • topic starter
  • Posts: 2,787
  • Joined: 19 Feb 2013

Posted 04 January 2016 - 06:46 PM

Thanks for posting your results. I'm still waiting for a non-rainy night so I can do further on-scope experiments with the observatory roof open to the night sky.

 

Mark



#27 schwim

schwim

    Viking 1

  • *****
  • Posts: 598
  • Joined: 23 Oct 2014

Posted 08 January 2016 - 03:33 AM

No new subs but have been experimenting...

 

6x1200s darks - no banding

16x450s darks - no banding

 

I took my last integration (previously pictured above with bands) and split the RGB channels out. Blue and Green look ok. The red channel looks like this:

 

Screen Shot 2016-01-08 at 12.59.26 AM.jpg

 

Noting that its easy to see in the red channel, I debayered my dark integrations and checked - maybe there's something there, but I'm really stretching my imagination to see it. I then checked my flat master. It's *very* subtle, but its there in the red channel. 

 

Screen Shot 2016-01-08 at 1.05.18 AM.jpg

 

Remembering that the bands/vertical split really "popped" when I ran a background extraction, I decided to run an ABE on my master flat after debayering. It's there, plain as day:

 

Screen Shot 2016-01-08 at 1.08.33 AM.jpg

 

For comparison I checked the master flat from a series I captured on 12/25, where interestingly the final light integration did not show the banding noticeably. I don't see the banding in that flat at all using the above methods. I don't remember ever seeing this banding prior to this either. This means I can point at a date - 12/26 - when this started happening to me. I'll need to investigate what changed. I did re-seat my filter due to dust (prior thread). I did start imaging at > 5min subs. I don't think anything else changed. Unfortunately, I've been deleting non-keeper RAW files like nuts because I keep running out of disk space, so I may not be able to go investigate the source data.

 

In summary:

  • Not present in darks, bias that I could confidently admit to
  • Present in flat master after ABE and STF
  • Present in flat master red channel
  • Present in light RGB integration
  • NOT meaningfully present in G B channels alone for all of the above
  • Present in the red channel clearly on light and flat frames

This tells me that light is part of the problem, which makes me think it could be a reflection of some sort. I may run down to best buy and grab an A7S to try out with the same scenario to see if I can reproduce it.

 

Has anyone with a non-modified or non-full spectrum camera been able to reproduce this?

 

EDIT: I just found it in a master flat from November. It looks a little different, and it didn't make it into the final integration as far as I can tell. I was using a different filter at that point so I need to test that out.


Edited by schwim, 08 January 2016 - 03:46 AM.


#28 sharkmelley

sharkmelley

    Mercury-Atlas

  • *****
  • topic starter
  • Posts: 2,787
  • Joined: 19 Feb 2013

Posted 08 January 2016 - 04:03 AM

I haven't seen the problem in darks either - last night I stacked 360x30sec darks and still no problem. However Mike Malik has posted a 20min dark that does appear to show the left/right discontinuity.

My flats taken before the A7S was modified still show similar banding.

I have a set of daylight experiments planned for the weekend which should help understand what is going on - to take a whole range of flats under identical conditions at various levels of sensor saturation.

Mark

#29 Corsica

Corsica

    Vendor (Innovationsforesight)

  • *****
  • Vendors
  • Posts: 647
  • Joined: 01 Mar 2010
  • Loc: PA - USA

Posted 08 January 2016 - 08:36 AM

I haven't seen the problem in darks either - last night I stacked 360x30sec darks and still no problem. However Mike Malik has posted a 20min dark that does appear to show the left/right discontinuity.

My flats taken before the A7S was modified still show similar banding.

I have a set of daylight experiments planned for the weekend which should help understand what is going on - to take a whole range of flats under identical conditions at various levels of sensor saturation.

Mark

Mark,

 

If you do some daylight tests I would suggest to try without the light pollution reduction filter, if any, too.


Edited by Corsica, 08 January 2016 - 08:39 AM.


#30 schwim

schwim

    Viking 1

  • *****
  • Posts: 598
  • Joined: 23 Oct 2014

Posted 08 January 2016 - 12:15 PM

I took the master flat that I ran ABE on as shown in this post above and split the R, G, and B channels out. The vertical split is present in all. The strong horizontal defects are clearly mostly in the red channel but there is some evidence of banding in all 3. It's amazing how the right side of the image is free of these defects.

 

RED

 

flat_R.jpg

 

GREEN

 

flat_G.jpg

 

BLUE

 

flat_B.jpg

 



#31 bwallan

bwallan

    Viking 1

  • -----
  • Posts: 563
  • Joined: 08 Jul 2010
  • Loc: Alberta, Canada

Posted 08 January 2016 - 05:45 PM

I took the master flat that I ran ABE on as shown in this post above and split the R, G, and B channels out. The vertical split is present in all. The strong horizontal defects are clearly mostly in the red channel but there is some evidence of banding in all 3. It's amazing how the right side of the image is free of these defects.

 

RED

 

attachicon.gifflat_R.jpg

 

GREEN

 

attachicon.gifflat_G.jpg

 

BLUE

 

attachicon.gifflat_B.jpg

All I can add is "interesting"...  Have never seen anything quite like this off any of the A7 cameras.  I have seen sort of similar streaking off a QHY8 Pro CCD camera resulting from a very bright star and photosite "overflow".  Its software had a checkbox to apply clamping which eliminated the problem.

 

bwa



#32 schwim

schwim

    Viking 1

  • *****
  • Posts: 598
  • Joined: 23 Oct 2014

Posted 08 January 2016 - 06:05 PM

Agreed that it is interesting. Also interesting is that the banding appears to be comprised of dim areas, almost as if these areas can't expose beyond that point but the surrounding areas can, or at least not equivalently. Just a wild theory, but a theory nevertheless. That, or there is a shadow, but it persists in the same exact area in spite of camera rotation or OTA orientation. And the sharp vertical "split" is difficult to explain were it optical in nature.

 

Also, the fact that it is in my (relatively) very short exposure flats, but *not* in darks/bias seems to point to it being a response to light. And the fact that some of my earlier projects don't show it at all. On 12/25 I imaged M76 using 300s subs - no banding that I can see. Since 12/26, all light sub integrations have banding but they're at 450s and well ETTR per the BOC and LR histo measurements. The major difference is the length of the exposure, and secondary is the part of the sky.

 

Of course, we've been in full-on rainy mode for Phoenix since I figured out this was happening, so all I can do is theorize and not experiment. Maybe tonight if it clears up.



#33 ccs_hello

ccs_hello

    Voyager 1

  • *****
  • Posts: 10,408
  • Joined: 03 Jul 2004

Posted 08 January 2016 - 08:18 PM

Using Schwim's R, G, B flats to review, they do not look like "bands" as seen in CCD (column readout or black-clipping) but

more like paintbrush streaks.

 

The artifacts in Red-channel flat even have wedged shape.  This is very, very odd.

 

Clear Skies!

 

ccs_hello



#34 sharkmelley

sharkmelley

    Mercury-Atlas

  • *****
  • topic starter
  • Posts: 2,787
  • Joined: 19 Feb 2013

Posted 09 January 2016 - 03:57 AM

That's a very informative test Schwim.  When I get time later this weekend I'll do exactly the same with one of my own my master flats.

 

I wouldn't care about these bands if they calibrated out of my images but they don't calibrate out with a standard flat using multiplicatively (i.e. dividing image by the flat).  Therefore I am thinking that an additional subtractive process ("Sony band removal") might be required in my processing sequence.

 

Mark



#35 sharkmelley

sharkmelley

    Mercury-Atlas

  • *****
  • topic starter
  • Posts: 2,787
  • Joined: 19 Feb 2013

Posted 09 January 2016 - 02:58 PM

I've taken some flats with a 50mm prime lens (no filters in the optical train) then split the master flat into 4 channels (R,G1,G2,B) and performed a PixInsight DBE background extraction.  Here are the results:

 

Channel1 = G1

e1.jpg

 

Channel2 = B

e2.jpg

 

Channel3 = R

e3.jpg

 

Channel4 = G2

e4.jpg

 

As with Schwim's results, the vertical discontinuity appears in all channels but the horizontal banding (left half of sensor) seems to be most obvious in the R and B channels.

 

Mark


Edited by sharkmelley, 09 January 2016 - 03:21 PM.


#36 schwim

schwim

    Viking 1

  • *****
  • Posts: 598
  • Joined: 23 Oct 2014

Posted 09 January 2016 - 06:25 PM

Good idea. 

 

I decided to rule out the optical train completely, so I took a series of 20 flats without a lens or filter attached to the camera. I kept the iso at 3200 and used an exposure of 1/8000 and pointed the camera at a uniformly lit wall. 20 subs integrated, debayered, split into RBG and ABE and stretch on each. 

 

Finding: The optical train does not appear to be contributing to the problem. This seems to be 100% a camera issue.

 

R

 

nolens_R.jpg

 

G

 

nolens_G.jpg

 

B

 

nolens_B.jpg

 

I'm not sure I see much point in experimenting with the camera beyond this. It looks like this is just the nature of the camera (at least our copies) and we may have to accept it for what it is. I'm going to start digging into how we might effectively process this out, or change the exposure strategy, or both. I do know that I have integrations that don't have this banding in it, but I'm not clear on how to get back there. The only thing I can think of at this point is that the subs were not so far ETTR . Still, the sample flats I took earlier today, at the 20-30% mark on the histo still showed the issue.

 

I'm kind of at a loss. My last project has very strong banding in it, and I'm able to *mostly* hide it through thoughful stretching, but I can still tell its there.

 

I posted over on the PixInsight forums a few days ago to see if anyone had an idea as to how to process this out but I haven't gotten a response yet.

 

http://pixinsight.co...hp?topic=9469.0


Edited by schwim, 09 January 2016 - 06:26 PM.


#37 sharkmelley

sharkmelley

    Mercury-Atlas

  • *****
  • topic starter
  • Posts: 2,787
  • Joined: 19 Feb 2013

Posted 09 January 2016 - 06:50 PM

The next experiment I want to do is to take sets of flats with the same optics and light conditions but with different exposure times to vary the amount of light hitting the sensor. 

 

I'm expecting to find that the banding differs according to the amount of light hitting the sensor i.e. it does not behave linearly.  If so, this will then explain why the banding does not calibrate out of my images and might also provide a clue about what kind of processing might remove the banding.

 

Mark



#38 schwim

schwim

    Viking 1

  • *****
  • Posts: 598
  • Joined: 23 Oct 2014

Posted 09 January 2016 - 07:12 PM

I tried that earlier today but wasn't after what you mention directly. That is, I put my "walking about" lens on the camera, stuck it atop a tripod and took flats. I took 20 each at different exposures, with the histogram curve at various spots - 25%, 55%, far ETTR. The banding/split is definitely more pronounced in the exposures that are farther to the right on the histogram, but its there in all of them. I was hoping there might be a "floor" on the histo under which they go away.

 

I suppose if we had a proper way of doing this we could build a response curve for the entire sensor. These sure do look like areas that are less sensitive relative to those around them, especially in the red.

 

I can re-take these tonight using a constant flat source and share them.

 

It would be interesting to see using a similar test whether our sensors have the banding in the same spot.



#39 bwallan

bwallan

    Viking 1

  • -----
  • Posts: 563
  • Joined: 08 Jul 2010
  • Loc: Alberta, Canada

Posted 09 January 2016 - 08:04 PM

Good idea. 

 

I decided to rule out the optical train completely, so I took a series of 20 flats without a lens or filter attached to the camera. I kept the iso at 3200 and used an exposure of 1/8000 and pointed the camera at a uniformly lit wall. 20 subs integrated, debayered, split into RBG and ABE and stretch on each. 

 

Finding: The optical train does not appear to be contributing to the problem. This seems to be 100% a camera issue.

 

R

 

attachicon.gifnolens_R.jpg

 

G

 

attachicon.gifnolens_G.jpg

 

B

 

attachicon.gifnolens_B.jpg

 

I'm not sure I see much point in experimenting with the camera beyond this. It looks like this is just the nature of the camera (at least our copies) and we may have to accept it for what it is. I'm going to start digging into how we might effectively process this out, or change the exposure strategy, or both. I do know that I have integrations that don't have this banding in it, but I'm not clear on how to get back there. The only thing I can think of at this point is that the subs were not so far ETTR . Still, the sample flats I took earlier today, at the 20-30% mark on the histo still showed the issue.

 

I'm kind of at a loss. My last project has very strong banding in it, and I'm able to *mostly* hide it through thoughful stretching, but I can still tell its there.

 

I posted over on the PixInsight forums a few days ago to see if anyone had an idea as to how to process this out but I haven't gotten a response yet.

 

http://pixinsight.co...hp?topic=9469.0

This discussion has tweaked my interest, so I shot a few flats and tried to duplicate your results; I couldn't.  Still not sure how you're getting this banding and image split??  Strange?

 

I shot RAW flats against the florescent light in my office with a Hutech HEUIB-II filter installed.  I converted the RAW flats to TIF format in Lightroom and then to Luminance & RGB files in ImagesPlus.  The results can be seen at: https://picasaweb.go...feat=directlink

 

What are you doing differently than me??

 

I should add the flats were shot at 1/8000 sec. & ISO 3200.

 

bwa


Edited by bwallan, 09 January 2016 - 08:05 PM.


#40 schwim

schwim

    Viking 1

  • *****
  • Posts: 598
  • Joined: 23 Oct 2014

Posted 10 January 2016 - 12:12 AM

Thanks for running your test!

 

For the flat testing above, I'm integrating the RAWs in PI, then splitting the channels, running an ABE against each, then stretching. Alternately, you can run an ABE directly against the integration. I get the same basic result.



#41 sharkmelley

sharkmelley

    Mercury-Atlas

  • *****
  • topic starter
  • Posts: 2,787
  • Joined: 19 Feb 2013

Posted 10 January 2016 - 03:47 AM

 

This discussion has tweaked my interest, so I shot a few flats and tried to duplicate your results; I couldn't.  Still not sure how you're getting this banding and image split??  Strange?

 

I shot RAW flats against the florescent light in my office with a Hutech HEUIB-II filter installed.  I converted the RAW flats to TIF format in Lightroom and then to Luminance & RGB files in ImagesPlus.  The results can be seen at: https://picasaweb.go...feat=directlink

 

What are you doing differently than me??

 

I should add the flats were shot at 1/8000 sec. & ISO 3200.

 

bwa

 

 

If you don't use PixInsight (or similar) then an alternative way to enhance the banding is to create a new layer in Photoshop where you blur the flat and then subtract the blurred version from the original.  Stretch the result as required.  I would suggest of blur of around 50 pixels as a starting point.

 

You won't see this problem in everyday terrestrial photos otherwise the reaction from the photographic community would be deafening!.  It only appears because background subtraction and stretching are typical operations in the processing sequence for astro-images.

 

Mark


Edited by sharkmelley, 10 January 2016 - 03:54 AM.


#42 sharkmelley

sharkmelley

    Mercury-Atlas

  • *****
  • topic starter
  • Posts: 2,787
  • Joined: 19 Feb 2013

Posted 10 January 2016 - 05:20 AM

I've now perfomed the experiment I have wanted to do for some time.  The idea is to produce a master flat at different exposure lengths to see if the banding changes.  In the images below I have shown channel 1 which is one of the two green channels.  The results will be split across two posts because of the Cloudy Nights limit on images per post.

 

ISO 100 10 x 1/3sec:

master1.jpg

 

ISO 100 10 x 1/5sec:

master2.jpg

 

ISO 100 10 x 1/10sec:

master3.jpg

 

ISO 100 10 x 1/20sec:

master4.jpg

 

continued on next post ...


Edited by sharkmelley, 10 January 2016 - 05:50 AM.


#43 sharkmelley

sharkmelley

    Mercury-Atlas

  • *****
  • topic starter
  • Posts: 2,787
  • Joined: 19 Feb 2013

Posted 10 January 2016 - 05:21 AM

continued from previous post ...

 

ISO 100 10 x 1/40sec:

master5.jpg

ISO 100 10 x 1/80sec:

master6.jpg

ISO 100 10 x 1/160sec:

master7.jpg

 

So there we have it - the banding most definitely changes according to how much light has hit the sensor.  This is the reason it doesn't calibrate out.

 

Mark


Edited by sharkmelley, 10 January 2016 - 05:50 AM.


#44 sharkmelley

sharkmelley

    Mercury-Atlas

  • *****
  • topic starter
  • Posts: 2,787
  • Joined: 19 Feb 2013

Posted 10 January 2016 - 10:20 AM

Here's an animation from one of those flats, cycling through 4 channels:

SonyA7SBandingAnimation.gif

 

Mark

 



#45 sharkmelley

sharkmelley

    Mercury-Atlas

  • *****
  • topic starter
  • Posts: 2,787
  • Joined: 19 Feb 2013

Posted 10 January 2016 - 10:30 AM

Animation from another of those flats, cycling through the 4 channels:

SonyA7SBandingAnimation2.gif

 

The only conclusion I've drawn so far is that the number and complexity of the banding increases with the amount of light captured.

 

I can't see any way at all of ever being able to calibrate out the banding.  This has become a serious showstopper for me.  I have so many images waiting to be processed and I have no way to remove the banding.

 

It is far worse to remove than Canon banding because:

1) It tends to be wedge shaped

2) It is restricted to one half of the sensor.

 

Mark


Edited by sharkmelley, 10 January 2016 - 10:47 AM.


#46 ccs_hello

ccs_hello

    Voyager 1

  • *****
  • Posts: 10,408
  • Joined: 03 Jul 2004

Posted 10 January 2016 - 11:53 AM

re:   schwim post #36

 

I think it might be sensor temperature related, mainly from main motherboard heat sources  ISO3200 with stacking/stretch just made it more detectable

 

re: Mark's study

 

I think I might have a reason for that

and sadly, it is related to imperfect stitching.  Here it is...

 

In a normal, non-stitched sensor, there are support circuitry surrounding the actual sensel (photon sensitive, the pixels) area.

Then there are some metallic links (still circuitry) passing signals from pixel to neighboring pixels.  They need to be as uniform as possible.

 

It is likely Mark's copy (the IMX235AQR sensor) isn't the ideal one...

 

In a stitched sensor, in the middle boundary line (the stitching seam), there is no room to add any peripheral-positioned supporting circuitry in between.

So the right-half of the sensor has to pass signal to the left-half side.

Minor imperfect connection (few ohm differences, a bit more or a bit less capacitive crosstalk) will make the signal passing thru less ideal.

Worse one will have longer lines until it cancelled out (say a complete RESET voltage level to clear the quantum well) and less one will see such artifact cancelled earlier.

The combination of several lines will make it look like a wedge where in the middle stitching point is widest.

 

R, G1. G2, and B channels are having different signal paths thus will look a bit different.

 

Above is just a hypothesis.

 

------------

 

I am not casting a nail on the FF image sensor.

I think in the long run (or with enough complains), SONY will move on to the stacked sensor design.  (Today only see in Exmor-RS in its BSI products.)

The stacked image sensor, the circuit is in the back and will have much larger room to play with. A minor difference in stitching variance will have little impact due to thicker geometry overall can be used.

 

BTW, now I am curious why don't we heard similar complains on other A7 family of FF mirrorless?

 

Clear Skies!

 

ccs_hello


Edited by ccs_hello, 10 January 2016 - 11:54 AM.


#47 sharkmelley

sharkmelley

    Mercury-Atlas

  • *****
  • topic starter
  • Posts: 2,787
  • Joined: 19 Feb 2013

Posted 10 January 2016 - 12:06 PM

Here is the result from another experiment.  This time I have taken flats at ISO 100 using every shutter speed from 1/30 sec to 1 sec - i.e. 16 master flat frames recorded all in continuous session under exactly the same daylight conditions.  The animated gif (too large to post on this forum but see link) clearly shows that the bands widen as the exposure time increases and more light reaches the sensor.

 

https://drive.google...GkzLWN4RHc/view

 

I have a theory.  As ccs_hello explains, the banding is "seeded" by the discontinuity.  For some reason I don't fully understand, the width of the banding increases with the amount of light falling.  This might explain the wedge shaped nature of the bands - as the band enters the vignette region, the light falls off and so does the width of the band.

 

Anyway, I am now on the point of junking my Sony A7S and buying a DSLR more suited to serious deep sky astrophotography.  But before I do so, can I be sure that full frame Sony Exmor sensors in Nikon cameras do not have the same issue?  Or should I just go for a APS-C?

 

Mark


Edited by sharkmelley, 10 January 2016 - 12:52 PM.


#48 schwim

schwim

    Viking 1

  • *****
  • Posts: 598
  • Joined: 23 Oct 2014

Posted 10 January 2016 - 01:36 PM

Wow. That's crazy. Since it effectively "moves" when you change exposure, maybe if you capture the flats in a wide range of exposures you may be able to get the differences out?



#49 sharkmelley

sharkmelley

    Mercury-Atlas

  • *****
  • topic starter
  • Posts: 2,787
  • Joined: 19 Feb 2013

Posted 10 January 2016 - 01:52 PM

Wow. That's crazy. Since it effectively "moves" when you change exposure, maybe if you capture the flats in a wide range of exposures you may be able to get the differences out?

 

Yes, it is completely crazy.  I've never experienced any imaging problem as intractable as this one.  Even during a single night's imaging, the sky background levels and therefore the banding will change from frame to frame.  I am convinced it is completely impossible to calibrate out.

 

Mark


Edited by sharkmelley, 10 January 2016 - 01:53 PM.


#50 schwim

schwim

    Viking 1

  • *****
  • Posts: 598
  • Joined: 23 Oct 2014

Posted 10 January 2016 - 02:45 PM

Last night I experimented with my light capturing. I pointed at M81 & M82 as per my previous images in this thread, and started adjusting my exposure down until I had a bit more separation from the right in the histo as seen in Lightroom. All were at iso3200. I started at 450s exposures, which is what I used above. This consistently had the curve about 10% separated from the right. Interestingly, it wasn't until I got to less than half that time (180s) when the histo curve started moving appreciably to the left. I settled on 120s subs, which put the curve at about 65% from the left.

 

I captured 30x120s lights, then 30x120s darks. I took flats this morning by putting t-shirt taught over the end of the OTA while pointed skyward. I adjusted iso and exposure to put the curve about 45-50%.

 

The banding and split remains in the flat. However, the fully calibrated light integration looks "OK". Not perfect - I can still see some of the banding. Leaving the flats out of the integration, the banding is difficult to see, even if it is there, but the result is clearly not flattened, there's a lot more noise and the banding could well be lost in it.

 

This is disappointing because the shorter exposures mean less detail in the lights.

 

I'm thinking that skyfog is contributing to all of this strongly. I have terrible skyfog here ("grey" area, one step below white per darksitefinder.com).  The longer exposed lights would seemingly have more propensity to "band and split" as your experiments show - more light thrown at the sensor accentuates this. Add the "always on" banding in the flats and I think it augments the problem rather than reduces it.

 

My RAWs are here if anyone wants to take them for a spin:

 

https://www.dropbox....K6d7VsVoia?dl=0

 

And the master files:

 

https://www.dropbox....NuOoODgMXa?dl=0

 

NOTE: These master files are "quick and dirty". I just wanted a quick result rather than taking the time to tweak the integrations.


Edited by schwim, 10 January 2016 - 02:46 PM.



CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.


Recent Topics






Cloudy Nights LLC
Cloudy Nights Sponsor: Astronomics