Jump to content

  •  

CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.

Photo

Pattern noise with asi178mm non-cooled with H-alpha imaging

solar astrophotography imaging
  • Please log in to reply
8 replies to this topic

#1 AJamesB

AJamesB

    Vostok 1

  • *****
  • topic starter
  • Posts: 170
  • Joined: 17 Nov 2015
  • Loc: Kihei, HI

Posted 02 April 2020 - 07:39 PM

Sorry if this is a topic that has been hammered to death.  I did research it and didn't find much, and what I did find was at least a year old, often much older.

 

Up until recently I'd been using a DMK41 for my solar h-alpha imaging.  I recently acquired an asi178mm.  Love the camera (the small pixels work great when using a reducer to get full disk images), however when pushing the wavelet sharpening as much as I did with the DMK, I get some very obvious pattern noise.  I can add blur in wavelets to remove it, but then I lose a decent portion of the sharpening.  What I did find while researching seemed to indicate that pattern noise manifests itself when using USB3, but is diminished with USB2, but obviously the latter is a much slower frame rate than the former.

 

Included is a sample of what I'm talking about.  I've pushed it a bit further than what I could normally do with the DMK just to make it extra obvious.  Is this a known issue?  If so, is there a simple remedy, or do I need to revert to using usb2?  Or is this just a natural limit of the camera and all is working as intended? 

 

Capture was on a lunt 100, no reducer or powermate.  During the capture I used max frame rate, USB 3.0, gain of 0, I forget the shutter speed but it was on the very low end, histogram pushed to the right without clipping, and a ROI of about 75% of max, giving me about 50-60fps.  Image is a stack of about 750 frames.

 

Let me know if you need any additional information, and thanks in advance!

 

jk0Tra.jpg


Edited by ammonthenephite, 02 April 2020 - 07:51 PM.

  • ghutton likes this

#2 jwestervelt

jwestervelt

    Viking 1

  • *****
  • Posts: 748
  • Joined: 01 Sep 2012
  • Loc: Phoenix, Arizona

Posted 02 April 2020 - 08:03 PM

Congratulations, you've stumbled across the dreaded Sony Exmor flaw. 

At this point, I am 99.99% certain that this is a sensor defect and not an issue with the camera circuitry or drivers.  In other words, ASI, QHY, and consumers like you and I are the victims and Sony is in the wrong.

I feel that I have sufficient evidence to support my claim that the sensor is defective, and there could even be a class action suit brought about, so I'm not going to get into the details of what I have discovered along with a few others who are actively researching this. Just know that you are not alone in this matter.

The TL;DR of what I can share at this point is that the sensor acts as if it has a Bayer filter in place despite being monochrome.  Of course, it doesn't really have a filter, but there is a definite bias in the readout of the sensor that follows a strict 2x2 image site pattern exactly like a bayer mask.  For example, assign the 2x2 matrix as follows, where g and G are green1 and green2...
Rg
GB

What you find is that when taking flats, G and g are perfectly balanced, but R is overexposed and B is underexposed.  EVERY Exmor sensor I have tested has exhibited an identical behavior.  Additionally, this problem only manifests itself during short exposure mode (AKA video) and will not happen when the camera is put into long exposure (LX) mode. 

Unfortunately, the reason why this absolutely blows for owners of this camera is that planetary, lunar, and solar imaging is wrecked.  Image stacking software will "micro-align" on the "Bayer" pattern and your effective resolution is quartered.  Additionally, when setting your exposure value by watching the histogram, it is the "red" pixels that are driving the upper end of the histogram so the "green" and "blue" pixels, basically 75% of the sensor, gets underexposed. 

Your options are as follows:

  • Use 2x2 binning and kiss most of your resolution goodbye.
  • Correct the images in post-processing, critically this must be done BEFORE stacking 
  • Live with it.

 

Regarding the image correction, I have a utility that will fix the .ser file, but it only works in linux at the moment and it only works with the .ser file format.  Additionally, if you overexposed the video and the red pixels clipped (255 or 65535 depending upon bit depth) then there isn't much that can be done, but no one should be making that mistake in the first place.

If you want to verify that you have this issue, you can send me a .ser file with a single frame of video captured, and I can provide a report showing the bias of your 4 pixel groups.  This analysis is more accurate for a image flat, but I can use images of the sun or moon with pretty good results.


Edited by jwestervelt, 02 April 2020 - 08:04 PM.

  • marktownley and ghutton like this

#3 AJamesB

AJamesB

    Vostok 1

  • *****
  • topic starter
  • Posts: 170
  • Joined: 17 Nov 2015
  • Loc: Kihei, HI

Posted 02 April 2020 - 08:14 PM

Wow, thank you for the detailed response.  The info is, well, disappointing, lol.  Everything else about the camera is great, but ya, I instantly noticed the pattern as I tried to run it through wavelet sharpening in registax.  I won't trouble ya with a sample .ser file, given your extensive experience with this I trust this is the issue.

 

Since I just got the camera, I wonder if ZWO would do a refund or exchange.  Is there another similar sensor (pixel size and overall sensor size) you could recommend, or is sony pretty much the 'go to' sensor now a days?

 

Thank you again for this info.


Edited by ammonthenephite, 02 April 2020 - 08:21 PM.


#4 jwestervelt

jwestervelt

    Viking 1

  • *****
  • Posts: 748
  • Joined: 01 Sep 2012
  • Loc: Phoenix, Arizona

Posted 02 April 2020 - 08:25 PM

I would take it up with ZWO.  I have not contacted them yet, but since they are being impacted by this I suspect that they would like to hear from me and my team.  I'm positive that you won't be the first one to raise this with ZWO.

This CAN be addressed with a driver patch, but the question is whether or not the camera manufacturers are willing to do so.  So far, I have gotten no traction from QHY to address the problem, even though it should be trivial to fix at the driver level, albeit with some CPU cost.


  • ghutton likes this

#5 AJamesB

AJamesB

    Vostok 1

  • *****
  • topic starter
  • Posts: 170
  • Joined: 17 Nov 2015
  • Loc: Kihei, HI

Posted 02 April 2020 - 08:29 PM

Ah, okay, I will contact them tonight.  Okay if I just point them to this thread?



#6 DSOs4Me

DSOs4Me

    Viking 1

  • *****
  • Posts: 554
  • Joined: 20 Jan 2017
  • Loc: South Jersey

Posted 02 April 2020 - 08:30 PM

I've been lucky. I use Registax and wavelets all the time and haven't had that problem (and hope not to).



#7 jwestervelt

jwestervelt

    Viking 1

  • *****
  • Posts: 748
  • Joined: 01 Sep 2012
  • Loc: Phoenix, Arizona

Posted 02 April 2020 - 08:32 PM

You can... There have been numerous other discussions online.  It impacts QHY cameras as well, and there are discussions on their forums that go into detail.  I've complained about the problem here on CN a few times, but given what I found out in the past week, I don't really have too much more to add publicly in the event that this goes the legal route and punitive damages are sought.  Obviously I'll want to bring as many camera manufactures on board as possible since they are getting bitten by this, but I would be floored if ZWO doesn't know about this yet.  For all I know, they've already raised this issue with Sony.

The cameras most people notice this problem with are ones built using the IMX178 and the IMX183.  It impacts a few others, at least from what I've seen of data provided to me from independent sources. 

I was supposed to call Point Grey this week to speak to their team since they manufacture cameras using this sensor.  They may or may not know about this issue, and if they do, they may have even addressed it already via driver or firmware.  I will make an effort to do this tomorrow and follow up in this thread.


Edited by jwestervelt, 02 April 2020 - 08:39 PM.

  • AJamesB likes this

#8 AJamesB

AJamesB

    Vostok 1

  • *****
  • topic starter
  • Posts: 170
  • Joined: 17 Nov 2015
  • Loc: Kihei, HI

Posted 02 April 2020 - 09:06 PM

Awesome, thank you so much.  The amount of knowledge and support on these forums never ceases to amaze me, thank you.



#9 AJamesB

AJamesB

    Vostok 1

  • *****
  • topic starter
  • Posts: 170
  • Joined: 17 Nov 2015
  • Loc: Kihei, HI

Posted 19 February 2021 - 04:37 PM

You can... There have been numerous other discussions online.  It impacts QHY cameras as well, and there are discussions on their forums that go into detail.  I've complained about the problem here on CN a few times, but given what I found out in the past week, I don't really have too much more to add publicly in the event that this goes the legal route and punitive damages are sought.  Obviously I'll want to bring as many camera manufactures on board as possible since they are getting bitten by this, but I would be floored if ZWO doesn't know about this yet.  For all I know, they've already raised this issue with Sony.

The cameras most people notice this problem with are ones built using the IMX178 and the IMX183.  It impacts a few others, at least from what I've seen of data provided to me from independent sources. 

I was supposed to call Point Grey this week to speak to their team since they manufacture cameras using this sensor.  They may or may not know about this issue, and if they do, they may have even addressed it already via driver or firmware.  I will make an effort to do this tomorrow and follow up in this thread.

Hey, was getting back in the market slowly for a camera replacement, and thought I'd reach out to ya again and see if you'd learned anything more about the issue, or if you had any camera recommendations that were similar in sensor size or pixel size, have good raw bit depth, but that don't suffer from this issue?  Thanks!


Edited by ammonthenephite, 19 February 2021 - 04:39 PM.



CNers have asked about a donation box for Cloudy Nights over the years, so here you go. Donation is not required by any means, so please enjoy your stay.


Recent Topics





Also tagged with one or more of these keywords: solar, astrophotography, imaging



Cloudy Nights LLC
Cloudy Nights Sponsor: Astronomics