114 replies to this topic

[Goofi]

If you can wait a couple of days, I might have some for you.

I'm considering a trip out to my dark site on Friday with my DSLR.

[David Ault]

You can use my data for M42.  This was done with a Canon 1100D (unmodified) and a Stellarvue SVR90T.  It's a big download at approximately 2GB.

 

Regards,

David

[mostlyemptyspace]

You can use my data for M42.  This was done with a Canon 1100D (unmodified) and a Stellarvue SVR90T.  It's a big download at approximately 2GB.

 

Regards,

David

 

Excellent, thanks! How did you take your flats? Did you just point your scope at the blue sky?

[David Ault]

I believe I had a flat panel by that time, but I'm not sure and I wasn't taking that many notes then either.  Given that the flux between them frames is almost identical I'm going to say I was using a flat panel already.

 

Regards,

David

[newman]

I have seen a Geoptik flat field generators for a couple hundred bucks, but I haven't seen anyone who has actually used them...anyone have experience with this? Is it a gimmick? 

[David Ault]

I haven't see the Geoptik ones before.  I'm using one of these currently.

 

Regards,

David

[mostlyemptyspace]

Ok so here is a stacked image of M42 I took tonight. I loaded it up in PixInsight, cropped the edges, and hit my first roadblock doing the DBE. No matter what I tried, the image looked worse after DBE than before. Yes, it removed the gradient on the edges, but it introduced a ton of noise and general ugliness. Can anyone get DBE to work well with this image and tell me what you did?

[Madratter]

I gave it a quick pass with cropping, DBE, and then some stretching to show the results.

 

The data is quite noisy. There is both a lot of correlated noise, vertical banding, and blotchiness in general.

 

[David Ault]

DBE can't really add noise.  It's likely it was present before but based on how you or the STF function stretches the data after DBE it becomes more visible.  I took your data leveled the background in both versions and added an offset to the DBE version so that the medians of both images matched (this is because DBE shifts the data when it subtracts the model).  When I stretch them the same amount I see no additional noise.

 

No DBE:

 

With DBE:

 

You can see that the gradient is still visible in the first image and if I wanted to stretch the image more I could do it more easily with the DBE version.

 

Regards,

David

[Goofi]

As others noted, there's still a lot of noise ... did you dither?

 

I found ABE did a better job than DBE ... here's my DBE since that's what you asked for.

I cropped it a little to remove stacking artifacts, and ran two passes of Canon Banding Script (1 each - horizontal and vertical) 

 

 

Attached Thumbnails

• 

[mostlyemptyspace]

DBE can't really add noise.  It's likely it was present before but based on how you or the STF function stretches the data after DBE it becomes more visible.  I took your data leveled the background in both versions and added an offset to the DBE version so that the medians of both images matched (this is because DBE shifts the data when it subtracts the model).  When I stretch them the same amount I see no additional noise.

 

No DBE:

 

 

With DBE:

 

 

You can see that the gradient is still visible in the first image and if I wanted to stretch the image more I could do it more easily with the DBE version.

 

Regards,

David

 

David, These look much better than what I could come up with. 

Can you tell me what parameters you used so I can mimic the results?

 

So you're saying then that the ugliness that I saw post DBE is more a matter of the STF than the DBE messing up the image?

Edited by mostlyemptyspace, 15 January 2015 - 03:13 PM.

[mostlyemptyspace]

Ok, so here is my attempt at processing this stacked image with PixInsight. What do you guys think? I think it turned out pretty well considering the conditions: taken from a red zone, unguided, 120x30sec exposures, taken with a noisy Canon 1000D and a low-end 75-300mm telephoto lens. Not too shabby I think.

 

The only thing I noticed is if you zoom in the stars are really globby. I think I may have messed up the deconvolution step. It could also be that the images were slightly out of focus. I'm not sure. Any tips on tightening them up?

[David Ault]

Right, the noise didn't change, but you removed an offset when doing DBE that altered how STF 'perceived' your data so it stretches it differently.  All I did was manually add the offset back in using PixelMath and the STF gives essentially the same result (it is a little different because the gradient is no longer present).

 

Here's how I processed your data.

 

The first thing I did was crop to the higher signal to noise ratio interior:

 

Then I ran DBE with a reasonable number of samples, keeping away from stars and nebulosity:

 

Then I stretched the image with the HistogramTransformation tool:

 

Lastly, I did a curves adjustment to increase the contrast a bit and darken the background (to hide some of the noise):

 

You can do a lot more to try and bring out the center of the trapezium and fainter regions.  This was just a quick process to show you how DBE wasn't adding noise.

 

Regards,

David

[mostlyemptyspace]

So I spent many many hours processing this data. And I came up with something pretty decent. I went back and took a lot more dark/flat/bias frames for a 2nd pass. I learned an enormous amount from this effort. There's one thing that is bugging me though. There are these vertical bands that you can see striped across the image. I don't know if that's noise, nebula dust, or if the bias/dark patterns weren't properly subtracted. I'm pretty sure I did everything right in the tutorials. I don't see these bands in the registered images, but I do in the stacked image. I took 200 bias frames, 70 darks, and 70 flats. I figured that should be more than enough. I attached a stretched jpeg of my calibration images to show that they look sane. Any ideas?

 

Edit: Looking at the images one on top of each other, I can see the bias pattern lining up directly with the final image.. If my bias master looks good, then why didn't the calibration step remove it from the lights?

Attached Thumbnails

• 
• 
• 
• 

Edited by mostlyemptyspace, 17 January 2015 - 03:08 PM.

[Goofi]

Trying running the Canon Banding Script in Pixinsight ...

[mostlyemptyspace]

That's a thing? Why doesn't calibrating with the bias work?

[Jon Rista]

The Canon Banding Script in PI is only for horizontal banding...it does nothing for vertical banding. A much better vertical band reducer is Carboni's Actions...he has both horizontal and vertical banding noise reduction scripts. They are amazingly good.

 

Mes (Your name is so long, I'm just going to start calling you "Mes" now, Mostlyemptyspace. ), your bias and dark...is that what they really look like? On my screen, they look as though they are 16-color images. If that is what you see on your screen, then I think you may have decimated the bit depth of your calibration files somewhere along the line, and they are probably not calibrating properly. If that is just because of jpeg compression or something, then you can ignore me.

[mostlyemptyspace]

Yeah I had to mash em up pretty good to be able to upload them. I'm sure they're 32 bit fit files, but I'll double check.

Are those banding scripts included or do I need to download them?

[mostlyemptyspace]

I tried the Canon banding thing and it worked after a fashion. Since I had horizontal and vertical banding, I had to run the script, rotate the image 90 degrees, and run it again. It still left a few bands, but it's definitely an improvement.

It looks like those Carboni tools are plugins for Photoshop, no? Is there no better way to do this in PixInsight?

I'm still confused why I'm even having to do this. Why isn't the bias calibration taking care of it?

[Jon Rista]

Carboni's Actions are for Photoshop. They come with the Astronomy Tools Actions. I've found Carboni's banding NR to be vastly superior to the Canon Banding Reduction in PI. I want to figure out exactly how he does it, see if I can replicate it for PI. 

[mostlyemptyspace]

Okay! I think this is as good as it's going to get with this data. Not too shabby for 30sec exposures from a red zone at sea level. So the first image you see lots of dark banding (slightly rotated). It's been largely (though not entirely) removed, except for one thin strip right through the center, of course. There is still some background noise, but it's more evenly distributed so it isn't an obvious banding pattern. I think I was too heavy handed with the deconvolution before, so now I think it looks more natural also. Removing the banding also darkened the background and made the subject pop quite a bit more. I'm pretty happy with it! Anything else you guys think I can do to improve this, given the limitations of the original data?

 

I definitely want to thank you all, Madratter and Jon Rista especially, for being so helpful and generous with your time. This has been a great learning experience, and I can't wait to try again with some fresh data!

Attached Thumbnails

• 
• 

[Madratter]

Looks like you are well on your way.

[mostlyemptyspace]

Looks like you are well on your way.

 

Dude, your tutorial was absolutely vital. I've had every step open in a separate tab and I've been flipping around them for the past 3 days. I've gone through the entire process at least 3 times now, after going back and collecting more calibration frames, and trying out this banding removal script. I pretty much have the workflow memorized by now

Seriously though, I went through Harry's tutorial videos first, and they only kinda sorta helped. They're more geared towards individual processes, not the workflow as a whole. The workflow is everything, and your walkthrough made it super simple. It will live in my bookmarks for all time Thanks again.

[Madratter]

[David Ault]

Mostlyemptyspace,

 

It looks like a good improvement and I'm glad you are making progress.

 

I have found some features will simply not calibrate correctly if the gradients from local lights, light pollution or the Moon are very strong in your light frame data.  I have this problem when I do broadband imaging from my house or the nearby university and schools where I do a lot of outreach.  I have tried all sorts of things, like doing a 3 to 8 point DBE on each subframe to remove the light pollution gradient before calibration to just spending a lot of time with DBE at the end of the integration and even modeling a flat with the gradient built in.  All of these things have had moderate success, but not nearly as much as I would like for the amount of work I put into it.  I don't have any of these issues when I switch to my 3nm Ha filter or when I'm doing broadband from a dark sky site.

 

Regards,

David