157 replies to this topic

[revans]

Up until a few days ago my image processing used only adaptive software to sharpen images or remove noise.  Usually these are final steps after initial processing has already happened.  Sometimes my finished product showed color noise, like someone dusted a little bit of colored baby powder onto the surface of my image.  And then there are other types of noise.... what to do about eliminating these problems?

 

Recently my attention was called to software that uses artificial intelligence (AI) to remove noise, sharpen images, and enlarge images without the expected detail loss. Many such programs are available and they come as plug-ins to other programs or as standalone programs in some cases.  Some programs tackle one of these issues (denoise, sharpen, enlarge), enhance) and some tackle them all.  They don't eliminate the need for a general image processing program like Siril or Gimp or whatever you use.

 

Users of PixInsight or Photoshop will probably like a plug-in like noiseXterminator.  Personally, I haven't learned how to use PixInsight yet, although it is in my future I expect.  I currently use Siril, Gimp, ImagesPlus, and Photoshop Elements.  Recently I added the Topaz Labs suite of standalone programs for image noise reduction, sharpening, enhancement and enlargement.  These all use artificial intelligence.

 

The results over adaptive software are striking and I think the difference is day and night.  Go with artificial intelligence over adaptive software.  So far I have not found AI software to shrink stars.  Topaz Labs software does not do that.  ImagesPlus will, and I expect PixInsight will although I have not used it, and Gimp can if you know the technique (which isn't obvious and involves selection of stars by color followed by erosion).  Also, in my brief experience with it, AI noise removal in Topaz Labs does not remove "walking noise" well so you still have to dither unless you find an AI program that will do this   Neither will these programs shrink star trails to a point or tackle this issue at all.  I wish I could find a program that would (if anyone knows of one, let me in on the secret)   These programs will not change mis-shapen stars to a point either.  In fact, sometimes too heavy an application of noise reduction might create mis-shapen stars using them. 

 

My experience with Topaz Labs products is that getting these AI image processing programs was worth the money and there is the advantage that they work with normal everyday daytime photos (which they were designed for in the first place).  I've had to go back and reprocess many images in my library using the Topaz Labs products.  There was a big improvement.  But these programs don't work well with every single image.  They have their limits.  But overall my gamble getting this software was worth it.  

 

Just saying...

 

Rick

Edited by revans, 02 December 2022 - 11:13 AM.

[happylimpet]

It looks pretty, but its fakery.

 

Its not improving your image, its inventing an image that looks like it thinks yours might have looked like without its perceived defects. Its guessing, and throwing in its own information/detail/structure etc.

 

It must be avoided, always, if you wish the image to have any credibility. Of course, not everyone does, and i suppose thats fine. Your processed AI image will be full of invented, unreal detail.

 

There's a place for it I'm sure. I'm just not sure what it is.

 

I know I would never use it, as I've put a huge amount of effort (and some cash!) into getting a system together that can image real fine detail in the universe. I'm not about to compromise that.

[Ken Sturrock]

There have been a number of discussion threads around this topic. Obviously, the AI-based software can create very pleasing images but the routines work towards the goals that they are programmed to work towards. As image processing tools have become more automated, I think that each of us will reach a point on the continuum where we wonder what the point of it all has become: Are we employing autonomous routines to process images to have a pretty picture with limited effort? Or are we processing an image more-or-less ourselves for the stimulation of learning, solving a "problem" and introducing our own artistic spin on the end result?

 

Modern image processing software is pretty amazing, and the routines have varying levels of automation. Now, I'm not totally opposed to artificial intelligence routines (that train has left the station) but I'm generally happier when I remain in control of the things that I can handle or have an opinion about.

[idclimber]

It looks pretty, but its fakery.

 

Its not improving your image, its inventing an image that looks like it thinks yours might have looked like without its perceived defects. Its guessing, and throwing in its own information/detail/structure etc.

 

It must be avoided, always, if you wish the image to have any credibility. Of course, not everyone does, and i suppose thats fine. Your processed AI image will be full of invented, unreal detail.

 

There's a place for it I'm sure. I'm just not sure what it is.

 

I know I would never use it, as I've put a huge amount of effort (and some cash!) into getting a system together that can image real fine detail in the universe. I'm not about to compromise that.

While I do not disagree with this assessment and choose not to use some of these tools. I do believe the only fair judge on this is the imager/artist. If Rick or any other imager chooses to use them, I am perfectly fine with that. 

 

Overall as long as imagers are honest, I think it is fine. 

[revans]

It looks pretty, but its fakery.

 

Its not improving your image, its inventing an image that looks like it thinks yours might have looked like without its perceived defects. Its guessing, and throwing in its own information/detail/structure etc.

 

It must be avoided, always, if you wish the image to have any credibility. Of course, not everyone does, and i suppose thats fine. Your processed AI image will be full of invented, unreal detail.

 

There's a place for it I'm sure. I'm just not sure what it is.

 

I know I would never use it, as I've put a huge amount of effort (and some cash!) into getting a system together that can image real fine detail in the universe. I'm not about to compromise that.

You know, when it comes down to it.... dirt is still dirt and blur is still blur... I'll live with the AI solution.... no dirt.... no blur (if your image was decent to begin with and not real crumby).

 

Rick

[revans]

There have been a number of discussion threads around this topic. Obviously, the AI-based software can create very pleasing images but the routines work towards the goals that they are programmed to work towards. As image processing tools have become more automated, I think that each of us will reach a point on the continuum where we wonder what the point of it all has become: Are we employing autonomous routines to process images to have a pretty picture with limited effort? Or are we processing an image more-or-less ourselves for the stimulation of learning, solving a "problem" and introducing our own artistic spin on the end result?

 

Modern image processing software is pretty amazing, and the routines have varying levels of automation. Now, I'm not totally opposed to artificial intelligence routines (that train has left the station) but I'm generally happier when I remain in control of the things that I can handle or have an opinion about.

AI or not, if you are using image processing then you really are not in control... you are in a state of dependency on the software you are using.  If its AI or if its adaptive or otherwise.... still the same to me.  And without it.... lousy images....  at least most of the time...

 

Rick

[Ken Sturrock]

AI or not, if you are using image processing then you really are not in control... you are in a state of dependency on the software you are using.  If its AI or if its adaptive or otherwise.... still the same to me.  And without it.... lousy images....  at least most of the time...

 

Rick

This is pretty much what I said - none of us are rubbing sticks together. But it's all on a spectrum. As I also said, the AI-based routines can produce nice results. Moreover, as discussed in some of past threads, there are a lot of ways to go about designing the routines as well as what "AI" even means. At the end of the day, if you are enjoying yourself in the hobby and like the results then that's what matters.

[lancing]

I knows its big in the photography world but I think the issue with astro is AI may not be able to accurately interpret what you are processing all the time and a best guess on its behalf maybe way off the actual visual of the real target.

A lot of the stuff we shoot is fairly unique in the photography world.

That said AI probably could help on the image acquisition side by analysing shots and continuously adjusting settings to compensate for target distortion and making sure we have an adequate number of frames for stacking.

Edited by lancing, 02 December 2022 - 11:56 AM.

[smiller]

This will be food for endless discussion and debate of “what is real”.  For me that Rubicon was crossed simply with narrowband color assignments that didn’t match the human visual system, one could argue we’ve crossed from the real into the artistic/abstract years ago.   I like them and do them, but I know it’s become a bit of an interpretation.   But that’s just me. 

 

With respect to the noise reduction algorithms, I don’t have too much of a problem if what they are attempting to do is get closer to the ground truth image given the data it has.   But I agree that there’s a delicate balance there because if it’s trained based on a zillion known images and, in the limit, it’s really doing replacement, that crosses a threshold for me.

 

 I don’t see noise reduction crossing the threshold yet… the defects that are in my images are still in my images, I believe the noise reduction (NoiseXterminator) looks to be just properly applying the noise reduction that I would really want to see if I was able to do it in some other manual manner….  In other words it’s the noise reduction “my mind’s eye” does when I look at the image to discern the details and see through the noise.   But again, that’s just me.

 

Not a simple subject with a simple answer.   And it won’t be getting any simpler as these methods get more sophisticated.

 

There are possibly some ways to test these algorithms.   And it’s done where one decimates an image outside the training set, uses the algorithm to reconstruct it, and you compare it against the ground truth undecimated image.   I co-sponsored research with Purdue University in the 90s for a resolution enhancement algorithm that did this.  This was before deep learning deep learning training, it was using void and cluster classification algorithms, but the general concept is roughly similar.

[idclimber]

One of the brand new tools in PI is SPCC. With it we can do unprecedented accurate color calibration of our RGB images. Cool, exactly how many of us will not push the saturation, or otherwise adjust the color to make the image more pleasing and attractive? Probably very few. 

 

Even using this tool and not altering the color is arguably a problem as it relies on a huge data set taken from other telescopes. 

 

It is so easy to make rules for others, which I interpret as.... you must do it my way, or your image is not worthy. 

Edited by idclimber, 02 December 2022 - 12:09 PM.

[jdupton]

Rick,

 

   As has been covered already, AI is working its way into astrophotography slowly but surely. Like Ken, I see it on a continuum of helping get the most out of an image versus creating a new image that might resemble your input.

 

   I am OK with AI that is trained to look at noise and then treat it in a way that preserves whatever is buried in that noise. I don't care for the overly detailed images that AI can spit out after attempting to "enhance" an image. For instance, what if you took an abstract AI package and trained it by looking a millions of spider webs and then applied it clean up an M16 or M42 image? Presto, would you then see all manner of tiny sharp filaments of star forming regions... I don't know and would not care to use such a thing.

 

   There are new options just around the corner. They are already arriving today. The technical news sites have been covering the Text-To-Image AI programs. Who knows, in a few years, we can retire our telescopes and image processing software. You'll be able to sit down in front of the computer and type out "I want an image of the Orion Nebula showing details of the embedded star forming globules.." The result will be a jaw-dropping image -- or sci-fi depiction of your "target" for the evening.

 

 

John

 

PS: As you can tell, I am too old to embrace some of what is coming.

Edited by jdupton, 02 December 2022 - 12:14 PM.

[astrokeith]

These 'AI' suites have recently been reviewed (AstronomyNow) by a very respected imager.

 

I would say they arent really 'AI', rather the very next generation of tuned tools. They arent 'thinking' per se, but using neural networks to apply tuned astroimage algorithms to remove known artefacts.

 

Some would argue they are better than the old way of image processing, as they do less harm, as they are more tuned to the AP tasks and specifics.

[CowTipton]

My images are very short exposures (1-5 seconds) and integration time (5mins or less) and therefore super noisy so I often use Topaz Denoise AI to help clean that up.  Mostly masked to the noisy background.

Is it fakery?  Sure but it's no more fake to me than using software to shrink stars (which I also do,) or using something like Starnet++ to create a starless image to be processed separately.

Hubble palette? Fake.  Those are not the actual colors captured.  Fake Hubble Palette, just as fake but no more so. 

etc.

 

In my opinion, astrophotography is the task of capturing data and then the art of processing it.

Art means freedom of expression.  Do whatever you want, it's your creation.  Some may like it and some may not.  They don't get to determine its validity.  You do.  You're the artist.

 

It's like telling a musician that he can only use certain notes over certain chords.  That's nonsense.  He can use whatever notes he wishes.  One note is not more ethical than another just because it fits into a certain scale.

If it sounds good, it IS good.

Likewise when it comes to photos: if it looks good, it IS good.

 

IMO of course.

[pedxing]

I for one, welcome our star removing overlords!

I'm less convinced about the noise removal and "enhancement"..

But having spent uncounted hours trying to make and use effective star masks, I'm glad to throw the star removal task over to the neural algorithms.

[happylimpet]

My images are very short exposures (1-5 seconds) and integration time (5mins or less) and therefore super noisy so I often use Topaz Denoise AI to help clean that up.  Mostly masked to the noisy background.

Is it fakery?  Sure but it's no more fake to me than using software to shrink stars (which I also do,) or using something like Starnet++ to create a starless image to be processed separately.

Hubble palette? Fake.  Those are not the actual colors captured.  Fake Hubble Palette, just as fake but no more so. 

etc.

 

In my opinion, astrophotography is the task of capturing data and then the art of processing it.

Art means freedom of expression.  Do whatever you want, it's your creation.  Some may like it and some may not.  They don't get to determine its validity.  You do.  You're the artist.

 

It's like telling a musician that he can only use certain notes over certain chords.  That's nonsense.  He can use whatever notes he wishes.  One note is not more ethical than another just because it fits into a certain scale.

If it sounds good, it IS good.

Likewise when it comes to photos: if it looks good, it IS good.

 

IMO of course.

I think this makes a very important point. The use of AI means that the image is now entirely an artwork created by a computer, inspired by an image taken by a person (presumably).

 

If thats how its described, I have no problem with it. When people post images of Mars with > hubble resolution, and refuse to accept that the detail isnt real, thats when i lose my cool. Of course, theres a lot in the middle!

[revans]

I had a similar sort of debate with myself when I stopped star hopping and manually centering my target in favor of an automated imaging process and using GOTO and plate solving.  I had the feeling I was losing touch with what I liked best about being out under the stars and using hard won knowledge I'd accumulated for years.  

 

I am getting over that in the interest of efficiency and saving time and getting more imaging done in the time available.  I shamelessly use GOTO and plate solving and automated imaging now.

 

I don't see much of a difference between that and using AI software to correct image defects.  We are still directing what we want done and if we don't like the result we can toss it and do something else.  I think its getting more into "art" when your result starts to look unlike what your target would look like if you could see it from a lesser distance and narrowband imaging doesn't bother me.... to me it is still science. 

 

Rick

Edited by revans, 02 December 2022 - 02:20 PM.

[oneredpanther]

Ahh yes the old debate from the purists and their artisanal hand-crafted small-batch photons gathered from a free-range family farm in rural Sagittarius, on organic beef-stock film emulsion, using the very latest techniques from the 1800's, with home-ground mirrors, on a wooden mount, hand-cranked by an old Swiss man, like his father did, as did his father before him. Unless you're doing THAT... it's all fakery. 

Edited by oneredpanther, 02 December 2022 - 03:29 PM.

[Alex Ranous]

I you're going to be a purist about it, the minute you do a nonlinear stretch, you're faking it.  I think as long as you use AI in moderation, you're fine. When I started playing with this stuff, I was using Topaz Denoise.  While it was fun to see what it could do when you cranked up the sharpening, it was clear it was inventing stuff.  The results looked like something from Hubble, but if you compared it actual Hubble data, you could see it was fake.

 

The thing is, Topaz's machine learning model was trained with terrestrial data. It's really good at sharpening thin filaments like hair and fur.  Applied to astro data, it took that fine nebulosity and did to it what it would do to hair and fur.  I'd only use Topaz with the sharpening turned down to zero.  Also, the noise reduction tended to mess up the stars, so I would only use it on data where I had removed the stars.

 

NoiseXTerminator was a game changer.  A machine learning model depends on the data it was trained with.  NoiseXTerminator (and StarXTerminator) was trained on astronomical images.  I see many more non-real artifacts created with traditional noise reduction than what I do with NoiseXTerminator.

[Spaceman 56]

 

In my opinion, astrophotography is the task of capturing data and then the art of processing it.

 

Art means freedom of expression.  Do whatever you want, it's your creation.  Some may like it and some may not.  They don't get to determine its validity.  You do.  You're the artist.

 

 

I tend to agree with this. 

 

 

Spaceman 56

[soojooko]

I think as long as you use AI in moderation, you're fine.

Like most things in life.

[revans]

Like most things in life.

Yes, I feel this way also.  I think when using Topaz products, I know very well that it was designed for terrestrial, but still I find it useful for astro images... provided you really use it lightly and don't let "hairs grow" on your image.  I agree and have known that likely NoiseXterminator is the better choice but frankly I am just not ready for PixInsight... yet... and I wanted a quick fix to my noise problems without it.  

 

Eventually I"ll learn PixInsight, but it is expensive at least to me and requires PayPal which I have never wanted anything to do with.  The latter is probably my main reason for avoiding it.  But  NoiseXExterminator is a plug-in at present for PixInsight or Photoshop.  I don't mind PhotoshopElements and I do use it, but the full version of Photoshop has got my hackles up for years now (why? well I spent many hundreds... $600 for one version...  on older versions and they are useless now because they are no longer supported even to the extent that you can't log into them anymore because they shut down the access licensing confirmation server.... and that was fatal to their products as far as I was concerned because I lost access to many plug-ins for it that I'd payed hundreds more for years ago)  and I won't get PixInsight because I don't want a PayPal account. 

 

So for now, I use Topaz, "hair and all."  But I don't let it grow Tiger Hair on my images.... I stop far far short of that.  But it is better than my adapative programs for what it does.

 

That is just where I am.  I'm an old geezer now and the young probably scoff at my logic.  But that is how I feel based on past experience and years doing this even if my images don't look like I'm very experienced...  (mostly being humble, I do have one or two decent ones)...

 

Rick

Edited by revans, 03 December 2022 - 07:46 AM.

[soojooko]

Yes, I feel this way also.  I think when using Topaz products, I know very well that it was designed for terrestrial, but still I find it useful for astro images... provided you really use it lightly and don't let "hairs grow" on your image.  I agree and have known that likely NoiseXterminator is the better choice but frankly I am just not ready for PixInsight... yet... and I wanted a quick fix to my noise problems without it.  

 

Eventually I"ll learn PixInsight, but it is expensive at least to me and requires PayPal which I have never wanted anything to do with.  The latter is probably my main reason for avoiding it.  But  NoiseXExterminator is a plug-in at present for PixInsight or Photoshop.  I don't mind PhotoshopElements and I do use it, but the full version of Photoshop has got my hackles up for years now (why? well I spent many hundreds... $600 for one version...  on older versions and they are useless now because they are no longer supported even to the extent that you can't log into them anymore because they shut down the access licensing confirmation server.... and that was fatal to their products as far as I was concerned because I lost access to many plug-ins for it that I'd payed hundreds more for years ago)  and I won't get PixInsight because I don't want a PayPal account. 

 

So for now, I use Topaz, "hair and all."  But I don't let it grow Tiger Hair on my images.... I stop far far short of that.  But it is better than my adapative programs for what it does.

 

That is just where I am.  I'm an old geezer now and the young probably scoff at my logic.  But that is how I feel based on past experience and years doing this even if my images don't look like I'm very experienced...  (mostly being humble, I do have one or two decent ones)...

 

Rick

Have you considered Affinity Photo? I use all 3 RC-Astro plugins on Affinity ( StarX, NoiseX and GradientX ). PLus many other Photoshop plugins work well with it. Its very cheap - especially right now where they have a 40% discount off the newly released V2 ( around $40 ). No subscription. All updates to v2.* are free. It's extremely capable for Astro work.

[Ken Sturrock]

and I won't get PixInsight because I don't want a PayPal account.

I totally get this but you don't need a PayPal account. Like a lot of online vendors, Pleiades just uses PayPal as a payment processor. You can pay with a card and skip joining PayPal.
 

[dciobota]

Funny thing, our own brains do context interpretation (which is what most AI are trained for, otherwise also known as pattern recognition back in the old days).  Famous pictures where your brain interprets two faces or a chalice, moving spirals in a static image, etc.  Most AI training is designed to do exactly that, mimic the way we as humans interpret data to fill in "missing pieces".  How many of us have looked at a noisy image and could determine what the nebulosity was, the edges of a blurry crater, etc.  All AI is doing is a more empirical than mathematical approach (say, edge detection algorithms vs training an AI on millions of edge examples).  I see no ethical issues with that in the art community, which the vast majority of astro images fall into. 

 

As they say,, vive la difference!  

[newbie SA]

In my opinion, astrophotography is the task of capturing data and then the art of processing it.

 

 

IMO of course.

I find this round it up quite nicely. 

To try to paint a photograph so that you can't see a difference might be a nice exercise, however, to bring out detail, without adding what was not in the photograph, is art.
There is never what "WE" see. If we where closer to our object, it would look different, if we had the vision of a dog, it would look different, if out atmothphere had a different composition it would look different. 
So what are we trying to archive?

An image, we think is pleasing!  Nothing else!! NO as close as possible to reality.

So AI is just as fallible as us. it just does everything faster. The same then us it depends on input. I watch you tube tutorials, ask at Cloudy Nights, I copy outomations from other people. AI is not that different.
In my opinion, denying AI its place in the future of Astrophotography is equal to "I know everything there is to know, and do like I did the last 10 years"

Edited by newbie SA, 03 December 2022 - 01:56 PM.