How long was the exposure? It looks like a combination of soft focus and poor tracking.
Regarding the Bahtinov mask, I'm going to go outside of conventional wisdom: I don't care for them. I have used them, and I agree that they can help you to get close. But (except for the case of software analysis of the diffraction pattern) they are far too subjective for me. It's not like they can give you a yes/no answer as to whether you are in focus or not. As you get closer to focus, it becomes harder to see mis-focus in that pattern. Without careful use, it is far too easy to believe that you are in focus, when you actually just close, but not in the critical focus zone.
Here's an experiment to try: Use the Bahtinov mask and get the best focus you can. Then, turn the fine focus knob an 8th of a turn. Does the Bahtinov mask show you being out of focus? Because at this point, you are probably out of focus (on most systems).
If you are imaging, the best way to get the best focus is by picking a star at the center of the field and doing a repeated subframe capture of it with exposures that are short enough to avoid saturating it. Then get your imaging software to show you the FWHM of that star and let it take 4 or 5 exposures. The FWHM will probably be changing a bit, and that's ok. You are looking for the trend. It also helps to zoom the subframe to at least 400% so that you can get a good look at the star and see its profile. Then, turn the focus knob and let it take 4 or 5 more exposures. You are looking at three things: Did the zoomed image of the star get smaller or larger? Did any faint stars appear that weren't there before (or did any faint stars that were there before disappear)? And did the FWHM number trend get higher or lower?
You are looking to see the subject star get smaller and brighter (it may get bright enough that you need to lower the exposure time). You are looking to see increasing numbers of faint stars in the field. And you are looking to see the FWHM number decrease.
As you get closer to focus, you will be making smaller and smaller adjustments to focus on each iteration (kind of like collimation). I find that about about a minute of this, I can get it to best focus. At that point, I am making really tiny turns of the fine focus knob, like 1/32nd or 1/64th of a turn.
Also, be aware that you'll almost certainly pass the point of best focus (how else could you know that you find the best focus?) When this happens, if your focus mechanism has backlash - and most do, especially SCTs - then you'll need to make note of the smallest FWHM trend that you did, back up and approach it again. Make sure that you back up far enough to actually clear the backlash. This will probably be a much bigger turn than you were making as you approached focus.
And finally, it really helps to approach best focus by turning the knob in the direction that works against gravity. In the case of an SCT, this would be a counter-clockwise turn of the focus knob, which pushed the primary mirror forward in the tube. For a focuser with a drawtube, just look at the focuser when you adjust it so see which direction lifts the camera up closer to the OTA.