Hi, jupi10; yes indeed... can never ~have enough~ math! "Use it or lose it."
Depending on how much time and effort you want to dedicate to this... just revisiting any/all of your old plain math texts can work wonders. And the astronomy and physics texts, in parallel. I often read my old undergrad and grad texts again... cover to cover... amazing how fast that refresher exercise progresses. Enough iterations and can knock off one refresher a week.
I went further, and audited many courses at my old Alma Mater (U or Rochester, NY). For token fee and professors' OK, alumni are allowed to sit in, participate, do homeworks, tests, etc. Just not for (unneeded!) credit.
Then, I rematriculated and went ahead and actually earned another grad degree --- in math! Just a few semesters (three) part-time does it. [The GI Bill and my employer (aerospace) covered tuition... completely! Even allowed me to attend day classes and float my work schedule to compensate.]
On point --- I'm not aware of any books that specifically teach math through astronomy examples. Tom