I think you can't go wrong buying a special window for IR, but keep in mind the window has two characteristics: the inherent extinction coefficient of the material, and the thickness of the window. Plastic wrap may have higher extinction coefficient, but at the same time you know it is very thin. And if you mainly just want to know if it's clear or not - there is less need for an exact measurement of temperature.
So in the years I have been using these sensors, I either used them attached to the telescope and completely uncovered - or in dedicated setups, fully exposed to the environment but covered in plastic wrap.
As it happens, right now I am setting up a full spectrum measurement with multiple sensors from UV to IR - and I think plastic wrap will work for the whole range. It doesn't pass UV-C but I hope I don't have to worry about that.
And with all these things, if you have the sensors it is very easy to try different materials just by passing them over the sensor and seeing if the signal changes dramatically.