I put some masking tape on the side of the lens and the side of my NV unit, I carefully focused the 1X lens (TNV-14, PVS-14 type) while looking through it, and then I made a mark so that I knew where exact infinity focus was.
When doing afocal observing and taking photos, I made sure the lens was focused to infinity. It has a tendency to be inadventently defocused, resulting in poorer images at the edge, so you have to keep an eye on it.
I don't touch the diopter adjustment. That is an adjustment for eyes, and people can simply wear glasses when viewing.
I use the 55mm for widest-field observing, and the 41mm for when I want better field quality. The ENVIS/PVS-14 lens does a great job for what it is and how fast it is. Long focal length eyepieces are not going to be perfect in fast telescopes, but I am pretty happy with the 41mm Panoptic for taking photos through.