I rank about 15 different aspects of each binocular. Several are mechanical and several are optical. All of the mechanical properties I rated on a simple 1-3 score basis.
For some of the optical properties, score is based on a formula that takes into consideration such things as importance to the overall view, variance from spec, variance above or below the mean. For instance, there is a great deal of discussion about coatings. True, coatings are important, but I rated coatings subjectively on a simple 1-3 basis. However, I felt that the tests for transmission of light were better reflected in the values I obtained for internal vignette and illumination, both of which have a definitive measured value and a calculated score. You can have what look like the best coatings in the world, but if the binocular system does a poor job at delivering all that light to the exit pupil, scoring coatings high doesn't do you much good. Also, by reducing the weight given to the subjectiveness of looking at coatings, I include a more measurable objective score.
There are several aspects of binoculars measured and included in this report that I have not seen measured anywhere else. Some appear to me to have a significant impact on the performance of the binocular system. It is my opinion that these are very important to the overall measure of performance and therefore they are included here. For instance, while often we see measures of exit pupil, sometimes actually measured and other times simply reported based on Aper/mag, few binocular studies go to any length to verify magnification and then verify effective aperture. This study does so. Also, many studies indicate that field sharpness has been checked and it is sometimes reported as the outer 1/3 of fov or from about 60% out, but few if any others actually measure distortion in the fov. This study does. None that I know of actually take measures of internal vignette for an off-axis beam, showing tilt in the optical axis and percentage of rays actually reaching the exit pupil. And even fewer still use limiting magnitude to attempt to confirm the effects of vignette and illumination.
Binocular aperture was measured and verified by three different methods. Scores were punished for those binoculars that did not measure up to specified aperture by a shortfall of more than 1-2mm. A binocular may score very well in many categories, but if it is supposed to be 50mm aperture and the aperture measures only 44mm, it looses a lot of points in the aperture category and gets knocked down in overall rank.
Often we read claims of field sharpness that are grossly exagerated. There are actually only very few binoculars that are "sharp to the edge", and nearly all of them are in the class of "benchmark" binoculars. In this report, field sharpness, the collective presentation of all aberration across the outer fov, is actually definitively measured to position and degree. Sharpness across the field of view scores benefit when both Afov is wide AND field sharpness is very good, two qualities that seldom appear in a binocular together. Quite often we see a very sharp fov in a binocular that has a near orthoscopic view. That's much easier to do than it is to get that sharp fov in a wide-angle view. So in this report we may have two binoculars that both have equal field sharpness at 75-80% out, but one binocualr accomplished it with a 64Â° Afov and the other with only a 52Â° Afov. The wider Afov would score much higher.
Some categories take into consideration several different measures before assesment for scoring that category. For instance, Focus. To score focus, I considered easy of reaching and moving the focus dial, stability (did it ever move after I set focus), close focus distance, does either side not reach perfect focus? I had measured a great deal of data on slow focusing and fast focusing, but considering that slow is more desirable for astronomy and fast is more desirable for terrestrial, did not include it in the score. Another is eyecups. Are the eyecups comfortable, do they shield the eye well, do they provide the proper distance so there are no blackouts, do the permit seeing the entire fov, were there several settings, did they move once set? Eye relief is yet another. Not only is it important to have sufficient eye relief, but also it is very important for eyeglass wearers that the eye lens be recessed deep enough so that eye glasses do not touch either the metal rim or the lens. A few binoculars lost points for that.
Resolution is considered very important. I recorded three different measures for resolution; normal power resolution, 6x boosted power resolution, and normal power handheld resolution. Then not only do I look at single base arcseconds resolution, but also apparent resolution so all different powers of binocular can be compared to each other. I put more weight on normal power resolution and much less weight on boosted resolution, simply for the reason that boosted values can never be achieved in actual use. While boosted res tells me something about the perfomance of the system, in practice it is normal power that gives the view.
I tested binocular magnification. Although that value is not scored, it is very important to one method of testing aperture by measurement of exit pupil and it is of the utmost importance to measuring resolution. Comparative resolution vlaues are all factored by the magnification at which the reading is obtained and without a true measure of magnification, that comparative measure cannot be done.
Interesting to note, for the testing of magnification, even the tester needed to be tested. A small magnifier used to test both magnification and boosted resolution could not be taken at face value for the little monocular stated magnification. Indeed, it was actually higher than stated. After determining the actual magnification of the monocular tester, it was taped permanently in that position and has been at that position ever since.
The scoring for the most part does not discriminate against smaller sizes in preference of larger sizes. However, it is important and not ignored completely, and consideration is taken to give some credit where larger size can result in better performance. For instance, a larger size would be expected to produce a finer resolution or a deeper limiting magnitude. If all scoring were based on ultimate performance, all the largest binoculars would be at the top of the scores and all the smallest would be at the bottom. In categories where this would have occured, the score is given some weight for overall performance and greater weight for performance as related to the binocular size group. This can be seen to perhaps effectively have leveled the sizes in that there are a number of 8x40/42s near the top of the scores.
For terrestrial use, I rated pincushion and depth of field. Strong pinchusion gets a low rating. Good depth of field is almost always associated with significant field curvature. For astronomy use, viewers might be inclined to choose a binocular with a flat field, which would almost always be associated with a lower depth of field score. However, more important for astronomy is not just the influence of curvature, but the collective distortion of all aberrations across the fov, and this is scored in field sharpness.
Most of the optical scores are measured and reported objectively. There is little room for the influence of subjective input. Either the binocular can see the test object or it cannot. However, there is some room for subjective influence in some of the scoring, such as fit and feel type properties. For this reason, scores should not be taken as a definitive value specifically placing one model directly above or below the other. Scores should be considered in the view of the range in which a particular model ended up. It may certainly be reasonable that a particular model could move 1 or 2 points up or down. It would be highly unlikely that any binocular scored in this list would move 4-5 points up or down.
Finally I'll add, I refrained for a long time from posting these scores, for several reasons. They include; adding missing pieces of data, refinement of measurment methods, refinement of scoring method, retesting of models that seemed to be askew in the data (in very few instances did they change), disputes about measuring methods (some of those got worked out based on discussions during the 6 month test period) and critism from the masses for not seeing favorites where they might hope they would end up. Well, this list represents the most fair process and unbiased reporting that I could come up with. Results are what they are. Nothing in this list is purposely skewed. I hope you all find this information useful.
A few things I discovered throughout the course of this study:
8x40, 8x42 and 10x42 can perform very well for astronomy. This size provides a very good cross-over choice for terrestrial use.
robustness is desirable, but is not always an indicator of good performance.
some roof prism binoculars in both 8x42 and 10x42 sizes perform very well for astronomy.
roof prism models can be better, porro prism models can be better. IMO, it is not the design, it's the quality.
specifications can often be misleading.
most quality is not apparent on the outside.
there is no one best size.
there are a lot of surprises along the way to discovery.
Build = build quality
Er = eye relief with and without glasses, eye lens depth,
Cups = eye cups design and function, cups position to eye relief
Focus = focus reach, function, ease, and stability, close focus
Diop = diopter adjustment, ease of use, single and individual focus
Coat = coatings quality (reflectivity) for lenses and prisms
Pincu = pincushion, higher score for less pincushion
Dof = depth of field, lower dof often associated with flat field
Res = normal resolution and boosted resolution, USAF charts
Reshh = normal resolution hand held
Fov = true field of view and Apparent fov, +/- from specified
Sharp = Outer field sharpness, % fov usable
Vign = internal vignette, axial tilt, but not aperture stops
Illum = Illumination of the Exit Pupil, light meter test
LM = Limiting magnitude of faintest stars
Aper = reduction in aperture due to mis-sized baffles, prism stops and prism cutoff
Mech = Lower importance score of mechanical functions, build quality
Optic = Higher Importance scores of optical aspects and performance
Total = Total Score
% = percentage of maximum possible score
Cost = cheapest available cost I found to purchase these items.
Links to Small Binocular Tests and Pictures