 |
| We fix a lot of lenses, but not all lenses can be fixed. |
With the next two posts, I hope to end the seventh most common forum war; the ‘lens variation is a big problem!’ vs ‘I don’t believe it exists!’ argument. Like a lot of forum wars, it comes down to semantics: Variation and bad copies aren’t the same thing (actually they’re not really related at all), but people tend to use the terms interchangeably.
Even $ 2,000 lenses must have variation
Note that I said ‘must’. I didn’t say ‘might’ or ‘could’. I certainly didn’t say ‘shouldn’t at this price’. If you expect every copy of a lens to be perfect, then a dose of reality is in order – unreasonable expectations are a down payment on disappointment.
The key point is what amount of variation is acceptable.
Of course, I define ‘unacceptable’ by my standards. My standards are probably similar to 90% of your standards (and they’re higher than most manufacturer’s standards). A few of you will consider my standards either too low or too high. That’s reasonable. You and I might be looking at the same lens, but we’re doing different things with it, and probably doing them on different cameras. Later on, we’ll talk about the difference between ‘acceptable variation’ and a genuinely bad copy that I would consider unacceptable.
Why lenses must vary
Any manufactured part, from a washer on your kitchen faucet to a component in the Hubble telescope has some variation. Generally (up to a certain point – limited by the state of the technology) you can lower the variation of a part if you are willing to pay more. Why? Because entirely new machines or manufacturing processes may be required, and all of that costs money.
But just ordering more units means you can save money, right? Well yes – in very general terms, ordering larger quantities lowers per-unit costs, but in a fairly linear fashion. Doubling your order of something usually reduces the per-unit cost by some percentage, but certainly not by half. There is never a point where if you order a large enough quantity of an item you get it for free.
 |
This is a 15 cm diameter, 1/10 wavelength optical flat, eyeglasses for scale. |
As an example, we use optical flats to calibrate our test benches. The flats come in different accuracies: 1/4 , 1/10, or 1/20 wavelength of flatness. All of those are very flat indeed, and those accuracies cost $ 800, $ 2,200, and $ 3,800 respectively. There is no quantity I could buy that would let me get the 1/20 wavelength plates for the 1/4 wavelength price. And I can’t get 1/40 wavelength of flatness at any price. The technology simply isn’t available.
What varies in a lens? Everything. The screws, helicoids, plates, and spacers vary. Every glass melt is very slightly different, giving elements a very slightly different refractive index. Lens grinding introduces variation, as does the coating process. Even the shims that we use to adjust variance, they vary. And shims don’t come in infinite thicknesses, so if your thinnest shim is 0.01mm then +/- 0.01mm is your maximum attainable accuracy.
What can manufacturers do about this?
The first thing is tolerancing the design. Optical programs let the designers punch in various tolerances for parts, showing how a given variation will affect the overall performance of the lens. For the sake of argument, let’s say that one particular glass element is very critical and even a slight variation makes a big difference in how the lens resolves, while variation among other elements matters less. The manufacturer can pay to have that critical element made more accurately. They can also change the design to make the part less critical, but often only by sacrificing performance.
In addition, manufacturers can (notice I said ‘can’, not ‘always do’) place compensating elements in the lens, allowing for slight adjustments in tilt, spacing, and centering. Emphasis is on ‘compensating’, though: These adjustments compensate for the inevitable errors that accumulate in any manufactured device. They are not called ‘adjusted for absolute perfection’ elements.
 |
| The two most common types of lens adjustments: shims and eccentric collars. |
Not all lenses are equally adjustable. Some modern lenses may have five to eight different adjustable elements. Many have two or three. A fair number have none at all; what you get is what you get. Here’s a thought experiment for you: imagine you’re an optical engineer and you’ve been tasked with making an inexpensive lens. Knowing that adjustable elements are an expensive thing to put in a lens, what would you do?
I want to emphasize that optical adjustments in a modern lens are not there so that the lens can be tweaked to perfection; the adjustments are compensatory. There are trade-offs. Imagine you’re a technician working on a lens. You can correct the tilt on this element, but maybe that messes up the spacing here. Correcting the spacing issue changes centering there. Correcting the centering messes up tilt again. Eventually, in this hypothetical case, after a lot of back-and-forth you would arrive at a combination of trade-offs; you made the tilt a lot better, but not perfect. That’s the best compromise you can get.
Because many people think of distributions as the classic ‘bell curve’ or ‘normal distribution’ let’s get that particular wrongness out of the way. If you evaluate a group of lenses for resolution and graph the results it does NOT come out to be a normal distribution with a nice bell curve.
 |
Frequency graph of two lenses. For those of you tired of reading already, this graph sums up the rest of the article. The black lens is going to have more variation than the green one. Neither the black nor green graphs are at zero over there on the softest end, bad copies happen to either one, but not frequently. |
As common sense tells you it should be, lenses have a very skewed distribution. No lens is manufactured better than the perfection of theoretical design. Most come out fairly close to this theoretic perfection, and some a little less close. Some lenses are fairly tightly grouped around the sharpest area like the green curve in the graph above, others more spread out, like the black one. The big takeaway from that is you can’t say things like ‘95% of copies will be within 2 standard deviations of the mean.’
The Math of Variation
Don’t freak out, it’s not hard math and there’s no test. Plus, it has real world implications; it will explain why there’s a difference between ‘expected variation – up to spec’ and ‘unacceptable copy – out of spec’.
There are several ways to look at the math but the Root Sum Square method is the one I find easiest to understand: you square all the errors of whatever type you’re considering, add all the squares together, then take the square root of the total.
The total gives you an idea of how far off from the perfect, theoretical design a given lens is. Let’s use a simple example, a hypothetical lens with ten elements and we’ll just look at the spacing of each element in nm. (If you want to skip the math, the summary is in bold words a couple of paragraphs down.)
If we say each element has a 2 micron variation, then the formula is ?10 X 22 = 6.32. If I make a sloppier lens, say each element varies by 3 microns, then ?10 X 32 = 9.48. Nothing dramatic here, looser control of variation makes higher root sum square.
The important thing happens if everything isn’t smooth and even. Instead of 10 elements worse by 1 micron, let’s make 1 element worse by 10 microns. I’ll do the math in two steps:
? (9 X 22) + (1 X 102) = ? (36 + 100) = ?136 = 11.66
The summary is this: If you vary one element a lot you get a huge increase in root sum square. If you spread that same total variation over several elements, you get only a moderate increase in root sum square. That is basically the difference between a bad copy and higher variation.
If you have just one really bad element the performance of the lens goes all to hell
The math reflects what we see in the real world. If you let all the elements in a lens vary a little bit, some copies are a little softer than others. Pixel peepers might tell, but most people won’t care. But if you have one really bad element (it can be more than one, but one is enough) the performance of the lens goes all to hell and you’re looking at a bad copy that nobody wants.
More real world: if one element is way out of wack, we can usually find it and fix it. If ten elements are a little bit out, not so much. In fact, trying to make it better usually makes it worse. (I know this from a lot of painful experience.)
What does this look like in the lab?
If you want to look at what I do when I set standards, here are the MTF graphs of multiple copies of two different 35mm F1.4 lenses. The dotted lines show the mean of all the samples; these are the numbers I give you when I publish the MTF of a lens. The colored area shows the range of acceptability. If the actual MTF of a lens falls within that range, it meets my standards.
 |
Mean (lines) and range (area) for two 35mm lenses. The mean is pretty similar, but the lens on the right has more variation. |
For those of you who noted the number of samples, 15 samples means 60 test runs, since each lens is tested at four rotations. The calculations for variation range include things about how much a lens varies itself (how different is the right upper quadrant from the left lower, etc.) as well as how much lenses vary between themselves and some other stuff that’s beyond the scope of this article.
So, in my lab, once we get these numbers we test all lenses over and over. If it falls in the expected range, it meets our standards. The range is variation; it’s what is basically inevitable for multiple copies of that lens. You can tell me I should only keep the ones that are above average if you want. Think about that for a bit, before you say it in the comments, though.
The math suggests a bad copy, one with something really out of whack, doesn’t fall in the range. That’s correct and usually it’s not even close. When a lens doesn’t make it, it REALLY doesn’t make it.
 |
A copy that obviously doesn’t meet standards. The vast majority of the time, one of these can be adjusted to return to expected range. |
We took that copy above, optically adjusted it, and afterwards it was right back in the expected range. So an out-of-spec copy can be fixed and brought back into range; we do that several times every day.
But we can’t optically adjust a lens that’s in the lower 1/3 of the range and put it into the upper 1/3, at least not often. Trust me, we’ve tried. That makes sense; if one thing is way out of line we can put it back. If a dozen things are a tiny bit out of line, well, not so much.
I know what you’re thinking
You’re thinking, ‘Roger, you’re obviously geeking out on this stuff, but does it make one damned bit of difference to me, a real photographer who gives zero shirts about your lab stuff? I want to see something real world’. OK, fine. here you go.
A Nikon 70-200mm F2.8 VR II lens is a really good lens with very low (for a zoom) variation. But if you drop it just right, the 9th element can actually pop out of its molded plastic holder a tiny bit without causing any obvious external damage. It doesn’t happen very often, but when it does, it always pops out about 0.5mm, which, in optical terms, is a huge amount. This is the ‘one bad element’ scenario outlined in our mathematical experiment earlier.
Below are images of the element popped out (left) and popped back in (right) and below each image is the picture taken by the lens in that condition. Any questions?
 |
| On top you see the 9th element ‘popped out’ (left) and replaced (right). Below each is the picture of a test chart made with the lens in that condition. |
So, what did we learn today?
We learned that variation among lenses is not the same thing as ‘good’ and ‘bad’ copies. Some of you who’ve read my stuff for a long time might remember I used to put out a Variation Number on those graphs, but I stopped doing that years ago, because people kept assuming that the higher the variation, the higher their chances were of getting a bad copy, which isn’t true. You see, bad copies are – well, bad. Variation just causes slight differences.
I’m going to do a part II that will go into detail with examples about how much you should expect lenses to vary, what the difference is between variation and a genuinely bad copy, and why some people act like jerks on forums. Well, maybe just the first two.
As a bonus, I will tell you the horrifying story of how manufacturers optically adjust a lens that’s really not optically adjustable. And for a double bonus I will show how variation means that there are actually two versions of the classic Zeiss 21mm F2.8 Distagon.
In other words, if you struggled through this article, hopefully the next one will be enough fun that you think it’s worth it. Delayed gratification and all that…
Roger
Roger Cicala is the founder of Lensrentals.com. He started by writing about the history of photography a decade ago, but now mostly writes about the testing, construction and repair of lenses and cameras. He follows Josh Billings’ philosophy: “It’s better to know nothing than to know what ain’t so.”
Articles: Digital Photography Review (dpreview.com)