The World According to Your AF Sensor

Extract from thread initiated by Rainer Hönle (Nightshot) on

www.dslr-forum.de

Foreword

It is not easy to get access to the image data generated by the AF sensor, so I thought, if you remove the AF chip from the AF module and substitute it with your own sensor then you can capture that image at will.The experiment was performed with the AF module of a 7D, but the principles outlined here are applicable to other cameras as well, except the AF point layout will vary.

The AF moduleHow the 5D AF module is constructed

Secondary Image Registration (SIR) optics

Just in front of the AF sensor we find SIR optics, sometimes also called separator lenses. The image above is from a 5D, I forgot to take one of the 7D.

Secondary Image Registration (SIR) optics

Are the SIR optics and the AF sensor offset?

As I cannot take pictures with my modified mirror box, the image above is taken by another camera. In real life this might not be better however as the AF may be displaced by 1mm to the right or left during assembly. Only the Canon 1D series specifies an adjustment. How other manufactures handle this I do not know.

The 7D AF sensor

The dark lines are the arrays of light sensitive pixels. The remainder are structures of the read out circuitry, etc. When viewed under microscopethe AF pixels are rectangular.

The 7D AF sensor

….in more detail. By means of the overlaid scale I was able to determine the size of the AF pixel to be 14.3 x 125µm, about 40 times the area of an average image sensor pixel.

The 7D AF sensor

The lower left f/2.8 array of 7D AF sensor. The AF pixels of the diagonal f/2.8 sensitive arrays are not aligned with the array direction, but are composed by staggered horizontal and vertical pixels respectively. The spacing between the (black) pixels are “dead zones”- one reason that the central vertical f/5.6 sensitive AF points are in reality double pixels housed in two approx. ½ pitch offset, staggered arrays.

Phase Detection AF Basics

Before we head into what the AF sensor sees, let’s just briefly look at how the light rays used by the AF sensor get there. The AF unit receives only about 30% of the light, the rest is reflected towards the view finder by the main mirror.

1 2

3

4

56

8

9

7

1 Lens2 Main mirror3 Secondary mirror4 Matte screen5 Penta prism6 Viewfinder lens7 AF unit8 Shutter9 Image Sensor10 Rear LCD 10

Phase Detection AF Basics

When the image on the image sensor is in focus, the secondary image distance on the AF sensor array pair corresponds to what is called the “baseline”. In case of front focus the images move closer and in case of rear focus, they move apart.

The AF sensor view

This is our focus subject as seen through the view finder.

The AF sensor view

These are the images projected by the SIR optics onto the central part of the AF sensor. The limited size of my image sensor made me concentrate on this part. The four middle areas in “+” shape correspond to the f/5.6 sensitive sensor range. The four corner areas make up the f/2.8 sensitive part. The real AF sensor does not record these images but only the areas marked with red.

The AF sensor view

When all the individual SIR images cover the same area, as shown in the previous image, the picture is in focus. How it looks when not in focus is shown here. The individual SIR images start to move, the more they move the larger the degree of defocus. I have also overlaid the original initial position in order to show the direction and the displacement of the SIR images.

The AF sensor view

The direction of movement as indicated by the arrows tells the camera in which direction to move the focus lens group, and the image displacement determines the movement amount. As can be seen, the images on the f/2.8 sensors move double the amount of those on the f/5.6 sensors, and thus exhibit the double (de)focus sensitivity. Also a decrease in sensor image sharpness can be seen.

The AF sensor viewTo make this even more evident I have made a small video. The left half of the screen in the video is what sensor sees and the right part is the view through the viewfinder.

The AF sensor viewWhat I show here is part of the autofocus compendium that I have been planning for a long time. I admit getting a little fast into it, as I believed the basics to be sufficiently well known. I have thus prepared a nicer presentation. This is the new image showing what you see in the viewfinder.

The AF sensor view

This animation shows the AF sensor itself, the part images projected onto the AF sensor, and finally, since the AF sensor consists of rows, what part of the projections are captured by these rows.

The AF sensor view

Let‘s assume that the active AF point is on the mountaineer. The camera captures all the rows that cover the targeted subject and compares these. As the mountaineer is in focus the row images have the same content (i.e. cover the same part of the subject).

The AF sensor view

What would happen if the active focus point was changed to the central one? The camera again compares the associated rows. As the mountain in the background is out of focus, a displacement between the row images exists. From the direction of the displacement, the camera can calculate if the subject is in front or behind the current focal plane and from the amount the distance the lens has to travel to bring the subject in focus is computed. If the lens is bright enough the f/2.8 sensors can be used. For the same focus error these exhibit double the amount of row image displacement and are thus more accurate.

The AF sensor view

The misconception

And now for a widespread misconception. When the AF does not perform well in the dark, you buy a brighter (large aperture) lens, as the AF is in need or more light to focus and the AF is performed at full aperture (wide open). The animation shows what happens when the aperture is progressively stopped down.

The misconception

The lens used is a 35/1.4. During the capture, the gain (ISO) and exposure time of the sensor were kept constant. As can be readily seen, no change occurs between f/1.4 and f/2.8. The images are neither brighter nor sharper – the AF can however still measure more accurately. At f/4.0 the f/2.8 sensor images start to vignette, at f/5.6 only the associated image fields are exposed and at f/8 only the most central part of the latter. At f/11 it’s all over, no similarity between the images to be compared longer exits.

The misconception

Canon has published a couple of nice pictures showing the ray traces in the AF module. The fact that they detail the inner workings of the 1D is not important.

The misconception

I have recreated the Canon drawings here. Each of the AF fields view the target subject through the red colored surfaces marked on the front element of the lens. One field primarily looks from above onto the paper, one from a very low vantage point, another one from the left, and one from the right. So much for the theory. Now would it not be nice if this was visible on the AF sensor also…

The misconception

fortunately that is exactly the way things are!On the picture you may notice a very large depth of focus for a macro. I have subsequently tried to stop down in order to achieve the same depth of focus for the image sensor as for the AF sensor.

The misconception

The EXIF shows f/14, but since this is a macro, we need to add two additional stops. The cone of rays that illuminate the AF sensor thus correspond with f/29. That the images on the AF sensor, as seen the above video, remain relatively sharp while the right picture only shows a washed out greyish, is hence no incidence.

The misconception

Also the red markings on the front element of the lens have been intentionally kept small. For the AF measurement, only very small parts of the lens are used. The small (effective) aperture also means that dust on the AF optics will be imaged with high sharpness on the AF sensor.

The misconception

Even though my optics looked absolutely clean and were cleaned by myself, some spots are visible on the pictures shown above. If the signal from the dust is higher than the signal from the AF subject, faulty operation is likely to occur.

The misconception

Thus, next time you are out shooting in lowly lit surroundings with a highly praised prime lens, think about what performance the AF is supposed to achieve – it only receives the amount of light corresponding to what the image sensor gets at f/29. Since the gain (ISO of the AF sensor) for this purpose has to be set to its maximum with the result that the noise level on the sensor is much higher, a certain variation in the AF performance is to be expected.

AF Generations

Let me shed a little light in the dark, even though this might be rather Canon specific. On the image I have indicated through which areas of the lens front element measurements are made for the different AF modules and ranges. Earlier on, for many

years, only the clasic f/2.8 and f/5.6 line sensors existed and served the purpose, including for the 5DmkII.

AF Generations

With the introduction of the EOS1 a new AF module saw the light, a module that performed area comparison rather than line comparison, and further had a modified ray configuration. From that time onward, lens adjust points did not cover just the classic f/2.8 and f/5.6 sensors, but also

those dedicated to the area sensors. With the 40D came the f/2.8 diagonal sensors that caused further addition to the list of focus points, as these were looking out of a different part of the lens.

AF Generations

The AF systems were continously expanded and improved, but the older lenses do not offer all the parameters needed, and therefore in case of a parameter retrival error, a value that resemble the expected value most closely will be used.

Unfortunately no list exists of which lenses master which command set, and to increase the confusion, Canon sometimes change the electronics during production and suddenly a newly shipped copy can do things the older siblings could not, even though they carry the same type and model numbers.

The test rig

I replaced the dedicated AF sensor it with a small “printed circuit board camera” with a 1/2,5" 5MPix image sensor and USB connectivity. Pixelsize 2.2 x 2.2µm. The camera is B&W only, which is no problem however since the AF sensor sees B&W only as well.

You cannot see much of the camera module in the picture as it was quite challenging to make it “light tight” – but this is how the test setup looks…. Bare essentials only, is the motto….

AF sensor vignetting

The nature of the AF sensor vignetting is dependent on the focal length used. The 50% value is generally reached at the same position. A short focal length exhibit as sharp edge, a long focal length has a very soft edge. Shown are two extreme examples with a 15mm fish eye and a 300/4.0 with a 2 x converter, both set at f/8.

AF sensor vignetting

What confuses the AF is when edge shading is decentered – if one or the other array in a pair is darker than the other, this might be interpreted as the actual signal and the AF will find nothing. With the 600mm shown, the center brightness already starts to decrease, the vignetting towards the edge is however still rather soft. If the subject is illuminated well at the center and exhibits strong contrast, this will overcome the vignetting and the AF will find its target.

Accuracy considerations

I have now repeated my search to see if there really is no difference amongst lenses. In the following images the focus target is the roof, and the subject of interest is the tree in front on the right. I have copied the associated images from the AF sensor using the same field of view and animated these as GIFs. Since the tree is not in focus it appears to jump back and forth, and the displacement corresponds to the adjustment that has to be performed by the lens. In other words, the more the tree moves, the more accurate the camera is able to command the lens to the position of best focus.

Accuracy considerations

We have already proven, that the secondary image brightness on the AF sensor is independent of the brightness (i.e. max aperture) of the lens, but also in regards to the accuracy (tree displacement), I have not been able to find any significant difference.

Accuracy considerations

Our point focus subject still is the roof of the house but now the AF is asked to focus on the tree. In each of the 3 situations the lens drive has to cover the same distance. At 24mm the AF only sees a minute difference, the focus error therefore will be larger at 105mm, where the AF is able to see with a much higher resolution due to the lens magnification.

What makes a clear difference however is the focal length in use:

Accuracy considerations

As previously mentioned the difference between the f/5.6 and f/2.8 fields is that the images are displaced by about the double amount in the latter compared to the former and in this case diagonal as this is how the arrays and optics are configured. So not much new to learn here.

This fact however is less dramatic than it might seem as the depth of field of the wide angle lens is larger than that of the long lens, it however hints of the deficit of such a system in terms of perfectly focusing wide angle lenses.

Accuracy considerations

On a final note what is to be further noticed with these images is that my sensor has rendered all of the images at a resolution, i.e. pixel size of 2.2 x 2.2µm, whereas the real AF sensor uses a pixel size of 14.3 x 125µm. The image viewed by the AF sensor hence rather looks like the rightmost one.

Color mattersFinally I tested how the AF reacts to varying color temperatures of the illumination. For this purpose the lens (50/1.8) was securely attached and to avoid any shake or changes the color filters were held in front of the light source. This is how the “whole” image looks.

Color mattersIn order to best to be able to see the effect, the f/2.8 AF field in the lower right corner was investigated. In the animation the green illumination reprensents the middle reference that is in turn compared with blue and red illumination respectively. Note the change near the arrow that indicates the direction of displacement.

Color mattersDepending on the color, the image on the sensor moves a tiny bit. The AF would hence focus at another point under red or blue light than under green.And now we need a small detour into Optics.Most current lenses are chromatically corrected. In this example blue and red.

Color mattersTherefore the focal plane for green light is in front of the blue and the red. With white light the focal plane is located where the colors form the smallest coincident spot. Now you think, great, the AF works as it should. It however sees the blue light focus at another position and accordingly positions the lens slightly differently. This means the world would be a perfect place if the AF and the image sensors had the same color sensitivity – which is exactly were the problem lies. The sensitivities for Blue and green are pretty close, but the AF sensor is much more sensitive to red than the image sensor. (The proof has to wait until my spectrometer arrives.) One reason might be to accommodate the red AF assist light emitted by the flash during focusing – if the AF sensor was not especially sensitive at this wavelength, the AF assist emission would serve no purpose.

Color mattersWith warm incandescent (tungsten) light the difference in sensitivity plays an important role. The AF sensor sees the relatively larger red component very clearly and accordingly focuses at a slightly displaced position.As the red light reaching the image sensor is however much more intensively filtered, the focus shift does not appear on the sensor image, with the result being a front focused image.Newer camera models therefore attempt to compensate for this displacement by measuring the color temperature of the light and offsetting the focus position calculated by AF a bit backwards when the light color is very warm.

f/29 Diffraction limitsThe AF pixels that we have seen so far are 14.3 x 125µm. Would downscaling the AF pixels to increase the AF sensor pitch cause diffraction issues?Actually – even with diffraction present, the better you resolve the airy disk, the better you can resolve the projected SIR image displacement. The real problem is the light sensitivity. My own 2.2 x 2.2µm sensor, requires about 2 seconds for a decent exposure – it’s doubtful that anybody would want to wait that long for the lens make its first move.