As described in previous entries using markers to perform a tracking presents more disadvantages than using the object itself as a target to be tracked. Some of those disadvantages are the need to print the marker or that the tracking can fail due to occlusions. Also, these markers are invasive to the environment, using marketing expression, “do not keep the packaging clean”.
Due to these reasons, many researchers and companies have focused on developing markerless tracking systems instead of marker-based tracking systems. The former will be the subject of this entry.
Figure 1. Online Monocular techniques scheme.
Techniques developed for online monocular markerless augmented reality systems can be classified into two sub-branches: model based and Structure from Motion (SfM) based. The difference is that while in the former a previous knowledge about the real world before the tracking is performed is required, in the later this knowledge is acquired during the tracking. Inside these two sub-branches two different approaches can be taken into consideration according the nature of the tracking. The first of them, known as recursive tracking, uses the previous known pose to estimate the current one. The second option, which is called tracking by detection, allows to calculate the pose estimation without any previous knowledge or estimation, which can be better for recovering from failures.
Furthermore, the model based approaches which use a recursive tracking can be classified in three branches or categories: edge based, optical flow based and textured based. In the other hand, the approaches covered by tracking by detection techniques are: edge based techniques and texture-based techniques. Although techniques based on tracking by detection seem to be a better option, several things have to be taken into consideration before to choose any of them in order to select the option which fits our requirements, like frame rate, accuracy or even object tracked.
In the edge based approach the camera pose is estimated by matching a wireframe 3D model of an object with the real world image edge information. Then, using different techniques, the geometrical edges given in the detection can be matched against a set of sample points in order to recover and correct any displacement that may occur. This technique has a low processing time, but it is very sensible to fast camera movements and hard in a cluttered background or shadows’ presence, which may cause matching errors. Furthermore, in this approach, the use of a 3D CAD Model is required in order to perform a good tracking. As strong points we can say that it is robust to illumination changes and partial and self-occlusions problems.
Figure 2. Example of edge-based tracking technique.
Optical Flow Based
The optical flow technique relies in the temporal information extracted from the relative movement of the object projection onto the image in order performance the track, and it has also a low processing time, but its weakness are the cumulative errors. If throughout the tracking process this accumulative error is not solved using a recovery approach, it becomes wrong and then it provides a bad pose estimation. We also have to take into consideration that this approach is sensible to fast camera movements and light changes.
The third approach, texture based , takes into consideration texture information available on images for tracking. Into this category template matching approach and interest point based technique are present. One of the differences between them is that while in the former a global search is performed in order to detect a pattern, in the later local features are used in order to match one or more patches of the image. Although they belong to the texture based approach, they have different requirements and features not only in computational cost, but also in strong points. Template matching techniques have a low processing time, being highly accurate. As weak points can be said that these techniques are sensible to fast camera movements, lighting changes on the scene and occlusion problems. In the other hand, Interest Point based techniques are only sensible to fast camera movements, but they have a high processing time.
Unlike the recursive tracking techniques where there is no image or object detection, techniques known as tracking by detection, perform a continuos detection in order to carry out the tracking. This drives to a higher computational and processing time than recursive tracking techniques, but these techniques will be described in detail in future entries.
Regarding the techniques described here, all of them are valid, but the selection of any of them should be done according the environment where they will be use and the system requirements. Thus, whether the target is a polygonal object or an object with strong contours, edge based approaches are the most suitable ones. If the objects to be tracked present textures on its surfaces, optical flow techniques should be used, always taking into consideration its constraints like light changes or very large camera displacements. Furthermore, if the textured object is planar, either template matching methods or interest point based methods should be the ones selected.Tags: AR, ARLab, Augmented Reality, Image Tracking, markerless AR