Archive for August, 2012

How to solve the Image Distortion Problem

Posted on: No Comments

      Theoretically, it is possible to define lens which will not introduce distortions. In practice, however, no lens is perfect. This is mainly due to manufacturing factors; it is much easier to make a “spherical” lens than to make a more mathematically ideal “parabolic” lens. It is also difficult to mechanically align the lens and imager exactly. Here we describe the two main lens distortions and how to model them. Radial distortions arise as a result of the shape of lens, whereas tangential distortions arise from the assembly process of the camera as a whole.

      We start with the radial distortion. The lenses of real cameras often noticeably distort the location of pixels near the edges of the imager. This bulging phenomenon is the source of the “barrel” or “fish-eye” effect. Figure1 gives some intuition as to why radial distortion occurs. With some lenses, rays farther from the center of the lens are bent more than those closer in. A typical inexpensive lens is, in effect, stronger than it ought to be as you get farther from the center. Barrel distortion is particularly noticeable in cheap web cameras but less apparent in high-end cameras, where a lot of effort is put into fancy lens systems that minimize radial distortion.

Figure 1.

        For radial distortions, the distortion is 0 at the (optical) center of the imager and increases as we move toward the periphery. In practice, this distortion is small and can be characterized by the first few terms of a Taylor series expansion around r = 0. For cheaper web cameras, we generally use the first two such terms; the first of which is conventionally called k1 and the second k2. For highly distorted cameras such as fish-eye lenses we can use a third radial distortion term k3.

In general, the radial location of a point on the image will be rescaled according to the following equations:

Here, (x, y) is the original location (on the imager) of the distorted point and (xcorrected, ycorrected) is the new location as a result of the correction.

The second largest common distortion is known as tangential distortion. This distortion is due to manufacturing defects resulting from the lens not being exactly parallel to the imaging plane.

Tangential distortion is minimally characterized by two additional parameters, p1 and p2, such that:

Thus, in total five (or six in some cases) distortion coefficients are going to be required.

Distortion example

      To ilustrate these theoretical points, I am going to show a couple of images taken by the same camera. The first of these images will show the picture with the distortion effect, whereas the second one will show the result of applying “undistortion” functions.

Figure 2. The image on the left shows a distorted image, while the right image shows an image in which the distortion has been corrected through the methods explained.


Stabilization.Speed without accuracy…it’s worth nothing.

Posted on: No Comments

      In previous entries we were talking about speeding up the processes on mobiles devices through assembler optimization and its importance when we are integrating the code developed on PC into mobile devices. As we saw this optimization is crucial so that algorithm can run with a real-time rate on such devices. But the speed alone is nothing when we want an accurate result. We also need to stabilize the image and the 3D Object or any model that we wish to overlay on the target.

       We need to take into consideration that, when we are pointing to the image and we are static, the 3D Model overlaid should not move at all, but when a sudden movement is done, the reaction of the overlaid model should be immediately carried out. If the overlaid model is moving although the image is static the user won´t feel the Augmented Reality experience as he should. In the other hand, if the user moves the camera but the overlaid model doesn´t follow the target in real-time he will appreciate a delay in the target tracking.


Promising Augmented Reality’s Predictions for next years

Posted on: No Comments

      Last months have come plenty of news about the Augmented Reality world and its growing spread around all the sectors in which it may be applied, but it seems that these predictions are more than that, they are a fact.

      Some reports conducted by marketing and consulting research firms shows the impact of this growing technology on daily life and the effect it produces on the markets which make use of Augmented Reality. These reports suggest that, despite the fact that the age of this technology may be date back to when 3D technology was born, it is still in its early stages and consequently, its economic potential and impact on consumers has not been fully investigated. One of these reports talks about the rise of augmented reality technology in the marketing industry and how it has come to change the way consumers communicate with each other and with companies.

      The interest shown in this technology and its capabilities in generating interactive digital content is growing quickly amongst both companies and consumers. At the same time, augmented reality becomes more important and promising than other technologies, like the QR codes or NFC. This is because Augmented reality is less invasive with the environment than the laters and it can be use for more than marketing. Another fact that help to this fast growing is the continuos mobile enhancement that big brands carry out. As high-end mobile devices become more common amongst users and more powerful devices, augmented reality is expected to play a bigger role amongst consumers, allowing them to be part of the augmented world and interact with it, expanding into more practical territory.


How AR will save new powerfull devices

Posted on: 1 Comment

       Great and powerful mobile devices are willing for Augmented Reality and not the opposite

      The lastest smartphones generations are targeting to become much more powerful than most of the personal computers normal users have at home. This is a great opportunity because all the tasks which a user had solely been able to carry out in a personal computer, nowadays, they can be done in those such powerful devices. Furthermore, the hardware integrated within these mobile devices allows users to have games or applications with amazing graphics designs. But the question that may come to the people´s mind is, are such powerful devices needed? or are the mobile brands offering more power than current users demand?

      Those questions drive us to the main claim in this post. Augmented reality is not willing for fast and powerful mobile devices. It is the new powerful mobile devices that are willing for augmented reality.

      Although augmented reality has been honored with the description of a cool and fancy feature, it has as well won the bad reputation of sensitive performance with a lot of demands and constraints. Those tough requirements mainly come from the computer vision sides involved in augmented reality. Heavy image processing tasks, plus manipulation of the camera itself and the 3D world rendering have gathered all to make augmented reality “impossible”.

Figure 1. This figure shows the evolutions of the mobiles in recent years.


iPhone video output setting YUV or BGR CVImageBufferRef to IplImage [Code snippet]

Posted on: No Comments
  • We are going to publish a series of posts with some useful code snippets for iOS and Android.

Any suggestion or contributions are welcomed ;)

Here we show how to get and IplImage from CVImageBufferRef or CMSampleBufferGetImageBuffer with the output video set to:

kCVPixelFormatType_32BGRA or

kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange or


Here we go:

- (IplImage *)createIplImageFromBuffer:(CVImageBufferRef)imageBuffer 
                                        withChannels:(int)channels {

    IplImage *iplimage = 0;
    if (imageBuffer) {
        // From CMSampleBufferGetImageBuffer:
        // CVImageBufferRef imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer);
        // lock buffer
        CVPixelBufferLockBaseAddress(imageBuffer, 0);
        // get buffer's image information
        uint8_t *bufferBaseAddress = (uint8_t *)CVPixelBufferGetBaseAddress(imageBuffer);
        size_t bufferWidth = CVPixelBufferGetWidth(imageBuffer);
        size_t bufferHeight = CVPixelBufferGetHeight(imageBuffer);
        size_t bufferBytesPerRow = CVPixelBufferGetBytesPerRow(imageBuffer);
        // create IplImage
        if (bufferBaseAddress) {
            iplimage = cvCreateImage(cvSize(bufferWidth, bufferHeight), IPL_DEPTH_8U, channels);
        // from YUV  - 1 channel (grey image)
        // from BGRA - 4 channels (colourful image)
        if(channels == 4)
            vImage_Buffer src;
            src.width = bufferWidth;
            src.height = bufferHeight;
            src.rowBytes = bufferBytesPerRow;
   = bufferBaseAddress;
            vImage_Buffer dest;
            dest.height = bufferHeight;
            dest.width = bufferWidth;
            dest.rowBytes = bufferBytesPerRow;
   = iplimage->imageData;
            // swap the pixel channels from BGRA to RGBA.
            const uint8_t map[4] = { 2, 1, 0, 3 };
            vImagePermuteChannels_ARGB8888(&src, &dest, map, kvImageNoFlags);
            iplimage->imageData = (char*)bufferBaseAddress;
        // unlock buffer
        CVPixelBufferUnlockBaseAddress(imageBuffer, 0);
        NSLog(@"No sampleBuffer!!");
    return iplimage;

Making 2D components in the 3D World using the ARLab 3D Render

Posted on: No Comments

      One of our key products is the 3D Rendering SDK. With that library we can make a lot of different stuff. The goal of this entry is just talking about making 2D components in the 3D world, such as a loading or a simple animation that will be layered on the screen. Imagine that we have a plane in which we can set up a texture, if we set up the texture several times, we will have the appearance of having a loading or an animation. And if we are able to set up these textures several times, up to 24 frames per second, we will have even a video textured on the screen.


LinkAR, the AR community for developers.

Posted on:

      Today ARLab has just released linkAR, the new community thought by and for developers, where you will be able to share, to know, and to ask to other developers which also belong to the community. The main goal of this community is that different developers contact each other to create a greater interaction amongst them , which will sometimes benefit both the users and the companies. In linkAR you will be able to buy licenses of our SDKs, publish your own applications and promote them. The idea is to create an AR ecosystem where  you can learn from other developers, share your ideas and win and make business with the apps and/or projects you may develop. Of course we will contribute not only supporting each single part of your development, even if is not AR related, but also we will release full open source projects like: BoxOffice developed with the ImageMatching SDK and ARPicBrowser developer with the AR Browser SDK.


Image Locals Features Descriptors in Augmented Reality

Posted on: 1 Comment

          As computer vision experts, we have to handle almost daily with the information “hidden” on images in order to generate information “visible” and useful for our algorithms. In this entry I want to talk about the Image Matching process. The image matching is the technique used in Computer Vision to find enough patches or strong features in a couple – or more- images in order to be able to state that one of these images is contained on the other one, or that both images are the same image. For this purpose several approaches have been proposed in the literature, but we are going to focus on local features approaches.

     Local Feature representation of images are widely used for matching and recognition in the field of computer vision and, lately also used in  Augmented Reality applications by adding any augmented information on the real world. Robust feature descriptors such as SIFT, SURF, FAST, Harris-Affine or GLOH (to name some examples) have become a core component in those kind of applications. The main idea about it is first detect features and then compute a set of descriptors for these features. One important thing to keep in mind is that all these methods will be ported to mobile devices later, and they could drive to very heavy processes, not reaching real-time rates. Thus, several techniques are lately developed so that the features detection and descriptors extraction methods selected can be implemented in mobiles devices with a real-time performance. But this is another step I do not want to focus on in this entry.

Local Features

           But what is a local feature? A local feature is an image pattern which differs from its immediate neighborhood. This difference can be associated with a change of an image property, which the commonly considered are texture, intensity and color.