Different types of augmented reality

Smartphones are probably one of the first objects where us mortals got to experience augmented reality; after all they are omnipresent. However, with time, as we look around us and observe closely, we find that augmented reality is prevalent elsewhere too.


Augmented reality does not end at your smartphone. There are many more ways to enhance your perception of the world

Smartphones are probably one of the first objects where us mortals got to experience augmented reality; after all they are omnipresent. However, with time, as we look around us and observe closely, we find that augmented reality is prevalent elsewhere too.

Let us have a look at the types of AR. Before we begin, understand that lines are blurred, broken and twisted. Definitions explained here are not written in stone.

Projection based AR

Just like anything else which is beyond our reach, projection based AR feels more attractive (at least as of now) compared to an AR app you can install on your phone. As is obvious by its name, projection based AR functions using projection onto objects. What makes it interesting is the wide array of possibilities.

Projection-based AR can build you a castle in air, or a dialer on hand

One of the simplest is projection of light on a surface. Speaking of lights, surfaces and AR, did you ever think those lines on your fingers (which divide each finger into three parts) can create 12 buttons? Have a look at the image and you would quickly grasp what we’re talking about. The picture depicts one of the simplest uses of projection based AR where light is fired onto a surface and the interaction is done by touching the projected surface with hand. The detection of where the user has touched the surface is done by differentiating between an expected (or known) projection image and the projection altered by interference of user’s hand.

One of the widespread uses of projection based AR techniques is noninteractive. Projection on objects can be used to create deception about the position, orientation and depth of an object. In such a case an object is taken into consideration and its structure is studied in depth. The object’s distance from the projection is calculated and the projection light sequence is then designed carefully to deceive the viewer’s mind.

The most exciting implementation of projection-based AR is perhaps the most exciting one of all uses. We are sure it can appeal to your inner scientist and even the child inside you. Pop over to your PC or Smartphone and watch this video:http://dgit.in/U954At. It shows Laser Plasma technology in action creating 3D images in air. That’s right - 3D images in thin air. This is one of those technologies which can not only be used for AR, but for a whole lot more. Not only can this piece of tech be used for creating virtual object in the air itself, its future developments may make it useful enough for much larger deployments. In some cases, it might eliminate the use of special gear (such as goggles and head mounted displays) for experiencing AR.

Recognition based AR

Recognition based AR focuses on recognition of objects and then provide us more information about the object. e.g. when using your mobile phone to scan a barcode or QR code, you actually use object recognition technology. Fact is, except location based AR systems, all other types do use some type of recognition system to detect the type of object over which augmentation has to be done.

Recognition based AR technology has varied uses as well. One of them is to detect the object in front of the camera and provide information about the object on screen. This is something similar to the AR apps for travellers (location browsers). However, the difference lies in the fact that the AR location browsers usually do not know about the objects that they see while recognition based AR apps do.

A second type of recognition-based AR application is to recognize the object and replace it with something else. The applications, once again are in abundance and possibilities endless. Some examples are given below:

1 Simulation of objects in 3D. In this case, printed version of a recognizable object (such as a card with QR code printed on it, or a picture provided by the app printed on paper) is shown to the camera. This printed version is called “Augmented Reality Marker” and acts as a reference for the AR app running on the system. The augmentation app detects and recognizes the marker and tries to understand the distance and orientation of the print.

Recognition based AR at work

Once the recognition is complete, it replaces the marker on screen with a 3D version of the corresponding object. This allows the user to investigate the object in more detail and from various angles. Rotating the marker would rotate the 3D imagery as well.

2 Yet another famous use of recognition AR tech is - translation of words on the fly. In this case, the app reads the words seen by the camera and tries to recognize the words using OCR (Optical Character Recognition) technology and then replaces the words on screen with their translated versions. This can be immensely useful for tourists when travelling to places where the locally prevalent language is not known.

3 Recognition based AR can also be used in education. Markers of more than two objects are kept together. The app detects the multiple markers and tries to simulate relationships among them. For example, one can use printed cards to represent atoms (in say chemistry class) and based on their mutual distance the AR app can show how a reaction would take place; and that would be just one use of AR in education.

Augemented Reality for simulating chemical formulae

Detection of drawings and sketches by more intelligent apps can help teach small children. e.g. a picture of a giraffe be replaced with a living 3D version of a giraffe and children could see how it looks in reality and they could interact with the same on a touchscreen!

4 Recognition of printed versions of 3D objects can help create 3D simulations of those objects without having to actually build a physical model. This can be of great aid to people who constantly work with 3D applications such as architects and animators. We will talk about them in later chapters.

5 Recognition based AR can be used in projectors to automatically detect a projectable surface and project on only the projectable area. The projection can be made interactive by using dynamic objects in the surroundings to command the projector. This can eventually be used with the projection mapping technique to auto-detect various types of objects and send out projection imagery according to the size, distance and colour of the surface on which projection could be done.

With projection based AR, your imagination is the only limit. There is a lot of research going on in this field and with time, more and more applications would pour in. If you are really excited about how you can create something like that of your own, we have tips for you at the end of this book. For now, let us see the location based AR.

Location based AR

It would be an injustice not to mention this category when talking about AR. Location based augmented reality is one of most widely implemented applications of AR. The strongest force behind this is the easy availability of smartphones and the features that they provide in terms of location detection. Location based AR is mostly used to help travellers in their journey.

Location based AR in most cases is used for AR location browsers which help users discover interesting places within their current location. This method works by detecting the user’s location and orientation by reading data from the mobile’s GPS, digital compass and accelerometer and predicting where the user is looking; then adding related information on screen about the objects that can be seen from the camera. Wikitude is one such application. If you’ve never used the app fear not, this video would help you realize how it looks in action: http://dgit.in/WkXxwS There are other interesting uses of location based AR which we will talk about in the coming chapters.

Outlining AR

Though the human eye is known to be the best camera in the world, there are limitations. We cannot look at things for too long. We cannot see well in low light conditions and sure as anything, your eye cannot see in infrared. For such cases, special cameras were built. Augmented reality apps which perform outlining use such cameras. Once again, object recognition sits behind all that outlining AR can do. Let us begin with a life-saving implementation example.

1 When driving a car on a road in foggy weather, the boundaries of the road may not be very visible to the human eye, leading to mishaps. Advanced cameras tuned specially to see the surroundings in low light conditions can be used to outline the road boundaries within which the car should stay. Such a system would prove very useful in avoiding accidents. With extra sensors capable of detecting objects around (e.g. by using ultrasound) the overall risk of hitting some living object can be minimized as well.

Outlining the road can help save you a mishap

The technology can help you save pedestrian lives as well. Outlining people crossing the road on a HUD (Heads Up Display) windscreen can be more useful than having a separate infrared video feed.

2 An AR app known as 110 stories allows you to see the twin towers of world trade center in New York as an outline. This might a tad morbid but it certainly is interesting. Outlining monuments from history can be utilized for many purposes the foremost is of course education.

Outlining the buildings from yesteryears is a creative use of outlining based AR

3 Outlining AR can be used by engineers and architects to look at a building and study the location of supporting pillars and metal bars. Such an application could fetch the building’s construction data from a database and provide a mesh/framework view of a building and can prove to be an aid in repairing a building or be used for educational purposes.

Superimposition based AR

Superimposition based AR provides an ‘alternate’ view of the object in concern, either by replacing the entire view with an augmented view of the object or by replacing a portion of the object view with an augmented view. In this case, once again, object recognition plays a vital role - logically, if the application does not know what it is looking at, it most certainly cannot replace the original view with an augmented one.

Depending on what type of view is required, the technology can be used for multiple purposes.

1 Doctors can use the technology to examine the patient from various angles in real-time. A live feed from an X-Ray machine can be used to superimpose the X-Ray view of the patient‘s body part on the real image to provide better understanding of the damage to bones. The application can be made to work via a head mounted display or special goggles. In other uses, the view can be shown on a screen where the video feed is taken from a real camera and X-Ray vision can be imposed on it.

2 In military applications, superimposition based AR can provide multiple views of a target object without showing extra information in text and blocking the vision of soldier from other important objects around. If you have been shooting enemies via your computer mouse, you’d already know how it would appear. Superimposition of infrared view or radioactive view of an object or an area can help save lives; or win wars!

3 Superimposition of ancient pictures over real ones can provide interesting views of historical places. Broken monuments can come back to life in all their original glory. Perhaps different eras complete with landscapes can be re-lived with AR.

4 To allow a tiger or snake near you might be a horrifying experience with hazardous consequences, except when superimposition AR is used to bring them to you. Placing a person in a location or situation which is otherwise dangerous can be can be safely accomplished via superimposition AR!

5 Superimposing a real object with its internal view can be helpful in education as well, for instance, to study bone structure. Though we have touched some of the most important types of augmented reality, there are a few others which cannot be easily classified to fall in one of the above said ones.

Superimposition AR can be used for educational purposes as well

These types use more multiple sensors and may produce the result in different forms. Such systems employ special faculties of dedicated devices working in parallel with processing systems with algorithms developed especially for being used in AR. As more input and output devices converge to more powerful and efficient systems, we would come across more types of AR in future.