04

Ar: The limitless possibilities

Technology usually evolves through two pathways – someone “builds” a new technology out of necessity to solve a particular problem, or someone “discovers” an existing technology and finds ways in which it can be used to solve new problems (which the original technology may not have been designed for). Augmented Reality (AR) is one field that owes its evolution to both aspects equally. As new information demands emerged, new ways of adding information to reality were found. Each passing day sees an increasing number of applications of augmented reality, and its potential seems endless. In this chapter, we show you how AR is already modifying our lives in some key areas, and what to expect in the coming years.

AR: THE LIMITLESS POSSIBILITIES

Augmented reality is not just about fun and games. There are some substantially beneficial uses for the technology in everything from engineering and medicine to sports, education and more

Technology usually evolves through two pathways – someone “builds” a new technology out of necessity to solve a particular problem, or someone “discovers” an existing technology and finds ways in which it can be used to solve new problems (which the original technology may not have been designed for). Augmented Reality (AR) is one field that owes its evolution to both aspects equally. As new information demands emerged, new ways of adding information to reality were found. Each passing day sees an increasing number of applications of augmented reality, and its potential seems endless. In this chapter, we show you how AR is already modifying our lives in some key areas, and what to expect in the coming years.

Military

Although, the prime motivator behind all innovations in military technology is to make it easier to defeat the enemy and prevent the reverse, several technological breakthroughs do find their way into civilian hands. AR is one such advancement that has significantly increased the success rate for other systems and even military.

AR can be very useful in military

Wait a minute! Chances are that you are wondering when was the last time you used AR, and that too as a military application. First person shooter (FPS) games, of course. Most FPS games show some really innovative use of AR: name of buildings, teammates and weapons as you hover the mouse pointer over them, and indicating directions to the next checkpoint without ever having to shift your focus from the screen center.

However the future promises a lot more. Aided with superior tactical gear carried by military personnel during an operation, AR can even reveal things that are hidden to the naked eye – enemy movements behind physical objects, type and range of enemy weapons, points of vulnerability in the enemy’s defences, simultaneous tracking of multiple enemy personnel – all of which soon prove indispensable to infantry of tomorrow.

Picture is a screenshot from a game. Please crop the more interesting area if needed. Can be used as-is though (text provides grounds for relevance)

Vehicle Windscreens

Gaming is so much of an inspiration for AR that this application has also materialized out of popular games. Being a smart Digit reader, you would have already guessed what genre of games we are talking about: racing.

When you race cars, bikes, trucks, kayaks (basically, anything under the sun) in a game, a lot of information is usually displayed on screen. This may include speed (displayed either on a digital or an analog speedometer), distance from a vehicle, milestone, or random objects in front of you, a map of the path that you are following and visual indications of a sharp turn ahead. Depending on the game, your virtual windscreen may also tell your where cops are waiting for you and whether or not they are following you in a chase. While several developers are working on it, the technology still has to see the light of the day in the real world. One simple application of would be to send a visual alarm as soon as you near or exceed the speed limit.

The innovation and possibilities do not end here. If AR is blended with MANETs (Mobile Ad Hoc NETworks), it can be used to provide drivers with information slow-moving traffic or nearby accidents. On detecting low fuel levels in your gas take, in addition to alerting you, AR may also visually guide you to the nearest fuel station. Through visual notifications and icons displayed directly on-screen, AR will guide you to your desired destination in a new city, without needing you to calculate the ‘next turn according to map’ by looking down at an LCD screen.

We are not going to put any pictures but links to two videos:
1. GM testing AR with one of their cars: http://www.youtube.com/ watch? v=94dg2D-jAhM
2. BMW’s HUD: http://www.youtube.com/watch?v=b4jpuoGP2C8&feature=endscreen

HUDs

HUDs (Heads Up Displays) are spectacles, windscreens, or other surfaces which allow the viewer to see through them but add extra information over (or around) some specific objects.

AR in Heads up display

They allow the viewer to get more information about things that he is looking at, without having to shift his focus to a secondary panel. Fighter jets are among the most loyal consumers of HUDs, as popularized in various movies, such as Top Gun way back in 1986! HUDs are one of the most important implementations of AR. Fighter planes are built to be used in close-combat situations, where an even the slightest mistake can cause heavy losses. Hence it becomes extremely important that the human operator flying the plane has access to as much information as he can about the things he is seeing without having to shift his eyes from the scene up ahead.

HUDs are being brought into the hands of the common public via spectacles (it’s gonna take some time though). Remember Google’s Project Glass? Please keep the picture as large as possible to show the details

Medical Sciences

The fact that AR can provide more information and with minimum distraction makes it a prime candidate for the medical sciences. There are already apps such as Mole Detective that determine the likelihood of skin cancer by analyzing an image taken using your mobile phone’s camera! Of course we wouldn’t vouch for their efficacy at this stage.

AR has considerable potential in saving both doctors’ and patients’ time. AR when combined with current advanced medical equipment can help visualize and graphically display the medical issue with the patient. Special goggles can superimpose an x-ray image, previously captured by an x-ray machine, with the real-world view of the patient lying on the operation table.

AR in XRay

AR is also used in MIS (Minimally Invasive Surgery) where a small camera bis inserted into the patient’s body. One use-case involves using AR to help visualize the brain inside a patient’s head. Another involves highlighting the location and extent of hairline fractures in a patient’s bones, and recommending ideal next steps an orthopedic surgeon. Other uses include visualization of anatomical joints in live motion, tracking small surgery tools during an operation and simulating the birth of a baby to check for any possible complications.

Shopping

Sales are what makes our world go round, and hence developers always seek to connect technology to sales. AR may prove to have immense impact on retail sales.
Although the internet has revolutionized shopping to this day, we still tend to visit brick-and-mortar outlets on a regular basis. One of the major reasons people capable of buying goods on the internet go to physical outlets is to ‘see, touch and feel’ the product before they buy them. Malls can leverage the capability of AR to its best by allowing consumers to choose the best product for themselves via ‘virtual trial rooms’. which allow shoppers to look at themselves in particular outfit or an accessory without having to actually wear it. 

AR in virtual trial room

The technology works by using a camera (or multiple cameras) to capture an image of the shopper, and superimposing the clothing item or accessory of the ideal size over the shopper’s body (instead of that of the model used in the advertisement). A look at the numbers and you would be able to imagine how much profitable AR can prove to retail businesses. Imaginate, a company that developed a similar product says that the business is worth $600 million if implemented by today’s brick-and-mortar shopping industry. Augmented reality can equally benefit web-based retailers. Do watch this video to see how AR has already made an entry into the retail sphere: http://www.youtube.com/watch?v=NxQZuo6 
pFUw&feature=player_embedded

Tourism

Tourism is another industry that may can utilize AR to its advantage. Even according to Wikipedia ,“Augmented reality (AR) is a live, direct or indirect, view of a physical, real-world environment whose elements are augmented by computer-generated sensory input such as sound, video, graphics or GPS data.” Including ‘GPS’ with sound and video in AR makes it really valuable for travel and tourism business. Let us first take a small step back to try and understand the ‘why’ behind the AR--Tourism relationship.

Travel is both a luxury and a necessity of modern life. No matter what the reason be, when travelling, we are always looking at things or looking for something. For example, if you are travelling in Kolkata and you pass by Victoria Memorial not knowing what that building is and its place in history, you would probably say “ah, that looks like a nice colonial palace” and stop at that. However, while travelling, we are continually looking out for information; let us say the nearest restaurant in a city unknown to you. An AR browser application (which helps you browse your locality with AR information) could visually guide you to your point of interest.

From yet another perspective, travel was for long been associated with carrying paper maps, books to read during a journey, and paper to take notes on. The mighty mobile phone has replaced most of these items. Further, the development of smartphones has reduced (if not eliminated) the need to carry multiple ‘information sources’. A single device can now encompass a GPS, camera, accelerometer, internet connectivity and a compass in our mobile phones.

AR browsers can guide you step-by-step from point A to point B without ever needing to look at a map. However most AR applications currently do not ‘understand’ what is in front of you. For instance, current AR applications do not transmit the live video feed sourced through a mobile camera to some remote server to tell you what you are looking at (such an approach would be very expensive from the computational viewpoint and also consume a lot of bandwidth). Instead, information about where you are standing and what direction your camera is facing, in the form of GPS, compass and gyro-sensor data, is sent to the backend server of the app.

However, these technologies still require several improvements in accuracy and usability. Since the application does not understand what you are seeing and depends on GPS, gyro sensor and compass data, the unavailability of any of these services will degrade the experience of the app by a large degree. For example, if the compass in your phone accurately calibrated by 90 degrees (i.e., east becomes north and north becomes west), then depending on AR app would require you to hold the phone in front of you and walk to your left! If the gyro-sensor fails, then the app might just tell you to dig the ground to reach your destination. However, such a case would be rare. What is more common is losing the GPS signal which would make the AR browser totally useless.

AR can be fun as well as useful when travelling

Even with GPS, the information being displayed can be slightly wrong. GPS detects your position with an approximate error of anything between 10 to 20 meters. Such an error can show you wrong information when you are at crossroad where multiple roads intersect or in a historical place where you are jam packed with points of significance. Also, in most cases, people tend to ignore the AR applications and instead rely on mapping services such as Google maps to get what they want. Again, it is quite a difficult task to keep holding your mobile phone in front of you at all times to learn about the place you are in.

That was all about how AR can be implemented in the tourism industry. It’s time we find out what is already out there.
Multiple applications with AR capabilities exist today, most of which work by combining GPS, compass and gyro-sensor information to figure out where you stand and what you are looking at. However, these applications are still incomplete in terms of data. A major portion of the world still needs to be mapped and it is going to take significant amount of time, effort, money and computational resources to store and deliver that information. One of the most famous AR applications available for iPhone and Android is Layar (pronounced ‘layer’) where users can create their own layers filled with information of places they like and the appropriate layer can then be loaded on the application to assist you using the AR techniques built into the app. The by the application creates the augmented reality by deriving information from the layer’s data.

AR can be used to relive the past!

Individual users creating layers of information for small places with a layer-based approach makes it a lot more easier and helpful, than for a single organization to try and map out the whole world. For example, one may create a layer containing information of a museum’s exhibits, and allow visitors to access this information by simply pointing the camera at an object of their interest. It might also be used to ‘re-live the past’. For example, if you see a sword in a museum, then AR could make it even more interesting by showing it in action --- such as presenting the image of a past emperor to whom it belonged actually holding the sword!

Another important use of AR for historical places can be in form of historical image superimposition. For example, you visit New York and point your camera to Statue of Liberty; an AR app then lets you choose images of the Statue and its surroundings at different points in time, and then superimposes them on the live image.

Architecture

If you have seen either of the movies ‘Prometheus’ or ‘Avatar’, then you already have a hint of how AR can help in architecture. AR can help visualize structures before they are actually built and simulate their behaviour under various weather conditions. Not only this, it can also help convert 2D prints into 3D visualizations and project time in space in front of you, thus providing a much better understanding of the building than what a normal print can convey.

AR in architecture can provide a high grade of imagination and aid to the architect

Yet another way of using AR in architecture is to utilize superimposition techniques to visualize how an architectural change would look in reality. Utilizing AR in an architect’s workspace can help the architect visualize his design as if it were live. Special goggles with little help from 3D processing units can convert the workspace into a super-advanced arena of imagination where 3D drawings made in CAD software on computers come alive much the same way as Ironman designs his toys.

Sports and Entertainment

All work and no play makes AR look like a dull technology which it is not. AR can be as much fun as it is useful. You have already seen AR at work at several recent sports events, though this might not be obvious at first glance. A prime example is displaying the score and names of players directly on the field of play. AR also provides invaluable help in analyzing crucial event in real time, which in some sports can totally change the outcome of the game. Does not sounds familiar? Let us augment your knowledge with familiarity then; let’s talk about cricket.

AR can have huge impacts on results of a game!

The LBW (Leg Before Wicket) is a well known method of dismissing a batsman, depending on (i) the trajectory of the ball and (ii) the object that the ball touches first in its trajectory. Instant action replays incorporating the AR technology called “Hawkeye” plot the trajectory of ball and predict whether the ball would have hit the stumps if it followed on its path without deviation. Another AR system known as “Hotspot” uses infrared cameras to highlight the point of contact between the ball and other surfaces based on show small deviations in temperature due to friction.

Besides cricket, AR is also used in other sports such as in swimming to compare the current position of the lead swimmer with those just behind him or with a projection of where the world holder would be if he were in the same race. The sport of tennis has adapted AR technologies brilliantly to assist umpires and viewers in line calls, and football seems to be following closely with proposals by FIFA to introduce goal-line technologies.

AR can be very entertaining

Advertisers are also utilizing AR to display ads in strategic positions on the field and having them blend with real-world elements.
AR techniques are also being widely used in amusement park rides and stalls where children can play with virtual animals ranging from small puppies to giant dinosaurs. About a year ago, National Geographic along with appshaker demonstrated the possibility of AR in a shopping mall in the UK where people were amazed to see their own images but with wild animals and astronauts. This video should give you a hint of the same: http://www.youtube.com/ watch? v=TL62txWNFMY and we recommend you better be prepared to be awed.

Education

AR sure has a huge role to play in the future of education. Instead of teaching ‘A for Apple and B for Ball’ to little children with picture books, AR can be used to visually project real-looking apples and balls in front them, and also have them experience these objects via other senses (touch, smell, sound and taste). Teachers could use applications which can simulate chemical reactions, physics phenomena and maths concepts. The may “liven up” biology classes by creating virtual operation tables. AR could considerably reduce teacher. For example, a simulation of how various atoms and molecules would interact when brought close to each other would allow students to experiment and learn the concepts on their own. Rules could be understood faster and more easily even when a teacher is not present.

Below is a list of a few interesting videos here to show how AR is going to revolutionize the way kids receive education in the future:

•    Chemistry lab simulation: http://www.youtube.com/watch?v=d8KXOrFLMvc

•    Yet another chemistry lab simulation: http://www.youtube.com/ watch?v=cPYKMhR5Oo0

•    Mathematics: http://www.youtube.com/watch?v=JKn7W8uAt3k

•    Biology: http://www.youtube.com/watch?v=6N8-x5WTJhA

Art

There is no better word to exemplify the uses of AR technologies in the field of art, than “Photoshop”. Adobe Photoshop, with the plethora of options and effects, is that special software that deserves recognition in IT heaven for turning many a nerd into capable artists. By superimposing images on top of each other, Photoshop allows new images and effects to be created. Detecting patterns and predicting their continuity in a scene can help turn boring things around us into pieces of art. Combined with dedicated and powerful hardware, AR could transform the sunset from your balcony into the most beautiful sunset on a beach! Mobile apps now let us “touch up” photos soon as we click them. The popular ‘negative’ and ‘sepia’ effects also qualify as means in which we have augmented our realities and produced pieces of art.

Machine-assisted translation

AR’s role in tourism does not end with maps, directions and points of interests. Wondering how else would a machine help you see a world differently when you are travelling? By assisting in translation, of course. Languages serve both as bonds and as boundaries when it comes to communication. When you travel far enough, bonds may easily turn into boundaries. For example, being able to understand your surroundings well, while you are in Spain would require you to understand Spanish. Billboards as well as restaurant menus may become intimidating in no time. Well, you do have a translation app or dictionary on your phone, but are you ready to translate all that you encounter into your preferred language? Well, good luck on your path to frustration and killing the joys of travelling.

AR can provide immense help while you travel by translating any content that you encounter on the fly. There is no need to keep tapping in words to understand what a restaurant offers on its menu. All you need to do is to point your mobile phone’s camera at the menu and look at the display and you would be reading the menu in your preferred language in no time. This is achieved through several apps available in app stores, which almost instantly translate content captured through mobile phone cameras. Some of the really clever apps have been able to completely blur the distinguishing points the augmented and real-world features.

One such application for iPhone is ‘Word Lens’ and we recommend you to watch it in action: http://www.youtube.com/watch?v=h2OfQdYrHRs. We are sure you would love the innovative uses it can have.