Augmented reality is mostly known for overlaying visual data into our physical space, however augmented reality's scope can actually cover all of the 5 senses being imposed into real space (sight, sound, smell, taste, and touch).
Visual augmentation is now becoming more and more available due to advancements in processors, cameras and software within mass market consumable level devices. This allows developers the ability to create content that can be used daily by end users across multiple platforms, rather than just in research labs or pop up experiences in populated areas. It uses fixed and relative positioning along with camera tracking technology to superimpose visuals into our environment.
The most prevalent way that augmented reality visuals can be triggered is through visual cues. For instance you can use a well known logo or a popular poster or even well recognised branded bottle of ketchup to initiate the augmentation of visuals and information. This creates an abundance of new opportunities for us to redevelop or expand on existing visuals that we have around us today, with the impact being that the way we look at our physical and virtual world is changing. Merging our, often highly curated, online lives with our offline lives into an integrated experience opens many opportunities across multiple fields, with this technology being able to enhance and alter almost anything we do as human beings today. This new and, what is currently highly, innovative technology is on the cusp of being part of the daily fabric that makes up our existence, as involved in our everyday lives as a smartphone is today. It is not, however, without it's limitations and challenges and whilst the technology has the capacity to enrich our lives there are risks and barriers to its use which may cause disconnection with our current ways of living.
For this essay I will be exploring both of these facets, focusing on one particular key area that augmentation will effect and that is of visual culture, the question is;
What are the benefits of augmented reality towards visual culture and what are the negative aspects that could hinder visual culture?
To begin we should take a brief look at the history of augmented reality's development. This will help to understand how it has been applied to date, how it has benefited us so far and how it has already impacted visual culture. Then we can explore the current already prevalent implications impacting visual culture today by the existence of augmented reality prior to considering possible future complications.
In 1960 Ivan Sutherland and his students at Harvard University developed the first prototypes of a see-through head mounted display, which superimposed 3D graphics into the view of the user, however it wasn\'t until the early 1990s that this technology was coined “Augmented Reality” by Caudell and Mizell, two scientists at Boeing. The practical uses of augmented reality started off being experimented with by Boeing, when they developed a new way of understanding the complex development of wiring harnesses, also known as a wiring looms; which are complex collections of electrical wiring banded together to form integrated parts of engineered products. Augmented reality's application to this tasks helped improve efficiency and reduced the requirement to use what was deemed a previously confusing instruction manual. Augmented reality was credited with making the task far less arduous and improving accuracy in execution through allowing Boeing to display wire bundle assembly schematics through a head mounted display.
It was in the 90s that augmented reality development and use really took off. It started to infiltrate into many different mediums; with it being designed for use in the mass market. In 1993 a system was developed by Feiner et al called KARMA, this system was able to give instruction sequences for repair and maintenance procedures. For example it could be used by a layperson to run through a maintenance procedure on a specific printer model, overlaying step by step instructions and highlighting the exact position of the step ahead.
This did not mean say that augmented reality wasn't still being used to help with complex environments too and in 1995 a new avenue for development came about in another field of research done by the State et al. at the University of North Carolina at Chapel Hill, with a medical application of the technology. In this instance augmented reality was used to meld a 3d scan of a baby inside the womb of a pregnant patient with a projected live feed onto their belly, this allowed the physician the ability to observe the subject in real time.
The first notable foray of augmented reality into game technology came in 1997, when the Japanese government and Canon inc joint funded a research company. Over four years they produced content for the digital entertainment market and mass market consumers got a new experience of this technology through RV-Border Guards, a multiplayer shooter game.
1999 saw the introduction of the first outdoor augmented reality system. This system came with a huge backpack to carry the technology. The software “The Touring Machine” gave you an augmented tour guide experience through Columbia University campus. This year also saw the development of the first outdoor augmented reality game QuakeAR.
Ever since the 1990s the technology has advanced every 2-3 years. If we fast forward to today, March 2017, we can see how consumer level products are now readily accessible that are far more advanced than all of the aforementioned achievements. Augmented reality access now comes in the form of free, easily downloadable software, directly available for our already owned handheld personal devices - with applications like Snapchat and the use of Quick Response codes being widely available and used by many. The range of uses that augmented reality is being applied to, in order to enhance our perception of the world we live in, is now in abundance, for example we use our handheld devices and the current technology daily for navigation. Guided tours are being augmented through the use of visual cues triggered by monuments, landmarks and sculpture, which are used to prompt information being relayed regarding an area, overlaying 3d rendered content to help visualise spaces being described into our current environment where they no longer exist, engrossing people into an experience to bring history and static objects to life.
The daily use of this particular form of augmented reality technology is not a fully immersible experience yet, as the peripheral vision is limited when it is used through a hand held mobile device. Whilst the technology is already available for it to be used with a head mountable device, which would make it more immersible, it is not affordable for most, however it is expected that it will be soon. Other limitations to applications such as these are based on them not having been designed to be completely mobile or their use being encumbered by limited network coverage/data access or prompts not being designed for a particular location. The content developer will design applications in order to enhance their own company position, to meet their own strategic objectives aligned with their interests; for example if using Yelp to look at restaurants in a particular area, using shop signage as the visual prompt, the augmented reality information displayed as you scan a location will only display information which favours the reviews that are available on their platform; not reviewing locations or providing information to businesses which do not use Yelp. There is currently no centralised platform taking the role of the world wide web we use today to bring everything together into one location.
What is clear is that in the majority of applications it is visuals that will be used as a trigger in the augmented world. When the technology has reached all five senses of augmentation we could expect that each time we view the McDonald\'s sign we would smell a delicious smell, taste french fries or feel an enjoyable sensation (like warmth or a hug) whilst hearing their theme tune. This might sound like far fetched tales of the imagination however most of the technology is already here. Sight and sound, as an augmented sensation, is already readily available and soon, it is anticipated, that an augmented taste sensation will be following shortly.
Augmented reality smell technology actually beat sound to the movie industry. Although it has not been as successful an addition as sound, it has been in active development for many years and the library of smells available is continuously being added to - such as fresh cut grass, burgers and bacon. Olorama is one such company that has “70 different scents and growing everyday” and is already designed to synchronise with visual augmented reality technology. Although this technology is not a portable experience, it can be placed within locations by companies to accompany the visual trigger of viewing the marketing material, heightening your experience.
The augmentation of sound is as you can imagine readily available and has been for many years by the use of headphones, tape players like Walkmen, CD players and mobile devices that can live stream audio data with many applications of use to entertainment and education.
To accompany visual, sound and smell augmentation there is another sense that has also been developed, touch also know as haptic feedback. The sensation of touch plays an important role for our need for feedback when interacting with virtual objects. As virtual objects are not present there is a requirement for there to be a means by which we are able to gauge physical feedback, for us to be able to connect with the objects, particularly for if they are to be manipulated or interacted with. A set of gloves with haptic sensors on each finger allow you to reach out and use gestures to manipulate a 3D object which could also trigger an interactive element of a poster or billboards or to rearrange digital assets throughout your home, for example a piece of digital art or the television position and size. Haptic technology provides instant feedback allowing you to understand when you have control over the object you are attempting to manipulate.
A crayon and some paper arguably the first creative outlet a person gets, kids today know how to navigate a tablet device and watch their favourite programs or play their favourite applications from as little as two years old, they\'re digital natives. It has been documented that in handing young children, as young as 1 year old, a magazine their first instance is to try to rearrange images or edit the text using their finger. They become irritated with its unresponsive design and chucking it to the side with disinterest, looking for their next more interactive fix.
An image of a child's hand colours in a colouring book which is being viewed live through a camera on a tablet device that has taken his input and wrapped it around an animated three dimensional character and augmented it into his reality. Creative applications of augmented reality are encouraging kids to continue to pick up crayons and get creative whilst embracing new technologies.
The limitations of this causes users to be confined by what someone else has already created. The character is pre designed on paper and in 3d space the only input a user has will be the colouring within the pre defined lines.
If you take a look at a child's drawing without limitations of any pre defined lines or complex medium you can see that they\'re highly uninfluenced by logic and their imaginations are incredibly creative; Simply giving them the crayon and a blank canvas will allow them to be unconfined from any influence, although this augmented concept will encourage the use of crayons which could lead to kids wanting to produce images of their own, it would not produce the interactive 3D model when they attempt to use the tablet device against their own creations.
This technology also applies to many more uses of creativity, instant 3d visuals based on your input can be useful for designers who want to visualise their patterns, colour schemes and more onto 3d trainer models, clothing, tattoos could be augmented onto a customers body to understand placement ideas or whether it's actually what they want.
One area of impact will be on visual artists themselves, this new medium for them to express their vision comes with a level of complexity. Understanding computer code is a career in itself, and it is also not for everybody. It is regularly perceived as a dull and nerdy process even though it results in some incredibly creative outcomes all on its own. Until developers build a set of creative tools and house them in a piece of software for example “Photoshop” for photography or “After Effects” for motion graphics the artist would with be required to educate themselves enough on the process to be able to do it themselves, or collaborate with developers. This requirement could lead to some visual artists to not pursue their vision through this medium and use something they feel more freedom with, rather than having to explain their vision in a hope that the developer completely understands their requirements.
Brian Mullins of Daqri has set out to elevate this need to understand code on a developers level by building a new platform that will allow a user the space and tools to produce their interactive 3D content and further publishing it into Daqri's market place, where people can download content and easily augment it into their world.
For the end user the augmented reality medium is mostly experienced through the screen of a phone removing the immersive experience.
When this technology becomes readily available in a unified manner and is more accessible via portable devices, manipulation of pretty much all of our world will be possible. Subsequently this is likely to result in a number of questions, particularly with regard to the legality of how and what can be augmented and where it can be placed.
As it stands today there is no unified platform; content is being developed on an application by application basis and the rules of the augmentation are set out by the creator of the app. Although this new medium isn\'t occupying a physical space accessible to anyone other than the user it is opening new questions as to what it is allowed to be manipulated. For example a ‘Pirates of the Caribbean' poster was used as a trigger point for an augmented reality application to overlay the image of Lloyd Blankein, the CEO of Goldman Sachs, referencing him as a modern day pirate, in place of Captain Barbossa.
As this manipulation is taking place in digital space, on the device as a layer on-top of the physical image it is not altering the physical image therefore it is not breaching any copyright laws today. Although for the manipulation to take place it is highly likely that the creator took the physical image and altered it in digital form to be able to create a believable alteration, colour matching and placement which does run the risk of breaching copyright laws.
Where augmented reality and copyright will play a big part is in the advertisement industry. Advertising has already began to embrace augmented reality for it's marketing campaigns, particular where billboards are concerned. For instance above we have a M&S advertisement telling you to download a specific application and point the camera at the billboard to trigger a superimposed augmented video in the billboards physical space thus extending their advertisement experience further.
The question is what if another app was developed to react to the same image to produce different content for example their competitors hire a company to produce a “smear campaign” or a protesting movement to produce a culture jamming alteration.
Technology even has the ability to react to the location of the billboard, rather than the image itself overlaying a different advertisement, these risks may seem like something we would actively have to seek out, which they are, however there is an ever increasing likelihood of them becoming issues on a regular basis as this technology is developed. When we are all walking around with head mountable displays being used as an extension of our phones and if all augmented content is projected from one centralised application, this issue will be a hot topic within the court rooms, when who owns what augmented reality triggering space and how our senses are triggered could happen without invitation or even in opposition to the intended wishes of the original content owner.
Another area of visual design that will come into question is within architecture. Developers of augmented reality have the ability to make virtual alterations to a building and does not have to adhere to structural limitations, this can be used for advertising, an extension of the design or even something more menacing by augmenting flames spewing from every window. Legally the owner of the building is able to alter or demolish the building without the architects consent. So naturally the owner could augment visuals to alter its appearance as they please. However, anybody can alter any building as they so please in a virtual world without any consent, as for the time being there is now law to say otherwise. This ability to add to the aesthetics of a building using augmented reality comes with an abundance of opportunities in design and advertisement, however like anything else it arrises questions towards future considerations.
An interesting side note from this with the ability to add flames to a building in mind, I can't help but wonder how a hallucinogenic experience could be created, or inflicted, without the need for taking substances - sensory overload could be mind altering. Equally there is the possibility creating more and more immersive entertainment in the future; for instance by augmenting a movie experience unfolding throughout your location.
Telling stories from a relative position in the street, picture being an extra in a classical scene from a movie, a fly on the wall, or a POV of the lead, or secondary character. For example, being on the pavement whilst Ryan Gosling and Rachel McAdams dance in the middle of the road, laying down and then seeing the car break up the romantic moment. An interesting concept that could alter the visual cultural experience of entertainment we create today.
If the location, building, sculpture or even billboard as to which augmented content is applied can be sold by its owner, can he sell the location multiple times to different applications, as for the moment they do not conflict with any other augmentation by another company.
Where advertisement and augmentation could develop, is in personable visuals for multiple individuals, by communicating with their device obtaining their profile image and augmenting it onto the advertisement, if you\'re with a friend who is connected to you on a social site it could place both of you into the advertisement at once. Whilst another person looking at it would only see themselves and not what you see.
This would have an impact on visual culture as graphic designers and artist when producing advertisements for companies, would need to adhere to the technology\'s requirements to be able to superimpose the subjects onto it.
With marketing in mind, consider the idea of using your HMD (head mountable device) to be able to have effectively a visual search engine guiding you throughout your day, whether it is directions for work or pleasure using personal or public transport to navigate the confusing underground systems or bus routes the HMD can take into account current traffic and anticipated traffic to suggest the best possible option. Think about when you enter a library and are able to be guided to a specific book you would like to loan, when you enter your local grocery store you would be able to pin point the location of each item on the shopping list that you stored on the device with an efficient route path (calculated by the devices knowledge of the floor plan) throughout the store.
This will have a huge impact on the way we view the visual culture of branding, when you are out shopping a branded product could be branded to appeal to your interests rather than having physical branding it could have a basic branding in the physical world to work as registration marks to trigger one of several brand styles dependent on your personal demographic or interests.
The image shown above allows brands to tailor the same product to multiple target demographics based on different data collected from the users device, enabling them to capture different audiences with the same product. Expanding the amount of visuals that need designing for each product.
To summarise, what we have discussed is a small fraction of how augmented reality technology will impact on visual culture as we know it today. It is allowing us new ways to be creative without completely moving away from the traditional methods. Adding more layers to architectural designs, more interactive content in our physical world. Offering new efficient ways to work and how we may be entertained in the future; but also opening new questions as to the legality of how it may be used. To augment is to make something greater by adding to it and we seem to be honouring definition with augmented reality whilst also questioning everything we develop for this platform and its potential effects towards our already established visual culture practises today.
Not without challenge; requires knowledge growth and technical expertise development - is coming; we need to adapt……harness it as an opportunity to make static objectives interactive and sensory……blah blah blah.
Although this technology has been around for some time, it is only now that it is coming out of its beta state and being born into our world. We will find that we will change with the technology there will be new normals for us and where there is change there will be conflict where we will have to debate moral questions of new rights and wrongs.
...(download the rest of the essay above)