I’ve been rethinking my major project in the last couple of weeks – the central idea remains the same, but I’ve completely turned the actual expression of it upside down into something that I think will be really cool.

I was wrestling with an interpretation of this augmented reality future technology. Not entirely happy with my whole interactive mannequin idea that I was running with for quite a while, I thought about maybe doing an animation instead, as I like to think I’m a reasonable artist and perhaps could concoct an animated sequence of a future where everyone’s world is warped individually by an augmented reality implant, turning people’s physical properties into avatars from something like Second Life and everything they could ever want is expressed digitally right before their eyes. They’ve never known life without this, so when the electricity runs out one day, they see a dark, grey world of nothingness that falls apart because they relied on technology so heavily.

The appeal of that is that technically, I know how to do it, it’s just it’s going to take a hell of a lot of work and it’s probably not the sort of thing they like to see from a final project on this degree – more for an animation one. So I needed something that I am enthusiastic about, but also that tutors will like, so I thought perhaps focusing on augmented reality quite so literally is where I’m going wrong. Why don’t I strip it back and simplify it a bit and think about what that technology is all about and why is it important? Similar to my dissertation, I figured that it’s about addiction to the internet and the constant feed of information that we all seem to crave. Some of us can’t go five minutes without checking our facebook, twitter, texts, you name it. So why not hook yourself up directly to the internet and just know when an update occurs and what it is. It’s the state of being a cyborg, man merged with machine, that I find fascinating about this whole area, and that’s what I want to play around with, rather than limiting myself to just a single type of futuristic technology.

The idea is to use a person’s senses – sight, sound and touch, to transfer information gathered from the internet directly into them. This is the next best thing after implanting the information straight into their heads, which isn’t really an option, but by using sensory perception in ways that haven’t been thoroughly explored before, I’m aiming to go for the idea of literally being able to feel the internet through your body. I plan on accepting various information feeds, twitter, digg, facebook, etc and parsing the data in some way, for example traffic volumes or searching for key words related to emotions, and then those are fed back to the body in various ways, so you can feel the activity in real time, as it’s happening.

It’s a simple concept but I really like it, and I’ve run it past both Shaun and Gianni, who both seem to understand it and share my enthusiasm, which is a great sign! Now comes the tricky part, transforming that slightly vague idea into a plan of various physical components. Sight and sound is easy enough. Well, not easy, but it’s within my comfort zone and knowledge that I may already have or am confident that I can acquire. Visually, I’ll use Processing to form a visualisation of the data, with sounds also generated and fed through headphones. I can work that out, but using touch will probably require me to come out of my comfortable world of software, into scary electronic components and making things with my hands and stuff. That’s the downside, as unfortunately it happens to be the most interesting part. Everyone uses sight and sound to feedback information but other than the vibration feature on your phone or game controller, or the physical depression of buttons that let you know when you’ve pressed them, we don’t take advantage of touch when building software and hardware, in quite the same way. What I want to do is be able to place components in various places on the body that the user will sense when they are activated. Imagine an electrical signal being felt riding slowly up your arm or down your spine. When in sync with what you’re seeing and hearing, it should create an amazingly involving effect.

Gianni pointed me towards Arduino, which is a programming environment that allows the computer to reach out to electronic components through an Arduino circuit board. Sounds good to me, but I’ve been having a lot of trouble finding just the right components to use. Gianni seemed particularly interested in hot and cold emitters which would be a really nice, subtle effect to achieve. An alternative are small electrical pulse stimulators like you get in electric shock games (though hopefully much gentler) or vibrating components. In an ideal world, it would be great to use all of these to create a really varied effect.

The problem is knowing what to search for to find them, as I’ve tried so many different searches to try and get what I want, but to no avail. Eventually I managed to find a small vibrating component that I reckon would do the trick, but it seems a bit pricey, considering I’ll need quite a few of them. Potentially I could create a cooling sensation with the use of tiny fans such as those found in cooling computer hard drives and stuff, which I should be able to turn on and off through the arduino board. It’s been a real struggle trying to find the right kind of component that would generate heat however. It seemed everything was focused on sensing heat and capturing it rather than putting it out. Clearly a lot of electrical stuff, particularly lighting, generates heat anyway, but not knowing hardly anything about electronics, I’ve no idea what kind of things generate the right amount of heat that can easily be turned on and off and have an instant effect, and be small enough. These heat pads seem quite interesting but you can’t turn them on and off, so you’d have to have a mechanism where they’d be lifted on and off by mechanical arms, and that sounds like way more trouble than it’s worth.

I’m starting to form an idea of how it would all look, and it needs to be something that is relatively easy to just put on and take off. There will be some experimenting to do with what kind of sensation works best on what parts of the body, but I’m thinking a focus on the arms and back would probably be best, therefore some sort of jacket lined with components and hooked up to the board would make sense. Ironically the whole point of the project is to merge man with machine and jump from a handheld device straight to the person’s head and become a true cyborg, but since that isn’t possible, a jacket that tells you whenever someone tweets or posts a photo will have to do! It’s somewhere in between head and handheld device anyway. In that wearable computing vein, I then found this which is a tutorial explaining how to create your own self-heating clothing for practical purposes. It would seem that, in a similar principal to a toaster, to generate heat, you just need to pass a current through a high resistance wire – a concept that I just about grasp, but it means rather than getting loads of separate heating components I could line the clothing with wire, as the tutorial does, which would give an all over effect. It does mean that it’s difficult to then control in terms of switching on and off easily to create the kinds of effects I have in mind, such as a surging heat that passes across your arm. However, perhaps I could use the fans to control this and normalise the areas that I don’t want heated. Sounds a bit fiddly to me, but I’ll only find out through playing around with it and seeing what I can make.

So I have a slightly scary task ahead of me, considering I have a terrible record with building anything tangible that doesn’t fall apart or look really tacky, but fortunately I do really like the idea, and I think it could be amazing if I can pull it off, plus I’d really like to be able to make this kind of stuff! It’s a big if, and will probably turn out to be quite an expensive project, but if I start now, I have enough time to get something out of it, I’m sure. Let’s do it!

Working Title: ‘The Update’ – Are we controlling technology, or does it control us?

Argument: The availability of online social networking and leisure services and the capabilities of mobile phones and other portable devices have surrounded us in a seductive world of digital information in the last decade. My argument is that we are becoming too overly dependent on the luxuries that we are now being offered as we buy into advertised lifestyles and new gadgets without question, and reduce our attention spans through constant interaction with networking sites and information feeds. These trends could naturally progress into a state where we become more and more dependent on technology, even physically in sync with it, for every aspect of our lives and society that if we should lose it, we would become unable to function.

Introduction

Convergence of Technology

Multiple services on multiple devices
- Devices once used for one specific task are now multi-functional, e.g. mobile phones, media players, games consoles, etc. Online services once only conceivably possible through a computer now appear across multiple platforms, surrounding us in complete, always-accessible solutions.
- Smartphones use touch screen technology to eliminate specific controls designed for a particular purpose (e.g standard telephone numbered button interfaces). Now, the software has near complete determination over what the device can do and how people can interact with it.
- Both software and hardware companies originally set to only one particular industry can now explore each others territories in search of providing complete lifestyle solutions (e.g. Apple), gaining more control and profits and making less room for smaller, specialised companies.

Throw-away society
- Some markets are so fiercely competitive that new models are constantly being released that push capabilities further and further, particularly in the case of mobile phones, for which many people feel it necessary to have an annual upgrade in order to keep up.
- This creates a throw away society. Products are no longer built to last and people become used to throwing out old devices and appliances without sentimentality, causing waste and damage to the environment.
-  Are we becoming slaves to consumerism? Are companies all too easily able to tempt us with the latest gadget through clever marketing?

Drive toward new formats
- Constant drive toward more efficient, more capable hardware, often leading to changes of format. New generations of games consoles make previous games and older hardware obsolete. DVDs replacing video, to be replaced by Blu-Ray? Consumers are forced to repurchase existing products in new formats.
- However, in many cases this can be a relatively slow process, as people do become used to traditional formats (e.g. on-demand TV hasn’t yet completely replaced broadcast because people are still used to the schedule system. Downloadable music and video hasn’t yet replaced CDs and DVDs).
- Downloadable content is now the main focus, as download speeds and storage space increase exponentially. This leaves content more easily open to corruption or data loss and crucially, piracy. New methods of selling products digitally and reducing piracy still need to be explored.

Stuck in the Loop

What is the Loop?
- Brief history of Cybernetics and the study of how a system evolves and changes its behaviour according to the feedback it receives.
- The internet provides us with a such a system that we can have constant interaction with and develop addiction to.

Addiction to the loop
- Manifests in our obsession with receiving information as soon as it becomes available, e.g email, text messaging, social networking sites Facebook, Twitter, instant messaging clients, RSS feeds, Youtube and on-demand video.
- These services can provide big distractions when working, degrading our ability to focus on a set task as we continually demand updates from our social circles and areas of interest through direct, instant feeds.
- Short attention spans are evident through television in the form of multiple channels, flashy adverts and messages, quick shots and modern mechanics in TV shows and movies that jump quickly between plots and time periods. We’re bombarded by information overload that causes us to become too impatient.

Examples of  addiction to the loop
- The allure of social networking sites and their ability to create a global village and an easy way to exchange instant information. It replicates school playground mentality as the number of Facebook friends becomes a kind of symbol for popularity.
- Online games such as World of Warcraft and Call of Duty. When a need to temporarily escape reality actually becomes more important than reality itself, through the dangerous ability to communicate and play with millions of others.
- Online acronyms and shorthand becoming more and more well known and widespread, and representative of our need to quickly express ourselves.

Why the internet empowers us
- The internet provides us with a universal, anonymous world free of the kind of consequences in the real world, e.g. physical pain or humiliation. Web 2.0 services allow us to communicate to others in ways never before possible which gives us a sense of control, but also a struggle as we attempt to get our voices heard.
- Twitter provides us with a voice, allowing us to commentate on our own lives in as much detail as we choose. There is also an interesting link with celebrities in that they are placed within the same environment and become much more approachable.
- Digitally enhances our minds and provides us with super human capabilities, e.g Wikipedia, GPS navigations.
- Many avenues for creation and sharing have opened up allowing the celebration of amateur work, possibly at the expense of high quality professionalism. For example, amateur journalist bloggers, Youtube etc.
- The ability to review and rate products and services has lead to the more successful and well known constantly being favoured over the lesser known products, which are being pushed further and further down the list even if they are of equal or better quality, or more relevant. Google search results or mobile app stores are prime examples.

Conclusion / The Ultimate Update: Augmented Reality

The complete solution to convergence
- How current trends for converging technologies may ultimately result in a single solution that utilises augmented reality to replace computer and mobile phone interfaces. Gesture controls for hands will provide a sophisticated method of syncing our body movements with digital manipulations.

What the augmented reality technology could do
- The information of the internet could break out from the traditional desktop and become tagged to real places, items and people, creating a rich world of information that we would become absorbed in.
- The ability to consume media and communicate with others wherever you are will inevitably lead to further reduced concentration and a tendency to skip between tasks, not directing full attention to any one thing.
- At what point does it stop? Will there be a future where we’re hard wired directly into information straight to our brains, communicating through electrical signals? Education could become partially irrelevant if information is automatically available to us instantly.
- What happens if we lose power needed for these advancements? How will we then cope without them and communicate with each other?

I’m faced with a major project proposal tomorrow, and I have plenty of ideas and cultural concepts to explore, all related to each other but I just need to unpack them and determine which ones I want to focus on, both in terms of the major project and the dissertation. So this is just a post to organise and formulate my thoughts and project ideas.

They all seem to come under a general theme of science fiction, based on cultural obsessions with technology. They’re led by this idea of a futuristic augmented reality display system designed to replace computers, mobile phones, sat navs, games consoles, mp3’s and all manner of gadgets, merging all of those technologies into a single piece of mobile equipment that integrates with the body. Information from the internet could be tagged to real life objects and you could communicate, work and entertain yourself through a constantly connected interface overlaid onto your vision. See previous posts for various practical uses in day to day living.

That’s what kicked off this train of thought, simply, “how cool would that be”, but over the weeks I’ve found at least a couple of different important cultural angles regarding both the cause and effect of this technology being produced, so that the augmented reality device has become more of a tool with which I intend to illustrate these points.

Converging technologies and multi-tasking devices

First of all there’s the idea that this technology would only need to come about through a strong cycle of consumerism and the merging of different platforms, creating multi-tasking devices that each evolve from having originally had only one particular purpose. Mobile phone companies are in constant competition with each other because they always need to be able to offer new innovative features, in addition to keeping up with the features of their rival handsets. As soon as something becomes popular, it quickly becomes essential, and customer demand drives more and more advanced, multi-tasking products, overlapping other specialist industries such as portable games consoles, computers and cameras, which in turn cause customers to exchange gadgets more and more frequently and become more demanding. This never ending loop could eventually cause the augmented reality system mentioned above, almost biologically linking us to technological enhancements and crucially, each other, through the web. We would become cyborgs and have a tremendous reliance on the kind of tools we could previously have done fine without.

Stuck “in the loop”

My other parallel idea to also drive us toward this utopian/dystopian future is inspired by some thoughts presented in a very recent book I’m reading called Cyburbia by James Harkin, who writes about the modern obsession of being “in the loop”, crucially driven by the popularity and availability of the internet. Because of what’s available to us, we’re becoming constantly in need of updates on all of our interests and social stories, be it through email, facebook, twitter, youtube and on demand video, RSS feeds, texts, instant messaging and so on. The need to know what’s going on appeals to natural human curiosity and has been with us with newspapers, books and even just through verbal stories, but now everything is so instantaneous that we can’t bear not to be able to access it as soon as it becomes available. Some of us (myself included) often find themselves at a bit of a loss as to what to do when we temporarily lose our internet connections, or even worse, eletricity in general – automatically opening our email or turning the TV on without thinking, knowing full well that they’re out of action. We’re obsessed with the idea of generating feedback by posting messages and receiving other messages in response, thereby altering the linearity of what we’re interacting with, creating particular satisfaction. This obsession fits neatly with augmented reality – we’ll no longer need to have a computer or gadget handy to know what’s going on or even be in an appropriate place. Our crucial information will always appear before our eyes as soon as possible, ready for immediate interaction and our attention will constantly fluctuate between reality and the virtual network.

There are numerous smaller concepts to dissect from the implications of a such a technology, for example in an advanced time, eyes could record what they see, keeping a complete history of the user’s interactions and events stored in a digitally enhanced memory. What effects does that have on our privacy? With the ability to place 3D models directly into our world, bringing games and simulations to life like never before, why do anything else? Plus, there’s something in the idea of taking the internet and dispersing it across the real world so that information taken from websites, maps, GPS signals etc pops out of real life objects, creating a kind of digital tagging mechanism, making our lives simpler and more convenient – perhaps too much so. These are more alternative, related areas that the project could evolve into.

Project ideas

I really think these kinds of idea are pretty strong, and there may be room for both of these key points in my project and dissertation under a general umbrella idea of the causes of a cyborg future. Although I haven’t quite worked out my plan of attack for my dissertation, I think there’s enough there to go on and I’m sure I can work out how to summarise it all neatly. I must admit I’m finding it hard to settle on a major project idea that sums it up though, or focus on a particular point that I could clearly express.

The obvious first choice would be to attempt to go some way toward making it, like this. I know I couldn’t get anywhere near my crazy science fiction ideas for it though, and in such early phases it could be more use to particular industries (tourism, medical procedures, etc) than consumer use, which my whole idea is based. Plus, I don’t have much enthusiasm for it and it’s not a particularly original project idea. If anything I would rather do something a bit artier, which surprises me a little because before starting the year, I didn’t think I would.

The iPhone application/narrative

I then came toward the idea of compressing it into an iPhone application, which would allow me to focus on the more software side of things, and design the interface for it, using the features of the platform such as its camera and touch screen. The inevitable problem though is that it doesn’t really capture the idea of the technology being integrated with the human, it’s just appearing as a kind of vague approximation on a platform for which many apps already exist that do anything that I would be capable of making. I did however wonder about the idea of wrapping the technology and its use in society around a narrative, incorporating hired actors in some sort of futuristic plot. Although I quite like this idea, I haven’t yet thought up a story that would have any real meaningful purpose other than simply demonstrate what the technology would be capable of. With the more theoretical ideas that I have about the cultural trends that would cause this cyborg system to be created in the first place, I want to make something that proves these points, in addition to just showing how cool it would be.

The installation

My current idea is not fully thought through and as such I’m not sure if it’s the final one I want to go for, but it is at least a step in the right direction and one that I would like to do, but it’ll depend on whether it gets the ok from my tutor or if they suggest something else or adapting it. I’m considering switching the roles around a little bit – rather than give the user these futuristic abilities, I’m considering creating someone who has them and is used to how they work, so that the user interacts with this person from the future, is able to see what they see and therefore how their world is augmented by this digital information. As an installation, the future person could take the form of a mannequin, potentially with articulated limbs and head to demonstrate a hand gesture interface, but crucially with a webcam in its head, capturing a feed that is presented on a monitor next to them. Most likely using Actionscript, I will be able to superimpose various interface components over the feed, simulating the augmented reality and hopefully I’ll be able to have it recognise certain objects such as human faces. It should demonstrate their divided attention between reality (the people they meet) and incoming emails, facebook, twitter, RSS feeds, media, etc. Passing users will be able to supply their own messages and profiles through these various services to interact with the person. Other concepts for what the technology could do should also be incorporated such as the ability to recognise certain objects (additional information such as reviews, clips from the web etc could pop out of a particular DVD cover that is held up, for example), they could remember the face of a visitor and record their meeting, and when they return, the option to play the movie back could appear, and it’ll just generally be able to associate pieces of information with that person on-screen, such as tweets or photos.

The problem with grounding the interface to a particular spot is that it removes one of the key features of the technology – that it’s portable, and can identify places and things on the move and attach information to their images. However, making it actually work in a physical space, along with appropriate mobile hardware, would probably be quite hard, at least in terms of getting anything meaningful out of it. By grounding it and restraining the incoming feed to only one spot, I can have much more control of what it’s capable of seeing and analysing and therefore provide the most effective preview of what it can do.

I’ve been drawn to the idea of future technologies and gadgets that aren’t yet available or fully possible but are currently being researched and developed. I like the whole idea of technologies from the realm of science fiction becoming a reality, for example, there’s something about a mobile phone that, twenty years ago, would have been the kind of thing seen in science fiction visions of the future. What particularly caught my eye is augmented reality, and the idea of adding a digital overlay directly to our vision, able to recognise things that we’re actually seeing and add digital tags and information onto them.

Augmented Reality in science fiction

The piece would take the form of contact lenses or glasses that could conceivably replace your computer and mobile phone by adding the various interface windows to your field of vision, moved and reshaped with the user’s hands, in a Minority Report style gesture-based control system. If stable and reliable, the possibilities for such a system could be limitless.

For one, it wouldn’t just be the case that you’d have an internet browser with you wherever you go, information from the internet could actually break out of a browser window and appear alongside real objects in just about every aspect of our lives. If you’re passing by a shop, you could touch or reach out to it and find out if it has a particular product in stock, if it’s a restaurant you could see how many tables are free, what the soup of the day is, etc. If you’re picking up a DVD or book from the shelf, the cover or barcode could be read and reviews and ratings, or a list of keywords could be pulled from the web, even clips from youtube etc.

People could make their facebook or twitter profiles public so that they would just appear as icons either side of them when you meet them, which you can touch to access. When going somewhere you’re not familiar with, you could have a big arrow at the top of your vision pointing the way, or a window that displays the locations of your friends nearby.

Audio through your earphones (or directly into your head perhaps) enables the intake of media and voice communication. Perhaps even further, an accurate 3D representation of the other person could be projected in front of you as they speak, making it seems as though you’re in the same room.

Even the camera features of mobiles could potentially be included as your eyes could take pictures and videos of exactly what you see, allowing detailed recordings of all of the events in your life, essentially providing digital assistance to your own memory. It’s the ultimate convergence of different technology and the merging of man with machine.

The ideas could go on and on, but obviously, I can’t make this system and I’m not that interested in attempted to do so, in terms of how it could technically work. It’s been done before, I wouldn’t get very far with it, and I wouldn’t enjoy it. I’d be much more interested in demonstrating these kinds of capabilities from a software point of view so I’d be looking more at a demonstration of what it could do.

At first I considered an iPhone application that could recognise certain things or perhaps place the user in a location-based game or story set in the future, centered around ideas for this sort of technology. But then there are applications capable of doing anything that I could make on the iPhone, and it wouldn’t really demonstrate the capabilities of the future technology, more the capabilities of the iPhone really. Plus, it would be hard to control how well it actually worked.

So I thought rather than placing the user in the future, how about the other way around, and the user meets someone from the future with this technology. They can see what the future person is seeing, with their augmented overlay and see how their actions can trigger certain effects in the technology. So even though I wasn’t sure about doing one at first, I’m now considering more of an installation. I’d take a manaquin and place a webcam in its head, so that what the manaquin sees could be displayed on a monitor next to it, on which I can overlay a fully designed augmented interface.

Using face and object recognition, the future person would be able to meet people who stand in front of him, and record the meetings so that if they come back, it could have the option of selecting the videos of the previous meetings. It could take various information from the internet that it could bring up when presented with certain objects, access people’s twitter and facebook pages, you could email and text him and see the messages pop up in his field of vision. I could potentially even actuate his arms to show how the gesture-based interface could work, but in general, the aim would be to demonstrate a future of where technology could end up, based on current trends. A person that becomes one with digital life as it integrates with his own knowledge, memory and method of interacting with others and his environment.

Devices and technologies were once designed and used only with a specific purpose in mind, but now, as a result of fierce competition between software and hardware companies as they break into new markets, mobile phones, computers, personal media players and games consoles frequently share common features. Internet access and all the various competing communications services that come with it, download and storage of music and movies, games and other applications converge together across a wide range of devices, all with their own separate origins, but now offering similar experiences. This is both the cause and effect of an ever more demanding public, so used to multi-tasking gadgets and the constant drive toward the next update.

The fact that these separate services can be contained within a single device suggests that specialist hardware for each doesn’t necessarily need to exist anymore. Modern smart phones featuring touch screen technology promote the capabilities of the applications running on them, each with their own unique interfaces to suit their nature, whether phone, web browser, game or camera. An array of bespoke buttons, thus limiting the flexibility of the device’s interactions, is no longer necessary. Pioneering smart phones such as the Apple iPhone and modern games consoles such as the Nintendo Wii bring a popular trend of gesture based controls, exploring new methods of communicating with our technology.

With a constant drive to innovate and impress, are technology companies quickly advancing toward a single hardware solution, combining the trends of converging software capabilities and interactivity designed to accommodate human gestures? This dissertation aims to gaze into the future and propose a single piece of technology, capable of providing any conceivable type of software, and suited to modern human requirements at any time or place, with the potential to completely replace traditional bespoke gadgets and appliances.

The proposed device would place the user in an augmented reality, directly intercepting their vision and overlaying it with software graphics, either appropriate to visible surroundings or separate. All of the capabilities of a personal computer could be combined with the instant communications and messaging aspects of a mobile phone, but with the ability to push and pull 3D objects with hand gestures, as if they were real. Information could be tagged onto digitally recognised places and people as the web breaks out from the restrictions of a browser window, into real life itself, where we become fully connected with the world at all times. Friends or family members communicating from different locations could have themselves projected in front of each other, as if they were in the same room, and our eyes could act as cameras, perfectly capturing our first person views of moments in our lives for playback or sharing.

Crucially, this dissertation aims to seek the consequences of these concepts for social interactions, commerce and entertainment industries, and how they would adapt to such technologies. Despite its many obvious advantages and innovations, would it be a necessary upgrade and if so, does it further our increasing dependency on technology in such a way that we would one day be unable to function in a world without the internet and digital realities as our tools?

Bibliography

Hainich, R. (2006) The End of Hardware: A Novel Approach to Augmented Reality

Harkin, J. (2009) Cyburbia: The Dangerous Idea That’s Changing How we Live and Who we Are.

Kirby, A. (2009) Digimodernism: How New Technologies Dismantle the Postmodern and Reconfigure Our Culture.

Feenburg, A. (1999) Questioning Technology.

MacFee, G. (2002) The Architecture of the Visible: Technology and Urban Visual Culture.

Tofts, D. (2000) Parallax: Essays on Art, Culture and Technology.

Lacy, S. (2008) The Stories of Facebook, Youtube and Myspace.