Email
Around 1914 the first practical domestic refrigerator was invented. Early in 2014, just one hundred years later, came the first instance of this home help misbehaving online. The fridge, one of numerous modern smart devices containing a computer and internet connection, was hacked and started sending spam emails.
Smart devices such as the fridge are part of a new technological trend where all kinds of previously unconnected devices can now communicate with themselves and exchange information, and even take intelligent decisions. From a fridge being able to let you know it’s run out of milk and reorder it online, to a room light that knows when it’s getting dark or when your car is turning into your home street, but that on Tuesday you visit the neighbour next door first for on average 30 minutes so it will save switching on till then, this is the brave new world of the Internet of Things.
The vision is for all manner of devices to connect together and share data to provide intelligent services, tools and support for the users. The data could come from almost anything, from sensors in your home or in your clothes, to your social media presence to your online shopping habits and weather forecasts for your location.
That’s a lot of possible data and doing anything useful with it needs new mathematics and technology for processing it. It’s not called big data for nothing, and your world is now swimming in it. Making use of big data is tough, while it’s all essentially binary numbers at heart, the actual data can exist as numbers, words, pictures, sounds or movies, it’s often unstructured, so it’s hard to find the useful stuff and there is a lot of it to check.
One way of thinking about one particular piece of data is as a point in space. We are used to the idea that our location can be given by our latitude and longitude; that’s what GPS gives us thanks to the satellite networks in the sky. That location can be thought of as an x and y coordinate, two numbers giving us a position on a flat two dimensional space. Add altitude and your position is now given by three numbers in a three dimensional space.
But suppose we now get even more ambitious, our position and the temperature there, that’s four numbers in a four dimensional space. We can still do maths on this, it’s just difficult to draw it. From here on in things get even bigger, we can step up to n dimensions, where n could be thousands of numbers that each characterise a particular piece of data; a piece of music might be represented as thousands of notes, or a picture as thousands of pixels. Each piece of the data jigsaw is a point in a cloud of data in a high dimensional space, but we want to find the interesting stuff, and that means we need some way to find it.
Enter machine learning, a branch of artificial intelligence which allows the useful patterns that lurk in these big data clouds, hidden from a casual human observer, to be extracted. Machine learning uses specialised mathematics, or algorithms, to sift and simplify the myriads of numbers.
One popular approach is to look for clusters, groups of data that have some similarity or common trait, and the connections between these clusters. Like a digital sheepdog a clustering algorithm decides which pen (cluster) the data should belong in. It may take different music and decide which genre it belongs to, learning to look for the tempo and drum sounds of heavy metal, or the orchestration of a classical piece. The machine learns where the best clusters are and can even move the boundaries as more data becomes available.
Clustering isn’t the only way to understand the trends in big data; exotic mathematical functions can snake through the clouds predicting changes over time, populations of complex mathematical rule sets can be evolved with only the best surviving. Whatever the method, churning away on the powerful state of the art computer server farms around the globe, patterns emerge from the big clouds of data, like complex weather systems we can predict and track. It is from this mathematics of learning and big data that a machine can achieve the ability to generate ‘intelligent’ decisions.
Being able to predict how an individual might behave based on their past activities and the information pulled from the world around them is potentially a useful ability for a device. We are all used to our computers suggesting things we might like to buy online or videos to watch given our previous choices. But we now have the ability to take this intelligence and embed it all over the place: a smart umbrella that beeps when you pass it in the hall and it knows it’s going to rain later; a room in a care home that knows when the occupant is still as they may just be contented reading rather than having had a fall; a smart car that as well as letting you know how to avoid that morning’s traffic snarl up, can communicate with the train it predicts you will be able to catch and find out if there are any of your favourite brand Danish pastries left in the buffet car or if instead you have to buy one at the coffee shop before boarding. But as the devices around us move from old world passive to being able to accumulate information and offer decisions, to live with us in harmony, they will need to learn manners.
Human society on the whole follows social rules, manners and etiquette. While perhaps not as rigorous as the gentile past of Downton Abbey, this allows us to have some expectation of what’s going to happen in social situations. We tend to know when not to interrupt someone, how close to stand to individuals, and when eye contact is and is not appropriate, for example.
Intelligent devices also need some understanding of these rules for us to be able to interact with them as naturally as possible. Once a device is able to offer opinions or take actions as well as carry out its main function, we start to consider it differently. The TV comedy gold of Basil Fawlty thrashing his car for ‘deliberately’ breaking down on Gourmet Night is an illustration of our human tendency towards anthropomorphism - attributing human like characteristics to non-human things.
The spam sending fridge from earlier this year could easily be seen as being a ‘bad refrigerator’, not because of an inappropriate interior temperature but because of its coerced online activity. Add to the mix the potential for affective computing; the ability for a computer device to recognise your facial expression or predict your mood through your gestures or body pose and respond appropriately; increasingly sophisticated language recognition and speech production technologies, and this illusion of humanity will become even stronger.
Basic digital personalities can be created through mathematical rules associating the information the device gets from the world with how it reacts to the world. As smart devices start to work with us, and understand our social rules, we may increasingly see them as human like - a world filled with tools designed to be our friends.
As intelligent behaviour starts to permeate our increasingly connected domestic technology and the potential for those devices to become socially aware starts to filter into our world, a raft of new opportunities and challenges open up. The opportunities are clear - new ways to do new things to help improve our lives, be it a home helper, a musical companion, a driving buddy or a fitness adviser.
There are of course technical challenges too - how to build such devices and the maths to make them smart, sociable, safe and secure. But that’s only a small part of the picture. In our homes there needs to be a commercial need to build a fairy tale world of taking mirrors, singing kettles and sassy chatty wardrobes, and a wish from consumers to live in it.
But there are broader ethical issues to consider. Where would the areas be where such devices would not be appropriate to use? What about our personal privacy, both in the collection of data about us, but also in the way it’s combined and acted on? How should the law deal with smart devices, if an incorrect decision is acted on, who or what is responsible? There are many others. We are now in the early days of the Internet of Things but perhaps in the not-so-distant future our ’ things’ may become just a little bit more human like us.
Peter McOwan is professor of computer science at Queen Mary, University of London.
;