Augmented Communication: are mobile devices really ruining face-to-face communication?

R.S. Pinner
9 min readOct 22, 2020
Ever show your phone to someone as part of the conversation… maybe to win an argument or learn something on the fly? Smart phones are said to be ruining conversation, but is that really the case?

Microchips may not yet be part of our brains, but they are part of our lives, and they are already becoming a part of how we communicate. This may seem like a glaringly obvious statement, but this article is not just about the fact that communication has changed how we talk in the traditional sense of online communication. There are thousands of articles on the subject, it would be like stating that the earth is still round. This article is about the fact that we now rely on our phones and other networked devices to make conversation, even when we are talking face-to-face with people in real-time. As this practice lends a great deal of added technological enhancement to daily conversations, I refer to this as augmented communication. This is an area into which I am currently researching, and in this article I will explain what I have observed so far.

The term Augmentative and Alternative Communication (AAC) is already in use to describe how hearing and speech-impaired people can utilise technology to help them communicate more easily and convey their messages with greater accuracy. For example, people work with those who have lost the power of speech using tools as simple as a board with the alphabet on, which can be used to spell out words. When journalist and editor of Elle magazine, Jean-Dominique Bauby, was paralysed after a stroke, he was left with only the ability to blink his left eyelid, a condition known as locked-in syndrome. With the help of his speech therapist Claude Mendibil (and their shared patience and motivation) the novel The Diving Bell and The Butterfly was written using a process called partner-assisted scanning, a form of AAC.

Stephen Hawking could still write books and deliver lectures thanks to AAC

Professor Stephen Hawking, after his tracheotomy in 1985, was still able to give lectures, thanks to AAC. When my step-father lost his facility to speak coherently after a brain operation, my mum created a book of photos and high frequency symbols which we used in order to facilitate conversations. In this way, technology has been enabling those for whom language might be lost to remain in contact with society and to contribute as equals. Because language is perhaps the greatest tool at our disposal for constructing our identities, the importance of this work is clearly paramount.

AAC as utilised by those who need assistance to speak is generally much slower than the rate of normal verbal communication. However, what happens when people with normal speech utilise augmented communication techniques to enhance their face-to-face conversations? Before I discuss this, let us take a look at how augmented communication appears in everyday life in developed countries where many people now carry smartphones and other networked devices in their pockets.

Sound Familiar?

Imagine you are meeting with a new friend. You know this person only a little, but you have clicked and are sharing a social drink or meal together to get to know each other a little more personally. You begin to talk about your family. Perhaps as little as a decade ago, you might naturally have taken a photograph out of your wallet or purse and shown the friend a picture. These days, it would be much more natural to simply show a selection of photos on your phone. However, this is just the beginning. As the conversation progresses, you realise you have a mutual friend who neither of you has seen for quite some time. You wonder what this friend is doing these days. Again, without really thinking anything of it, you reach for your phone and begin to google the mutual friend. You can quickly and easily find out what they are up to now, and use this new information to build a new direction for the face-to-face interaction you are having. It would not even be beyond the realms of possibility to add the person to the conversation via a video call, although in my experience so far this has not become common practice, yet.

Friends become “friends” on social media during a face-to-face meeting

Further examples of augmented communication might be googling a fact in order to maintain a conversation about which you know only a little, ‘checking in’ to a bar or restaurant and uploading pictures of a gathering to social networking sites so that mutual friends (and people you barely know) can ‘like’ and comment asynchronously, and of course adding visual aids to conversations in order to more effectively communicate your message or add emphasis to a point. Do any of these things sound familiar?

An area for study?

Of course, this type of interaction has arisen very naturally, albeit very recently, so why should it be of interest to linguists or worthy of study? What interests me is the way that augmented communication impacts our conversations. In particular, I am curious about how this practice will contribute to the already burgeoning practice of sharing memes. Aaron Lynch, in his fascinating book Thought Contagion, discusses how memes are spreading across the developed world in a fashion akin to a deadly virus, and that the most extreme versions of these memes tend to be the ones people abide to most. Looking at the rise of Donald Trump’s Doomsday rhetoric during the US Presidential race, one cannot help but feel that this may indeed be the case.

Memes might become part of face-to-face communication thanks to augmented communication

Interestingly, there seems to be little interest in the way handheld networked technology can enhance face-to-face communication at this point. Most of the research seems to focus on the damaging effect this technology has. In particular, the depersonalising effect of technology and the way that smartphones and the like have reduced our ability to connect in real time. I would imagine I am not alone in having found it very offensive when someone snubs me in favour of their phone. It is hurtful to be side-lined from a personal interaction by an electronic one. This has been the feature of many discussions, one famous example is the viral YouTube video Look Up, which contains a poem by Gary Turk, lamenting people for being constantly glued to their phones. With over 60 million views, this video seems to have struck a chord with the online community, causing it to be shared and viewed many times over. And, ironically (as more than one YouTube commenter has also wryly noted) quite often this sharing would be taking place on smartphones. So, effectively people are using their phones and are quite aware of the anti-social implications, yet they continue to do so anyway. Perhaps the phone is simply a more convenient way of escaping unwanted conversations, or perhaps the devices have some addictive properties that make people act in a rude way during face-to-face conversation.

Another criticism of the smartphone is linked to the phenomena of dumbwalking, or walking whilst on your phone. In Tokyo where I live and work, there has been a marked increase in the number of public warnings and announcements regarding this issue, and in Antwerp, Belgium there are even now ‘texting lanes’ where those who wish to walk whilst looking at their screen can shuffle along at a slower pace, leaving room for commuters and others who can walk in what is effectively the fast lane. China has also followed suit with these phone lanes.

Special lanes for those who want to walk and text

Although this does not affect face-to-face communication per se, it does illustrate how people seem to find it hard to know when it is appropriate to use their phones. Some people have even put themselves and others at physical risk, such is the allure of the Black Mirror, as it has been dubbed by British satirist, Charlie Brooker. A prime example of this would be the recent wave of accidents related to Pokémon Go. If people cannot even walk down the street without having an accident as a result of their phone, it does not bode well for face-to-face communication, which involves a personal engagement with other individuals or groups, and the constant restructuring of self which is necessary to move between various social situations. Quite simply, if you are on your phone when other people are around and talking, you are likely to be isolating yourself from the group and creating issues that will make it harder to socialise in the future.

this is uncharted territory

However, returning to the idea of augmented communication, how much does having a phone in your hand really alter the conversation you would have anyway? Is this really such a big deal?

I would answer yes, simply because this is uncharted territory. At this point, we do not know how much more convincing someone is if they use a networked device to augment their speech with visuals and to fill in holes in their knowledge with search engines. For example, if a door-to-door salesperson or a Jehovah’s Witness used augmented communication, would they be able to convince more people? Certainly, we know from a number of studies that image-based advertising can convince people to buy things they do not need en masse, something which Arran Stibbe calls ‘damaging discourse’ and links to ecological destruction in his book Ecolinguistics. Is it possible that people using augmented communication, for example to check facts on the fly during talk, will fail to apply their usual critical filter to the information source, leading to conversations which are not backed up by balanced evidence. Is it a good thing to be able to talk about a subject whilst we are simultaneously learning about it from our phone screens? The need for critical thinking whilst selecting information during a conversation is more acute with the rise of fake news.

One example of how much superior a person’s access to knowledge is when armed with a smartphone is demonstrated by the security measures put in place during standardised exams to ensure nobody is cheating. Phones are strictly prohibited from the exam hall, and with very good reason. It seems quite likely that, if during a debate one group of people had smartphones and the other group could rely only on their wits and existing knowledge, the technologically advanced group would win. This leads to a larger issue, that of global competition. Economies and companies with higher technology are already out-competing less well-equipped ones. Are we going to find that, on a day-to-day and face-to-face scale this type of digital divide is going to affect the structure of our social hierarchies? Will people who cannot afford a smartphone be left out of the conversation? All of these questions beg more informed answers.

Although augmented communication is something in need of deeper understanding, it is certainly something that we can all easily observe. I hope that, now we have a name for this phenomenon and a brief outline of how it might impact us, we can begin to look more closely at the implications and start to look up from our phones a little more.

Whilst it is certainly a good thing that speakers can empower themselves and their conversations through augmented communication, it does raise certain other issues. Who is going to be left behind in these technologically charged conversations? Will it lead to a decrease in our working memory? Will using a phone to augment a conversation become the norm, or has it already become the norm in many situations? My research into augmented communication has so far been ethnographic and small-scale. It would be good to augment this research with other viewpoints.

--

--

R.S. Pinner

I’m an Associate Professor in Literature and Applied Linguistics, and author of several books. Interested in digital media, language and stories.