Could physical language be the key to unlocking the full potential of global collaboration?

And the Lord said, "Look, they are one people, and they have all one language, and this is only the beginning of what they will do; nothing that they propose to do will now be impossible for them. Let us go down, and there confound their language, that they may not understand one another's speech.” - Genesis 11:6-7

From nomadic tribes to agrarian cultures to nation states, the power of speech has helped humans become the dominant species on the planet. But it has also kept us apart. Ever since Genesis, when God recognized the infinite power of global collaboration - and didn’t much like where it was heading.

Is it any surprise that we’re still feeling the aftereffects at work? Language is a facilitator, but it can also be a barrier that gets in the way of great ideas. A barrier that, until recently, felt insurmountable.

But that might be changing. Modern technology in the form of AI, machine learning, and raw computing power is bringing a world of universal understanding slowly into view. And, with all due respect to Douglas Adams, you don’t even need to put a fish in your ear.

With so many technical breakthroughs preparing to launch humanity into a new era of autonomous, universal communication, it is probably worth investigating why the great wall of words has stood for so long. Where better to investigate this than the polylingual hub of business, Hong Kong?

Dr. Wong Man Ho, known to her colleagues as ‘Ivy,’ is the Assistant Professor of Linguistics and Director of General Education at Hong Kong Shue Yan University. In her offices overlooking Victoria Harbour, Ivy has spent much of her academic career investigating the relationship between visualisation and language acquisition. Ivy is beaming with enthusiasm as she begins to explain her approach to linguistics, translation, and research in a polylingual environment.

So do the text-based translation tools, now available on every smartphone, present a viable solution for collaboration in a multilingual operating environment? Ivy continues to track the progress of such tools, regularly putting them to the test with complex text samples. “It’s actually really impressive how they translate. Every few months I feed a different poem in and the performance improves. For translating factual things, it is easy and quite accurate.”

She has no fear, however, of these tools rendering language learning obsolete. “What is amazing about the human mind is that we don't need that much data. We are not computers. We can meet 100 people to increase collocational awareness [our understanding of grammatical word arrangement]. I don't have to train with five-million sentences in order to know the best collocation.”

At this juncture, with a pressing community engagement across the bay, the interview transitions from the Shue Yan University campus to the comfort of a red Hong Kong taxi. Ivy elaborates upon some of the limitations of these text-based translation tools.

“When my friends and I were in Korea, the cab drivers didn't speak English. They expect everybody to speak Korean. So we had to use automated text translation to communicate. We tried to say that we were sorry that we didn’t speak the language. We used the tool to get our factual message across, but we had to try and compensate with our smiles, gestures, and body language.”

As the taxi descends Braemar Hill, a mere 200m above sea level, it passes Hong Kong Stadium. While social interactions may be hampered by the mechanical nature of automated text translation, is it not also the case that the in-person translations of so many post-match interviews at sporting events fail to capture the passions, the agonies, and the ecstasies of our sporting idols?

“The translator’s gestures are often a little bit stiff, because they are really thinking, ‘I have to make sure this is accurate,’ rather than a more natural method of translation. Usually, if translators cannot find a way to translate a more culturally constrained meaning, they would remove it and just translate the facts.”

The conversation is briefly interrupted by an audible protestation from the cab driver as he ‘negotiates’ with other road users, funnelling into the correct lane for the tunnel under Victoria Harbour. Whilst the gesticulations of a frustrated cab driver may be universally understood, within the context of the global office, are the physical aspects of communication more culturally contextual? Not necessarily, says Ivy. “Physicality is universal because we all have these sensory motors and all of these contribute to meaning making.”

Having emerged from the tunnel, the journey continues on the north side of Victoria Harbour. Ivy further illustrates the natural relationship between physicality and communication, pointing out the window as the cab passes the world-renowned Yuen Po Street Bird Garden. “With other species, like songbirds, if you play a song through a speaker, they have difficulty learning the song, but if you cover up the speaker with a wooden songbird model, they can actually pick it up better.”

Visualization and physicality are a core aspect of Ivy’s research. Initially, her work was limited to two dimensions, developing new techniques for language acquisition supported by diagrammatic visual aids. “I realised there are demographic variations among native speakers as well as regional variations. It’s not just a matter of grammatical rules. It’s about how you conceptualise the scene. With the pandemic, there were social-distancing rules, so we moved everything to computer-assisted training, and we tried to use animated schematic diagrams with text to explain.”

Ivy’s research into the power of visually reinforced linguistics has evolved since then, graduating to the third dimension. “I'm developing a virtual reality scene in a supermarket, where you can practise using classifiers within the simulated environment.” This implementation of VR includes animated characters, where multilingual communication skills can be practised autonomously, harnessing the many benefits of more immersive training platforms.

The cab ride concludes, arriving at Fa Hui Park on a bright spring evening; commuters, school children, and families all enjoying the vibrant community gardens. According to Ivy, much like the songbird, human beings also benefit from a bond between language acquisition and three-dimensional space. “Babies actually learn more effectively when they are looking at real people. If you just show them a video, it’s two dimensional, the effectiveness decreases substantially.”

In this age of global work, remote communication and collaboration are here to stay. AI enhancements, such as real-time translation, are fast becoming a viable solution and will, of course, improve as wider deployment generates greater data sets for further nuanced machine learning. So, how can Ivy’s research be leveraged for greater understanding of globally distributed workforces? “For my experiments, I don't want to be biased. I want to remove that human element. But in workshops with students that I have been working with for months, gestures and facial expressions are very important. It’s about the nonverbal cues. These relationships become very powerful.”

Translation tools will never replace the essential, natural human component of communication. However, with only 9% of businesses hiring talent in other countries, there are certainly improvements that can be made to the existing two-dimensional infrastructure of video conferencing if organisations are to harness the full potential of the global office.

Supporting individual workers’ own nonverbal communication skills can lead to improved understanding between colleagues. Combining instant translation technology with a three-dimensional platform has the potential to liberate contextual gesticulations currently obscured by the video frame, empowering users with the freedom to collaborate better wherever they are in whatever language they are speaking.