powerful, and full of rich nuance. It's a tough task to build a computer than can “hear” spoken natural human language, break it down into its component parts, infer meaning and intent, and then act on that information to do something useful. Artificial intelligence is behind many incredible applications of natural language processing. It can find spelling and grammatical errors, translate from one language to another, review legal contracts, and summarize the important points of a report.
Natural language processing (NLP) is an umbrella term that describes a machine's ability to understand, process, and communicate using natural human language. NLP consists of two pieces: natural language understanding (NLU) and natural language generation (NLG). You can think of one as being language input and the other as language output. In more technical terms, NLU converts unstructured human language data to structured data that a computer can understand while NLG converts structured data to unstructured data in the form of human language. Artificial intelligence is essential to both NLU and NLG. Natural language processing sits at the heart of voice interfaces, language translation services, email sentiment analysis, and many other applications that involve human language.
Natural Language Understanding (NLU)
Computers that understand human language perform many useful business tasks. Sentiment detection assesses text to determine if it conveys positive or negative emotion, for example, to highlight positive online product reviews or surface negative customer emails that require a swift response. NLU is also used to detect profanity, hate speech, threats, abuse, and other conversation that may be deemed inappropriate.
Natural language understanding is used to analyze documents and provide decision support. Scriptbook, a startup from Antwerp, Belgium, reviews movie screenplays to predict their likely box office failure or success. The software helps studios to make greenlight decisions on scripts. Scriptbook analyzed the scripts of 62 movies released in 2015 and 2016. Thirty of these movies were box office successes; 32 were failures and lost money. Scriptbook's AI correctly predicted all 30 of the box office hits and correctly called 22 of the movies that were duds. With 52 correct calls made on 62 movies, the AI scored far better than the Hollywood moguls had. Scriptbook also uses NLU to predict a film's likely MPAA rating, the likeability of characters, and the countries where a movie will find most success, all based on its script.
LawGeex, an Israeli company, uses natural language processing to automate the review of legal contracts and nondisclosure agreements. LawGeex challenged 20 U.S.-trained lawyers to identify legal issues in five real-life nondisclosure agreements (NDAs), faster than their AI. The test was overseen by an independent lawyer and performed with input from legal experts and law professors. The lawyers took an average of 92 minutes to review all five NDAs and achieved an average accuracy rate of 85%. The LawGeex AI scored an accuracy of 94%, equal to the best lawyer's score, and completed the entire task in just 26 seconds. Lawyers that I've told about this AI are generally delighted. Reviewing NDAs is not a favorite part of their work and they are excited to offload routine tasks and focus more time on higher value, higher revenue work.
Natural language understanding speeds data entry. AIs identify email addresses, physical addresses, dates of birth, and phone numbers from nonstandard forms, automatically. This technology, sometimes combined with handwriting recognition (thanks again, AI!), makes short work of data entry.
Voice recognition combines speech-to-text capabilities with NLU. This technology automatically creates subtitles for videos and presentations. Microsoft now includes this capability with some versions of its PowerPoint application.
Natural Language Generation (NLG)
Computers with the ability to write or speak in natural human language are a huge breakthrough. AIs generate language either from source data or from source text. For example, an auto-generated weather report is created from weather forecast data, while a translation from one language to another is performed based on source text.
Natural language generation (NLG) has many valuable business applications. Language translation is an obvious one. Another is the automatic creation of summaries and abstracts. These might be summaries of financial reports, legal documents, operations reports, performance reviews, news articles, or medical records. NLG is combined with image recognition to automatically caption or describe images. This is a valuable function for people with visual impairment and also improves the quality of image searches.
In March 2014, the Los Angeles Times published a short story on an earthquake in Beverly Hills, California. The article described the location, time, and strength of the earthquake (4.4 on the Richter scale, if you're interested) and posted it to the L.A. Times website within three minutes of the earthquake. The article was written by a simple piece of software that sourced seismic data from the U.S. Geological Survey. This simple automation populated data into a prewritten template but demonstrates the power of automation. In a world of 24-hour news cycles and shrinking ad revenue, NLG frees human reporters to focus on higher-value stories and investigative journalism. More advanced automation now writes weather and traffic reports, summarizes business results, and covers sports events. Wordsmith, an automated reporting platform used by the Associated Press (AP), uses NLG to generate stories about minor league baseball games, college basketball, and quarterly corporate earnings reports. The AP claims that Wordsmith produces more than 4,400 corporate recaps each quarter, more than 15 times the number it could previously handle using a human writing staff. Wordsmith offers a glimpse of the more sophisticated NLG capabilities coming in the near future.
Simplish uses AI to convert complex text, with a vocabulary of more than 100,000 words, into simpler text with a vocabulary of less than 2,000 words. NLG will make complex language accessible to broader audiences with lower levels of education, for example, an academic text translated for lay people or children. Future NLP applications might provide comprehensive copyediting services or disrupt the businesses of companies like CliffsNotes.
Quillionz uses AI to automatically generate questions, quizzes, and assessments from a body of text. This provides an amazing aid for teachers, who can guide the AI to focus on particular keyword topics and automatically create multiple-choice questions.
There is a dark side to NLG technology. If we think that online fake news is a problem now, just wait until next-generation NLG is unleashed by the Russian troll farms. A human troll farm might create a hundred fake “news” articles in a day; a weaponized AI could pump out a million pieces of disinformation in a single hour. OpenAI claims to have created NLG software that writes such high-quality text that they have chosen to withhold its release, based on such concerns.
A 2015 survey of 352 leading AI experts by the University of Oxford predicted that AI will translate languages better than human translators by 2024, will write high-school-level essays by 2026, but won't write a best-selling novel until 2049. It's a giant leap from teenage essays to Tolkein, but the direction is clear. While we are a long way from the first Pulitzer Prize–winning novel written by AI, sophisticated NLG technology will soon create complex documents that rival human output.
Voice Agents
Voice agents like Apple Siri, Microsoft Cortana, Amazon Alexa, Samsung Bixby, and Google Assistant improve significantly every year. Some add new capabilities almost every week. Google claims their voice assistant is now available on more than a billion devices. These “conversational computing” or “digital dialogue” platforms will reshape the way we work and become an increasingly important part of our lives. Paradoxically, voice agents may become both our managers, and our subordinates, guiding our actions and performing our errands.
Voice interfaces are an important component of hands-free computing and a natural complement to augmented reality. Hands-free technology offers the prospect of “computing for the rest of us” and voice interfaces
