Natural Language Processing

By Kathy Xing

Graphic Credit: Getty Images

Natural language processing (NLP) is a branch of artificial intelligence that broadly focuses on interactions between human language and computers. It has a broad goal of enabling computers to understand and derive meaning from natural language—the way humans communicate—in a smart and useful way. 

According to Forbes, NLP first arose as machine translation in the 1950s as was meant to help with code-breaking. In the 1960s, language programs like SHRDLU successfully enabled user interaction in a block game where the computer would respond to requests to move or manipulate blocks. ELIZA, the first chatbox, was also developed during this time. Up until the 1980s, hand-written rules and parameters guided NLP, but the introduction of machine learning algorithms and statistical NLP enabled the shift to NLP as we know it today.

In modern times, NLP combines artificial intelligence with computational linguistics and computer science to analyze human language. According to Investopedia, this can be broken down into a series of tasks. The computer first needs to understand the language received with a built-in statistical model that breaks speech down into tiny units in order to statistically find the most likely words and sentences that were said. Then, the computer defines words grammatically as nouns, verbs, etc. with coded lexicon rules. After these first two steps, the computer is able to gain a general idea as to what was said. Finally, the computer programming language must be converted to an audible or textual response from the user input.

Currently, NLP has countless applications and is used in predictive word suggestions on mobile devices and Google searches as well as voice-activated assistants like Siri. Millions of people are incorporating smart speakers like Alexa into their homes; this technology is entirely based around NLP as it intakes voice commands and uses algorithms to decipher the meaning and provide an appropriate response. More sophisticated chatbots that help answer customer questions are also on the rise. In fact, according to a survey from Oracle Corporation, 80 percent of sales and marketing leaders have implemented or plan to implement chatbots in order to better serve customers. 

While much progress has been made on NLP since the 1950s, there are still various difficulties. Human language in and of itself is ambiguous, and this ambiguity is the cause of most difficulties with NLP. For example, at the word level, it is challenging for a computer to distinguish whether “board” is being used as a noun or a verb, and at the sentence level, it can be difficult to understand whether “He lifted the beetle with a red cap” means that the beetle was moved with a red cap or if the beetle that was lifted had a red cap. As a result, even though many prominent companies are improving upon NLP and have even created successful products based around NLP, there still has not been success in creating a holistic cognitive platform that understands human language at the level of an actual human.

Sources:
Forbes
(https://www.forbes.com/sites/forbestechcouncil/2018/11/06/the-evolution-of-natural-language-processing-and-its-impact-on-ai/#63f325761119)
Investopedia
(https://www.investopedia.com/terms/n/natural-language-processing-nlp.asp)
Oracle Corporation
(https://www.oracle.com/webfolder/s/delivery_production/docs/FY16h1/doc35/CXResearchVirtualExperiences.pdf)