Natural language processing with Python

Last updated on March 25, 2021 by Enrique Bruzual

As humans, we use natural language to communicate through different mediums. Natural Language Processing (NLP) is generally known as the computational processing of language used in everyday communication by humans. NLP has a general scope definition, as the field is broad and continues to evolve.

NLP has been around since the 1950s, starting with automatic translation experiments. Back then, researchers predicted that there would be complete computational translation in a 3 to 5 years time frame, but due to the lack of computer power, the time-frame went unfulfilled.

NLP has continued to evolve, and most recently, with the assistance of Machine Learning tools, increased computational power and big data, we have seen rapid development and implementation of NLP tasks. Nowadays many commercial products use NLP. Its real-world uses range from auto-completion in smartphones, personal assistants, search engines, voice-activated GPS systems, and the list goes on.

Python has become the most preferred language for NLP because of its great library ecosystem, platform independence, and ease of use. Especially its extensive NLP library catalog has made Python more accessible to developers, enabling them to research the field and create new NLP tools to share with the open-source community.

In the following, let's find out what are the common real-world uses of NLP and what open-source Python tools and libraries are available for the NLP tasks.

Open-Source Python Projects for Natural Language Processing

Optical Character Recognition (OCR)

OCR is the conversion of analog text into its digital form. By digitally scanning an analog version of any text, OCR software can detect the rasterized text, isolate it and finally match every character to its digital counterpart. OpenCV-python and Pytesseract are two major Python libraries commonly used for OCR. These are Python bindings for OpenCV and Tesseract, respectively. OpenCV is an open-source library of computer vision and machine learning, while Tesseract is an open-source OCR engine by Google.

Real-world use cases of OCR are license plate reader, where a license plate is identified and isolated from a photo image, and the OCR task is performed to extract license number. A single-board computer, such as the Raspberry Pi loaded with a camera module and the OCR software, makes it a viable testing platform.

Speech to Text (STT)

Speech recognition is the task of converting digitized voice recordings into text. The more effective systems use Machine Learning to train models and have new recordings compare against them to increase their accuracy. SpeechRecognition is a Python library for performing speech recognition online or offline. It supports multiple recognition engines such as CMU Sphinx, Google Cloud Speech, Microsoft Bing Voice Recognition, etc.

Text-to-Speech (TTS)

Text-to-Speech is an artificially generated voice able to speak text in real-time. Some synthesized voices available today are very close to human speech. Text-to-Speech software integrates accents, intonations, exclamation, and nuances allowing digital voices to closely approximate human speech. Several Python libraries are available for TTS. Pyttsx3 is a TTS library that performs text-to-speed conversion offline. gTTS is a Python library that performs TTS with Google Translate's text-to-speech API. TTS is a text-to-speech library that is driven by the state-of-the-art deep learning models.

Sentiment Analysis

NLP can extract the sentiment polarity and objectivity of a given sentence or phrase by implementing the subtasks mentioned above with other specialized algorithms. Sentiment analysis classifies the tone of a particular text as positive or negative, as well as the level of subjectivity. Gauging people's opinions on social media using sentiment analysis is a common practice for product reviews. The best-known Python library for sentiment analysis is NLTK (Natural Language Toolkit), which is a powerful NLP platform that offers a range of text processing capabilities including semantic reasoning. Several Python implementations are available (e.g., twitter-sentiment-analysis, pytorch-sentment-analysis).

Document Classification

Document classification is a generalization of sentiment analysis, where the goal is to label documents with one of N categories based on their content. In general, documents may contain a mix of text, images and videos, but in the context of NLP, they are primarily text-based. Supervised deep learning is the proven technology for this type of task that requires complex semantic analysis. The Python-based machine learning frameworks such as Scikit-learn, TensorFlow, Keras, Pytorch, combined with NumPy math library are the go-to solution for document classification. Real-world use cases of document classification is spam detection filter, where the goal is to classify email content as spam or non-spam. A number of Python projects are available on this topic.

Conversational Chatbot

Chatbots are very common nowadays, as they can help automate customer service and minimize company costs. Chatbots integrated with services such as WhatsApp, allowing customers to interact with automated customer service. The customer can ask questions, schedule appointments using natural language, and the chatbot would respond appropriately.

Twilio offers a service allowing programmers to integrate a chatbot with phone services giving access to platforms such as WhatsApp, and Alexa. Thus a programmer can write a chatbot using Python (e.g., ChatterBot, Chatbot), deploying it to the cloud, and finally connecting it to WhatsApp and Alexa through the Twilio API.

Sign Language Recognition

A sign language translator application can assist with the communication process among those who are sign language literate and those who are not. The application could have a meaningful impact as there are millions of deaf people around the world.

With the help of Python libraries such as OpenCV, TensorFlow and Keras, developers can write applications capable of processing live video. Detect hand signs and translate them into any language.

Roku Voice-Activated Remote

Roku is a digital media player offering access to a variety of streaming media services. Roku gives access to an IPA leveraged with the help of the Roku Python library.

Using Python's SpeechRecognition library, a developer can write an application capable of converting a user's utterances into Roku text commands and sending them to the Roku by using the API.

Blog Keyword Generator

Using a library such as TextBlob or NLTK, developers can write an application capable of preprocessing a blog post. Perform a word frequency count and extract the most frequently used words.

Conclusion

Natural Language Processing has been around for a while, but thanks to increased computer power and advances of Machine Learning technology, NLP has seen rapid growth. With a few libraries and a single-board computer, Python is a great language to test NLP ideas and projects.

I hope this article answers questions you may have about Natural Language Processing and perhaps inspire you to try writing some NLP applications.

Support Xmodulo

This website is made possible by minimal ads and your gracious donation via PayPal or credit card

Please note that this article is published by Xmodulo.com under a Creative Commons Attribution-ShareAlike 3.0 Unported License. If you would like to use the whole or any part of this article, you need to cite this web page at Xmodulo.com as the original source.