There is about 1.1 zettabytes (nearly one billion gigabytes) flow of data over the internet, globally, every month. And this is to say, just the beginning of the digital era.
The distribution of this digital world is, well quite similar to ours, 70% textual data to 30% audio/visual data. Clearly, it isn’t humanly possible to catalog this argentinosaurus of knowledge. Hence, you let a machine do a machine’s job!
Machine learning (here on referred to as ML) is the field of computer science that, to describe it in the most non-technical lingo, deals with the computer learning theory and pattern recognition. And Natural Language Processing is a further sub-division of ML, which as the name suggests processes language. And agree it or not, that 70% of textual data isn’t going anywhere anytime soon.
Wikipedia defines Natural Language Processing (here on referred to as NLP) as a field of computer science, artificial intelligence, and computational linguistics concerned with the interactions between computers and human (natural) languages. As such, NLP is related to the area of human-computer interaction.
So what does an individual understand by the term NLP? – A machine that can understand a natural language like humans.
And what should, an individual understand by that term? – The use and ability of systems to understand natural language such as English, rather a specialized computer language such as Python
Yes, you going “oooh” right now would totally make sense. To explain it in the most understandable way, a machine works on binary – set of 0’s and 1’s. Any language that is understood by humans, even the programming languages, are understood and processed by a machine only after converting it into sets of 0’s and 1’s – the only language, understood by a machine. This is exactly where NLP is different and unique. It tries to make the machine understand our letters, words, and sentences. It tries to teach the machine, the human language as it is.
So how is this actually done? As all science fanatic students believe, everything, from the life on earth to serendipity, to even chaos can be represented by mathematical equations. The NLP magic works pretty much on the same principal. The working of NLP or a machine that understands human language is also derived through Math.
The (machine) learning of this method is precisely like a child learning A-B-C. The machine is made to feed on huge amounts of data, the machine then assigns a unique token value to each of the words and creates a pattern of its own to remember every word and its usage. Yes, exactly the way we remember our school books, by creating a pattern unique to us. The more data it feeds on, the better it fine-tunes its pattern recognition – up to a point where it understands as well as generates human languages like we do.
The patterns are identified by the machine on its own. And it can keep on evolving it, keep on making it better. That is the beauty.
The search engines we use every day to correct our spellings, the social media platforms we use every other minute to pass our time, the voice recognition apps that help us multi-task are the very basics of what NLP does. The ultimate hero of automation – Jarvis, is what NLP is capable of and more.
In this world of conventions, there are many that are doing some amazing work in this field, busting all myths. The much-hyped jargon – big data and automation, which are doing rounds around the market these days are all a part of this small magical community – Machine Learning.