Did you shoot a man in Reno? I don’t mean to pry.
Words don’t exist in a vacuum. Each word interacts with the words around it. The word games we play as kids are actually us playing with how we can make those interactions funny, or clever. Detecting word play is more than just computational linguistics, or data mining. It requires deep understanding of the various signals encoded in sentences.
Phonetics and Alliteration
How words sound is important to determining if a phrase is meant to be funny, or melodic, or even if the right (vs write) word was chosen.
Education and Technical Level
Whether matching an the message to the audience, or determining the psychographic to target with marketing, the reading level, and the technical difficulty of text matters.
Sentiment and Bias
Determining if someone is truly happy, or just biased is something humans struggle with, but when looked at through the cold unfeeling smarts of NLP this is a task that a computer can do better than a human.
The word “dog” is three letters, but it evokes an emotion, image, and sound when it is said. Language is so much more than the conveyance of three letters from one person to another; it is the conveyance of ideas and shared experience. A common device in action movies with teams of people who have worked together is to say the name of a place, and that be how they are going to beat the bad guy. One word conveys to the other person an entire plan. An investor might say to another, “I think this the next Instagram” even if the startup they are talking about has nothing to do with images. The implication is that it is a fast growing company ready for acquisition at a high valuation. Computers don’t have this shorthand. NLP converts shorthand to long hand and generates data points that Machine Learning or AI can leverage to make decisions. “Hog, Frog, Dog,” and “pig, toad, puppy” are the “same thing” but only if you look at them as concepts not as phonetics. Humor is lost on computers, art is lost on computers, alliteration is lost on computers. It is only through Natural Language Processing that computers can fake that “Natural” ability of humans to see things as more than the sum of the parts. Without a shared collective history, and sense of culture, machines can’t learn to understand people or understand the patterns in the text they create. NLP is a crutch for ML and AI to lean on until they can develop these shared experiences on their own. It is many more things as well, but at its core this is the differentiator that makes Machine Learning, without Natural Language Processing, impossible.
Detectors, Signals, and Meta Data
- Part of Speech Tagging
- Named Entity Recognition
- Sentence Disambiguation
- KeyWord Extraction
- Summarization and Sentence Significance
- Sentiment Analysis
- Alliteration Detection
- Word Sense Disambiguation
- Logistic Regression Scoring
- Tagging for Latent Semantic Indexing
- Tagging for Singular Value Decomposition
- Phonetic Decomposition
- Reading Difficulty Modeling
- Technical Difficulty Modeling
- Spelling Correction
- String Comparison and Plagiarism Detection
- Author Profiling
- Psychographic Modeling
- Fact and Statistic Extraction
- Ism Extraction
- Character Language Modeling