Introduction to NLP

Gil Katz gil@recast.ai @gil_katz

Introduction to NLP

Gil Katz gil@recast.ai @gil_katz

Collaborative end-to-end bot platform

BUILD CONNECT MONITOR TRAIN

1950: Turing Test (“Computing Machinery and Intelligence” / Alan Turing)

Historic Background

50’s-60’s: Machine translation to be reality in three to five years (Georgetown experiment)

60’s-80’s: NLP systems based on hard rules, some quite impressive!

80’s-90’s: Machine learning for NLP, starting with simple algorithms (decision trees)

Historic Background

90’s-2000’s: Increasingly statistical models are used (starting with HMM for POS tagging)

2000’s-today: Neural Networks added to the mix

NLU over translation Written over spoken Chatbots and Recast.AI

This Talk

State of the Art Intents Entities Skills

Intents Subject of communication

Entities Important objects

Skills Take action

greetings

I have no internet 4G doesn’t work No signal at my place

report-issue weather

report-issue greetings

I have no internet 4G doesn’t work No signal at my place

weather

report-issue greetings

I have no internet 4G doesn’t work No signal at my place

weather

report-issue greetings

I have no internet 4G doesn’t work No signal at my place

weather

- Where are you located?

Powered by Machine Learning

Code

ML

data answers

rules

answers

data rules data answers

Supervised Unsupervised

Classification Regression

Preprocessing Be Prepared!

J’ai envie d’une pizza aujourd’hui

J’ai envie d’une pizza aujourd’hui (fr)

J ’ ai envie d ’ une pizza aujourd’hui (fr) PRON

VERB

NOUN

ADP

DET

ADV

J ’ ai envie d ’ une pizza aujourd’hui (fr)

Intent Classification

Naive Bayes

Bayes’ Rule: P(i|w) = P(w|i)P(i)/P(w)

‘Sparse’ approach

L’enfant a mangé un poisson

Un poisson a mangé l’enfant

No context management

Neural Networks

Words are represented densely in an n-dimensional space

Representations are learned through context

Main problem: How to represent sentences?

vct(Berlin) - vct(Germany) + vct(France) = vct(Paris)

SVM

RNNs (LSTM/GRU/QRNN/...)

Bi-LSTM with Attention

What’s more?

Siamese Networks

Named Entity Recognition

Named Entity Recognition Sequence Labeling

I have no in yesterday since internet Nancy

I have no in yesterday since internet Nancy

O O O CONNECTION O LOCATION O DATETIME

? ? ? ? ? ? ? ?

I have no in yesterday since internet Nancy

? ? ? ? ? ? ? ?

I have no in yesterday since internet Nancy

PERSON

LOCATION

? ? ? ? ? ? ? ?

I have no in yesterday since internet Nancy

LOCATION

Hidden Markov Chains

SSVM

CRF

RNN (LSTM/GRU/QRNN/...)

What’s more?

Entity Enrichment

I have no internet in Nancy since yesterday

{

canonical: …

value: …

}

{

formatted: …

lat: …

lng: …

}

{

formatted: …

iso: …

accuracy: …

}

Dialog Management

Markov Decision Process Finite State Architecture Frame Based

User Initiative Greetings

System Initiative Payment

Mixed Initiative QA/Track/Order

Wrap Up

ML and NLP as engine behind chatbots

A chatbot is built on many ML tasks

Different algorithms fit different tasks

Recast.AI as an end-to-end chatbot development platform

Questions?