Nexus Learn

Lecture 30: Natural Language Processing Basics

Lecture Slides CS-511 Artificial Intelligence 7 min read Updated 8 Nov 2026

↓ Download slides (PowerPoint)

Recap of Lecture 29 and Homework Check

LAST LECTURE · 5 MIN

Today: a new kind of data - words. How does Gmail know a message is spam, or a shop know its reviews are angry?

Today's Plan (50 minutes)

ROADMAP

0-5

Recap of Lecture 29 and homework check

S02

5-8

Plan and learning outcomes

S03-S04

8-15

Module 1: What NLP is and why language is hard

S05-S07

15-27

Module 2: Turning text into numbers

S08-S11

27-35

Module 3: Sentiment analysis and language models

S12-S14

35-43

Class activity: text to numbers by hand

S15

43-48

Common mistakes and recap quiz

S16-S17

48-50

Homework and next lecture

S18

Learning Outcomes (SLOs)

BY THE END OF THIS LECTURE YOU WILL BE ABLE TO

SLO 30.1

Explain what natural language processing is, name its main tasks and applications, and explain why language is hard for computers

→ CLO1 · C2 Understand

SLO 30.2

Turn short texts into numbers by hand: tokens, stop words, a bag of words and TF-IDF

→ CLO1 · C3 Apply

SLO 30.3

Explain how naive Bayes classifies the sentiment of a review, and how modern language models such as ChatGPT go beyond word counts

→ CLO1 · C2 Understand

  • Course Learning Outcome(s) served
  • CLO1: Explain the applications of artificial intelligence and the agent-based approach to AI.

Slide 5

MODULE 1

What Is NLP?

Computers and human language

Language Tasks All Around Us

MODULE 1 · APPLICATIONS

Classification

Translation

Speech to text

Chatbots

Spam filter; angry or happy review.

Urdu to English and back.

Urdu voice typing; voice notes.

Bank helpline; ChatGPT, Gemini.

Natural language = a language people speak (Urdu, English, Punjabi) - not a programming language.

Why Language Is Hard for Computers

MODULE 1 · AMBIGUITY

Ambiguity: one sentence, more than one meaning. People use context and world knowledge; a computer has to learn them from data.

Slide 8

MODULE 2

Turning Text into Numbers

Models need numbers, not words

Step 1: Tokens and Stop Words

MODULE 2 · PRE-PROCESSING

REAL OUTPUT: L30/TOKENS.PY

  • Sentence : The new mobile is very good, but the battery is NOT good!
  • Tokens : ['the', 'new', 'mobile', 'is', 'very', 'good', 'but', 'the', 'battery', 'is', 'not', 'good'] (12)
  • Our stop words removed: ['new', 'mobile', 'good', 'battery', 'not', 'good']
  • scikit-learn list removed: ['new', 'mobile', 'good', 'battery', 'good'] ('not' is in its list!)
  • Roman Urdu, same meaning: [['khana', 'bohat', 'acha', 'tha'], ['khana', 'bht', 'achha', 'tha']]

Careful: a ready-made stop-word list removed 'not' - and 'NOT good' became 'good'. Keep negation words for sentiment.

Step 2: Bag of Words

MODULE 2 · COUNTING WORDS

REAL OUTPUT: L30/BOW.PY

  • Reviews (stop words 'was', 'and' removed):
  • R1: food good service good
  • R2: food cold service slow
  • R3: good biryani good price
  • Bag of words (counts)
  • word R1 R2 R3 df idf
  • biryani 0 0 1 1 0.477
  • cold 0 1 0 1 0.477
  • food 1 1 0 2 0.176
  • good 2 0 2 2 0.176
  • price 0 0 1 1 0.477
  • service 1 1 0 2 0.176
  • slow 0 1 0 1 0.477

Vocabulary

Every different word in all the reviews: 7 here.

One row per text

R1 = 0 0 1 2 0 1 0 - a vector of counts.

Order is lost

'dog bites man' = 'man bites dog'.

Picture: tip the words of a review into a bag and count them.

Step 3: TF-IDF - Which Words Matter?

MODULE 2 · WEIGHTING WORDS

FORMULA (N TEXTS, DF = TEXTS WITH THE WORD)

BOARD

  • TF = count of the word in this text
  • IDF = log10( N / df )
  • TF-IDF = TF x IDF
  • 'good': df = 2, IDF = log10(3/2) = 0.176
  • R1: 2 x 0.176 = 0.352
  • 'biryani': df = 1, IDF = log10(3/1) = 0.477

REAL OUTPUT: L30/BOW.PY

  • TF-IDF of R1 (TF x IDF):
  • food 1 x 0.176 = 0.176
  • good 2 x 0.176 = 0.352
  • service 1 x 0.176 = 0.176
  • TF-IDF of R3: biryani 0.477, good 0.352, price 0.477

Idea: a word that appears everywhere tells us nothing; a rare word tells us a lot about the text.

Slide 12

MODULE 3

Sentiment and Language Models

Is this review happy or angry?

Sentiment Analysis with Naive Bayes

MODULE 3 · TEXT CLASSIFICATION

REAL OUTPUT: L30/SENTIMENT.PY (MADE-UP PRACTICE REVIEWS)

  • Training: 8 positive, 8 negative reviews; vocabulary 41; positive words 36, negative words 34
  • P('khana'|pos) = (1+1)/(36+41) = 0.0260 P('khana'|neg) = (1+1)/(34+41) = 0.0267
  • P('acha'|pos) = (2+1)/(36+41) = 0.0390 P('acha'|neg) = (0+1)/(34+41) = 0.0133
  • P('tha'|pos) = (1+1)/(36+41) = 0.0260 P('tha'|neg) = (1+1)/(34+41) = 0.0267
  • P(positive | 'khana acha tha') = 0.735
  • scikit-learn, single words (41 features):
  • khana acha tha P(positive) = 0.735 -> positive
  • service was slow and food cold P(positive) = 0.243 -> negative
  • food was not good P(positive) = 0.458 -> negative
  • biryani was not bad P(positive) = 0.070 -> negative
  • scikit-learn, words + word pairs (92 features):
  • khana acha tha P(positive) = 0.844 -> positive
  • service was slow and food cold P(positive) = 0.238 -> negative
  • food was not good P(positive) = 0.211 -> negative
  • biryani was not bad P(positive) = 0.024 -> negative

Word pairs help: 'food was not good' 0.458 -> 0.211 once 'not good' is a feature.

Still wrong: 'biryani was not bad' is a good review, but both models say negative - 'not bad' was never seen.

From Word Counts to ChatGPT

MODULE 3 · LANGUAGE MODELS

Language model: predicts the next word. 'Pakistan ne match ...' -> 'jeet liya'. Trained on huge text, it learns grammar, facts and 'not bad'.

Urdu is a low-resource language (less text online) - Urdu NLP is an open research area.

Activity: Text to Numbers by Hand

CLASS ACTIVITY · PAIRS · 8 MIN

Pairs, 6 min + 2 min check. Lower case, remove punctuation, split on spaces.

Tokenise 'Match was GREAT, Pakistan won!' How many tokens?

3 texts. IDF of a word in 1 text? In all 3 texts?

Remove the stop word 'was'. How many tokens are left?

Name the task: Urdu voice typing; Gmail spam folder; Google Translate.

Bag of words for S1 'pakistan won match' and S2 'india won match match'.

Why is it dangerous to remove 'not' before sentiment analysis?

Common Mistakes

WATCH OUT

WRONG IDEA

CORRECT IDEA

✗ Computers understand words like we do.

✓ They work with numbers made from words (counts, TF-IDF, embeddings).

✗ Always remove every stop word.

✓ Keep negation words ('not', 'nahi') for sentiment.

✗ Bag of words keeps the word order.

✓ Order is lost: 'dog bites man' = 'man bites dog'.

✗ A high count always means an important word.

✓ Words in every text get IDF 0 - TF-IDF fixes this.

✗ ChatGPT looks words up in a dictionary.

✓ It is a language model that predicts the next word.

Recap Quiz

CHECK YOUR UNDERSTANDING · 3 MIN

  • Q1. Splitting text into words is called:
  • (a) Stemming (b) Tokenisation (c) Translation (d) Crossover
  • Q2. 5 texts; a word appears in all 5. Its IDF is:
  • (a) 1 (b) 5 (c) 0.699 (d) 0
  • Q3. 'food good good' with vocabulary (bad, food, good) as a bag of words:
  • (a) 0 1 1 (b) 1 1 2 (c) 0 1 2 (d) 2 1 0
  • Q4. Deciding whether a review is happy or angry is:
  • (a) Sentiment analysis (b) Machine translation (c) Speech recognition (d) Question answering

Homework and Next Lecture

BEFORE THE NEXT CLASS

  • Task 1: Three SMS: 'free prize win', 'win the match', 'free tickets free'. Make the bag-of-words table with df and IDF = log10(3 / df) for every word.
  • Task 2: Find the TF-IDF of 'free' in SMS 3, 'win' in SMS 1 and 'tickets' in SMS 3. Which word best describes SMS 3?
  • Task 3: Review 'bura khana'. Positive words 36, negative words 34, vocabulary 41. 'bura' 0 times positive, 1 negative; 'khana' 1 and 1. Find P(positive) with +1 smoothing and equal priors.

NEXT LECTURE

  • Lecture 31:
  • Machine Vision

← All CS-511 material