NLP for SEOs: How to
optimise your
content for BERT
@jcbhaagen
Josephine Haagen
Journey Further
https://www.slideshare.net/
JosephineHaagen
2
What is
NLP?
©2022 Journey
Further. All rights
reserved. All related
logos, assets
messaging are
trademarks of
Journey Further Ltd
How machines
understand language.
3
©2022 Journey
Further. All rights
reserved. All related
logos, assets
messaging are
trademarks of
Journey Further Ltd
4
Or, in SEO terms:
Takes unstructured data
and attempts to
structure it.
©2022 Journey
Further. All rights
reserved. All related
logos, assets
messaging are
trademarks of
Journey Further Ltd
©2022 Journey
Further. All rights
reserved. All related
logos, assets
messaging are
trademarks of
Journey Further Ltd
5
What can NLP
do for you?
6
A content strategy led by NLP
©2022 Journey
Further. All rights
reserved. All related
logos, assets
messaging are
trademarks of
Journey Further Ltd
7
A content strategy led by NLP
©2022 Journey
Further. All rights
reserved. All related
logos, assets
messaging are
trademarks of
Journey Further Ltd
©2022 Journey
Further. All rights
reserved. All related
logos, assets
messaging are
trademarks of
Journey Further Ltd
8
What did we
do?
9
©2022 Journey
Further. All rights
reserved. All related
logos, assets
messaging are
trademarks of
Journey Further Ltd
01
Prioritised key
pages based on
potential traffic
10
©2022 Journey
Further. All rights
reserved. All related
logos, assets
messaging are
trademarks of
Journey Further Ltd
02
Optimised them
(based on
prioritised
order)
11
©2022 Journey
Further. All rights
reserved. All related
logos, assets
messaging are
trademarks of
Journey Further Ltd
03
Watched the
line go up
12
©2022 Journey
Further. All rights
reserved. All related
logos, assets
messaging are
trademarks of
Journey Further Ltd
How does
NLP
work?
13
Processing
©2022 Journey
Further. All rights
reserved. All related
logos, assets
messaging are
trademarks of
Journey Further Ltd
INPUT TOKENISATION
14
Processing
©2022 Journey
Further. All rights
reserved. All related
logos, assets
messaging are
trademarks of
Journey Further Ltd
TOKENISATION
POS TAG
SALIENCE
STEMS
DEPENDENCY
15
NLP in action
©2022 Journey
Further. All rights
reserved. All related
logos, assets
messaging are
trademarks of
Journey Further Ltd
16
©2022 Journey
Further. All rights
reserved. All related
logos, assets
messaging are
trademarks of
Journey Further Ltd
17
What do we know about Alice?
©2022 Journey
Further. All rights
reserved. All related
logos, assets
messaging are
trademarks of
Journey Further Ltd
Alice is swimming against the current.
Noun (aux) V Verb Preposition Determiner Noun
Parsed by Google’s NLP tool
Parsed manually
18
BERT
©2022 Journey
Further. All rights
reserved. All related
logos, assets
messaging are
trademarks of
Journey Further Ltd
©2022 Journey
Further. All rights
reserved. All related
logos, assets
messaging are
trademarks of
Journey Further Ltd
19
BERT
NLU icon
Token masking
Next sentence
prediction
20
©2022 Journey
Further. All rights
reserved. All related
logos, assets
messaging are
trademarks of
Journey Further Ltd
“In autumn the ______
fall from the trees.”
21
©2022 Journey
Further. All rights
reserved. All related
logos, assets
messaging are
trademarks of
Journey Further Ltd
At dinner, Ernie
ordered a pizza
and tiramisu.
Do we still call
it link juice?
After a long
day of work,
Ernie went out
to dinner with
friends.
IsNext
NotNext
22
©2022 Journey
Further. All rights
reserved. All related
logos, assets
messaging are
trademarks of
Journey Further Ltd
How to optimise
content with
NLP
23
©2022 Journey
Further. All rights
reserved. All related
logos, assets
messaging are
trademarks of
Journey Further Ltd
“First-time
buyer”
Adjective (first-time)
supporting a noun
(buyer)
Maximum
three word
phrase
Noun is not also
an adjective
No numbers in
the keyword
24
©2022 Journey
Further. All rights
reserved. All related
logos, assets
messaging are
trademarks of
Journey Further Ltd
25
©2022 Journey
Further. All rights
reserved. All related
logos, assets
messaging are
trademarks of
Journey Further Ltd
In the rugged Colorado Desert of
California, there lies buried a treasure
ship sailed there hundreds of years
ago by either Viking or Spanish
explorers. Some say this is legend;
others insist it is fact. A few have
even claimed to have seen the ship,
its wooden remains poking through
the sand like the skeleton of a
prehistoric beast. Among those who
say they’ve come close to the ship is
small-town librarian Myrtle Botts. In
1933, she was hiking with her
husband in the Anza-Borrego Desert,
not far from the border with Mexico.
Text Relevance
rugged
Colorado
Desert of
California
0.89
treasure ship 0.78
Anza-
Borrego
Desert
0.75
Spanish
explorers
0.6
26
©2022 Journey
Further. All rights
reserved. All related
logos, assets
messaging are
trademarks of
Journey Further Ltd
Villas in Spain are incredibly popular, and
for good reason too! With beautiful
beaches and a charming countryside,
bustling cities and idyllic towns, a
Spanish villa holiday will
certainly be one to remember.
27
©2022 Journey
Further. All rights
reserved. All related
logos, assets
messaging are
trademarks of
Journey Further Ltd
Topic Original Updated Difference
holiday
villas
0.00 0.58 0.58
Spanish
villa
properties
0.64 0.69 0.05
In Salient, salience is measured as a
score out of 1.
28
©2022 Journey
Further. All rights
reserved. All related
logos, assets
messaging are
trademarks of
Journey Further Ltd
Changes go through
29
©2022 Journey
Further. All rights
reserved. All related
logos, assets
messaging are
trademarks of
Journey Further Ltd
Introduce
focus
keywords
in the first
paragraph
Experiment
with
sentence
structure
Look at
what the
tool sees
as
relevant
Do
30
©2022 Journey
Further. All rights
reserved. All related
logos, assets
messaging are
trademarks of
Journey Further Ltd
Don’t
Keyword
stuff
Write for
robots
31
So why
use NLP?
©2022 Journey
Further. All rights
reserved. All related
logos, assets
messaging are
trademarks of
Journey Further Ltd
32
©2022 Journey
Further. All rights
reserved. All related
logos, assets
messaging are
trademarks of
Journey Further Ltd
01
Because
you can
02
Because
Google is
03
Because
it works!
Thank
you
33
34
Appendix
©2022 Journey
Further. All rights
reserved. All related
logos, assets
messaging are
trademarks of
Journey Further Ltd
35
©2022 Journey
Further. All rights
reserved. All related
logos, assets
messaging are
trademarks of
Journey Further Ltd
Tools mentioned:
SALIENT
Google’s natural language API demo
Understanding BERT:
● Open Sourcing BERT: State-of-the-Art Pre-training for Natural Language
Processing
● BERT Explained: State of the art language model for NLP
● What’s My Line? Next Sentence Prediction in RunwayML with BERT
● BERT For Next Sentence Prediction
36
Syntax & Semantics
©2022 Journey
Further. All rights
reserved. All related
logos, assets
messaging are
trademarks of
Journey Further Ltd
Syntax is the grammatical structure of a
sentence, the order each word goes into.
Semantics is the intent part of a sentence, how
do the words in the sentence affect each other?
How does the order in which they come in
change your meaning?
NLP turns syntax and semantics into its own
data structure which specifies the relationship
between words and phrases. This is known as
ontology.
NLU takes this ontology and attempts to
prescribe intent to it.
“Patient’s mother was involved in a car accident
and injured her hand.”
37
Parts of speech
©2022 Journey
Further. All rights
reserved. All related
logos, assets
messaging are
trademarks of
Journey Further Ltd
Nouns - keywords are usually nouns. Nouns are
things.
Verbs - doing words
Adjectives - describing words
Superlatives - adjectives on a scale. Best,
worst, greatest, biggest, smallest.
Adverbs - describes a word. Quickly, slowly,
really, very.
Prepositions - Links a noun to another noun.
To, at, after, on.
Conjunctions - Joins clauses, sentences, or
words together. And, but, when.
38
Sentence structures/syntax
©2022 Journey
Further. All rights
reserved. All related
logos, assets
messaging are
trademarks of
Journey Further Ltd
Independent clause - a part of a sentence that
could stand alone. A subject, a verb, and a
complete idea.
I like SEO.
He is optimising a page.
Dependent clause - Not a complete sentence.
Sits with an independent clause to become a
complete sentence.
Because she said that,
As I’m currently looking into this,
Subject - Whatever’s doing the action in your
sentence.
I like SEO.
Verb - the action in your sentence.
I like SEO.
Object - What receives the action in
a sentence.
I like SEO.
Prepositional phrase - A phrase
that begins with a preposition, and
modifies a word in the sentence.
I like SEO on smaller websites.
39
Types of sentences
©2022 Journey
Further. All rights
reserved. All related
logos, assets
messaging are
trademarks of
Journey Further Ltd
Simple sentences - a sentence that could stand
alone. A subject, a verb, and a complete idea.
I like SEO.
He is optimising a page.
Compound sentences - A sentence that
contains two or more independent clauses, they
can be combined with a comma, a conjunction,
or with a semicolon.
I like SEO, I’m good at it.
I like SEO, but I’m no good at it.
Complex sentences - an independent
clause and one dependent clause (a
part of a sentence that can’t stand on
its own).
Although I like SEO, I don’t like site
speed.
Compound-complex sentences - a
sentence that has at least two
independent clauses and at least one
dependent clause. These can be
joined via commas or conjunctions,
though they can get confusing if your
comma game isn’t strong.
I like SEO, even though I spent
hours trying to edit a robots.txt, and
my laptop crashed trying to run a
crawl.

BrightonSEO - NLP for SEOs - How to optimise your content for BERT.pptx

  • 1.
    NLP for SEOs:How to optimise your content for BERT @jcbhaagen Josephine Haagen Journey Further https://www.slideshare.net/ JosephineHaagen
  • 2.
    2 What is NLP? ©2022 Journey Further.All rights reserved. All related logos, assets messaging are trademarks of Journey Further Ltd
  • 3.
    How machines understand language. 3 ©2022Journey Further. All rights reserved. All related logos, assets messaging are trademarks of Journey Further Ltd
  • 4.
    4 Or, in SEOterms: Takes unstructured data and attempts to structure it. ©2022 Journey Further. All rights reserved. All related logos, assets messaging are trademarks of Journey Further Ltd
  • 5.
    ©2022 Journey Further. Allrights reserved. All related logos, assets messaging are trademarks of Journey Further Ltd 5 What can NLP do for you?
  • 6.
    6 A content strategyled by NLP ©2022 Journey Further. All rights reserved. All related logos, assets messaging are trademarks of Journey Further Ltd
  • 7.
    7 A content strategyled by NLP ©2022 Journey Further. All rights reserved. All related logos, assets messaging are trademarks of Journey Further Ltd
  • 8.
    ©2022 Journey Further. Allrights reserved. All related logos, assets messaging are trademarks of Journey Further Ltd 8 What did we do?
  • 9.
    9 ©2022 Journey Further. Allrights reserved. All related logos, assets messaging are trademarks of Journey Further Ltd 01 Prioritised key pages based on potential traffic
  • 10.
    10 ©2022 Journey Further. Allrights reserved. All related logos, assets messaging are trademarks of Journey Further Ltd 02 Optimised them (based on prioritised order)
  • 11.
    11 ©2022 Journey Further. Allrights reserved. All related logos, assets messaging are trademarks of Journey Further Ltd 03 Watched the line go up
  • 12.
    12 ©2022 Journey Further. Allrights reserved. All related logos, assets messaging are trademarks of Journey Further Ltd How does NLP work?
  • 13.
    13 Processing ©2022 Journey Further. Allrights reserved. All related logos, assets messaging are trademarks of Journey Further Ltd INPUT TOKENISATION
  • 14.
    14 Processing ©2022 Journey Further. Allrights reserved. All related logos, assets messaging are trademarks of Journey Further Ltd TOKENISATION POS TAG SALIENCE STEMS DEPENDENCY
  • 15.
    15 NLP in action ©2022Journey Further. All rights reserved. All related logos, assets messaging are trademarks of Journey Further Ltd
  • 16.
    16 ©2022 Journey Further. Allrights reserved. All related logos, assets messaging are trademarks of Journey Further Ltd
  • 17.
    17 What do weknow about Alice? ©2022 Journey Further. All rights reserved. All related logos, assets messaging are trademarks of Journey Further Ltd Alice is swimming against the current. Noun (aux) V Verb Preposition Determiner Noun Parsed by Google’s NLP tool Parsed manually
  • 18.
    18 BERT ©2022 Journey Further. Allrights reserved. All related logos, assets messaging are trademarks of Journey Further Ltd
  • 19.
    ©2022 Journey Further. Allrights reserved. All related logos, assets messaging are trademarks of Journey Further Ltd 19 BERT NLU icon Token masking Next sentence prediction
  • 20.
    20 ©2022 Journey Further. Allrights reserved. All related logos, assets messaging are trademarks of Journey Further Ltd “In autumn the ______ fall from the trees.”
  • 21.
    21 ©2022 Journey Further. Allrights reserved. All related logos, assets messaging are trademarks of Journey Further Ltd At dinner, Ernie ordered a pizza and tiramisu. Do we still call it link juice? After a long day of work, Ernie went out to dinner with friends. IsNext NotNext
  • 22.
    22 ©2022 Journey Further. Allrights reserved. All related logos, assets messaging are trademarks of Journey Further Ltd How to optimise content with NLP
  • 23.
    23 ©2022 Journey Further. Allrights reserved. All related logos, assets messaging are trademarks of Journey Further Ltd “First-time buyer” Adjective (first-time) supporting a noun (buyer) Maximum three word phrase Noun is not also an adjective No numbers in the keyword
  • 24.
    24 ©2022 Journey Further. Allrights reserved. All related logos, assets messaging are trademarks of Journey Further Ltd
  • 25.
    25 ©2022 Journey Further. Allrights reserved. All related logos, assets messaging are trademarks of Journey Further Ltd In the rugged Colorado Desert of California, there lies buried a treasure ship sailed there hundreds of years ago by either Viking or Spanish explorers. Some say this is legend; others insist it is fact. A few have even claimed to have seen the ship, its wooden remains poking through the sand like the skeleton of a prehistoric beast. Among those who say they’ve come close to the ship is small-town librarian Myrtle Botts. In 1933, she was hiking with her husband in the Anza-Borrego Desert, not far from the border with Mexico. Text Relevance rugged Colorado Desert of California 0.89 treasure ship 0.78 Anza- Borrego Desert 0.75 Spanish explorers 0.6
  • 26.
    26 ©2022 Journey Further. Allrights reserved. All related logos, assets messaging are trademarks of Journey Further Ltd Villas in Spain are incredibly popular, and for good reason too! With beautiful beaches and a charming countryside, bustling cities and idyllic towns, a Spanish villa holiday will certainly be one to remember.
  • 27.
    27 ©2022 Journey Further. Allrights reserved. All related logos, assets messaging are trademarks of Journey Further Ltd Topic Original Updated Difference holiday villas 0.00 0.58 0.58 Spanish villa properties 0.64 0.69 0.05 In Salient, salience is measured as a score out of 1.
  • 28.
    28 ©2022 Journey Further. Allrights reserved. All related logos, assets messaging are trademarks of Journey Further Ltd Changes go through
  • 29.
    29 ©2022 Journey Further. Allrights reserved. All related logos, assets messaging are trademarks of Journey Further Ltd Introduce focus keywords in the first paragraph Experiment with sentence structure Look at what the tool sees as relevant Do
  • 30.
    30 ©2022 Journey Further. Allrights reserved. All related logos, assets messaging are trademarks of Journey Further Ltd Don’t Keyword stuff Write for robots
  • 31.
    31 So why use NLP? ©2022Journey Further. All rights reserved. All related logos, assets messaging are trademarks of Journey Further Ltd
  • 32.
    32 ©2022 Journey Further. Allrights reserved. All related logos, assets messaging are trademarks of Journey Further Ltd 01 Because you can 02 Because Google is 03 Because it works!
  • 33.
  • 34.
    34 Appendix ©2022 Journey Further. Allrights reserved. All related logos, assets messaging are trademarks of Journey Further Ltd
  • 35.
    35 ©2022 Journey Further. Allrights reserved. All related logos, assets messaging are trademarks of Journey Further Ltd Tools mentioned: SALIENT Google’s natural language API demo Understanding BERT: ● Open Sourcing BERT: State-of-the-Art Pre-training for Natural Language Processing ● BERT Explained: State of the art language model for NLP ● What’s My Line? Next Sentence Prediction in RunwayML with BERT ● BERT For Next Sentence Prediction
  • 36.
    36 Syntax & Semantics ©2022Journey Further. All rights reserved. All related logos, assets messaging are trademarks of Journey Further Ltd Syntax is the grammatical structure of a sentence, the order each word goes into. Semantics is the intent part of a sentence, how do the words in the sentence affect each other? How does the order in which they come in change your meaning? NLP turns syntax and semantics into its own data structure which specifies the relationship between words and phrases. This is known as ontology. NLU takes this ontology and attempts to prescribe intent to it. “Patient’s mother was involved in a car accident and injured her hand.”
  • 37.
    37 Parts of speech ©2022Journey Further. All rights reserved. All related logos, assets messaging are trademarks of Journey Further Ltd Nouns - keywords are usually nouns. Nouns are things. Verbs - doing words Adjectives - describing words Superlatives - adjectives on a scale. Best, worst, greatest, biggest, smallest. Adverbs - describes a word. Quickly, slowly, really, very. Prepositions - Links a noun to another noun. To, at, after, on. Conjunctions - Joins clauses, sentences, or words together. And, but, when.
  • 38.
    38 Sentence structures/syntax ©2022 Journey Further.All rights reserved. All related logos, assets messaging are trademarks of Journey Further Ltd Independent clause - a part of a sentence that could stand alone. A subject, a verb, and a complete idea. I like SEO. He is optimising a page. Dependent clause - Not a complete sentence. Sits with an independent clause to become a complete sentence. Because she said that, As I’m currently looking into this, Subject - Whatever’s doing the action in your sentence. I like SEO. Verb - the action in your sentence. I like SEO. Object - What receives the action in a sentence. I like SEO. Prepositional phrase - A phrase that begins with a preposition, and modifies a word in the sentence. I like SEO on smaller websites.
  • 39.
    39 Types of sentences ©2022Journey Further. All rights reserved. All related logos, assets messaging are trademarks of Journey Further Ltd Simple sentences - a sentence that could stand alone. A subject, a verb, and a complete idea. I like SEO. He is optimising a page. Compound sentences - A sentence that contains two or more independent clauses, they can be combined with a comma, a conjunction, or with a semicolon. I like SEO, I’m good at it. I like SEO, but I’m no good at it. Complex sentences - an independent clause and one dependent clause (a part of a sentence that can’t stand on its own). Although I like SEO, I don’t like site speed. Compound-complex sentences - a sentence that has at least two independent clauses and at least one dependent clause. These can be joined via commas or conjunctions, though they can get confusing if your comma game isn’t strong. I like SEO, even though I spent hours trying to edit a robots.txt, and my laptop crashed trying to run a crawl.

Editor's Notes

  • #3 Include introduction slide - where did I start with this?
  • #6 There's more - much more - you can do to create a content ideation that is based on actual insights and not just keyword research
  • #7 Average rank has gone down across most of the pages we’ve optimised (this isn’t a magic bullet, using NLP doesn’t guarantee results just like the rest of SEO)
  • #8 Average rank has gone down across most of the pages we’ve optimised (this isn’t a magic bullet, using NLP doesn’t guarantee results just like the rest of SEO)
  • #17 Sentiment’s hokeyyyyy
  • #18 Explain difference between adjective and noun
  • #20 A machine learning technique, aimed at understanding intent and the relationship between queries. The blueprint for every NLP experiment since. Used in pretty much every English language query + 70 other languages across Google Search. Pre-trained on English-language Wikipedia and BooksCorpus - all the text it was trained on (its corpus) was in order and not shuffled. BERT is a pre-training model - you can use BERT as a base for your own NLP needs (provided you know Python!) There are two things that BERT does best: language modelling and next sentence prediction. Token masking: BERT was trained to predict the next word in a sentence, based off of the text that it was trained on. Bidirectional: BERT was the first NLP model that was fully bidirectional, able to use the context of both the words around masked tokens. BERT was made bidirectional by its token masking process - Input tokens were masked randomly, and the program would predict what their tokens are. Next sentence prediction: The program would be given a first sentence, then would have to guess if the next sentence given IsNext or is NotNext. 50% of second sentences would be a random sentence from the corpus, while the other half would be the actual sentence that followed the first one. This is an important process in understanding the relationship between sentences and how they work together.
  • #27 Make keywords larger
  • #36 Make keywords larger
  • #37 Average rank has gone down across most of the pages we’ve optimised (this isn’t a magic bullet, using NLP doesn’t guarantee results just like the rest of SEO)
  • #38 Average rank has gone down across most of the pages we’ve optimised (this isn’t a magic bullet, using NLP doesn’t guarantee results just like the rest of SEO)
  • #39 Average rank has gone down across most of the pages we’ve optimised (this isn’t a magic bullet, using NLP doesn’t guarantee results just like the rest of SEO)
  • #40 Average rank has gone down across most of the pages we’ve optimised (this isn’t a magic bullet, using NLP doesn’t guarantee results just like the rest of SEO)