How to get synonyms from nltk WordNet Python
Question:
WordNet is great, but I’m having a hard time getting synonyms in nltk. If you search similar to for the word ‘small’ like here, it shows all of the synonyms.
Basically I just need to know the following:
wn.synsets('word')[i].option() Where option can be hypernyms and antonyms, but what is the option for getting synonyms?
Answers:
You might be interested in a Synset:
>>> wn.synsets('small')
[Synset('small.n.01'),
Synset('small.n.02'),
Synset('small.a.01'),
Synset('minor.s.10'),
Synset('little.s.03'),
Synset('small.s.04'),
Synset('humble.s.01'),
Synset('little.s.07'),
Synset('little.s.05'),
Synset('small.s.08'),
Synset('modest.s.02'),
Synset('belittled.s.01'),
Synset('small.r.01')]
That’s the same list of top-level entries that the web interface gave you.
If you also want the "similar to" list, that’s not the same thing as the synonyms. For that, you call similar_tos() on each Synset.
So, to show the same information as the website, start with something like this:
for ss in wn.synsets('small'):
print(ss)
for sim in ss.similar_tos():
print(' {}'.format(sim))
Of course the website is also printing the part of speech (sim.pos), list of lemmas (sim.lemma_names), definition (sim.definition), and examples (sim.examples) for each synset at both levels. and it’s grouping them by parts of speech, and it’s added in links to other things that you can follow, and so forth. But that should be enough to get you started.
If you want the synonyms in the synset (aka the lemmas that make up the set), you can get them with lemma_names():
>>> for ss in wn.synsets('small'):
>>> print(ss.name(), ss.lemma_names())
small.n.01 ['small']
small.n.02 ['small']
small.a.01 ['small', 'little']
minor.s.10 ['minor', 'modest', 'small', 'small-scale', 'pocket-size', 'pocket-sized']
little.s.03 ['little', 'small']
small.s.04 ['small']
humble.s.01 ['humble', 'low', 'lowly', 'modest', 'small']
...
You can use wordnet.synset and lemmas in order to get all the synonyms:
example :
from itertools import chain
from nltk.corpus import wordnet
synonyms = wordnet.synsets(text)
lemmas = set(chain.from_iterable([word.lemma_names() for word in synonyms]))
Demo:
>>> synonyms = wordnet.synsets('change')
>>> set(chain.from_iterable([word.lemma_names() for word in synonyms]))
set([u'interchange', u'convert', u'variety', u'vary', u'exchange', u'modify', u'alteration', u'switch', u'commute', u'shift', u'modification', u'deepen', u'transfer', u'alter', u'change'])
This worked for me
wordnet.synsets('change')[0].hypernyms()[0].lemma_names()
Simplest program to print the synonyms of a given word
from nltk.corpus import wordnet
for syn in wordnet.synsets("good"):
for name in syn.lemma_names():
print(name)
I’ve code Thesaurus Lookup for Synonym recently, I used this function :
def find_synonyms(keyword) :
synonyms = []
for synset in wordnet.synsets(keyword):
for lemma in synset.lemmas():
synonyms.append(lemma.name())
return str(synonyms)
But if you prefer to host your own Dictionary, you might interested with my project on offline synonym dictionary lookup on my github page :
https://github.com/syauqiex/offline_english_synonym_dictionary
Here are some helper functions to make NLTK easier to use, and two examples of how those functions can be used.
def download_nltk_dependencies_if_needed():
try:
nltk.word_tokenize('foobar')
except LookupError:
nltk.download('punkt')
try:
nltk.pos_tag(nltk.word_tokenize('foobar'))
except LookupError:
nltk.download('averaged_perceptron_tagger')
def get_some_word_synonyms(word):
word = word.lower()
synonyms = []
synsets = wordnet.synsets(word)
if (len(synsets) == 0):
return []
synset = synsets[0]
lemma_names = synset.lemma_names()
for lemma_name in lemma_names:
lemma_name = lemma_name.lower().replace('_', ' ')
if (lemma_name != word and lemma_name not in synonyms):
synonyms.append(lemma_name)
return synonyms
def get_all_word_synonyms(word):
word = word.lower()
synonyms = []
synsets = wordnet.synsets(word)
if (len(synsets) == 0):
return []
for synset in synsets:
lemma_names = synset.lemma_names()
for lemma_name in lemma_names:
lemma_name = lemma_name.lower().replace('_', ' ')
if (lemma_name != word and lemma_name not in synonyms):
synonyms.append(lemma_name)
return synonyms
Example 1: get_some_word_synonyms
This approach tends to return the most relevant synonyms, but some words like "angry" won’t return any synonyms.
download_nltk_dependencies_if_needed()
words = ['dog', 'fire', 'erupted', 'throw', 'sweet', 'center', 'said', 'angry', 'iPhone', 'ThisIsNotARealWorddd', 'awesome', 'amazing', 'jim dandy', 'change']
for word in words:
print('Synonyms for {}:'.format(word))
synonyms = get_some_word_synonyms(word)
for synonym in synonyms:
print(" {}".format(synonym))
Example 1 output:
Synonyms for dog:
domestic dog
canis familiaris
Synonyms for fire:
Synonyms for erupted:
erupt
break out
Synonyms for throw:
Synonyms for sweet:
henry sweet
Synonyms for center:
centre
middle
heart
eye
Synonyms for said:
state
say
tell
Synonyms for angry:
Synonyms for iPhone:
Synonyms for ThisIsNotARealWorddd:
Synonyms for awesome:
amazing
awe-inspiring
awful
awing
Synonyms for amazing:
amaze
astonish
astound
Synonyms for jim dandy:
Synonyms for change:
alteration
modification
Example 2: get_all_word_synonyms
This approach will return all possible synonyms, but some may not be very relevant.
download_nltk_dependencies_if_needed()
words = ['dog', 'fire', 'erupted', 'throw', 'sweet', 'center', 'said', 'angry', 'iPhone', 'ThisIsNotARealWorddd', 'awesome', 'amazing', 'jim dandy', 'change']
for word in words:
print('Synonyms for {}:'.format(word))
synonyms = get_some_word_synonyms(word)
for synonym in synonyms:
print(" {}".format(synonym))
Example 2 output:
Synonyms for dog:
domestic dog
canis familiaris
frump
cad
bounder
blackguard
hound
heel
frank
frankfurter
hotdog
hot dog
wiener
wienerwurst
weenie
pawl
detent
click
andiron
firedog
dog-iron
chase
chase after
trail
tail
tag
give chase
go after
track
Synonyms for fire:
firing
flame
flaming
ardor
ardour
fervor
fervour
fervency
fervidness
attack
flak
flack
blast
open fire
discharge
displace
give notice
can
dismiss
give the axe
send away
sack
force out
give the sack
terminate
go off
arouse
elicit
enkindle
kindle
evoke
raise
provoke
burn
burn down
fuel
Synonyms for erupted:
erupt
break out
irrupt
flare up
flare
break open
burst out
ignite
catch fire
take fire
combust
conflagrate
come out
break through
push through
belch
extravasate
break
burst
recrudesce
Synonyms for throw:
stroke
cam stroke
shed
cast
cast off
shake off
throw off
throw away
drop
thrust
give
flip
switch
project
contrive
bewilder
bemuse
discombobulate
hurl
hold
have
make
confuse
fox
befuddle
fuddle
bedevil
confound
Synonyms for sweet:
henry sweet
dessert
afters
confection
sweetness
sugariness
angelic
angelical
cherubic
seraphic
dulcet
honeyed
mellifluous
mellisonant
gratifying
odoriferous
odorous
perfumed
scented
sweet-scented
sweet-smelling
fresh
unfermented
sugared
sweetened
sweet-flavored
sweetly
Synonyms for center:
centre
middle
heart
eye
center field
centerfield
midpoint
kernel
substance
core
essence
gist
heart and soul
inwardness
marrow
meat
nub
pith
sum
nitty-gritty
center of attention
centre of attention
nerve center
nerve centre
snapper
plaza
mall
shopping mall
shopping center
shopping centre
focus on
center on
revolve around
revolve about
concentrate on
concentrate
focus
pore
rivet
halfway
midway
Synonyms for said:
state
say
tell
allege
aver
suppose
read
order
enjoin
pronounce
articulate
enounce
sound out
enunciate
aforesaid
aforementioned
Synonyms for angry:
furious
raging
tempestuous
wild
Synonyms for iPhone:
Synonyms for ThisIsNotARealWorddd:
Synonyms for awesome:
amazing
awe-inspiring
awful
awing
Synonyms for amazing:
amaze
astonish
astound
perplex
vex
stick
get
puzzle
mystify
baffle
beat
pose
bewilder
flummox
stupefy
nonplus
gravel
dumbfound
astonishing
awe-inspiring
awesome
awful
awing
Synonyms for jim dandy:
Synonyms for change:
alteration
modification
variety
alter
modify
vary
switch
shift
exchange
commute
convert
interchange
transfer
deepen
Perhaps these are not synonyms in the proper terminology of wordnet. But I also want my function to return all similar words, like ‘weeny’, ‘flyspeck’ etc. You can see them for the word ‘small’ in the author link. I used these code:
from nltk.corpus import wordnet as wn
def get_all_synonyms(word):
synonyms = []
for ss in wn.synsets(word):
synonyms.extend(ss.lemma_names())
for sim in ss.similar_tos():
synonyms_batch = sim.lemma_names()
synonyms.extend(synonyms_batch)
synonyms = set(synonyms)
if word in synonyms:
synonyms.remove(word)
synonyms = [synonym.replace('_',' ') for synonym in synonyms]
return synonyms
get_all_synonyms('small')
WordNet is great, but I’m having a hard time getting synonyms in nltk. If you search similar to for the word ‘small’ like here, it shows all of the synonyms.
Basically I just need to know the following:
wn.synsets('word')[i].option() Where option can be hypernyms and antonyms, but what is the option for getting synonyms?
You might be interested in a Synset:
>>> wn.synsets('small')
[Synset('small.n.01'),
Synset('small.n.02'),
Synset('small.a.01'),
Synset('minor.s.10'),
Synset('little.s.03'),
Synset('small.s.04'),
Synset('humble.s.01'),
Synset('little.s.07'),
Synset('little.s.05'),
Synset('small.s.08'),
Synset('modest.s.02'),
Synset('belittled.s.01'),
Synset('small.r.01')]
That’s the same list of top-level entries that the web interface gave you.
If you also want the "similar to" list, that’s not the same thing as the synonyms. For that, you call similar_tos() on each Synset.
So, to show the same information as the website, start with something like this:
for ss in wn.synsets('small'):
print(ss)
for sim in ss.similar_tos():
print(' {}'.format(sim))
Of course the website is also printing the part of speech (sim.pos), list of lemmas (sim.lemma_names), definition (sim.definition), and examples (sim.examples) for each synset at both levels. and it’s grouping them by parts of speech, and it’s added in links to other things that you can follow, and so forth. But that should be enough to get you started.
If you want the synonyms in the synset (aka the lemmas that make up the set), you can get them with lemma_names():
>>> for ss in wn.synsets('small'):
>>> print(ss.name(), ss.lemma_names())
small.n.01 ['small']
small.n.02 ['small']
small.a.01 ['small', 'little']
minor.s.10 ['minor', 'modest', 'small', 'small-scale', 'pocket-size', 'pocket-sized']
little.s.03 ['little', 'small']
small.s.04 ['small']
humble.s.01 ['humble', 'low', 'lowly', 'modest', 'small']
...
You can use wordnet.synset and lemmas in order to get all the synonyms:
example :
from itertools import chain
from nltk.corpus import wordnet
synonyms = wordnet.synsets(text)
lemmas = set(chain.from_iterable([word.lemma_names() for word in synonyms]))
Demo:
>>> synonyms = wordnet.synsets('change')
>>> set(chain.from_iterable([word.lemma_names() for word in synonyms]))
set([u'interchange', u'convert', u'variety', u'vary', u'exchange', u'modify', u'alteration', u'switch', u'commute', u'shift', u'modification', u'deepen', u'transfer', u'alter', u'change'])
This worked for me
wordnet.synsets('change')[0].hypernyms()[0].lemma_names()
Simplest program to print the synonyms of a given word
from nltk.corpus import wordnet
for syn in wordnet.synsets("good"):
for name in syn.lemma_names():
print(name)
I’ve code Thesaurus Lookup for Synonym recently, I used this function :
def find_synonyms(keyword) :
synonyms = []
for synset in wordnet.synsets(keyword):
for lemma in synset.lemmas():
synonyms.append(lemma.name())
return str(synonyms)
But if you prefer to host your own Dictionary, you might interested with my project on offline synonym dictionary lookup on my github page :
https://github.com/syauqiex/offline_english_synonym_dictionary
Here are some helper functions to make NLTK easier to use, and two examples of how those functions can be used.
def download_nltk_dependencies_if_needed():
try:
nltk.word_tokenize('foobar')
except LookupError:
nltk.download('punkt')
try:
nltk.pos_tag(nltk.word_tokenize('foobar'))
except LookupError:
nltk.download('averaged_perceptron_tagger')
def get_some_word_synonyms(word):
word = word.lower()
synonyms = []
synsets = wordnet.synsets(word)
if (len(synsets) == 0):
return []
synset = synsets[0]
lemma_names = synset.lemma_names()
for lemma_name in lemma_names:
lemma_name = lemma_name.lower().replace('_', ' ')
if (lemma_name != word and lemma_name not in synonyms):
synonyms.append(lemma_name)
return synonyms
def get_all_word_synonyms(word):
word = word.lower()
synonyms = []
synsets = wordnet.synsets(word)
if (len(synsets) == 0):
return []
for synset in synsets:
lemma_names = synset.lemma_names()
for lemma_name in lemma_names:
lemma_name = lemma_name.lower().replace('_', ' ')
if (lemma_name != word and lemma_name not in synonyms):
synonyms.append(lemma_name)
return synonyms
Example 1: get_some_word_synonyms
This approach tends to return the most relevant synonyms, but some words like "angry" won’t return any synonyms.
download_nltk_dependencies_if_needed()
words = ['dog', 'fire', 'erupted', 'throw', 'sweet', 'center', 'said', 'angry', 'iPhone', 'ThisIsNotARealWorddd', 'awesome', 'amazing', 'jim dandy', 'change']
for word in words:
print('Synonyms for {}:'.format(word))
synonyms = get_some_word_synonyms(word)
for synonym in synonyms:
print(" {}".format(synonym))
Example 1 output:
Synonyms for dog:
domestic dog
canis familiaris
Synonyms for fire:
Synonyms for erupted:
erupt
break out
Synonyms for throw:
Synonyms for sweet:
henry sweet
Synonyms for center:
centre
middle
heart
eye
Synonyms for said:
state
say
tell
Synonyms for angry:
Synonyms for iPhone:
Synonyms for ThisIsNotARealWorddd:
Synonyms for awesome:
amazing
awe-inspiring
awful
awing
Synonyms for amazing:
amaze
astonish
astound
Synonyms for jim dandy:
Synonyms for change:
alteration
modification
Example 2: get_all_word_synonyms
This approach will return all possible synonyms, but some may not be very relevant.
download_nltk_dependencies_if_needed()
words = ['dog', 'fire', 'erupted', 'throw', 'sweet', 'center', 'said', 'angry', 'iPhone', 'ThisIsNotARealWorddd', 'awesome', 'amazing', 'jim dandy', 'change']
for word in words:
print('Synonyms for {}:'.format(word))
synonyms = get_some_word_synonyms(word)
for synonym in synonyms:
print(" {}".format(synonym))
Example 2 output:
Synonyms for dog:
domestic dog
canis familiaris
frump
cad
bounder
blackguard
hound
heel
frank
frankfurter
hotdog
hot dog
wiener
wienerwurst
weenie
pawl
detent
click
andiron
firedog
dog-iron
chase
chase after
trail
tail
tag
give chase
go after
track
Synonyms for fire:
firing
flame
flaming
ardor
ardour
fervor
fervour
fervency
fervidness
attack
flak
flack
blast
open fire
discharge
displace
give notice
can
dismiss
give the axe
send away
sack
force out
give the sack
terminate
go off
arouse
elicit
enkindle
kindle
evoke
raise
provoke
burn
burn down
fuel
Synonyms for erupted:
erupt
break out
irrupt
flare up
flare
break open
burst out
ignite
catch fire
take fire
combust
conflagrate
come out
break through
push through
belch
extravasate
break
burst
recrudesce
Synonyms for throw:
stroke
cam stroke
shed
cast
cast off
shake off
throw off
throw away
drop
thrust
give
flip
switch
project
contrive
bewilder
bemuse
discombobulate
hurl
hold
have
make
confuse
fox
befuddle
fuddle
bedevil
confound
Synonyms for sweet:
henry sweet
dessert
afters
confection
sweetness
sugariness
angelic
angelical
cherubic
seraphic
dulcet
honeyed
mellifluous
mellisonant
gratifying
odoriferous
odorous
perfumed
scented
sweet-scented
sweet-smelling
fresh
unfermented
sugared
sweetened
sweet-flavored
sweetly
Synonyms for center:
centre
middle
heart
eye
center field
centerfield
midpoint
kernel
substance
core
essence
gist
heart and soul
inwardness
marrow
meat
nub
pith
sum
nitty-gritty
center of attention
centre of attention
nerve center
nerve centre
snapper
plaza
mall
shopping mall
shopping center
shopping centre
focus on
center on
revolve around
revolve about
concentrate on
concentrate
focus
pore
rivet
halfway
midway
Synonyms for said:
state
say
tell
allege
aver
suppose
read
order
enjoin
pronounce
articulate
enounce
sound out
enunciate
aforesaid
aforementioned
Synonyms for angry:
furious
raging
tempestuous
wild
Synonyms for iPhone:
Synonyms for ThisIsNotARealWorddd:
Synonyms for awesome:
amazing
awe-inspiring
awful
awing
Synonyms for amazing:
amaze
astonish
astound
perplex
vex
stick
get
puzzle
mystify
baffle
beat
pose
bewilder
flummox
stupefy
nonplus
gravel
dumbfound
astonishing
awe-inspiring
awesome
awful
awing
Synonyms for jim dandy:
Synonyms for change:
alteration
modification
variety
alter
modify
vary
switch
shift
exchange
commute
convert
interchange
transfer
deepen
Perhaps these are not synonyms in the proper terminology of wordnet. But I also want my function to return all similar words, like ‘weeny’, ‘flyspeck’ etc. You can see them for the word ‘small’ in the author link. I used these code:
from nltk.corpus import wordnet as wn
def get_all_synonyms(word):
synonyms = []
for ss in wn.synsets(word):
synonyms.extend(ss.lemma_names())
for sim in ss.similar_tos():
synonyms_batch = sim.lemma_names()
synonyms.extend(synonyms_batch)
synonyms = set(synonyms)
if word in synonyms:
synonyms.remove(word)
synonyms = [synonym.replace('_',' ') for synonym in synonyms]
return synonyms
get_all_synonyms('small')