UnicodeDecodeError when reading file in Python


I am currently learning Python and came across the following error:

Traceback (most recent call last):   
  File "file.py", line 22, in module
    for word in file.read():
  File "C:UsersuserAppDataLocalContinuumAnaconda3libencodingscp1252.py" line 23, in decode
    return codecs.charmap_decode(input,self.errors,decoding_table)[0] 
UnicodeDecodeError: 'charmap' codec can't decode byte 0x9d in position 6552: character maps to undefined

This is my code:



ignorelist=set( line.strip() for line in open('ignorelist'))

for word in file.read():
    word = word.replace(".","")
    word = word.replace(",","")

    if word not in ignorelist:
        if word not in dict:
            dict[word] = 1
            dict[word] += 1


for word, count in d.most_common(10):
    print(word, ": ", count)

Why does this happen?

Asked By: dzitrus



Could you try with this change, by specifying the encoding:

file=open('xyz.txt', encoding='utf8')

(The ignorelist file may need it too )

Answered By: PRMoureu