UnicodeDecodeError when reading file in Python

Question

I am currently learning Python and came across the following error:

Traceback (most recent call last):   
  File "file.py", line 22, in module
    for word in file.read():
  File "C:UsersuserAppDataLocalContinuumAnaconda3libencodingscp1252.py" line 23, in decode
    return codecs.charmap_decode(input,self.errors,decoding_table)[0] 
UnicodeDecodeError: 'charmap' codec can't decode byte 0x9d in position 6552: character maps to undefined

This is my code:

file=open('xyz.txt')

dict={}

ignorelist=set( line.strip() for line in open('ignorelist'))

for word in file.read():
    word = word.replace(".","")
    word = word.replace(",","")

    if word not in ignorelist:
        if word not in dict:
            dict[word] = 1
        else:
            dict[word] += 1

d=collections.Counter(dict)

for word, count in d.most_common(10):
    print(word, ": ", count)

Why does this happen?

Asked By: dzitrus

||

Source

Answer 1

Could you try with this change, by specifying the encoding:

file=open('xyz.txt', encoding='utf8')

(The ignorelist file may need it too )

Answered By: PRMoureu

UnicodeDecodeError when reading file in Python

Question:

Answers: