fake_useragent module not connecting properly – IndexError: list index out of range

Question

I tried to use fake_useragent module with this block

from fake_useragent import UserAgent

ua = UserAgent()
print(ua.random)

But when the execution reached this line ua = UserAgent(), it throws this error

Traceback (most recent call last):
  File "/home/hadi/Desktop/excel/gatewayform.py", line 191, in <module>
    gate = GateWay()
  File "/home/hadi/Desktop/excel/gatewayform.py", line 23, in __init__
    ua = UserAgent()
  File "/usr/local/lib/python3.9/dist-packages/fake_useragent/fake.py", line 69, in __init__
    self.load()
  File "/usr/local/lib/python3.9/dist-packages/fake_useragent/fake.py", line 75, in load
    self.data = load_cached(
  File "/usr/local/lib/python3.9/dist-packages/fake_useragent/utils.py", line 250, in load_cached
    update(path, use_cache_server=use_cache_server, verify_ssl=verify_ssl)
  File "/usr/local/lib/python3.9/dist-packages/fake_useragent/utils.py", line 245, in update
    write(path, load(use_cache_server=use_cache_server, verify_ssl=verify_ssl))
  File "/usr/local/lib/python3.9/dist-packages/fake_useragent/utils.py", line 178, in load
    raise exc
  File "/usr/local/lib/python3.9/dist-packages/fake_useragent/utils.py", line 154, in load
    for item in get_browsers(verify_ssl=verify_ssl):
  File "/usr/local/lib/python3.9/dist-packages/fake_useragent/utils.py", line 99, in get_browsers
    html = html.split('<table class="w3-table-all notranslate">')[1]
IndexError: list index out of range

I use linux and I have installed the module using this command pip3 install fake_useragent --upgrade.

Is there any solution for this issue? if not, is there a better module to use?

Asked By: Hadi Ayoub

||

Source

Answer 1

There is a solution for this, from Github pull request #110. Basically, all you need to do is change one character in one line of the fake_useragent/utils.py source code.

To do this on your system, open /usr/local/lib/python3.9/dist-packages/fake_useragent/utils.py^† in your favorite text editor using admin privileges. Go to line 99, and change the w3

    html = html.split('<table class="w3-table-all notranslate">')[1]
#                                    ^^ change this

to ws:

    html = html.split('<table class="ws-table-all notranslate">')[1]
#                                    ^^ to this

Save the file (with admin permissions), restart your Python session, and your code should work just fine.

† To find the fake_useragent directory in which utils.py resides, run the following code:

import fake_useragent
print(fake_useragent.__file__)

For example, on my Windows laptop, this printed

'C:\Users\mattdmo\AppData\Roaming\Python\Python310\site-packages\fake_useragent\__init__.py'

so the folder to open is C:UsersmattdmoAppDataRoamingPythonPython310site-packagesfake_useragent.

Answered By: MattDMo

Answer 2

I tried UserAgent(use_cache_server=False, verify_ssl=False) but didn’t work.

Upgrading the version to 0.1.13 worked for me.

pip3 install fake-useragent==0.1.13

I tried pip install fake-useragent -U but it somehow didn’t upgrade the package correctly.

From the package owner: https://github.com/fake-useragent/fake-useragent/pull/136#issuecomment-1302431518

Answered By: Terry Tú Nguyễn

fake_useragent module not connecting properly – IndexError: list index out of range

Question:

Answers: