Response code 404 in python requests, opens normally in the browser


I’m trying to requests.get this page in python requests but get error code 404. It opens normally in the browser. What’s wrong here?

<Response [404]>
Asked By: the_oceanor



try with requests session:

import requests
from bs4 import BeautifulSoup

url = ""

headers = {'User-Agent': 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:76.0) Gecko/20100101 Firefox/76.0'}
session = requests.Session()
r = session.get(url, timeout=30, headers=headers)     



{'companyName': 'NewMedia',
 'productName': 'Shararam',
 'productVersion': '0.2',
 'dataUrl': '',
 'wasmCodeUrl': '2022.09.20.wasm.code.unityweb',
 'wasmFrameworkUrl': '2022.09.20.wasm.framework.unityweb',
 'graphicsAPI': ['WebGL 2.0', 'WebGL 1.0'],
 'webglContextAttributes': {'preserveDrawingBuffer': False},
 'splashScreenStyle': 'Dark',
 'backgroundColor': '#231F20',
 'developmentBuild': False,
 'multithreading': False,
 'unityVersion': '2019.4.21f1'}
Answered By: khaled koubaa

You should always figure out the correct headers that are needed from the server to accept your request. In my case i tried with:

headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/ Safari/537.36', 
           'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9',
           'Accept-Language': 'en-US,en;q=0.5',
           'Accept-Encoding': 'gzip, deflate, br'}

And worked. It seems that accept encoding was the key in this case.

Answered By: Juan Federico