why can't it not find this cookie?

Question

I want to scrape this website: https://dbh.smartschool.be/ for a school project but I always run into a problem whit the authentication and I have no cleu why.
this is my code:

import requests

URL = "https://dbh.smartschool.be"
LOGIN_ROUTE = "/login"

HEADERS = {
   "User-Agent" : "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36" ,
   "orgin" : URL,
   "referer" : URL + "/"
}

s = requests.session()

csrf_token = s.get(URL).cookies["PHPSESSID"]

login_payload ={
   "login_form[_username]": "Roel.bellemans",
   "login_form[_password]": "######",
   "login_form[_generationTime]": "1665576878",
   "login_form[_token]": csrf_token
}

login_req = s.post(URL + LOGIN_ROUTE, headers=HEADERS, data=login_payload)
print(login_req)

and this is what I get back:

Traceback (most recent call last):
  File "C:UsersroelbDesktopcookie_auth_scraperlogIn.py", line 15, in <module>
    csrf_token = s.get(URL).cookies["PHPSESSID"]
  File "C:UsersroelbAppDataLocalProgramsPythonPython310libsite-packagesrequestscookies.py", line 334, in __getitem__        
    return self._find_no_duplicates(name)
  File "C:UsersroelbAppDataLocalProgramsPythonPython310libsite-packagesrequestscookies.py", line 413, in _find_no_duplicates
    raise KeyError(f"name={name!r}, domain={domain!r}, path={path!r}")
KeyError: "name='PHPSESSID', domain=None, path=None"

but there is a PHPSESSID

any thoughts?

Asked By: Roel Bellemans

||

Source

Answer 1

Your cookie issue falls on this line:

csrf_token = s.get(URL).cookies["PHPSESSID"]   # CAUSES ERROR

The cookie issue can be fixed with this:

import requests
URL = "https://dbh.smartschool.be"
s = requests.Session()
response = s.get(URL)
csrf_token = s.cookies.get_dict()['PHPSESSID']
print(csrf_token)

OUTPUT:

0ss1g37jsa5sg2sd2beu91vnqc8vamv3bmi03lmfc0p9c8kai3

If I leave out PHPSESSID : and print the dictionary instead like this:

csrf_token = s.cookies.get_dict()
print(csrf_token)

OUTPUT:

{'PHPSESSID': '0ss1g37jsa5sg2sd2beu91vnqc8vamv3bmi03lmfc0p9c8kai3', 'pid': 'e2163707-0803-49c8-af39-f8c499508845'}

Answered By: ScottC

why can't it not find this cookie?

Question:

Answers: