JSON from webpage into Python script: urllib.error.HTTPError: HTTP Error 403: Forbidden

Question

I made a program and it works with a local json (data) file!

Code Block:

def datas(self):        
        
        with open ("C:\Users\Messi\Desktop\Python\\tek.json", "r") as dosya:            
            dataApi = json.load(dosya)     
        return dataApi

I uploaded this data json to a website which is a Lemp Server!
https://bestpurpleshampoo.com/tek.json

I changed the code block to the:

import urllib.request, json 
with urllib.request.urlopen("https://bestpurpleshampoo.com/tek.json") as url:
    dataApi= json.load(url)
return dataApi

There is an error:

    Traceback (most recent call last):
   File "c:UsersMessiDesktopPythonProjectsFootball TipsKivymain.py", line 150, in <module>
     BetpredictorApp().run()
   File "C:UsersMessianaconda3envskivyEnvlibsite-packageskivyapp.py", line 954, in run
     self._run_prepare()
   File "C:UsersMessianaconda3envskivyEnvlibsite-packageskivyapp.py", line 923, in _run_prepare
     self.load_kv(filename=self.kv_file)
   File "C:UsersMessianaconda3envskivyEnvlibsite-packageskivyapp.py", line 696, in load_kv
     root = Builder.load_file(rfilename)
   File "C:UsersMessianaconda3envskivyEnvlibsite-packageskivylangbuilder.py", line 305, in load_file
     return self.load_string(data, **kwargs)
   File "C:UsersMessianaconda3envskivyEnvlibsite-packageskivylangbuilder.py", line 403, in load_string
     widget = Factory.get(parser.root.name)(__no_builder=True)
   File "c:UsersMessiDesktopPythonProjectsFootball TipsKivymain.py", line 16, in __init__
     self.data = self.datas()
   File "c:UsersMessiDesktopPythonProjectsFootball TipsKivymain.py", line 134, in datas
     with urllib.request.urlopen("https://bestpurpleshampoo.com/tek.json") as url:
   File "C:UsersMessianaconda3envskivyEnvliburllibrequest.py", line 214, in urlopen
     return opener.open(url, data, timeout)
   File "C:UsersMessianaconda3envskivyEnvliburllibrequest.py", line 523, in open
     response = meth(req, response)
   File "C:UsersMessianaconda3envskivyEnvliburllibrequest.py", line 632, in http_response
     response = self.parent.error(
   File "C:UsersMessianaconda3envskivyEnvliburllibrequest.py", line 561, in error
     return self._call_chain(*args)
   File "C:UsersMessianaconda3envskivyEnvliburllibrequest.py", line 494, in _call_chain
     result = func(*args)
   File "C:UsersMessianaconda3envskivyEnvliburllibrequest.py", line 641, in http_error_default
     raise HTTPError(req.full_url, code, msg, hdrs, fp)
 urllib.error.HTTPError: HTTP Error 403: Forbidden

Also:
https://www.pythonpool.com/urllib-error-httperror-http-error-403-forbidden/

from urllib import request
from urllib.request import Request, urlopen     
url = "https://bestpurpleshampoo.com/tek.json"
request_site = Request(url, headers={"User-Agent": "Mozilla/5.0"})
webpage = urlopen(request_site).read()
print(webpage[:500])

In above solution i got:

Traceback (most recent call last):
   File "c:UsersMessiDesktopPythonProjectsFootball TipsKivymain.py", line 151, in <module>
     BetpredictorApp().run()
   File "C:UsersMessianaconda3envskivyEnvlibsite-packageskivyapp.py", line 954, in run
     self._run_prepare()
   File "C:UsersMessianaconda3envskivyEnvlibsite-packageskivyapp.py", line 923, in _run_prepare
     self.load_kv(filename=self.kv_file)
   File "C:UsersMessianaconda3envskivyEnvlibsite-packageskivyapp.py", line 696, in load_kv
     root = Builder.load_file(rfilename)
   File "C:UsersMessianaconda3envskivyEnvlibsite-packageskivylangbuilder.py", line 305, in load_file
     return self.load_string(data, **kwargs)
   File "C:UsersMessianaconda3envskivyEnvlibsite-packageskivylangbuilder.py", line 403, in load_string
     widget = Factory.get(parser.root.name)(__no_builder=True)
   File "c:UsersMessiDesktopPythonProjectsFootball TipsKivymain.py", line 16, in __init__
     self.Today = self.todayMatches()
   File "c:UsersMessiDesktopPythonProjectsFootball TipsKivymain.py", line 124, in todayMatches
     for i in range(len(self.data['today']["home"])):
 TypeError: byte indices must be integers or slices, not str

How possible is it that this json is works perfect locally! Team names are STR and stats are also integer! But with above method that i get error

I tried some other methods in but not tried all!
How to get JSON from webpage into Python script

Could you please help me to fix this 🙂
I have 500 hours on Python that i like it but stucked here

Thanks

Asked By: Mecra Yavçın

||

Source

Answer 1

I’m not 100% sure I understand what the problem is, but if I use this code it works fine from my system:

import requests
url = "https://bestpurpleshampoo.com/tek.json"
response = requests.get(url).json()
print(response['today']['home'])

Result:

['Ajax', 'Liverpool', 'Club Brugge', 'FC Porto'... truncated ...]

If you need to install requests library, use: pip install requests

Answered By: user56700

JSON from webpage into Python script: urllib.error.HTTPError: HTTP Error 403: Forbidden

Question:

Answers: