Python Get Request Loop

Question

hopefully there is a kind soul out there which will offer to graciously answer my question as I continue my learning experience.

I have written the following simple python code to bring back a sporting web site I would like to scrape.

I want to be able to scrape from race-1 to race X without having to manually change the url. So I was wondering is it possible to include a loop to scrape all races from 1 to end and consolidate in one print file.

import requests

url = "https://s3-ap-southeast-2.amazonaws.com/racevic.static/2018-01-01/flemington/sectionaltimes/race-1.json?callback=sectionaltimes_callback"

payload={}
headers = {}

response = requests.request("GET", url, headers=headers, data=payload)

print(response.text)

Asked By: Learner

||

Source

Answer 1

Use string formatting to insert the current race number into the URL.

import requests
payload = {}
headers = {}

for race in range(1, X+1):
    url = f"https://s3-ap-southeast-2.amazonaws.com/racevic.static/2018-01-01/flemington/sectionaltimes/race-{race}.json?callback=sectionaltimes_callback"
    response = requests.request("GET", url, headers=headers, data=payload)
    json_string = re.sub(r'^sectionaltimes_callback((.*))$', r'1', response.text)
    data = json.loads(json_string)
    print(data)

Answered By: Barmar

Python Get Request Loop

Question:

Answers: