Python request.get error on Wikipedia image URL

Question

Requests.get() does not seem to be returning the expected bytes for Wikipedia image URLs, such as https://upload.wikimedia.org/wikipedia/commons/0/05/20100726_Kalamitsi_Beach_Ionian_Sea_Lefkada_island_Greece.jpg:

import wikipedia
import requests

page = wikipedia.page("beach")
first_image_link = page.images[0]
req = requests.get(first_image_link)
req.content

b'<!DOCTYPE html>n<html lang="en">n<meta charset="utf-8">n<title>Wikimedia Error</title>n<style>n*...

Asked By: placeholder

||

Source

Answer 1

I typed your code and it seems to be an "Error: 403, Forbidden.". Wikipedia requires a user agent header in the request.

import wikipedia
import requests

headers = {
    'User-Agent': 'My User Agent 1.0'
}

page = wikipedia.page("beach")
first_image_link = page.images[0]
req = requests.get(first_image_link, headers=headers, stream=True)
req.content

For the user agent, you should probably supply something a bit more descriptive than the placeholder I use in my example. Maybe the name of your script, or just the word "script" or something like that.
I tested it and it works fine. You will get back the image as you are expecting.

Answered By: Nova

Answer 2

Most websites block requests that come in without a valid browser as a User-Agent. Wikimedia is one such.

import requests
headers={'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.0.0 Safari/537.36'}
res = requests.get('https://upload.wikimedia.org/wikipedia/commons/0/05/20100726_Kalamitsi_Beach_Ionian_Sea_Lefkada_island_Greece.jpg', headers=headers)
res.content

which will give you expected output

Answered By: Himanshuman

Python request.get error on Wikipedia image URL

Question:

Answers: