Webscrapping with bs4: Type Error – 'NoneType' object is not subscriptable

Question

I am trying to extract "video" from the url and print how many there are in the console.

But I get this Error:

TypeError: 'NoneType' object is not subscriptable

Here is my code:

import requests
from bs4 import BeautifulSoup

Web_url = "https://watch.plex.tv/show/hannibal/season/1/episode/9"
r = requests.get(Web_url)

soup = BeautifulSoup(r.content, 'html.parser')

video_tags = soup.find_all("video")
print("Total", len(video_tags), "videos found")

if len(video_tags) !=0:
    for video_tag in video_tags:
        video_url = video_tag.find("a")['href']
        print(video_url)```

Asked By: Suikurix

||

Source

Answer 1

You’re probably getting the error on the video_url = video_tag.find("a")['href'] line, since I cannot see any other place where you’re indexing, which is what can cause this error. So, the error would happen because video_tag.find("a") returns None. You should wrap it for some error handling like:

for video_tag in video_tags:
    video_hl = video_tag.find("a")
    if video_hl is None:
        print("No hyperlinks found inside the video tag", video_tag.attrs)
        continue
    video_url = video_hl['href']
    print(video_url)

(Printing the video_tag.attrs can help you find or select that video element for further inspection.)

[Actually, I can’t get to that line when I run your code because I get Total 0 videos found so the for-loop never runs; but when I open and inspect the plex link you’re using, the only video tag I can find is the "Watch Now" button (which does not get scraped when I run your code, for some reason), and that does not have any a tags in it – nor do any of the several other videos I inspected on the plex site.]

If you’re trying to get the video source link rather that hyperlinks that will show up when a browser doesn’t support HTML video, you can get the src attribute with video_src = video_tag.get('src')

Answered By: Driftr95

Webscrapping with bs4: Type Error – 'NoneType' object is not subscriptable

Question:

Answers: