using try-except block to scrape data from twitter api

Question

I’m scraping data from twiter with using their tweet ids but some of the tweets have been deleted so my code throws an error when it gets to those tweets ids

how do I use "try-except" to just skip those tweet ids or populate the with NaN
my code below:

tweet_ids = twitter_archive['tweet_id']

tweet_id = []
likes = []
retweets = []

try:
    for ids in tweet_ids:
        tweet = api.get_status(ids)
        tweet_id.append(ids)
        likes.append(tweet.favorite_count)
        retweets.append(tweet.retweet_count)
except:
     #I need help with the except block
tweet_info = pd.DataFrame({'tweet_id': tweet_id, 'likes': likes, 'retweets': retweets})

Asked By: Khola

||

Source

Answer 1

If you want to skip those tweets. You can add try-except inside the loop for checking tweets as a better practice. Reason to add this inside the loop is that if you create a list for the missing tweets as well, you can add the missing IDs there easily.

failed_tweets = []

for ids in tweet_ids:
    try:
        tweet = api.get_status(ids)
        tweet_id.append(ids)
        likes.append(tweet.favorite_count)
        retweets.append(tweet.retweet_count)
    except:
        failed_tweets.append(ids)

This is not going to raise an error if tweet is not found and it will continue to loop through other tweets without a problem. At the same time, iti s going to save failed tweets to a different list called failed_tweets.

Answered By: g14u

using try-except block to scrape data from twitter api

Question:

Answers: