From yesterday i'm facing the issue of snscrape with twitter
Question:
ScraperException: 4 requests to https://api.twitter.com/2/search/adaptive.json?include_profile_interstitial_type=1&include_blocking=1&include_blocked_by=1&include_followed_by=1&include_want_retweets=1&include_mute_edge=1&include_can_dm=1&include_can_media_tag=1&skip_status=1&cards_platform=Web-12&include_cards=1&include_ext_alt_text=true&include_quote_count=true&include_reply_count=1&tweet_mode=extended&include_entities=true&include_user_entities=true&include_ext_media_color=true&include_ext_media_availability=true&send_error_codes=true&simple_quoted_tweets=true&q=%28from%3AZeeNewsEnglish%29+until%3A2023-01-12+since%3A2023-01-08+-filter%3Areplies&count=100&query_source=spelling_expansion_revert_click&pc=1&spelling_corrections=1&ext=mediaStats%2ChighlightedLabel failed, giving up.
I tried following code :
import snscrape.modules.twitter as sntwitter
import time
query5 = "(from:BBC) until:2023-01-12 since:2023-01-08 -filter:replies"
news = [query5]
tweets = []
for news_data in news:
limit = 500
for tweet in sntwitter.TwitterSearchScraper(news_data).get_items():
# print(vars(tweet))
# break
if len(tweets) == limit:
break
else:
tweets.append([tweet.date, tweet.username, tweet.content])
time.sleep(2)
Answers:
Faced the same issue. I guess snscraper, made use of Twitter API (elevated access) in the backend. Twitter shut down all the bots which were fetching the API data. Twitter essentially wants you to make authentic use of their data. I suggest signing up on twitter’s developer account and requesting the elevated environment. Notice the first line in error makes a call to Twitter API.
you have to install the last version of snscrape 0.5.0.20230113.
ScraperException: 4 requests to https://api.twitter.com/2/search/adaptive.json?include_profile_interstitial_type=1&include_blocking=1&include_blocked_by=1&include_followed_by=1&include_want_retweets=1&include_mute_edge=1&include_can_dm=1&include_can_media_tag=1&skip_status=1&cards_platform=Web-12&include_cards=1&include_ext_alt_text=true&include_quote_count=true&include_reply_count=1&tweet_mode=extended&include_entities=true&include_user_entities=true&include_ext_media_color=true&include_ext_media_availability=true&send_error_codes=true&simple_quoted_tweets=true&q=%28from%3AZeeNewsEnglish%29+until%3A2023-01-12+since%3A2023-01-08+-filter%3Areplies&count=100&query_source=spelling_expansion_revert_click&pc=1&spelling_corrections=1&ext=mediaStats%2ChighlightedLabel failed, giving up.
I tried following code :
import snscrape.modules.twitter as sntwitter
import time
query5 = "(from:BBC) until:2023-01-12 since:2023-01-08 -filter:replies"
news = [query5]
tweets = []
for news_data in news:
limit = 500
for tweet in sntwitter.TwitterSearchScraper(news_data).get_items():
# print(vars(tweet))
# break
if len(tweets) == limit:
break
else:
tweets.append([tweet.date, tweet.username, tweet.content])
time.sleep(2)
Faced the same issue. I guess snscraper, made use of Twitter API (elevated access) in the backend. Twitter shut down all the bots which were fetching the API data. Twitter essentially wants you to make authentic use of their data. I suggest signing up on twitter’s developer account and requesting the elevated environment. Notice the first line in error makes a call to Twitter API.
you have to install the last version of snscrape 0.5.0.20230113.