Is there a library for parsing through JSON data using Spotipy?
Question:
I’ve been using Spotipy as my first intermediate project, and when I grab data from my account, it gives me a bunch of data in JSON format. It’s possible I’m just doing it all wrong, but from what I see, it’s really difficult to grab more than 50 of any given item from a Spotify user using the Web API. This limitation is easy to get passed, just by using multiple API calls with an offset.
My goal is to have a decently readable dictionary so I can grab any given data that I want. This is the dictionary setup I have so far, and I’m planning on making it bigger
spot_obj = spotipy.Spotify(auth=token_info['access_token'])
spot_dict['user'] = spot_obj.current_user()
spot_dict['liked_songs'] = spot_obj.current_user_saved_tracks()
spot_dict['playlists'] = spot_obj.current_user_playlists()
spot_dict['saved_albums'] = spot_obj.current_user_saved_albums()
spot_dict['num_of_liked_songs'] = spot_obj.current_user_saved_tracks()['total']
However, when I try to grab more than 50 items, by making a for loop with an offset, the newly edited JSON file has different formatting such that the 51+ songs and on are somewhere else. The following is an edited version of the code in a response here: Limits for Spotipy?
tracks = []
for x in range(1, 10):
response = spot_obj.current_user_saved_tracks(limit=50,offset=x * 50)
if len(response) == 0:
break
tracks.append(response)
return tracks
The problem here is that the tracks.append(response) makes a new list (array? it’s separated by []). Such that if I want to grab the second iteration, I need to make a new for-loop to access it. This is really inefficient and clunky. (also the if statement never becomes true because Spotify will always return something, it’s just that the fields will be empty)
Essentially, I’m asking to for a direction on how to scrub through JSON data easier, and I’m not sure what how the json built-in would help. I’m sure it would somewhere, but I don’t think it’s an encoding issue at all, Python sees all this as a dictionary, list, string, etc. What would be best is an easily accessible module that makes parsing though this, and getting around the ’50 limitation’ easier. I can’t find anything that does this, and if I figure it out I’d be more than glad to make it myself and post it, I just need a pointer
This is how I’ve been formatting the data that I want, and this is why I need all the data to be accessible in one place
song_list = []
for track in spot_dict['liked_songs']['items']:
song = track['track']['name']
artist = track['track']['album']['artists'][0]['name']
album = track['track']['album']['name']
release = track['track']['album']['release_date']
song_tup = (song, artist, album, release)
song_list.append(song_tup)
Answers:
Use extend() rather than append() to concatenate lists, rather than creating nested lists.
The list of tracks is in the items key of the response, so concatenate those lists.
tracks.extend(response['items'])
edit: if anyone cares i made a pagination tool for this https://github.com/nate-llc/spotipy-pagination
I got it working.
def songs(spot_obj):
total = response = spot_obj.current_user_saved_tracks(limit=1)['total']
iterations = (int((total / 50)) + (total % 50 > 0))
tracks = []
for x in range(iterations):
response = spot_obj.current_user_saved_tracks(limit=50,offset=x * 50)
tracks.append(response)
#looks inside of the tracks list, yoinks the ['items'], then adds them to one list
song_list = []
for iteration in tracks:
for x in range(len(iteration['items'])):
song_list.append(iteration['items'][x])
return song_list
songs(spot_obj = spotipy.Spotify(auth=token_info['access_token'])
so this assumes you already have the spotipy user object. then we look inside of the different iterations of current_user_saved_tracks(), take the [‘items’] values and add them to one bigger, combined list. (if this is confusing to you, look at the formatting json of current_user_saved_tracks())
we get the iterations by looking at the total number of songs, dividing that number by 50, then round up.
otherwise, i think if someone were to run this code they would see why it’s useful. if anyone has questions, feel free to dm me or whatever you do on this site
I’ve been using Spotipy as my first intermediate project, and when I grab data from my account, it gives me a bunch of data in JSON format. It’s possible I’m just doing it all wrong, but from what I see, it’s really difficult to grab more than 50 of any given item from a Spotify user using the Web API. This limitation is easy to get passed, just by using multiple API calls with an offset.
My goal is to have a decently readable dictionary so I can grab any given data that I want. This is the dictionary setup I have so far, and I’m planning on making it bigger
spot_obj = spotipy.Spotify(auth=token_info['access_token'])
spot_dict['user'] = spot_obj.current_user()
spot_dict['liked_songs'] = spot_obj.current_user_saved_tracks()
spot_dict['playlists'] = spot_obj.current_user_playlists()
spot_dict['saved_albums'] = spot_obj.current_user_saved_albums()
spot_dict['num_of_liked_songs'] = spot_obj.current_user_saved_tracks()['total']
However, when I try to grab more than 50 items, by making a for loop with an offset, the newly edited JSON file has different formatting such that the 51+ songs and on are somewhere else. The following is an edited version of the code in a response here: Limits for Spotipy?
tracks = []
for x in range(1, 10):
response = spot_obj.current_user_saved_tracks(limit=50,offset=x * 50)
if len(response) == 0:
break
tracks.append(response)
return tracks
The problem here is that the tracks.append(response) makes a new list (array? it’s separated by []). Such that if I want to grab the second iteration, I need to make a new for-loop to access it. This is really inefficient and clunky. (also the if statement never becomes true because Spotify will always return something, it’s just that the fields will be empty)
Essentially, I’m asking to for a direction on how to scrub through JSON data easier, and I’m not sure what how the json built-in would help. I’m sure it would somewhere, but I don’t think it’s an encoding issue at all, Python sees all this as a dictionary, list, string, etc. What would be best is an easily accessible module that makes parsing though this, and getting around the ’50 limitation’ easier. I can’t find anything that does this, and if I figure it out I’d be more than glad to make it myself and post it, I just need a pointer
This is how I’ve been formatting the data that I want, and this is why I need all the data to be accessible in one place
song_list = []
for track in spot_dict['liked_songs']['items']:
song = track['track']['name']
artist = track['track']['album']['artists'][0]['name']
album = track['track']['album']['name']
release = track['track']['album']['release_date']
song_tup = (song, artist, album, release)
song_list.append(song_tup)
Use extend() rather than append() to concatenate lists, rather than creating nested lists.
The list of tracks is in the items key of the response, so concatenate those lists.
tracks.extend(response['items'])
edit: if anyone cares i made a pagination tool for this https://github.com/nate-llc/spotipy-pagination
I got it working.
def songs(spot_obj):
total = response = spot_obj.current_user_saved_tracks(limit=1)['total']
iterations = (int((total / 50)) + (total % 50 > 0))
tracks = []
for x in range(iterations):
response = spot_obj.current_user_saved_tracks(limit=50,offset=x * 50)
tracks.append(response)
#looks inside of the tracks list, yoinks the ['items'], then adds them to one list
song_list = []
for iteration in tracks:
for x in range(len(iteration['items'])):
song_list.append(iteration['items'][x])
return song_list
songs(spot_obj = spotipy.Spotify(auth=token_info['access_token'])
so this assumes you already have the spotipy user object. then we look inside of the different iterations of current_user_saved_tracks(), take the [‘items’] values and add them to one bigger, combined list. (if this is confusing to you, look at the formatting json of current_user_saved_tracks())
we get the iterations by looking at the total number of songs, dividing that number by 50, then round up.
otherwise, i think if someone were to run this code they would see why it’s useful. if anyone has questions, feel free to dm me or whatever you do on this site