TIP: around 12:17 when you run the API runner for the first time, I had to replace "df = df2.append(df2)" with df_append = pd.DataFrame(df2) df = pd.concat([df2,df_append]) because frame.append is deprecated; new format is panda.concat. I kept getting an annoying little warning message that I didn't want to show every time the runner was successful. I typed this out on my own after a little research and I have to give a HUGE shoutout to Alex for the videos. I am FAR from versed in Python.... Just a little research and trial and error. Hope this helps someone!
And at 14:25 when the error is detected & corrected (LOL), here's the update I made to the correction: df_append = pd.DataFrame(df2) df = pd.concat([df,df_append]) Worked perfectly.
@@krystlestevens2585 Really helpful! Thank you. One question: Every time I run api_runner() it still prints out the unorganized data. Were you able to just print 'API Runner Complete" without all the data each time? Here is my code so you can see if I messed anything up: df2 = pd.json_normalize(data['data']) df2['timestamp'] = pd.to_datetime('now') df_append = pd.DataFrame(df2) df = pd.concat([df2, df_append]) import os from time import time from time import sleep for i in range(333): api_runner() print('API Runner Fetch Completed') sleep(30) exit()
Well 27:25, another way you can achieve removing scientific notation is by pd.options.display.float_format = '{:,.2f}'.format and just hit shift+enter in jupyter.
very good content Alex. Keep it up. suggestion: the reset_index() will automatically generated sequential numbers so there's no need to created number sequence for indexing purpose. directly call the reset_index() function
These sort of video's are exactly what I'm looking for!!! Don't have time to go throught them at the moment, but am bookmarking them for the weekend! Keep it up!
00:01 Automating data collection from a crypto API using Python 02:38 Adding timestamp to automation scripts for tracking run time 08:25 Automating the process of appending data frames and creating a loop for automatic execution. 11:32 Automating API calls to run every minute 17:03 Automating data export to CSV from API response using Python 19:48 Automating file creation and data appending using Python 25:30 Exploring data visualization and transformation using Python for crypto website API 28:27 Grouping data in Python using Pandas groupby function 34:25 Convert series to data frame in Python 37:13 Resetting index for proper data formatting 43:14 Updating and visualizing crypto data for different time frames 46:08 Creating Dataframe and Querying Data in Python for Cryptocurrency Prices Crafted by Merlin AI.
I had the same issue, I used df7.query instead of df10.query and then all of the values appeared. So instead of df10=df10.query("name=='Bitcoin'") try df10=df7.query("name=='Bitcoin'") and see if that works.
I plan on doing some larger, full scale projects once I finish the Python beginner tutorial series. I'll definitely be making more complex, challenging projects for people to try out :)
Alex, can u please put the "How to use a Public API" video on the bootcamp? I was searching for it and I couldn't find and being frustrated cause I couldn't understand where this video was,since I followed all the bootcamp videos until this one,and I didn't go through it. Thanks God u have shown the link above, and in a moment I saw it.
Alex, great video as usual. I do have a topic for you if you like to discuss in another video. How much statistics is required for a data analyst to know? Maybe do a comparison to a data scientist
Hey Alex, It was a great video and I am gonna tweak and experiment a bit with it on my end. I have a question though. While automating the script, how are we gonna update our API key? Can we use just one API key 333 times a day and we'll be getting fresh updated data every time our script runs? Or are we gonna get 333 different API keys for each time our script runs? In either case, how would we be updating our API key?
sns.set_theme(style= 'darkgrid') # or sns.set_style('darkgrid') axes = sns.lineplot(x='timestamp', y='quote.USD.price', data=df11) axes.set_xticklabels(ax.get_xticklabels(), rotation=45); 🙌🏽 That sorted it out the messy x axis but you may get a warning message.
I don't know how many hours you spend preparing each of these but I am sure it is a non-trivial amount! I am curious, how much Python do you use on a day-to-day basis?
Hi Alex, great video. I am working on this project while the video plays but it seems as though I reach the daily credit limit after running the script for 4 completions. Do you have any suggestions on how to resolve this?
UserWarning: Creating legend with loc="best" can be slow with large amounts of data. self._figure.tight_layout(*args, **kwargs) i am getting this error on 41:09 . Help me if anybody knows please .
Hello Alex. I'm Godfred and i just join your channel cos your doing extremely well with the education but my problem is not about this particular video but Excel. Pls I wanna know if there is any way I can learn the excel vidoes without laptop? cos I was finding it difficult in understanding and yet and I'm also interested in the course.😥
hey there I am a stay at home mom and would love to teach myself everything there is to learn to in data and possibly in the near future get a data analyst job. If you could recommend a platform or just 1 program could you please tell me which one you would recommend and why? it is very overwhelming seeing all these options online, I have also read many reviews of people who did the google cert and didn;t get any luck getting a entry level job as an analyst . I plan on choosing a platform and your videos for practice. Thank you in advance
I'd probably look at Coursera - if you're just starting out you can look at the IBM Data Analyst specialization or the Google Data Analyst Certification on Coursera. Both are pretty good for beginners and have a good path to learn.
@Alex The Analyst hi I need your help I was going through your SQL based project in this series but can you tell how to upload that to my resume and github
what is wrong im not able to understand. import os from time import time from time import sleep for i in range(333): api_runner() print('API Runner completed') sleep(60) #sleep for 1 minute exit() stuck in this part again and again. AttributeError: 'DataFrame' object has no attribute 'append', this is error i get everytime
I got an error here --> sns.lineplot(x='timestamp', y='quote.USD.price', data = df8) because: ValueError Traceback (most recent call last) Input In [48], in () 1 sns.set_theme(style='darkgrid') ----> 2 sns.lineplot(x='timestamp', y='quote.USD.price', data = df8) File ~\anaconda3\lib\site-packages\seaborn\_decorators.py:46, in _deprecate_positional_args..inner_f(*args, **kwargs) 36 warnings.warn( 37 "Pass the following variable{} as {}keyword arg{}: {}. " 38 "From version 0.12, the only valid positional argument " (...) 43 FutureWarning 44 ) 45 kwargs.update({k: arg for k, arg in zip(sig.parameters, args)}) ---> 46 return f(**kwargs) what should I do
@@barkamolurinboev6311 thanks for your help, where I have to reset it, bc only works like one time and no more, even I'm resetting, but for sure another question is which one df I have to reset? Thanks.
Thanks, @AlexTheAnalyst! I love your bootcamp! I also have a trouble: df10 = df[['name','quote.USD.price','timestamp']] df10 = df10.query("name == 'Bitcoin'") df10. You have 5 outputs, but I always have only one: name quote.USD.price timestamp 0 Bitcoin 30619.117651 2023-07-11 23:18:07.626573+00:00 Can you advise me please, how to fix it? Thanks!
TIP: around 12:17 when you run the API runner for the first time, I had to replace "df = df2.append(df2)" with
df_append = pd.DataFrame(df2)
df = pd.concat([df2,df_append])
because frame.append is deprecated; new format is panda.concat. I kept getting an annoying little warning message that I didn't want to show every time the runner was successful.
I typed this out on my own after a little research and I have to give a HUGE shoutout to Alex for the videos. I am FAR from versed in Python.... Just a little research and trial and error. Hope this helps someone!
And at 14:25 when the error is detected & corrected (LOL), here's the update I made to the correction:
df_append = pd.DataFrame(df2)
df = pd.concat([df,df_append])
Worked perfectly.
thank you! this helps me so much!
@@krystlestevens2585 thank you for these tips, I'm also experiencing the same thing
@@krystlestevens2585 Really helpful! Thank you. One question: Every time I run api_runner() it still prints out the unorganized data. Were you able to just print 'API Runner Complete" without all the data each time? Here is my code so you can see if I messed anything up:
df2 = pd.json_normalize(data['data'])
df2['timestamp'] = pd.to_datetime('now')
df_append = pd.DataFrame(df2)
df = pd.concat([df2, df_append])
import os
from time import time
from time import sleep
for i in range(333):
api_runner()
print('API Runner Fetch Completed')
sleep(30)
exit()
Thank youuu
Great demo Alex. As someone who is nearing the end of my DA degree and starting to put together a portfolio this really helped!
Well 27:25, another way you can achieve removing scientific notation is by
pd.options.display.float_format = '{:,.2f}'.format
and just hit shift+enter in jupyter.
very good content Alex. Keep it up.
suggestion: the reset_index() will automatically generated sequential numbers so there's no need to created number sequence for indexing purpose. directly call the reset_index() function
I am always smiling when Alex makes mistakes keep them in the videos, because I make them too. Same for spelling mistakes. Haha. :)
Spelling things is my weakness :D
These sort of video's are exactly what I'm looking for!!! Don't have time to go throught them at the moment, but am bookmarking them for the weekend!
Keep it up!
fantastic, even if i didn't see it. coz you're the one who is doing great videos.
This is extremely insightful, Alex! Great video and follow up from last week's video!
This is very helpful Alex. Thanks and lots of love from India.
Thank you for this, dear unknown person.
Awesome video! You explain everything very clearly. I also dig you are showing the errors!
You are the man! 🤜🤛
Love your content! You motivated me to start my own Tech UA-cam channel and I hope one day it will grow like yours did. That's it ,just thank you!
cool upload Alex The Analyst. I crushed that thumbs up on your video. Keep up the very good work.
Loved this❤️💯, thank you Alex😊used to think automation was a mountain now you’ve broken that barrier 🥂
Great topic as usual, waiting...
00:01 Automating data collection from a crypto API using Python
02:38 Adding timestamp to automation scripts for tracking run time
08:25 Automating the process of appending data frames and creating a loop for automatic execution.
11:32 Automating API calls to run every minute
17:03 Automating data export to CSV from API response using Python
19:48 Automating file creation and data appending using Python
25:30 Exploring data visualization and transformation using Python for crypto website API
28:27 Grouping data in Python using Pandas groupby function
34:25 Convert series to data frame in Python
37:13 Resetting index for proper data formatting
43:14 Updating and visualizing crypto data for different time frames
46:08 Creating Dataframe and Querying Data in Python for Cryptocurrency Prices
Crafted by Merlin AI.
Thank you so much Alex, I've just completed the project successfully 👍
Thank you Alex, really great piece.
Thanks a ton, Alex! What did I do to deserve this🥺?!! How do you make things this easy!
hello, can someone tell me why do i have only 1 timestamt at 48:15 ?
I had the same issue, I used df7.query instead of df10.query and then all of the values appeared. So instead of df10=df10.query("name=='Bitcoin'") try df10=df7.query("name=='Bitcoin'") and see if that works.
ensure your dataframe(df) is updating and not commented out in the api_runner function with line: df = pd.concat([df,df_append])
Pls i need more site with APIs
Thanks alex ...u are good
Was there ever a follow up video with more on the data visualization part? Thank you! Great video!
Absolutely love this! Please tell me a webscrapping tutorial is in the works 🙏🏿🤞🏿
Hi Alex
You are doing great job. I would request you to make project on SQL and power bi (complex one, big dataset and multiple tables)
Hes already made an sql project palv
I plan on doing some larger, full scale projects once I finish the Python beginner tutorial series. I'll definitely be making more complex, challenging projects for people to try out :)
Alex, can u please put the "How to use a Public API" video on the bootcamp? I was searching for it and I couldn't find and being frustrated cause I couldn't understand where this video was,since I followed all the bootcamp videos until this one,and I didn't go through it.
Thanks God u have shown the link above, and in a moment I saw it.
its linked in the description
Alex, great video as usual. I do have a topic for you if you like to discuss in another video. How much statistics is required for a data analyst to know? Maybe do a comparison to a data scientist
Hey Alex, It was a great video and I am gonna tweak and experiment a bit with it on my end. I have a question though. While automating the script, how are we gonna update our API key? Can we use just one API key 333 times a day and we'll be getting fresh updated data every time our script runs? Or are we gonna get 333 different API keys for each time our script runs? In either case, how would we be updating our API key?
Awesome Alex!!!
Dude, you are the best :)
How does one at 48:35 get to shorten the timestamps labled at the bottom. Mine are super messy, similar to the percent change before we changed it.
sns.set_theme(style= 'darkgrid') # or sns.set_style('darkgrid')
axes = sns.lineplot(x='timestamp', y='quote.USD.price', data=df11)
axes.set_xticklabels(ax.get_xticklabels(), rotation=45); 🙌🏽
That sorted it out the messy x axis but you may get a warning message.
Just checking. Will there be a playback version of this video?
Yep - once it "Premiers" it says it will go up as a regular video just like normal.
Is there a part 2 to this video ? cheers
I don't know how many hours you spend preparing each of these but I am sure it is a non-trivial amount! I am curious, how much Python do you use on a day-to-day basis?
A 5-10 hours or so I would say - and I used to use it a lot, but in my manager role not a lot.
Help Please ! 44:16
UserWarning: The figure layout has changed to tight
self._figure.tight_layout(*args, **kwargs)
Thank you..
Hi Alex, great video. I am working on this project while the video plays but it seems as though I reach the daily credit limit after running the script for 4 completions. Do you have any suggestions on how to resolve this?
UserWarning: Creating legend with loc="best" can be slow with large amounts of data.
self._figure.tight_layout(*args, **kwargs)
i am getting this error on 41:09 . Help me if anybody knows please .
hey Alex, running the program not saved by the excel format
My kernel keeps dying while running api_runner . What should I do?
I tried many times, even tried to virtual env
Hello Alex. I'm Godfred and i just join your channel cos your doing extremely well with the education but my problem is not about this particular video but Excel. Pls I wanna know if there is any way I can learn the excel vidoes without laptop? cos I was finding it difficult in understanding and yet and I'm also interested in the course.😥
Does Transposing DataFrame help while changing the columns to rows?
Hi Alex! Can you share how to upload files from a folder using API of a website?
hey there I am a stay at home mom and would love to teach myself everything there is to learn to in data and possibly in the near future get a data analyst job. If you could recommend a platform or just 1 program could you please tell me which one you would recommend and why? it is very overwhelming seeing all these options online, I have also read many reviews of people who did the google cert and didn;t get any luck getting a entry level job as an analyst . I plan on choosing a platform and your videos for practice. Thank you in advance
I'd probably look at Coursera - if you're just starting out you can look at the IBM Data Analyst specialization or the Google Data Analyst Certification on Coursera. Both are pretty good for beginners and have a good path to learn.
anyone getting "UnboundLocalError: cannot access local variable 'df' where it is not associated with a value" when running the for loop?
Yess
@Alex The Analyst hi I need your help I was going through your SQL based project in this series but can you tell how to upload that to my resume and github
Can someone explain why it is range(333) given? I mean how to calculate this 333
It is the daily limit usage of the API key.
what is wrong im not able to understand. import os
from time import time
from time import sleep
for i in range(333):
api_runner()
print('API Runner completed')
sleep(60) #sleep for 1 minute
exit() stuck in this part again and again. AttributeError: 'DataFrame' object has no attribute 'append', this is error i get everytime
Refer to the comment above, append is no longer available
Use concat
Df= pd.concat([df,df_append])
My Seaborn catplot is not showing, please what can I do?
df = df.reset_index()
My lineplot didn’t make a line on the graph 🥲 any ideas on how to fix this? 🙏🏼
ensure your dataframe(df) is updating and not commented out in the api_runner function with line: df = pd.concat([df,df_append])
Can you please do pyspark tutorials for us jus like you did SQL @alex
What the Heyll!?!?
okay, bald!! disliked
with peace and love
I got an error here --> sns.lineplot(x='timestamp', y='quote.USD.price', data = df8)
because:
ValueError Traceback (most recent call last)
Input In [48], in ()
1 sns.set_theme(style='darkgrid')
----> 2 sns.lineplot(x='timestamp', y='quote.USD.price', data = df8)
File ~\anaconda3\lib\site-packages\seaborn\_decorators.py:46, in _deprecate_positional_args..inner_f(*args, **kwargs)
36 warnings.warn(
37 "Pass the following variable{} as {}keyword arg{}: {}. "
38 "From version 0.12, the only valid positional argument "
(...)
43 FutureWarning
44 )
45 kwargs.update({k: arg for k, arg in zip(sig.parameters, args)})
---> 46 return f(**kwargs)
what should I do
df = df.reset_index()
@@barkamolurinboev6311 thanks for your help, where I have to reset it, bc only works like one time and no more, even I'm resetting, but for sure another question is which one df I have to reset? Thanks.
Thanks, @AlexTheAnalyst! I love your bootcamp!
I also have a trouble:
df10 = df[['name','quote.USD.price','timestamp']]
df10 = df10.query("name == 'Bitcoin'")
df10.
You have 5 outputs, but I always have only one:
name quote.USD.price timestamp
0 Bitcoin 30619.117651 2023-07-11 23:18:07.626573+00:00
Can you advise me please, how to fix it? Thanks!