End-to-End Data Analysis Project 2024 | SQL & Power BI | Beginner Friendly
Вставка
- Опубліковано 2 жов 2024
- You will build an end-to-end data analysis project in less an 1 hour. This is aimed at beginners who want to understand the whole end-to-end process of data analysis. It will help you develop skills and see a typical analysis request.
You are the Data Analyst in this Story:
You will act as a data analyst helping Toman Bike Share to understand if increasing prices next year is possible. All files and data provided. Here is the workflow below
1. Build a Database
2. Develop SQL queries
3. Connect Power BI to your database
4. Build a Dashboard
5. Make Analysis Recommendations
You can find all the files here for download
github.com/Gae...
click the data file and download it as raw file.
Check me out on LinkedIn
/ gaelimholland
Video Time Stamps:
0:49 Data Analysis Request Email
2:01 Downloading and Install Microsoft SQL Server and SQL Server Studio
5:22 Setting up your Database and Tables
10:25 Developing your SQL queries
22:43 Dashboard Preview
23:49 Connect Power BI to the Database
27:44 Building the Dashboard
55:05 Answer the Analysis Question
1:00 Recommending Price Increase
Check me out on Linked In or DM me:
For up to 50% at DataCamp, check out my affiliate link:
datacamp.pxf.i...
This is phenomenal. I am an experienced Data Analyst and just clicked through some of the video, but the learning here is invaluable for people wanting to break into the field. I use Tableau in my job and some Power BI, so even I'm learning some new things. Thanks!
That is a great comment and endorsement based on your experience. This definitely motivates me to keep sharing content! Thank you!
@@absentdatamy thoughts exactly
Very important is that the per rider you have to subtract the COGS. in your video you made to much profit that makes no senes everey unit needs the same amount of COGS. (riders * price -(COGS * riders))/ 100 as profit
I think you got the profit margin formulal wrong. It should be Profit / Revenue
This is super good content! I'm a BI Analyst myself and when I was learning to be an analyst, I could not find anything like what you just did in this video.
Another masterpiece, you always deliver great content. Question, should profit margin be revenue - cogs which gives gross profit or profit margin, then divide gross profit by revenue to get profit margin%? Thank you.
Great presentation. I have stuck with SUM issue. When I perform Measure = (SUM(Query1[revenue]-SUM(Query1[profit])/SUM(Query1[profit] function, the error pops up "The SUM function only accepts a column reference as an argument." Ive tried different approaches without success. Also, there is a message "cannot find parameter [revenue] Please help.
Hi, thanks a lot for the video. i'm having some issue to calculate the revenue. everytime I try I have this message "Operand data type nvarchar is invalid for multiply operator" . i try to change the data type even if I can see number in the price riders column but then the price column is not regnized. Do you have an idea ? thank you
Pls do more of these! Your vids r super helpful for newcomers who r tryna break into DA industry.
Awesome, Please share these videos. I would love to do more. Your support is appreciated.
Hi, thank you for sharing this cool project! I use Mac M1 and have to download Azure data studio instead. Azure couldn't map the dteday column as datetime format and the files couldn't be imported. I had to format that column as string instead and it went through fine. I am wondering if there's any other way to import the file with the dteday properly imported as datetime with Azure?
For me, There is no option coming in new card to add image URL.
Here in image field it is coming like add data, when I click that it is showing all the Columns names and a search window 😢
I didn't understand how did you calculate the price settings, please clarify
Thank you so much, I'm so grateful for this. Regarding the profit margin, it should be 68.8% right? Because:
Profit Margin = (Total Profit / Total Revenue) * 100
= (10448578.9299999 / 15187365.2100005) * 100
= 68.8%
Or there is something that I am not understanding?
yeah, i was stuck there too, I thought profit margin was just profit/revenue. If the long way around it would be sum(revenue)-sumx(COGS*riders)/sum(revenue) which would still give the same as profit/revenue. I think the formula given may be wrong coz (revenue-profit)/profit seems more like a COGS margin. I believe your answer is correct
Hi Gaelim,
I hope this message finds you well.
I’m currently transitioning from an Electrical Engineering background into Data Analytics, and I must say, your videos have been incredibly helpful in providing clarity on where and how to begin this journey.
I've watched several of your tutorials, particularly those on becoming a Data Analyst, and they’ve been very insightful.
I’m based in Australia, and I’m starting with SQL Server and Power BI, aiming to master these tools as they are widely used in the majority of companies here. Would you recommend any other key skills or tools that would be valuable for me to focus on at this stage?
Thank you for your time and great content.
How did you get to format the dteday column as a datetime?
mine is importing as varchar.
This was the most awaited video thanku so much 😊❤
Young Robert Williams Ronald Thompson David
I love all your vudeos i just finished your first two Data Analyst Portfolio projects and i can't wait to do the third. That would make it my fourth data analyst portfolio project
Awesome! Thank you so much for watching the videos.
Hey, I just wanted to let you know that I found out about your channel tonight and it is amazing. There are many great data-related channels on UA-cam, but yours is pure gold. I would say this is the most practical content I have seen so far. One tip would be to switch to a better microphone because at some points, the audio quality is not great.
Thank you for this. I was able to complete this and add to my portfolio
Awesome! I am glad you completed the dashboard. Feel free to share it on social media also
Thank You For this video, As a Beginner in Data Analytics it was extreemly helpful for me, got the understanding of how a Data analytics process would be done in a company.
Glad you found it helpful
Hi, not sure if I'm the only one running into this issue but for both the bike share tables the date format changes from cell A288 downwards and excel refuses to recognize it as a date. Any help with that would be appreciated.
These are CSV, not Excel files. Follow the instructions there should not be any issues. Make sure you download the files as raw from Github
@@absentdata let me clarify. I know they are CSVs. I opened them in Excel to inspect the data. Uploaded them into Power BI using MySQL and I noticed an error with the date format. So went back into the file to try and figure out what's wrong. That's when I noticed the difference.
There are no instructions for MYSQL this is a completely different DBMS. I would require different instructions
I was having a similar issue, but just click on the link to open the files & download them somewhere on your pc w/o changing the format...I was thrown off because the files in the video are Excel but the files in the link are csv.
On another note, you can convert csv files to excel, but that's not req'd for this step.
I hope I made sense.
What a fabulous video. The sql query was very beginner level. The dashboard was relatively easy to make. Ran into a few issues in PowerBI that I had to research google to fix, but that was my issue, not due to your video.
I struggle mostly with the recommendation at the end only because I've never been asked analyze anything before. I'll have to review the numbers more so i can understand what you recommended.
The only issue I really have with it, is that the csv date days don't match. In other words, I don't think day 0 equals Sunday. However that would be data cleaning and the point of the video was dashboard project practice, which I'm grateful for.
Thanks for making this, it was super helpful. Can you make an intermediate or advanced beginner level video?
What area do you want to improve? The channel has some intermediate to advanced SQL analysis and Python videos
@@absentdata I've gotten more practice with queries and dashboards. Personally, the recommendations are my struggle. Providing the solution to the stakeholder. I feel like I can only solve so much via fake data for projects. I'll look into your other videos. Thanks for all your fantastic content.
@@newenglandnomad9405 If you are struggling with recommendations. I would advise you to start off with a actual problem statement or hypothesis. This gives you a clear path to answering stakeholder questions because this problem statement or hypothesis should be derived with the stakeholder. Then you can prove or disprove the hypothesis through analysis. Just randomly analyzing data and looking for insights is not effective at all. Here are a few videos that will help ua-cam.com/video/oWP00-NNpYE/v-deo.htmlsi=LZMwOd9S7lMNxuRk ua-cam.com/video/lzvCn8Tt5_U/v-deo.html&ab_channel=AbsentData
You can algo make those formulas using DAX in Powerbi, but its good no know how to do it in sql.
I totally agree with you.
This is a great video from beginning to end. Using the new card visual as a placeholder for the sections is a great idea. But I just wonder whether this will slow down the visualization. Can we just use the Shape for the section background?
There shouldn't be any performance issue if the value is turned off. If you did have a value there as long as it's not a calculated value that was created with DAX, no additional resources are needed. So either way no dlowdown
Great Content. Picked up good analysis strategies and query techniques
Glad to hear that
Iam unable to use riders individually 😢 tried a lot
Great vid- practical, easy to follow, applicable… thank you!
This is my second video on data from Absent Data. I am extremely grateful for the content he has been providing.
Glad you like the channel.
The measure for the profit margin should be calculated as : Measure(Profit Margin) = (sum(Query 1[revenue])- sum(Query 1[profit]))/sum(Query1[revenue]). This is because the definition of profit margin is that it is the difference between total revenue and profit all divided by total revenue, to understand what part of total revenue is real profit. We shouldn't divide it by profit itself.
Definition of revenue: Total money that you get from sales.
Definition of profit: Total money from sales after you have subtracted all business expenses(rent, electricity bills, gas, debt payments).
Thank you for the helpful tutorial.
do we have any version of SQL Server Management Studio (SSMS) for MAC users?
I'm not sure about MAC but you should be able to use it
my Price column and my COGS column from cost_table is not a float value its 399 & 124 rather than 3.99 and 1.24 what can i do to fix it. Appears at minute 20:29.
I couldn't fix it... I implement the list new and in the process i set the DATATYPE to navchar(50) now it shows correct float values BUT i cant multiply the values to revenue bbecuase of incorrect DATATYPE
in first instance i fixed it with this piece of sh...
with cte as (
select * from bike_share_yr_0
union
select * from bike_share_yr_1)
select
dteday,
season,
a.yr,
weekday,
hr,
rider_type,
riders,
price / 100 AS price,
COGS / 100 AS price,
riders * price / 100 as revenue
from cte a
left join cost_table b
on a.yr = b.yr
maybe its a bug in Microsfot SSMS
This was amazing, I learned so much.
Do you have videos on how to write a recommendation ......PLEASE 🥺
Great content always 😍. Can you please upload advanced level complete Data Analytics projects in Power BI, Excel & SQL
9:00 i did exactly like you but in results give me error about column "atemp"
i fixed this problem by allow nulls but again give me warning about 6 data deleted.
Hello,
during this project I am facing an issue to load data into power bi from sql server. I tried several ways to resolve it but unable to solve.
error occurs when I load the data into power bi in the last tab by selecting "Use Current credentials" after this step the error message is " We couldn't authenticate with the credentials provided. Please try again."
IF anyone can help me with this it will be very helpful for me.
Why do you calculate profit margin this way? Shouldn't it be (Revenue - COGS)/Revenue or simply Profit/Revenue?
Yes you can calculate it that way
@@absentdata The number is different then
@@mrbartuss1 please you will get same result if you calculate the profit by subtracting value of revenue from the COGS column. alternatively in the video he made it explicit by using the riders * price which will give you same value as the Revenue. so it's same thing.
Thank you for this video I love it. but when I try to import data, the columns in the temp and atemp sections don't have dots (.) why? even though the data type is already float. Please reply
Hello sir, in the weekday column, what days do the numbers 0 to 6 represent?
I use linux , so what should i be downloading?? can anyone help me setup everything in linux please
Very informative 🙏💯
THANK U SO MUCH FOR THE INFORMATIVE TUTORIAL .. APPRECIATE IT
My pleasure!
Please show how to connect realtime data streaming from Mysql to power bi
Shukriya Bhai is video k liye 😢،
Allama Ehsan Elahi Zaheer ko Allah tala unko jannat firdous ala darzah de
Amazing content as always
Does anyone here provide 1:1 powerbi mentoring? At a decent price
The most accurate thing here was "...Please provide this asap". 😂
Great video man 💯…was so lost in power BI but now I now I can integrate SQL, Excel and Bi all at once thanks to you
So glad. It helped you make sense of things. Please share it with anyone you think it will help
Hello, what if the data needs to be cleaned before analysis? How does 1 use SQL to go about that?
While that can be som data cleaning done tgrough SQL. It's better to have this done prior to loading in a database or do the cleaning after you extract the data from the database
@@absentdata Any instructional video on how to go about that?
I need the link of your giphy animation...
Great video walkthrough on using SQL and Power BI. I learned so much!
Very helpful video for beginners. Please upload more videos like this which will help beginners to step into data analyst job. Thanks for your valuable content
Check out the playlist
Thank you for this video, it is really helpful. A quick question, how did you take out the highlight 'invalid object name bike_share_yr_1' and for the cost_table?
The error will be removed in SSMS by refreshing the local cache. Edit -> IntelliSense -> Refresh Local Cache
Thank you so much. This video has given me great insight on a real-life project as I transition into data analysis. I appreciate this. 🙌
Hot dang, Kermit. Great job.
Hilarious 😂
1) 12:10 how are we able to see that the query has gotten larger?
2) would you advise that I complete this project & use it for my GitHub that's linked to my resume?
1. Do you mean the data from the query. You can look at the rows of data brought in from the query over time. if more data is added the size of the data will change.
2. Its always good to have dashboards to present and portfolio and walk the interviewer through the data pipeline, transformation and visualization steps.
I want to say thanks a lot for your video.
I decided to switch my career to be a data analyst/business analyst and started the journey a few weeks ago.
After looking at other people's CV & portfolios, who switched careers, I wanted to start building some projects before I apply to any data analyst roles.
Then I found this video and started to follow along two days ago. Given the limited time I have, because I also have a corporate job, I've finished the process today and feel like I've achieved something meaningful.
Thanks again and please keep sharing.
Best regards.
Hii I wanted to wish you the best in your career shift. I know its a great idea because the demand for good analyst is growing. I am really happy my video help you. Please share it with anyone who needs help.
hi, i also want to switch my career, can you please help me regarding this ?
Great video i love it!
One thing if the price is per rider the cost should be per rider as well on the revenue. Minute 22:21.
Thank you so much for the video
You solve it a min 44:15 LOL.
It was all planned to cover Powerquery? Dude loved your video so much. Thank you!
Thank you, my server name didnt populate while installing, what do i do
Be more specific. What name didn't populate
This is easy and very helpful! Can I make this and add it for my portfolio?
Of course, you can add it your portfolio
Great content. Just a newbie question, may I ask why did you create database from SQL when you can just get the data from excel to PBI?
Most data does not exist in Excel sheets. Most data is going to be in SQL databases. So as an analyst you need to know how to extract data from databases.
@@absentdata and I need hands on experience with it, so this project is EXACTLY what I need....THANK YOU A MILLION TIMES OVER!!!!!!
Hello, great content, very educational. But, I have a question:
Why would you calculate the profit margin as "( revenue - profit) / revenue" when in SQL you already did "revenue(riders*price) - COGS(riders*COGS)" and named it as "profit".
It doesn't make sense to me to deduct the profit from the revenue and then divide it with the revenue.
Shouldn't it be " profit/revenue " to get the profit margin? Thank you.
Yes, that's fine way to do it.
@@absentdata okay, thank you!!
@absentdata sir in my data price showing 3.9900000, cogs showing 01:24:00.0000000,COGS showing time(7) as type, please sir help me in this
Change the data types
@@absentdata done sir.... about to finish this project, thank u nd best explanation
Awesome Visualization, please are u on LinkedIn?
Yes my LinkedIn is in the description
I had some problems with different values than yours when it came to the dashboard, I don't know what happened, for example: Number of riders on yours was 3M while on mine it was 5M, along with several other different values. I did everything like you from the beginning, but I downloaded the database via csv on kaggle, but I believe that wasn't what made this happen.
Thanks for this video, i made my first Data Analyst project for my portfolio. Now I will search a project realized in Python and other in Excel for fill my Portfolio moreover.
try some of the projects on this channel which cover Python also.
@@absentdata for sure. I loved this channel and I will improve myself with the projects I learn here. Thank you again!
This awesome as usual easy to understand and follow
Glad that is was easy to follow
Thank you for making this video! it helped me get started with sql and power bi
Glad to hear the video was easy ro approach.
I had trouble getting the data into SQL Server. It would not allow the date/time2 format for the dteday column. Did anyone else experiance this?
Maybe turn of auto or rich data type feature
i am facing the same issue. Any luck solving it?
we need more content like that please !
Glad you enjoy this walkthrough. Please share this video with anyone you think will help! I would love to make more if it's successful!
Dude how do you show the column tools on 33:50 ? I can see you skip the process.. I can't find the column tools in report view, only in data view.
To make the colum tools appear in the top menu, you just need to click on a column in your table, and the colum tools menu will appear in the top ribbon.
@@absentdata Thank you so much. I have found it. Now I would like to know how do I adjust the month to numeric type? Appreciate you
Thank you sir. can you share the dataset? excel file or
Yes all of this is in the description
Thank you! Great content. 👏
Thanks for the video!! How came your dtedate at 36:42 happened to include all the year, months, day and quarter??
Dates in Power BI. Have an automatic hierarchy. So date --> month--> quarter--> year is the hierarchy
@@absentdatahow do I fix this?
Thank you so much im learning some new things.
Happy you are learning
Starting to love the content, but I'm facing difficulty inserting the XML file to my SQL database. It keeps giving No null error fr atemp, and temp column
There is no part in the video that has XML files
@@absentdata I have resolved the issue, thanks
Awesome!
so let say this is a real life scenario, will the company provide the data you work with?
Yes, in a real-life scenario, a company will give you access to their data. Most companies house their data in a database
@@absentdata Thanks!
Good explanation...Please do more videos like these
Thank you! I will. Please help me share the video.
Good video but where is the need to bring in MySQL here, as you are not doing any transaction here. All excel files like bikeshare and others could have been straightaway imported into power BI, Tables could be merged in power query and links between tables could have been established in data model option
Most data lives in databases. It's important as an analyst to be able to use SQL to extract data
True
Please for god's sake improve your mic's quality, the video and the project is great, but the voice/mic is not 😭
Thank you. Very informative
im an aspiring data analyst and you channel is a gold mine thank you so much
Great to hear! Its great that you a starting your journey as a data analyst. We need more good ones!
Where can i find sample data as these?
Check Kaggle. Its a good source for sample data. You can also check out my website that has some data sources. absentdata.com/data-analysis/where-to-find-data/#:~:text=1%20https%3A%2F%2Fwww.kaggle.com%20This%20is%20one%20of%20the%20best,Data%20repo%20from%20the%20US%20government%20More%20items
Thank you so much!
Hi sir, just a quick question. The .csv files are normally extracted from a sales system or such. Do we need to clean the data in Excel again or no?
It really depends on the data source. So data can be extract from a sales platform that is dirty and needs cleaning. However, usually when data is in a database. It has undergone some ETL which may fix many cleaning issues.
OMG this is so helpful. Thank you very much sir! more of this kind of tutorial/walkthrough for Data Analytics. more power to you as well!
Thank you for the support. Please share it to anyone who you think I can help.
oh shiii Tokyo Revengers???
Huh?
@@absentdata nvm i thought you know the "Toman bike" reference. That sample data you got was probably from someone who's a fan of the anime Tokyo revengers. Basically Toman is the name of the gang of bikers that's in the anime. Nvm
Thank you
May you send the data and others file to make this project and add on my resume.😊
Thank you sir
You're welcome
awesome work
Thank you
Thank you for your video. It clarifies what a data analyst will possibly work on the job.
I am glad it was helpful!
i pray you are muslim and go to heaven.u are so amazing job