Customer churn prediction using ANN | Deep Learning Tutorial 18 (Tensorflow2.0, Keras & Python)
Вставка
- Опубліковано 22 тра 2024
- In this video we will build a customer churn prediction model using artificial neural network or ANN. Customer churn measures how and why are customers leaving the business. WE will use telecom customer churn dataset from kaggle (link below) and build a deep learning model for churn prediction. We will also understand precision,recalll and accuracy of this model by using confusion matrix and classification report
#customerchurnprediction #customerchurn #customerchurnpredictionusingANN #ANN #deeplearningANN #deeplearningtutorial
Do you want to learn technology from me? Check codebasics.io/?... for my affordable video courses.
Code: github.com/codebasics/deep-le...
Exercise: For exercise go at the end of above notebook. There is a link to kaggle dataset and exercise description
Dataset: www.kaggle.com/blastchar/telc...
Next video: • Precision, Recall, F1 ...
Previous video: • GPU bench-marking with...
Deep learning playlist: • Deep Learning With Ten...
Machine learning playlist : • Machine Learning Tutor...
#️⃣ Social Media #️⃣
🔗 Discord: / discord
📸 Dhaval's Personal Instagram: / dhavalsays
📸 Instagram: / codebasicshub
🔊 Facebook: / codebasicshub
📝 Linkedin (Personal): / dhavalsays
📝 Linkedin (Codebasics): / codebasics
📱 Twitter: / codebasicshub
🔗 Patreon: www.patreon.com/codebasics?fa...
DISCLAIMER: All opinions expressed in this video are of my own and not that of my employers'.
Do you want to learn technology from me? Check codebasics.io/ for my affordable video courses.
Dude I ran a similar churn model with three outcomes.
When I run the dataset without oversample the NN gives me a nice accuracy.
When I run the dataset WITH oversample, somehow the NN gives me a very bad accuracy. WHY?
the dataset has already column if a customer churned or not. Then what are we really predicting ?
Hi Sir! at 29:04 why are you calling first layer as ''Hidden layer"? I think first layer should be Input layer. I am confused.
You have such an amazing talent to simplify complex issues and explain them so they look simple. It comes with a lot of hard work and it is very very much appreciated .
Clarification: This video was just to demonstrate how to build neural network for structured dataset like customer churn. However there is an imbalance in this dataset (number of samples with churn=Yes are much less than churn=No), I will add a new video in this series showing how to handle this imbalance. That will improve the f1 score.
sir i want to do my MS thesis on Churn prediction. is it possible to use this code and make it???
At 11:54 you said green mean customer staying and red means customer leaving, and at 13:15 you said opposite of that, plz clear the statement.
BTW your videos are helping so much!
Hello
May be I missed it, but what's your name please?
I had to run all your videos at 2x speed else I would have slept or got distracted...hahaha
I have been following all your videos and they are all extremely helpful. Thanks for your efforts :)
Excellent order of contents delivery in a possible simple way. Made each session interesting and linked with the previous sessions. Thank you so much for your efforts.
Your videos are getting better and better, Thanks man , love your videos
Thanks for teaching such extensively all the projects and ML methods
Thank you ! I had a hard time struggling with the coding things, but you make learning ML or DL so easy and fun！
Excellent video on business use case hands-on lab !!!!
cleaning the data is very very essential i any project what you did in this project is very good hope you do more advanced video on preparing the data excellent work
Learnt a lot on this video. My focus mainly was on cleaning, and the part from 27:58
This was so good! Keep it coming!
Thank you so much for this great tutorial, and your endless effort to share your knowledge.
☺️🙏
This was very helpful. Thanks a lot!
Thanks a lot sir fo making this topic sooo easy to understand🤗
awsome tutorial . thank you for taching useful methods
Yes Its Informative, Explained each and everything, Thanks Bro!
Glad it was helpful!
After long search I found ANN for classification problem. Thank you for such awesome video. Sir I have a question: in ML classification we have 5-6 also for solving classification problem. Do we also have similar algo in ANN for solving multi class or binary classification problem ?
Sir you are great
Your humor is also good for refresher
.
ahh thank you :)
Amazing lecture 💯
There is one thing - the notebooks on your GitHub sometimes have half lines like they are cut due to their long lengths at some places.
Sir do you have any video regarding svm for various inputs??
Or Ann for more than 3 inputs??
Actually I have both classification and non classification problem
Thanks for giving the dataset sir. Really helpful for practise.😁
Yup, practice is very important
That's a fantastic video. But, was there any specific reason you didn't remove one dummy variable when you used the hot-encoding method? (for preventing multicollinearity).
i am forever grateful to you and I continuous which you success, I love that for a every questions I want to ask you provide answer like why use replace instead of one hot encode? thanx
@Imran H : Well I think colors are compared with "Traffic Lights": Green=Go/Leave and Red=Waiting/Stay! So if you are leaving/moving(Churn=Yes) it's Green-Lights/Go and if you are Staying(Churn=No) it's Red-Lights/Wait/Stay!
very good knowledge
Hi, thank you for your courses
Something different:
I am tired recently
I take moringa
Do you know moringa and is it really healthy and safe?
23:34 for those people who are getting dummy variables to be True or False, just supply (dtype=int) in the get_dummies function.
thank you so much. It took my whole day to understand.
No problem bro@@bhumeshpanchal4904
thank you so much
Thank you, sir. This was a fantastic video tutorial. Very clearly explained.
Great Video.. So easy to understand Thanks alot Sir.
Also I would like to learn after the model is ready, how do I run this model on other dataset. Please share your knowlegde on who to use the model on other datasets of same type.
You are a great teacher thanku
I am happy this was helpful to you.
Hi, Wonderful lecture. If I want to use Genetic Algorithm for feature selection. How can i implement in this analysis?
Churn=Yes should be red and Churn=No should be green, however it is shown the other way around
Correct, proper EDA shall be done for Stakeholders to be able to understand the problem better
Glad I wasn't the only one who noticed this.
@@tylerpoore1487 nice flex
@Imran H Well I think colors are compared with "Traffic Lights": Green=Go/Leave and Red=Waiting/Stay! So if you are leaving/moving(Churn=Yes) it's Green-Lights/Go and if you are Staying(Churn=No) it's Red-Lights/Wait/Stay!
@@lanreuzamere4994 idea is good, but it would be so dumb to show customers being loyal in red flag. haha
Thank you.
Using balanced dataset and some permutations on layers/neurons , i got ~91.3% accuracy. Is this optimal or do we have more scope here! Thanks.
can you upload any regression problem using ANN on skewed data without removing skewness.how we will train ANN in such case?
Hello !
What is a good approach for a database that has questions in the columns and in the rows the answers to those questions (multiple selection where the clients chose 1 answer from 4 options).
you have good sense of humor😂
wonderful explanation
Glad it was helpful!
Instead of using threshold value for making yp 0 and 1 we can use :
yp = model.predict_classes(X_test)
it will give up as 0 and 1 value.
wow thats some new info to me. Thanks bud. i was using round(yp,0) as it will convert all
Thankyou sir
hi could you explain what y_test is? Is that the validation data? is x_test the training data? also, is y_pred the data in your prediction? Im trying to do this but with images. I have a folder of images (prediction folder) and I'm trying to print out the array with the results instead of images. right now i have it set so when i ran the prediction loop, images are loaded along with their 1 or 0. so I visually see if the result is accurate or not. I want to use arrays instead of images tho.
Your the best, when it comes to an explanation. Do you have any community where all AI, ML or deep learning talk happen?
Yes I am going to launch a discord community. I will announce soon, stay tuned. I will be doing a live session with some big announcements
Hello sir,
First of all, love your content and teaching style. Thank you.
Just wanted to clear a small doubt.
Do we need to worry about the " Dummy Variable Trap" in deep learning models too? I noticed, we have not dropped any of the generated dummies after OneHot Encoding.
It is always good to avoid dummy variable trap.
In monthly charge histogram,with a high monthly charge histogram shows that churn=no that means customers are not leaving right?but it's not true ..
sir, thank you for this video. I have a question, how can we predict the customer churn. I mean, I want to see a way that I can give the model some information about a specific customer, then the model should give its prediction about this specific customer either yes or no churn. I asked you this question because I see you that you stopped in your video after you get the recall results. I was waiting to see you insert customer's data and get the result from the model. Thanks
Thanks for this wonderful video.How can I build an interface for this model
Do we just build model in Random forest and ANN or we can predict value of target label by putting values of observation for which we do not know the target label. Like in Linear regression we get equation where can put value of Xs for corresponding coefficient????????
thanks
does this one model/dataset will suit for any type of company. I mean, This one dataset will help to predict churn for companies like bank, Telecom, Wireless Services, Food delivering services so on.
Sir please make a video on advanced house price regression from kaggle .
Is feature engineering a precursor to ANN unlike other deep learning techniques.Here we are doing feature engineering first then applying ANN
In 31:35, class y have 2 labels 1 and 0 (
mean Yes or No ), then why your final dense 1, that must be 2 right ?
sir, i have train ann on my dataset and the done the preprocess stage. however, the accuracy of my ann model does not change during the epochs running. does i have problem with my dataset? tq
thank you for the great tutorial. however there´s one question (sorry if asked before): don´t we have to take into account the obvious class imbalance of the target variable?
You are absolutely right that we didn't account for imbalance in the dataset. I will add a new video on how to tackle that.
very good tutorial, thank you.
please I am getting ValueError: Must pass 2-d input, when I try to run the confusion matrix heatmap
use an additional bracket [ ]
it will work nor
Hey sir, can i use this model to predict the Hw dataset?
Do a project to predict what type or what subject video will give you more views and engagement .
Hi Sir,
Can you please do a series on ML solutions for Anti-Money Laundering, Anomaly Detection , Market Manipulation etc. in Banking and Financial Services. I guess these are in much demand now and openings are also many. This would help a lot?. from Krish
Nice video! By the way, scaling before the train-test split caused data leakage. How can we trust the predictions then?
@@languagetech2675 U need to rely on explainable machine learning tools to get what you want.
Sir please make a video on hyper parameter tuning using keras tuner. i hardly missing your video bcz i didn't find perfect each and every component explanation at anywhere.
what is keras tuner?
Thank you for making such wonderful step by step tutorials. I have learned more from your tutorials than from my classes. I was wondering if you'd clarify a little confusion I have. We initially saw the data had more than 7000 instances. But the confusion matrix shows lot less than that number. Shouldn't the matrix show the result for all numbers?
This is because he did the test split for training and testing
confusion matrix shows only for X_test
Hey you are using an ANN but what kind of ANN is it, like is it backpropagation, Delta, Kohonen. Which model is used here?
How to know the number of hidden layers need to be built and the number of neurons in each hidden layer ?
Great tutorial ! One question though. Instead of one ot encoding, can't we use nominal scaled values as 1,2,3 for such attributes like Internet_Service?
We prefer that when the categorical data is ordinal i.e it can be ranked. Otherwise, OHE is preferred
@@gautamprabhu3267 Thanks!
Sir i am getting error as expected axis -1 of input shape to have value 2 but received input with shape [none 5]
My yp values are very low as compared to your values and getting errors while creating confusion matrix.Please suggest changes.
Great tutorial!, can you make a video on stock price prediction using lstm?
Yes I can
After successful model.fit(), when I am doing "y_predict = model.predict(x_test)", I am getting error: WARNING:tensorflow:Falling back from v2 loop because of error: Failed to find data adapter that can handle input: , Plz help me to understand the issue
Quick doubt: When should the scaling be done? After train_test_split or before? In this video you did it before but I heard it can lead to data leakage problem. Thank you for the content
yes you should scale the data after the train test split to avoid data leakage from our test split into our training split
That's your choice, but it'd be better if you scale the data set before splitting as you won't need to scale the test and validation dataset separately .
Sir , Can you make one video on keras tuner ???
I am new to ML, how do you create 'Churn'? I only know variable values.
i have done exactly like you did but my total charges values are still the object type what should i do?
I didn't get the point, why separately calculating precision and recall for class 0 and class1 , what is macro AVG and what is weighted average?I know generally precision and recall calculating for entire dataset . Why accuracy not considered?
I didn't get why my output is 1, and ini tthhe last video were 10. Wasn't it supposed to be 2? like 0 or 1? In the previous video, where we want to classify 10 groups, we use 10 as output. I know that i'm wrong but i don't understand why. Can someone help? :/
Thank you for making the video. Sir, I have one doubt regarding one hot encoding. When you made dummy variables and you got 3 new columns, you used all 3 new dummy colums in your model. In such scenario there would be chances of dummy variables trap or multicolliniarity. So to avoid that we always remove one colums from our dummy variables. Could you please make a separate video on it. Thank you.
Hey yes. You can remove that column. In one of my videos in ML playlist I have discussed this
14:11 sir can you explain about that blood_sugar_men and blood_sugar_women
sir i have problem with tensorflow not working in my laptop i try my best to install.. but still face problem regarding module error not found
I have a doubt: he performed the preprocessing on the entire dataset and split it into X_train and X_test. Isn't it incorrect? Shouldn't he call the X_test X_validation, since the test set must not be pre-processed with the training set.
the dataset has already column if a customer churned or not. Then what are we really predicting ?
What is the use of blood_sugar_men and blood_sugar_women
precision recall f1-score support
0 0.96 0.87 0.91 1745
1 0.46 0.73 0.56 255
accuracy 0.85 2000
macro avg 0.71 0.80 0.74 2000
weighted avg 0.89 0.85 0.87 2000
exercise answers, 40 epochs did the work,
what is the end result of this prediction like kya fayda hora hai??
how can we build a decision boundry for this problem
At 18.32 The replace method is not working...
Even printing the unique values there was no replacement
What should be the approach to make that work??
Is there a video where we can build a simple UI and deploy this project. It would good practice and making this end to end. A simple flask .api with a seirlized model. Does your channel have a video on this, thanks in advance
yes I have done few end to end projects. In youtube search "codebasics data science projects" you will find few playlists where I covered building UI, deployment using both fastap and flask.
@@codebasics Thank you, I'll check those out. What do you think a good simple deployment for this project would consist of ?
at 29:35 the input_shape is (26,) that is 26 columns but what about 5625 rows ??? I'm finding input shape for this dataset quite different than the Handwriiten Digits Classification
This is the same observation I made as well, do you understand why?
My accuracy of every epoch remains the same. It does not change at all. What is the problem?
Thanks for the wonderful clip..can you make a tutorial on optimizing NN using GA.. Since you explain things so nicely this will really help
Point noted.
My accuracy of every epoch remains same. It does not change at all. What is the problem of my model?
8.42 how did you removed that warning message??
use this code you will not get any warnings
import warnings
warnings.filterwarnings('ignore')
I have 7 columns input and one column in output .
What you suggest for predicting model ??
ANN??
Depends. you can use ANN or SVM,logistic regression if it is a classification problem
sir can you plz interpret the output in business term to a layman person . You just stopped in confusion matrix
Sure it is hard to add it in video but once you have built the model you can run it on current set of customers and find out who are going to leave most likely. And then focus on business strategy so that they don't leave (by giving them special discount, increasing engagement, providing better service etc) also this python notebook gives insights on due to which factors customer is leaving and again you can modify business strategy to reduce the churn rate
Hello sir,
Thanks for the nice tutorial, I attempted the assignment and accuracy was around 80%, but model is only predicting the 0 cases.
Is it the overfit model?
When I checked for dataset the value for Exited columns is highly imbalanced (80:20) Approx.
Now the question is how to handle the imbalance dataset?
Yup the dataset is indeed imbalanced. I will add new video and solve this exercise and show you how to tackle this imbalance.
@@codebasics Hi did you add a new video about the data imbalance
nice t-shirt. where i can get this?
after importing tensorflow , it shows error after run .why?
In my dataset, there are three different output variables i.e. good, excellent, and poor. So, I am using categorical_crossentropy and softmax as activation in the output layer.
ann.add(tf.keras.layers.Dense(units=6, activation='relu'))
ann.add(tf.keras.layers.Dense(units='softmax'
ann.fit(X_train, y_train, batch_size = 32, epochs = 100)
an error occurred
ValueError: Shapes (None, 1) and (None, 3) are incompatible
could you please let me know the soln for the same
For multiclass classification use sparse categorical cross entropy as the loss function and Label encode the target variable. That should solve the problem
27:23 O_O dude forgot it's not time for his evil mastermind to come out LMAO
18:30 the df1.replace('No internet service', 'No', inplace=True) is not working
Can anyone suggest any help?
I solved it here is the code:
def print_unique_col_values(df1):
for col in df1.columns:
if df1[col].dtype == 'object':
df1.loc[:, col].replace({'No internet service': 'No', 'No phone service' : 'No'}, inplace=True)
print(f'{col} : {df1[col].unique()}')
Hello @codebasics... my one hot encoding does not porduce 1 and 0, instead it generated false and true. could you pls assist me... here is the code line
import pandas as pd
df=pd.read_csv(r'C:\Users\Desktop\WA_Fn-UseC_-Telco-Customer-Churn.csv')
pd1=pd.get_dummies(data=df, columns=['InternetService']) #one hot encoding
Plz explain about NLP ,can we make a model from scratch in python .plz make a video on NLP ...thank you sir
yes NLP videos will be coming soon