Oh Lawd have mercy, this is one of the best, if not the best, tutorials that I have watched on the subject matter. Excellent lecture Dr. Rai. Subscribed.
Thank You for your lectures, Sir! I normalized independent variables one by one and it could improve the model. I replaced line 14 as shown below: data[, 1]
This is so helpful and so well-explained! I simply loved it. I have found your video on upschannel, but I have come here just to be able to say 'Thanks!' And I usually never comment on videos!
@@bkrai Namaskar Dr. I would like to conduct a sensitivity analysis of my model to know which input/s has greater importance for each output/s. How may I implement this on R ? Thank you for your kind response.
Dear Dr. Bharatendra, it is a very helpful video, I learned many things but I am not able to use the table function because the tensor flow version is changing and I am not able to proceed ahead with my development. please help with newer version
Hi Dr Rai, Is it possible to use boosting (ensemble methods) technique in Deep Neural networks in R? Or use boosting technique in neural networks similar to boosting tree based models? Thank you
Thank you for the great tutorial. Is it possible to create a loop that will test various parameters such as node density etc, to obtain the best combination of nodes (and layers)? What will the function loop look like?
Thank you Sir, I have a question. While I am running the 'tuning_run', at each iteration either '41' or '24' data points are getting considered while my train data has '1860' data points. I don't have any clue why it is happing. I am using '20%' validation_split.
Hello Sir, While writing code for compile I am not getting the optimiser nor the metrics..can you please suggest, are you using any additional package besides keras ?
@@bkrai thank you sir for replying so soon.At The complie part of neural network with categorical data I was facing issue but now problem is resolved. I could run the code with my data file. But the output is horrible 10% accuracy.. I guess I need to filter out imp variables and then use the model again.
Dear Prof Rai, Using this deep learning method, if we have large variables, how we can find the important features of the model? Just like finding VIP in PLSDA..
Sir I have one more request could you make a video to explain these concepts K fold cross validation, boosting and bagging, gradient descent, grid search and other boosting algorithm? I have read a lot but I never understood how it works and how to use it. Thank you Sir.
hi bharatendra, Firstly, great video. Two Questions. Q1 Is there any inherent cross-validation specified in the example you run in the video? My my data set is 'relatively small', so I need some kind of C-V. Q2 I have a reasonably high numbers of Missing data points (sparse-data). The data is point-in-time but not a time series. I realise the 'best' approach for handling NA's is data dependent, buy any suggestions how to handle this would be very much appreciated. Alex
When fitting the model, we can specify validation_split. The example in video uses 0.2 or 20%. Handling missing data will depend on the specific data that you have. Since you have relatively small data, probably it doesn't have too much missing data. But any missing data will have to be addressed before developing the model.
Hi sir, such a good explanation. Sir, I watch your video explained about neuralnet package and keras package. is there any differences between neuralnet package and keras package? Because both of the package we can do the multi hidden layer neural network.
Thanks for this @Dr. Bharatendra Rai, would you have any videos on RNN and LSTM for panel data with continuous response variables implemented in R. I have been looking for that and seem not to get anything. If you can refer me to any of your or other related videos I will really appreciate. Thanks.
Nice video. One question: do you know how to deal with factor variable also in the input data? I have one dataset with several of these. Thank you so much.
Thank you sir for your reply, yes, please make a video how we can use this in actual life through any project or any data because I write R code and learn so many thing from your channel it is not difficult to understand codes but the actual use of it. If training model work well at given set of data can performance of model be changed with addtional information
Hi sir, thanks for the video but I want to ask,.. In your video the independent variables are numeric(integer) at all can we use categorical variable as independent variable?
Thank you very much sir for all your wonderful tutorials. Vey clear and easy to follow. If you have, could you please share how to forecast for next period (out of sample) using LSTM/SVM or other ML tools.
Sir I tried to run this code on my dataset but after fit model I got value error a target error with shape(10610,4 ) was passed for an output of shape and error in py_call_impl
Thank you for this video Dr.Bharat. How can I get more rows of CTG dataset. Could you please provide us with the source of data? The other important thing, you used this data in this video, multinomial logistic regression video and ordinal logistic regression video but you did not look at the class imbalance or bias in the mutli-class response variable? Is it better to do so before applying all those techniques? Thanks.
Amount of data available for CTG is as in the CSV file. For class imbalance with deep learning models, you can refer to: ua-cam.com/video/4KfiQRqn_vA/v-deo.html
Hello, The imbalance helped a lot. I am now running another model in which my dependent variables range from 1-100 Is there anyway to predict that range of values in keras ?
Thanks for the video sir. I would like to oversample my training data. So, is it ok to do partitioning and oversample before converting into matrix form?
Hi Sir, great video, I would like to ask can I perform deep learning on a data set which is a mix of categorical and continuous variable where levels of few variables are more than 60 ? Also my target variable is a continuous variable. Awaiting your response. Thank you
hi Dr. Bharatendra, I got the same plot the first time, but when I run it the second time both plots ACC[val loss - loss], as well as LOSS[ val acc - acc], become parallel straight-lined and stable from 1st epoch and got the same score of loss and acc. 19/19 ===== 0s 659us/step - loss: 0.3937 - accuracy: 0.8590. is it normal?. also, I have doubt about [19/19] as it should reflect the whole rows, like in your example [1218/1218]
Thank you so much for this helpful video. I am learning a lot from your videos and the explanation is wonderful. Thanks once again. I had a doubt here. How do we split the data for 10fold cross-validation or K cross-validation instead of 70/30? And besides accuracy how do we calculate the other evaluation metrics? If you can demonstrate this in your video then it would be great.
I have dataset with 3 variables numeric, and target class with 3 levels, I am getting this error when using replace =T, Error in sample.int(x, size, replace, prob) : invalid 'replace' argument. I cannot find any solution for this, can you give me some suggestion about this.
I don't get the same confusion matrix every time I run this model despite I am using set.seed() before building the model? Is there a parameter we can use when building the model like the one we used in extreme Gradient Boosting? Thanks.
Dr. Bharatendra Rai so is there any parameter we can use in the model like what you did with xgboost? So how we can prove that the results we get belong to this model ?
Your videos are really helpful and informative. I like the way you put across the contents to make your viewers understand the concepts. Could you please make a video on DNN, CNN, RNN, and LSTM for binary classification on a tabular data(.csv) which is not an image classification problem? Also, If it is possible to show us how to implement K fold CV and an external test dataset prediction would be great. Thanks in advance!
Hello Dr. Rai.. I am trying to do something similar on a data but am facing issues. I have sent you a dm separately on another social media site (with an image of the data set). Thank you so much for these videos. They are extremely helpful.
hi sir, was wondering how to fix this error? Error in py_call_impl(callable, dots$args, dots$keywords) : r-miniconda/envs/r-reticulate/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py:805 train_function *
@@bkrai thank you very much sir! will try it out! and by the way, thanks to you vids, really helps me a lot to understand concepts in neural network and random forest… kudos sir!
Your tutorials are awesome. But could you tell me how we can separate the important variables using deep learning? I have more than 100 variables for my model and definitely don't want to use them all since most of them are correlated and have less importance.
Thank you for this nice tutorial. When creating the dummy variables using one hot encoding, shouldn't we remove the first column because of the "dummy variable trap"? Correct me if I am wrong but my understanding is that if we have K categories, we need to have K-1 dummy variables.
Thanks for your reply. I confused the encoding of attributes versus the encoding of the dependent variable. In this particular scenario, you are encoding the labels (dependent variable). Am I right with this logic?
Yes it can be factor. The example in this video uses response as factor. However, you need to convert it's values from text to numbers and do one hot encoding.
Dr. Bharatendra Rai I can’t stop to ask you. about Neural network for sounds. I would like to learn fo you, and that me can do it in the future. Regards, and I hope for you answer
Your tutorial is awsome. But no matter what i tried to do i cant indtsll keras and tenserflor in my r studio. Can you make video of how to install from scratch? including all error and how handle them..?
There some error when I run the command on 73rd row. model =keras_model_sequential() Error comes as Error in py_discover_config(required_module, use_environment) Python specified in RETICULATE _PYTHON Kindly help how to solve it
Sir Thanks for such an clear and understandable videos. Sir Can you also make some tutorials on RNN and LSTM. It would be very helpful if there are some theoretical understandings of the deep learning.Thanks in advance!
hi sir , your tutorials are awesome so helpful! i need help with classification neural network and the data is text and numeric , the layers should learn and convert text data and combine with the numeric data than integrated model for classification . can you help ? will LSTM layers be helpful ??
I have solved it. Add as.numeric() using pipe operator to k_argmax function. k_argmax(pred) %>% as.numeric() This will make d pred object to be numeric datatype which would be same datatype as testingtarget object you would use in the table(). As such you get the confusion matrix without errors
Many have reported issues when trying with new data. At least some coding will help in seeing what is going on and adapt to a totally different data. Blindly running codes doesn't help.
Sir, I did this above model it worked for me. But after confusion matrix i want to denormalize it, I used traditional formula denormalized = (normalized)*(max(x)-min(x))+min(x) as a function not working? suggest me any other
Hello Dear Sir, Many Thanks for your video. I have a dataset with 4 input variables and one output variable. the output variable has 101 different categories. I used your code but it gives worst accuracy such as 0.1391 on categorical cross entropy but for the same dataset it gives 99% accuracy on binary cross entropy..another problem is it gives mostly constant accuracy which is 99% on binary cross entropy even in dynamic plot. How to tackle this problem??. Please help
@@bkrai not the epochs sir, it is taking only taking 23 values of x, I mean training data. It's not considering the full data like how in your video it is around 1200/1200. Please help me thank you 🙏
If you are referring to 1218/1218, then note that the training data has 1523 and 80% of that is approximately this number. Remaining are used for validation related calculations.
sir , this video makes me comfortable to implement deep learning algo in R but sir how can i run this python code in R x = x.reshape((1,) + x.shape) # this is a Numpy array with shape (1, 3, 150, 150)
@@bkrai I tried the same method for binary but it stops at history %> .. my response is 0 and 1 (numeric) but the same method is not working. Can we know please why you used (data$NSP -1) is it just for changing to 0,1,2 or it is for another purpose? When omitting this line , the method again stops at history?
@@bkrai Yes it is 2 units and binary_crossentropy . Thanks a million . Anyone who is trying this example and then wants to apply it on binary classification will have this error : "Error in (function (classes, fdef, mtable) : unable to find an inherited method for function ‘fit’ for signature ‘"keras.engine.sequential.Sequential", "matrix"’ .. Just close R and reopen it again, then it will work smoothly.
Here is a link that provides such an example: machinelearningmastery.com/evaluate-performance-deep-learning-models-keras/machinelearningmastery.com/evaluate-performance-deep-learning-models-keras/
Hey there! Graet work my friend! I've been trying to follow your steps with another data frame, but after installing everything, and running i get the same problem each time... I've tried everything, but I can't get it to work :( I was wondering if you could help me pleeease! This are the code lines and the message I get: data[,-c(6,19,20)]
Bharatendra Rai the matrix corresponds to the classic client churn problem... in this case, those variables correspond to numeric values, the rest of them are characters that I turned into factors, obviously the first line of the code is an error, but only because I've trying different options. Any way, I tried with other random NUMERIC matrixes, each of them correctly defined, as the one you use in your example, and even with that it didn't worked for me :( Any ideas on what could it be? PD: I also upgraded and downgraded all the packages in r and python, no solution...
Looks like 149.4 Gb is too big for your computer. You can go for a smaller data by taking sample and also close any data file that you are not using in the RStudio.
can u plzz tell me how to do it for 4 class variable sir.....cause for 3 class variable it works fine....when it is implemented on 4 class variables...it only predicts 3 classes....thanku
Oh Lawd have mercy, this is one of the best, if not the best, tutorials that I have watched on the subject matter. Excellent lecture Dr. Rai. Subscribed.
Thanks for your comments!
Sir, your way of explanations are really great!!. I have completed "Top 10 Machine Learning Methods" and started deep learning videos
Thanks for the feedback!
Thank You for your lectures, Sir! I normalized independent variables one by one and it could improve the model. I replaced line 14 as shown below:
data[, 1]
That's great, thanks for the update!
Yes I had the models improved but "val loss" increases as "loss" decreases which means we are doing overfitting!!!!
Great explanation Doctor 👌👍 oversimplified the complex problem 👏👌👍
Thanks for comments!
Great video! Production value is top notch, as is the content. Carefully and thoughtfully planned discussion is tremendous. Thanx ;)
Thanks for your encouraging comments!
Hello Doc. Can you do a video on time series forecast using keras transformer model in R?
Your channel give me hope and confidence
Thanks for comments!
This is so helpful and so well-explained! I simply loved it. I have found your video on upschannel, but I have come here just to be able to say 'Thanks!' And I usually never comment on videos!
Thanks for comments!
Excellent illustration Dr. Thank you so much
please, can I find a video about the "the Adaptive neuro fuzzy inference system (ANFIS)" in R
Thanks for the suggestion, I've added it to my list!
@@bkrai Much appreciated professor
Doc, I am overwhelmed with this video tutorial. Thank you so much
Thanks for comments! Don't have right now, but will look into it.
@@bkrai Namaskar Dr. I would like to conduct a sensitivity analysis of my model to know which input/s has greater importance for each output/s. How may I implement this on R ? Thank you for your kind response.
I think the problem similar to backpropagation, analyze the weights (importance, value) from output and backtrack until the input
Can you please explain or suggets diffrent types of loss and optimiser functions. Thanks
You can refer to this playlist where different types are used:
ua-cam.com/play/PL34t5iLfZddtC6LqEfalIBhQGSZX77bOn.html
😀 Great Video and thanks for sharing the data set used!
You are welcome!
Could you please help me locate the CTG.csv data on the git hub link you provided? So I can follow the learning actively using your video. Thanks!
Here is the link:
raw.githubusercontent.com/bkrai/DeepLearningR/master/Cardiotocographic.csv
Dear Dr. Bharatendra, it is a very helpful video, I learned many things but I am not able to use the table function because the tensor flow version is changing and I am not able to proceed ahead with my development. please help with newer version
after this trainLabels
See if this helps:
ua-cam.com/video/-IYYqdxdYXk/v-deo.html
Awesome video. Thank you. Can the same technique be used for regression (predicting a numerical value). Thoughts?
yes.
Best vedio on ml on UA-cam
Thanks for comments!
Hi Dr Rai,
Is it possible to use boosting (ensemble methods) technique in Deep Neural networks in R?
Or use boosting technique in neural networks similar to boosting tree based models?
Thank you
Yes, absolutely!
@@bkrai could you kindly recommend a source where boosted neural network is implemented ?
Thank you for the great tutorial. Is it possible to create a loop that will test various parameters such as node density etc, to obtain the best combination of nodes (and layers)? What will the function loop look like?
You can refer to this:
ua-cam.com/video/FscOZT0_ObA/v-deo.html
Thank you Sir,
I have a question.
While I am running the 'tuning_run', at each iteration either '41' or '24' data points are getting considered while my train data has '1860' data points. I don't have any clue why it is happing. I am using '20%' validation_split.
Earned yourself a sub. Great job. Very well explained
Thanks for comments!
Hello Sir, While writing code for compile I am not getting the optimiser nor the metrics..can you please suggest, are you using any additional package besides keras ?
What code are you using?
@@bkrai thank you sir for replying so soon.At The complie part of neural network with categorical data I was facing issue but now problem is resolved. I could run the code with my data file. But the output is horrible 10% accuracy.. I guess I need to filter out imp variables and then use the model again.
Thanks for the update!
Have you anytime given a thought of shifting to Python ? Just curious. For conducting your research do you prefer R or Python.
I prefer R.
THANK YOU!!!!!!!!!!!!
The best tutorial ever!!!!!
So good, so clear!
Thank you for share this
Thanks for comments!
Dear sir, I humbly request you to do a video on 'AUTOKERAS'. I will be eagerly waiting for your valuable video sir. Thank you
Thanks, I've added it to my list of future videos.
Hello Sir,
i took binary dataset. I added loss = binary_crossentropy. Getting error while running Fit Model. plz suggest ?
What error are you getting?
Dear Prof Rai,
Using this deep learning method, if we have large variables, how we can find the important features of the model? Just like finding VIP in PLSDA..
Deep networks do feature selection automatically, we don't need to find them.
Sir I have one more request could you make a video to explain these concepts K fold cross validation, boosting and bagging, gradient descent, grid search and other boosting algorithm? I have read a lot but I never understood how it works and how to use it. Thank you Sir.
Dr. Do you have any tutorial of NN with multiple outputs?
Thanks, I've added it to my list.
Great video! I'm getting a NaN value in my testtarget.. how to resolve this? Thanks in advance
NaN is 'not a number'. Check your code.
@@bkrai Thanks for your reply.. this is my code
library(keras)
> data str(data)
'data.frame': 789 obs. of 6 variables:
$ TATAMOTORS.NS.Open : num 431 429 440 430 431 ...
$ TATAMOTORS.NS.High : num 436 441 441 433 436 ...
$ TATAMOTORS.NS.Low : num 422 422 432 426 430 ...
$ TATAMOTORS.NS.Close : num 424 439 434 430 432 ...
$ TATAMOTORS.NS.Volume : int 6807536 15331261 9794953 8395377 7021611 4635214 12629045 4702579 4564895 7408870 ...
$ TATAMOTORS.NS.Adjusted: num 424 439 434 430 432 ...
> data dimnames(data) data[, 1:5] data[,6] summary(data)
V1 V2 V3 V4
Min. :0.0000005 Min. :0.0000005 Min. :0.0000004 Min. :0.0000004
1st Qu.:0.0000024 1st Qu.:0.0000025 1st Qu.:0.0000024 1st Qu.:0.0000024
Median :0.0000061 Median :0.0000062 Median :0.0000060 Median :0.0000061
Mean :0.0000130 Mean :0.0000131 Mean :0.0000128 Mean :0.0000129
3rd Qu.:0.0000156 3rd Qu.:0.0000156 3rd Qu.:0.0000151 3rd Qu.:0.0000153
Max. :0.0001084 Max. :0.0001088 Max. :0.0001066 Max. :0.0001074
NA's :2 NA's :2 NA's :2 NA's :2
V5 V6
Min. :1 Min. : 64.3
1st Qu.:1 1st Qu.:133.6
Median :1 Median :174.8
Mean :1 Mean :197.6
3rd Qu.:1 3rd Qu.:257.4
Max. :1 Max. :438.3
NA's :2 NA's :2
> set.seed(1234)
> ind training test trainingtarget testtarget trainLabels testLabels
hi bharatendra, Firstly, great video. Two Questions. Q1 Is there any inherent cross-validation specified in the example you run in the video? My my data set is 'relatively small', so I need some kind of C-V. Q2 I have a reasonably high numbers of Missing data points (sparse-data). The data is point-in-time but not a time series. I realise the 'best' approach for handling NA's is data dependent, buy any suggestions how to handle this would be very much appreciated. Alex
When fitting the model, we can specify validation_split. The example in video uses 0.2 or 20%. Handling missing data will depend on the specific data that you have. Since you have relatively small data, probably it doesn't have too much missing data. But any missing data will have to be addressed before developing the model.
Hi sir, such a good explanation. Sir, I watch your video explained about neuralnet package and keras package. is there any differences between neuralnet package and keras package?
Because both of the package we can do the multi hidden layer neural network.
keras run much faster.
Thanks for this @Dr. Bharatendra Rai, would you have any videos on RNN and LSTM for panel data with continuous response variables implemented in R. I have been looking for that and seem not to get anything. If you can refer me to any of your or other related videos I will really appreciate. Thanks.
It's on my list of future video!
Doc, Is there any function where i can calculate the following metrics( mse, f score, precision, recall, etc.) of the nn model using this process?
Nice video. One question: do you know how to deal with factor variable also in the input data? I have one dataset with several of these. Thank you so much.
You can do one-hot encoding for factor variables.
I am finding difficulties with installing keras on windows. Is there any resource available for guiding me towards the end?
I have installed python 3.7, still getting a message- 'virturalenv' not available.
see if this works:
ua-cam.com/video/-IYYqdxdYXk/v-deo.html
Hello Sir, this is great video but how can we use training model in actual life along with raw data. please make any video on this
Thanks! You can also refer to this playlist for more and from time to time I'll be adding more topics:
ua-cam.com/video/hd81EH1g1bE/v-deo.html
Thank you sir for your reply, yes, please make a video how we can use this in actual life through any project or any data because I write R code and learn so many thing from your channel it is not difficult to understand codes but the actual use of it. If training model work well at given set of data can performance of model be changed with addtional information
Hi sir, thanks for the video but I want to ask,.. In your video the independent variables are numeric(integer) at all can we use categorical variable as independent variable?
Yes categorical as independent variables can be used with one hot encoding.
@@bkrai okay thanks a lot
You are welcome!
Thank you very much sir for all your wonderful tutorials. Vey clear and easy to follow. If you have, could you please share how to forecast for next period (out of sample) using LSTM/SVM or other ML tools.
Sir I tried to run this code on my dataset but after fit model I got value error a target error with shape(10610,4 ) was passed for an output of shape and error in py_call_impl
Difficukt to say much without looking at the code.
Please have Video on Unet ,Image ,mask and filter stride pooling etc
Thanks for the suggestion! I've added this to my list.
Thank you very much for the excellent lecture!
I solved the problem that I had written earlier, thank you!
Thanks for the comments and update!
Thank you for this video Dr.Bharat. How can I get more rows of CTG dataset. Could you please provide us with the source of data? The other important thing, you used this data in this video, multinomial logistic regression video and ordinal logistic regression video but you did not look at the class imbalance or bias in the mutli-class response variable? Is it better to do so before applying all those techniques? Thanks.
Amount of data available for CTG is as in the CSV file. For class imbalance with deep learning models, you can refer to:
ua-cam.com/video/4KfiQRqn_vA/v-deo.html
Dr. Bharatendra Rai thank you very much Dr.
Sir make full series on deep learning in R
Yeah....plz Sir
I've added many more to the playlist now:
ua-cam.com/play/PL34t5iLfZddtC6LqEfalIBhQGSZX77bOn.html
The deep learning playlist now has many more videos. You can also refer to my book titled 'Advanced Deep Learning with R'.
Hey, great tutorial
For some reason, the keras model is only predicting 0 instead of 0's and 1's.
Do you know what could be the problem
How many 0s and how many 1s do you have in the data set?
73586 - 0
1565 - 1
Your model may be too biased towards predicting zero due to heavy imbalance. This link will be useful:
ua-cam.com/video/Ho2Klvzjegg/v-deo.html
Hello,
The imbalance helped a lot.
I am now running another model in which my dependent variables range from 1-100
Is there anyway to predict that range of values in keras ?
Your dependent variable is categorical or numeric?
Thanks for the video sir. I would like to oversample my training data. So, is it ok to do partitioning and oversample before converting into matrix form?
If you are trying to address class imbalance, refer to this:
ua-cam.com/video/4KfiQRqn_vA/v-deo.html
Hi Sir, great video, I would like to ask can I perform deep learning on a data set which is a mix of categorical and continuous variable where levels of few variables are more than 60 ?
Also my target variable is a continuous variable.
Awaiting your response.
Thank you
For numeric response, use this:
ua-cam.com/video/SrQw_fWo4lw/v-deo.html
Dear Professor, how can we find the weight of each independent variables?
hi Dr. Bharatendra, I got the same plot the first time, but when I run it the second time both plots ACC[val loss - loss], as well as LOSS[ val acc - acc], become parallel straight-lined and stable from 1st epoch and got the same score of loss and acc. 19/19 ===== 0s 659us/step - loss: 0.3937 - accuracy: 0.8590. is it normal?.
also, I have doubt about [19/19] as it should reflect the whole rows, like in your example [1218/1218]
If you run the same model again, it starts from where it left in the previous run. To run it from beginning, you can re-run codes from line-32 onward.
@@bkrai thanks a lot
Please, Can u make a recurrent neural network for predictin continuos values of stocks?
Thanks, I've added this to my list.
Thank you so much for this helpful video. I am learning a lot from your videos and the explanation is wonderful. Thanks once again. I had a doubt here. How do we split the data for 10fold cross-validation or K cross-validation instead of 70/30? And besides accuracy how do we calculate the other evaluation metrics? If you can demonstrate this in your video then it would be great.
Thanks for the comments and suggestion!
I have dataset with 3 variables numeric, and target class with 3 levels, I am getting this error when using replace =T, Error in sample.int(x, size, replace, prob) : invalid 'replace' argument. I cannot find any solution for this, can you give me some suggestion about this.
sir I badly need a help from u
please hep me in resolving this error
for (i in training_set)
{
p
Try this link where I read several image files: ua-cam.com/video/5bso_5X7Zu4/v-deo.html
thank you! keras R
You are welcome!
Why you change the NSP value from 0 to 2? what's the problem with 1-3?
The method used here required that specific format.
I don't get the same confusion matrix every time I run this model despite I am using set.seed() before building the model? Is there a parameter we can use when building the model like the one we used in extreme Gradient Boosting? Thanks.
That's true with deep learning models that you will get slightly different results each time you re-run the model.
Dr. Bharatendra Rai so is there any parameter we can use in the model like what you did with xgboost? So how we can prove that the results we get belong to this model ?
You can save the model and reuse it to get same results.
Dr. Bharatendra Rai ok thank you so much Sir.
Welcome!
sir thank you for tutorial. Can you tell me which application you use for screen grab?
It's quick view.
Your videos are really helpful and informative. I like the way you put across the contents to make your viewers understand the concepts. Could you please make a video on DNN, CNN, RNN, and LSTM for binary classification on a tabular data(.csv) which is not an image classification problem? Also, If it is possible to show us how to implement K fold CV and an external test dataset prediction would be great. Thanks in advance!
Thanks for suggestions! I've added them to my list.
sir thanks for the vide..super....when we have 2 outcomes (binomial)..do we need to do one encoding?
If two values are 0 or 1, it is not needed.
Very well explained, thanks!
You're welcome!
Hello Dr. Rai.. I am trying to do something similar on a data but am facing issues. I have sent you a dm separately on another social media site (with an image of the data set). Thank you so much for these videos. They are extremely helpful.
Thanks, I'll look into it.
@@bkrai .. thank you sir... Looking forward
@@bkrai .. sure.. looking forward. Have pinged you on linkedin.
Hello sir,.. the video is of world-class quality .. is it possible to plot the multilayer perceptron model just like nn plot
You can use this link:
ua-cam.com/video/SrQw_fWo4lw/v-deo.html
hi sir, was wondering how to fix this error?
Error in py_call_impl(callable, dots$args, dots$keywords) :
r-miniconda/envs/r-reticulate/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py:805 train_function *
Likely due to Tensorflow not installed properly. You can try working in RStudio cloud.
ua-cam.com/video/SFpzr21Pavg/v-deo.html
@@bkrai thank you very much sir! will try it out! and by the way, thanks to you vids, really helps me a lot to understand concepts in neural network and random forest… kudos sir!
Thanks for comments!
Your tutorials are awesome. But could you tell me how we can separate the important variables using deep learning? I have more than 100 variables for my model and definitely don't want to use them all since most of them are correlated and have less importance.
You can use this link for identifying important variables:
ua-cam.com/video/VEBax2WMbEA/v-deo.html
Watch encoder-decoder network then you could reduce the size of your input by taking the bottle neck as new variables
This was fantastic. Thank you!!!
You are welcome!
Sir, for a large datatset(order of 5 lakh cases) with a binary outcome and over 30 inputs, which package should I use? keras or neuralnet
Definitely keras.
@@bkrai Sir, thank you for the help. I have one doubt. How do we change the default cut off probability in keras?
Thank you once again sir🙏🏻 and wishing you happy new year 2018.
Thanks and a very happy, healthy, and prosperous happy new year to you too!
Hello Sir, is it possible to detect mask object detection in R like we do in Python??
Thank you for this nice tutorial.
When creating the dummy variables using one hot encoding, shouldn't we remove the first column because of the "dummy variable trap"? Correct me if I am wrong but my understanding is that if we have K categories, we need to have K-1 dummy variables.
Here it is ok since we use three nodes in the output layer. Each column with 0 and 1 represents absence or presence of that category.
It's not particularly necessary no
One hot encoding is necessary here. It helps to make response or output value lie between 0 and 1.
Thanks for your reply. I confused the encoding of attributes versus the encoding of the dependent variable. In this particular scenario, you are encoding the labels (dependent variable). Am I right with this logic?
That's correct!
Thank you sir for the excellent lecture. sir can you share your knowledge on Deep Learning for Text Classification with Keras.
Thanks, I've added this to my list.
excellent sir. can the response variable be "FACTOR" ?.....same example in random forest you considered response variable as factors...please clarify
Yes it can be factor. The example in this video uses response as factor. However, you need to convert it's values from text to numbers and do one hot encoding.
ok sir
Sir I had a problem to install keras
I love your explanations
Thanks for comments!
Thank a lot master!!! Saludos desde Chile.
Thanks for comments!
Dr. Bharatendra Rai I can’t stop to ask you. about Neural network for sounds. I would like to learn fo you, and that me can do it in the future. Regards, and I hope for you answer
how can i download the Dataset of the Work ?
Link is available in the description area below this video.
Your tutorial is awsome. But no matter what i tried to do i cant indtsll keras and tenserflor in my r studio.
Can you make video of how to install from scratch? including all error and how handle them..?
You can refer to this link:
tensorflow.rstudio.com/installation/
There some error when I run the command on 73rd row.
model =keras_model_sequential()
Error comes as
Error in py_discover_config(required_module, use_environment)
Python specified in RETICULATE _PYTHON
Kindly help how to solve it
Sir Thanks for such an clear and understandable videos. Sir Can you also make some tutorials on RNN and LSTM. It would be very helpful if there are some theoretical understandings of the deep learning.Thanks in advance!
Thanks for the suggestions, it on my list for future videos.
Nice to hear that. Please help me to understand how the data processing works for LSTM
you made my day... what if i'm interested in predicting a continuous variable ?
Here is the link:
ua-cam.com/video/SrQw_fWo4lw/v-deo.html
hi sir ,
your tutorials are awesome so helpful! i need help with classification neural network and the data is text and numeric , the layers should learn and convert text data and combine with the numeric data than integrated model for classification . can you help ? will LSTM layers be helpful ??
I'll do LSTM probably next month.
The predict_classes function didn't work for me so I used
pred %
predict(test)
pred
Hello Sir, I got the exact same error message. How did you go about solving it?
@@uchekalu7725 haven't solved it. I believe the owners of the package retired the function Dr R used.
I have solved it. Add as.numeric() using pipe operator to k_argmax function.
k_argmax(pred) %>% as.numeric()
This will make d pred object to be numeric datatype which would be same datatype as testingtarget object you would use in the table(). As such you get the confusion matrix without errors
@@uchekalu7725 thank you!!
Thanks for the update!
why have you removed parts of your R code online?
Many have reported issues when trying with new data. At least some coding will help in seeing what is going on and adapt to a totally different data. Blindly running codes doesn't help.
Sir, I did this above model it worked for me. But after confusion matrix i want to denormalize it, I used traditional formula denormalized = (normalized)*(max(x)-min(x))+min(x) as a function not working? suggest me any other
There is no need to denormalize.
Sir another doubt how can I make a test dataset with no label for output in this above example(NSP)
How we can plot any of those MLPs to see the network?
You can use this link to visualize the network:
ua-cam.com/video/-Vs9Vae2KI0/v-deo.html
Dr. Bharatendra Rai thanks a million Sir.
You are welcome!
Sir if I want 0 and 1 only than what should I put over -1
It depends on what are original values.
> data[, 1:21]
The error message clearly says what needs to be done.
Hello Dear Sir, Many Thanks for your video. I have a dataset with 4 input variables and one output variable. the output variable has 101 different categories. I used your code but it gives worst accuracy such as 0.1391 on categorical cross entropy but for the same dataset it gives 99% accuracy on binary cross entropy..another problem is it gives mostly constant accuracy which is 99% on binary cross entropy even in dynamic plot. How to tackle this problem??. Please help
Go with the one that gives better result.
Thanks for this interesting video.
Can't you just install keras from RStudio, using install.packages('keras')?
Yes, that's the 1st thing we do. And then it will ask you to run two specific lines of command from terminal or command prompt of your computer.
After installing using install.packages('keras') I wasn't asked to input additional commands, and seems to work fine.
That's good!
Hello sir, in my dataset there 200 rows but when i run the training it shows only 23/23. please help sir
If you are using 23 epochs, after running all it will show 23/23.
@@bkrai not the epochs sir, it is taking only taking 23 values of x, I mean training data. It's not considering the full data like how in your video it is around 1200/1200. Please help me thank you 🙏
If you are referring to 1218/1218, then note that the training data has 1523 and 80% of that is approximately this number. Remaining are used for validation related calculations.
@@bkrai yes sir, for mine the training data has 200 values but my model is taking only 20 values of it. what should i do. Please help, thank you.
Hi Sir, you are a machine. Could you make videos using R on topics like Customer Life Cycle, Lifetime Value, Cross and Up-sellling?
Thanks for comments and suggestion! I've added it to my list.
Did this trick can be usefull on 2021?
Yes deep learning model should work fine.
sir , this video makes me comfortable to implement deep learning algo in R
but sir how can i run this python code in R x = x.reshape((1,) + x.shape) # this is a Numpy array with shape (1, 3, 150, 150)
Probably someone with better expertise in python will be able to answer this.
Okk
How I can apply this multilayer perceptron for a binary classification? Thanks.
This same method will work for binary too.
@@bkrai I tried the same method for binary but it stops at history %> .. my response is 0 and 1 (numeric) but the same method is not working. Can we know please why you used (data$NSP -1) is it just for changing to 0,1,2 or it is for another purpose? When omitting this line , the method again stops at history?
@@bkrai This is where I changed NSP to binary :
data
Make sure to make necessary changes elsewhere. For example, last layer should now have 2 units and not 3.
@@bkrai Yes it is 2 units and binary_crossentropy . Thanks a million . Anyone who is trying this example and then wants to apply it on binary classification will have this error : "Error in (function (classes, fdef, mtable) : unable to find an inherited method for function ‘fit’ for signature ‘"keras.engine.sequential.Sequential", "matrix"’ .. Just close R and reopen it again, then it will work smoothly.
How can i use k fold cross validation with keras in R?
Here is a link that provides such an example:
machinelearningmastery.com/evaluate-performance-deep-learning-models-keras/machinelearningmastery.com/evaluate-performance-deep-learning-models-keras/
Thank you, sir, for such a wonderful video
It will work If I have more than 200 categorical variables ....?
No harm in trying.
Hey there! Graet work my friend!
I've been trying to follow your steps with another data frame, but after installing everything, and running i get the same problem each time... I've tried everything, but I can't get it to work :(
I was wondering if you could help me pleeease!
This are the code lines and the message I get:
data[,-c(6,19,20)]
What type of variables 6, 9 and 20 are?
Bharatendra Rai the matrix corresponds to the classic client churn problem... in this case, those variables correspond to numeric values, the rest of them are characters that I turned into factors, obviously the first line of the code is an error, but only because I've trying different options. Any way, I tried with other random NUMERIC matrixes, each of them correctly defined, as the one you use in your example, and even with that it didn't worked for me :(
Any ideas on what could it be?
PD: I also upgraded and downgraded all the packages in r and python, no solution...
What is the purpose of these two lines? If you want to convert all 20 variables to numeric, then one line should have been enough.
data[,-c(6,19,20)]
Respected Sir,
Can you plz extend this code for further evaluating ROC and AUC. It is will be a great help.
Hello sir ,your video lectures help me alot. Thanks and keep posting..Please help me on this-
Error: cannot allocate vector of size 149.4 Gb
Looks like 149.4 Gb is too big for your computer. You can go for a smaller data by taking sample and also close any data file that you are not using in the RStudio.
@@bkrai actually the size of data set is only 4mb but when I am applying svm algo after preprocessing then it gives this error..
It is creating something that is 149.4 Gb in size.
how can we decide the number of units to be 50?
Is it random ?
It needs some experimentation. You may start with a number and then try higher and lower numbers to see if it helps.
Sir can you upload a video for drawing roc curve of this problem
Try link below, but use one level Vs other:
ua-cam.com/video/ypO1DPEKYFo/v-deo.html
@@bkrai Sir, I used plotROC by installing RSNNS package. Curve has drawn. But it is without formatting.
It is showing only one curve as there are 6 classes in my data set, so the number of curves should be 6
There should be only one curve and not 6. The curve indicates performance of the entire model.
@@bkrai thanks a lot sir. You have solved my problem. You are such a pure soul
can u plzz tell me how to do it for 4 class variable sir.....cause for 3 class variable it works fine....when it is implemented on 4 class variables...it only predicts 3 classes....thanku
Note that in the last layer I used units = 3 because I had 3 classes. If you have 4, you should use 4.