For someone who doesn't even have good coding experience. I had to finish a demo for the work. I was able to complete it with your tutorial. Excellent work.
DAMN.....!!!!!!!!!! I tried so hard to learn ML on other channels. All of them were so confusing and longer than my attention span. But your videos keep me concentrated all the way. Great job man, hope UA-cam search engine will be more generous on you.
interesting project.... your way of teaching is very simple and explanations are really good... looking forward for more projects to ad in my resume...😅😅
Hi great video! I personally think the axises during the 'Quantifying the similarity' part can be a bit confusing. If you are measuring the similarities of the Users, then u are comparing the User's rating score on different movies. So it should be U1 and U2 marked on the dots instead of M1 and M2, and M1 & M2 on the axises. Because when it expands from 2D to nD, the vector will be combinations of U1 and U2's rating on different movies, and each of the axis will be M1, M2....Mn
Hi, can make a video of how after training a model which predicts rating by a user of an item, multiple recommendations for a particular user based on their item rakings are generated?? It would be really helpful
There is an error in how the non-rated entries are treated. specifically, if nans are replaced by 0 BEFORE mean normalization, you are basically penalising that unrated item. The right thing to do is to take the unrated items as 0 AFTER mean normalization, therefore giving them the same user rating as the mean.
i was also wondering whether there were other approaches... in a large normal dataset, most (the vast majority) of the dataset would wind up being values that are NaNs converted to means... having such a large proportion of imputed values must impact the quality of the predictions, no?
Thank you for the video! One question, though.. would it be better to use df.mean().mean() as the mean rating of all users instead of 2.5? Many thanks!
Very well explained! Thanks. Could you please explain me how the mean 2.5 was subtracted from user's rating which you mentioned as a trick @21:02? I was trying out this on books recommendation dataset.
you all prolly dont care at all but does anyone know a method to get back into an Instagram account?? I was stupid forgot the password. I love any help you can give me
@Alejandro Oliver I really appreciate your reply. I found the site through google and I'm waiting for the hacking stuff now. I see it takes quite some time so I will reply here later with my results.
Hello sir, I have some confusion, our csv will not be the same as in this tutorial, i.e movies in column and ratings in rows our data will be like i'll have three column one is user_id , ratings, product_id how will I apply this filtering on such data, please guide me
Hi, thanks for the great tutorial. In the toy dataset example, what do I need to do in order to find similar users (not movies) based on action_lover's ratings?
Does standardise() function remain same? eg. if we wish to perform user-user CF, do we need to standardise column instead? amazing tutorial, thanks.. :))
you teaching skills are way better...thanks for this...also i have a doubt. suppose i wanna build a hybrid system(content based+collaberative) the way to do is that get recommendations(content based+collaberative) from both and combine to form one .example: recommendation1=[m1,m3,m6] and recom2=[m4,m5,m6,m8,m9] then final recomndation=[m6,m1,m3,m6,m4,m5,m8,m9] . is this the way how it should be done or there are other better way please tell.
There is no such problem because ONLY if (user_Ratings-2.5) is negative (i.e small rating) the multiplication will move it even more negative when if it is positive the product will sure be a positive number.
According to your steps shown in the website for application development on medium,when we are trying to open test.html file it appears blank.Can you please explain the testing API step again.
I have a dataset and I want to create a model which will predict top3 movies to user, such that final output should be CSV file with columns user_id and top3 how to do this , pls help me
Please please please help urgently When i get to the recommendation part... I'm getting an error No matter what movie i say it gives me a KeyError and says there has been an exception handling error Please help Asap
you might have written: user_ratings=rating.pivot_table(index=['userId'],columns=['title'],values=['rating']) instead of: user_ratings=rating.pivot_table(index=['userId'],columns=['title'],values='rating') remove the braces from rating
hmm how can I get list similar movies afterall? It seems like the final list we have is the list of rating values, not the movieId, so I don't know how to get information of the movies from the final recommended list?
Sir, while working with a bigger dataset of movielens , dataframe of shape(22884377, 4), i am unable to create a pivot tabel. The error is: Unstacked DataFrame is too big, causing int32 overflow PLEASE PROVIDE A SOLUTION TO THIS SIR!!
error in ----> 3 from sklearn.metrics.pairwise import consine_similarity ImportError: cannot import name 'consine_similarity' from 'sklearn.metrics.pairwise' (/usr/local/lib/python3.7/dist-packages/sklearn/metrics/pairwise.py) ---------------------------------------------------------------------------
I really liked the way you implemented and i started using them but i got stuck at a point when i am using the pivot_table. My jupyter notebook kernel dies every time when i try running it. Can someone please help me how to overcome that.
hi what would you do if you had 278k users and 250k books and the pivot gets out bounds so you have ratings in three columns - user movie rating? thanks
You will have to use a distributed / parallel approach. Have a look at this: endymecy.gitbooks.io/spark-ml-source-analysis/content/%E6%8E%A8%E8%8D%90/papers/Large-scale%20Parallel%20Collaborative%20Filtering%20the%20Netflix%20Prize.pdf
Hlo mihir sir, the cource is very good i want to suggest you please update code heroku I want to become campus ambassador of this platform can you provide me.
sir, nice video . but i want to know how to do user similarity recommended system of videos, here the data consists only users and they liked videos (no rating required), can u help me
Hi Qaisar, could you please elaborate on what you mean by multiple criteria for a movie? Do you mean having different features such as movie director, tags etc.?
You can, however, you will have to find some frameworks and tools to do so. Python and R are used most of the times as they have wonderful tools to help out as sci-kit learn. If you understood the maths completely and can implement it by yourself go for it. However, I still will recommend at least looking up for a matrix multiplication library for C#. This is vital as multiplying millions of numbers is really time-consuming and your code may take days to train. Or may not train at all. Libraries have really efficient implementations of matrix multiplication and even utilise hardware acceleration with GPU sometimes.
Hi, i have used your approach for my master thesis, but there is a big question, how the prediction will make? the prediction is the last step in collaborative filtering. How can i cover your approach in my letter?
@@torrestam8527 I think it doesn't matters if you didn't add that user to matrix ,because you are finding the similar movie on the rating given to previous movies as we are using the item centered method.
Please get it from here: raw.githubusercontent.com/codeheroku/Introduction-to-Machine-Learning/master/Collaborative%20Filtering/dataset/toy_dataset.csv
For someone who doesn't even have good coding experience. I had to finish a demo for the work. I was able to complete it with your tutorial. Excellent work.
DAMN.....!!!!!!!!!!
I tried so hard to learn ML on other channels.
All of them were so confusing and longer than my attention span.
But your videos keep me concentrated all the way.
Great job man, hope UA-cam search engine will be more generous on you.
This tutorial had really helped me a lot ! Thank you so much for this wonderful content and explaining it so clearly ! I wish you all the best !!
Such a great explaination. You need many more subscribers.
Hi! thanks for your motivating words and support 😀
interesting project.... your way of teaching is very simple and explanations are really good... looking forward for more projects to ad in my resume...😅😅
Thanks for your kind words :)
such a simple teacher , he made hard stuff very easy
Hi great video! I personally think the axises during the 'Quantifying the similarity' part can be a bit confusing. If you are measuring the similarities of the Users, then u are comparing the User's rating score on different movies. So it should be U1 and U2 marked on the dots instead of M1 and M2, and M1 & M2 on the axises. Because when it expands from 2D to nD, the vector will be combinations of U1 and U2's rating on different movies, and each of the axis will be M1, M2....Mn
Great Tutorial, it helps me to understand how to implement a recommendation system in my projects about anime
Did you complete your project??
@@dhaanish0264 Yep, It usually takes a long time (10 min) to run the algorithm because the amount of data but it works
@@juandiegorodriguez7583 could you share me the code , I want a reference as I'm working on the same project but with different website
Hi, can make a video of how after training a model which predicts rating by a user of an item, multiple recommendations for a particular user based on their item rakings are generated?? It would be really helpful
There is an error in how the non-rated entries are treated. specifically, if nans are replaced by 0 BEFORE mean normalization, you are basically penalising that unrated item. The right thing to do is to take the unrated items as 0 AFTER mean normalization, therefore giving them the same user rating as the mean.
i was also wondering whether there were other approaches... in a large normal dataset, most (the vast majority) of the dataset would wind up being values that are NaNs converted to means... having such a large proportion of imputed values must impact the quality of the predictions, no?
Can we use the built in standard scaler instead of creating that standardize helper method?
You are the best sir, teaching method really helps!
This was so incredibly helpful, thank you so much!
Thank you for the video! One question, though.. would it be better to use df.mean().mean() as the mean rating of all users instead of 2.5? Many thanks!
Very well explained! Thanks. Could you please explain me how the mean 2.5 was subtracted from user's rating which you mentioned as a trick @21:02? I was trying out this on books recommendation dataset.
Because all the ratings at that time were on a scale of 0 to 5, and he converted it to the scale of -2.5 to 2.5 for obvious reasons. Hope this helps 😃
@@Just_Moh_it Thanks!
It would be so great if you can do something more about evaluation metrics to evaluate your model
Thank you
you all prolly dont care at all but does anyone know a method to get back into an Instagram account??
I was stupid forgot the password. I love any help you can give me
@Elian Christian Instablaster :)
@Alejandro Oliver I really appreciate your reply. I found the site through google and I'm waiting for the hacking stuff now.
I see it takes quite some time so I will reply here later with my results.
@Alejandro Oliver it worked and I actually got access to my account again. Im so happy!
Thank you so much you really help me out !
@Elian Christian You are welcome :)
very Nice, easy to understand all points
The session was really helpful.Thankyou!
Thank you 🙌
Kindly tell me the accuracy...
Thanks. Very helpful I managed to redo your code without problem
Hello sir, I have some confusion, our csv will not be the same as in this tutorial, i.e movies in column and ratings in rows our data will be like i'll have three column one is user_id , ratings, product_id how will I apply this filtering on such data, please guide me
notebooks.azure.com/hello-codeheroku/projects/collab-filtering here u will get
how to evaluate such type of recommendation system?
Nice, could you please shown how to deploy the model and connect with a front end react app
HI, could you please do a video on creating a hybrid recommendation system
Very nice explanation thanks alot
Shouldn't it be axis=1 in standardize method
can you please suggest how should we split the data into train/test and evalute our model
For your standardize function, why do you include the 0s in your calculation of the mean when normalizing?
Hi, thanks for the great tutorial. In the toy dataset example, what do I need to do in order to find similar users (not movies) based on action_lover's ratings?
Does standardise() function remain same? eg. if we wish to perform user-user CF, do we need to standardise column instead?
amazing tutorial, thanks.. :))
13:49
I'm not sure if that's the correct way to standardize data. The correct formula should be:
x_new = (x - mean)/std
I have the same question too
really appreciated. helps me a lot. thank you. Good luck for channel
you teaching skills are way better...thanks for this...also i have a doubt. suppose i wanna build a hybrid system(content based+collaberative) the way to do is that get recommendations(content based+collaberative) from both and combine to form one .example: recommendation1=[m1,m3,m6] and recom2=[m4,m5,m6,m8,m9] then final recomndation=[m6,m1,m3,m6,m4,m5,m8,m9] . is this the way how it should be done or there are other better way please tell.
I have a small doubt when you penalize the user rating by -2.5 for the case of romantic then won't that affect the scores for action movies.
There is no such problem because ONLY if (user_Ratings-2.5) is negative (i.e small rating) the multiplication will move it even more negative when if it is positive the product will sure be a positive number.
Nice demonstration. Question: i have developed model based cf and predicted values. How can i recommend items from this model?please help me
According to your steps shown in the website for application development on medium,when we are trying to open test.html file it appears blank.Can you please explain the testing API step again.
Yes, there is no API located at /movies/default/call/json/get_recommendations
Great explanation!
Thank You Sir
I have a dataset and I want to create a model which will predict top3 movies to user, such that final output should be CSV file with columns user_id and top3 how to do this , pls help me
very well explained... Thank you!
how to split the training and test data set and measure the mean squared error, please help me in this problem
Great tutorial! how can I deploy it in the android app? can we convert it into TFlite?
Yes you can . It has been already done . Check the official repo of tf
@@karuneshpalekar5212 Thanks!! but i want to make tf model using this tutorial approch.
I understood the tutorial but how we can integrate this system into real website ?nobody explains it in detailed. Just Jupiter notebooks.!!!
In the third cell why the output is not displaying all the user id's?!!!.... And why it displays Only Five user id's
Please please please help urgently
When i get to the recommendation part... I'm getting an error
No matter what movie i say it gives me a KeyError and says there has been an exception handling error
Please help Asap
you might have written: user_ratings=rating.pivot_table(index=['userId'],columns=['title'],values=['rating'])
instead of: user_ratings=rating.pivot_table(index=['userId'],columns=['title'],values='rating')
remove the braces from rating
hmm how can I get list similar movies afterall? It seems like the final list we have is the list of rating values, not the movieId, so I don't know how to get information of the movies from the final recommended list?
Sir, while working with a bigger dataset of movielens , dataframe of shape(22884377, 4), i am unable to create a pivot tabel. The error is: Unstacked DataFrame is too big, causing int32 overflow
PLEASE PROVIDE A SOLUTION TO THIS SIR!!
thanks a lot for your really really useful tutorial. Keep going .....
can you please upload the toy dataset again, the given link is not working. If anyone else has it then pls reply.
So this is user based collaborative filtering??
Very good and funny videos bring a great sense of entertainment!
error in
----> 3 from sklearn.metrics.pairwise import consine_similarity
ImportError: cannot import name 'consine_similarity' from 'sklearn.metrics.pairwise' (/usr/local/lib/python3.7/dist-packages/sklearn/metrics/pairwise.py)
---------------------------------------------------------------------------
I really liked the way you implemented and i started using them but i got stuck at a point when i am using the pivot_table. My jupyter notebook kernel dies every time when i try running it. Can someone please help me how to overcome that.
Hi, Could you please share Car Recommendation system
Were the issues/challenges dealt with in any other tutorial or video?
is it possible to include an evaluation metrics?
hi what would you do if you had 278k users and 250k books and the pivot gets out bounds so you have ratings in three columns - user movie rating? thanks
You will have to use a distributed / parallel approach. Have a look at this:
endymecy.gitbooks.io/spark-ml-source-analysis/content/%E6%8E%A8%E8%8D%90/papers/Large-scale%20Parallel%20Collaborative%20Filtering%20the%20Netflix%20Prize.pdf
I liked it??? I loved it!!!
I subscribed man......great content
Plase help me here....Why there is a use of Standardize function? please anyone!!
Sir how we predict . If one user gives ratings one movie1 to 5 then how we predict for movie2.
really good video and explanation, +1 sub bro
Thanks for your motivation :)
Loved it.
Can I take this code for my final year project?
I just loved it
Can I Have more than 1 indexes because I want to filter movie on user and genres basis
Can you make me a collaborative recommendations system project
But what if I want to find out the NAN values using python?
Sir how can we deploy this model in flutter application
Item based : movie 6
really helpful...
What if in my dataset, instead of NaN, all the values are already 0. Then should I use the fillna(0) function?
function call will have no effect on your code because your dataset already contain 0 value
When I ran the final cell it was coming series object has no attribute sort
it was great
hello , how can i access to the power point of this video
How to convert this to a web application?
Does it not have ui ???
Hlo mihir sir, the cource is very good i want to suggest you please update code heroku I want to become campus ambassador of this platform can you provide me.
sir, nice video .
but i want to know how to do user similarity recommended system of videos, here the data consists only users and they liked videos (no rating required), can u help me
Based on the duration of a particular genre that a user watched ....
hi,, can i get the codes for same example in R
Which algorithm is implemented here?
What if we have multiple criteria for movie? Then how can we find similarities.
Hi Qaisar, could you please elaborate on what you mean by multiple criteria for a movie? Do you mean having different features such as movie director, tags etc.?
Where can I find python file for this
i want dataset of movie and rating i didn't found can you send me the link here
drive.google.com/file/d/1WWQCl9w52M1sXNWd4JSKL7q-HHywk03p/view
how to do the rmse?
The website is not working!!!
can we make recommender system using collaborative filtering in simple Visual C#
You can, however, you will have to find some frameworks and tools to do so.
Python and R are used most of the times as they have wonderful tools to help out as sci-kit learn.
If you understood the maths completely and can implement it by yourself go for it. However, I still will recommend at least looking up for a matrix multiplication library for C#.
This is vital as multiplying millions of numbers is really time-consuming and your code may take days to train. Or may not train at all.
Libraries have really efficient implementations of matrix multiplication and even utilise hardware acceleration with GPU sometimes.
Please how can I join your whatsapp group ? I need further guide on a project I am working on
Can we do it with free Azure account?
Yes. And you don't need an Azure account, you just need a Microsoft account.
Hi, i have used your approach for my master thesis, but there is a big question, how the prediction will make? the prediction is the last step in collaborative filtering. How can i cover your approach in my letter?
Specify the goal and need money
I have a question, is "Action_lover" a new user to the system? Or this user has already been in the system?
It's a new user
@@omkarshete185 so what if "action_lover" is old user, should I add his ratings to the matrix before calculating cosine similarity?
@@torrestam8527 I think it doesn't matters if you didn't add that user to matrix ,because you are finding the similar movie on the rating given to previous movies as we are using the item centered method.
@@torrestam8527 are you working on this project ?
@@omkarshete185 yes I am :)) I'm 90% through it, thanks to this video.
14:39 where did you get the number 5?
Ratings are on the scale of 1 to 5
toy_dataset is not there in the google drive
Please get it from here: raw.githubusercontent.com/codeheroku/Introduction-to-Machine-Learning/master/Collaborative%20Filtering/dataset/toy_dataset.csv
hebat lu bro
How can we find the accuracy score here?
i need help with that too did u do it?
@@anishkc3234 Hi, I used RMSE , and MAE (aka root mean square error and mean absolute error)
@@giannismaris13 can i get the code??i need it
Not able to find the data😭
drive.google.com/file/d/1WWQCl9w52M1sXNWd4JSKL7q-HHywk03p/view
raw.githubusercontent.com/codeheroku/Introduction-to-Machine-Learning/master/Collaborative%20Filtering/dataset/toy_dataset.csv
Toydataset link?
It's in the Azure Notebook project
notebooks.azure.com/hello-codeheroku/projects/collab-filtering
@@CodeHeroku The Microsoft Azure Notebooks preview website will be retired on October 9th, 2020. Please transfer the course file to another platform.
Item to Item collaborating filtering explanation was bad
which algorithm is this??
did you find out?
Which algorithm is used here?