15:42 why didn't you choose the path 1->2->4->5->6 the sum here is 380 here and in 1->2->3 the sum is 190 only , we choose this path over 1->2->4->5->6 ?
i have given one optimal policy, u can choose any but in your path without passing through 3 you cannot go to 4 since 3 is the goal state we will stop there. Moreover the agent is not aware of the rewards , so in trail and error basis only he has to proceed. After certain number of iterations the agent might find the highest profitable path
For 13:51, the Q(1,2) = 90, which is correct. If we would have started solving the problem from state 1 --> 2, then Q(1,2) would have been 0 right, in the first step ????????
yes u are correct.So only u have to start with some non-zero value,then only the q-matix will be updated.Since we are doing it manually we started with Q(2,3),the same is not the case with the algorithm
Why are we taking next states that have only non negative Q values? And there are 6 cells/states that one can end up in. But I can see only 4 actions(up,down,right,left). So why do we have 6x6 matrix?
As per the problem definition reward should be maximized,so take only non negative Q values.u can take a matrix of any size.This is like a board game where u can move only one step up,down,right,left
Does we have to update the R matrix also? Because certain point of time rewards in Q matrix gets updated and R will be remain as it is. And till what time we have to use R matrix if it is not get updated?
no, we dont have to update the r matrix. the r matrix remains constant...... only the q matrix changes..........r matrix serves as a guidance for updating the q matrix with right values.......
15:42 why didn't you choose the path 1->2->4->5->6 the sum here is 380 here and in 1->2->3 the sum is 190 only , we choose this path over 1->2->4->5->6 ?
i have given one optimal policy, u can choose any but in your path without passing through 3 you cannot go to 4 since 3 is the goal state we will stop there. Moreover the agent is not aware of the rewards , so in trail and error basis only he has to proceed. After certain number of iterations the agent might find the highest profitable path
Thank Sriveni ! A very good explained example!
Glad you liked it.Please do check other videos and share to all those who will be benefited.
For 13:51, the Q(1,2) = 90, which is correct. If we would have started solving the problem from state 1 --> 2, then Q(1,2) would have been 0 right, in the first step ????????
yes u are correct.So only u have to start with some non-zero value,then only the q-matix will be updated.Since we are doing it manually we started with Q(2,3),the same is not the case with the algorithm
Thank u for the lecture maam❤️
Thank you 😊
Why are we taking next states that have only non negative Q values? And there are 6 cells/states that one can end up in. But I can see only 4 actions(up,down,right,left). So why do we have 6x6 matrix?
As per the problem definition reward should be maximized,so take only non negative Q values.u can take a matrix of any size.This is like a board game where u can move only one step up,down,right,left
@@sriveniI understand. One more question though. Why don't we include the learning rate alpha? Or at what step we need it?
@@ramazandurmaz3012 you can use to update the Q value using the formula
Q(s,a)
Does we have to update the R matrix also? Because certain point of time rewards in Q matrix gets updated and R will be remain as it is. And till what time we have to use R matrix if it is not get updated?
no, we dont have to update the r matrix. the r matrix remains constant...... only the q matrix changes..........r matrix serves as a guidance for updating the q matrix with right values.......
yes,u need not update the R-matrix
hi mam,
the maximum cumulative reward by seletcing the path 4,5,6 is 81+90+100=271 mam..
Yes ,two paths are mentioned in the solution 1st path is the shortest
First path is 1,,2,3 and 2nd path is 4,5,6 and 3.Both are mentioned in the solution graph
mam Can you please explain same Q learning concept with another example?
little busy,i will do it
hme kb pta chlega ki hmara Q matrix ab update nhi hoga ??
In the process of updating if the values won't change after 2 to 3 iterations we will stop
only you said so or you said ok.
I didn't get your doubt
Parabens
Thank you