Solved Example on Q-learning in Reinforcement Learning/Q-Learning example

Q-learning Task, Q-learning Algorithm, Solved Example on Q-Learning

[@i_izhar03]

15:42 why didn't you choose the path 1->2->4->5->6 the sum here is 380 here and in 1->2->3 the sum is 190 only , we choose this path over 1->2->4->5->6 ?

[@sriveni]

i have given one optimal policy, u can choose any but in your path without passing through 3 you cannot go to 4 since 3 is the goal state we will stop there. Moreover the agent is not aware of the rewards , so in trail and error basis only he has to proceed. After certain number of iterations the agent might find the highest profitable path

[@Solinvencible]

Thank Sriveni ! A very good explained example!

[@sriveni]

Glad you liked it.Please do check other videos and share to all those who will be benefited.

[@starultra2863]

For 13:51, the Q(1,2) = 90, which is correct. If we would have started solving the problem from state 1 --> 2, then Q(1,2) would have been 0 right, in the first step ????????

[@sriveni]

yes u are correct.So only u have to start with some non-zero value,then only the q-matix will be updated.Since we are doing it manually we started with Q(2,3),the same is not the case with the algorithm

[@MS_P007]

Thank u for the lecture maam❤️

[@sriveni]

Thank you 😊

[@ramazandurmaz3012]

Why are we taking next states that have only non negative Q values? And there are 6 cells/states that one can end up in. But I can see only 4 actions(up,down,right,left). So why do we have 6x6 matrix?

[@sriveni]

As per the problem definition reward should be maximized,so take only non negative Q values.u can take a matrix of any size.This is like a board game where u can move only one step up,down,right,left

[@ramazandurmaz3012]

​@@sriveniI understand. One more question though. Why don't we include the learning rate alpha? Or at what step we need it?

[@sriveni]

@@ramazandurmaz3012 you can use to update the Q value using the formula
Q(s,a)

[@user-cu6gf2rj6y]

Does we have to update the R matrix also? Because certain point of time rewards in Q matrix gets updated and R will be remain as it is. And till what time we have to use R matrix if it is not get updated?

[@starultra2863]

no, we dont have to update the r matrix. the r matrix remains constant...... only the q matrix changes..........r matrix serves as a guidance for updating the q matrix with right values.......

[@sriveni]

yes,u need not update the R-matrix

[@sundarammuthu5020]

hi mam,
the maximum cumulative reward by seletcing the path 4,5,6 is 81+90+100=271 mam..

[@sriveni]

Yes ,two paths are mentioned in the solution 1st path is the shortest

[@sriveni]

First path is 1,,2,3 and 2nd path is 4,5,6 and 3.Both are mentioned in the solution graph

[@rams9256]

mam Can you please explain same Q learning concept with another example?

[@kartikeysokhal301]

[@sriveni]

little busy,i will do it

[@rahuljacker7171]

hme kb pta chlega ki hmara Q matrix ab update nhi hoga ??

[@sriveni]

In the process of updating if the values won't change after 2 to 3 iterations we will stop

[@afshinmonfared187]

only you said so or you said ok.

[@sriveni]

I didn't get your doubt

[@skayllacodm]

Parabens

[@sriveni]

Thank you