Causal Reinforcement Learning -- Part 1/2 (ICML tutorial)

Поділитися
Вставка
  • Опубліковано 30 жов 2024

КОМЕНТАРІ • 11

  • @olivrobinson
    @olivrobinson 3 роки тому +5

    Really clear and awesome material. Thank you for this! I'll be moving on to the next video...

  • @BenOgorek
    @BenOgorek 3 роки тому +3

    51:15 head hurting thinking about distinction between watching an agent do() something and watching an agent do something

    • @CausalAI
      @CausalAI  3 роки тому +2

      Hey Ben, I think the discussion after this summary slide may provide further elaboration, but let me know... -E

  • @AjayTalati
    @AjayTalati 3 роки тому +4

    What does he mean when he says the agents causal graph G, captures the "invariants" of the SCM M of the environment? Any simple example?

    • @spitfirerulz
      @spitfirerulz 3 роки тому +1

      I think it means that information about causal relationships of the variables in an SCM M (e.g. which variables "listen" to which others) can be adequately described by graph G. This captures the key properties of the causal relationship which do not vary in different circumstances.
      We would still need M because, for instance, we need to describe whether the functions are linear, complicated etc.

  • @edupignatelli
    @edupignatelli Рік тому

    Is there any publication that put these presentation in formal writing?

  • @shashank7601
    @shashank7601 4 роки тому +2

    I'm not sure if this is the right place to ask this question, but if hypothetically you give an arbitary SCM to an RL agent, will it then be able to perform all layers of the ladder of causation including counterfactuals? And how would this arbitary SCM look like (ie. how is it robust enough to perform counterfactuals). Is this SCM just hard coded if - then statements given to the agent?

    • @CausalAI
      @CausalAI  4 роки тому +13

      Hi there,
      That's an excellent and somewhat popular question, thank you for the opportunity of clarifying. I hypothesize that this is the case since it goes against our strongly held belief that ae ll what we need is more data, or that data is enough, not the case in causal inference. I'll try to elaborate next.
      Given a fully specified SCM, all the three layers (i.e., any counterfactual) are immediately computable through definitions 2, 5, and 7, as discussed in the PCH chapter (causalai.net/r60.pdf). Call this SCM M.
      Unfortunately, there is NOTHING about the output of M's evaluation that makes it more or less related to the actual SCM that underlies the environment, say M*.
      The first main result in the aforementioned chapter is called the "Causal Hierarchy Theorem" (CHT) (page 22), which says that even if we train the SCM M with layer 1 data, it still doesn't say anything about layers 2 or 3. I will leave you to check this statement (hint: the chapter should help). In other words, it makes not so much sense to ask about the "robustness" of M's predictions, given that they are unrelated to M*.
      Cheers, Elias

  • @sujith3914
    @sujith3914 4 роки тому +3

    It is unfortunate that the mindset of scaling up is sufficient to achieve the most sophisticated AI is a rather prevalent one and not one that is adopted by only a few. I guess there is a bright side to it that it provides people with very few resources a chance to make good contributions as well, because just scaling up is not sufficient.

    • @CausalAI
      @CausalAI  4 роки тому +5

      Hi Sujith, my hope with the tutorial is that if the examples and tasks are minimally clear, the understanding that scaling-up is not the only issue will follow naturally. In other words, there is no controversial statement, it's just basic logic. Deliberately, we designed the minimal or easiest possible examples so that this point could be understood; obviously, things just get more involved in larger settings.

  • @user-wr4yl7tx3w
    @user-wr4yl7tx3w Рік тому

    7:50