From 5a4cb88b1e27f01fe49c21a705f20b5a9076d9b6 Mon Sep 17 00:00:00 2001 From: simoninithomas Date: Sat, 3 Dec 2022 11:09:46 +0100 Subject: [PATCH] Modified Unit 1 with Sayak review + add final illustrations --- units/en/unit1/conclusion.mdx | 5 +++-- units/en/unit1/exp-exp-tradeoff.mdx | 2 +- units/en/unit1/hands-on.mdx | 8 ++++---- units/en/unit1/introduction.mdx | 2 +- units/en/unit1/quiz.mdx | 4 ++-- units/en/unit1/two-methods.mdx | 2 +- units/en/unit1/what-is-rl.mdx | 6 +++--- 7 files changed, 15 insertions(+), 14 deletions(-) diff --git a/units/en/unit1/conclusion.mdx b/units/en/unit1/conclusion.mdx index f41ef14..b501e20 100644 --- a/units/en/unit1/conclusion.mdx +++ b/units/en/unit1/conclusion.mdx @@ -10,6 +10,7 @@ Naturally, during the course, we’re going to use and explain these terms again In the next chapter, we’re going to reinforce what we just learn by **training Huggy the Dog to fetch the stick**. -You will be able then to play with him 🤗. +You will be able then to play with him 🤗. -ADD GIF HUGGY + diff --git a/units/en/unit1/exp-exp-tradeoff.mdx b/units/en/unit1/exp-exp-tradeoff.mdx index b616f81..65682b5 100644 --- a/units/en/unit1/exp-exp-tradeoff.mdx +++ b/units/en/unit1/exp-exp-tradeoff.mdx @@ -19,7 +19,7 @@ But if our agent does a little bit of exploration, it can **discover the big re This is what we call the exploration/exploitation trade-off. We need to balance how much we **explore the environment** and how much we **exploit what we know about the environment.** -Therefore, we must **define a rule that helps to handle this trade-off**. We’ll see in future chapters different ways to handle it. +Therefore, we must **define a rule that helps to handle this trade-off**. We’ll see the different ways to handle it in the future chapters. If it’s still confusing, **think of a real problem: the choice of a restaurant:** diff --git a/units/en/unit1/hands-on.mdx b/units/en/unit1/hands-on.mdx index 624e134..40e9c0f 100644 --- a/units/en/unit1/hands-on.mdx +++ b/units/en/unit1/hands-on.mdx @@ -1,12 +1,12 @@ # Hands on [[hands-on]] -Now that you've studied the bases of Reinforcement Learning, you’re ready to train your first two agents and share it with the community through the Hub 🔥: +Now that you've studied the bases of Reinforcement Learning, you’re ready to train your first agent and share it with the community through the Hub 🔥: -- A Lunar Lander agent that will learn to land correctly on the Moon 🌕 -- A car that needs to reach the top of the mountain ⛰️ . +A Lunar Lander agent that will learn to land correctly on the Moon 🌕 -TODO: Add illustration MountainCar and MoonLanding +LunarLander +And finally, you'll **upload this trained agent to the Hugging Face Hub 🤗, a free, open platform where people can share ML models, datasets, and demos.** Thanks to our leaderboard, you'll be able to compare your results with other classmates and exchange the best practices to improve your agent's scores Who will win the challenge for Unit 1 🏆? diff --git a/units/en/unit1/introduction.mdx b/units/en/unit1/introduction.mdx index bcdc0ea..337e264 100644 --- a/units/en/unit1/introduction.mdx +++ b/units/en/unit1/introduction.mdx @@ -13,7 +13,7 @@ Then, you'll **train your Deep Reinforcement Learning agent, a lunar lander to l LunarLander -And finally, you'll **upload it to the Hugging Face Hub 🤗, a free, open platform where people can share ML models, datasets, and demos.** +And finally, you'll **upload this trained agent to the Hugging Face Hub 🤗, a free, open platform where people can share ML models, datasets, and demos.** It's essential **to master these elements** before diving into implementing Deep Reinforcement Learning agents. The goal of this chapter is to give you solid foundations. diff --git a/units/en/unit1/quiz.mdx b/units/en/unit1/quiz.mdx index 1f55843..b29b41f 100644 --- a/units/en/unit1/quiz.mdx +++ b/units/en/unit1/quiz.mdx @@ -33,7 +33,7 @@ At every step: }, { text: "state s0, state s0, action a0, new state s1, reward r1", - explain: "The self-attention layer does contain attention \"heads,\" but these are not adaptation heads.", + explain: "", correct: true }, { @@ -57,7 +57,7 @@ At every step: explain: "" }, { - text: "The observation* is a complete description of the state of the world (there is no hidden information)", + text: "The observation is a complete description of the state of the world (there is no hidden information)", explain: "" }, { diff --git a/units/en/unit1/two-methods.mdx b/units/en/unit1/two-methods.mdx index cd3600d..88789e6 100644 --- a/units/en/unit1/two-methods.mdx +++ b/units/en/unit1/two-methods.mdx @@ -73,7 +73,7 @@ If we recap: ## Value-based methods [[value-based]] -In Value-based methods, instead of training a policy function, we **train a value function** that maps a state to the expected value **of being at that state.** +In value-based methods, instead of training a policy function, we **train a value function** that maps a state to the expected value **of being at that state.** The value of a state is the **expected discounted return** the agent can get if it **starts in that state, and then act according to our policy.** diff --git a/units/en/unit1/what-is-rl.mdx b/units/en/unit1/what-is-rl.mdx index cb1aa66..f807efe 100644 --- a/units/en/unit1/what-is-rl.mdx +++ b/units/en/unit1/what-is-rl.mdx @@ -6,9 +6,9 @@ To understand Reinforcement Learning, let’s start with the big picture. The idea behind Reinforcement Learning is that an agent (an AI) will learn from the environment by **interacting with it** (through trial and error) and **receiving rewards** (negative or positive) as feedback for performing actions. -Learning from interaction with the environment **comes from our natural experiences.** +Learning from interactions with the environment **comes from our natural experiences.** -For instance, imagine putting your little brother in front of a video game he never played, a controller in his hands, and letting him alone. +For instance, imagine putting your little brother in front of a video game he never played, a controller in his hands, and leaving him alone. Illustration_1 @@ -34,4 +34,4 @@ If we take now a formal definition: Reinforcement learning is a framework for solving control tasks (also called decision problems) by building agents that learn from the environment by interacting with it through trial and error and receiving rewards (positive or negative) as unique feedback. -But how Reinforcement Learning works? +But how does Reinforcement Learning work?