CS_ALearn/deep-rl-class
1
0
Fork 0
mirror of https://github.com/huggingface/deep-rl-class.git synced 2026-04-13 18:00:45 +08:00
Code Issues Projects Releases Wiki Activity

Finalize introduction and marl

Browse Source
This commit is contained in:
simoninithomas
2023-01-31 15:57:01 +01:00
parent e4f039d2cc
commit bc7238c1bf
2 changed files with 34 additions and 23 deletions
Download Patch File Download Diff File Expand all files Collapse all files

18
units/en/unit7/introduction-to-marl.mdx
View File

@@ -2,10 +2,10 @@
## From single agent to multiple agents
From the first unit, we learned to train agents in a single-agent system. Where our agent was alone in its environment: **it was not cooperating or collaborating with other agents**.
In the first unit, we learned to train agents in a single-agent system. Where our agent was alone in its environment: **it was not cooperating or collaborating with other agents**.
<figure>
<img src=”https://huggingface.co/datasets/huggingface-deep-rl-course/course-images/resolve/main/en/unit10/patchwork.jpg alt=”Patchwork”/>
<img src="https://huggingface.co/datasets/huggingface-deep-rl-course/course-images/resolve/main/en/unit10/patchwork.jpg" alt="Patchwork"/>
<figcaption>
A patchwork of all the environments you've trained your agents on since the beginning of the course
</figcaption>
@@ -16,19 +16,19 @@ When we do Multi agents reinforcement learning (MARL), we are in a situation whe
For instance, you can think of a warehouse where **multiple robots need to navigate to load and unload packages**.
<figure>
<img src=”https://huggingface.co/datasets/huggingface-deep-rl-course/course-images/resolve/main/en/unit10/warehouse.jpg” alt=”Warehouse”/>
<img src="https://huggingface.co/datasets/huggingface-deep-rl-course/course-images/resolve/main/en/unit10/warehouse.jpg" alt="Warehouse"/>
<figcaption> <a href="[https://www.freepik.com/free-vector/robots-warehouse-interior-automated-machines_32117680.htm#query=warehouse robot&position=17&from_view=keyword](https://www.freepik.com/free-vector/robots-warehouse-interior-automated-machines_32117680.htm#query=warehouse%20robot&position=17&from_view=keyword)">Image by upklyak</a> on Freepik </figcaption>
Or a road with **several autonomous vehicles**.
<figure>
<img src=”https://huggingface.co/datasets/huggingface-deep-rl-course/course-images/resolve/main/en/unit10/selfdrivingcar.jpg” alt=”Self driving cars”/>
<img src="https://huggingface.co/datasets/huggingface-deep-rl-course/course-images/resolve/main/en/unit10/selfdrivingcar.jpg" alt="Self driving cars"/>
<figcaption>
<a href="[https://www.freepik.com/free-vector/autonomous-smart-car-automatic-wireless-sensor-driving-road-around-car-autonomous-smart-car-goes-scans-roads-observe-distance-automatic-braking-system_26413332.htm#query=self driving cars highway&position=34&from_view=search&track=ais](https://www.freepik.com/free-vector/autonomous-smart-car-automatic-wireless-sensor-driving-road-around-car-autonomous-smart-car-goes-scans-roads-observe-distance-automatic-braking-system_26413332.htm#query=self%20driving%20cars%20highway&position=34&from_view=search&track=ais)">Image by jcomp</a> on Freepik
</figcaption>
</figure>
In these examples, we have multiple agents interacting in the environment and with the other agents. This implies defining a multi-agent system. But first, let's understand the different types of multi-agent environments.
In these examples, we have **multiple agents interacting in the environment and with the other agents**. This implies defining a multi-agents system. But first, let's understand the different types of multi-agent environments.
## Different types of multi-agent environments
@@ -42,16 +42,16 @@ For instance, in a warehouse, **robots must collaborate to load and unload the p
For example, in a game of tennis, **each agent wants to beat the other agent**.
<img src=”https://huggingface.co/datasets/huggingface-deep-rl-course/course-images/resolve/main/en/unit10/tennis.png alt=”Tennis”/>
<img src="https://huggingface.co/datasets/huggingface-deep-rl-course/course-images/resolve/main/en/unit10/tennis.png alt="Tennis"/>
- *Mixed of both adversarial and cooperative*: like in our SoccerTwos environment, two agents are part of a team (blue or purple): they need to cooperate with each other and beat the opponent team.
<figure>
<img src=”https://huggingface.co/datasets/huggingface-deep-rl-course/course-images/resolve/main/en/unit10/soccertwos.gif” alt=”SoccerTwos”/>
<img src="https://huggingface.co/datasets/huggingface-deep-rl-course/course-images/resolve/main/en/unit10/soccertwos.gif" alt="SoccerTwos"/>
</figure>
<figcaption>This environment was made by the <a href=”https://github.com/Unity-Technologies/ml-agents”>Unity MLAgents Team</a></figcaption>
<figcaption>This environment was made by the <a href="https://github.com/Unity-Technologies/ml-agents">Unity MLAgents Team</a></figcaption>
So now we can ask how we design these multi-agent systems. Said differently, how can we train agents in a multi-agent setting?
So now we can ask how we design these multi-agent systems. Said differently, **how can we train agents in a multi-agents setting** ?

39
units/en/unit7/introduction.mdx
View File

@@ -1,25 +1,36 @@
# Introduction [[introduction]]
<img src=”https://huggingface.co/datasets/huggingface-deep-rl-course/course-images/resolve/main/en/unit0/thumbnail.jpg” alt=”Thumbnail”/>
<img src="https://huggingface.co/datasets/huggingface-deep-rl-course/course-images/resolve/main/en/unit0/thumbnail.png" alt="Thumbnail"/>
Since the beginning of this course, we learned to train agents in a single-agent system. Where our agent was alone in its environment: it was not cooperating or collaborating with other agents.
Since the beginning of this course, we learned to train agents in a *single-agent system* where our agent was alone in its environment: it was **not cooperating or collaborating with other agents**.
Our different agents worked great, and the single-agent system is useful for many applications.
This worked great, and the single-agent system is useful for many applications.
But, as humans, we live in a multi-agent world. Our intelligence comes from interaction with other agents. And so, our goal is to create agents that can interact with other humans and other agents.
Consequently, we must study how to train deep reinforcement learning agents in a multi-agent system to build robust agents that can adapt, collaborate, or compete.
So today, we’re going to learn the basics of this fascinating topic of multi-agents reinforcement learning (MARL).
And the most exciting part is that during this unit, you’re going to train your first agents in a multi-agents system: a 2vs2 soccer team that needs to beat the opponent team.
And you’re going to participate in AI vs. AI challenges where your trained agent will compete against other classmates’ agents every day and be ranked on a new leaderboard.
<figure>
<img src=”https://huggingface.co/datasets/huggingface-deep-rl-course/course-images/resolve/main/en/unit10/soccertwos.gif” alt=”SoccerTwos”/>
<figcaption>This environment was made by the <a href=”https://github.com/Unity-Technologies/ml-agents”>Unity MLAgents Team</a></figcaption>
<img src="https://huggingface.co/datasets/huggingface-deep-rl-course/course-images/resolve/main/en/unit10/patchwork.jpg" alt="Patchwork"/>
<figcaption>
A patchwork of all the environments you’ve trained your agents on since the beginning of the course
</figure>
But, as humans, **we live in a multi-agent world**. Our intelligence comes from interaction with other agents. And so, our **goal is to create agents that can interact with other humans and other agents**.
Consequently, we must study how to train deep reinforcement learning agents in a *multi-agents system* to build robust agents that can adapt, collaborate, or compete.
So today, we’re going to **learn the basics of this fascinating topic of multi-agents reinforcement learning (MARL)**.
And the most exciting part is that during this unit, you’re going to train your first agents in a multi-agents system: **a 2vs2 soccer team that needs to beat the opponent team**.
And you’re going to participate in **AI vs. AI challenge** where your trained agent will compete against other classmates’ agents every day and be ranked on a [new leaderboard]().
<figure>
<img src="https://huggingface.co/datasets/huggingface-deep-rl-course/course-images/resolve/main/en/unit10/soccertwos.gif" alt=”SoccerTwos”/>
<figcaption>This environment was made by the <a href="https://github.com/Unity-Technologies/ml-agents">Unity MLAgents Team</a></figcaption>
</figure>