mirror of
https://github.com/huggingface/deep-rl-class.git
synced 2026-02-03 02:14:53 +08:00
Merge pull request #587 from nnilayy/fix_mlagents_env_issue
Fix: `mlagents-learn: command not found`
This commit is contained in:
Thomas Simonini
@@ -3,8 +3,8 @@
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "view-in-github",
|
||||
"colab_type": "text"
|
||||
"colab_type": "text",
|
||||
"id": "view-in-github"
|
||||
},
|
||||
"source": [
|
||||
"<a href=\"https://colab.research.google.com/github/huggingface/deep-rl-class/blob/main/notebooks/bonus-unit1/bonus-unit1.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
|
||||
@@ -21,31 +21,34 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "FMYrDriDujzX"
|
||||
},
|
||||
"source": [
|
||||
"<img src=\"https://huggingface.co/datasets/huggingface-deep-rl-course/course-images/resolve/main/en/unit2/thumbnail.png\" alt=\"Bonus Unit 1Thumbnail\">\n",
|
||||
"\n",
|
||||
"In this notebook, we'll reinforce what we learned in the first Unit by **teaching Huggy the Dog to fetch the stick and then play with it directly in your browser**\n",
|
||||
"\n",
|
||||
"⬇️ Here is an example of what **you will achieve at the end of the unit.** ⬇️ (launch ▶ to see)"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "FMYrDriDujzX"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"%%html\n",
|
||||
"<video controls autoplay><source src=\"https://huggingface.co/datasets/huggingface-deep-rl-course/course-images/resolve/main/en/notebooks/unit-bonus1/huggy.mp4\" type=\"video/mp4\"></video>"
|
||||
],
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "PnVhs1yYNyUF"
|
||||
},
|
||||
"execution_count": null,
|
||||
"outputs": []
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"%%html\n",
|
||||
"<video controls autoplay><source src=\"https://huggingface.co/datasets/huggingface-deep-rl-course/course-images/resolve/main/en/notebooks/unit-bonus1/huggy.mp4\" type=\"video/mp4\"></video>"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "x7oR6R-ZIbeS"
|
||||
},
|
||||
"source": [
|
||||
"### The environment 🎮\n",
|
||||
"\n",
|
||||
@@ -54,22 +57,22 @@
|
||||
"### The library used 📚\n",
|
||||
"\n",
|
||||
"- [MLAgents](https://github.com/Unity-Technologies/ml-agents)"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "x7oR6R-ZIbeS"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"We're constantly trying to improve our tutorials, so **if you find some issues in this notebook**, please [open an issue on the Github Repo](https://github.com/huggingface/deep-rl-class/issues)."
|
||||
],
|
||||
"metadata": {
|
||||
"id": "60yACvZwO0Cy"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"We're constantly trying to improve our tutorials, so **if you find some issues in this notebook**, please [open an issue on the Github Repo](https://github.com/huggingface/deep-rl-class/issues)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "Oks-ETYdO2Dc"
|
||||
},
|
||||
"source": [
|
||||
"## Objectives of this notebook 🏆\n",
|
||||
"\n",
|
||||
@@ -80,23 +83,23 @@
|
||||
"- Be able to play **with your trained Huggy directly in your browser**.\n",
|
||||
"\n",
|
||||
"\n"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "Oks-ETYdO2Dc"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "mUlVrqnBv2o1"
|
||||
},
|
||||
"source": [
|
||||
"## This notebook is from Deep Reinforcement Learning Course\n",
|
||||
"<img src=\"https://huggingface.co/datasets/huggingface-deep-rl-course/course-images/resolve/main/en/notebooks/deep-rl-course-illustration.jpg\" alt=\"Deep RL Course illustration\"/>"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "mUlVrqnBv2o1"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "pAMjaQpHwB_s"
|
||||
},
|
||||
"source": [
|
||||
"In this free course, you will:\n",
|
||||
"\n",
|
||||
@@ -110,13 +113,13 @@
|
||||
"\n",
|
||||
"\n",
|
||||
"The best way to keep in touch is to join our discord server to exchange with the community and with us 👉🏻 https://discord.gg/ydHrjt3WP5"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "pAMjaQpHwB_s"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "6r7Hl0uywFSO"
|
||||
},
|
||||
"source": [
|
||||
"## Prerequisites 🏗️\n",
|
||||
"\n",
|
||||
@@ -125,33 +128,30 @@
|
||||
"🔲 📚 **Develop an understanding of the foundations of Reinforcement learning** (MC, TD, Rewards hypothesis...) by doing Unit 1\n",
|
||||
"\n",
|
||||
"🔲 📚 **Read the introduction to Huggy** by doing Bonus Unit 1"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "6r7Hl0uywFSO"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "DssdIjk_8vZE"
|
||||
},
|
||||
"source": [
|
||||
"## Set the GPU 💪\n",
|
||||
"- To **accelerate the agent's training, we'll use a GPU**. To do that, go to `Runtime > Change Runtime type`\n",
|
||||
"\n",
|
||||
"<img src=\"https://huggingface.co/datasets/huggingface-deep-rl-course/course-images/resolve/main/en/notebooks/gpu-step1.jpg\" alt=\"GPU Step 1\">"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "DssdIjk_8vZE"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "sTfCXHy68xBv"
|
||||
},
|
||||
"source": [
|
||||
"- `Hardware Accelerator > GPU`\n",
|
||||
"\n",
|
||||
"<img src=\"https://huggingface.co/datasets/huggingface-deep-rl-course/course-images/resolve/main/en/notebooks/gpu-step2.jpg\" alt=\"GPU Step 2\">"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "sTfCXHy68xBv"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
@@ -159,7 +159,7 @@
|
||||
"id": "an3ByrXYQ4iK"
|
||||
},
|
||||
"source": [
|
||||
"## Clone the repository and install the dependencies 🔽\n",
|
||||
"## Clone the repository 🔽\n",
|
||||
"\n",
|
||||
"- We need to clone the repository, that contains **ML-Agents.**"
|
||||
]
|
||||
@@ -177,6 +177,77 @@
|
||||
"!git clone --depth 1 https://github.com/Unity-Technologies/ml-agents"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Setup the Virtual Environment 🔽\n",
|
||||
"- In order for the **ML-Agents** to run successfully in Colab, Colab's Python version must meet the library's Python requirements.\n",
|
||||
"\n",
|
||||
"- We can check for the supported Python version under the `python_requires` parameter in the `setup.py` files. These files are required to set up the **ML-Agents** library for use and can be found in the following locations:\n",
|
||||
" - `/content/ml-agents/ml-agents/setup.py`\n",
|
||||
" - `/content/ml-agents/ml-agents-envs/setup.py`\n",
|
||||
"\n",
|
||||
"- Colab's Current Python version(can be checked using `!python --version`) doesn't match the library's `python_requires` parameter, as a result installation may silently fail and lead to errors like these, when executing the same commands later:\n",
|
||||
" - `/bin/bash: line 1: mlagents-learn: command not found`\n",
|
||||
" - `/bin/bash: line 1: mlagents-push-to-hf: command not found`\n",
|
||||
"\n",
|
||||
"- To resolve this, we'll create a virtual environment with a Python version compatible with the **ML-Agents** library.\n",
|
||||
"\n",
|
||||
"`Note:` *For future compatibility, always check the `python_requires` parameter in the installation files and set your virtual environment to the maximum supported Python version in the given below script if the Colab's Python version is not compatible*"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Colab's Current Python Version (Incompatible with ML-Agents)\n",
|
||||
"!python --version"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Install virtualenv and create a virtual environment\n",
|
||||
"!pip install virtualenv\n",
|
||||
"!virtualenv myenv\n",
|
||||
"\n",
|
||||
"# Download and install Miniconda\n",
|
||||
"!wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh\n",
|
||||
"!chmod +x Miniconda3-latest-Linux-x86_64.sh\n",
|
||||
"!./Miniconda3-latest-Linux-x86_64.sh -b -f -p /usr/local\n",
|
||||
"\n",
|
||||
"# Activate Miniconda and install Python ver 3.10.12\n",
|
||||
"!source /usr/local/bin/activate\n",
|
||||
"!conda install -q -y --prefix /usr/local python=3.10.12 ujson # Specify the version here\n",
|
||||
"\n",
|
||||
"# Set environment variables for Python and conda paths\n",
|
||||
"!export PYTHONPATH=/usr/local/lib/python3.10/site-packages/\n",
|
||||
"!export CONDA_PREFIX=/usr/local/envs/myenv"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Python Version in New Virtual Environment (Compatible with ML-Agents)\n",
|
||||
"!python --version"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Installing the dependencies 🔽"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
@@ -218,23 +289,23 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"We downloaded the file Huggy.zip from https://github.com/huggingface/Huggy using `wget`"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "IHh_LXsRrrbM"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"We downloaded the file Huggy.zip from https://github.com/huggingface/Huggy using `wget`"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"!wget \"https://github.com/huggingface/Huggy/raw/main/Huggy.zip\" -O ./trained-envs-executables/linux/Huggy.zip"
|
||||
],
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "8xNAD1tRpy0_"
|
||||
},
|
||||
"execution_count": null,
|
||||
"outputs": []
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"!wget \"https://github.com/huggingface/Huggy/raw/main/Huggy.zip\" -O ./trained-envs-executables/linux/Huggy.zip"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
@@ -270,6 +341,9 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "dYKVj8yUvj55"
|
||||
},
|
||||
"source": [
|
||||
"## Let's recap how this environment works\n",
|
||||
"\n",
|
||||
@@ -307,13 +381,13 @@
|
||||
"- *Time penalty*: a fixed-time penalty given at every action to **force him to get to the stick as fast as possible**.\n",
|
||||
"- *Rotation penalty*: we penalize Huggy if **he spins too much and turns too quickly**.\n",
|
||||
"- *Getting to the target reward*: we reward Huggy for **reaching the target**."
|
||||
],
|
||||
"metadata": {
|
||||
"id": "dYKVj8yUvj55"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "NAuEq32Mwvtz"
|
||||
},
|
||||
"source": [
|
||||
"## Create the Huggy config file\n",
|
||||
"\n",
|
||||
@@ -333,13 +407,15 @@
|
||||
" <img src=\"https://huggingface.co/datasets/huggingface-deep-rl-course/course-images/resolve/main/en/unit1/create-huggy.png\" alt=\"Create huggy.yaml\" width=\"20%\">\n",
|
||||
"\n",
|
||||
"- Copy and paste the content below 🔽"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "NAuEq32Mwvtz"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "loQ0N5jhXW71"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"behaviors:\n",
|
||||
" Huggy:\n",
|
||||
@@ -367,24 +443,22 @@
|
||||
" max_steps: 2e6\n",
|
||||
" time_horizon: 1000\n",
|
||||
" summary_freq: 50000"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "loQ0N5jhXW71"
|
||||
},
|
||||
"execution_count": null,
|
||||
"outputs": []
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"- Don't forget to save the file!"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "oakN7UHwXdCX"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"- Don't forget to save the file!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "r9wv5NYGw-05"
|
||||
},
|
||||
"source": [
|
||||
"- **In the case you want to modify the hyperparameters**, in Google Colab notebook, you can click here to open the config.yaml: `/content/ml-agents/config/ppo/Huggy.yaml`\n",
|
||||
"\n",
|
||||
@@ -394,10 +468,7 @@
|
||||
"\n",
|
||||
"=> Just keep in mind that **decreasing the `checkpoint_interval` means more models to upload to the Hub and so a longer uploading time**\n",
|
||||
"We’re now ready to train our agent 🔥."
|
||||
],
|
||||
"metadata": {
|
||||
"id": "r9wv5NYGw-05"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
@@ -426,12 +497,12 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"The training will take 30 to 45min depending on your machine (don't forget to **set up a GPU**), go take a ☕️you deserve it 🤗."
|
||||
],
|
||||
"metadata": {
|
||||
"id": "lN32oWF8zPjs"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"The training will take 30 to 45min depending on your machine (don't forget to **set up a GPU**), go take a ☕️you deserve it 🤗."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
@@ -457,6 +528,9 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "izT6FpgNzZ6R"
|
||||
},
|
||||
"source": [
|
||||
"To be able to share your model with the community there are three more steps to follow:\n",
|
||||
"\n",
|
||||
@@ -469,10 +543,7 @@
|
||||
"\n",
|
||||
"- Copy the token\n",
|
||||
"- Run the cell below and paste the token"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "izT6FpgNzZ6R"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
@@ -488,12 +559,12 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"If you don't want to use a Google Colab or a Jupyter Notebook, you need to use this command instead: `huggingface-cli login`"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "ew59mK19zjtN"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"If you don't want to use a Google Colab or a Jupyter Notebook, you need to use this command instead: `huggingface-cli login`"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
@@ -508,6 +579,9 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "KK4fPfnczunT"
|
||||
},
|
||||
"source": [
|
||||
"And we define 4 parameters:\n",
|
||||
"\n",
|
||||
@@ -516,10 +590,7 @@
|
||||
"3. `--repo-id`: the name of the Hugging Face repo you want to create or update. It’s always <your huggingface username>/<the repo name>\n",
|
||||
"If the repo does not exist **it will be created automatically**\n",
|
||||
"4. `--commit-message`: since HF repos are git repository you need to define a commit message."
|
||||
],
|
||||
"metadata": {
|
||||
"id": "KK4fPfnczunT"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
@@ -534,6 +605,9 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "yborB0850FTM"
|
||||
},
|
||||
"source": [
|
||||
"Else, if everything worked you should have this at the end of the process(but with a different url 😆) :\n",
|
||||
"\n",
|
||||
@@ -546,22 +620,22 @@
|
||||
"It’s the link to your model repository. The repository contains a model card that explains how to use the model, your Tensorboard logs and your config file. **What’s awesome is that it’s a git repository, which means you can have different commits, update your repository with a new push, open Pull Requests, etc.**\n",
|
||||
"\n",
|
||||
"<img src=\"https://huggingface.co/datasets/huggingface-deep-rl-course/course-images/resolve/main/en/notebooks/unit-bonus1/modelcard.png\" alt=\"ml learn function\" width=\"100%\">"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "yborB0850FTM"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"But now comes the best: **being able to play with Huggy online 👀.**"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "5Uaon2cg0NrL"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"But now comes the best: **being able to play with Huggy online 👀.**"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "VMc4oOsE0QiZ"
|
||||
},
|
||||
"source": [
|
||||
"## Play with your Huggy 🐕\n",
|
||||
"\n",
|
||||
@@ -572,30 +646,30 @@
|
||||
"- Click on Play with my Huggy model\n",
|
||||
"\n",
|
||||
"<img src=\"https://huggingface.co/datasets/huggingface-deep-rl-course/course-images/resolve/main/en/notebooks/unit-bonus1/load-huggy.jpg\" alt=\"load-huggy\" width=\"100%\">"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "VMc4oOsE0QiZ"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "Djs8c5rR0Z8a"
|
||||
},
|
||||
"source": [
|
||||
"1. In step 1, type your username (your username is case sensitive: for instance, my username is ThomasSimonini not thomassimonini or ThOmasImoNInI) and click on the search button.",
|
||||
"1. In step 1, type your username (your username is case sensitive: for instance, my username is ThomasSimonini not thomassimonini or ThOmasImoNInI) and click on the search button.\n",
|
||||
"\n",
|
||||
"2. In step 2, select your model repository.",
|
||||
"2. In step 2, select your model repository.\n",
|
||||
"\n",
|
||||
"3. In step 3, **choose which model you want to replay**:",
|
||||
"3. In step 3, **choose which model you want to replay**:\n",
|
||||
" - I have multiple ones, since we saved a model every 500000 timesteps.\n",
|
||||
" - But since I want the more recent, I choose `Huggy.onnx`\n",
|
||||
"\n",
|
||||
"👉 What’s nice **is to try with different models steps to see the improvement of the agent.**"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "Djs8c5rR0Z8a"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "PI6dPWmh064H"
|
||||
},
|
||||
"source": [
|
||||
"Congrats on finishing this bonus unit!\n",
|
||||
"\n",
|
||||
@@ -605,18 +679,15 @@
|
||||
"\n",
|
||||
"\n",
|
||||
"## Keep Learning, Stay awesome 🤗"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "PI6dPWmh064H"
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"accelerator": "GPU",
|
||||
"colab": {
|
||||
"provenance": [],
|
||||
"include_colab_link": true,
|
||||
"private_outputs": true,
|
||||
"include_colab_link": true
|
||||
"provenance": []
|
||||
},
|
||||
"gpuClass": "standard",
|
||||
"kernelspec": {
|
||||
|
||||
@@ -3,8 +3,8 @@
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "view-in-github",
|
||||
"colab_type": "text"
|
||||
"colab_type": "text",
|
||||
"id": "view-in-github"
|
||||
},
|
||||
"source": [
|
||||
"<a href=\"https://colab.research.google.com/github/huggingface/deep-rl-class/blob/main/notebooks/bonus-unit1/bonus_unit1.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
|
||||
@@ -21,31 +21,34 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "FMYrDriDujzX"
|
||||
},
|
||||
"source": [
|
||||
"<img src=\"https://huggingface.co/datasets/huggingface-deep-rl-course/course-images/resolve/main/en/unit2/thumbnail.png\" alt=\"Bonus Unit 1Thumbnail\">\n",
|
||||
"\n",
|
||||
"In this notebook, we'll reinforce what we learned in the first Unit by **teaching Huggy the Dog to fetch the stick and then play with it directly in your browser**\n",
|
||||
"\n",
|
||||
"⬇️ Here is an example of what **you will achieve at the end of the unit.** ⬇️ (launch ▶ to see)"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "FMYrDriDujzX"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"%%html\n",
|
||||
"<video controls autoplay><source src=\"https://huggingface.co/datasets/huggingface-deep-rl-course/course-images/resolve/main/en/notebooks/unit-bonus1/huggy.mp4\" type=\"video/mp4\"></video>"
|
||||
],
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "PnVhs1yYNyUF"
|
||||
},
|
||||
"execution_count": null,
|
||||
"outputs": []
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"%%html\n",
|
||||
"<video controls autoplay><source src=\"https://huggingface.co/datasets/huggingface-deep-rl-course/course-images/resolve/main/en/notebooks/unit-bonus1/huggy.mp4\" type=\"video/mp4\"></video>"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "x7oR6R-ZIbeS"
|
||||
},
|
||||
"source": [
|
||||
"### The environment 🎮\n",
|
||||
"\n",
|
||||
@@ -54,22 +57,22 @@
|
||||
"### The library used 📚\n",
|
||||
"\n",
|
||||
"- [MLAgents](https://github.com/Unity-Technologies/ml-agents)"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "x7oR6R-ZIbeS"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"We're constantly trying to improve our tutorials, so **if you find some issues in this notebook**, please [open an issue on the Github Repo](https://github.com/huggingface/deep-rl-class/issues)."
|
||||
],
|
||||
"metadata": {
|
||||
"id": "60yACvZwO0Cy"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"We're constantly trying to improve our tutorials, so **if you find some issues in this notebook**, please [open an issue on the Github Repo](https://github.com/huggingface/deep-rl-class/issues)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "Oks-ETYdO2Dc"
|
||||
},
|
||||
"source": [
|
||||
"## Objectives of this notebook 🏆\n",
|
||||
"\n",
|
||||
@@ -80,23 +83,23 @@
|
||||
"- Be able to play **with your trained Huggy directly in your browser**.\n",
|
||||
"\n",
|
||||
"\n"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "Oks-ETYdO2Dc"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "mUlVrqnBv2o1"
|
||||
},
|
||||
"source": [
|
||||
"## This notebook is from Deep Reinforcement Learning Course\n",
|
||||
"<img src=\"https://huggingface.co/datasets/huggingface-deep-rl-course/course-images/resolve/main/en/notebooks/deep-rl-course-illustration.jpg\" alt=\"Deep RL Course illustration\"/>"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "mUlVrqnBv2o1"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "pAMjaQpHwB_s"
|
||||
},
|
||||
"source": [
|
||||
"In this free course, you will:\n",
|
||||
"\n",
|
||||
@@ -110,13 +113,13 @@
|
||||
"\n",
|
||||
"\n",
|
||||
"The best way to keep in touch is to join our discord server to exchange with the community and with us 👉🏻 https://discord.gg/ydHrjt3WP5"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "pAMjaQpHwB_s"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "6r7Hl0uywFSO"
|
||||
},
|
||||
"source": [
|
||||
"## Prerequisites 🏗️\n",
|
||||
"\n",
|
||||
@@ -125,41 +128,36 @@
|
||||
"🔲 📚 **Develop an understanding of the foundations of Reinforcement learning** (MC, TD, Rewards hypothesis...) by doing Unit 1\n",
|
||||
"\n",
|
||||
"🔲 📚 **Read the introduction to Huggy** by doing Bonus Unit 1"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "6r7Hl0uywFSO"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "DssdIjk_8vZE"
|
||||
},
|
||||
"source": [
|
||||
"## Set the GPU 💪\n",
|
||||
"- To **accelerate the agent's training, we'll use a GPU**. To do that, go to `Runtime > Change Runtime type`\n",
|
||||
"\n",
|
||||
"<img src=\"https://huggingface.co/datasets/huggingface-deep-rl-course/course-images/resolve/main/en/notebooks/gpu-step1.jpg\" alt=\"GPU Step 1\">"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "DssdIjk_8vZE"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "sTfCXHy68xBv"
|
||||
},
|
||||
"source": [
|
||||
"- `Hardware Accelerator > GPU`\n",
|
||||
"\n",
|
||||
"<img src=\"https://huggingface.co/datasets/huggingface-deep-rl-course/course-images/resolve/main/en/notebooks/gpu-step2.jpg\" alt=\"GPU Step 2\">"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "sTfCXHy68xBv"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "an3ByrXYQ4iK"
|
||||
},
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Clone the repository and install the dependencies 🔽\n",
|
||||
"## Clone the repository 🔽\n",
|
||||
"\n",
|
||||
"- We need to clone the repository, that contains **ML-Agents.**"
|
||||
]
|
||||
@@ -167,9 +165,7 @@
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "6WNoL04M7rTa"
|
||||
},
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"%%capture\n",
|
||||
@@ -177,12 +173,81 @@
|
||||
"!git clone --depth 1 https://github.com/Unity-Technologies/ml-agents"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Setup the Virtual Environment 🔽\n",
|
||||
"- In order for the **ML-Agents** to run successfully in Colab, Colab's Python version must meet the library's Python requirements.\n",
|
||||
"\n",
|
||||
"- We can check for the supported Python version under the `python_requires` parameter in the `setup.py` files. These files are required to set up the **ML-Agents** library for use and can be found in the following locations:\n",
|
||||
" - `/content/ml-agents/ml-agents/setup.py`\n",
|
||||
" - `/content/ml-agents/ml-agents-envs/setup.py`\n",
|
||||
"\n",
|
||||
"- Colab's Current Python version(can be checked using `!python --version`) doesn't match the library's `python_requires` parameter, as a result installation may silently fail and lead to errors like these, when executing the same commands later:\n",
|
||||
" - `/bin/bash: line 1: mlagents-learn: command not found`\n",
|
||||
" - `/bin/bash: line 1: mlagents-push-to-hf: command not found`\n",
|
||||
"\n",
|
||||
"- To resolve this, we'll create a virtual environment with a Python version compatible with the **ML-Agents** library.\n",
|
||||
"\n",
|
||||
"`Note:` *For future compatibility, always check the `python_requires` parameter in the installation files and set your virtual environment to the maximum supported Python version in the given below script if the Colab's Python version is not compatible*"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "d8wmVcMk7xKo"
|
||||
},
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Colab's Current Python Version (Incompatible with ML-Agents)\n",
|
||||
"!python --version"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Install virtualenv and create a virtual environment\n",
|
||||
"!pip install virtualenv\n",
|
||||
"!virtualenv myenv\n",
|
||||
"\n",
|
||||
"# Download and install Miniconda\n",
|
||||
"!wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh\n",
|
||||
"!chmod +x Miniconda3-latest-Linux-x86_64.sh\n",
|
||||
"!./Miniconda3-latest-Linux-x86_64.sh -b -f -p /usr/local\n",
|
||||
"\n",
|
||||
"# Activate Miniconda and install Python ver 3.10.12\n",
|
||||
"!source /usr/local/bin/activate\n",
|
||||
"!conda install -q -y --prefix /usr/local python=3.10.12 ujson # Specify the version here\n",
|
||||
"\n",
|
||||
"# Set environment variables for Python and conda paths\n",
|
||||
"!export PYTHONPATH=/usr/local/lib/python3.10/site-packages/\n",
|
||||
"!export CONDA_PREFIX=/usr/local/envs/myenv"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Python Version in New Virtual Environment (Compatible with ML-Agents)\n",
|
||||
"!python --version"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Installing the dependencies 🔽"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"%%capture\n",
|
||||
@@ -218,23 +283,23 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"We downloaded the file Huggy.zip from https://github.com/huggingface/Huggy using `wget`"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "IHh_LXsRrrbM"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"We downloaded the file Huggy.zip from https://github.com/huggingface/Huggy using `wget`"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"!wget \"https://github.com/huggingface/Huggy/raw/main/Huggy.zip\" -O ./trained-envs-executables/linux/Huggy.zip"
|
||||
],
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "8xNAD1tRpy0_"
|
||||
},
|
||||
"execution_count": null,
|
||||
"outputs": []
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"!wget \"https://github.com/huggingface/Huggy/raw/main/Huggy.zip\" -O ./trained-envs-executables/linux/Huggy.zip"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
@@ -270,6 +335,9 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "dYKVj8yUvj55"
|
||||
},
|
||||
"source": [
|
||||
"## Let's recap how this environment works\n",
|
||||
"\n",
|
||||
@@ -307,13 +375,13 @@
|
||||
"- *Time penalty*: a fixed-time penalty given at every action to **force him to get to the stick as fast as possible**.\n",
|
||||
"- *Rotation penalty*: we penalize Huggy if **he spins too much and turns too quickly**.\n",
|
||||
"- *Getting to the target reward*: we reward Huggy for **reaching the target**."
|
||||
],
|
||||
"metadata": {
|
||||
"id": "dYKVj8yUvj55"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "NAuEq32Mwvtz"
|
||||
},
|
||||
"source": [
|
||||
"## Create the Huggy config file\n",
|
||||
"\n",
|
||||
@@ -333,13 +401,15 @@
|
||||
" <img src=\"https://huggingface.co/datasets/huggingface-deep-rl-course/course-images/resolve/main/en/unit1/create-huggy.png\" alt=\"Create huggy.yaml\" width=\"20%\">\n",
|
||||
"\n",
|
||||
"- Copy and paste the content below 🔽"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "NAuEq32Mwvtz"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "loQ0N5jhXW71"
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"behaviors:\n",
|
||||
" Huggy:\n",
|
||||
@@ -367,24 +437,22 @@
|
||||
" max_steps: 2e6\n",
|
||||
" time_horizon: 1000\n",
|
||||
" summary_freq: 50000"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "loQ0N5jhXW71"
|
||||
},
|
||||
"execution_count": null,
|
||||
"outputs": []
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"- Don't forget to save the file!"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "oakN7UHwXdCX"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"- Don't forget to save the file!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "r9wv5NYGw-05"
|
||||
},
|
||||
"source": [
|
||||
"- **In the case you want to modify the hyperparameters**, in Google Colab notebook, you can click here to open the config.yaml: `/content/ml-agents/config/ppo/Huggy.yaml`\n",
|
||||
"\n",
|
||||
@@ -394,10 +462,7 @@
|
||||
"\n",
|
||||
"=> Just keep in mind that **decreasing the `checkpoint_interval` means more models to upload to the Hub and so a longer uploading time**\n",
|
||||
"We’re now ready to train our agent 🔥."
|
||||
],
|
||||
"metadata": {
|
||||
"id": "r9wv5NYGw-05"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
@@ -426,12 +491,12 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"The training will take 30 to 45min depending on your machine (don't forget to **set up a GPU**), go take a ☕️you deserve it 🤗."
|
||||
],
|
||||
"metadata": {
|
||||
"id": "lN32oWF8zPjs"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"The training will take 30 to 45min depending on your machine (don't forget to **set up a GPU**), go take a ☕️you deserve it 🤗."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
@@ -457,6 +522,9 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "izT6FpgNzZ6R"
|
||||
},
|
||||
"source": [
|
||||
"To be able to share your model with the community there are three more steps to follow:\n",
|
||||
"\n",
|
||||
@@ -469,10 +537,7 @@
|
||||
"\n",
|
||||
"- Copy the token\n",
|
||||
"- Run the cell below and paste the token"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "izT6FpgNzZ6R"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
@@ -488,12 +553,12 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"If you don't want to use a Google Colab or a Jupyter Notebook, you need to use this command instead: `huggingface-cli login`"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "ew59mK19zjtN"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"If you don't want to use a Google Colab or a Jupyter Notebook, you need to use this command instead: `huggingface-cli login`"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
@@ -508,6 +573,9 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "KK4fPfnczunT"
|
||||
},
|
||||
"source": [
|
||||
"And we define 4 parameters:\n",
|
||||
"\n",
|
||||
@@ -516,10 +584,7 @@
|
||||
"3. `--repo-id`: the name of the Hugging Face repo you want to create or update. It’s always <your huggingface username>/<the repo name>\n",
|
||||
"If the repo does not exist **it will be created automatically**\n",
|
||||
"4. `--commit-message`: since HF repos are git repository you need to define a commit message."
|
||||
],
|
||||
"metadata": {
|
||||
"id": "KK4fPfnczunT"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
@@ -534,6 +599,9 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "yborB0850FTM"
|
||||
},
|
||||
"source": [
|
||||
"Else, if everything worked you should have this at the end of the process(but with a different url 😆) :\n",
|
||||
"\n",
|
||||
@@ -546,22 +614,22 @@
|
||||
"It’s the link to your model repository. The repository contains a model card that explains how to use the model, your Tensorboard logs and your config file. **What’s awesome is that it’s a git repository, which means you can have different commits, update your repository with a new push, open Pull Requests, etc.**\n",
|
||||
"\n",
|
||||
"<img src=\"https://huggingface.co/datasets/huggingface-deep-rl-course/course-images/resolve/main/en/notebooks/unit-bonus1/modelcard.png\" alt=\"ml learn function\" width=\"100%\">"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "yborB0850FTM"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"But now comes the best: **being able to play with Huggy online 👀.**"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "5Uaon2cg0NrL"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"But now comes the best: **being able to play with Huggy online 👀.**"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "VMc4oOsE0QiZ"
|
||||
},
|
||||
"source": [
|
||||
"## Play with your Huggy 🐕\n",
|
||||
"\n",
|
||||
@@ -572,13 +640,13 @@
|
||||
"- Click on Play with my Huggy model\n",
|
||||
"\n",
|
||||
"<img src=\"https://huggingface.co/datasets/huggingface-deep-rl-course/course-images/resolve/main/en/notebooks/unit-bonus1/load-huggy.jpg\" alt=\"load-huggy\" width=\"100%\">"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "VMc4oOsE0QiZ"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "Djs8c5rR0Z8a"
|
||||
},
|
||||
"source": [
|
||||
"1. In step 1, choose your model repository which is the model id (in my case ThomasSimonini/ppo-Huggy).\n",
|
||||
"\n",
|
||||
@@ -587,13 +655,13 @@
|
||||
" - But since I want the more recent, I choose `Huggy.onnx`\n",
|
||||
"\n",
|
||||
"👉 What’s nice **is to try with different models steps to see the improvement of the agent.**"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "Djs8c5rR0Z8a"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "PI6dPWmh064H"
|
||||
},
|
||||
"source": [
|
||||
"Congrats on finishing this bonus unit!\n",
|
||||
"\n",
|
||||
@@ -603,18 +671,15 @@
|
||||
"\n",
|
||||
"\n",
|
||||
"## Keep Learning, Stay awesome 🤗"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "PI6dPWmh064H"
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"accelerator": "GPU",
|
||||
"colab": {
|
||||
"provenance": [],
|
||||
"include_colab_link": true,
|
||||
"private_outputs": true,
|
||||
"include_colab_link": true
|
||||
"provenance": []
|
||||
},
|
||||
"gpuClass": "standard",
|
||||
"kernelspec": {
|
||||
|
||||
@@ -3,8 +3,8 @@
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "view-in-github",
|
||||
"colab_type": "text"
|
||||
"colab_type": "text",
|
||||
"id": "view-in-github"
|
||||
},
|
||||
"source": [
|
||||
"<a href=\"https://colab.research.google.com/github/huggingface/deep-rl-class/blob/main/notebooks/unit5/unit5.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
|
||||
@@ -22,6 +22,9 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "97ZiytXEgqIz"
|
||||
},
|
||||
"source": [
|
||||
"<img src=\"https://huggingface.co/datasets/huggingface-deep-rl-course/course-images/resolve/main/en/unit7/thumbnail.png\" alt=\"Thumbnail\"/>\n",
|
||||
"\n",
|
||||
@@ -33,33 +36,33 @@
|
||||
"After that, you'll be able **to watch your agents playing directly on your browser**.\n",
|
||||
"\n",
|
||||
"For more information about the certification process, check this section 👉 https://huggingface.co/deep-rl-course/en/unit0/introduction#certification-process"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "97ZiytXEgqIz"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"⬇️ Here is an example of what **you will achieve at the end of this unit.** ⬇️\n"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "FMYrDriDujzX"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"⬇️ Here is an example of what **you will achieve at the end of this unit.** ⬇️\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "cBmFlh8suma-"
|
||||
},
|
||||
"source": [
|
||||
"<img src=\"https://huggingface.co/datasets/huggingface-deep-rl-course/course-images/resolve/main/en/unit7/pyramids.gif\" alt=\"Pyramids\"/>\n",
|
||||
"\n",
|
||||
"<img src=\"https://huggingface.co/datasets/huggingface-deep-rl-course/course-images/resolve/main/en/unit7/snowballtarget.gif\" alt=\"SnowballTarget\"/>"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "cBmFlh8suma-"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "A-cYE0K5iL-w"
|
||||
},
|
||||
"source": [
|
||||
"### 🎮 Environments:\n",
|
||||
"\n",
|
||||
@@ -69,22 +72,22 @@
|
||||
"### 📚 RL-Library:\n",
|
||||
"\n",
|
||||
"- [ML-Agents](https://github.com/Unity-Technologies/ml-agents)\n"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "A-cYE0K5iL-w"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"We're constantly trying to improve our tutorials, so **if you find some issues in this notebook**, please [open an issue on the GitHub Repo](https://github.com/huggingface/deep-rl-class/issues)."
|
||||
],
|
||||
"metadata": {
|
||||
"id": "qEhtaFh9i31S"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"We're constantly trying to improve our tutorials, so **if you find some issues in this notebook**, please [open an issue on the GitHub Repo](https://github.com/huggingface/deep-rl-class/issues)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "j7f63r3Yi5vE"
|
||||
},
|
||||
"source": [
|
||||
"## Objectives of this notebook 🏆\n",
|
||||
"\n",
|
||||
@@ -92,20 +95,17 @@
|
||||
"\n",
|
||||
"- Understand how works **ML-Agents**, the environment library.\n",
|
||||
"- Be able to **train agents in Unity Environments**.\n"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "j7f63r3Yi5vE"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "viNzVbVaYvY3"
|
||||
},
|
||||
"source": [
|
||||
"## This notebook is from the Deep Reinforcement Learning Course\n",
|
||||
"<img src=\"https://huggingface.co/datasets/huggingface-deep-rl-course/course-images/resolve/main/en/notebooks/deep-rl-course-illustration.jpg\" alt=\"Deep RL Course illustration\"/>"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "viNzVbVaYvY3"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
@@ -141,6 +141,9 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "xYO1uD5Ujgdh"
|
||||
},
|
||||
"source": [
|
||||
"# Let's train our agents 🚀\n",
|
||||
"\n",
|
||||
@@ -148,66 +151,130 @@
|
||||
"\n",
|
||||
"- For `Pyramids` : Mean Reward = 1.75\n",
|
||||
"- For `SnowballTarget` : Mean Reward = 15 or 30 targets hit in an episode.\n"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "xYO1uD5Ujgdh"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "DssdIjk_8vZE"
|
||||
},
|
||||
"source": [
|
||||
"## Set the GPU 💪\n",
|
||||
"- To **accelerate the agent's training, we'll use a GPU**. To do that, go to `Runtime > Change Runtime type`\n",
|
||||
"\n",
|
||||
"<img src=\"https://huggingface.co/datasets/huggingface-deep-rl-course/course-images/resolve/main/en/notebooks/gpu-step1.jpg\" alt=\"GPU Step 1\">"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "DssdIjk_8vZE"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "sTfCXHy68xBv"
|
||||
},
|
||||
"source": [
|
||||
"- `Hardware Accelerator > GPU`\n",
|
||||
"\n",
|
||||
"<img src=\"https://huggingface.co/datasets/huggingface-deep-rl-course/course-images/resolve/main/en/notebooks/gpu-step2.jpg\" alt=\"GPU Step 2\">"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "sTfCXHy68xBv"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "an3ByrXYQ4iK"
|
||||
},
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Clone the repository and install the dependencies 🔽\n"
|
||||
"## Clone the repository 🔽\n",
|
||||
"\n",
|
||||
"- We need to clone the repository, that contains **ML-Agents.**"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "6WNoL04M7rTa"
|
||||
},
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"%%capture\n",
|
||||
"# Clone the repository\n",
|
||||
"# Clone the repository (can take 3min)\n",
|
||||
"!git clone --depth 1 https://github.com/Unity-Technologies/ml-agents"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Setup the Virtual Environment 🔽\n",
|
||||
"- In order for the **ML-Agents** to run successfully in Colab, Colab's Python version must meet the library's Python requirements.\n",
|
||||
"\n",
|
||||
"- We can check for the supported Python version under the `python_requires` parameter in the `setup.py` files. These files are required to set up the **ML-Agents** library for use and can be found in the following locations:\n",
|
||||
" - `/content/ml-agents/ml-agents/setup.py`\n",
|
||||
" - `/content/ml-agents/ml-agents-envs/setup.py`\n",
|
||||
"\n",
|
||||
"- Colab's Current Python version(can be checked using `!python --version`) doesn't match the library's `python_requires` parameter, as a result installation may silently fail and lead to errors like these, when executing the same commands later:\n",
|
||||
" - `/bin/bash: line 1: mlagents-learn: command not found`\n",
|
||||
" - `/bin/bash: line 1: mlagents-push-to-hf: command not found`\n",
|
||||
"\n",
|
||||
"- To resolve this, we'll create a virtual environment with a Python version compatible with the **ML-Agents** library.\n",
|
||||
"\n",
|
||||
"`Note:` *For future compatibility, always check the `python_requires` parameter in the installation files and set your virtual environment to the maximum supported Python version in the given below script if the Colab's Python version is not compatible*"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "d8wmVcMk7xKo"
|
||||
},
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Colab's Current Python Version (Incompatible with ML-Agents)\n",
|
||||
"!python --version"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Install virtualenv and create a virtual environment\n",
|
||||
"!pip install virtualenv\n",
|
||||
"!virtualenv myenv\n",
|
||||
"\n",
|
||||
"# Go inside the repository and install the package\n",
|
||||
"# Download and install Miniconda\n",
|
||||
"!wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh\n",
|
||||
"!chmod +x Miniconda3-latest-Linux-x86_64.sh\n",
|
||||
"!./Miniconda3-latest-Linux-x86_64.sh -b -f -p /usr/local\n",
|
||||
"\n",
|
||||
"# Activate Miniconda and install Python ver 3.10.12\n",
|
||||
"!source /usr/local/bin/activate\n",
|
||||
"!conda install -q -y --prefix /usr/local python=3.10.12 ujson # Specify the version here\n",
|
||||
"\n",
|
||||
"# Set environment variables for Python and conda paths\n",
|
||||
"!export PYTHONPATH=/usr/local/lib/python3.10/site-packages/\n",
|
||||
"!export CONDA_PREFIX=/usr/local/envs/myenv"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Python Version in New Virtual Environment (Compatible with ML-Agents)\n",
|
||||
"!python --version"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Installing the dependencies 🔽"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"%%capture\n",
|
||||
"# Go inside the repository and install the package (can take 3min)\n",
|
||||
"%cd ml-agents\n",
|
||||
"!pip3 install -e ./ml-agents-envs\n",
|
||||
"!pip3 install -e ./ml-agents"
|
||||
@@ -215,15 +282,15 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "R5_7Ptd_kEcG"
|
||||
},
|
||||
"source": [
|
||||
"## SnowballTarget ⛄\n",
|
||||
"\n",
|
||||
"If you need a refresher on how this environments work check this section 👉\n",
|
||||
"https://huggingface.co/deep-rl-course/unit5/snowball-target"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "R5_7Ptd_kEcG"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
@@ -252,32 +319,32 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"We downloaded the file SnowballTarget.zip from https://github.com/huggingface/Snowball-Target using `wget`"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "ekSh8LWawkB5"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"We downloaded the file SnowballTarget.zip from https://github.com/huggingface/Snowball-Target using `wget`"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"!wget \"https://github.com/huggingface/Snowball-Target/raw/main/SnowballTarget.zip\" -O ./training-envs-executables/linux/SnowballTarget.zip"
|
||||
],
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "6LosWO50wa77"
|
||||
},
|
||||
"execution_count": null,
|
||||
"outputs": []
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"!wget \"https://github.com/huggingface/Snowball-Target/raw/main/SnowballTarget.zip\" -O ./training-envs-executables/linux/SnowballTarget.zip"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"We unzip the executable.zip file"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "_LLVaEEK3ayi"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"We unzip the executable.zip file"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
@@ -313,6 +380,9 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "NAuEq32Mwvtz"
|
||||
},
|
||||
"source": [
|
||||
"### Define the SnowballTarget config file\n",
|
||||
"- In ML-Agents, you define the **training hyperparameters into config.yaml files.**\n",
|
||||
@@ -333,7 +403,7 @@
|
||||
" checkpoint_interval: 50000\n",
|
||||
" max_steps: 200000\n",
|
||||
" time_horizon: 64\n",
|
||||
" threaded: true\n",
|
||||
" threaded: false\n",
|
||||
" hyperparameters:\n",
|
||||
" learning_rate: 0.0003\n",
|
||||
" learning_rate_schedule: linear\n",
|
||||
@@ -353,31 +423,28 @@
|
||||
" gamma: 0.99\n",
|
||||
" strength: 1.0\n",
|
||||
"```"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "NAuEq32Mwvtz"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "4U3sRH4N4h_l"
|
||||
},
|
||||
"source": [
|
||||
"<img src=\"https://huggingface.co/datasets/huggingface-deep-rl-course/course-images/resolve/main/en/unit7/snowballfight_config1.png\" alt=\"Config SnowballTarget\"/>\n",
|
||||
"<img src=\"https://huggingface.co/datasets/huggingface-deep-rl-course/course-images/resolve/main/en/unit7/snowballfight_config2.png\" alt=\"Config SnowballTarget\"/>"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "4U3sRH4N4h_l"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "JJJdo_5AyoGo"
|
||||
},
|
||||
"source": [
|
||||
"As an experimentation, you should also try to modify some other hyperparameters. Unity provides very [good documentation explaining each of them here](https://github.com/Unity-Technologies/ml-agents/blob/main/docs/Training-Configuration-File.md).\n",
|
||||
"\n",
|
||||
"Now that you've created the config file and understand what most hyperparameters do, we're ready to train our agent 🔥."
|
||||
],
|
||||
"metadata": {
|
||||
"id": "JJJdo_5AyoGo"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
@@ -406,12 +473,12 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"The training will take 10 to 35min depending on your config, go take a ☕️you deserve it 🤗."
|
||||
],
|
||||
"metadata": {
|
||||
"id": "lN32oWF8zPjs"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"The training will take 10 to 35min depending on your config, go take a ☕️you deserve it 🤗."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
@@ -437,6 +504,9 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "izT6FpgNzZ6R"
|
||||
},
|
||||
"source": [
|
||||
"To be able to share your model with the community there are three more steps to follow:\n",
|
||||
"\n",
|
||||
@@ -449,10 +519,7 @@
|
||||
"\n",
|
||||
"- Copy the token\n",
|
||||
"- Run the cell below and paste the token"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "izT6FpgNzZ6R"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
@@ -468,15 +535,18 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"If you don't want to use a Google Colab or a Jupyter Notebook, you need to use this command instead: `huggingface-cli login`"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "aSU9qD9_6dem"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"If you don't want to use a Google Colab or a Jupyter Notebook, you need to use this command instead: `huggingface-cli login`"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "KK4fPfnczunT"
|
||||
},
|
||||
"source": [
|
||||
"Then, we simply need to run `mlagents-push-to-hf`.\n",
|
||||
"\n",
|
||||
@@ -493,21 +563,18 @@
|
||||
"For instance:\n",
|
||||
"\n",
|
||||
"`!mlagents-push-to-hf --run-id=\"SnowballTarget1\" --local-dir=\"./results/SnowballTarget1\" --repo-id=\"ThomasSimonini/ppo-SnowballTarget\" --commit-message=\"First Push\"`"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "KK4fPfnczunT"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"!mlagents-push-to-hf --run-id=\"SnowballTarget1\" --local-dir=\"./results/SnowballTarget1\" --repo-id=\"ThomasSimonini/ppo-SnowballTarget\" --commit-message=\"First Push\""
|
||||
],
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "kAFzVB7OYj_H"
|
||||
},
|
||||
"execution_count": null,
|
||||
"outputs": []
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"!mlagents-push-to-hf --run-id=\"SnowballTarget1\" --local-dir=\"./results/SnowballTarget1\" --repo-id=\"ThomasSimonini/ppo-SnowballTarget\" --commit-message=\"First Push\""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
@@ -522,6 +589,9 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "yborB0850FTM"
|
||||
},
|
||||
"source": [
|
||||
"Else, if everything worked you should have this at the end of the process(but with a different url 😆) :\n",
|
||||
"\n",
|
||||
@@ -532,22 +602,22 @@
|
||||
"```\n",
|
||||
"\n",
|
||||
"It’s the link to your model, it contains a model card that explains how to use it, your Tensorboard and your config file. **What’s awesome is that it’s a git repository, that means you can have different commits, update your repository with a new push etc.**"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "yborB0850FTM"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"But now comes the best: **being able to visualize your agent online 👀.**"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "5Uaon2cg0NrL"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"But now comes the best: **being able to visualize your agent online 👀.**"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "VMc4oOsE0QiZ"
|
||||
},
|
||||
"source": [
|
||||
"### Watch your agent playing 👀\n",
|
||||
"\n",
|
||||
@@ -558,19 +628,19 @@
|
||||
"2. Launch the game and put it in full screen by clicking on the bottom right button\n",
|
||||
"\n",
|
||||
"<img src=\"https://huggingface.co/datasets/huggingface-deep-rl-course/course-images/resolve/main/en/unit7/snowballtarget_load.png\" alt=\"Snowballtarget load\"/>"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "VMc4oOsE0QiZ"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "Djs8c5rR0Z8a"
|
||||
},
|
||||
"source": [
|
||||
"1. In step 1, type your username (your username is case sensitive: for instance, my username is ThomasSimonini not thomassimonini or ThOmasImoNInI) and click on the search button.",
|
||||
"1. In step 1, type your username (your username is case sensitive: for instance, my username is ThomasSimonini not thomassimonini or ThOmasImoNInI) and click on the search button.\n",
|
||||
"\n",
|
||||
"2. In step 2, select your model repository.",
|
||||
"2. In step 2, select your model repository.\n",
|
||||
"\n",
|
||||
"3. In step 3, **choose which model you want to replay**:",
|
||||
"3. In step 3, **choose which model you want to replay**:\n",
|
||||
" - I have multiple ones, since we saved a model every 500000 timesteps.\n",
|
||||
" - But since I want the more recent, I choose `SnowballTarget.onnx`\n",
|
||||
"\n",
|
||||
@@ -579,13 +649,13 @@
|
||||
"And don't hesitate to share the best score your agent gets on discord in #rl-i-made-this channel 🔥\n",
|
||||
"\n",
|
||||
"Let's now try a harder environment called Pyramids..."
|
||||
],
|
||||
"metadata": {
|
||||
"id": "Djs8c5rR0Z8a"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "rVMwRi4y_tmx"
|
||||
},
|
||||
"source": [
|
||||
"## Pyramids 🏆\n",
|
||||
"\n",
|
||||
@@ -593,39 +663,36 @@
|
||||
"- Our environment executable is in a zip file.\n",
|
||||
"- We need to download it and place it to `./training-envs-executables/linux/`\n",
|
||||
"- We use a linux executable because we use colab, and colab machines OS is Ubuntu (linux)"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "rVMwRi4y_tmx"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"We downloaded the file Pyramids.zip from from https://huggingface.co/spaces/unity/ML-Agents-Pyramids/resolve/main/Pyramids.zip using `wget`"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "x2C48SGZjZYw"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"We downloaded the file Pyramids.zip from from https://huggingface.co/spaces/unity/ML-Agents-Pyramids/resolve/main/Pyramids.zip using `wget`"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"!wget \"https://huggingface.co/spaces/unity/ML-Agents-Pyramids/resolve/main/Pyramids.zip\" -O ./training-envs-executables/linux/Pyramids.zip"
|
||||
],
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "eWh8Pl3sjZY2"
|
||||
},
|
||||
"execution_count": null,
|
||||
"outputs": []
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"!wget \"https://huggingface.co/spaces/unity/ML-Agents-Pyramids/resolve/main/Pyramids.zip\" -O ./training-envs-executables/linux/Pyramids.zip"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"We unzip the executable.zip file"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "V5LXPOPujZY3"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"We unzip the executable.zip file"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
@@ -661,6 +728,9 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "fqceIATXAgih"
|
||||
},
|
||||
"source": [
|
||||
"### Modify the PyramidsRND config file\n",
|
||||
"- Contrary to the first environment which was a custom one, **Pyramids was made by the Unity team**.\n",
|
||||
@@ -672,32 +742,29 @@
|
||||
"👉 To do that, we go to config/ppo/PyramidsRND.yaml,**and modify these to max_steps to 1000000.**\n",
|
||||
"\n",
|
||||
"<img src=\"https://huggingface.co/datasets/huggingface-deep-rl-course/course-images/resolve/main/en/unit7/pyramids-config.png\" alt=\"Pyramids config\"/>"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "fqceIATXAgih"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "RI-5aPL7BWVk"
|
||||
},
|
||||
"source": [
|
||||
"As an experimentation, you should also try to modify some other hyperparameters, Unity provides a very [good documentation explaining each of them here](https://github.com/Unity-Technologies/ml-agents/blob/main/docs/Training-Configuration-File.md).\n",
|
||||
"\n",
|
||||
"We’re now ready to train our agent 🔥."
|
||||
],
|
||||
"metadata": {
|
||||
"id": "RI-5aPL7BWVk"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "s5hr1rvIBdZH"
|
||||
},
|
||||
"source": [
|
||||
"### Train the agent\n",
|
||||
"\n",
|
||||
"The training will take 30 to 45min depending on your machine, go take a ☕️you deserve it 🤗."
|
||||
],
|
||||
"metadata": {
|
||||
"id": "s5hr1rvIBdZH"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
@@ -723,50 +790,53 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"!mlagents-push-to-hf --run-id= # Add your run id --local-dir= # Your local dir --repo-id= # Your repo id --commit-message= # Your commit message"
|
||||
],
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"id": "yiEQbv7rB4mU"
|
||||
},
|
||||
"execution_count": null,
|
||||
"outputs": []
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"!mlagents-push-to-hf --run-id= # Add your run id --local-dir= # Your local dir --repo-id= # Your repo id --commit-message= # Your commit message"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "7aZfgxo-CDeQ"
|
||||
},
|
||||
"source": [
|
||||
"### Watch your agent playing 👀\n",
|
||||
"\n",
|
||||
"👉 https://huggingface.co/spaces/unity/ML-Agents-Pyramids"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "7aZfgxo-CDeQ"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "hGG_oq2n0wjB"
|
||||
},
|
||||
"source": [
|
||||
"### 🎁 Bonus: Why not train on another environment?\n",
|
||||
"Now that you know how to train an agent using MLAgents, **why not try another environment?**\n",
|
||||
"\n",
|
||||
"MLAgents provides 17 different and we’re building some custom ones. The best way to learn is to try things of your own, have fun.\n",
|
||||
"\n"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "hGG_oq2n0wjB"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
""
|
||||
],
|
||||
"metadata": {
|
||||
"id": "KSAkJxSr0z6-"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "YiyF4FX-04JB"
|
||||
},
|
||||
"source": [
|
||||
"You have the full list of the Unity official environments here 👉 https://github.com/Unity-Technologies/ml-agents/blob/develop/docs/Learning-Environment-Examples.md\n",
|
||||
"\n",
|
||||
@@ -775,13 +845,13 @@
|
||||
"For now we have integrated:\n",
|
||||
"- [Worm](https://huggingface.co/spaces/unity/ML-Agents-Worm) demo where you teach a **worm to crawl**.\n",
|
||||
"- [Walker](https://huggingface.co/spaces/unity/ML-Agents-Walker) demo where you teach an agent **to walk towards a goal**."
|
||||
],
|
||||
"metadata": {
|
||||
"id": "YiyF4FX-04JB"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"id": "PI6dPWmh064H"
|
||||
},
|
||||
"source": [
|
||||
"That’s all for today. Congrats on finishing this tutorial!\n",
|
||||
"\n",
|
||||
@@ -790,18 +860,15 @@
|
||||
"See you on Unit 6 🔥,\n",
|
||||
"\n",
|
||||
"## Keep Learning, Stay awesome 🤗"
|
||||
],
|
||||
"metadata": {
|
||||
"id": "PI6dPWmh064H"
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"accelerator": "GPU",
|
||||
"colab": {
|
||||
"provenance": [],
|
||||
"include_colab_link": true,
|
||||
"private_outputs": true,
|
||||
"include_colab_link": true
|
||||
"provenance": []
|
||||
},
|
||||
"gpuClass": "standard",
|
||||
"kernelspec": {
|
||||
|
||||
@@ -81,19 +81,67 @@ Before diving into the notebook, you need to:
|
||||
|
||||
<img src="https://huggingface.co/datasets/huggingface-deep-rl-course/course-images/resolve/main/en/notebooks/gpu-step2.jpg" alt="GPU Step 2">
|
||||
|
||||
## Clone the repository and install the dependencies 🔽
|
||||
- We need to clone the repository that **contains the experimental version of the library that allows you to push your trained agent to the Hub.**
|
||||
## Clone the repository 🔽
|
||||
|
||||
- We need to clone the repository, that contains **ML-Agents.**
|
||||
|
||||
```bash
|
||||
# Clone the repository
|
||||
# Clone the repository (can take 3min)
|
||||
git clone --depth 1 https://github.com/Unity-Technologies/ml-agents
|
||||
```
|
||||
|
||||
## Setup the Virtual Environment 🔽
|
||||
|
||||
- In order for the **ML-Agents** to run successfully in Colab, Colab's Python version must meet the library's Python requirements.
|
||||
|
||||
- We can check for the supported Python version under the `python_requires` parameter in the `setup.py` files. These files are required to set up the **ML-Agents** library for use and can be found in the following locations:
|
||||
- `/content/ml-agents/ml-agents/setup.py`
|
||||
- `/content/ml-agents/ml-agents-envs/setup.py`
|
||||
|
||||
- Colab's Current Python version(can be checked using `!python --version`) doesn't match the library's `python_requires` parameter, as a result installation may silently fail and lead to errors like these, when executing the same commands later:
|
||||
- `/bin/bash: line 1: mlagents-learn: command not found`
|
||||
- `/bin/bash: line 1: mlagents-push-to-hf: command not found`
|
||||
|
||||
- To resolve this, we'll create a virtual environment with a Python version compatible with the **ML-Agents** library.
|
||||
|
||||
`Note:` *For future compatibility, always check the `python_requires` parameter in the installation files and set your virtual environment to the maximum supported Python version in the given below script if the Colab's Python version is not compatible*
|
||||
|
||||
```bash
|
||||
# Go inside the repository and install the package
|
||||
cd ml-agents
|
||||
pip install -e ./ml-agents-envs
|
||||
pip install -e ./ml-agents
|
||||
# Colab's Current Python Version (Incompatible with ML-Agents)
|
||||
!python --version
|
||||
```
|
||||
|
||||
```bash
|
||||
# Install virtualenv and create a virtual environment
|
||||
!pip install virtualenv
|
||||
!virtualenv myenv
|
||||
|
||||
# Download and install Miniconda
|
||||
!wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
|
||||
!chmod +x Miniconda3-latest-Linux-x86_64.sh
|
||||
!./Miniconda3-latest-Linux-x86_64.sh -b -f -p /usr/local
|
||||
|
||||
# Activate Miniconda and install Python ver 3.10.12
|
||||
!source /usr/local/bin/activate
|
||||
!conda install -q -y --prefix /usr/local python=3.10.12 ujson # Specify the version here
|
||||
|
||||
# Set environment variables for Python and conda paths
|
||||
!export PYTHONPATH=/usr/local/lib/python3.10/site-packages/
|
||||
!export CONDA_PREFIX=/usr/local/envs/myenv
|
||||
```
|
||||
|
||||
```bash
|
||||
# Python Version in New Virtual Environment (Compatible with ML-Agents)
|
||||
!python --version
|
||||
```
|
||||
|
||||
## Installing the dependencies 🔽
|
||||
|
||||
```bash
|
||||
# Go inside the repository and install the package (can take 3min)
|
||||
%cd ml-agents
|
||||
pip3 install -e ./ml-agents-envs
|
||||
pip3 install -e ./ml-agents
|
||||
```
|
||||
|
||||
## SnowballTarget ⛄
|
||||
|
||||
@@ -64,15 +64,62 @@ Before diving into the notebook, you need to:
|
||||
|
||||
<img src="https://huggingface.co/datasets/huggingface-deep-rl-course/course-images/resolve/main/en/notebooks/gpu-step2.jpg" alt="GPU Step 2">
|
||||
|
||||
## Clone the repository and install the dependencies 🔽
|
||||
## Clone the repository 🔽
|
||||
|
||||
- We need to clone the repository, that contains ML-Agents.
|
||||
- We need to clone the repository, that contains **ML-Agents.**
|
||||
|
||||
```bash
|
||||
# Clone the repository (can take 3min)
|
||||
git clone --depth 1 https://github.com/Unity-Technologies/ml-agents
|
||||
```
|
||||
|
||||
## Setup the Virtual Environment 🔽
|
||||
|
||||
- In order for the **ML-Agents** to run successfully in Colab, Colab's Python version must meet the library's Python requirements.
|
||||
|
||||
- We can check for the supported Python version under the `python_requires` parameter in the `setup.py` files. These files are required to set up the **ML-Agents** library for use and can be found in the following locations:
|
||||
- `/content/ml-agents/ml-agents/setup.py`
|
||||
- `/content/ml-agents/ml-agents-envs/setup.py`
|
||||
|
||||
- Colab's Current Python version(can be checked using `!python --version`) doesn't match the library's `python_requires` parameter, as a result installation may silently fail and lead to errors like these, when executing the same commands later:
|
||||
- `/bin/bash: line 1: mlagents-learn: command not found`
|
||||
- `/bin/bash: line 1: mlagents-push-to-hf: command not found`
|
||||
|
||||
- To resolve this, we'll create a virtual environment with a Python version compatible with the **ML-Agents** library.
|
||||
|
||||
`Note:` *For future compatibility, always check the `python_requires` parameter in the installation files and set your virtual environment to the maximum supported Python version in the given below script if the Colab's Python version is not compatible*
|
||||
|
||||
```bash
|
||||
# Colab's Current Python Version (Incompatible with ML-Agents)
|
||||
!python --version
|
||||
```
|
||||
|
||||
```bash
|
||||
# Install virtualenv and create a virtual environment
|
||||
!pip install virtualenv
|
||||
!virtualenv myenv
|
||||
|
||||
# Download and install Miniconda
|
||||
!wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
|
||||
!chmod +x Miniconda3-latest-Linux-x86_64.sh
|
||||
!./Miniconda3-latest-Linux-x86_64.sh -b -f -p /usr/local
|
||||
|
||||
# Activate Miniconda and install Python ver 3.10.12
|
||||
!source /usr/local/bin/activate
|
||||
!conda install -q -y --prefix /usr/local python=3.10.12 ujson # Specify the version here
|
||||
|
||||
# Set environment variables for Python and conda paths
|
||||
!export PYTHONPATH=/usr/local/lib/python3.10/site-packages/
|
||||
!export CONDA_PREFIX=/usr/local/envs/myenv
|
||||
```
|
||||
|
||||
```bash
|
||||
# Python Version in New Virtual Environment (Compatible with ML-Agents)
|
||||
!python --version
|
||||
```
|
||||
|
||||
## Installing the dependencies 🔽
|
||||
|
||||
```bash
|
||||
# Go inside the repository and install the package (can take 3min)
|
||||
%cd ml-agents
|
||||
|
||||
Reference in New Issue
Block a user