From 8de22a76191a3a57e1133c6bd12a19364200894d Mon Sep 17 00:00:00 2001 From: unknown Date: Sat, 18 Jan 2025 15:02:00 +0530 Subject: [PATCH] Add new python venv for ML-Agents & Colab compatibility (Bonus-Unit-1) --- notebooks/bonus-unit1/bonus_unit1.ipynb | 311 ++++++++++++++---------- 1 file changed, 188 insertions(+), 123 deletions(-) diff --git a/notebooks/bonus-unit1/bonus_unit1.ipynb b/notebooks/bonus-unit1/bonus_unit1.ipynb index 2b43631..a85452b 100644 --- a/notebooks/bonus-unit1/bonus_unit1.ipynb +++ b/notebooks/bonus-unit1/bonus_unit1.ipynb @@ -3,8 +3,8 @@ { "cell_type": "markdown", "metadata": { - "id": "view-in-github", - "colab_type": "text" + "colab_type": "text", + "id": "view-in-github" }, "source": [ "\"Open" @@ -21,31 +21,34 @@ }, { "cell_type": "markdown", + "metadata": { + "id": "FMYrDriDujzX" + }, "source": [ "\"Bonus\n", "\n", "In this notebook, we'll reinforce what we learned in the first Unit by **teaching Huggy the Dog to fetch the stick and then play with it directly in your browser**\n", "\n", "⬇️ Here is an example of what **you will achieve at the end of the unit.** ⬇️ (launch ▶ to see)" - ], - "metadata": { - "id": "FMYrDriDujzX" - } + ] }, { "cell_type": "code", - "source": [ - "%%html\n", - "" - ], + "execution_count": null, "metadata": { "id": "PnVhs1yYNyUF" }, - "execution_count": null, - "outputs": [] + "outputs": [], + "source": [ + "%%html\n", + "" + ] }, { "cell_type": "markdown", + "metadata": { + "id": "x7oR6R-ZIbeS" + }, "source": [ "### The environment 🎮\n", "\n", @@ -54,22 +57,22 @@ "### The library used 📚\n", "\n", "- [MLAgents](https://github.com/Unity-Technologies/ml-agents)" - ], - "metadata": { - "id": "x7oR6R-ZIbeS" - } + ] }, { "cell_type": "markdown", - "source": [ - "We're constantly trying to improve our tutorials, so **if you find some issues in this notebook**, please [open an issue on the Github Repo](https://github.com/huggingface/deep-rl-class/issues)." - ], "metadata": { "id": "60yACvZwO0Cy" - } + }, + "source": [ + "We're constantly trying to improve our tutorials, so **if you find some issues in this notebook**, please [open an issue on the Github Repo](https://github.com/huggingface/deep-rl-class/issues)." + ] }, { "cell_type": "markdown", + "metadata": { + "id": "Oks-ETYdO2Dc" + }, "source": [ "## Objectives of this notebook 🏆\n", "\n", @@ -80,23 +83,23 @@ "- Be able to play **with your trained Huggy directly in your browser**.\n", "\n", "\n" - ], - "metadata": { - "id": "Oks-ETYdO2Dc" - } + ] }, { "cell_type": "markdown", + "metadata": { + "id": "mUlVrqnBv2o1" + }, "source": [ "## This notebook is from Deep Reinforcement Learning Course\n", "\"Deep" - ], - "metadata": { - "id": "mUlVrqnBv2o1" - } + ] }, { "cell_type": "markdown", + "metadata": { + "id": "pAMjaQpHwB_s" + }, "source": [ "In this free course, you will:\n", "\n", @@ -110,13 +113,13 @@ "\n", "\n", "The best way to keep in touch is to join our discord server to exchange with the community and with us 👉🏻 https://discord.gg/ydHrjt3WP5" - ], - "metadata": { - "id": "pAMjaQpHwB_s" - } + ] }, { "cell_type": "markdown", + "metadata": { + "id": "6r7Hl0uywFSO" + }, "source": [ "## Prerequisites 🏗️\n", "\n", @@ -125,41 +128,36 @@ "🔲 📚 **Develop an understanding of the foundations of Reinforcement learning** (MC, TD, Rewards hypothesis...) by doing Unit 1\n", "\n", "🔲 📚 **Read the introduction to Huggy** by doing Bonus Unit 1" - ], - "metadata": { - "id": "6r7Hl0uywFSO" - } + ] }, { "cell_type": "markdown", + "metadata": { + "id": "DssdIjk_8vZE" + }, "source": [ "## Set the GPU 💪\n", "- To **accelerate the agent's training, we'll use a GPU**. To do that, go to `Runtime > Change Runtime type`\n", "\n", "\"GPU" - ], - "metadata": { - "id": "DssdIjk_8vZE" - } + ] }, { "cell_type": "markdown", + "metadata": { + "id": "sTfCXHy68xBv" + }, "source": [ "- `Hardware Accelerator > GPU`\n", "\n", "\"GPU" - ], - "metadata": { - "id": "sTfCXHy68xBv" - } + ] }, { "cell_type": "markdown", - "metadata": { - "id": "an3ByrXYQ4iK" - }, + "metadata": {}, "source": [ - "## Clone the repository and install the dependencies 🔽\n", + "## Clone the repository 🔽\n", "\n", "- We need to clone the repository, that contains **ML-Agents.**" ] @@ -167,9 +165,7 @@ { "cell_type": "code", "execution_count": null, - "metadata": { - "id": "6WNoL04M7rTa" - }, + "metadata": {}, "outputs": [], "source": [ "%%capture\n", @@ -177,12 +173,81 @@ "!git clone --depth 1 https://github.com/Unity-Technologies/ml-agents" ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Setup the Virtual Environment 🔽\n", + "- In order for the **ML-Agents** to run successfully in Colab, Colab's Python version must meet the library's Python requirements.\n", + "\n", + "- We can check for the supported Python version under the `python_requires` parameter in the `setup.py` files. These files are required to set up the **ML-Agents** library for use and can be found in the following locations:\n", + " - `/content/ml-agents/ml-agents/setup.py`\n", + " - `/content/ml-agents/ml-agents-envs/setup.py`\n", + "\n", + "- Colab's Current Python version(can be checked using `!python --version`) doesn't match the library's `python_requires` parameter, as a result installation may silently fail and lead to errors like these, when executing the same commands later:\n", + " - `/bin/bash: line 1: mlagents-learn: command not found`\n", + " - `/bin/bash: line 1: mlagents-push-to-hf: command not found`\n", + "\n", + "- To resolve this, we'll create a virtual environment with a Python version compatible with the **ML-Agents** library.\n", + "\n", + "`Note:` *For future compatibility, always check the `python_requires` parameter in the installation files and set your virtual environment to the maximum supported Python version in the given below script if the Colab's Python version is not compatible*" + ] + }, { "cell_type": "code", "execution_count": null, - "metadata": { - "id": "d8wmVcMk7xKo" - }, + "metadata": {}, + "outputs": [], + "source": [ + "# Colab's Current Python Version (Incompatible with ML-Agents)\n", + "!python --version" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Install virtualenv and create a virtual environment\n", + "!pip install virtualenv\n", + "!virtualenv myenv\n", + "\n", + "# Download and install Miniconda\n", + "!wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh\n", + "!chmod +x Miniconda3-latest-Linux-x86_64.sh\n", + "!./Miniconda3-latest-Linux-x86_64.sh -b -f -p /usr/local\n", + "\n", + "# Activate Miniconda and install Python ver 3.10.12\n", + "!source /usr/local/bin/activate\n", + "!conda install -q -y --prefix /usr/local python=3.10.12 ujson # Specify the version here\n", + "\n", + "# Set environment variables for Python and conda paths\n", + "!export PYTHONPATH=/usr/local/lib/python3.10/site-packages/\n", + "!export CONDA_PREFIX=/usr/local/envs/myenv" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Python Version in New Virtual Environment (Compatible with ML-Agents)\n", + "!python --version" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Installing the dependencies 🔽" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, "outputs": [], "source": [ "%%capture\n", @@ -218,23 +283,23 @@ }, { "cell_type": "markdown", - "source": [ - "We downloaded the file Huggy.zip from https://github.com/huggingface/Huggy using `wget`" - ], "metadata": { "id": "IHh_LXsRrrbM" - } + }, + "source": [ + "We downloaded the file Huggy.zip from https://github.com/huggingface/Huggy using `wget`" + ] }, { "cell_type": "code", - "source": [ - "!wget \"https://github.com/huggingface/Huggy/raw/main/Huggy.zip\" -O ./trained-envs-executables/linux/Huggy.zip" - ], + "execution_count": null, "metadata": { "id": "8xNAD1tRpy0_" }, - "execution_count": null, - "outputs": [] + "outputs": [], + "source": [ + "!wget \"https://github.com/huggingface/Huggy/raw/main/Huggy.zip\" -O ./trained-envs-executables/linux/Huggy.zip" + ] }, { "cell_type": "code", @@ -270,6 +335,9 @@ }, { "cell_type": "markdown", + "metadata": { + "id": "dYKVj8yUvj55" + }, "source": [ "## Let's recap how this environment works\n", "\n", @@ -307,13 +375,13 @@ "- *Time penalty*: a fixed-time penalty given at every action to **force him to get to the stick as fast as possible**.\n", "- *Rotation penalty*: we penalize Huggy if **he spins too much and turns too quickly**.\n", "- *Getting to the target reward*: we reward Huggy for **reaching the target**." - ], - "metadata": { - "id": "dYKVj8yUvj55" - } + ] }, { "cell_type": "markdown", + "metadata": { + "id": "NAuEq32Mwvtz" + }, "source": [ "## Create the Huggy config file\n", "\n", @@ -333,13 +401,15 @@ " \"Create\n", "\n", "- Copy and paste the content below 🔽" - ], - "metadata": { - "id": "NAuEq32Mwvtz" - } + ] }, { "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "loQ0N5jhXW71" + }, + "outputs": [], "source": [ "behaviors:\n", " Huggy:\n", @@ -367,24 +437,22 @@ " max_steps: 2e6\n", " time_horizon: 1000\n", " summary_freq: 50000" - ], - "metadata": { - "id": "loQ0N5jhXW71" - }, - "execution_count": null, - "outputs": [] + ] }, { "cell_type": "markdown", - "source": [ - "- Don't forget to save the file!" - ], "metadata": { "id": "oakN7UHwXdCX" - } + }, + "source": [ + "- Don't forget to save the file!" + ] }, { "cell_type": "markdown", + "metadata": { + "id": "r9wv5NYGw-05" + }, "source": [ "- **In the case you want to modify the hyperparameters**, in Google Colab notebook, you can click here to open the config.yaml: `/content/ml-agents/config/ppo/Huggy.yaml`\n", "\n", @@ -394,10 +462,7 @@ "\n", "=> Just keep in mind that **decreasing the `checkpoint_interval` means more models to upload to the Hub and so a longer uploading time**\n", "We’re now ready to train our agent 🔥." - ], - "metadata": { - "id": "r9wv5NYGw-05" - } + ] }, { "cell_type": "markdown", @@ -426,12 +491,12 @@ }, { "cell_type": "markdown", - "source": [ - "The training will take 30 to 45min depending on your machine (don't forget to **set up a GPU**), go take a ☕️you deserve it 🤗." - ], "metadata": { "id": "lN32oWF8zPjs" - } + }, + "source": [ + "The training will take 30 to 45min depending on your machine (don't forget to **set up a GPU**), go take a ☕️you deserve it 🤗." + ] }, { "cell_type": "code", @@ -457,6 +522,9 @@ }, { "cell_type": "markdown", + "metadata": { + "id": "izT6FpgNzZ6R" + }, "source": [ "To be able to share your model with the community there are three more steps to follow:\n", "\n", @@ -469,10 +537,7 @@ "\n", "- Copy the token\n", "- Run the cell below and paste the token" - ], - "metadata": { - "id": "izT6FpgNzZ6R" - } + ] }, { "cell_type": "code", @@ -488,12 +553,12 @@ }, { "cell_type": "markdown", - "source": [ - "If you don't want to use a Google Colab or a Jupyter Notebook, you need to use this command instead: `huggingface-cli login`" - ], "metadata": { "id": "ew59mK19zjtN" - } + }, + "source": [ + "If you don't want to use a Google Colab or a Jupyter Notebook, you need to use this command instead: `huggingface-cli login`" + ] }, { "cell_type": "markdown", @@ -508,6 +573,9 @@ }, { "cell_type": "markdown", + "metadata": { + "id": "KK4fPfnczunT" + }, "source": [ "And we define 4 parameters:\n", "\n", @@ -516,10 +584,7 @@ "3. `--repo-id`: the name of the Hugging Face repo you want to create or update. It’s always /\n", "If the repo does not exist **it will be created automatically**\n", "4. `--commit-message`: since HF repos are git repository you need to define a commit message." - ], - "metadata": { - "id": "KK4fPfnczunT" - } + ] }, { "cell_type": "code", @@ -534,6 +599,9 @@ }, { "cell_type": "markdown", + "metadata": { + "id": "yborB0850FTM" + }, "source": [ "Else, if everything worked you should have this at the end of the process(but with a different url 😆) :\n", "\n", @@ -546,22 +614,22 @@ "It’s the link to your model repository. The repository contains a model card that explains how to use the model, your Tensorboard logs and your config file. **What’s awesome is that it’s a git repository, which means you can have different commits, update your repository with a new push, open Pull Requests, etc.**\n", "\n", "\"ml" - ], - "metadata": { - "id": "yborB0850FTM" - } + ] }, { "cell_type": "markdown", - "source": [ - "But now comes the best: **being able to play with Huggy online 👀.**" - ], "metadata": { "id": "5Uaon2cg0NrL" - } + }, + "source": [ + "But now comes the best: **being able to play with Huggy online 👀.**" + ] }, { "cell_type": "markdown", + "metadata": { + "id": "VMc4oOsE0QiZ" + }, "source": [ "## Play with your Huggy 🐕\n", "\n", @@ -572,13 +640,13 @@ "- Click on Play with my Huggy model\n", "\n", "\"load-huggy\"" - ], - "metadata": { - "id": "VMc4oOsE0QiZ" - } + ] }, { "cell_type": "markdown", + "metadata": { + "id": "Djs8c5rR0Z8a" + }, "source": [ "1. In step 1, choose your model repository which is the model id (in my case ThomasSimonini/ppo-Huggy).\n", "\n", @@ -587,13 +655,13 @@ " - But since I want the more recent, I choose `Huggy.onnx`\n", "\n", "👉 What’s nice **is to try with different models steps to see the improvement of the agent.**" - ], - "metadata": { - "id": "Djs8c5rR0Z8a" - } + ] }, { "cell_type": "markdown", + "metadata": { + "id": "PI6dPWmh064H" + }, "source": [ "Congrats on finishing this bonus unit!\n", "\n", @@ -603,18 +671,15 @@ "\n", "\n", "## Keep Learning, Stay awesome 🤗" - ], - "metadata": { - "id": "PI6dPWmh064H" - } + ] } ], "metadata": { "accelerator": "GPU", "colab": { - "provenance": [], + "include_colab_link": true, "private_outputs": true, - "include_colab_link": true + "provenance": [] }, "gpuClass": "standard", "kernelspec": {