From 89a025e2458f3987b6df96fb533ccb3e5c39a20e Mon Sep 17 00:00:00 2001
From: Mike-Wazowsk1 <104924053+Mike-Wazowsk1@users.noreply.github.com>
Date: Tue, 11 Oct 2022 01:29:59 +0300
Subject: [PATCH] links fix
Replace 'utils' to 'rl_zoo3' for functions: load_from_hub and push_to_hab
https://github.com/DLR-RM/rl-baselines3-zoo doesn't have folder 'utils'
---
unit3/unit3.ipynb | 774 ++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 774 insertions(+)
create mode 100644 unit3/unit3.ipynb
diff --git a/unit3/unit3.ipynb b/unit3/unit3.ipynb
new file mode 100644
index 0000000..7f3d397
--- /dev/null
+++ b/unit3/unit3.ipynb
@@ -0,0 +1,774 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "colab_type": "text",
+ "id": "view-in-github"
+ },
+ "source": [
+ "![\"Open](\"https://colab.research.google.com/assets/colab-badge.svg\")
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "k7xBVPzoXxOg"
+ },
+ "source": [
+ "# Unit 3: Deep Q-Learning with Atari Games ๐พ using RL Baselines3 Zoo\n",
+ "\n",
+ "In this notebook, **you'll train a Deep Q-Learning agent** playing Space Invaders using [RL Baselines3 Zoo](https://github.com/DLR-RM/rl-baselines3-zoo), a training framework based on [Stable-Baselines3](https://stable-baselines3.readthedocs.io/en/master/) that provides scripts for training, evaluating agents, tuning hyperparameters, plotting results and recording videos.\n",
+ "\n",
+ "We're using the [RL-Baselines-3 Zoo integration, a vanilla version of Deep Q-Learning](https://stable-baselines3.readthedocs.io/en/master/modules/dqn.html) with no extensions such as Double-DQN, Dueling-DQN, and Prioritized Experience Replay.\n",
+ "\n",
+ "โ If you have questions, please post them on #study-group-unit3 discord channel ๐ https://discord.gg/aYka4Yhff9\n",
+ "\n",
+ "๐ฎ Environments: \n",
+ "- SpacesInvadersNoFrameskip-v4\n",
+ "\n",
+ "๐ RL-Library: [Stable-Baselines3](https://stable-baselines3.readthedocs.io/en/master/)\n",
+ " \n",
+ "โฌ๏ธ Here is an example of what **you will achieve** โฌ๏ธ"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "J9S713biXntc"
+ },
+ "outputs": [],
+ "source": [
+ "%%html\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "wciHGjrFYz9m"
+ },
+ "source": [
+ "## Objectives of this notebook ๐\n",
+ "At the end of the notebook, you will:\n",
+ "- Be able to understand deeper **how RL Baselines3 Zoo works**.\n",
+ "- Be able to **push your trained agent and the code to the Hub** with a nice video replay and an evaluation score ๐ฅ.\n",
+ "\n",
+ "\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "7Hac0z2QZdOm"
+ },
+ "source": [
+ "## This notebook is from Deep Reinforcement Learning Class\n",
+ "\n",
+ "\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "nw6fJHIAZd-J"
+ },
+ "source": [
+ "In this free course, you will:\n",
+ "\n",
+ "- ๐ Study Deep Reinforcement Learning in **theory and practice**.\n",
+ "- ๐งโ๐ป Learn to **use famous Deep RL libraries** such as Stable Baselines3, RL Baselines3 Zoo, and RLlib.\n",
+ "- ๐ค Train **agents in unique environments** \n",
+ "\n",
+ "And more check ๐ the syllabus ๐ https://github.com/huggingface/deep-rl-class\n",
+ "\n",
+ "The best way to keep in touch is to join our discord server to exchange with the community and with us ๐๐ป https://discord.gg/aYka4Yhff9"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "0vgANIBBZg1p"
+ },
+ "source": [
+ "## Prerequisites ๐๏ธ\n",
+ "Before diving into the notebook, you need to:\n",
+ "\n",
+ "๐ฒ ๐ [Read the Unit 3 Readme](https://github.com/huggingface/deep-rl-class/blob/main/unit3/README.md) that contains all the information.\n",
+ "\n",
+ "๐ฒ ๐ [Read **Deep Q-Learning**](https://huggingface.co/blog/deep-rl-dqn) \n",
+ "\n",
+ "๐ฒ ๐ข Sign up to [our Discord Server](https://discord.gg/aYka4Yhff9) if it's not already done and **introduce yourself to #introduce-yourself channel ๐ฅณ**\n",
+ "\n",
+ "๐ฒ ๐ Are you new to Discord? Check our **discord 101 to get the best practices** ๐ https://github.com/huggingface/deep-rl-class/blob/main/DISCORD.Md\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "QR0jZtYreSI5"
+ },
+ "source": [
+ "# Let's train a Deep Q-Learning agent playing Atari' Space Invaders ๐พ and upload it to the Hub."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "ESJV9guAee3P"
+ },
+ "source": [
+ "### Step 0: Set the GPU ๐ช\n",
+ "- To **faster the agent's training, we'll use a GPU** to do that go to `Runtime > Change Runtime type`\n",
+ "\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "Zg8eWiQ1efcU"
+ },
+ "source": [
+ "- `Hardware Accelerator > GPU`"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "eoey4CFpejM5"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "P6XBkQ5qelOV"
+ },
+ "source": [
+ "### Step 0+: Setup a Virtual Display ๐ป\n",
+ "\n",
+ "During the notebook, we'll need to generate a replay video. To do so, with colab, **we need to have a virtual screen to be able to render the environment** (and thus record the frames). \n",
+ "\n",
+ "Hence the following cell will install virtual screen libraries and create and run a virtual screen ๐ฅ"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "jV6wjQ7Be7p5"
+ },
+ "outputs": [],
+ "source": [
+ "%%capture\n",
+ "!pip install pyglet==1.5.1 \n",
+ "!apt install python-opengl\n",
+ "!apt install ffmpeg\n",
+ "!apt install xvfb\n",
+ "!pip3 install pyvirtualdisplay\n",
+ "\n",
+ "# Additional dependencies for RL Baselines3 Zoo\n",
+ "!apt-get install swig cmake freeglut3-dev \n",
+ "\n",
+ "# Virtual display\n",
+ "from pyvirtualdisplay import Display\n",
+ "\n",
+ "virtual_display = Display(visible=0, size=(1400, 900))\n",
+ "virtual_display.start()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "mYIMvl5X9NAu"
+ },
+ "source": [
+ "### Step 1: Clone RL-Baselines3 Zoo Repo ๐"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "eu5ZDPZ09VNQ"
+ },
+ "outputs": [],
+ "source": [
+ "!git clone https://github.com/DLR-RM/rl-baselines3-zoo"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "HCIoSbvbfAQh"
+ },
+ "source": [
+ "### Step 2: Install dependencies ๐ฝ\n",
+ "The first step is to install the dependencies RL-Baselines3 Zoo needs (this can take 5min โฒ)\n",
+ "\n",
+ "But we'll also install:\n",
+ "- `huggingface_sb3`: Additional code for Stable-baselines3 to load and upload models from the Hugging Face ๐ค Hub."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "s2QsFAk29h-D"
+ },
+ "outputs": [],
+ "source": [
+ "%cd /content/rl-baselines3-zoo/"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "3QaOS7Xj9j1s"
+ },
+ "outputs": [],
+ "source": [
+ "!pip install -r requirements.txt"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "RLRGKFR39l9s"
+ },
+ "outputs": [],
+ "source": [
+ "%%capture\n",
+ "!pip install huggingface_sb3"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "5iPgzluo9z-u"
+ },
+ "source": [
+ "### Step 3: Train our Deep Q-Learning Agent to Play Space Invaders ๐พ\n",
+ "\n",
+ "To train an agent with RL-Baselines3-Zoo, we just need to do two things:\n",
+ "1. We define the hyperparameters in `rl-baselines3-zoo/hyperparams/dqn.yml`\n",
+ "\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "oInM0jLkDPfL"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "_VjblFSVDQOj"
+ },
+ "source": [
+ "Here we see that:\n",
+ "- We use the Atari Wrapper that preprocess the input (Frame reduction ,grayscale, stack 4 frames)\n",
+ "- We use `CnnPolicy`, since we use Convolutional layers to process the frames\n",
+ "- We train it for 10 million `n_timesteps` \n",
+ "- Memory (Experience Replay) size is 100000\n",
+ "\n",
+ "๐ก My advice is to **reduce the training timesteps to 1M,** which will take about 90 minutes on a P100. `!nvidia-smi` will tell you what GPU you're using. At 10 million steps, this will take about 9 hours, which could likely result in Colab timing out. I recommend running this on your local computer (or somewhere else). Just click on: `File>Download`. "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "5qTkbWrkECOJ"
+ },
+ "source": [
+ "You can check the documentation to understand what each hyperparameters does: https://stable-baselines3.readthedocs.io/en/master/modules/dqn.html?highlight=deep%20q%20learning#parameters"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "Hn8bRTHvERRL"
+ },
+ "source": [
+ "2. We run `train.py` and save the models on `logs` folder ๐"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "Xr1TVW4xfbz3"
+ },
+ "outputs": [],
+ "source": [
+ "!python train.py --algo ________ --env SpaceInvadersNoFrameskip-v4 -f _________"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "SeChoX-3SZfP"
+ },
+ "source": [
+ "#### Solution"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "PuocgdokSab9"
+ },
+ "outputs": [],
+ "source": [
+ "!python train.py --algo dqn --env SpaceInvadersNoFrameskip-v4 -f logs/"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "_dLomIiMKQaf"
+ },
+ "source": [
+ "### Step 4: Let's evaluate our agent ๐\n",
+ "- RL-Baselines3-Zoo provides `enjoy.py` to evaluate our agent.\n",
+ "- Let's evaluate it for 5000 timesteps ๐ฅ"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "co5um_KeKbBJ"
+ },
+ "outputs": [],
+ "source": [
+ "!python enjoy.py --algo dqn --env SpaceInvadersNoFrameskip-v4 --no-render --n-timesteps _________ --folder logs/"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "Q24K1tyWSj7t"
+ },
+ "source": [
+ "#### Solution"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "P_uSmwGRSk0z"
+ },
+ "outputs": [],
+ "source": [
+ "!python enjoy.py --algo dqn --env SpaceInvadersNoFrameskip-v4 --no-render --n-timesteps 5000 --folder logs/"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "liBeTltiHJtr"
+ },
+ "source": [
+ "### Step 5: Publish our trained model on the Hub ๐\n",
+ "Now that we saw we got good results after the training, we can publish our trained model on the hub ๐ค with one line of code.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "O6L41QiMHYM2"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "ezbHS1q3HYVV"
+ },
+ "source": [
+ "By using `utils.push_to_hub.py` **you evaluate, record a replay, generate a model card of your agent and push it to the hub**.\n",
+ "\n",
+ "This way:\n",
+ "- You can **showcase our work** ๐ฅ\n",
+ "- You can **visualize your agent playing** ๐\n",
+ "- You can **share with the community an agent that others can use** ๐พ\n",
+ "- You can **access a leaderboard ๐ to see how well your agent is performing compared to your classmates** ๐ https://huggingface.co/spaces/chrisjay/Deep-Reinforcement-Learning-Leaderboard"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "XMSeZRBiHk6X"
+ },
+ "source": [
+ "To be able to share your model with the community there are three more steps to follow:\n",
+ "\n",
+ "1๏ธโฃ (If it's not already done) create an account to HF โก https://huggingface.co/join\n",
+ "\n",
+ "2๏ธโฃ Sign in and then, you need to store your authentication token from the Hugging Face website.\n",
+ "- Create a new token (https://huggingface.co/settings/tokens) **with write role**"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "9ToyuaYwHmxG"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "9O6FI0F8HnzE"
+ },
+ "source": [
+ "- Copy the token \n",
+ "- Run the cell below and past the token"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "Ppu9yePwHrZX"
+ },
+ "outputs": [],
+ "source": [
+ "from huggingface_hub import notebook_login # To log to our Hugging Face account to be able to upload models to the Hub.\n",
+ "notebook_login()\n",
+ "!git config --global credential.helper store"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "2RVEdunPHs8B"
+ },
+ "source": [
+ "If you don't want to use a Google Colab or a Jupyter Notebook, you need to use this command instead: `huggingface-cli login`"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "dSLwdmvhHvjw"
+ },
+ "source": [
+ "3๏ธโฃ We're now ready to push our trained agent to the ๐ค Hub ๐ฅ"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "PW436XnhHw1H"
+ },
+ "source": [
+ "Let's run push_to_hub.py file to upload our trained agent to the Hub.\n",
+ "\n",
+ "`--repo-name `: The name of the repo\n",
+ "\n",
+ "`-orga`: Your Hugging Face username"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "9YO_qZTWeRHl"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "Ygk2sEktTDEw"
+ },
+ "outputs": [],
+ "source": [
+ "!python -m rl_zoo3.push_to_hub --algo dqn --env SpaceInvadersNoFrameskip-v4 --repo-name _____________________ -orga _____________________ -f logs/"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "otgpa0rhS9wR"
+ },
+ "source": [
+ "#### Solution"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "_HQNlAXuEhci"
+ },
+ "outputs": [],
+ "source": [
+ "!python -m rl_zoo3.push_to_hub --algo dqn --env SpaceInvadersNoFrameskip-v4 --repo-name dqn-SpaceInvadersNoFrameskip-v4 -orga ThomasSimonini -f logs/"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "0D4F5zsTTJ-L"
+ },
+ "source": [
+ "###."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "ff89kd2HL1_s"
+ },
+ "source": [
+ "Congrats ๐ฅณ you've just trained and uploaded your first Deep Q-Learning agent using RL-Baselines-3 Zoo. The script above should have displayed a link to a model repository such as https://huggingface.co/ThomasSimonini/dqn-SpaceInvadersNoFrameskip-v4. When you go to this link, you can:\n",
+ "* see a video preview of your agent at the right. \n",
+ "* click \"Files and versions\" to see all the files in the repository.\n",
+ "* click \"Use in stable-baselines3\" to get a code snippet that shows how to load the model.\n",
+ "* a model card (`README.md` file) which gives a description of the model and the hyperparameters you used.\n",
+ "\n",
+ "Under the hood, the Hub uses git-based repositories (don't worry if you don't know what git is), which means you can update the model with new versions as you experiment and improve your agent.\n",
+ "\n",
+ "Compare the results of your LunarLander-v2 with your classmates using the leaderboard ๐ ๐ https://huggingface.co/spaces/chrisjay/Deep-Reinforcement-Learning-Leaderboard"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "fyRKcCYY-dIo"
+ },
+ "source": [
+ "### Step 6: Load a powerful trained model ๐ฅ\n",
+ "- The Stable-Baselines3 team uploaded **more than 150 trained Deep Reinforcement Learning agents on the Hub**.\n",
+ "\n",
+ "You can find them here: ๐ https://huggingface.co/sb3\n",
+ "\n",
+ "Some examples:\n",
+ "- Asteroids: https://huggingface.co/sb3/dqn-AsteroidsNoFrameskip-v4\n",
+ "- Beam Rider: https://huggingface.co/sb3/dqn-BeamRiderNoFrameskip-v4\n",
+ "- Breakout: https://huggingface.co/sb3/dqn-BreakoutNoFrameskip-v4\n",
+ "- Road Runner: https://huggingface.co/sb3/dqn-RoadRunnerNoFrameskip-v4\n",
+ "\n",
+ "Let's load an agent playing Beam Rider: https://huggingface.co/sb3/dqn-BeamRiderNoFrameskip-v4"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "B-9QVFIROI5Y"
+ },
+ "outputs": [],
+ "source": [
+ "%%html\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "7ZQNY_r6NJtC"
+ },
+ "source": [
+ "1. We download the model using `utils.load_from_hub`, and place it in a new folder that we can call `rl_trained`"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "OdBNZHy0NGTR"
+ },
+ "outputs": [],
+ "source": [
+ "# Download model and save it into the logs/ folder\n",
+ "!python -m rl_zoo3.load_from_hub --algo dqn --env BeamRiderNoFrameskip-v4 -orga sb3 -f rl_trained/"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "LFt6hmWsNdBo"
+ },
+ "source": [
+ "2. Let's evaluate if for 5000 timesteps"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "aOxs0rNuN0uS"
+ },
+ "outputs": [],
+ "source": [
+ "!python enjoy.py --algo dqn --env BeamRiderNoFrameskip-v4 -n 5000 -f rl_trained/"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "kxMDuDfPON57"
+ },
+ "source": [
+ "Why not trying to train your own **Deep Q-Learning Agent playing BeamRiderNoFrameskip-v4? ๐.**\n",
+ "\n",
+ "If you want to try, check https://huggingface.co/sb3/dqn-BeamRiderNoFrameskip-v4#hyperparameters **in the model card, you have the hyperparameters of the trained agent.**"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "xL_ZtUgpOuY6"
+ },
+ "source": [
+ "We'll see in the next Unit, how we can **use Optuna for optimizing the Hyperparameters ๐ฅ.**\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "-pqaco8W-huW"
+ },
+ "source": [
+ "## Some additional challenges ๐\n",
+ "The best way to learn **is to try things by your own**!\n",
+ "\n",
+ "In the [Leaderboard](https://huggingface.co/spaces/chrisjay/Deep-Reinforcement-Learning-Leaderboard) you will find your agents. Can you get to the top?\n",
+ "\n",
+ "Here's a list of environments you can try to train your agent with:\n",
+ "- BeamRiderNoFrameskip-v4\n",
+ "- BreakoutNoFrameskip-v4 \n",
+ "- EnduroNoFrameskip-v4\n",
+ "- PongNoFrameskip-v4\n",
+ "\n",
+ "\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "JNeyXYt-PtCQ"
+ },
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "paS-XKo4-kmu"
+ },
+ "source": [
+ "________________________________________________________________________\n",
+ "Congrats on finishing this chapter!\n",
+ "\n",
+ "If youโre still feel confused with all these elements...it's totally normal! **This was the same for me and for all people who studied RL.**\n",
+ "\n",
+ "Take time to really **grasp the material before continuing and try the additional challenges**. Itโs important to master these elements and having a solid foundations.\n",
+ "\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "5WRx7tO7-mvC"
+ },
+ "source": [
+ "\n",
+ "\n",
+ "### This is a course built with you ๐ท๐ฟโโ๏ธ\n",
+ "\n",
+ "Finally, we want to improve and update the course iteratively with your feedback. If you have some, please fill this form ๐ https://forms.gle/3HgA7bEHwAmmLfwh9\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "fS3Xerx0fIMV"
+ },
+ "source": [
+ "### Keep Learning, Stay Awesome ๐ค"
+ ]
+ }
+ ],
+ "metadata": {
+ "colab": {
+ "collapsed_sections": [],
+ "include_colab_link": true,
+ "name": "Copie de Unit 3: Deep Q-Learning with Space Invaders.ipynb",
+ "private_outputs": true,
+ "provenance": []
+ },
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.10.6"
+ },
+ "varInspector": {
+ "cols": {
+ "lenName": 16,
+ "lenType": 16,
+ "lenVar": 40
+ },
+ "kernels_config": {
+ "python": {
+ "delete_cmd_postfix": "",
+ "delete_cmd_prefix": "del ",
+ "library": "var_list.py",
+ "varRefreshCmd": "print(var_dic_list())"
+ },
+ "r": {
+ "delete_cmd_postfix": ") ",
+ "delete_cmd_prefix": "rm(",
+ "library": "var_list.r",
+ "varRefreshCmd": "cat(var_dic_list()) "
+ }
+ },
+ "types_to_exclude": [
+ "module",
+ "function",
+ "builtin_function_or_method",
+ "instance",
+ "_Feature"
+ ],
+ "window_display": false
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 1
+}