diff --git a/unit4/unit4.ipynb b/unit4/unit4.ipynb
index 97963b1..79e53aa 100644
--- a/unit4/unit4.ipynb
+++ b/unit4/unit4.ipynb
@@ -7,7 +7,17 @@
"colab_type": "text"
},
"source": [
- "![\"Open](\"https://colab.research.google.com/assets/colab-badge.svg\")
"
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "2D3NL_e4crQv"
+ },
+ "source": [
+ "# Unit 4: Let's learn about Unity ML-Agents with Hugging Face 🤗\n",
+ "\n"
]
},
{
@@ -21,12 +31,71 @@
},
{
"cell_type": "markdown",
- "metadata": {
- "id": "2D3NL_e4crQv"
- },
"source": [
- "#Pyramids Unity3D "
- ]
+ "In this notebook, you'll learn about ML-Agents and use one of the pre-made environments: Pyramids. \n",
+ "\n",
+ "In this environment, we’ll train an agent that needs to press a button to spawn a pyramid, then navigate to the pyramid, knock it over, and move to the gold brick at the top.\n",
+ "\n",
+ "❓ If you have questions, please post them on #study-group-unit1 discord channel 👉 https://discord.gg/aYka4Yhff9\n",
+ "\n",
+ "🎮 Environments: \n",
+ "- [Pyramids](https://github.com/Unity-Technologies/ml-agents/blob/main/docs/Learning-Environment-Examples.md#pyramids)\n",
+ "\n",
+ "\n",
+ "⬇️ Here is an example of what **you will achieve at the end of this notebook.** ⬇️"
+ ],
+ "metadata": {
+ "id": "FMYrDriDujzX"
+ }
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ ""
+ ],
+ "metadata": {
+ "id": "cBmFlh8suma-"
+ }
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "## This notebook is from Deep Reinforcement Learning Class\n",
+ "\n",
+ "This notebook was written by [Abid Ali Awan alias kingabzpro](https://github.com/kingabzpro) 🤗\n",
+ "\n",
+ ""
+ ],
+ "metadata": {
+ "id": "mUlVrqnBv2o1"
+ }
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "In this free course, you will:\n",
+ "\n",
+ "- 📖 Study Deep Reinforcement Learning in **theory and practice**.\n",
+ "- 🧑💻 Learn to **use famous Deep RL libraries** such as Stable Baselines3, RL Baselines3 Zoo, and RLlib.\n",
+ "- 🤖 Train **agents in unique environments** \n",
+ "\n",
+ "And more check 📚 the syllabus 👉 https://github.com/huggingface/deep-rl-class\n",
+ "\n",
+ "The best way to keep in touch is to join our discord server to exchange with the community and with us 👉🏻 https://discord.gg/aYka4Yhff9"
+ ],
+ "metadata": {
+ "id": "pAMjaQpHwB_s"
+ }
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "## Prerequisites 🏗️\n",
+ "To dive in this notebook, **you need to read in parallel the tutorial** 👉 https://link.medium.com/KOpvPdyz4qb\n"
+ ],
+ "metadata": {
+ "id": "6r7Hl0uywFSO"
+ }
},
{
"cell_type": "markdown",
@@ -34,7 +103,8 @@
"id": "an3ByrXYQ4iK"
},
"source": [
- "##Getting Started"
+ "### Step 1: Clone the repository and install the dependencies 🔽\n",
+ "- We need to clone the repository, that **contains the experimental version of the library that allows you to push your trained agent to the Hub.**"
]
},
{
@@ -46,7 +116,7 @@
"outputs": [],
"source": [
"%%capture\n",
- "# Clone the repository\n",
+ "# Clone the repository (can take 1min)\n",
"!git clone https://github.com/huggingface/ml-agents/"
]
},
@@ -71,7 +141,9 @@
"id": "HRY5ufKUKfhI"
},
"source": [
- "## Download and move the zip file in `./trained-envs-executables/linux/`"
+ "### Step 2: Download and move the environment zip file in `./trained-envs-executables/linux/`\n",
+ "- Our environment executable is in a zip file.\n",
+ "- We need to download it and place it to `./trained-envs-executables/linux/`"
]
},
{
@@ -130,272 +202,9 @@
"cell_type": "code",
"execution_count": null,
"metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "8FPx0an9IAwO",
- "outputId": "54c8310a-c988-46d0-f90c-1a513c437c42"
+ "id": "8FPx0an9IAwO"
},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "Archive: ./trained-envs-executables/linux/Pyramids.zip\n",
- " creating: ./trained-envs-executables/linux/Pyramids/\n",
- " inflating: ./trained-envs-executables/linux/Pyramids/UnityPlayer.so \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids \n",
- " creating: ./trained-envs-executables/linux/Pyramids/Pyramids_BurstDebugInformation_DoNotShip/\n",
- " creating: ./trained-envs-executables/linux/Pyramids/Pyramids_BurstDebugInformation_DoNotShip/Data/\n",
- " creating: ./trained-envs-executables/linux/Pyramids/Pyramids_BurstDebugInformation_DoNotShip/Data/Plugins/\n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_BurstDebugInformation_DoNotShip/Data/Plugins/lib_burst_generated.txt \n",
- " creating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/\n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/resources.assets \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/app.info \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/boot.config \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/level0 \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/globalgamemanagers \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/ScriptingAssemblies.json \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/sharedassets0.assets \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/sharedassets0.assets.resS \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/RuntimeInitializeOnLoads.json \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/globalgamemanagers.assets \n",
- " creating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/\n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.StreamingModule.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.VideoModule.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.SharedInternalsModule.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.PerformanceReportingModule.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/Unity.InputSystem.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.UIElementsModule.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.UnityCurlModule.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.InputModule.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.DSPGraphModule.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.ScreenCaptureModule.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.CrashReportingModule.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/Unity.Burst.Cecil.Mdb.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.UI.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/Mono.Security.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.UnityWebRequestAssetBundleModule.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/Assembly-CSharp.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.VirtualTexturingModule.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.UnityWebRequestWWWModule.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.DSPGraphModule.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/Unity.ML-Agents.Extensions.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.SubstanceModule.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/Google.Protobuf.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/System.Transactions.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.JSONSerializeModule.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/Unity.Barracuda.ONNX.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/Unity.Barracuda.BurstBLAS.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.TerrainPhysicsModule.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.SubsystemsModule.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/System.IO.Abstractions.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/Unity.ML-Agents.Extensions.Input.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.Physics2DModule.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.InputLegacyModule.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.IMGUIModule.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.ClothModule.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.ClusterRendererModule.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/System.Runtime.Serialization.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.SpriteShapeModule.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/System.Numerics.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/Unity.ML-Agents.CommunicatorObjects.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.UI.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.TLSModule.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.VRModule.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.VirtualTexturingModule.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.VFXModule.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.GIModule.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/Unity.Burst.Unsafe.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.UNETModule.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.UIElementsModule.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/System.IO.Abstractions.TestingHelpers.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.AndroidJNIModule.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.UnityTestProtocolModule.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/Unity.Barracuda.ONNX.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.SubsystemsModule.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.UnityWebRequestAudioModule.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/System.Data.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.SpriteMaskModule.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.AIModule.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.ScreenCaptureModule.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.ProfilerModule.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.TilemapModule.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/System.ComponentModel.Composition.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.UmbraModule.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.PerformanceReportingModule.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/Unity.Barracuda.BurstBLAS.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.SpriteShapeModule.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.HotReloadModule.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.GameCenterModule.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/System.EnterpriseServices.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.VFXModule.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.PhysicsModule.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/System.Drawing.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/Unity.InputSystem.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/Google.Protobuf.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/Unity.Barracuda.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.TextRenderingModule.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/Unity.Mathematics.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.DirectorModule.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/Unity.ML-Agents.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.AccessibilityModule.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/System.Xml.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.UmbraModule.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.LocalizationModule.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.PhysicsModule.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.TextRenderingModule.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.ClusterRendererModule.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.VehiclesModule.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/System.IO.Compression.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/Unity.ML-Agents.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.UnityWebRequestTextureModule.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.TerrainModule.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.UnityTestProtocolModule.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.UIElementsNativeModule.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.XRModule.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.HotReloadModule.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.ImageConversionModule.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.ClothModule.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.UIElementsNativeModule.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.TextCoreFontEngineModule.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.UnityWebRequestAudioModule.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.Physics2DModule.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.TextCoreTextEngineModule.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.SpriteMaskModule.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.CrashReportingModule.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/System.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.GIModule.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.DirectorModule.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/Unity.Mathematics.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/System.Core.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.UnityAnalyticsModule.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/Newtonsoft.Json.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.UnityWebRequestAssetBundleModule.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.SharedInternalsModule.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.AudioModule.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/netstandard.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.UnityWebRequestWWWModule.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.StreamingModule.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.GridModule.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.UnityAnalyticsModule.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/System.Runtime.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/System.Interactive.Async.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.VideoModule.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/System.Security.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/System.IO.Compression.FileSystem.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/Unity.ML-Agents.CommunicatorObjects.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.TilemapModule.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.UnityWebRequestTextureModule.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.AccessibilityModule.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.AndroidJNIModule.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.TextCoreFontEngineModule.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.AIModule.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/System.Configuration.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/Unity.Burst.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.UnityConnectModule.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.UIModule.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.UnityCurlModule.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.AssetBundleModule.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.UnityConnectModule.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.VRModule.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/Unity.Burst.Cecil.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.SubstanceModule.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.ClusterInputModule.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.UIModule.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.UnityWebRequestModule.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.InputModule.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/Unity.ML-Agents.Extensions.Input.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.AnimationModule.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.JSONSerializeModule.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.UNETModule.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.ProfilerModule.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/System.Data.DataSetExtensions.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.VehiclesModule.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.XRModule.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.GameCenterModule.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.ImageConversionModule.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.CoreModule.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/System.ServiceModel.Internals.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.TerrainModule.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/Assembly-CSharp.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.TerrainPhysicsModule.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/System.Net.Http.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.AudioModule.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/Newtonsoft.Json.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.TextCoreTextEngineModule.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.ParticleSystemModule.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.IMGUIModule.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.WindModule.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.AssetBundleModule.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/Unity.Barracuda.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.InputLegacyModule.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.ParticleSystemModule.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.UnityWebRequestModule.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.WindModule.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.LocalizationModule.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.ClusterInputModule.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.RuntimeInitializeOnLoadManagerInitializerModule.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.GridModule.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/Unity.Burst.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/Unity.ML-Agents.Extensions.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.AnimationModule.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/System.Xml.Linq.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.TLSModule.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.CoreModule.pdb \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/UnityEngine.RuntimeInitializeOnLoadManagerInitializerModule.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/Unity.Burst.Cecil.Rocks.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/Unity.Burst.Cecil.Pdb.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/Grpc.Core.dll \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Managed/mscorlib.dll \n",
- " creating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/MonoBleedingEdge/\n",
- " creating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/MonoBleedingEdge/etc/\n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/MonoBleedingEdge/etc/config \n",
- " creating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/MonoBleedingEdge/etc/mono/\n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/MonoBleedingEdge/etc/mono/config \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/MonoBleedingEdge/etc/mono/browscap.ini \n",
- " creating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/MonoBleedingEdge/etc/mono/4.0/\n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/MonoBleedingEdge/etc/mono/4.0/DefaultWsdlHelpGenerator.aspx \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/MonoBleedingEdge/etc/mono/4.0/settings.map \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/MonoBleedingEdge/etc/mono/4.0/web.config \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/MonoBleedingEdge/etc/mono/4.0/machine.config \n",
- " creating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/MonoBleedingEdge/etc/mono/4.0/Browsers/\n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/MonoBleedingEdge/etc/mono/4.0/Browsers/Compat.browser \n",
- " creating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/MonoBleedingEdge/etc/mono/4.5/\n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/MonoBleedingEdge/etc/mono/4.5/settings.map \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/MonoBleedingEdge/etc/mono/4.5/DefaultWsdlHelpGenerator.aspx \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/MonoBleedingEdge/etc/mono/4.5/web.config \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/MonoBleedingEdge/etc/mono/4.5/machine.config \n",
- " creating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/MonoBleedingEdge/etc/mono/4.5/Browsers/\n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/MonoBleedingEdge/etc/mono/4.5/Browsers/Compat.browser \n",
- " creating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/MonoBleedingEdge/etc/mono/2.0/\n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/MonoBleedingEdge/etc/mono/2.0/DefaultWsdlHelpGenerator.aspx \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/MonoBleedingEdge/etc/mono/2.0/settings.map \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/MonoBleedingEdge/etc/mono/2.0/web.config \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/MonoBleedingEdge/etc/mono/2.0/machine.config \n",
- " creating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/MonoBleedingEdge/etc/mono/2.0/Browsers/\n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/MonoBleedingEdge/etc/mono/2.0/Browsers/Compat.browser \n",
- " creating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/MonoBleedingEdge/etc/mono/mconfig/\n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/MonoBleedingEdge/etc/mono/mconfig/config.xml \n",
- " creating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/MonoBleedingEdge/x86_64/\n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/MonoBleedingEdge/x86_64/libmonobdwgc-2.0.so \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/MonoBleedingEdge/x86_64/libmono-native.so \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/MonoBleedingEdge/x86_64/libMonoPosixHelper.so \n",
- " creating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Resources/\n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Resources/unity default resources \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Resources/UnityPlayer.png \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Resources/unity_builtin_extra \n",
- " creating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Plugins/\n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Plugins/libgrpc_csharp_ext.x64.so \n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/Plugins/lib_burst_generated.so \n",
- " creating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/ML-Agents/\n",
- " creating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/ML-Agents/Timers/\n",
- " inflating: ./trained-envs-executables/linux/Pyramids/Pyramids_Data/ML-Agents/Timers/Pyramids_timers.json \n"
- ]
- }
- ],
+ "outputs": [],
"source": [
"%%capture\n",
"!unzip -d ./trained-envs-executables/linux/ ./trained-envs-executables/linux/Pyramids.zip"
@@ -421,229 +230,96 @@
"!chmod -R 755 ./trained-envs-executables/linux/Pyramids/Pyramids"
]
},
+ {
+ "cell_type": "markdown",
+ "source": [
+ "### Step 3: Modify the PyramidsRND config file\n",
+ "- In ML-Agents, you define the **training hyperparameters into config.yaml files.**\n",
+ "For this first training, we’ll modify one thing:\n",
+ "- The total training steps hyperparameter is too high since we can hit the benchmark in only 1M training steps.\n",
+ "👉 To do that, we go to config/ppo/PyramidsRND.yaml,**and modify these to max_steps to 1000000.**"
+ ],
+ "metadata": {
+ "id": "NAuEq32Mwvtz"
+ }
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "- Click here to open the config.yaml: /content/ml-agents/config/ppo/PyramidsRND.yaml"
+ ],
+ "metadata": {
+ "id": "r9wv5NYGw-05"
+ }
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "- Modify the `max_steps` to 500,000"
+ ],
+ "metadata": {
+ "id": "ob_crJkhygQz"
+ }
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ ""
+ ],
+ "metadata": {
+ "id": "LY7GI7QpymSu"
+ }
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "As an experimentation, you should also try to modify some other hyperparameters, Unity provides a very [good documentation explaining each of them here](https://github.com/Unity-Technologies/ml-agents/blob/main/docs/Training-Configuration-File.md).\n",
+ "\n",
+ "We’re now ready to train our agent 🔥."
+ ],
+ "metadata": {
+ "id": "JJJdo_5AyoGo"
+ }
+ },
{
"cell_type": "markdown",
"metadata": {
"id": "f9fI555bO12v"
},
"source": [
- "##Train\n",
+ "### Step 4: Train our agent\n",
+ "\n",
+ "To train our agent, we just need to **launch mlagents-learn and select the executable containing the environment.**\n",
+ "\n",
+ "We define four parameters:\n",
+ "\n",
+ "1. `mlagents-learn `: the path where the hyperparameter config file is.\n",
+ "2. `--env`: where the environment executable is.\n",
+ "3. `--run_id`: the name you want to give to your training run id.\n",
+ "4. `--no-graphics`: to not launch the visualization during the training.\n",
"\n",
"Train the model and use the `--resume` flag to continue training in case of interruption. \n",
"\n",
- "\n",
- "\n",
"> It will fail first time when you use `--resume`, try running the block again to bypass the error. \n",
"\n"
]
},
+ {
+ "cell_type": "markdown",
+ "source": [
+ "The training will take 30 to 45min depending on your machine, go take a ☕️you deserve it 🤗."
+ ],
+ "metadata": {
+ "id": "lN32oWF8zPjs"
+ }
+ },
{
"cell_type": "code",
"execution_count": null,
"metadata": {
- "colab": {
- "background_save": true,
- "base_uri": "https://localhost:8080/"
- },
- "id": "bS-Yh1UdHfzy",
- "outputId": "ee7ddaae-aa82-45d7-84f1-a38e1a395c60"
+ "id": "bS-Yh1UdHfzy"
},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "\n",
- " ┐ ╖\n",
- " ╓╖╬│╡ ││╬╖╖\n",
- " ╓╖╬│││││┘ ╬│││││╬╖\n",
- " ╖╬│││││╬╜ ╙╬│││││╖╖ ╗╗╗\n",
- " ╬╬╬╬╖││╦╖ ╖╬││╗╣╣╣╬ ╟╣╣╬ ╟╣╣╣ ╜╜╜ ╟╣╣\n",
- " ╬╬╬╬╬╬╬╬╖│╬╖╖╓╬╪│╓╣╣╣╣╣╣╣╬ ╟╣╣╬ ╟╣╣╣ ╒╣╣╖╗╣╣╣╗ ╣╣╣ ╣╣╣╣╣╣ ╟╣╣╖ ╣╣╣\n",
- " ╬╬╬╬┐ ╙╬╬╬╬│╓╣╣╣╝╜ ╫╣╣╣╬ ╟╣╣╬ ╟╣╣╣ ╟╣╣╣╙ ╙╣╣╣ ╣╣╣ ╙╟╣╣╜╙ ╫╣╣ ╟╣╣\n",
- " ╬╬╬╬┐ ╙╬╬╣╣ ╫╣╣╣╬ ╟╣╣╬ ╟╣╣╣ ╟╣╣╬ ╣╣╣ ╣╣╣ ╟╣╣ ╣╣╣┌╣╣╜\n",
- " ╬╬╬╜ ╬╬╣╣ ╙╝╣╣╬ ╙╣╣╣╗╖╓╗╣╣╣╜ ╟╣╣╬ ╣╣╣ ╣╣╣ ╟╣╣╦╓ ╣╣╣╣╣\n",
- " ╙ ╓╦╖ ╬╬╣╣ ╓╗╗╖ ╙╝╣╣╣╣╝╜ ╘╝╝╜ ╝╝╝ ╝╝╝ ╙╣╣╣ ╟╣╣╣\n",
- " ╩╬╬╬╬╬╬╦╦╬╬╣╣╗╣╣╣╣╣╣╣╝ ╫╣╣╣╣\n",
- " ╙╬╬╬╬╬╬╬╣╣╣╣╣╣╝╜\n",
- " ╙╬╬╬╣╣╣╜\n",
- " ╙\n",
- " \n",
- " Version information:\n",
- " ml-agents: 0.29.0.dev0,\n",
- " ml-agents-envs: 0.29.0.dev0,\n",
- " Communicator API: 1.5.0,\n",
- " PyTorch: 1.8.1+cu102\n",
- "E0704 08:25:54.156778894 1219 fork_posix.cc:76] Other threads are currently calling into gRPC, skipping fork() handlers\n",
- "[INFO] Connected to Unity environment with package version 2.2.1-exp.1 and communication version 1.5.0\n",
- "[INFO] Connected new brain: Pyramids?team=0\n",
- "[INFO] Hyperparameters for behavior name Pyramids: \n",
- "\ttrainer_type:\tppo\n",
- "\thyperparameters:\t\n",
- "\t batch_size:\t128\n",
- "\t buffer_size:\t2048\n",
- "\t learning_rate:\t0.0003\n",
- "\t beta:\t0.01\n",
- "\t epsilon:\t0.2\n",
- "\t lambd:\t0.95\n",
- "\t num_epoch:\t3\n",
- "\t learning_rate_schedule:\tlinear\n",
- "\t beta_schedule:\tlinear\n",
- "\t epsilon_schedule:\tlinear\n",
- "\tnetwork_settings:\t\n",
- "\t normalize:\tFalse\n",
- "\t hidden_units:\t512\n",
- "\t num_layers:\t2\n",
- "\t vis_encode_type:\tsimple\n",
- "\t memory:\tNone\n",
- "\t goal_conditioning_type:\thyper\n",
- "\t deterministic:\tFalse\n",
- "\treward_signals:\t\n",
- "\t extrinsic:\t\n",
- "\t gamma:\t0.99\n",
- "\t strength:\t1.0\n",
- "\t network_settings:\t\n",
- "\t normalize:\tFalse\n",
- "\t hidden_units:\t128\n",
- "\t num_layers:\t2\n",
- "\t vis_encode_type:\tsimple\n",
- "\t memory:\tNone\n",
- "\t goal_conditioning_type:\thyper\n",
- "\t deterministic:\tFalse\n",
- "\t rnd:\t\n",
- "\t gamma:\t0.99\n",
- "\t strength:\t0.01\n",
- "\t network_settings:\t\n",
- "\t normalize:\tFalse\n",
- "\t hidden_units:\t64\n",
- "\t num_layers:\t3\n",
- "\t vis_encode_type:\tsimple\n",
- "\t memory:\tNone\n",
- "\t goal_conditioning_type:\thyper\n",
- "\t deterministic:\tFalse\n",
- "\t learning_rate:\t0.0001\n",
- "\t encoding_size:\tNone\n",
- "\tinit_path:\tNone\n",
- "\tkeep_checkpoints:\t5\n",
- "\tcheckpoint_interval:\t500000\n",
- "\tmax_steps:\t3000000\n",
- "\ttime_horizon:\t128\n",
- "\tsummary_freq:\t30000\n",
- "\tthreaded:\tFalse\n",
- "\tself_play:\tNone\n",
- "\tbehavioral_cloning:\tNone\n",
- "[INFO] Resuming from results/Pyramids Training/Pyramids.\n",
- "[INFO] Resuming training from step 44290.\n",
- "[INFO] Pyramids. Step: 60000. Time Elapsed: 32.700 s. Mean Reward: -0.804. Std of Reward: 0.679. Training.\n",
- "[INFO] Pyramids. Step: 90000. Time Elapsed: 90.815 s. Mean Reward: -0.574. Std of Reward: 0.911. Training.\n",
- "[INFO] Pyramids. Step: 120000. Time Elapsed: 149.772 s. Mean Reward: -0.654. Std of Reward: 0.847. Training.\n",
- "[INFO] Pyramids. Step: 150000. Time Elapsed: 206.909 s. Mean Reward: -0.653. Std of Reward: 0.791. Training.\n",
- "[INFO] Pyramids. Step: 180000. Time Elapsed: 261.982 s. Mean Reward: -1.000. Std of Reward: 0.000. Training.\n",
- "[INFO] Pyramids. Step: 210000. Time Elapsed: 317.543 s. Mean Reward: -1.000. Std of Reward: 0.000. Training.\n",
- "[INFO] Pyramids. Step: 240000. Time Elapsed: 375.899 s. Mean Reward: -1.000. Std of Reward: 0.000. Training.\n",
- "[INFO] Pyramids. Step: 270000. Time Elapsed: 431.732 s. Mean Reward: -0.770. Std of Reward: 0.703. Training.\n",
- "[INFO] Pyramids. Step: 300000. Time Elapsed: 488.390 s. Mean Reward: -0.720. Std of Reward: 0.757. Training.\n",
- "[INFO] Pyramids. Step: 330000. Time Elapsed: 542.836 s. Mean Reward: -0.902. Std of Reward: 0.470. Training.\n",
- "[INFO] Pyramids. Step: 360000. Time Elapsed: 599.255 s. Mean Reward: -1.000. Std of Reward: 0.000. Training.\n",
- "[INFO] Pyramids. Step: 390000. Time Elapsed: 659.726 s. Mean Reward: -0.714. Std of Reward: 0.746. Training.\n",
- "[INFO] Pyramids. Step: 420000. Time Elapsed: 719.115 s. Mean Reward: -0.620. Std of Reward: 0.888. Training.\n",
- "[INFO] Pyramids. Step: 450000. Time Elapsed: 781.232 s. Mean Reward: -0.505. Std of Reward: 0.992. Training.\n",
- "[INFO] Pyramids. Step: 480000. Time Elapsed: 841.041 s. Mean Reward: -0.451. Std of Reward: 1.012. Training.\n",
- "[INFO] Exported results/Pyramids Training/Pyramids/Pyramids-499951.onnx\n",
- "[INFO] Pyramids. Step: 510000. Time Elapsed: 903.377 s. Mean Reward: -0.295. Std of Reward: 1.055. Training.\n",
- "[INFO] Pyramids. Step: 540000. Time Elapsed: 963.867 s. Mean Reward: -0.176. Std of Reward: 1.145. Training.\n",
- "[INFO] Pyramids. Step: 570000. Time Elapsed: 1022.686 s. Mean Reward: -0.191. Std of Reward: 1.125. Training.\n",
- "[INFO] Pyramids. Step: 600000. Time Elapsed: 1082.886 s. Mean Reward: 0.053. Std of Reward: 1.181. Training.\n",
- "[INFO] Pyramids. Step: 630000. Time Elapsed: 1142.497 s. Mean Reward: 0.302. Std of Reward: 1.251. Training.\n",
- "[INFO] Pyramids. Step: 660000. Time Elapsed: 1204.061 s. Mean Reward: 0.790. Std of Reward: 1.144. Training.\n",
- "[INFO] Pyramids. Step: 690000. Time Elapsed: 1266.511 s. Mean Reward: 1.144. Std of Reward: 0.919. Training.\n",
- "[INFO] Pyramids. Step: 720000. Time Elapsed: 1330.114 s. Mean Reward: 1.307. Std of Reward: 0.765. Training.\n",
- "[INFO] Pyramids. Step: 750000. Time Elapsed: 1395.904 s. Mean Reward: 1.144. Std of Reward: 0.943. Training.\n",
- "[INFO] Pyramids. Step: 780000. Time Elapsed: 1461.999 s. Mean Reward: 1.271. Std of Reward: 0.747. Training.\n",
- "[INFO] Pyramids. Step: 810000. Time Elapsed: 1522.645 s. Mean Reward: 1.020. Std of Reward: 1.033. Training.\n",
- "[INFO] Pyramids. Step: 840000. Time Elapsed: 1582.109 s. Mean Reward: 1.030. Std of Reward: 1.011. Training.\n",
- "[INFO] Pyramids. Step: 870000. Time Elapsed: 1646.121 s. Mean Reward: 1.433. Std of Reward: 0.673. Training.\n",
- "[INFO] Pyramids. Step: 900000. Time Elapsed: 1711.327 s. Mean Reward: 1.282. Std of Reward: 0.845. Training.\n",
- "[INFO] Pyramids. Step: 930000. Time Elapsed: 1776.580 s. Mean Reward: 1.133. Std of Reward: 0.962. Training.\n",
- "[INFO] Pyramids. Step: 960000. Time Elapsed: 1838.970 s. Mean Reward: 1.208. Std of Reward: 0.936. Training.\n",
- "[INFO] Pyramids. Step: 990000. Time Elapsed: 1902.757 s. Mean Reward: 1.373. Std of Reward: 0.732. Training.\n",
- "[INFO] Exported results/Pyramids Training/Pyramids/Pyramids-999977.onnx\n",
- "[INFO] Pyramids. Step: 1020000. Time Elapsed: 1966.600 s. Mean Reward: 1.445. Std of Reward: 0.653. Training.\n",
- "[INFO] Pyramids. Step: 1050000. Time Elapsed: 2033.775 s. Mean Reward: 1.515. Std of Reward: 0.607. Training.\n",
- "[INFO] Pyramids. Step: 1080000. Time Elapsed: 2100.586 s. Mean Reward: 1.487. Std of Reward: 0.630. Training.\n",
- "[INFO] Pyramids. Step: 1110000. Time Elapsed: 2165.209 s. Mean Reward: 1.586. Std of Reward: 0.362. Training.\n",
- "[INFO] Pyramids. Step: 1140000. Time Elapsed: 2232.162 s. Mean Reward: 1.599. Std of Reward: 0.447. Training.\n",
- "[INFO] Pyramids. Step: 1170000. Time Elapsed: 2295.385 s. Mean Reward: 1.224. Std of Reward: 0.940. Training.\n",
- "[INFO] Pyramids. Step: 1200000. Time Elapsed: 2361.663 s. Mean Reward: 1.590. Std of Reward: 0.465. Training.\n",
- "[INFO] Pyramids. Step: 1230000. Time Elapsed: 2426.524 s. Mean Reward: 1.436. Std of Reward: 0.729. Training.\n",
- "[INFO] Pyramids. Step: 1260000. Time Elapsed: 2494.057 s. Mean Reward: 1.578. Std of Reward: 0.531. Training.\n",
- "[INFO] Pyramids. Step: 1290000. Time Elapsed: 2562.419 s. Mean Reward: 1.549. Std of Reward: 0.588. Training.\n",
- "[INFO] Pyramids. Step: 1320000. Time Elapsed: 2628.902 s. Mean Reward: 1.676. Std of Reward: 0.214. Training.\n",
- "[INFO] Pyramids. Step: 1350000. Time Elapsed: 2694.671 s. Mean Reward: 1.588. Std of Reward: 0.534. Training.\n",
- "[INFO] Pyramids. Step: 1380000. Time Elapsed: 2759.915 s. Mean Reward: 1.572. Std of Reward: 0.522. Training.\n",
- "[INFO] Pyramids. Step: 1410000. Time Elapsed: 2825.777 s. Mean Reward: 1.483. Std of Reward: 0.727. Training.\n",
- "[INFO] Pyramids. Step: 1440000. Time Elapsed: 2890.511 s. Mean Reward: 1.586. Std of Reward: 0.529. Training.\n",
- "[INFO] Pyramids. Step: 1470000. Time Elapsed: 2958.030 s. Mean Reward: 1.571. Std of Reward: 0.598. Training.\n",
- "[INFO] Pyramids. Step: 1500000. Time Elapsed: 3026.486 s. Mean Reward: 1.638. Std of Reward: 0.415. Training.\n",
- "[INFO] Exported results/Pyramids Training/Pyramids/Pyramids-1499948.onnx\n",
- "[INFO] Pyramids. Step: 1530000. Time Elapsed: 3092.382 s. Mean Reward: 1.613. Std of Reward: 0.432. Training.\n",
- "[INFO] Pyramids. Step: 1560000. Time Elapsed: 3163.294 s. Mean Reward: 1.703. Std of Reward: 0.299. Training.\n",
- "[INFO] Pyramids. Step: 1590000. Time Elapsed: 3234.616 s. Mean Reward: 1.670. Std of Reward: 0.479. Training.\n",
- "[INFO] Pyramids. Step: 1620000. Time Elapsed: 3302.403 s. Mean Reward: 1.693. Std of Reward: 0.166. Training.\n",
- "[INFO] Pyramids. Step: 1650000. Time Elapsed: 3372.770 s. Mean Reward: 1.716. Std of Reward: 0.302. Training.\n",
- "[INFO] Pyramids. Step: 1680000. Time Elapsed: 3442.514 s. Mean Reward: 1.700. Std of Reward: 0.390. Training.\n",
- "[INFO] Pyramids. Step: 1710000. Time Elapsed: 3511.489 s. Mean Reward: 1.659. Std of Reward: 0.414. Training.\n",
- "[INFO] Pyramids. Step: 1740000. Time Elapsed: 3583.989 s. Mean Reward: 1.650. Std of Reward: 0.480. Training.\n",
- "[INFO] Pyramids. Step: 1770000. Time Elapsed: 3654.035 s. Mean Reward: 1.686. Std of Reward: 0.386. Training.\n",
- "[INFO] Pyramids. Step: 1800000. Time Elapsed: 3725.231 s. Mean Reward: 1.730. Std of Reward: 0.146. Training.\n",
- "[INFO] Pyramids. Step: 1830000. Time Elapsed: 3797.474 s. Mean Reward: 1.758. Std of Reward: 0.129. Training.\n",
- "[INFO] Pyramids. Step: 1860000. Time Elapsed: 3866.379 s. Mean Reward: 1.635. Std of Reward: 0.493. Training.\n",
- "[INFO] Pyramids. Step: 1890000. Time Elapsed: 3936.319 s. Mean Reward: 1.719. Std of Reward: 0.289. Training.\n",
- "[INFO] Pyramids. Step: 1920000. Time Elapsed: 4007.064 s. Mean Reward: 1.650. Std of Reward: 0.479. Training.\n",
- "[INFO] Pyramids. Step: 1950000. Time Elapsed: 4078.127 s. Mean Reward: 1.704. Std of Reward: 0.390. Training.\n",
- "[INFO] Pyramids. Step: 1980000. Time Elapsed: 4149.322 s. Mean Reward: 1.674. Std of Reward: 0.469. Training.\n",
- "[INFO] Exported results/Pyramids Training/Pyramids/Pyramids-1999905.onnx\n",
- "[INFO] Pyramids. Step: 2010000. Time Elapsed: 4219.756 s. Mean Reward: 1.751. Std of Reward: 0.146. Training.\n",
- "[INFO] Pyramids. Step: 2040000. Time Elapsed: 4288.297 s. Mean Reward: 1.661. Std of Reward: 0.477. Training.\n",
- "[INFO] Pyramids. Step: 2070000. Time Elapsed: 4357.614 s. Mean Reward: 1.678. Std of Reward: 0.472. Training.\n",
- "[INFO] Pyramids. Step: 2100000. Time Elapsed: 4425.649 s. Mean Reward: 1.684. Std of Reward: 0.393. Training.\n",
- "[INFO] Pyramids. Step: 2130000. Time Elapsed: 4495.805 s. Mean Reward: 1.678. Std of Reward: 0.405. Training.\n",
- "[INFO] Pyramids. Step: 2160000. Time Elapsed: 4570.061 s. Mean Reward: 1.743. Std of Reward: 0.281. Training.\n",
- "[INFO] Pyramids. Step: 2190000. Time Elapsed: 4639.448 s. Mean Reward: 1.653. Std of Reward: 0.527. Training.\n",
- "[INFO] Pyramids. Step: 2220000. Time Elapsed: 4708.968 s. Mean Reward: 1.717. Std of Reward: 0.294. Training.\n",
- "[INFO] Pyramids. Step: 2250000. Time Elapsed: 4776.413 s. Mean Reward: 1.664. Std of Reward: 0.539. Training.\n",
- "[INFO] Pyramids. Step: 2280000. Time Elapsed: 4845.380 s. Mean Reward: 1.674. Std of Reward: 0.465. Training.\n",
- "[INFO] Pyramids. Step: 2310000. Time Elapsed: 4917.064 s. Mean Reward: 1.760. Std of Reward: 0.149. Training.\n",
- "[INFO] Pyramids. Step: 2340000. Time Elapsed: 4985.848 s. Mean Reward: 1.693. Std of Reward: 0.456. Training.\n",
- "[INFO] Pyramids. Step: 2370000. Time Elapsed: 5056.830 s. Mean Reward: 1.724. Std of Reward: 0.295. Training.\n",
- "[INFO] Pyramids. Step: 2400000. Time Elapsed: 5128.556 s. Mean Reward: 1.745. Std of Reward: 0.361. Training.\n",
- "[INFO] Pyramids. Step: 2430000. Time Elapsed: 5199.095 s. Mean Reward: 1.732. Std of Reward: 0.286. Training.\n",
- "[INFO] Pyramids. Step: 2460000. Time Elapsed: 5269.909 s. Mean Reward: 1.714. Std of Reward: 0.382. Training.\n",
- "[INFO] Pyramids. Step: 2490000. Time Elapsed: 5342.099 s. Mean Reward: 1.729. Std of Reward: 0.293. Training.\n",
- "[INFO] Exported results/Pyramids Training/Pyramids/Pyramids-2499985.onnx\n",
- "[INFO] Pyramids. Step: 2520000. Time Elapsed: 5412.892 s. Mean Reward: 1.721. Std of Reward: 0.378. Training.\n",
- "[INFO] Pyramids. Step: 2550000. Time Elapsed: 5480.977 s. Mean Reward: 1.637. Std of Reward: 0.560. Training.\n",
- "[INFO] Pyramids. Step: 2580000. Time Elapsed: 5551.610 s. Mean Reward: 1.676. Std of Reward: 0.460. Training.\n",
- "[INFO] Pyramids. Step: 2610000. Time Elapsed: 5621.435 s. Mean Reward: 1.737. Std of Reward: 0.297. Training.\n",
- "[INFO] Pyramids. Step: 2640000. Time Elapsed: 5693.534 s. Mean Reward: 1.659. Std of Reward: 0.571. Training.\n",
- "[INFO] Pyramids. Step: 2670000. Time Elapsed: 5765.286 s. Mean Reward: 1.687. Std of Reward: 0.466. Training.\n",
- "[INFO] Pyramids. Step: 2700000. Time Elapsed: 5832.036 s. Mean Reward: 1.662. Std of Reward: 0.530. Training.\n",
- "[INFO] Pyramids. Step: 2730000. Time Elapsed: 5901.169 s. Mean Reward: 1.696. Std of Reward: 0.461. Training.\n",
- "[INFO] Pyramids. Step: 2760000. Time Elapsed: 5974.735 s. Mean Reward: 1.773. Std of Reward: 0.143. Training.\n",
- "[INFO] Pyramids. Step: 2790000. Time Elapsed: 6045.767 s. Mean Reward: 1.759. Std of Reward: 0.136. Training.\n",
- "[INFO] Pyramids. Step: 2820000. Time Elapsed: 6116.882 s. Mean Reward: 1.728. Std of Reward: 0.373. Training.\n",
- "[INFO] Pyramids. Step: 2850000. Time Elapsed: 6187.203 s. Mean Reward: 1.627. Std of Reward: 0.599. Training.\n",
- "[INFO] Pyramids. Step: 2880000. Time Elapsed: 6260.347 s. Mean Reward: 1.737. Std of Reward: 0.361. Training.\n",
- "[INFO] Pyramids. Step: 2910000. Time Elapsed: 6331.183 s. Mean Reward: 1.722. Std of Reward: 0.371. Training.\n",
- "[INFO] Pyramids. Step: 2940000. Time Elapsed: 6405.764 s. Mean Reward: 1.788. Std of Reward: 0.117. Training.\n",
- "[INFO] Pyramids. Step: 2970000. Time Elapsed: 6476.484 s. Mean Reward: 1.762. Std of Reward: 0.138. Training.\n",
- "[INFO] Pyramids. Step: 3000000. Time Elapsed: 6549.256 s. Mean Reward: 1.684. Std of Reward: 0.510. Training.\n",
- "[INFO] Exported results/Pyramids Training/Pyramids/Pyramids-2999989.onnx\n",
- "[INFO] Exported results/Pyramids Training/Pyramids/Pyramids-3000047.onnx\n",
- "[INFO] Copied results/Pyramids Training/Pyramids/Pyramids-3000047.onnx to results/Pyramids Training/Pyramids.onnx.\n"
- ]
- }
- ],
+ "outputs": [],
"source": [
"!mlagents-learn ./config/ppo/PyramidsRND.yaml --env=./trained-envs-executables/linux/Pyramids/Pyramids --run-id=\"Pyramids Training\" --no-graphics"
]
@@ -654,103 +330,224 @@
"id": "5Vue94AzPy1t"
},
"source": [
- "## Save the model to Hugging face"
+ "### Step 5: Push the agent to the 🤗 Hub\n",
+ "- Now that we trained our agent, we’re **ready to push it to the Hub and see him playing online 🔥.**"
]
},
{
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "rKt2vsYoK56o",
- "outputId": "ef39fe1d-d75f-4e13-c468-d933e2880388"
- },
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "\n",
- " _| _| _| _| _|_|_| _|_|_| _|_|_| _| _| _|_|_| _|_|_|_| _|_| _|_|_| _|_|_|_|\n",
- " _| _| _| _| _| _| _| _|_| _| _| _| _| _| _| _|\n",
- " _|_|_|_| _| _| _| _|_| _| _|_| _| _| _| _| _| _|_| _|_|_| _|_|_|_| _| _|_|_|\n",
- " _| _| _| _| _| _| _| _| _| _| _|_| _| _| _| _| _| _| _|\n",
- " _| _| _|_| _|_|_| _|_|_| _|_|_| _| _| _|_|_| _| _| _| _|_|_| _|_|_|_|\n",
- "\n",
- " To login, `huggingface_hub` now requires a token generated from https://huggingface.co/settings/tokens .\n",
- " \n",
- "Token: \n",
- "Login successful\n",
- "Your token has been saved to /root/.huggingface/token\n"
- ]
- }
- ],
+ "cell_type": "markdown",
"source": [
- "!huggingface-cli login"
- ]
+ "To be able to share your model with the community there are three more steps to follow:\n",
+ "\n",
+ "1️⃣ (If it's not already done) create an account to HF ➡ https://huggingface.co/join\n",
+ "\n",
+ "2️⃣ Sign in and then, you need to store your authentication token from the Hugging Face website.\n",
+ "- Create a new token (https://huggingface.co/settings/tokens) **with write role**"
+ ],
+ "metadata": {
+ "id": "izT6FpgNzZ6R"
+ }
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ ""
+ ],
+ "metadata": {
+ "id": "ynDmQo5TzdBq"
+ }
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
- "colab": {
- "background_save": true
- },
- "id": "HeylT8D0U4MB"
+ "id": "rKt2vsYoK56o"
},
"outputs": [],
"source": [
- "!git config --global credential.helper store"
+ "from huggingface_hub import notebook_login\n",
+ "notebook_login()"
]
},
+ {
+ "cell_type": "markdown",
+ "source": [
+ "If you don't want to use a Google Colab or a Jupyter Notebook, you need to use this command instead: `huggingface-cli login`"
+ ],
+ "metadata": {
+ "id": "ew59mK19zjtN"
+ }
+ },
{
"cell_type": "markdown",
"metadata": {
"id": "Xi0y_VASRzJU"
},
"source": [
- "Make changes so that it save to your id and correct run-id"
+ "Then, we simply need to run `mlagents-push-to-hf`.\n",
+ "\n"
]
},
+ {
+ "cell_type": "markdown",
+ "source": [
+ ""
+ ],
+ "metadata": {
+ "id": "NtuBFDHGzuG_"
+ }
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "And we define 4 parameters:\n",
+ "\n",
+ "1. `--run-id`: the name of the training run id.\n",
+ "2. `--local-dir`: where the agent was saved, it’s results/, so in my case results/First Training.\n",
+ "3. `--repo-id`: the name of the Hugging Face repo you want to create or update. It’s always /\n",
+ "If the repo does not exist **it will be created automatically**\n",
+ "4. `--commit-message`: since HF repos are git repository you need to define a commit message."
+ ],
+ "metadata": {
+ "id": "KK4fPfnczunT"
+ }
+ },
{
"cell_type": "code",
"execution_count": null,
"metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "dGEFAIboLVc6",
- "outputId": "d9d80a98-6735-40a1-d5ce-348b921dc4e4"
+ "id": "dGEFAIboLVc6"
},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "Pyramids Training\n",
- "kingabzpro/MLAgents-Pyramids2\n",
- "This function will create a model card and upload your Pyramids Training into HuggingFace Hub. This is a work in progress: If you encounter a bug, please send open an issue\n",
- "REPO NAME: MLAgents-Pyramids2\n",
- "ORGANIZATION: kingabzpro\n",
- "/usr/local/lib/python3.7/dist-packages/huggingface_hub/hf_api.py:94: FutureWarning: `name` and `organization` input arguments are deprecated and will be removed in v0.10. Pass `repo_id` instead.\n",
- " FutureWarning,\n"
- ]
- }
- ],
+ "outputs": [],
"source": [
- "!mlagents-push-to-hf --run-id=\"Pyramids Training\" --local-dir=\"./results/Pyramids Training\" --repo-id=\"kingabzpro/MLAgents-Pyramids2\" --commit-message=\"First Pyramids\""
+ "!mlagents-push-to-hf --run-id=\"Pyramids Training\" --local-dir=\"./results/Pyramids Training\" --repo-id=\"ThomasSimonini/testpyramidsrnd\" --commit-message=\"First Pyramids\""
]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "Else, if everything worked you should have this at the end of the process(but with a different url 😆) :\n",
+ "\n",
+ "\n",
+ "\n",
+ "```\n",
+ "Your model is pushed to the hub. You can view your model here: https://huggingface.co/ThomasSimonini/MLAgents-Pyramids\n",
+ "```\n",
+ "\n",
+ "It’s the link to your model, it contains a model card that explains how to use it, your Tensorboard and your config file. **What’s awesome is that it’s a git repository, that means you can have different commits, update your repository with a new push etc.**"
+ ],
+ "metadata": {
+ "id": "yborB0850FTM"
+ }
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ ""
+ ],
+ "metadata": {
+ "id": "6YSyjSqy0NJH"
+ }
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "But now comes the best: **being able to visualize your agent online 👀.**"
+ ],
+ "metadata": {
+ "id": "5Uaon2cg0NrL"
+ }
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "### Step 6: Watch our agent play 👀\n",
+ "\n",
+ "For this step it’s simple:\n",
+ "\n",
+ "Go to your repository\n",
+ "In Watch Your Agent Play section click on the link: https://huggingface.co/spaces/unity/ML-Agents-Pyramids"
+ ],
+ "metadata": {
+ "id": "VMc4oOsE0QiZ"
+ }
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ ""
+ ],
+ "metadata": {
+ "id": "Pp93P0fM0ZQP"
+ }
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "1. In step 1, choose your model repository which is the model id (in my case ThomasSimonini/MLAgents-Pyramids).\n",
+ "\n",
+ "2. In step 2, **choose what model you want to replay**:\n",
+ " - I have multiple one, since we saved a model every 500000 timesteps. \n",
+ " - But if I want the more recent I choose Pyramids.onnx\n",
+ "\n",
+ "👉 What’s nice **is to try with different models step to see the improvement of the agent.**"
+ ],
+ "metadata": {
+ "id": "Djs8c5rR0Z8a"
+ }
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "### 🎁 Bonus: Why not train on another environment?\n",
+ "Now that you know how to train an agent using MLAgents, **why not try another environment?** \n",
+ "\n",
+ "MLAgents provides 18 different and we’re building some custom ones. The best way to learn is to try things of your own, have fun.\n",
+ "\n"
+ ],
+ "metadata": {
+ "id": "hGG_oq2n0wjB"
+ }
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ ""
+ ],
+ "metadata": {
+ "id": "KSAkJxSr0z6-"
+ }
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "You have the full list of the one currently available on Hugging Face here 👉 https://github.com/huggingface/ml-agents#the-environments"
+ ],
+ "metadata": {
+ "id": "YiyF4FX-04JB"
+ }
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "That’s all for today. Congrats on finishing this tutorial! You’ve just trained your first ML-Agent and shared it to the Hub 🥳.\n",
+ "\n",
+ "The best way to learn is to practice and try stuff. Why not try another environment? ML-Agents has 18 different environments, but you can also create your own? Check the documentation and have fun!\n",
+ "\n",
+ "## Keep Learning, Stay awesome 🤗"
+ ],
+ "metadata": {
+ "id": "PI6dPWmh064H"
+ }
}
],
"metadata": {
"accelerator": "GPU",
"colab": {
"collapsed_sections": [],
- "name": "Unity3D DeepRL on Colab.ipynb",
+ "name": "Unit 4: Unity3D DeepRL on Colab.ipynb",
"provenance": [],
- "authorship_tag": "ABX9TyOOKb9z7tBT44PADMlCbLX/",
+ "private_outputs": true,
"include_colab_link": true
},
"gpuClass": "standard",
@@ -764,4 +561,4 @@
},
"nbformat": 4,
"nbformat_minor": 0
-}
+}
\ No newline at end of file