CS_ALearn/deep-rl-class
1
0
Fork 0
mirror of https://github.com/huggingface/deep-rl-class.git synced 2026-02-03 02:14:53 +08:00
Code Issues Projects Releases Wiki Activity

unit3 | deep-q-algorithm | catastrophic forgetting

Browse Source
This commit is contained in:
DESKTOP-AENDA0E\Fardin
2023-12-11 11:08:54 +01:00
parent 21df13744c
commit 8692616b7a
1 changed files with 2 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
units/en/unit3/deep-q-algorithm.mdx
View File

@@ -40,8 +40,8 @@ Experience replay helps by **using the experiences of the training more efficien
⇒ This allows the agent to **learn from the same experiences multiple times**.
2. **Avoid forgetting previous experiences and reduce the correlation between experiences**.
- The problem we get if we give sequential samples of experiences to our neural network is that it tends to forget **the previous experiences as it gets new experiences.** For instance, if the agent is in the first level and then in the second, which is different, it can forget how to behave and play in the first level.
2. **Avoid forgetting previous experiences (aka catastrophic interference, or catastrophic forgetting) and reduce the correlation between experiences**.
- **[catastrophic forgetting](https://en.wikipedia.org/wiki/Catastrophic_interference)**: The problem we get if we give sequential samples of experiences to our neural network is that it tends to forget **the previous experiences as it gets new experiences.** For instance, if the agent is in the first level and then in the second, which is different, it can forget how to behave and play in the first level.
The solution is to create a Replay Buffer that stores experience tuples while interacting with the environment and then sample a small batch of tuples. This prevents **the network from only learning about what it has done immediately before.**