diff --git a/pytorch/官方教程/01.md b/pytorch/官方教程/01.md deleted file mode 100644 index fef39340..00000000 --- a/pytorch/官方教程/01.md +++ /dev/null @@ -1 +0,0 @@ -# 学习 PyTorch \ No newline at end of file diff --git a/pytorch/官方教程/01概述.md b/pytorch/官方教程/01概述.md new file mode 100644 index 00000000..bac5dce5 --- /dev/null +++ b/pytorch/官方教程/01概述.md @@ -0,0 +1,88 @@ +# 学习 PyTorch + + +## 过程 + +1. 获取数据集 +2. 数据预处理 +3. 训练模型 + 1. 神经网络：torch.nn.Model.__init__定义具有一些可学习参数（或权重）的神经网络 + 2. 正向传播：torch.nn.Model.forward通过网络处理输入，进行正向传播 + 3. 计算损失：torch.nn.loss输出正确的距离有多远 + 4. 反向传播：torch.tensor.backward将梯度传播回网络参数 + 5. 更新权重：troch.optim通常使用简单的更新规则来更新网络的权重：weight = weight - learning_rate * gradient +4. 验证模型 +5. 使用模型 + + +## 术语 + +* 特征`3*32*32` +* 层：卷积层、池化层、全连接层 +* 算子：卷积层使用的卷积算子。池化层的赤化算子`3*3` +* 激活函数：卷积层的的每个卷积算子计算完成后输出一个新的特征。激活函数对这个卷积算子处理后的结果进行第二次处理。`1*1`。不改变特征的维度。只添加一个对特征的调整。 +* 权重。在深度神经网络中，权重使用来计算z的求和使用的。在卷积神经网络中， + + + +## 标准神经网络 + +* 现实生活中，channel表示通道的数量。神经网络的计算图，从前向后，有无数的通道。channel就是表示这个通道的数量。channel:某一层输入输出的数量。分别用inchannls和outchannels +* 通道的数量，指某一层的通道的数量，而非相对于某一个神经元来说。同一层的神经元遵循相同的计算方法（使用同一个算子） +* 在普通神经网络中，一个channel表示一条通道（连线），就只能产生一个值。这个值会×权重，作为下一层的激活值。然后调用激活函数，作为下一层的输出值。 +* 在卷积神经网络中，channel代表的含义也一样。一个channel表示一条通道（连线），但是这个通道上传递的是高维的特征数据，而非简单的一个数据。如conv2d，传递的可能是`32*32`的特征。第二层使用了inchannel3，outchannel6，卷积算子（卷积核）的大小为3.该层处理的输入特征为3个channel的`32*32`。输出特征是6个channel的`28*28`。表示该层 + +* 在标准神经网络的构建过程中，可以在前向传播过程中对函数过程进行控制。可以构造任意形状的计算图。每一层的神经元可以不一样。 +## 卷积神经网络 + + +* 与之前的卷积过程相比较，卷积神经网络的单层结构多了激活函数和偏移量；而与标准神经网络： + +$$Z^{[l]} = W^{[l]}A^{[l-1]}+b$$ +$$A^{[l]} = g^{[l]}(Z^{[l]})$$ + +* 相比，滤波器的数值对应着权重 $W^{[l]}$，卷积运算对应着 $W^{[l]}$与 $A^{[l-1]}$的乘积运算，所选的激活函数变为 ReLU。 + +* 对于一个 3x3x3 的滤波器，包括偏移量 $b$在内共有 28 个参数。不论输入的图片有多大，用这一个滤波器来提取特征时，参数始终都是 28 个，固定不变。即**选定滤波器组后，参数的数目与输入图片的尺寸无关**。因此，卷积神经网络的参数相较于标准神经网络来说要少得多。这是 CNN 的优点之一。 + +## 数据规模——普通神经网络 + +## 数据规模——卷积神经网络 + +### channel的理解 +> 与数据规模相关的量主要包括以下几种。他们共同构成了张量的各个维度。在torch.nn当中的运算，都是以层为单位进行计算的。也就是说，同一层只有一种神经元，多个神经元表示多个channel，同一层的输入输出使用一个张量来表示。 + +### 数据的规模 +对torch.nn来说。一个张量至少应该包括四个维度。batch_size,channesl,height,width。每一个张量表示某一层的所有输入或输出。 + + +* batch_size:张量的第一个维度。表示输入样本的个数。在传播计算和理解过程中，可以假设没有这个维度。但是在实际的运算过程中，第一维表示的batch_size +* channels:表示该层算子的个数、神经元的个数。in_channels表示输入的神经元的个数。out_channels表示输出神经元的个数。 +* height +* width[-depth]表示一张图片或者一个条数据的大小。 + + +### 参数的数量 +对于每一层的参数的数量的计算如下。也就是说每一个卷积层有两个参数， 一个参数是weight，另一个参数是bias。具体的就是conv2.weight conv2.bias + +```py +>>> conv2 = nn.Conv2d(in_channels=3, out_channels=32, kernel_size=(4,3,2)) +>>> for (name, param) in conv2.named_parameters(): +>>> print(name) +weight +bias +>>> print(conv2.weight.shape) +torch.Size([32, 3, 4, 3, 2]) +``` + + +## 构成 + +> 在构建普通神经网络的时候。可以实现不同的控制级别。 + +* 神经网络的构成主要包括以下几个内容：数据+算子+过程。实现模型的训练。 + * 数据（样本）或者说激活值。 + * 算子。 + * 自己定义的算子，需要自己声明参数并初始化。 + * 使用系统定义的算子，系统会自动添加参数。 + * 过程：前项传播的函数。后向传播的函数。误差计算的函数。梯度下降的函数。 \ No newline at end of file diff --git a/pytorch/官方教程/02.md b/pytorch/官方教程/02.md deleted file mode 100644 index ef79c60d..00000000 --- a/pytorch/官方教程/02.md +++ /dev/null @@ -1,39 +0,0 @@ -# PyTorch 深度学习：60分钟快速入门 - -> 原文： - -**作者**： [Soumith Chintala](http://soumith.ch) - - - -## 什么是 PyTorch？ - -PyTorch 是基于以下两个目的而打造的python科学计算框架： - -* 无缝替换NumPy，并且通过利用GPU的算力来实现神经网络的加速。 -* 通过自动微分机制，来让神经网络的实现变得更加容易。 - -## 本次教程的目标： - -* 深入了解PyTorch的张量单元以及如何使用Pytorch来搭建神经网络。 -* 自己动手训练一个小型神经网络来实现图像的分类。 - -注意 - -确保已安装[`torch`](https://github.com/pytorch/pytorch)和[`torchvision`](https://github.com/pytorch/vision)包。 - -![../_img/tensor_illustration_flat.png](img/0c7a402331744a44f5e17575b1607904.png) - -[张量](blitz/tensor_tutorial.html#sphx-glr-beginner-blitz-tensor-tutorial-py) - -![../_img/autodiff.png](img/0a7a97c39d6dfc0e08d2701eb7a49231.png) - -[`torch.autograd`的简要介绍](blitz/autograd_tutorial.html#sphx-glr-beginner-blitz-autograd-tutorial-py) - -![../_img/mnist1.png](img/be60e8e1f4baa0de87cf9d37c5325525.png) - -[神经网络简介](blitz/neural_networks_tutorial.html#sphx-glr-beginner-blitz-neural-networks-tutorial-py) - -![../_img/cifar101.png](img/7a28f697e6bab9f3d9b1e8da4a5a5249.png) - -[自己动手训练一个图像分类器](blitz/cifar10_tutorial.html#sphx-glr-beginner-blitz-cifar10-tutorial-py) diff --git a/pytorch/官方教程/02Pytorch.md b/pytorch/官方教程/02Pytorch.md new file mode 100644 index 00000000..6904f60c --- /dev/null +++ b/pytorch/官方教程/02Pytorch.md @@ -0,0 +1,28 @@ +# PyTorch 深度学习：60分钟快速入门 + +> 原文： + +**作者**： [Soumith Chintala](http://soumith.ch) + + + +## 什么是 PyTorch？ + +PyTorch 是基于以下两个目的而打造的python科学计算框架： + +* 无缝替换NumPy，并且通过利用GPU的算力来实现神经网络的加速。 +* 通过自动微分机制，来让神经网络的实现变得更加容易。 + +## 本次教程的目标： + +* 深入了解PyTorch的张量单元以及如何使用Pytorch来搭建神经网络。 +* 自己动手训练一个小型神经网络来实现图像的分类。 + + +## 目录 + +* [02Pytorch](02Pytorch.md) +* [03Tensor张量](03Tensor.md) +* [04autograd](04Autograd.md) +* [05神经网络](05NN.md) +* [06图像分类器](06Classification.md) diff --git a/pytorch/官方教程/03.md b/pytorch/官方教程/03Tensor.md similarity index 100% rename from pytorch/官方教程/03.md rename to pytorch/官方教程/03Tensor.md diff --git a/pytorch/官方教程/04.md b/pytorch/官方教程/04Autograd.md similarity index 100% rename from pytorch/官方教程/04.md rename to pytorch/官方教程/04Autograd.md diff --git a/pytorch/官方教程/05.md b/pytorch/官方教程/05NN.md similarity index 98% rename from pytorch/官方教程/05.md rename to pytorch/官方教程/05NN.md index 06eac934..9783d1b5 100644 --- a/pytorch/官方教程/05.md +++ b/pytorch/官方教程/05NN.md @@ -17,8 +17,8 @@ 神经网络的典型训练过程如下： * 定义具有一些可学习参数（或权重）的神经网络 -* 遍历输入数据集 -* 通过网络处理输入 +* 遍历输入数据集，进行数据预处理 +* 通过网络处理输入，进行正向传播 * 计算损失（输出正确的距离有多远） * 将梯度传播回网络参数 * 通常使用简单的更新规则来更新网络的权重：`weight = weight - learning_rate * gradient` diff --git a/pytorch/官方教程/06.md b/pytorch/官方教程/06ImageClassification.md similarity index 100% rename from pytorch/官方教程/06.md rename to pytorch/官方教程/06ImageClassification.md diff --git a/pytorch/官方教程/07.md b/pytorch/官方教程/07example.md similarity index 92% rename from pytorch/官方教程/07.md rename to pytorch/官方教程/07example.md index 4cedfe0b..691e68b4 100644 --- a/pytorch/官方教程/07.md +++ b/pytorch/官方教程/07example.md @@ -17,9 +17,7 @@ PyTorch 的核心是提供两个主要功能： 您可以在[本页](#examples-download)浏览各个示例。 -## 张量 - -### 预热：NumPy +## 1 预热：NumPy 在介绍 PyTorch 之前，我们将首先使用 numpy 实现网络。 @@ -68,7 +66,7 @@ print(f'Result: y = {a} + {b} x + {c} x^2 + {d} x^3') ``` -### PyTorch：张量 +## 2 PyTorch：张量 Numpy 是一个很棒的框架，但是它不能利用 GPU 来加速其数值计算。 对于现代深度神经网络，GPU 通常会提供 [50 倍或更高](https://github.com/jcjohnson/cnn-benchmarks)的加速，因此遗憾的是，numpy 不足以实现现代深度学习。 @@ -125,9 +123,9 @@ print(f'Result: y = {a.item()} + {b.item()} x + {c.item()} x^2 + {d.item()} x^3' ``` -## Autograd +## 3 Autograd -### PyTorch：张量和 Autograd +## 3.1 PyTorch：张量和 Autograd 在上述示例中，我们必须手动实现神经网络的前向和后向传递。 对于小型的两层网络，手动实现反向传递并不是什么大问题，但是对于大型的复杂网络来说，可以很快变得非常麻烦。 @@ -198,7 +196,7 @@ print(f'Result: y = {a.item()} + {b.item()} x + {c.item()} x^2 + {d.item()} x^3' ``` -### PyTorch：定义新的 Autograd 函数 +## 3.2 PyTorch：定义新的 Autograd 函数 在幕后，每个原始的 Autograd 运算符实际上都是在张量上运行的两个函数。 **正向**函数从输入张量计算输出张量。 **反向**函数接收相对于某个标量值的输出张量的梯度，并计算相对于相同标量值的输入张量的梯度。 @@ -293,9 +291,9 @@ print(f'Result: y = {a.item()} + {b.item()} * P3({c.item()} + {d.item()} x)') ``` -## `nn`模块 +## 4 `nn`模块 -### PyTorch：`nn` +## 4.1 PyTorch：`nn` 计算图和 Autograd 是定义复杂运算符并自动采用导数的非常强大的范例。 但是对于大型神经网络，原始的 Autograd 可能会太低级。 @@ -380,7 +378,7 @@ print(f'Result: y = {linear_layer.bias.item()} + {linear_layer.weight[:, 0].item ``` -### PyTorch：`optim` +## 4.2 PyTorch：`optim` 到目前为止，我们已经通过使用`torch.no_grad()`手动更改持有可学习参数的张量来更新模型的权重。 对于像随机梯度下降这样的简单优化算法来说，这并不是一个巨大的负担，但是在实践中，我们经常使用更复杂的优化器（例如 AdaGrad，RMSProp，Adam 等）来训练神经网络。 @@ -443,7 +441,7 @@ print(f'Result: y = {linear_layer.bias.item()} + {linear_layer.weight[:, 0].item ``` -### PyTorch：自定义`nn`模块 +## 4.3 PyTorch：自定义`nn`模块 有时，您将需要指定比一系列现有模块更复杂的模型。 对于这些情况，您可以通过子类化`nn.Module`并定义一个`forward`来定义自己的模块，该模块使用其他模块或在 Tensors 上的其他自动转换操作来接收输入 Tensors 并生成输出 Tensors。 @@ -510,7 +508,7 @@ print(f'Result: {model.string()}') ``` -### PyTorch：控制流 + 权重共享 +## 4.4 PyTorch：控制流 + 权重共享 作为动态图和权重共享的示例，我们实现了一个非常奇怪的模型：一个三阶多项式，在每个正向传播中选择 3 到 5 之间的一个随机数，并使用该阶数，多次使用相同的权重重复计算四和五阶。 @@ -586,46 +584,4 @@ for t in range(30000): print(f'Result: {model.string()}') -``` - -## 示例 - -您可以在此处浏览以上示例。 - -### 张量 - -![../_img/sphx_glr_polynomial_numpy_thumb.png](img/ea0bddb69dfbd67215b823007544ab8f.png) - -[热身：NumPy](examples_tensor/polynomial_numpy.html#sphx-glr-beginner-examples-tensor-polynomial-numpy-py) - -![../_img/sphx_glr_polynomial_tensor_thumb.png](img/04ee335faf821b337dba0c4d7ccb0b67.png) - -[PyTorch：张量](examples_tensor/polynomial_tensor.html#sphx-glr-beginner-examples-tensor-polynomial-tensor-py) - -### Autograd - -![../_img/sphx_glr_polynomial_autograd_thumb.png](img/ffad28c33f8a48d06521421f1aa441ed.png) - -[PyTorch：张量和 Autograd](examples_autograd/polynomial_autograd.html#sphx-glr-beginner-examples-autograd-polynomial-autograd-py) - -![../_img/sphx_glr_polynomial_custom_function_thumb.png](img/a5c5d931ed12e34bf68476f4f157b780.png) - -[PyTorch：定义新的 Autograd 函数](examples_autograd/polynomial_custom_function.html#sphx-glr-beginner-examples-autograd-polynomial-custom-function-py) - -### `nn`模块 - -![../_img/sphx_glr_polynomial_nn_thumb.png](img/335fb81e535f98bfda7cbdb3e50d8832.png) - -[PyTorch：`nn`](examples_nn/polynomial_nn.html#sphx-glr-beginner-examples-nn-polynomial-nn-py) - -![../_img/sphx_glr_polynomial_optim_thumb.png](img/87aa5017f5f0ba9a29d66e74ac6b3d1a.png) - -[PyTorch：`optim`](examples_nn/polynomial_optim.html#sphx-glr-beginner-examples-nn-polynomial-optim-py) - -![../_img/sphx_glr_polynomial_module_thumb.png](img/b3f0b96ed8ba751fee4a5fc7ca878eb1.png) - -[PyTorch：自定义`nn`模块](examples_nn/polynomial_module.html#sphx-glr-beginner-examples-nn-polynomial-module-py) - -![../_img/sphx_glr_dynamic_net_thumb.png](img/bf0b252ce2d39ba6da26c16bee984d39.png) - -[PyTorch：控制流 + 权重共享](examples_nn/dynamic_net.html#sphx-glr-beginner-examples-nn-dynamic-net-py) \ No newline at end of file +``` \ No newline at end of file diff --git a/pytorch/官方教程/08.md b/pytorch/官方教程/08.md index d78d6eb0..e6867d85 100644 --- a/pytorch/官方教程/08.md +++ b/pytorch/官方教程/08.md @@ -1,59 +1,971 @@ -# 热身：NumPy +# `torch.nn`到底是什么？ -> 原文： +> 原文： -经过训练的三阶多项式，可以通过最小化平方的欧几里得距离来预测`y = sin(x)`从`-pi`到`pi`。 +作者：Jeremy Howard，[fast.ai](https://www.fast.ai)。 感谢 Rachel Thomas 和 Francisco Ingham。 -此实现使用 numpy 手动计算正向传播，损失和后向通过。 +我们建议将本教程作为笔记本而不是脚本来运行。 要下载笔记本（`.ipynb`）文件，请单击页面顶部的链接。 -numpy 数组是通用的 n 维数组； 它对深度学习，梯度或计算图一无所知，而只是执行通用数值计算的一种方法。 +PyTorch 提供设计精美的模块和类[`torch.nn`](https://pytorch.org/docs/stable/nn.html)，[`torch.optim`](https://pytorch.org/docs/stable/optim.html)，[`Dataset`](https://pytorch.org/docs/stable/data.html?highlight=dataset#torch.utils.data.Dataset)和[`DataLoader`](https://pytorch.org/docs/stable/data.html?highlight=dataloader#torch.utils.data.DataLoader)神经网络。 为了充分利用它们的功能并针对您的问题对其进行自定义，您需要真正了解它们在做什么。 为了建立这种理解，我们将首先在 MNIST 数据集上训练基本神经网络，而无需使用这些模型的任何功能。 我们最初将仅使用最基本的 PyTorch 张量函数。 然后，我们将一次从`torch.nn`，`torch.optim`，`Dataset`或`DataLoader`中逐个添加一个函数，以准确显示每个函数，以及如何使代码更简洁或更有效。 灵活。 + +**本教程假定您已经安装了 PyTorch，并且熟悉张量操作的基础知识。** （如果您熟悉 Numpy 数组操作，将会发现此处使用的 PyTorch 张量操作几乎相同）。 + +## MNIST 数据集 + +我们将使用经典的 [MNIST](http://deeplearning.net/data/mnist/) 数据集，该数据集由手绘数字的黑白图像组成（0 到 9 之间）。 + +我们将使用[`pathlib`](https://docs.python.org/3/library/pathlib.html)处理路径（Python 3 标准库的一部分），并使用[`requests`](http://docs.python-requests.org/en/master/)下载数据集。 我们只会在使用模块时才导入它们，因此您可以确切地看到每个位置上正在使用的模块。 ```py -import numpy as np -import math +from pathlib import Path +import requests -# Create random input and output data -x = np.linspace(-math.pi, math.pi, 2000) -y = np.sin(x) +DATA_PATH = Path("data") +PATH = DATA_PATH / "mnist" -# Randomly initialize weights -a = np.random.randn() -b = np.random.randn() -c = np.random.randn() -d = np.random.randn() +PATH.mkdir(parents=True, exist_ok=True) -learning_rate = 1e-6 -for t in range(2000): - # Forward pass: compute predicted y - # y = a + b x + c x^2 + d x^3 - y_pred = a + b * x + c * x ** 2 + d * x ** 3 +URL = "https://github.com/pytorch/tutorials/raw/master/_static/" +FILENAME = "mnist.pkl.gz" - # Compute and print loss - loss = np.square(y_pred - y).sum() - if t % 100 == 99: - print(t, loss) - - # Backprop to compute gradients of a, b, c, d with respect to loss - grad_y_pred = 2.0 * (y_pred - y) - grad_a = grad_y_pred.sum() - grad_b = (grad_y_pred * x).sum() - grad_c = (grad_y_pred * x ** 2).sum() - grad_d = (grad_y_pred * x ** 3).sum() - - # Update weights - a -= learning_rate * grad_a - b -= learning_rate * grad_b - c -= learning_rate * grad_c - d -= learning_rate * grad_d - -print(f'Result: y = {a} + {b} x + {c} x^2 + {d} x^3') +if not (PATH / FILENAME).exists(): + content = requests.get(URL + FILENAME).content + (PATH / FILENAME).open("wb").write(content) ``` -**脚本的总运行时间**：（0 分钟 0.000 秒） +该数据集为 numpy 数组格式，并已使用`pickle`（一种用于序列化数据的 python 特定格式）存储。 -[下载 Python 源码：`polynomial_numpy.py`](https://pytorch.org/tutorials/_downloads/6287cd68dd239d4f34ac75d774a66e23/polynomial_numpy.py) +```py +import pickle +import gzip -[下载 Jupyter 笔记本：`polynomial_numpy.ipynb`](https://pytorch.org/tutorials/_downloads/d4cfaf6a36486a5e37afb34266028d9e/polynomial_numpy.ipynb) +with gzip.open((PATH / FILENAME).as_posix(), "rb") as f: + ((x_train, y_train), (x_valid, y_valid), _) = pickle.load(f, encoding="latin-1") + +``` + +每个图像为`28 x 28`，并存储为长度为`784 = 28x28`的扁平行。 让我们来看一个； 我们需要先将其重塑为 2d。 + +```py +from matplotlib import pyplot +import numpy as np + +pyplot.imshow(x_train[0].reshape((28, 28)), cmap="gray") +print(x_train.shape) + +``` + +![../_img/sphx_glr_nn_tutorial_001.png](img/7c783def0bbe536f41ed172041b7e89e.png) + +出： + +```py +(50000, 784) + +``` + +PyTorch 使用`torch.tensor`而不是 numpy 数组，因此我们需要转换数据。 + +```py +import torch + +x_train, y_train, x_valid, y_valid = map( + torch.tensor, (x_train, y_train, x_valid, y_valid) +) +n, c = x_train.shape +x_train, x_train.shape, y_train.min(), y_train.max() +print(x_train, y_train) +print(x_train.shape) +print(y_train.min(), y_train.max()) + +``` + +出： + +```py +tensor([[0., 0., 0., ..., 0., 0., 0.], + [0., 0., 0., ..., 0., 0., 0.], + [0., 0., 0., ..., 0., 0., 0.], + ..., + [0., 0., 0., ..., 0., 0., 0.], + [0., 0., 0., ..., 0., 0., 0.], + [0., 0., 0., ..., 0., 0., 0.]]) tensor([5, 0, 4, ..., 8, 4, 8]) +torch.Size([50000, 784]) +tensor(0) tensor(9) + +``` + +## 从零开始的神经网络（没有`torch.nn`） + +首先，我们仅使用 PyTorch 张量操作创建模型。 我们假设您已经熟悉神经网络的基础知识。 （如果不是，则可以在 [course.fast.ai](https://course.fast.ai) 中学习它们）。 + +PyTorch 提供了创建随机或零填充张量的方法，我们将使用它们来为简单的线性模型创建权重和偏差。 这些只是常规张量，还有一个非常特殊的附加值：我们告诉 PyTorch 它们需要梯度。 这使 PyTorch 记录了在张量上完成的所有操作，因此它可以在反向传播时*自动计算*的梯度！ + +**对于权重，我们在初始化之后设置`requires_grad`，因为我们不希望该步骤包含在梯度中。 （请注意，PyTorch 中的尾随`_`表示该操作是原地执行的。）** + +注意 + +我们在这里用 [Xavier 初始化](http://proceedings.mlr.press/v9/glorot10a/glorot10a.pdf)（通过乘以`1 / sqrt(n)`）来初始化权重。 + +```py +import math + +weights = torch.randn(784, 10) / math.sqrt(784) +weights.requires_grad_() +bias = torch.zeros(10, requires_grad=True) + +``` + +由于 PyTorch 具有自动计算梯度的功能，我们可以将任何标准的 Python 函数（或可调用对象）用作模型！ 因此，我们只需编写一个普通矩阵乘法和广播加法即可创建一个简单的线性模型。 我们还需要激活函数，因此我们将编写并使用`log_softmax`。 请记住：尽管 PyTorch 提供了许多预写的损失函数，激活函数等，但是您可以使用纯 Python 轻松编写自己的函数。 PyTorch 甚至会自动为您的函数创建快速 GPU 或向量化的 CPU 代码。 + +```py +def log_softmax(x): + return x - x.exp().sum(-1).log().unsqueeze(-1) + +def model(xb): + return log_softmax(xb @ weights + bias) + +``` + +在上面，`@`代表点积运算。 我们将对一批数据（在本例中为 64 张图像）调用函数。 这是一个*正向传播*。 请注意，由于我们从随机权重开始，因此在这一阶段，我们的预测不会比随机预测更好。 + +```py +bs = 64 # batch size + +xb = x_train[0:bs] # a mini-batch from x +preds = model(xb) # predictions +preds[0], preds.shape +print(preds[0], preds.shape) + +``` + +出： + +```py +tensor([-2.5964, -2.3153, -2.1321, -2.4480, -2.2930, -1.9507, -2.1289, -2.4175, + -2.5332, -2.3967], grad_fn=) torch.Size([64, 10]) + +``` + +如您所见，`preds`张量不仅包含张量值，还包含梯度函数。 稍后我们将使用它进行反向传播。 + +让我们实现负对数可能性作为损失函数（同样，我们只能使用标准 Python）： + +```py +def nll(input, target): + return -input[range(target.shape[0]), target].mean() + +loss_func = nll + +``` + +让我们使用随机模型来检查损失，以便我们稍后查看反向传播后是否可以改善我们的损失。 + +```py +yb = y_train[0:bs] +print(loss_func(preds, yb)) + +``` + +出： + +```py +tensor(2.3735, grad_fn=) + +``` + +我们还实现一个函数来计算模型的准确率。 对于每个预测，如果具有最大值的索引与目标值匹配，则该预测是正确的。 + +```py +def accuracy(out, yb): + preds = torch.argmax(out, dim=1) + return (preds == yb).float().mean() + +``` + +让我们检查一下随机模型的准确率，以便我们可以看出随着损失的增加，准确率是否有所提高。 + +```py +print(accuracy(preds, yb)) + +``` + +出： + +```py +tensor(0.0938) + +``` + +现在，我们可以运行一个训练循环。 对于每次迭代，我们将： + +* 选择一个小批量数据（大小为`bs`） +* 使用模型进行预测 +* 计算损失 +* `loss.backward()`更新模型的梯度，在这种情况下为`weights`和`bias`。 + +现在，我们使用这些梯度来更新权重和偏差。 我们在`torch.no_grad()`上下文管理器中执行此操作，因为我们不希望在下一步的梯度计算中记录这些操作。 [您可以在这里阅读有关 PyTorch 的 Autograd 如何记录操作的更多信息](https://pytorch.org/docs/stable/notes/autograd.html)。 + +然后，将梯度设置为零，以便为下一个循环做好准备。 否则，我们的梯度会记录所有已发生操作的运行记录（即`loss.backward()`将梯度添加到已存储的内容中，而不是替换它们）。 + +小费 + +您可以使用标准的 python 调试器逐步浏览 PyTorch 代码，从而可以在每一步检查各种变量值。 取消注释以下`set_trace()`即可尝试。 + +```py +from IPython.core.debugger import set_trace + +lr = 0.5 # learning rate +epochs = 2 # how many epochs to train for + +for epoch in range(epochs): + for i in range((n - 1) // bs + 1): + # set_trace() + start_i = i * bs + end_i = start_i + bs + xb = x_train[start_i:end_i] + yb = y_train[start_i:end_i] + pred = model(xb) + loss = loss_func(pred, yb) + + loss.backward() + with torch.no_grad(): + weights -= weights.grad * lr + bias -= bias.grad * lr + weights.grad.zero_() + bias.grad.zero_() + +``` + +就是这样：我们完全从头开始创建并训练了一个最小的神经网络（在这种情况下，是逻辑回归，因为我们没有隐藏的层）！ + +让我们检查损失和准确率，并将其与我们之前获得的进行比较。 我们希望损失会减少，准确率会增加，而且确实如此。 + +```py +print(loss_func(model(xb), yb), accuracy(model(xb), yb)) + +``` + +出： + +```py +tensor(0.0811, grad_fn=) tensor(1.) + +``` + +## 使用`torch.nn.functional` + +现在，我们将重构代码，使其执行与以前相同的操作，只是我们将开始利用 PyTorch 的`nn`类使其更加简洁和灵活。 从这里开始的每一步，我们都应该使代码中的一个或多个：更短，更易理解和/或更灵活。 + +第一步也是最简单的步骤，就是用`torch.nn.functional`（通常按照惯例将其导入到名称空间`F`中）替换我们的手写激活和损失函数，从而缩短代码长度。 该模块包含`torch.nn`库中的所有函数（而该库的其他部分包含类）。 除了广泛的损失和激活函数外，您还会在这里找到一些方便的函数来创建神经网络，例如合并函数。 （还有一些用于进行卷积，线性层等的函数，但是正如我们将看到的那样，通常可以使用库的其他部分来更好地处理这些函数。） + +如果您使用的是负对数似然损失和对数 softmax 激活，那么 Pytorch 会提供结合了两者的单一函数`F.cross_entropy`。 因此，我们甚至可以从模型中删除激活函数。 + +```py +import torch.nn.functional as F + +loss_func = F.cross_entropy + +def model(xb): + return xb @ weights + bias + +``` + +请注意，我们不再在`model`函数中调用`log_softmax`。 让我们确认我们的损失和准确率与以前相同： + +```py +print(loss_func(model(xb), yb), accuracy(model(xb), yb)) + +``` + +出： + +```py +tensor(0.0811, grad_fn=) tensor(1.) + +``` + +## 使用`nn.Module`重构 + +接下来，我们将使用`nn.Module`和`nn.Parameter`进行更清晰，更简洁的训练循环。 我们将`nn.Module`子类化（它本身是一个类并且能够跟踪状态）。 在这种情况下，我们要创建一个类，该类包含前进步骤的权重，偏置和方法。 `nn.Module`具有许多我们将要使用的属性和方法（例如`.parameters()`和`.zero_grad()`）。 + +注意 + +`nn.Module`（大写`M`）是 PyTorch 的特定概念，并且是我们将经常使用的一类。 不要将`nn.Module`与[模块](https://docs.python.org/3/tutorial/modules.html)（小写`m`）的 Python 概念混淆，该模块是可以导入的 Python 代码文件。 + +```py +from torch import nn + +class Mnist_Logistic(nn.Module): + def __init__(self): + super().__init__() + self.weights = nn.Parameter(torch.randn(784, 10) / math.sqrt(784)) + self.bias = nn.Parameter(torch.zeros(10)) + + def forward(self, xb): + return xb @ self.weights + self.bias + +``` + +由于我们现在使用的是对象而不是仅使用函数，因此我们首先必须实例化模型： + +```py +model = Mnist_Logistic() + +``` + +现在我们可以像以前一样计算损失。 请注意，`nn.Module`对象的使用就好像它们是函数一样（即，它们是*可调用的*），但是在后台 Pytorch 会自动调用我们的`forward`方法。 + +```py +print(loss_func(model(xb), yb)) + +``` + +出： + +```py +tensor(2.3903, grad_fn=) + +``` + +以前，在我们的训练循环中，我们必须按名称更新每个参数的值，并手动将每个参数的梯度分别归零，如下所示： + +```py +with torch.no_grad(): + weights -= weights.grad * lr + bias -= bias.grad * lr + weights.grad.zero_() + bias.grad.zero_() + +``` + +现在我们可以利用`model.parameters()`和`model.zero_grad()`（它们都由 PyTorch 为`nn.Module`定义）来使这些步骤更简洁，并且更不会出现忘记某些参数的错误，尤其是当我们有一个更复杂的模型的时候： + +```py +with torch.no_grad(): + for p in model.parameters(): p -= p.grad * lr + model.zero_grad() + +``` + +我们将把小的训练循环包装在`fit`函数中，以便稍后再运行。 + +```py +def fit(): + for epoch in range(epochs): + for i in range((n - 1) // bs + 1): + start_i = i * bs + end_i = start_i + bs + xb = x_train[start_i:end_i] + yb = y_train[start_i:end_i] + pred = model(xb) + loss = loss_func(pred, yb) + + loss.backward() + with torch.no_grad(): + for p in model.parameters(): + p -= p.grad * lr + model.zero_grad() + +fit() + +``` + +让我们仔细检查一下我们的损失是否减少了： + +```py +print(loss_func(model(xb), yb)) + +``` + +出： + +```py +tensor(0.0808, grad_fn=) + +``` + +## 使用`nn.Linear`重构 + +我们继续重构我们的代码。 代替手动定义和初始化`self.weights`和`self.bias`并计算`xb  @ self.weights + self.bias`，我们将对线性层使用 Pytorch 类[`nn.Linear`](https://pytorch.org/docs/stable/nn.html#linear-layers)，这将为我们完成所有工作。 Pytorch 具有许多类型的预定义层，可以大大简化我们的代码，并且通常也可以使其速度更快。 + +```py +class Mnist_Logistic(nn.Module): + def __init__(self): + super().__init__() + self.lin = nn.Linear(784, 10) + + def forward(self, xb): + return self.lin(xb) + +``` + +我们以与以前相同的方式实例化模型并计算损失： + +```py +model = Mnist_Logistic() +print(loss_func(model(xb), yb)) + +``` + +出： + +```py +tensor(2.4215, grad_fn=) + +``` + +我们仍然可以使用与以前相同的`fit`方法。 + +```py +fit() + +print(loss_func(model(xb), yb)) + +``` + +出： + +```py +tensor(0.0824, grad_fn=) + +``` + +## 使用`optim`重构 + +Pytorch 还提供了一个包含各种优化算法的包`torch.optim`。 我们可以使用优化器中的`step`方法采取向前的步骤，而不是手动更新每个参数。 + +这将使我们替换之前的手动编码优化步骤： + +```py +with torch.no_grad(): + for p in model.parameters(): p -= p.grad * lr + model.zero_grad() + +``` + +而是只使用： + +```py +opt.step() +opt.zero_grad() + +``` + +（`optim.zero_grad()`将梯度重置为 0，我们需要在计算下一个小批量的梯度之前调用它。） + +```py +from torch import optim + +``` + +我们将定义一个小函数来创建模型和优化器，以便将来重用。 + +```py +def get_model(): + model = Mnist_Logistic() + return model, optim.SGD(model.parameters(), lr=lr) + +model, opt = get_model() +print(loss_func(model(xb), yb)) + +for epoch in range(epochs): + for i in range((n - 1) // bs + 1): + start_i = i * bs + end_i = start_i + bs + xb = x_train[start_i:end_i] + yb = y_train[start_i:end_i] + pred = model(xb) + loss = loss_func(pred, yb) + + loss.backward() + opt.step() + opt.zero_grad() + +print(loss_func(model(xb), yb)) + +``` + +出： + +```py +tensor(2.2999, grad_fn=) +tensor(0.0823, grad_fn=) + +``` + +## 使用`Dataset`重构 + +PyTorch 有一个抽象的`Dataset`类。 数据集可以是具有`__len__`函数（由 Python 的标准`len`函数调用）和具有`__getitem__`函数作为对其进行索引的一种方法。 [本教程](https://pytorch.org/tutorials/beginner/data_loading_tutorial.html)演示了一个不错的示例，该示例创建一个自定义`FacialLandmarkDataset`类作为`Dataset`的子类。 + +PyTorch 的[`TensorDataset`](https://pytorch.org/docs/stable/_modules/torch/utils/data/dataset.html#TensorDataset)是一个数据集包装张量。 通过定义索引的长度和方式，这也为我们提供了沿张量的第一维进行迭代，索引和切片的方法。 这将使我们在训练的同一行中更容易访问自变量和因变量。 + +```py +from torch.utils.data import TensorDataset + +``` + +`x_train`和`y_train`都可以合并为一个`TensorDataset`，这将更易于迭代和切片。 + +```py +train_ds = TensorDataset(x_train, y_train) + +``` + +以前，我们不得不分别遍历`x`和`y`值的小批量： + +```py +xb = x_train[start_i:end_i] +yb = y_train[start_i:end_i] + +``` + +现在，我们可以一起执行以下两个步骤： + +```py +xb,yb = train_ds[i*bs : i*bs+bs] + +``` + +```py +model, opt = get_model() + +for epoch in range(epochs): + for i in range((n - 1) // bs + 1): + xb, yb = train_ds[i * bs: i * bs + bs] + pred = model(xb) + loss = loss_func(pred, yb) + + loss.backward() + opt.step() + opt.zero_grad() + +print(loss_func(model(xb), yb)) + +``` + +出： + +```py +tensor(0.0819, grad_fn=) + +``` + +## 使用`DataLoader`重构 + +Pytorch 的`DataLoader`负责批量管理。 您可以从任何`Dataset`创建一个`DataLoader`。 `DataLoader`使迭代迭代变得更加容易。 不必使用`train_ds[i*bs : i*bs+bs]`，`DataLoader`会自动为我们提供每个小批量。 + +```py +from torch.utils.data import DataLoader + +train_ds = TensorDataset(x_train, y_train) +train_dl = DataLoader(train_ds, batch_size=bs) + +``` + +以前，我们的循环遍历如下批量`(xb, yb)`： + +```py +for i in range((n-1)//bs + 1): + xb,yb = train_ds[i*bs : i*bs+bs] + pred = model(xb) + +``` + +现在，我们的循环更加简洁了，因为`(xb, yb)`是从数据加载器自动加载的： + +```py +for xb,yb in train_dl: + pred = model(xb) + +``` + +```py +model, opt = get_model() + +for epoch in range(epochs): + for xb, yb in train_dl: + pred = model(xb) + loss = loss_func(pred, yb) + + loss.backward() + opt.step() + opt.zero_grad() + +print(loss_func(model(xb), yb)) + +``` + +出： + +```py +tensor(0.0821, grad_fn=) + +``` + +得益于 Pytorch 的`nn.Module`，`nn.Parameter`，`Dataset`和`DataLoader`，我们的训练循环现在变得更小，更容易理解。 现在，让我们尝试添加在实践中创建有效模型所需的基本功能。 + +## 添加验证 + +在第 1 节中，我们只是试图建立一个合理的训练循环以用于我们的训练数据。 实际上，您也应该**始终**具有[验证集](https://www.fast.ai/2017/11/13/validation-sets/)，以便识别您是否过拟合。 + +[对训练数据进行打乱](https://www.quora.com/Does-the-order-of-training-data-matter-when-training-neural-networks)对于防止批量与过拟合之间的相关性很重要。 另一方面，无论我们是否打乱验证集，验证损失都是相同的。 由于打乱需要花费更多时间，因此打乱验证数据没有任何意义。 + +我们将验证集的批量大小设为训练集的两倍。 这是因为验证集不需要反向传播，因此占用的内存更少（不需要存储梯度）。 我们利用这一优势来使用更大的批量，并更快地计算损失。 + +```py +train_ds = TensorDataset(x_train, y_train) +train_dl = DataLoader(train_ds, batch_size=bs, shuffle=True) + +valid_ds = TensorDataset(x_valid, y_valid) +valid_dl = DataLoader(valid_ds, batch_size=bs * 2) + +``` + +我们将在每个周期结束时计算并打印验证损失。 + +（请注意，我们总是在训练之前调用`model.train()`，并在推理之前调用`model.eval()`，因为诸如`nn.BatchNorm2d`和`nn.Dropout`之类的层会使用它们，以确保这些不同阶段的行为正确。） + +```py +model, opt = get_model() + +for epoch in range(epochs): + model.train() + for xb, yb in train_dl: + pred = model(xb) + loss = loss_func(pred, yb) + + loss.backward() + opt.step() + opt.zero_grad() + + model.eval() + with torch.no_grad(): + valid_loss = sum(loss_func(model(xb), yb) for xb, yb in valid_dl) + + print(epoch, valid_loss / len(valid_dl)) + +``` + +出： + +```py +0 tensor(0.3743) +1 tensor(0.3316) + +``` + +## 创建`fit()`和`get_data()` + +现在，我们将自己进行一些重构。 由于我们经历了两次相似的过程来计算训练集和验证集的损失，因此我们将其设为自己的函数`loss_batch`，该函数可计算一批损失。 + +我们将优化器传入训练集中，然后使用它执行反向传播。 对于验证集，我们没有通过优化程序，因此该方法不会执行反向传播。 + +```py +def loss_batch(model, loss_func, xb, yb, opt=None): + loss = loss_func(model(xb), yb) + + if opt is not None: + loss.backward() + opt.step() + opt.zero_grad() + + return loss.item(), len(xb) + +``` + +`fit`运行必要的操作来训练我们的模型，并计算每个周期的训练和验证损失。 + +```py +import numpy as np + +def fit(epochs, model, loss_func, opt, train_dl, valid_dl): + for epoch in range(epochs): + model.train() + for xb, yb in train_dl: + loss_batch(model, loss_func, xb, yb, opt) + + model.eval() + with torch.no_grad(): + losses, nums = zip( + *[loss_batch(model, loss_func, xb, yb) for xb, yb in valid_dl] + ) + val_loss = np.sum(np.multiply(losses, nums)) / np.sum(nums) + + print(epoch, val_loss) + +``` + +`get_data`返回训练和验证集的数据加载器。 + +```py +def get_data(train_ds, valid_ds, bs): + return ( + DataLoader(train_ds, batch_size=bs, shuffle=True), + DataLoader(valid_ds, batch_size=bs * 2), + ) + +``` + +现在，我们获取数据加载器和拟合模型的整个过程可以在 3 行代码中运行： + +```py +train_dl, valid_dl = get_data(train_ds, valid_ds, bs) +model, opt = get_model() +fit(epochs, model, loss_func, opt, train_dl, valid_dl) + +``` + +出： + +```py +0 0.3120644524335861 +1 0.28915613491535186 + +``` + +您可以使用这些基本的 3 行代码来训练各种各样的模型。 让我们看看是否可以使用它们来训练卷积神经网络（CNN）！ + +## 切换到 CNN + +现在，我们将构建具有三个卷积层的神经网络。 由于上一节中的任何功能都不假设任何有关模型形式的信息，因此我们将能够使用它们来训练 CNN，而无需进行任何修改。 + +我们将使用 Pytorch 的预定义[`Conv2d`](https://pytorch.org/docs/stable/nn.html#torch.nn.Conv2d)类作为我们的卷积层。 我们定义了具有 3 个卷积层的 CNN。 每个卷积后跟一个 ReLU。 最后，我们执行平均池化。 （请注意，`view`是 numpy 的`reshape`的 PyTorch 版本） + +```py +class Mnist_CNN(nn.Module): + def __init__(self): + super().__init__() + self.conv1 = nn.Conv2d(1, 16, kernel_size=3, stride=2, padding=1) + self.conv2 = nn.Conv2d(16, 16, kernel_size=3, stride=2, padding=1) + self.conv3 = nn.Conv2d(16, 10, kernel_size=3, stride=2, padding=1) + + def forward(self, xb): + xb = xb.view(-1, 1, 28, 28) + xb = F.relu(self.conv1(xb)) + xb = F.relu(self.conv2(xb)) + xb = F.relu(self.conv3(xb)) + xb = F.avg_pool2d(xb, 4) + return xb.view(-1, xb.size(1)) + +lr = 0.1 + +``` + +[动量](https://cs231n.github.io/neural-networks-3/#sgd)是随机梯度下降的一种变体，它也考虑了以前的更新，通常可以加快训练速度。 + +```py +model = Mnist_CNN() +opt = optim.SGD(model.parameters(), lr=lr, momentum=0.9) + +fit(epochs, model, loss_func, opt, train_dl, valid_dl) + +``` + +出： + +```py +0 0.32337012240886687 +1 0.25021172934770586 + +``` + +## `nn.Sequential` + +`torch.nn`还有另一个方便的类，可以用来简化我们的代码：[`Sequential`](https://pytorch.org/docs/stable/nn.html#torch.nn.Sequential)。 `Sequential`对象以顺序方式运行其中包含的每个模块。 这是编写神经网络的一种简单方法。 + +为了利用这一点，我们需要能够从给定的函数轻松定义**自定义层**。 例如，PyTorch 没有视层，我们需要为我们的网络创建一个层。 `Lambda`将创建一个层，然后在使用`Sequential`定义网络时可以使用该层。 + +```py +class Lambda(nn.Module): + def __init__(self, func): + super().__init__() + self.func = func + + def forward(self, x): + return self.func(x) + +def preprocess(x): + return x.view(-1, 1, 28, 28) + +``` + +用`Sequential`创建的模型很简单： + +```py +model = nn.Sequential( + Lambda(preprocess), + nn.Conv2d(1, 16, kernel_size=3, stride=2, padding=1), + nn.ReLU(), + nn.Conv2d(16, 16, kernel_size=3, stride=2, padding=1), + nn.ReLU(), + nn.Conv2d(16, 10, kernel_size=3, stride=2, padding=1), + nn.ReLU(), + nn.AvgPool2d(4), + Lambda(lambda x: x.view(x.size(0), -1)), +) + +opt = optim.SGD(model.parameters(), lr=lr, momentum=0.9) + +fit(epochs, model, loss_func, opt, train_dl, valid_dl) + +``` + +出： + +```py +0 0.30119081069231035 +1 0.25335356528759 + +``` + +## 包装`DataLoader` + +Our CNN is fairly concise, but it only works with MNIST, because: + +* 假设输入为`28 * 28`长向量 +* 假设 CNN 的最终网格尺寸为`4 * 4`（因为这是平均值 + +我们使用的合并核大小） + +让我们摆脱这两个假设，因此我们的模型适用于任何 2d 单通道图像。 首先，我们可以删除初始的 Lambda 层，但将数据预处理移至生成器中： + +```py +def preprocess(x, y): + return x.view(-1, 1, 28, 28), y + +class WrappedDataLoader: + def __init__(self, dl, func): + self.dl = dl + self.func = func + + def __len__(self): + return len(self.dl) + + def __iter__(self): + batches = iter(self.dl) + for b in batches: + yield (self.func(*b)) + +train_dl, valid_dl = get_data(train_ds, valid_ds, bs) +train_dl = WrappedDataLoader(train_dl, preprocess) +valid_dl = WrappedDataLoader(valid_dl, preprocess) + +``` + +接下来，我们可以将`nn.AvgPool2d`替换为`nn.AdaptiveAvgPool2d`，这使我们能够定义所需的*输出*张量的大小，而不是所需的*输入*张量的大小。 结果，我们的模型将适用于任何大小的输入。 + +```py +model = nn.Sequential( + nn.Conv2d(1, 16, kernel_size=3, stride=2, padding=1), + nn.ReLU(), + nn.Conv2d(16, 16, kernel_size=3, stride=2, padding=1), + nn.ReLU(), + nn.Conv2d(16, 10, kernel_size=3, stride=2, padding=1), + nn.ReLU(), + nn.AdaptiveAvgPool2d(1), + Lambda(lambda x: x.view(x.size(0), -1)), +) + +opt = optim.SGD(model.parameters(), lr=lr, momentum=0.9) + +``` + +试试看： + +```py +fit(epochs, model, loss_func, opt, train_dl, valid_dl) + +``` + +出： + +```py +0 0.327303307390213 +1 0.2181092014491558 + +``` + +## 使用您的 GPU + +如果您足够幸运地能够使用具有 CUDA 功能的 GPU（可以从大多数云提供商处以每小时 0.50 美元的价格租用一个），则可以使用它来加速代码。 首先检查您的 GPU 是否在 Pytorch 中正常工作： + +```py +print(torch.cuda.is_available()) + +``` + +出： + +```py +True + +``` + +然后为其创建一个设备对象： + +```py +dev = torch.device( + "cuda") if torch.cuda.is_available() else torch.device("cpu") + +``` + +让我们更新`preprocess`，将批量移至 GPU： + +```py +def preprocess(x, y): + return x.view(-1, 1, 28, 28).to(dev), y.to(dev) + +train_dl, valid_dl = get_data(train_ds, valid_ds, bs) +train_dl = WrappedDataLoader(train_dl, preprocess) +valid_dl = WrappedDataLoader(valid_dl, preprocess) + +``` + +最后，我们可以将模型移至 GPU。 + +```py +model.to(dev) +opt = optim.SGD(model.parameters(), lr=lr, momentum=0.9) + +``` + +您应该发现它现在运行得更快： + +```py +fit(epochs, model, loss_func, opt, train_dl, valid_dl) + +``` + +出： + +```py +0 0.1833980613708496 +1 0.17365939717292786 + +``` + +## 总结 + +现在，我们有了一个通用的数据管道和训练循环，您可以将其用于使用 Pytorch 训练许多类型的模型。 要了解现在可以轻松进行模型训练，请查看`mnist_sample`示例笔记本。 + +当然，您需要添加很多内容，例如数据扩充，超参数调整，监控训练，迁移学习等。 这些功能可在 fastai 库中使用，该库是使用本教程中所示的相同设计方法开发的，为希望进一步推广其模型的从业人员提供了自然的下一步。 + +我们承诺在本教程开始时将通过示例分别说明`torch.nn`，`torch.optim`，`Dataset`和`DataLoader`。 因此，让我们总结一下我们所看到的： + +> * `torch.nn` +> * `Module`：创建一个行为类似于函数的可调用对象，但也可以包含状态（例如神经网络层权重）。 它知道其中包含的 `Parameter` ，并且可以将其所有坡度归零，遍历它们以进行权重更新等。 +> * `Parameter`：张量的包装器，用于告知 `Module` 具有在反向传播期间需要更新的权重。 仅更新具有`require_grad`属性集的张量 +> * `functional`：一个模块（通常按照惯例导入到 `F` 名称空间中），其中包含激活函数，损失函数等。 以及卷积和线性层等层的无状态版本。 +> * `torch.optim`：包含诸如 `SGD` 的优化程序，这些优化程序在后退步骤 +> * `Dataset` 中更新 `Parameter` 的权重。 具有 `__len__` 和 `__getitem__` 的对象，包括 Pytorch 提供的类，例如 `TensorDataset` +> * `DataLoader`：获取任何 `Dataset` 并创建一个迭代器，该迭代器返回批量数据。 + +**脚本的总运行时间**：（0 分钟 57.062 秒） + +[下载 Python 源码：`nn_tutorial.py`](../_downloads/a6246751179fbfb7cad9222ef1c16617/nn_tutorial.py) + +[下载 Jupyter 笔记本：`nn_tutorial.ipynb`](../_downloads/5ddab57bb7482fbcc76722617dd47324/nn_tutorial.ipynb) [由 Sphinx 画廊](https://sphinx-gallery.readthedocs.io)生成的画廊 \ No newline at end of file diff --git a/pytorch/官方教程/09.md b/pytorch/官方教程/09.md index 709223d7..12d5eeb4 100644 --- a/pytorch/官方教程/09.md +++ b/pytorch/官方教程/09.md @@ -1,64 +1,348 @@ -# PyTorch：张量 +# 使用 TensorBoard 可视化模型，数据和训练 -> 原文： +> 原文： -经过训练的三阶多项式，可以通过最小化平方的欧几里得距离来预测`y = sin(x)`从`-pi`到`pi`。 +在 [60 分钟突击](https://pytorch.org/tutorials/beginner/deep_learning_60min_blitz.html)中，我们向您展示了如何加载数据，如何通过定义为`nn.Module`子类的模型提供数据，如何在训练数据上训练该模型以及在测试数据上对其进行测试。 为了了解发生的情况，我们在模型训练期间打印一些统计数据，以了解训练是否在进行中。 但是，我们可以做得更好：PyTorch 与 TensorBoard 集成在一起，TensorBoard 是一种工具，用于可视化神经网络训练运行的结果。 本教程使用 [Fashion-MNIST 数据集](https://github.com/zalandoresearch/fashion-mnist)说明了其某些功能，可以使用`torchvision.datasets`将其读入 PyTorch。 -此实现使用 PyTorch 张量手动计算正向传播，损失和后向通过。 +在本教程中，我们将学习如何： -PyTorch 张量基本上与 numpy 数组相同：它对深度学习或计算图或梯度一无所知，只是用于任意数值计算的通用 n 维数组。 +> 1. 读取数据并进行适当的转换（与先前的教程几乎相同）。 +> 2. 设置 TensorBoard。 +> 3. 写入 TensorBoard。 +> 4. 使用 TensorBoard 检查模型架构。 +> 5. 使用 TensorBoard 来创建我们在上一个教程中创建的可视化的交互式版本，并使用较少的代码 -numpy 数组和 PyTorch 张量之间的最大区别是 PyTorch 张量可以在 CPU 或 GPU 上运行。 要在 GPU 上运行操作，只需将张量转换为 cuda 数据类型。 +具体来说，在第 5 点，我们将看到： + +> * 有两种方法可以检查我们的训练数据 +> * 在训练模型时如何跟踪其表现 +> * 在训练后如何评估模型的表现。 + +我们将从 [CIFAR-10 教程](https://pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html)中类似的样板代码开始： ```py +# imports +import matplotlib.pyplot as plt +import numpy as np + import torch -import math +import torchvision +import torchvision.transforms as transforms -dtype = torch.float -device = torch.device("cpu") -# device = torch.device("cuda:0") # Uncomment this to run on GPU +import torch.nn as nn +import torch.nn.functional as F +import torch.optim as optim -# Create random input and output data -x = torch.linspace(-math.pi, math.pi, 2000, device=device, dtype=dtype) -y = torch.sin(x) +# transforms +transform = transforms.Compose( + [transforms.ToTensor(), + transforms.Normalize((0.5,), (0.5,))]) -# Randomly initialize weights -a = torch.randn((), device=device, dtype=dtype) -b = torch.randn((), device=device, dtype=dtype) -c = torch.randn((), device=device, dtype=dtype) -d = torch.randn((), device=device, dtype=dtype) +# datasets +trainset = torchvision.datasets.FashionMNIST('./data', + download=True, + train=True, + transform=transform) +testset = torchvision.datasets.FashionMNIST('./data', + download=True, + train=False, + transform=transform) -learning_rate = 1e-6 -for t in range(2000): - # Forward pass: compute predicted y - y_pred = a + b * x + c * x ** 2 + d * x ** 3 +# dataloaders +trainloader = torch.utils.data.DataLoader(trainset, batch_size=4, + shuffle=True, num_workers=2) - # Compute and print loss - loss = (y_pred - y).pow(2).sum().item() - if t % 100 == 99: - print(t, loss) +testloader = torch.utils.data.DataLoader(testset, batch_size=4, + shuffle=False, num_workers=2) - # Backprop to compute gradients of a, b, c, d with respect to loss - grad_y_pred = 2.0 * (y_pred - y) - grad_a = grad_y_pred.sum() - grad_b = (grad_y_pred * x).sum() - grad_c = (grad_y_pred * x ** 2).sum() - grad_d = (grad_y_pred * x ** 3).sum() +# constant for classes +classes = ('T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat', + 'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle Boot') - # Update weights using gradient descent - a -= learning_rate * grad_a - b -= learning_rate * grad_b - c -= learning_rate * grad_c - d -= learning_rate * grad_d - -print(f'Result: y = {a.item()} + {b.item()} x + {c.item()} x^2 + {d.item()} x^3') +# helper function to show an image +# (used in the `plot_classes_preds` function below) +def matplotlib_imshow(img, one_channel=False): + if one_channel: + img = img.mean(dim=0) + img = img / 2 + 0.5 # unnormalize + npimg = img.numpy() + if one_channel: + plt.imshow(npimg, cmap="Greys") + else: + plt.imshow(np.transpose(npimg, (1, 2, 0))) ``` -**脚本的总运行时间**：（0 分钟 0.000 秒） +我们将在该教程中定义一个类似的模型架构，仅需进行少量修改即可解决以下事实：图像现在是一个通道而不是三个通道，而图像是`28x28`而不是`32x32`： -[下载 Python 源码：`polynomial_tensor.py`](https://pytorch.org/tutorials/_downloads/38bc029908996abe0c601bcf0f5fd9d8/polynomial_tensor.py) +```py +class Net(nn.Module): + def __init__(self): + super(Net, self).__init__() + self.conv1 = nn.Conv2d(1, 6, 5) + self.pool = nn.MaxPool2d(2, 2) + self.conv2 = nn.Conv2d(6, 16, 5) + self.fc1 = nn.Linear(16 * 4 * 4, 120) + self.fc2 = nn.Linear(120, 84) + self.fc3 = nn.Linear(84, 10) -[下载 Jupyter 笔记本：`polynomial_tensor.ipynb`](https://pytorch.org/tutorials/_downloads/1c715a0888ae0e33279df327e1653329/polynomial_tensor.ipynb) + def forward(self, x): + x = self.pool(F.relu(self.conv1(x))) + x = self.pool(F.relu(self.conv2(x))) + x = x.view(-1, 16 * 4 * 4) + x = F.relu(self.fc1(x)) + x = F.relu(self.fc2(x)) + x = self.fc3(x) + return x -[由 Sphinx 画廊](https://sphinx-gallery.readthedocs.io)生成的画廊 \ No newline at end of file +net = Net() + +``` + +我们将在之前定义相同的`optimizer`和`criterion`： + +```py +criterion = nn.CrossEntropyLoss() +optimizer = optim.SGD(net.parameters(), lr=0.001, momentum=0.9) + +``` + +## 1\. TensorBoard 设置 + +现在，我们将设置 TensorBoard，从`torch.utils`导入`tensorboard`并定义`SummaryWriter`，这是将信息写入 TensorBoard 的关键对象。 + +```py +from torch.utils.tensorboard import SummaryWriter + +# default `log_dir` is "runs" - we'll be more specific here +writer = SummaryWriter('runs/fashion_mnist_experiment_1') + +``` + +请注意，仅此行会创建一个`runs/fashion_mnist_experiment_1`文件夹。 + +## 2\. 写入 TensorBoard + +现在，使用[`make_grid`](https://pytorch.org/docs/stable/torchvision/utils.html#torchvision.utils.make_grid)将图像写入到 TensorBoard 中，具体来说就是网格。 + +```py +# get some random training images +dataiter = iter(trainloader) +images, labels = dataiter.next() + +# create grid of images +img_grid = torchvision.utils.make_grid(images) + +# show images +matplotlib_imshow(img_grid, one_channel=True) + +# write to tensorboard +writer.add_image('four_fashion_mnist_images', img_grid) + +``` + +正在运行 + +```py +tensorboard --logdir=runs + +``` + +从命令行，然后导航到`https://localhost:6006`应该显示以下内容。 + +![intermediate/../../_static/img/tensorboard_first_view.png](img/8b09d6361316e495383ceedf9b8407ea.png) + +现在您知道如何使用 TensorBoard 了！ 但是，此示例可以在 Jupyter 笔记本中完成-TensorBoard 真正擅长的地方是创建交互式可视化。 接下来，我们将介绍其中之一，并在本教程结束时介绍更多内容。 + +## 3\. 使用 TensorBoard 检查模型 + +TensorBoard 的优势之一是其可视化复杂模型结构的能力。 让我们可视化我们构建的模型。 + +```py +writer.add_graph(net, images) +writer.close() + +``` + +现在刷新 TensorBoard 后，您应该会看到一个`Graphs`标签，如下所示： + +![intermediate/../../_static/img/tensorboard_model_viz.png](img/8f596b99dbb3c262b61db267d5db2d63.png) + +继续并双击`Net`以展开它，查看构成模型的各个操作的详细视图。 + +TensorBoard 具有非常方便的功能，可在低维空间中可视化高维数据，例如图像数据。 接下来我们将介绍这一点。 + +## 4\. 在 TensorBoard 中添加“投影仪” + +我们可以通过[`add_embedding`](https://pytorch.org/docs/stable/tensorboard.html#torch.utils.tensorboard.writer.SummaryWriter.add_embedding)方法可视化高维数据的低维表示 + +```py +# helper function +def select_n_random(data, labels, n=100): + ''' + Selects n random datapoints and their corresponding labels from a dataset + ''' + assert len(data) == len(labels) + + perm = torch.randperm(len(data)) + return data[perm][:n], labels[perm][:n] + +# select random images and their target indices +images, labels = select_n_random(trainset.data, trainset.targets) + +# get the class labels for each image +class_labels = [classes[lab] for lab in labels] + +# log embeddings +features = images.view(-1, 28 * 28) +writer.add_embedding(features, + metadata=class_labels, + label_img=images.unsqueeze(1)) +writer.close() + +``` + +现在，在 TensorBoard 的“投影仪”选项卡中，您可以看到这 100 张图像-每个图像 784 维-向下投影到三维空间中。 此外，这是交互式的：您可以单击并拖动以旋转三维投影。 最后，一些技巧可以使可视化效果更容易看到：选择左上方的“颜色：标签”，以及启用“夜间模式”，这将使图像更容易看到，因为它们的背景是白色的： + +![intermediate/../../_static/img/tensorboard_projector.png](img/f4990a0920dff7e4647a23cfc1639a8a.png) + +现在我们已经彻底检查了我们的数据，让我们展示了 TensorBoard 如何从训练开始就可以使跟踪模型的训练和评估更加清晰。 + +## 5\. 使用 TensorBoard 跟踪模型训练 + +在前面的示例中，我们仅*每 2000 次迭代*打印该模型的运行损失。 现在，我们将运行损失记录到 TensorBoard 中，并通过`plot_classes_preds`函数查看模型所做的预测。 + +```py +# helper functions + +def images_to_probs(net, images): + ''' + Generates predictions and corresponding probabilities from a trained + network and a list of images + ''' + output = net(images) + # convert output probabilities to predicted class + _, preds_tensor = torch.max(output, 1) + preds = np.squeeze(preds_tensor.numpy()) + return preds, [F.softmax(el, dim=0)[i].item() for i, el in zip(preds, output)] + +def plot_classes_preds(net, images, labels): + ''' + Generates matplotlib Figure using a trained network, along with images + and labels from a batch, that shows the network's top prediction along + with its probability, alongside the actual label, coloring this + information based on whether the prediction was correct or not. + Uses the "images_to_probs" function. + ''' + preds, probs = images_to_probs(net, images) + # plot the images in the batch, along with predicted and true labels + fig = plt.figure(figsize=(12, 48)) + for idx in np.arange(4): + ax = fig.add_subplot(1, 4, idx+1, xticks=[], yticks=[]) + matplotlib_imshow(images[idx], one_channel=True) + ax.set_title("{0}, {1:.1f}%\n(label: {2})".format( + classes[preds[idx]], + probs[idx] * 100.0, + classes[labels[idx]]), + color=("green" if preds[idx]==labels[idx].item() else "red")) + return fig + +``` + +最后，让我们使用与之前教程相同的模型训练代码来训练模型，但是每 1000 批将结果写入 TensorBoard，而不是打印到控制台。 这是通过[`add_scalar`](https://pytorch.org/docs/stable/tensorboard.html#torch.utils.tensorboard.writer.SummaryWriter.add_scalar)函数完成的。 + +此外，在训练过程中，我们将生成一幅图像，显示该批量中包含的四幅图像的模型预测与实际结果。 + +```py +running_loss = 0.0 +for epoch in range(1): # loop over the dataset multiple times + + for i, data in enumerate(trainloader, 0): + + # get the inputs; data is a list of [inputs, labels] + inputs, labels = data + + # zero the parameter gradients + optimizer.zero_grad() + + # forward + backward + optimize + outputs = net(inputs) + loss = criterion(outputs, labels) + loss.backward() + optimizer.step() + + running_loss += loss.item() + if i % 1000 == 999: # every 1000 mini-batches... + + # ...log the running loss + writer.add_scalar('training loss', + running_loss / 1000, + epoch * len(trainloader) + i) + + # ...log a Matplotlib Figure showing the model's predictions on a + # random mini-batch + writer.add_figure('predictions vs. actuals', + plot_classes_preds(net, inputs, labels), + global_step=epoch * len(trainloader) + i) + running_loss = 0.0 +print('Finished Training') + +``` + +现在，您可以查看“标量”选项卡，以查看在 15,000 次训练迭代中绘制的运行损失： + +![intermediate/../../_static/img/tensorboard_scalar_runs.png](img/afda8238ecd1f547d61be4d155844f68.png) + +此外，我们可以查看整个学习过程中模型在任意批量上所做的预测。 查看“图像”选项卡，然后在“预测与实际”可视化条件下向下滚动以查看此内容； 这表明，例如，仅经过 3000 次训练迭代，该模型就已经能够区分出视觉上截然不同的类，例如衬衫，运动鞋和外套，尽管它并没有像后来的训练那样有信心： + +![intermediate/../../_static/img/tensorboard_images.png](img/d5ab1f07cb4a9d9200c2a2d3b238340d.png) + +在之前的教程中，我们研究了模型训练后的每类准确率； 在这里，我们将使用 TensorBoard 绘制每个类别的精确调用曲线（[在这里解释](https://www.scikit-yb.org/en/latest/api/classifier/prcurve.html)）。 + +## 6\. 使用 TensorBoard 评估经过训练的模型 + +```py +# 1\. gets the probability predictions in a test_size x num_classes Tensor +# 2\. gets the preds in a test_size Tensor +# takes ~10 seconds to run +class_probs = [] +class_preds = [] +with torch.no_grad(): + for data in testloader: + images, labels = data + output = net(images) + class_probs_batch = [F.softmax(el, dim=0) for el in output] + _, class_preds_batch = torch.max(output, 1) + + class_probs.append(class_probs_batch) + class_preds.append(class_preds_batch) + +test_probs = torch.cat([torch.stack(batch) for batch in class_probs]) +test_preds = torch.cat(class_preds) + +# helper function +def add_pr_curve_tensorboard(class_index, test_probs, test_preds, global_step=0): + ''' + Takes in a "class_index" from 0 to 9 and plots the corresponding + precision-recall curve + ''' + tensorboard_preds = test_preds == class_index + tensorboard_probs = test_probs[:, class_index] + + writer.add_pr_curve(classes[class_index], + tensorboard_preds, + tensorboard_probs, + global_step=global_step) + writer.close() + +# plot all the pr curves +for i in range(len(classes)): + add_pr_curve_tensorboard(i, test_probs, test_preds) + +``` + +现在，您将看到一个`PR Curves`选项卡，其中包含每个类别的精确调用曲线。 继续四处戳； 您会发现在某些类别中，模型的“曲线下面积”接近 100%，而在另一些类别中，该面积更低： + +![intermediate/../../_static/img/tensorboard_pr_curves.png](img/d15de2be2b754f9a4f46418764232b5e.png) + +这是 TensorBoard 和 PyTorch 与之集成的介绍。 当然，您可以在 Jupyter 笔记本中完成 TensorBoard 的所有操作，但是使用 TensorBoard 时，默认情况下会获得交互式的视觉效果。 \ No newline at end of file diff --git a/pytorch/官方教程/10.md b/pytorch/官方教程/10.md deleted file mode 100644 index da742903..00000000 --- a/pytorch/官方教程/10.md +++ /dev/null @@ -1,77 +0,0 @@ -# PyTorch：张量和 Autograd - -> 原文： - -经过训练的三阶多项式，可以通过最小化平方的欧几里得距离来预测`y = sin(x)`从`-pi`到`pi`。 - -此实现使用 PyTorch 张量上的运算来计算正向传播，并使用 PyTorch Autograd 来计算梯度。 - -PyTorch 张量表示计算图中的一个节点。 如果`x`是具有`x.requires_grad=True`的张量，则`x.grad`是另一个张量，其保持`x`相对于某个标量值的梯度。 - -```py -import torch -import math - -dtype = torch.float -device = torch.device("cpu") -# device = torch.device("cuda:0") # Uncomment this to run on GPU - -# Create Tensors to hold input and outputs. -# By default, requires_grad=False, which indicates that we do not need to -# compute gradients with respect to these Tensors during the backward pass. -x = torch.linspace(-math.pi, math.pi, 2000, device=device, dtype=dtype) -y = torch.sin(x) - -# Create random Tensors for weights. For a third order polynomial, we need -# 4 weights: y = a + b x + c x^2 + d x^3 -# Setting requires_grad=True indicates that we want to compute gradients with -# respect to these Tensors during the backward pass. -a = torch.randn((), device=device, dtype=dtype, requires_grad=True) -b = torch.randn((), device=device, dtype=dtype, requires_grad=True) -c = torch.randn((), device=device, dtype=dtype, requires_grad=True) -d = torch.randn((), device=device, dtype=dtype, requires_grad=True) - -learning_rate = 1e-6 -for t in range(2000): - # Forward pass: compute predicted y using operations on Tensors. - y_pred = a + b * x + c * x ** 2 + d * x ** 3 - - # Compute and print loss using operations on Tensors. - # Now loss is a Tensor of shape (1,) - # loss.item() gets the scalar value held in the loss. - loss = (y_pred - y).pow(2).sum() - if t % 100 == 99: - print(t, loss.item()) - - # Use autograd to compute the backward pass. This call will compute the - # gradient of loss with respect to all Tensors with requires_grad=True. - # After this call a.grad, b.grad. c.grad and d.grad will be Tensors holding - # the gradient of the loss with respect to a, b, c, d respectively. - loss.backward() - - # Manually update weights using gradient descent. Wrap in torch.no_grad() - # because weights have requires_grad=True, but we don't need to track this - # in autograd. - with torch.no_grad(): - a -= learning_rate * a.grad - b -= learning_rate * b.grad - c -= learning_rate * c.grad - d -= learning_rate * d.grad - - # Manually zero the gradients after updating weights - a.grad = None - b.grad = None - c.grad = None - d.grad = None - -print(f'Result: y = {a.item()} + {b.item()} x + {c.item()} x^2 + {d.item()} x^3') - -``` - -**脚本的总运行时间**：（0 分钟 0.000 秒） - -[下载 Python 源码：`polynomial_autograd.py`](https://pytorch.org/tutorials/_downloads/2956e289de4f5fdd59114171805b23d2/polynomial_autograd.py) - -[下载 Jupyter 笔记本：`polynomial_autograd.ipynb`](https://pytorch.org/tutorials/_downloads/e1d4d0ca7bd75ea2fff8032fcb79076e/polynomial_autograd.ipynb) - -[由 Sphinx 画廊](https://sphinx-gallery.readthedocs.io)生成的画廊 \ No newline at end of file diff --git a/pytorch/官方教程/11.md b/pytorch/官方教程/11.md deleted file mode 100644 index a7cda6fa..00000000 --- a/pytorch/官方教程/11.md +++ /dev/null @@ -1,103 +0,0 @@ -# PyTorch：定义新的 Autograd 函数 - -> 原文： - -经过训练的三阶多项式，可以通过最小化平方的欧几里得距离来预测`y = sin(x)`从`-pi`到`pi`。 而不是将多项式写为`y = a + bx + cx ^ 2 + dx ^ 3`，我们将多项式写为`y = a + b P[3](c + dx)`其中`P[3](x) = 1/2 (5x ^ 3 - 3x)`是三次的[勒让德多项式](https://en.wikipedia.org/wiki/Legendre_polynomials)。 - -此实现使用 PyTorch 张量上的运算来计算正向传播，并使用 PyTorch Autograd 来计算梯度。 - -在此实现中，我们实现了自己的自定义 Autograd 函数来执行`P'[3](x)`。 通过数学，`P'[3](x) = 3/2 (5x ^ 2 - 1)`： - -```py -import torch -import math - -class LegendrePolynomial3(torch.autograd.Function): - """ - We can implement our own custom autograd Functions by subclassing - torch.autograd.Function and implementing the forward and backward passes - which operate on Tensors. - """ - - @staticmethod - def forward(ctx, input): - """ - In the forward pass we receive a Tensor containing the input and return - a Tensor containing the output. ctx is a context object that can be used - to stash information for backward computation. You can cache arbitrary - objects for use in the backward pass using the ctx.save_for_backward method. - """ - ctx.save_for_backward(input) - return 0.5 * (5 * input ** 3 - 3 * input) - - @staticmethod - def backward(ctx, grad_output): - """ - In the backward pass we receive a Tensor containing the gradient of the loss - with respect to the output, and we need to compute the gradient of the loss - with respect to the input. - """ - input, = ctx.saved_tensors - return grad_output * 1.5 * (5 * input ** 2 - 1) - -dtype = torch.float -device = torch.device("cpu") -# device = torch.device("cuda:0") # Uncomment this to run on GPU - -# Create Tensors to hold input and outputs. -# By default, requires_grad=False, which indicates that we do not need to -# compute gradients with respect to these Tensors during the backward pass. -x = torch.linspace(-math.pi, math.pi, 2000, device=device, dtype=dtype) -y = torch.sin(x) - -# Create random Tensors for weights. For this example, we need -# 4 weights: y = a + b * P3(c + d * x), these weights need to be initialized -# not too far from the correct result to ensure convergence. -# Setting requires_grad=True indicates that we want to compute gradients with -# respect to these Tensors during the backward pass. -a = torch.full((), 0.0, device=device, dtype=dtype, requires_grad=True) -b = torch.full((), -1.0, device=device, dtype=dtype, requires_grad=True) -c = torch.full((), 0.0, device=device, dtype=dtype, requires_grad=True) -d = torch.full((), 0.3, device=device, dtype=dtype, requires_grad=True) - -learning_rate = 5e-6 -for t in range(2000): - # To apply our Function, we use Function.apply method. We alias this as 'P3'. - P3 = LegendrePolynomial3.apply - - # Forward pass: compute predicted y using operations; we compute - # P3 using our custom autograd operation. - y_pred = a + b * P3(c + d * x) - - # Compute and print loss - loss = (y_pred - y).pow(2).sum() - if t % 100 == 99: - print(t, loss.item()) - - # Use autograd to compute the backward pass. - loss.backward() - - # Update weights using gradient descent - with torch.no_grad(): - a -= learning_rate * a.grad - b -= learning_rate * b.grad - c -= learning_rate * c.grad - d -= learning_rate * d.grad - - # Manually zero the gradients after updating weights - a.grad = None - b.grad = None - c.grad = None - d.grad = None - -print(f'Result: y = {a.item()} + {b.item()} * P3({c.item()} + {d.item()} x)') - -``` - -**脚本的总运行时间**：（0 分钟 0.000 秒） - -[下载 Python 源码：`polynomial_custom_function.py`](https://pytorch.org/tutorials/_downloads/b7ec15fd7bec1ca3f921104cfb6a54ed/polynomial_custom_function.py) - -[下载 Jupyter 笔记本：`polynomial_custom_function.ipynb`](https://pytorch.org/tutorials/_downloads/0a64809624bf2f3eb497d30d5303a9a0/polynomial_custom_function.ipynb) - -[由 Sphinx 画廊](https://sphinx-gallery.readthedocs.io)生成的画廊 \ No newline at end of file diff --git a/pytorch/官方教程/12.md b/pytorch/官方教程/12.md deleted file mode 100644 index 9dc0b70d..00000000 --- a/pytorch/官方教程/12.md +++ /dev/null @@ -1,87 +0,0 @@ -# PyTorch：`nn` - -> 原文： - -经过训练的三阶多项式，可以通过最小化平方的欧几里得距离来预测`y = sin(x)`从`-pi`到`pi`。 - -此实现使用来自 PyTorch 的`nn`包来构建网络。 PyTorch Autograd 使定义计算图和获取梯度变得容易，但是原始的 Autograd 对于定义复杂的神经网络来说可能太低了。 这是`nn`包可以提供帮助的地方。 `nn`包定义了一组模块，您可以将其视为神经网络层，该神经网络层从输入产生输出并且可能具有一些可训练的权重。 - -```py -import torch -import math - -# Create Tensors to hold input and outputs. -x = torch.linspace(-math.pi, math.pi, 2000) -y = torch.sin(x) - -# For this example, the output y is a linear function of (x, x^2, x^3), so -# we can consider it as a linear layer neural network. Let's prepare the -# tensor (x, x^2, x^3). -p = torch.tensor([1, 2, 3]) -xx = x.unsqueeze(-1).pow(p) - -# In the above code, x.unsqueeze(-1) has shape (2000, 1), and p has shape -# (3,), for this case, broadcasting semantics will apply to obtain a tensor -# of shape (2000, 3) - -# Use the nn package to define our model as a sequence of layers. nn.Sequential -# is a Module which contains other Modules, and applies them in sequence to -# produce its output. The Linear Module computes output from input using a -# linear function, and holds internal Tensors for its weight and bias. -# The Flatten layer flatens the output of the linear layer to a 1D tensor, -# to match the shape of `y`. -model = torch.nn.Sequential( - torch.nn.Linear(3, 1), - torch.nn.Flatten(0, 1) -) - -# The nn package also contains definitions of popular loss functions; in this -# case we will use Mean Squared Error (MSE) as our loss function. -loss_fn = torch.nn.MSELoss(reduction='sum') - -learning_rate = 1e-6 -for t in range(2000): - - # Forward pass: compute predicted y by passing x to the model. Module objects - # override the __call__ operator so you can call them like functions. When - # doing so you pass a Tensor of input data to the Module and it produces - # a Tensor of output data. - y_pred = model(xx) - - # Compute and print loss. We pass Tensors containing the predicted and true - # values of y, and the loss function returns a Tensor containing the - # loss. - loss = loss_fn(y_pred, y) - if t % 100 == 99: - print(t, loss.item()) - - # Zero the gradients before running the backward pass. - model.zero_grad() - - # Backward pass: compute gradient of the loss with respect to all the learnable - # parameters of the model. Internally, the parameters of each Module are stored - # in Tensors with requires_grad=True, so this call will compute gradients for - # all learnable parameters in the model. - loss.backward() - - # Update the weights using gradient descent. Each parameter is a Tensor, so - # we can access its gradients like we did before. - with torch.no_grad(): - for param in model.parameters(): - param -= learning_rate * param.grad - -# You can access the first layer of `model` like accessing the first item of a list -linear_layer = model[0] - -# For linear layer, its parameters are stored as `weight` and `bias`. -print(f'Result: y = {linear_layer.bias.item()} + {linear_layer.weight[:, 0].item()} x + {linear_layer.weight[:, 1].item()} x^2 + {linear_layer.weight[:, 2].item()} x^3') - -``` - -**脚本的总运行时间**：（0 分钟 0.000 秒） - -[下载 Python 源码：`polynomial_nn.py`](https://pytorch.org/tutorials/_downloads/b4767df4367deade63dc8a0d3712c1d4/polynomial_nn.py) - -[下载 Jupyter 笔记本：`polynomial_nn.ipynb`](https://pytorch.org/tutorials/_downloads/7bc167d8b8308ae65a717d7461d838fa/polynomial_nn.ipynb) - -[由 Sphinx 画廊](https://sphinx-gallery.readthedocs.io)生成的画廊 \ No newline at end of file diff --git a/pytorch/官方教程/13.md b/pytorch/官方教程/13.md deleted file mode 100644 index ac2d8a92..00000000 --- a/pytorch/官方教程/13.md +++ /dev/null @@ -1,71 +0,0 @@ -# PyTorch：`optim` - -> 原文： - -经过训练的三阶多项式，可以通过最小化平方的欧几里得距离来预测`y = sin(x)`从`-pi`到`pi`。 - -此实现使用来自 PyTorch 的`nn`包来构建网络。 - -与其像以前那样手动更新模型的权重，不如使用`optim`包定义一个优化器，该优化器将为我们更新权重。 `optim`包定义了许多深度学习常用的优化算法，包括 SGD + 动量，RMSProp，Adam 等。 - -```py -import torch -import math - -# Create Tensors to hold input and outputs. -x = torch.linspace(-math.pi, math.pi, 2000) -y = torch.sin(x) - -# Prepare the input tensor (x, x^2, x^3). -p = torch.tensor([1, 2, 3]) -xx = x.unsqueeze(-1).pow(p) - -# Use the nn package to define our model and loss function. -model = torch.nn.Sequential( - torch.nn.Linear(3, 1), - torch.nn.Flatten(0, 1) -) -loss_fn = torch.nn.MSELoss(reduction='sum') - -# Use the optim package to define an Optimizer that will update the weights of -# the model for us. Here we will use RMSprop; the optim package contains many other -# optimization algorithms. The first argument to the RMSprop constructor tells the -# optimizer which Tensors it should update. -learning_rate = 1e-3 -optimizer = torch.optim.RMSprop(model.parameters(), lr=learning_rate) -for t in range(2000): - # Forward pass: compute predicted y by passing x to the model. - y_pred = model(xx) - - # Compute and print loss. - loss = loss_fn(y_pred, y) - if t % 100 == 99: - print(t, loss.item()) - - # Before the backward pass, use the optimizer object to zero all of the - # gradients for the variables it will update (which are the learnable - # weights of the model). This is because by default, gradients are - # accumulated in buffers( i.e, not overwritten) whenever .backward() - # is called. Checkout docs of torch.autograd.backward for more details. - optimizer.zero_grad() - - # Backward pass: compute gradient of the loss with respect to model - # parameters - loss.backward() - - # Calling the step function on an Optimizer makes an update to its - # parameters - optimizer.step() - -linear_layer = model[0] -print(f'Result: y = {linear_layer.bias.item()} + {linear_layer.weight[:, 0].item()} x + {linear_layer.weight[:, 1].item()} x^2 + {linear_layer.weight[:, 2].item()} x^3') - -``` - -**脚本的总运行时间**：（0 分钟 0.000 秒） - -[下载 Python 源码：`polynomial_optim.py`](https://pytorch.org/tutorials/_downloads/bcfec6f02e0fe747a42dbd1579267469/polynomial_optim.py) - -[下载 Jupyter 笔记本：`polynomial_optim.ipynb`](https://pytorch.org/tutorials/_downloads/8ef669b2c61c6c5aa47c54dceac4979e/polynomial_optim.ipynb) - -[由 Sphinx 画廊](https://sphinx-gallery.readthedocs.io)生成的画廊 \ No newline at end of file diff --git a/pytorch/官方教程/14.md b/pytorch/官方教程/14.md deleted file mode 100644 index 619ffac4..00000000 --- a/pytorch/官方教程/14.md +++ /dev/null @@ -1,75 +0,0 @@ -# PyTorch：自定义`nn`模块 - -> 原文： - -经过训练的三阶多项式，可以通过最小化平方的欧几里得距离来预测`y = sin(x)`从`-pi`到`pi`。 - -此实现将模型定义为自定义`Module`子类。 每当您想要一个比现有模块的简单序列更复杂的模型时，都需要以这种方式定义模型。 - -```py -import torch -import math - -class Polynomial3(torch.nn.Module): - def __init__(self): - """ - In the constructor we instantiate four parameters and assign them as - member parameters. - """ - super().__init__() - self.a = torch.nn.Parameter(torch.randn(())) - self.b = torch.nn.Parameter(torch.randn(())) - self.c = torch.nn.Parameter(torch.randn(())) - self.d = torch.nn.Parameter(torch.randn(())) - - def forward(self, x): - """ - In the forward function we accept a Tensor of input data and we must return - a Tensor of output data. We can use Modules defined in the constructor as - well as arbitrary operators on Tensors. - """ - return self.a + self.b * x + self.c * x ** 2 + self.d * x ** 3 - - def string(self): - """ - Just like any class in Python, you can also define custom method on PyTorch modules - """ - return f'y = {self.a.item()} + {self.b.item()} x + {self.c.item()} x^2 + {self.d.item()} x^3' - -# Create Tensors to hold input and outputs. -x = torch.linspace(-math.pi, math.pi, 2000) -y = torch.sin(x) - -# Construct our model by instantiating the class defined above -model = Polynomial3() - -# Construct our loss function and an Optimizer. The call to model.parameters() -# in the SGD constructor will contain the learnable parameters of the nn.Linear -# module which is members of the model. -criterion = torch.nn.MSELoss(reduction='sum') -optimizer = torch.optim.SGD(model.parameters(), lr=1e-6) -for t in range(2000): - # Forward pass: Compute predicted y by passing x to the model - y_pred = model(x) - - # Compute and print loss - loss = criterion(y_pred, y) - if t % 100 == 99: - print(t, loss.item()) - - # Zero gradients, perform a backward pass, and update the weights. - optimizer.zero_grad() - loss.backward() - optimizer.step() - -print(f'Result: {model.string()}') - -``` - -**脚本的总运行时间**：（0 分钟 0.000 秒） - -[下载 Python 源码：`polynomial_module.py`](https://pytorch.org/tutorials/_downloads/916a9c460c899330dbc53216cc775358/polynomial_module.py) - -[下载 Jupyter 笔记本：`polynomial_module.ipynb`](https://pytorch.org/tutorials/_downloads/19f4ecdd2763dd4b90693df4d6e10ebe/polynomial_module.ipynb) - -[由 Sphinx 画廊](https://sphinx-gallery.readthedocs.io)生成的画廊 \ No newline at end of file diff --git a/pytorch/官方教程/15.md b/pytorch/官方教程/15.md deleted file mode 100644 index 50fb29a1..00000000 --- a/pytorch/官方教程/15.md +++ /dev/null @@ -1,82 +0,0 @@ -# PyTorch：控制流 + 权重共享 - -> 原文： - -为了展示 PyTorch 动态图的强大功能，我们将实现一个非常奇怪的模型：一个三阶多项式，在每个正向传播中选择 3 到 5 之间的一个随机数，并使用该数量的阶次，多次使用相同的权重重复计算四和五阶。 - -```py -import random -import torch -import math - -class DynamicNet(torch.nn.Module): - def __init__(self): - """ - In the constructor we instantiate five parameters and assign them as members. - """ - super().__init__() - self.a = torch.nn.Parameter(torch.randn(())) - self.b = torch.nn.Parameter(torch.randn(())) - self.c = torch.nn.Parameter(torch.randn(())) - self.d = torch.nn.Parameter(torch.randn(())) - self.e = torch.nn.Parameter(torch.randn(())) - - def forward(self, x): - """ - For the forward pass of the model, we randomly choose either 4, 5 - and reuse the e parameter to compute the contribution of these orders. - - Since each forward pass builds a dynamic computation graph, we can use normal - Python control-flow operators like loops or conditional statements when - defining the forward pass of the model. - - Here we also see that it is perfectly safe to reuse the same parameter many - times when defining a computational graph. - """ - y = self.a + self.b * x + self.c * x ** 2 + self.d * x ** 3 - for exp in range(4, random.randint(4, 6)): - y = y + self.e * x ** exp - return y - - def string(self): - """ - Just like any class in Python, you can also define custom method on PyTorch modules - """ - return f'y = {self.a.item()} + {self.b.item()} x + {self.c.item()} x^2 + {self.d.item()} x^3 + {self.e.item()} x^4 ? + {self.e.item()} x^5 ?' - -# Create Tensors to hold input and outputs. -x = torch.linspace(-math.pi, math.pi, 2000) -y = torch.sin(x) - -# Construct our model by instantiating the class defined above -model = DynamicNet() - -# Construct our loss function and an Optimizer. Training this strange model with -# vanilla stochastic gradient descent is tough, so we use momentum -criterion = torch.nn.MSELoss(reduction='sum') -optimizer = torch.optim.SGD(model.parameters(), lr=1e-8, momentum=0.9) -for t in range(30000): - # Forward pass: Compute predicted y by passing x to the model - y_pred = model(x) - - # Compute and print loss - loss = criterion(y_pred, y) - if t % 2000 == 1999: - print(t, loss.item()) - - # Zero gradients, perform a backward pass, and update the weights. - optimizer.zero_grad() - loss.backward() - optimizer.step() - -print(f'Result: {model.string()}') - -``` - -**脚本的总运行时间**：（0 分钟 0.000 秒） - -[下载 Python 源码：`dynamic_net.py`](https://pytorch.org/tutorials/_downloads/3900c903cde097dc0088c3b06d588c0b/dynamic_net.py) - -[下载 Jupyter 笔记本：`dynamic_net.ipynb`](https://pytorch.org/tutorials/_downloads/ad230923bd9eb0d42576725b63ad8d91/dynamic_net.ipynb) - -[由 Sphinx 画廊](https://sphinx-gallery.readthedocs.io)生成的画廊 \ No newline at end of file diff --git a/pytorch/官方教程/16.md b/pytorch/官方教程/16.md deleted file mode 100644 index e6867d85..00000000 --- a/pytorch/官方教程/16.md +++ /dev/null @@ -1,971 +0,0 @@ -# `torch.nn`到底是什么？ - -> 原文： - -作者：Jeremy Howard，[fast.ai](https://www.fast.ai)。 感谢 Rachel Thomas 和 Francisco Ingham。 - -我们建议将本教程作为笔记本而不是脚本来运行。 要下载笔记本（`.ipynb`）文件，请单击页面顶部的链接。 - -PyTorch 提供设计精美的模块和类[`torch.nn`](https://pytorch.org/docs/stable/nn.html)，[`torch.optim`](https://pytorch.org/docs/stable/optim.html)，[`Dataset`](https://pytorch.org/docs/stable/data.html?highlight=dataset#torch.utils.data.Dataset)和[`DataLoader`](https://pytorch.org/docs/stable/data.html?highlight=dataloader#torch.utils.data.DataLoader)神经网络。 为了充分利用它们的功能并针对您的问题对其进行自定义，您需要真正了解它们在做什么。 为了建立这种理解，我们将首先在 MNIST 数据集上训练基本神经网络，而无需使用这些模型的任何功能。 我们最初将仅使用最基本的 PyTorch 张量函数。 然后，我们将一次从`torch.nn`，`torch.optim`，`Dataset`或`DataLoader`中逐个添加一个函数，以准确显示每个函数，以及如何使代码更简洁或更有效。 灵活。 - -**本教程假定您已经安装了 PyTorch，并且熟悉张量操作的基础知识。** （如果您熟悉 Numpy 数组操作，将会发现此处使用的 PyTorch 张量操作几乎相同）。 - -## MNIST 数据集 - -我们将使用经典的 [MNIST](http://deeplearning.net/data/mnist/) 数据集，该数据集由手绘数字的黑白图像组成（0 到 9 之间）。 - -我们将使用[`pathlib`](https://docs.python.org/3/library/pathlib.html)处理路径（Python 3 标准库的一部分），并使用[`requests`](http://docs.python-requests.org/en/master/)下载数据集。 我们只会在使用模块时才导入它们，因此您可以确切地看到每个位置上正在使用的模块。 - -```py -from pathlib import Path -import requests - -DATA_PATH = Path("data") -PATH = DATA_PATH / "mnist" - -PATH.mkdir(parents=True, exist_ok=True) - -URL = "https://github.com/pytorch/tutorials/raw/master/_static/" -FILENAME = "mnist.pkl.gz" - -if not (PATH / FILENAME).exists(): - content = requests.get(URL + FILENAME).content - (PATH / FILENAME).open("wb").write(content) - -``` - -该数据集为 numpy 数组格式，并已使用`pickle`（一种用于序列化数据的 python 特定格式）存储。 - -```py -import pickle -import gzip - -with gzip.open((PATH / FILENAME).as_posix(), "rb") as f: - ((x_train, y_train), (x_valid, y_valid), _) = pickle.load(f, encoding="latin-1") - -``` - -每个图像为`28 x 28`，并存储为长度为`784 = 28x28`的扁平行。 让我们来看一个； 我们需要先将其重塑为 2d。 - -```py -from matplotlib import pyplot -import numpy as np - -pyplot.imshow(x_train[0].reshape((28, 28)), cmap="gray") -print(x_train.shape) - -``` - -![../_img/sphx_glr_nn_tutorial_001.png](img/7c783def0bbe536f41ed172041b7e89e.png) - -出： - -```py -(50000, 784) - -``` - -PyTorch 使用`torch.tensor`而不是 numpy 数组，因此我们需要转换数据。 - -```py -import torch - -x_train, y_train, x_valid, y_valid = map( - torch.tensor, (x_train, y_train, x_valid, y_valid) -) -n, c = x_train.shape -x_train, x_train.shape, y_train.min(), y_train.max() -print(x_train, y_train) -print(x_train.shape) -print(y_train.min(), y_train.max()) - -``` - -出： - -```py -tensor([[0., 0., 0., ..., 0., 0., 0.], - [0., 0., 0., ..., 0., 0., 0.], - [0., 0., 0., ..., 0., 0., 0.], - ..., - [0., 0., 0., ..., 0., 0., 0.], - [0., 0., 0., ..., 0., 0., 0.], - [0., 0., 0., ..., 0., 0., 0.]]) tensor([5, 0, 4, ..., 8, 4, 8]) -torch.Size([50000, 784]) -tensor(0) tensor(9) - -``` - -## 从零开始的神经网络（没有`torch.nn`） - -首先，我们仅使用 PyTorch 张量操作创建模型。 我们假设您已经熟悉神经网络的基础知识。 （如果不是，则可以在 [course.fast.ai](https://course.fast.ai) 中学习它们）。 - -PyTorch 提供了创建随机或零填充张量的方法，我们将使用它们来为简单的线性模型创建权重和偏差。 这些只是常规张量，还有一个非常特殊的附加值：我们告诉 PyTorch 它们需要梯度。 这使 PyTorch 记录了在张量上完成的所有操作，因此它可以在反向传播时*自动计算*的梯度！ - -**对于权重，我们在初始化之后设置`requires_grad`，因为我们不希望该步骤包含在梯度中。 （请注意，PyTorch 中的尾随`_`表示该操作是原地执行的。）** - -注意 - -我们在这里用 [Xavier 初始化](http://proceedings.mlr.press/v9/glorot10a/glorot10a.pdf)（通过乘以`1 / sqrt(n)`）来初始化权重。 - -```py -import math - -weights = torch.randn(784, 10) / math.sqrt(784) -weights.requires_grad_() -bias = torch.zeros(10, requires_grad=True) - -``` - -由于 PyTorch 具有自动计算梯度的功能，我们可以将任何标准的 Python 函数（或可调用对象）用作模型！ 因此，我们只需编写一个普通矩阵乘法和广播加法即可创建一个简单的线性模型。 我们还需要激活函数，因此我们将编写并使用`log_softmax`。 请记住：尽管 PyTorch 提供了许多预写的损失函数，激活函数等，但是您可以使用纯 Python 轻松编写自己的函数。 PyTorch 甚至会自动为您的函数创建快速 GPU 或向量化的 CPU 代码。 - -```py -def log_softmax(x): - return x - x.exp().sum(-1).log().unsqueeze(-1) - -def model(xb): - return log_softmax(xb @ weights + bias) - -``` - -在上面，`@`代表点积运算。 我们将对一批数据（在本例中为 64 张图像）调用函数。 这是一个*正向传播*。 请注意，由于我们从随机权重开始，因此在这一阶段，我们的预测不会比随机预测更好。 - -```py -bs = 64 # batch size - -xb = x_train[0:bs] # a mini-batch from x -preds = model(xb) # predictions -preds[0], preds.shape -print(preds[0], preds.shape) - -``` - -出： - -```py -tensor([-2.5964, -2.3153, -2.1321, -2.4480, -2.2930, -1.9507, -2.1289, -2.4175, - -2.5332, -2.3967], grad_fn=) torch.Size([64, 10]) - -``` - -如您所见，`preds`张量不仅包含张量值，还包含梯度函数。 稍后我们将使用它进行反向传播。 - -让我们实现负对数可能性作为损失函数（同样，我们只能使用标准 Python）： - -```py -def nll(input, target): - return -input[range(target.shape[0]), target].mean() - -loss_func = nll - -``` - -让我们使用随机模型来检查损失，以便我们稍后查看反向传播后是否可以改善我们的损失。 - -```py -yb = y_train[0:bs] -print(loss_func(preds, yb)) - -``` - -出： - -```py -tensor(2.3735, grad_fn=) - -``` - -我们还实现一个函数来计算模型的准确率。 对于每个预测，如果具有最大值的索引与目标值匹配，则该预测是正确的。 - -```py -def accuracy(out, yb): - preds = torch.argmax(out, dim=1) - return (preds == yb).float().mean() - -``` - -让我们检查一下随机模型的准确率，以便我们可以看出随着损失的增加，准确率是否有所提高。 - -```py -print(accuracy(preds, yb)) - -``` - -出： - -```py -tensor(0.0938) - -``` - -现在，我们可以运行一个训练循环。 对于每次迭代，我们将： - -* 选择一个小批量数据（大小为`bs`） -* 使用模型进行预测 -* 计算损失 -* `loss.backward()`更新模型的梯度，在这种情况下为`weights`和`bias`。 - -现在，我们使用这些梯度来更新权重和偏差。 我们在`torch.no_grad()`上下文管理器中执行此操作，因为我们不希望在下一步的梯度计算中记录这些操作。 [您可以在这里阅读有关 PyTorch 的 Autograd 如何记录操作的更多信息](https://pytorch.org/docs/stable/notes/autograd.html)。 - -然后，将梯度设置为零，以便为下一个循环做好准备。 否则，我们的梯度会记录所有已发生操作的运行记录（即`loss.backward()`将梯度添加到已存储的内容中，而不是替换它们）。 - -小费 - -您可以使用标准的 python 调试器逐步浏览 PyTorch 代码，从而可以在每一步检查各种变量值。 取消注释以下`set_trace()`即可尝试。 - -```py -from IPython.core.debugger import set_trace - -lr = 0.5 # learning rate -epochs = 2 # how many epochs to train for - -for epoch in range(epochs): - for i in range((n - 1) // bs + 1): - # set_trace() - start_i = i * bs - end_i = start_i + bs - xb = x_train[start_i:end_i] - yb = y_train[start_i:end_i] - pred = model(xb) - loss = loss_func(pred, yb) - - loss.backward() - with torch.no_grad(): - weights -= weights.grad * lr - bias -= bias.grad * lr - weights.grad.zero_() - bias.grad.zero_() - -``` - -就是这样：我们完全从头开始创建并训练了一个最小的神经网络（在这种情况下，是逻辑回归，因为我们没有隐藏的层）！ - -让我们检查损失和准确率，并将其与我们之前获得的进行比较。 我们希望损失会减少，准确率会增加，而且确实如此。 - -```py -print(loss_func(model(xb), yb), accuracy(model(xb), yb)) - -``` - -出： - -```py -tensor(0.0811, grad_fn=) tensor(1.) - -``` - -## 使用`torch.nn.functional` - -现在，我们将重构代码，使其执行与以前相同的操作，只是我们将开始利用 PyTorch 的`nn`类使其更加简洁和灵活。 从这里开始的每一步，我们都应该使代码中的一个或多个：更短，更易理解和/或更灵活。 - -第一步也是最简单的步骤，就是用`torch.nn.functional`（通常按照惯例将其导入到名称空间`F`中）替换我们的手写激活和损失函数，从而缩短代码长度。 该模块包含`torch.nn`库中的所有函数（而该库的其他部分包含类）。 除了广泛的损失和激活函数外，您还会在这里找到一些方便的函数来创建神经网络，例如合并函数。 （还有一些用于进行卷积，线性层等的函数，但是正如我们将看到的那样，通常可以使用库的其他部分来更好地处理这些函数。） - -如果您使用的是负对数似然损失和对数 softmax 激活，那么 Pytorch 会提供结合了两者的单一函数`F.cross_entropy`。 因此，我们甚至可以从模型中删除激活函数。 - -```py -import torch.nn.functional as F - -loss_func = F.cross_entropy - -def model(xb): - return xb @ weights + bias - -``` - -请注意，我们不再在`model`函数中调用`log_softmax`。 让我们确认我们的损失和准确率与以前相同： - -```py -print(loss_func(model(xb), yb), accuracy(model(xb), yb)) - -``` - -出： - -```py -tensor(0.0811, grad_fn=) tensor(1.) - -``` - -## 使用`nn.Module`重构 - -接下来，我们将使用`nn.Module`和`nn.Parameter`进行更清晰，更简洁的训练循环。 我们将`nn.Module`子类化（它本身是一个类并且能够跟踪状态）。 在这种情况下，我们要创建一个类，该类包含前进步骤的权重，偏置和方法。 `nn.Module`具有许多我们将要使用的属性和方法（例如`.parameters()`和`.zero_grad()`）。 - -注意 - -`nn.Module`（大写`M`）是 PyTorch 的特定概念，并且是我们将经常使用的一类。 不要将`nn.Module`与[模块](https://docs.python.org/3/tutorial/modules.html)（小写`m`）的 Python 概念混淆，该模块是可以导入的 Python 代码文件。 - -```py -from torch import nn - -class Mnist_Logistic(nn.Module): - def __init__(self): - super().__init__() - self.weights = nn.Parameter(torch.randn(784, 10) / math.sqrt(784)) - self.bias = nn.Parameter(torch.zeros(10)) - - def forward(self, xb): - return xb @ self.weights + self.bias - -``` - -由于我们现在使用的是对象而不是仅使用函数，因此我们首先必须实例化模型： - -```py -model = Mnist_Logistic() - -``` - -现在我们可以像以前一样计算损失。 请注意，`nn.Module`对象的使用就好像它们是函数一样（即，它们是*可调用的*），但是在后台 Pytorch 会自动调用我们的`forward`方法。 - -```py -print(loss_func(model(xb), yb)) - -``` - -出： - -```py -tensor(2.3903, grad_fn=) - -``` - -以前，在我们的训练循环中，我们必须按名称更新每个参数的值，并手动将每个参数的梯度分别归零，如下所示： - -```py -with torch.no_grad(): - weights -= weights.grad * lr - bias -= bias.grad * lr - weights.grad.zero_() - bias.grad.zero_() - -``` - -现在我们可以利用`model.parameters()`和`model.zero_grad()`（它们都由 PyTorch 为`nn.Module`定义）来使这些步骤更简洁，并且更不会出现忘记某些参数的错误，尤其是当我们有一个更复杂的模型的时候： - -```py -with torch.no_grad(): - for p in model.parameters(): p -= p.grad * lr - model.zero_grad() - -``` - -我们将把小的训练循环包装在`fit`函数中，以便稍后再运行。 - -```py -def fit(): - for epoch in range(epochs): - for i in range((n - 1) // bs + 1): - start_i = i * bs - end_i = start_i + bs - xb = x_train[start_i:end_i] - yb = y_train[start_i:end_i] - pred = model(xb) - loss = loss_func(pred, yb) - - loss.backward() - with torch.no_grad(): - for p in model.parameters(): - p -= p.grad * lr - model.zero_grad() - -fit() - -``` - -让我们仔细检查一下我们的损失是否减少了： - -```py -print(loss_func(model(xb), yb)) - -``` - -出： - -```py -tensor(0.0808, grad_fn=) - -``` - -## 使用`nn.Linear`重构 - -我们继续重构我们的代码。 代替手动定义和初始化`self.weights`和`self.bias`并计算`xb  @ self.weights + self.bias`，我们将对线性层使用 Pytorch 类[`nn.Linear`](https://pytorch.org/docs/stable/nn.html#linear-layers)，这将为我们完成所有工作。 Pytorch 具有许多类型的预定义层，可以大大简化我们的代码，并且通常也可以使其速度更快。 - -```py -class Mnist_Logistic(nn.Module): - def __init__(self): - super().__init__() - self.lin = nn.Linear(784, 10) - - def forward(self, xb): - return self.lin(xb) - -``` - -我们以与以前相同的方式实例化模型并计算损失： - -```py -model = Mnist_Logistic() -print(loss_func(model(xb), yb)) - -``` - -出： - -```py -tensor(2.4215, grad_fn=) - -``` - -我们仍然可以使用与以前相同的`fit`方法。 - -```py -fit() - -print(loss_func(model(xb), yb)) - -``` - -出： - -```py -tensor(0.0824, grad_fn=) - -``` - -## 使用`optim`重构 - -Pytorch 还提供了一个包含各种优化算法的包`torch.optim`。 我们可以使用优化器中的`step`方法采取向前的步骤，而不是手动更新每个参数。 - -这将使我们替换之前的手动编码优化步骤： - -```py -with torch.no_grad(): - for p in model.parameters(): p -= p.grad * lr - model.zero_grad() - -``` - -而是只使用： - -```py -opt.step() -opt.zero_grad() - -``` - -（`optim.zero_grad()`将梯度重置为 0，我们需要在计算下一个小批量的梯度之前调用它。） - -```py -from torch import optim - -``` - -我们将定义一个小函数来创建模型和优化器，以便将来重用。 - -```py -def get_model(): - model = Mnist_Logistic() - return model, optim.SGD(model.parameters(), lr=lr) - -model, opt = get_model() -print(loss_func(model(xb), yb)) - -for epoch in range(epochs): - for i in range((n - 1) // bs + 1): - start_i = i * bs - end_i = start_i + bs - xb = x_train[start_i:end_i] - yb = y_train[start_i:end_i] - pred = model(xb) - loss = loss_func(pred, yb) - - loss.backward() - opt.step() - opt.zero_grad() - -print(loss_func(model(xb), yb)) - -``` - -出： - -```py -tensor(2.2999, grad_fn=) -tensor(0.0823, grad_fn=) - -``` - -## 使用`Dataset`重构 - -PyTorch 有一个抽象的`Dataset`类。 数据集可以是具有`__len__`函数（由 Python 的标准`len`函数调用）和具有`__getitem__`函数作为对其进行索引的一种方法。 [本教程](https://pytorch.org/tutorials/beginner/data_loading_tutorial.html)演示了一个不错的示例，该示例创建一个自定义`FacialLandmarkDataset`类作为`Dataset`的子类。 - -PyTorch 的[`TensorDataset`](https://pytorch.org/docs/stable/_modules/torch/utils/data/dataset.html#TensorDataset)是一个数据集包装张量。 通过定义索引的长度和方式，这也为我们提供了沿张量的第一维进行迭代，索引和切片的方法。 这将使我们在训练的同一行中更容易访问自变量和因变量。 - -```py -from torch.utils.data import TensorDataset - -``` - -`x_train`和`y_train`都可以合并为一个`TensorDataset`，这将更易于迭代和切片。 - -```py -train_ds = TensorDataset(x_train, y_train) - -``` - -以前，我们不得不分别遍历`x`和`y`值的小批量： - -```py -xb = x_train[start_i:end_i] -yb = y_train[start_i:end_i] - -``` - -现在，我们可以一起执行以下两个步骤： - -```py -xb,yb = train_ds[i*bs : i*bs+bs] - -``` - -```py -model, opt = get_model() - -for epoch in range(epochs): - for i in range((n - 1) // bs + 1): - xb, yb = train_ds[i * bs: i * bs + bs] - pred = model(xb) - loss = loss_func(pred, yb) - - loss.backward() - opt.step() - opt.zero_grad() - -print(loss_func(model(xb), yb)) - -``` - -出： - -```py -tensor(0.0819, grad_fn=) - -``` - -## 使用`DataLoader`重构 - -Pytorch 的`DataLoader`负责批量管理。 您可以从任何`Dataset`创建一个`DataLoader`。 `DataLoader`使迭代迭代变得更加容易。 不必使用`train_ds[i*bs : i*bs+bs]`，`DataLoader`会自动为我们提供每个小批量。 - -```py -from torch.utils.data import DataLoader - -train_ds = TensorDataset(x_train, y_train) -train_dl = DataLoader(train_ds, batch_size=bs) - -``` - -以前，我们的循环遍历如下批量`(xb, yb)`： - -```py -for i in range((n-1)//bs + 1): - xb,yb = train_ds[i*bs : i*bs+bs] - pred = model(xb) - -``` - -现在，我们的循环更加简洁了，因为`(xb, yb)`是从数据加载器自动加载的： - -```py -for xb,yb in train_dl: - pred = model(xb) - -``` - -```py -model, opt = get_model() - -for epoch in range(epochs): - for xb, yb in train_dl: - pred = model(xb) - loss = loss_func(pred, yb) - - loss.backward() - opt.step() - opt.zero_grad() - -print(loss_func(model(xb), yb)) - -``` - -出： - -```py -tensor(0.0821, grad_fn=) - -``` - -得益于 Pytorch 的`nn.Module`，`nn.Parameter`，`Dataset`和`DataLoader`，我们的训练循环现在变得更小，更容易理解。 现在，让我们尝试添加在实践中创建有效模型所需的基本功能。 - -## 添加验证 - -在第 1 节中，我们只是试图建立一个合理的训练循环以用于我们的训练数据。 实际上，您也应该**始终**具有[验证集](https://www.fast.ai/2017/11/13/validation-sets/)，以便识别您是否过拟合。 - -[对训练数据进行打乱](https://www.quora.com/Does-the-order-of-training-data-matter-when-training-neural-networks)对于防止批量与过拟合之间的相关性很重要。 另一方面，无论我们是否打乱验证集，验证损失都是相同的。 由于打乱需要花费更多时间，因此打乱验证数据没有任何意义。 - -我们将验证集的批量大小设为训练集的两倍。 这是因为验证集不需要反向传播，因此占用的内存更少（不需要存储梯度）。 我们利用这一优势来使用更大的批量，并更快地计算损失。 - -```py -train_ds = TensorDataset(x_train, y_train) -train_dl = DataLoader(train_ds, batch_size=bs, shuffle=True) - -valid_ds = TensorDataset(x_valid, y_valid) -valid_dl = DataLoader(valid_ds, batch_size=bs * 2) - -``` - -我们将在每个周期结束时计算并打印验证损失。 - -（请注意，我们总是在训练之前调用`model.train()`，并在推理之前调用`model.eval()`，因为诸如`nn.BatchNorm2d`和`nn.Dropout`之类的层会使用它们，以确保这些不同阶段的行为正确。） - -```py -model, opt = get_model() - -for epoch in range(epochs): - model.train() - for xb, yb in train_dl: - pred = model(xb) - loss = loss_func(pred, yb) - - loss.backward() - opt.step() - opt.zero_grad() - - model.eval() - with torch.no_grad(): - valid_loss = sum(loss_func(model(xb), yb) for xb, yb in valid_dl) - - print(epoch, valid_loss / len(valid_dl)) - -``` - -出： - -```py -0 tensor(0.3743) -1 tensor(0.3316) - -``` - -## 创建`fit()`和`get_data()` - -现在，我们将自己进行一些重构。 由于我们经历了两次相似的过程来计算训练集和验证集的损失，因此我们将其设为自己的函数`loss_batch`，该函数可计算一批损失。 - -我们将优化器传入训练集中，然后使用它执行反向传播。 对于验证集，我们没有通过优化程序，因此该方法不会执行反向传播。 - -```py -def loss_batch(model, loss_func, xb, yb, opt=None): - loss = loss_func(model(xb), yb) - - if opt is not None: - loss.backward() - opt.step() - opt.zero_grad() - - return loss.item(), len(xb) - -``` - -`fit`运行必要的操作来训练我们的模型，并计算每个周期的训练和验证损失。 - -```py -import numpy as np - -def fit(epochs, model, loss_func, opt, train_dl, valid_dl): - for epoch in range(epochs): - model.train() - for xb, yb in train_dl: - loss_batch(model, loss_func, xb, yb, opt) - - model.eval() - with torch.no_grad(): - losses, nums = zip( - *[loss_batch(model, loss_func, xb, yb) for xb, yb in valid_dl] - ) - val_loss = np.sum(np.multiply(losses, nums)) / np.sum(nums) - - print(epoch, val_loss) - -``` - -`get_data`返回训练和验证集的数据加载器。 - -```py -def get_data(train_ds, valid_ds, bs): - return ( - DataLoader(train_ds, batch_size=bs, shuffle=True), - DataLoader(valid_ds, batch_size=bs * 2), - ) - -``` - -现在，我们获取数据加载器和拟合模型的整个过程可以在 3 行代码中运行： - -```py -train_dl, valid_dl = get_data(train_ds, valid_ds, bs) -model, opt = get_model() -fit(epochs, model, loss_func, opt, train_dl, valid_dl) - -``` - -出： - -```py -0 0.3120644524335861 -1 0.28915613491535186 - -``` - -您可以使用这些基本的 3 行代码来训练各种各样的模型。 让我们看看是否可以使用它们来训练卷积神经网络（CNN）！ - -## 切换到 CNN - -现在，我们将构建具有三个卷积层的神经网络。 由于上一节中的任何功能都不假设任何有关模型形式的信息，因此我们将能够使用它们来训练 CNN，而无需进行任何修改。 - -我们将使用 Pytorch 的预定义[`Conv2d`](https://pytorch.org/docs/stable/nn.html#torch.nn.Conv2d)类作为我们的卷积层。 我们定义了具有 3 个卷积层的 CNN。 每个卷积后跟一个 ReLU。 最后，我们执行平均池化。 （请注意，`view`是 numpy 的`reshape`的 PyTorch 版本） - -```py -class Mnist_CNN(nn.Module): - def __init__(self): - super().__init__() - self.conv1 = nn.Conv2d(1, 16, kernel_size=3, stride=2, padding=1) - self.conv2 = nn.Conv2d(16, 16, kernel_size=3, stride=2, padding=1) - self.conv3 = nn.Conv2d(16, 10, kernel_size=3, stride=2, padding=1) - - def forward(self, xb): - xb = xb.view(-1, 1, 28, 28) - xb = F.relu(self.conv1(xb)) - xb = F.relu(self.conv2(xb)) - xb = F.relu(self.conv3(xb)) - xb = F.avg_pool2d(xb, 4) - return xb.view(-1, xb.size(1)) - -lr = 0.1 - -``` - -[动量](https://cs231n.github.io/neural-networks-3/#sgd)是随机梯度下降的一种变体，它也考虑了以前的更新，通常可以加快训练速度。 - -```py -model = Mnist_CNN() -opt = optim.SGD(model.parameters(), lr=lr, momentum=0.9) - -fit(epochs, model, loss_func, opt, train_dl, valid_dl) - -``` - -出： - -```py -0 0.32337012240886687 -1 0.25021172934770586 - -``` - -## `nn.Sequential` - -`torch.nn`还有另一个方便的类，可以用来简化我们的代码：[`Sequential`](https://pytorch.org/docs/stable/nn.html#torch.nn.Sequential)。 `Sequential`对象以顺序方式运行其中包含的每个模块。 这是编写神经网络的一种简单方法。 - -为了利用这一点，我们需要能够从给定的函数轻松定义**自定义层**。 例如，PyTorch 没有视层，我们需要为我们的网络创建一个层。 `Lambda`将创建一个层，然后在使用`Sequential`定义网络时可以使用该层。 - -```py -class Lambda(nn.Module): - def __init__(self, func): - super().__init__() - self.func = func - - def forward(self, x): - return self.func(x) - -def preprocess(x): - return x.view(-1, 1, 28, 28) - -``` - -用`Sequential`创建的模型很简单： - -```py -model = nn.Sequential( - Lambda(preprocess), - nn.Conv2d(1, 16, kernel_size=3, stride=2, padding=1), - nn.ReLU(), - nn.Conv2d(16, 16, kernel_size=3, stride=2, padding=1), - nn.ReLU(), - nn.Conv2d(16, 10, kernel_size=3, stride=2, padding=1), - nn.ReLU(), - nn.AvgPool2d(4), - Lambda(lambda x: x.view(x.size(0), -1)), -) - -opt = optim.SGD(model.parameters(), lr=lr, momentum=0.9) - -fit(epochs, model, loss_func, opt, train_dl, valid_dl) - -``` - -出： - -```py -0 0.30119081069231035 -1 0.25335356528759 - -``` - -## 包装`DataLoader` - -Our CNN is fairly concise, but it only works with MNIST, because: - -* 假设输入为`28 * 28`长向量 -* 假设 CNN 的最终网格尺寸为`4 * 4`（因为这是平均值 - -我们使用的合并核大小） - -让我们摆脱这两个假设，因此我们的模型适用于任何 2d 单通道图像。 首先，我们可以删除初始的 Lambda 层，但将数据预处理移至生成器中： - -```py -def preprocess(x, y): - return x.view(-1, 1, 28, 28), y - -class WrappedDataLoader: - def __init__(self, dl, func): - self.dl = dl - self.func = func - - def __len__(self): - return len(self.dl) - - def __iter__(self): - batches = iter(self.dl) - for b in batches: - yield (self.func(*b)) - -train_dl, valid_dl = get_data(train_ds, valid_ds, bs) -train_dl = WrappedDataLoader(train_dl, preprocess) -valid_dl = WrappedDataLoader(valid_dl, preprocess) - -``` - -接下来，我们可以将`nn.AvgPool2d`替换为`nn.AdaptiveAvgPool2d`，这使我们能够定义所需的*输出*张量的大小，而不是所需的*输入*张量的大小。 结果，我们的模型将适用于任何大小的输入。 - -```py -model = nn.Sequential( - nn.Conv2d(1, 16, kernel_size=3, stride=2, padding=1), - nn.ReLU(), - nn.Conv2d(16, 16, kernel_size=3, stride=2, padding=1), - nn.ReLU(), - nn.Conv2d(16, 10, kernel_size=3, stride=2, padding=1), - nn.ReLU(), - nn.AdaptiveAvgPool2d(1), - Lambda(lambda x: x.view(x.size(0), -1)), -) - -opt = optim.SGD(model.parameters(), lr=lr, momentum=0.9) - -``` - -试试看： - -```py -fit(epochs, model, loss_func, opt, train_dl, valid_dl) - -``` - -出： - -```py -0 0.327303307390213 -1 0.2181092014491558 - -``` - -## 使用您的 GPU - -如果您足够幸运地能够使用具有 CUDA 功能的 GPU（可以从大多数云提供商处以每小时 0.50 美元的价格租用一个），则可以使用它来加速代码。 首先检查您的 GPU 是否在 Pytorch 中正常工作： - -```py -print(torch.cuda.is_available()) - -``` - -出： - -```py -True - -``` - -然后为其创建一个设备对象： - -```py -dev = torch.device( - "cuda") if torch.cuda.is_available() else torch.device("cpu") - -``` - -让我们更新`preprocess`，将批量移至 GPU： - -```py -def preprocess(x, y): - return x.view(-1, 1, 28, 28).to(dev), y.to(dev) - -train_dl, valid_dl = get_data(train_ds, valid_ds, bs) -train_dl = WrappedDataLoader(train_dl, preprocess) -valid_dl = WrappedDataLoader(valid_dl, preprocess) - -``` - -最后，我们可以将模型移至 GPU。 - -```py -model.to(dev) -opt = optim.SGD(model.parameters(), lr=lr, momentum=0.9) - -``` - -您应该发现它现在运行得更快： - -```py -fit(epochs, model, loss_func, opt, train_dl, valid_dl) - -``` - -出： - -```py -0 0.1833980613708496 -1 0.17365939717292786 - -``` - -## 总结 - -现在，我们有了一个通用的数据管道和训练循环，您可以将其用于使用 Pytorch 训练许多类型的模型。 要了解现在可以轻松进行模型训练，请查看`mnist_sample`示例笔记本。 - -当然，您需要添加很多内容，例如数据扩充，超参数调整，监控训练，迁移学习等。 这些功能可在 fastai 库中使用，该库是使用本教程中所示的相同设计方法开发的，为希望进一步推广其模型的从业人员提供了自然的下一步。 - -我们承诺在本教程开始时将通过示例分别说明`torch.nn`，`torch.optim`，`Dataset`和`DataLoader`。 因此，让我们总结一下我们所看到的： - -> * `torch.nn` -> * `Module`：创建一个行为类似于函数的可调用对象，但也可以包含状态（例如神经网络层权重）。 它知道其中包含的 `Parameter` ，并且可以将其所有坡度归零，遍历它们以进行权重更新等。 -> * `Parameter`：张量的包装器，用于告知 `Module` 具有在反向传播期间需要更新的权重。 仅更新具有`require_grad`属性集的张量 -> * `functional`：一个模块（通常按照惯例导入到 `F` 名称空间中），其中包含激活函数，损失函数等。 以及卷积和线性层等层的无状态版本。 -> * `torch.optim`：包含诸如 `SGD` 的优化程序，这些优化程序在后退步骤 -> * `Dataset` 中更新 `Parameter` 的权重。 具有 `__len__` 和 `__getitem__` 的对象，包括 Pytorch 提供的类，例如 `TensorDataset` -> * `DataLoader`：获取任何 `Dataset` 并创建一个迭代器，该迭代器返回批量数据。 - -**脚本的总运行时间**：（0 分钟 57.062 秒） - -[下载 Python 源码：`nn_tutorial.py`](../_downloads/a6246751179fbfb7cad9222ef1c16617/nn_tutorial.py) - -[下载 Jupyter 笔记本：`nn_tutorial.ipynb`](../_downloads/5ddab57bb7482fbcc76722617dd47324/nn_tutorial.ipynb) - -[由 Sphinx 画廊](https://sphinx-gallery.readthedocs.io)生成的画廊 \ No newline at end of file diff --git a/pytorch/官方教程/17.md b/pytorch/官方教程/17.md deleted file mode 100644 index 12d5eeb4..00000000 --- a/pytorch/官方教程/17.md +++ /dev/null @@ -1,348 +0,0 @@ -# 使用 TensorBoard 可视化模型，数据和训练 - -> 原文： - -在 [60 分钟突击](https://pytorch.org/tutorials/beginner/deep_learning_60min_blitz.html)中，我们向您展示了如何加载数据，如何通过定义为`nn.Module`子类的模型提供数据，如何在训练数据上训练该模型以及在测试数据上对其进行测试。 为了了解发生的情况，我们在模型训练期间打印一些统计数据，以了解训练是否在进行中。 但是，我们可以做得更好：PyTorch 与 TensorBoard 集成在一起，TensorBoard 是一种工具，用于可视化神经网络训练运行的结果。 本教程使用 [Fashion-MNIST 数据集](https://github.com/zalandoresearch/fashion-mnist)说明了其某些功能，可以使用`torchvision.datasets`将其读入 PyTorch。 - -在本教程中，我们将学习如何： - -> 1. 读取数据并进行适当的转换（与先前的教程几乎相同）。 -> 2. 设置 TensorBoard。 -> 3. 写入 TensorBoard。 -> 4. 使用 TensorBoard 检查模型架构。 -> 5. 使用 TensorBoard 来创建我们在上一个教程中创建的可视化的交互式版本，并使用较少的代码 - -具体来说，在第 5 点，我们将看到： - -> * 有两种方法可以检查我们的训练数据 -> * 在训练模型时如何跟踪其表现 -> * 在训练后如何评估模型的表现。 - -我们将从 [CIFAR-10 教程](https://pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html)中类似的样板代码开始： - -```py -# imports -import matplotlib.pyplot as plt -import numpy as np - -import torch -import torchvision -import torchvision.transforms as transforms - -import torch.nn as nn -import torch.nn.functional as F -import torch.optim as optim - -# transforms -transform = transforms.Compose( - [transforms.ToTensor(), - transforms.Normalize((0.5,), (0.5,))]) - -# datasets -trainset = torchvision.datasets.FashionMNIST('./data', - download=True, - train=True, - transform=transform) -testset = torchvision.datasets.FashionMNIST('./data', - download=True, - train=False, - transform=transform) - -# dataloaders -trainloader = torch.utils.data.DataLoader(trainset, batch_size=4, - shuffle=True, num_workers=2) - -testloader = torch.utils.data.DataLoader(testset, batch_size=4, - shuffle=False, num_workers=2) - -# constant for classes -classes = ('T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat', - 'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle Boot') - -# helper function to show an image -# (used in the `plot_classes_preds` function below) -def matplotlib_imshow(img, one_channel=False): - if one_channel: - img = img.mean(dim=0) - img = img / 2 + 0.5 # unnormalize - npimg = img.numpy() - if one_channel: - plt.imshow(npimg, cmap="Greys") - else: - plt.imshow(np.transpose(npimg, (1, 2, 0))) - -``` - -我们将在该教程中定义一个类似的模型架构，仅需进行少量修改即可解决以下事实：图像现在是一个通道而不是三个通道，而图像是`28x28`而不是`32x32`： - -```py -class Net(nn.Module): - def __init__(self): - super(Net, self).__init__() - self.conv1 = nn.Conv2d(1, 6, 5) - self.pool = nn.MaxPool2d(2, 2) - self.conv2 = nn.Conv2d(6, 16, 5) - self.fc1 = nn.Linear(16 * 4 * 4, 120) - self.fc2 = nn.Linear(120, 84) - self.fc3 = nn.Linear(84, 10) - - def forward(self, x): - x = self.pool(F.relu(self.conv1(x))) - x = self.pool(F.relu(self.conv2(x))) - x = x.view(-1, 16 * 4 * 4) - x = F.relu(self.fc1(x)) - x = F.relu(self.fc2(x)) - x = self.fc3(x) - return x - -net = Net() - -``` - -我们将在之前定义相同的`optimizer`和`criterion`： - -```py -criterion = nn.CrossEntropyLoss() -optimizer = optim.SGD(net.parameters(), lr=0.001, momentum=0.9) - -``` - -## 1\. TensorBoard 设置 - -现在，我们将设置 TensorBoard，从`torch.utils`导入`tensorboard`并定义`SummaryWriter`，这是将信息写入 TensorBoard 的关键对象。 - -```py -from torch.utils.tensorboard import SummaryWriter - -# default `log_dir` is "runs" - we'll be more specific here -writer = SummaryWriter('runs/fashion_mnist_experiment_1') - -``` - -请注意，仅此行会创建一个`runs/fashion_mnist_experiment_1`文件夹。 - -## 2\. 写入 TensorBoard - -现在，使用[`make_grid`](https://pytorch.org/docs/stable/torchvision/utils.html#torchvision.utils.make_grid)将图像写入到 TensorBoard 中，具体来说就是网格。 - -```py -# get some random training images -dataiter = iter(trainloader) -images, labels = dataiter.next() - -# create grid of images -img_grid = torchvision.utils.make_grid(images) - -# show images -matplotlib_imshow(img_grid, one_channel=True) - -# write to tensorboard -writer.add_image('four_fashion_mnist_images', img_grid) - -``` - -正在运行 - -```py -tensorboard --logdir=runs - -``` - -从命令行，然后导航到`https://localhost:6006`应该显示以下内容。 - -![intermediate/../../_static/img/tensorboard_first_view.png](img/8b09d6361316e495383ceedf9b8407ea.png) - -现在您知道如何使用 TensorBoard 了！ 但是，此示例可以在 Jupyter 笔记本中完成-TensorBoard 真正擅长的地方是创建交互式可视化。 接下来，我们将介绍其中之一，并在本教程结束时介绍更多内容。 - -## 3\. 使用 TensorBoard 检查模型 - -TensorBoard 的优势之一是其可视化复杂模型结构的能力。 让我们可视化我们构建的模型。 - -```py -writer.add_graph(net, images) -writer.close() - -``` - -现在刷新 TensorBoard 后，您应该会看到一个`Graphs`标签，如下所示： - -![intermediate/../../_static/img/tensorboard_model_viz.png](img/8f596b99dbb3c262b61db267d5db2d63.png) - -继续并双击`Net`以展开它，查看构成模型的各个操作的详细视图。 - -TensorBoard 具有非常方便的功能，可在低维空间中可视化高维数据，例如图像数据。 接下来我们将介绍这一点。 - -## 4\. 在 TensorBoard 中添加“投影仪” - -我们可以通过[`add_embedding`](https://pytorch.org/docs/stable/tensorboard.html#torch.utils.tensorboard.writer.SummaryWriter.add_embedding)方法可视化高维数据的低维表示 - -```py -# helper function -def select_n_random(data, labels, n=100): - ''' - Selects n random datapoints and their corresponding labels from a dataset - ''' - assert len(data) == len(labels) - - perm = torch.randperm(len(data)) - return data[perm][:n], labels[perm][:n] - -# select random images and their target indices -images, labels = select_n_random(trainset.data, trainset.targets) - -# get the class labels for each image -class_labels = [classes[lab] for lab in labels] - -# log embeddings -features = images.view(-1, 28 * 28) -writer.add_embedding(features, - metadata=class_labels, - label_img=images.unsqueeze(1)) -writer.close() - -``` - -现在，在 TensorBoard 的“投影仪”选项卡中，您可以看到这 100 张图像-每个图像 784 维-向下投影到三维空间中。 此外，这是交互式的：您可以单击并拖动以旋转三维投影。 最后，一些技巧可以使可视化效果更容易看到：选择左上方的“颜色：标签”，以及启用“夜间模式”，这将使图像更容易看到，因为它们的背景是白色的： - -![intermediate/../../_static/img/tensorboard_projector.png](img/f4990a0920dff7e4647a23cfc1639a8a.png) - -现在我们已经彻底检查了我们的数据，让我们展示了 TensorBoard 如何从训练开始就可以使跟踪模型的训练和评估更加清晰。 - -## 5\. 使用 TensorBoard 跟踪模型训练 - -在前面的示例中，我们仅*每 2000 次迭代*打印该模型的运行损失。 现在，我们将运行损失记录到 TensorBoard 中，并通过`plot_classes_preds`函数查看模型所做的预测。 - -```py -# helper functions - -def images_to_probs(net, images): - ''' - Generates predictions and corresponding probabilities from a trained - network and a list of images - ''' - output = net(images) - # convert output probabilities to predicted class - _, preds_tensor = torch.max(output, 1) - preds = np.squeeze(preds_tensor.numpy()) - return preds, [F.softmax(el, dim=0)[i].item() for i, el in zip(preds, output)] - -def plot_classes_preds(net, images, labels): - ''' - Generates matplotlib Figure using a trained network, along with images - and labels from a batch, that shows the network's top prediction along - with its probability, alongside the actual label, coloring this - information based on whether the prediction was correct or not. - Uses the "images_to_probs" function. - ''' - preds, probs = images_to_probs(net, images) - # plot the images in the batch, along with predicted and true labels - fig = plt.figure(figsize=(12, 48)) - for idx in np.arange(4): - ax = fig.add_subplot(1, 4, idx+1, xticks=[], yticks=[]) - matplotlib_imshow(images[idx], one_channel=True) - ax.set_title("{0}, {1:.1f}%\n(label: {2})".format( - classes[preds[idx]], - probs[idx] * 100.0, - classes[labels[idx]]), - color=("green" if preds[idx]==labels[idx].item() else "red")) - return fig - -``` - -最后，让我们使用与之前教程相同的模型训练代码来训练模型，但是每 1000 批将结果写入 TensorBoard，而不是打印到控制台。 这是通过[`add_scalar`](https://pytorch.org/docs/stable/tensorboard.html#torch.utils.tensorboard.writer.SummaryWriter.add_scalar)函数完成的。 - -此外，在训练过程中，我们将生成一幅图像，显示该批量中包含的四幅图像的模型预测与实际结果。 - -```py -running_loss = 0.0 -for epoch in range(1): # loop over the dataset multiple times - - for i, data in enumerate(trainloader, 0): - - # get the inputs; data is a list of [inputs, labels] - inputs, labels = data - - # zero the parameter gradients - optimizer.zero_grad() - - # forward + backward + optimize - outputs = net(inputs) - loss = criterion(outputs, labels) - loss.backward() - optimizer.step() - - running_loss += loss.item() - if i % 1000 == 999: # every 1000 mini-batches... - - # ...log the running loss - writer.add_scalar('training loss', - running_loss / 1000, - epoch * len(trainloader) + i) - - # ...log a Matplotlib Figure showing the model's predictions on a - # random mini-batch - writer.add_figure('predictions vs. actuals', - plot_classes_preds(net, inputs, labels), - global_step=epoch * len(trainloader) + i) - running_loss = 0.0 -print('Finished Training') - -``` - -现在，您可以查看“标量”选项卡，以查看在 15,000 次训练迭代中绘制的运行损失： - -![intermediate/../../_static/img/tensorboard_scalar_runs.png](img/afda8238ecd1f547d61be4d155844f68.png) - -此外，我们可以查看整个学习过程中模型在任意批量上所做的预测。 查看“图像”选项卡，然后在“预测与实际”可视化条件下向下滚动以查看此内容； 这表明，例如，仅经过 3000 次训练迭代，该模型就已经能够区分出视觉上截然不同的类，例如衬衫，运动鞋和外套，尽管它并没有像后来的训练那样有信心： - -![intermediate/../../_static/img/tensorboard_images.png](img/d5ab1f07cb4a9d9200c2a2d3b238340d.png) - -在之前的教程中，我们研究了模型训练后的每类准确率； 在这里，我们将使用 TensorBoard 绘制每个类别的精确调用曲线（[在这里解释](https://www.scikit-yb.org/en/latest/api/classifier/prcurve.html)）。 - -## 6\. 使用 TensorBoard 评估经过训练的模型 - -```py -# 1\. gets the probability predictions in a test_size x num_classes Tensor -# 2\. gets the preds in a test_size Tensor -# takes ~10 seconds to run -class_probs = [] -class_preds = [] -with torch.no_grad(): - for data in testloader: - images, labels = data - output = net(images) - class_probs_batch = [F.softmax(el, dim=0) for el in output] - _, class_preds_batch = torch.max(output, 1) - - class_probs.append(class_probs_batch) - class_preds.append(class_preds_batch) - -test_probs = torch.cat([torch.stack(batch) for batch in class_probs]) -test_preds = torch.cat(class_preds) - -# helper function -def add_pr_curve_tensorboard(class_index, test_probs, test_preds, global_step=0): - ''' - Takes in a "class_index" from 0 to 9 and plots the corresponding - precision-recall curve - ''' - tensorboard_preds = test_preds == class_index - tensorboard_probs = test_probs[:, class_index] - - writer.add_pr_curve(classes[class_index], - tensorboard_preds, - tensorboard_probs, - global_step=global_step) - writer.close() - -# plot all the pr curves -for i in range(len(classes)): - add_pr_curve_tensorboard(i, test_probs, test_preds) - -``` - -现在，您将看到一个`PR Curves`选项卡，其中包含每个类别的精确调用曲线。 继续四处戳； 您会发现在某些类别中，模型的“曲线下面积”接近 100%，而在另一些类别中，该面积更低： - -![intermediate/../../_static/img/tensorboard_pr_curves.png](img/d15de2be2b754f9a4f46418764232b5e.png) - -这是 TensorBoard 和 PyTorch 与之集成的介绍。 当然，您可以在 Jupyter 笔记本中完成 TensorBoard 的所有操作，但是使用 TensorBoard 时，默认情况下会获得交互式的视觉效果。 \ No newline at end of file diff --git a/pytorch/官方教程/README.md b/pytorch/官方教程/README.md deleted file mode 100644 index cd349818..00000000 --- a/pytorch/官方教程/README.md +++ /dev/null @@ -1,35 +0,0 @@ -# PyTorch 中文官方教程 1.7 - -> 原文：[WELCOME TO PYTORCH TUTORIALS](https://pytorch.org/tutorials/) -> -> 协议：[CC BY-NC-SA 4.0](http://creativecommons.org/licenses/by-nc-sa/4.0/) -> -> 自豪地采用[谷歌翻译](https://translate.google.cn/) -> -> 不要担心自己的形象，只关心如何实现目标。——《原则》，生活原则 2.3.c - -* [在线阅读](https://dl.apachecn.org) -* [ApacheCN 面试求职交流群 724187166](https://jq.qq.com/?_wv=1027&k=54ujcL3) -* [ApacheCN 学习资源](http://www.apachecn.org/) - -## 贡献指南 - -本项目需要校对，欢迎大家提交 Pull Request。 - -> 请您勇敢地去翻译和改进翻译。虽然我们追求卓越，但我们并不要求您做到十全十美，因此请不要担心因为翻译上犯错——在大部分情况下，我们的服务器已经记录所有的翻译，因此您不必担心会因为您的失误遭到无法挽回的破坏。（改编自维基百科） - -## 联系方式 - -### 负责人 - -* [飞龙](https://github.com/wizardforcel): 562826179 - -### 其他 - -* 在我们的 [apachecn/apachecn-tf-zh](https://github.com/apachecn/apachecn-tf-zh) github 上提 issue. -* 发邮件到 Email: `apachecn@163.com`. -* 在我们的 [组织学习交流群](http://www.apachecn.org/organization/348.html) 中联系群主/管理员即可. - -## 赞助我们 - -![](http://data.apachecn.org/img/about/donate.jpg) diff --git a/pytorch/官方教程/book.json b/pytorch/官方教程/book.json deleted file mode 100644 index 2d99e483..00000000 --- a/pytorch/官方教程/book.json +++ /dev/null @@ -1,176 +0,0 @@ -{ - "title" : "Pytorch 中文文档", - "author" : "ApacheCN", - "description" : "Pytorch 中文文档: 教程和文档", - "language" : "zh-hans", - "plugins": [ - "github", - "github-buttons", - "-sharing", - "insert-logo", - "sharing-plus", - "back-to-top-button", - "code", - "copy-code-button", - "katex", - "pageview-count", - "edit-link", - "emphasize", - "alerts", - "auto-scroll-table", - "popup", - "hide-element", - "page-toc-button", - "tbfed-pagefooter", - "sitemap", - "advanced-emoji", - "expandable-chapters", - "splitter", - "search-pro" - ], - "pluginsConfig": { - "github": { - "url": "https://github.com/apachecn/pytorch-doc-zh" - }, - "github-buttons": { - "buttons": [ - { - "user": "apachecn", - "repo": "pytorch-doc-zh", - "type": "star", - "count": true, - "size": "small" - } - ] - }, - "insert-logo": { - "url": "http://data.apachecn.org/img/logo.jpg", - "style": "background: none; max-height: 150px; min-height: 150px" - }, - "hide-element": { - "elements": [".gitbook-link"] - }, - "edit-link": { - "base": "https://github.com/apachecn/pytorch-doc-zh/blob/master/docs/1.7", - "label": "编辑本页" - }, - "sharing": { - "qzone": true, - "weibo": true, - "twitter": false, - "facebook": false, - "google": false, - "qq": false, - "line": false, - "whatsapp": false, - "douban": false, - "all": [ - "qq", "douban", "facebook", "google", "linkedin", "twitter", "weibo", "whatsapp" - ] - }, - "page-toc-button": { - "maxTocDepth": 4, - "minTocSize": 4 - }, - "tbfed-pagefooter": { - "copyright":"Copyright © ibooker.org.cn 2019", - "modify_label": "该文件修订时间： ", - "modify_format": "YYYY-MM-DD HH:mm:ss" - }, - "sitemap": { - "hostname": "http://pytorch.apachecn.org" - } - }, - "my_links" : { - "sidebar" : { - "Home" : "https://www.baidu.com" - } - }, - "my_plugins": [ - "donate", - "todo", - "-lunr", - "-search", - "expandable-chapters-small", - "chapter-fold", - "expandable-chapters", - "expandable-chapters-small", - "back-to-top-button", - "ga", - "baidu", - "sitemap", - "tbfed-pagefooter", - "advanced-emoji", - "sectionx", - "page-treeview", - "simple-page-toc", - "ancre-navigation", - "theme-apachecn@git+https://github.com/apachecn/theme-apachecn#HEAD", - "pagefooter-apachecn@git+https://github.com/apachecn/gitbook-plugin-pagefooter-apachecn#HEAD" - ], - "my_pluginsConfig": { - "github-buttons": { - "buttons": [ - { - "user": "apachecn", - "repo": "pytorch-doc-zh", - "type": "star", - "count": true, - "size": "small" - }, - { - "user": "apachecn", - "width": "160", - "type": "follow", - "count": true, - "size": "small" - } - ] - }, - "ignores": ["node_modules"], - "simple-page-toc": { - "maxDepth": 3, - "skipFirstH1": true - }, - "page-toc-button": { - "maxTocDepth": 2, - "minTocSize": 2 - }, - "page-treeview": { - "copyright": "Copyright © aleen42", - "minHeaderCount": "2", - "minHeaderDeep": "2" - }, - "donate": { - "wechat": "微信收款的二维码URL", - "alipay": "支付宝收款的二维码URL", - "title": "", - "button": "赏", - "alipayText": "支付宝打赏", - "wechatText": "微信打赏" - }, - "page-copyright": { - "description": "modified at", - "signature": "你的签名", - "wisdom": "Designer, Frontend Developer & overall web enthusiast", - "format": "YYYY-MM-dd hh:mm:ss", - "copyright": "Copyright © 你的名字", - "timeColor": "#666", - "copyrightColor": "#666", - "utcOffset": "8", - "style": "normal", - "noPowered": false - }, - "ga": { - "token": "UA-102475051-10" - }, - "baidu": { - "token": "75439e2cbd22bdd813226000e9dcc12f" - }, - "pagefooter-apachecn": { - "copyright":"Copyright © ibooker.org.cn 2019", - "modify_label": "该文件修订时间： ", - "modify_format": "YYYY-MM-DD HH:mm:ss" - } - } -} diff --git a/pytorch/实战/.gitignore b/pytorch/实战/.gitignore new file mode 100644 index 00000000..6320cd24 --- /dev/null +++ b/pytorch/实战/.gitignore @@ -0,0 +1 @@ +data \ No newline at end of file diff --git a/pytorch/实战/1.ipynb b/pytorch/实战/1.ipynb index 4ff48f15..bf237334 100644 --- a/pytorch/实战/1.ipynb +++ b/pytorch/实战/1.ipynb @@ -193,21 +193,197 @@ }, { "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], + "execution_count": 103, + "metadata": { + "tags": [] + }, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "Net(\n (conv1): Conv2d(1, 6, kernel_size=(3, 3), stride=(1, 1))\n (conv2): Conv2d(6, 16, kernel_size=(3, 3), stride=(1, 1))\n (fc1): Linear(in_features=576, out_features=120, bias=True)\n (fc2): Linear(in_features=120, out_features=84, bias=True)\n (fc3): Linear(in_features=84, out_features=10, bias=True)\n)\n10\ntorch.Size([6, 1, 3, 3])\n" + ] + } + ], "source": [ "import torch\n", "import torch.nn as nn\n", - "import torch.nn.functional as f\n", + "import torch.nn.functional as F\n", "\n", "class Net(nn.Module):\n", + "\n", " def __init__(self):\n", + " super(Net,self).__init__()\n", " self.conv1 = nn.Conv2d(1,6,3)\n", " self.conv2 = nn.Conv2d(6,16,3)\n", " self.fc1 = nn.Linear(16*6*6,120)\n", - " self" + " self.fc2 = nn.Linear(120,84)\n", + " self.fc3 = nn.Linear(84,10)\n", + "\n", + " def forward(self,x):\n", + " x = F.max_pool2d(F.relu(self.conv1(x)),(2,2))\n", + " x = F.max_pool2d(F.relu(self.conv2(x)),2)\n", + " x = x.view(-1,self.num_flat_features(x))\n", + " x = F.relu(self.fc1(x))\n", + " x = F.relu(self.fc2(x))\n", + " x = self.fc3(x)\n", + " return x\n", + " \n", + " def num_flat_features(self,x):\n", + " size = x.size()[1:]\n", + " num_features = 1\n", + " for s in size:\n", + " num_features *=s\n", + " return num_features\n", + "\n", + "net = Net()\n", + "print(net)\n", + "params = list(net.parameters())\n", + "print(len(params))\n", + "print(params[0].size())" ] + }, + { + "cell_type": "code", + "execution_count": 104, + "metadata": {}, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "tensor([[ 0.0492, -0.0594, -0.1026, -0.0511, 0.0224, -0.0672, 0.1048, 0.0772,\n -0.1358, -0.0327]], grad_fn=)\n" + ] + } + ], + "source": [ + "input = torch.randn(1,1,32,32)\n", + "out = net(input)\n", + "print(out)\n", + "\n", + "net.zero_grad()\n", + "out.backward(torch.randn(1,10))" + ] + }, + { + "cell_type": "code", + "execution_count": 83, + "metadata": {}, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "tensor([[0.9332, 0.9385],\n [0.5645, 0.9817],\n [0.0998, 0.0800],\n [0.3189, 0.7160],\n [0.4157, 0.3705]])\n" + ] + } + ], + "source": [ + "# tensor,的第一个维度表示数量。第二个维度开始表示数据的格式。如果第一维的维度>1说明存在一个以上的数据条目\n", + "a = torch.rand(2,5)\n", + "# b = np.rand(2,3)\n", + "print(a.T)" + ] + }, + { + "cell_type": "code", + "execution_count": 96, + "metadata": {}, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "None\nTrue\ntensor([[-0.8533, -1.8255, 0.1003, -0.2550, 0.0429, -2.4029, -2.1907, -0.3119,\n -0.5956, 0.5517]])\ntensor(1.5207, grad_fn=)\n\n\n\n" + ] + } + ], + "source": [ + "# 定义损失\n", + "print(input.grad)\n", + "print(input.is_leaf)\n", + "output = net(input)\n", + "target = torch.randn(10)\n", + "target = target.view(1,-1)\n", + "print(target)\n", + "criterion = nn.MSELoss()\n", + "loss = criterion(output,target)\n", + "# 输出了反向传播过程中的函数\n", + "print(loss)\n", + "print(loss.grad_fn)\n", + "print(loss.grad_fn.next_functions[0][0])\n", + "print(loss.grad_fn.next_functions[0][0].next_functions[0][0])" + ] + }, + { + "cell_type": "code", + "execution_count": 100, + "metadata": {}, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "tensor([0., 0., 0., 0., 0., 0.])\n" + ] + }, + { + "output_type": "error", + "ename": "RuntimeError", + "evalue": "Trying to backward through the graph a second time, but the saved intermediate results have already been freed. Specify retain_graph=True when calling .backward() or autograd.grad() the first time.", + "traceback": [ + "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m", + "\u001b[1;31mRuntimeError\u001b[0m Traceback (most recent call last)", + "\u001b[1;32m\u001b[0m in \u001b[0;36m\u001b[1;34m\u001b[0m\n\u001b[0;32m 4\u001b[0m \u001b[0mprint\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mnet\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mconv1\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mbias\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mgrad\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 5\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m----> 6\u001b[1;33m \u001b[0mloss\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mbackward\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m 7\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 8\u001b[0m \u001b[0mprint\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mnet\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mconv1\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mbias\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mgrad\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n", + "\u001b[1;32mC:\\Python\\lib\\site-packages\\torch\\tensor.py\u001b[0m in \u001b[0;36mbackward\u001b[1;34m(self, gradient, retain_graph, create_graph, inputs)\u001b[0m\n\u001b[0;32m 243\u001b[0m \u001b[0mcreate_graph\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mcreate_graph\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 244\u001b[0m inputs=inputs)\n\u001b[1;32m--> 245\u001b[1;33m \u001b[0mtorch\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mautograd\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mbackward\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mgradient\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mretain_graph\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mcreate_graph\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0minputs\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0minputs\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m 246\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 247\u001b[0m \u001b[1;32mdef\u001b[0m \u001b[0mregister_hook\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mhook\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n", + "\u001b[1;32mC:\\Python\\lib\\site-packages\\torch\\autograd\\__init__.py\u001b[0m in \u001b[0;36mbackward\u001b[1;34m(tensors, grad_tensors, retain_graph, create_graph, grad_variables, inputs)\u001b[0m\n\u001b[0;32m 143\u001b[0m \u001b[0mretain_graph\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mcreate_graph\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 144\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 145\u001b[1;33m Variable._execution_engine.run_backward(\n\u001b[0m\u001b[0;32m 146\u001b[0m \u001b[0mtensors\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mgrad_tensors_\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mretain_graph\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mcreate_graph\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0minputs\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 147\u001b[0m allow_unreachable=True, accumulate_grad=True) # allow_unreachable flag\n", + "\u001b[1;31mRuntimeError\u001b[0m: Trying to backward through the graph a second time, but the saved intermediate results have already been freed. Specify retain_graph=True when calling .backward() or autograd.grad() the first time." + ] + } + ], + "source": [ + "# 进行反向传播\n", + "net.zero_grad()\n", + "\n", + "print(net.conv1.bias.grad)\n", + "loss.backward()\n", + "\n", + "print(net.conv1.bias.grad)\n", + "# 更新网络权重weight = weight -learning_rate*gradient\n", + "learning_rate = 0.01\n", + "for f in net.parameters():\n", + " f.data.sub_(f.grad.data*learning_rate)\n" + ] + }, + { + "cell_type": "code", + "execution_count": 106, + "metadata": {}, + "outputs": [], + "source": [ + "# 常用的优化器\n", + "import torch.optim as optim\n", + "\n", + "optimizer = optim.SGD(net.parameters(),lr=0.01)\n", + "\n", + "optimizer.zero_grad()\n", + "\n", + "output = net(input)\n", + "loss = criterion(output,target)\n", + "loss.backward()\n", + "optimizer.step()" + ] + }, + { + "source": [ + "### 卷积的计算公式\n", + "$$\n", + "d = (d - kennelsize + 2 * padding) / stride + 1\n", + "$$" + ], + "cell_type": "markdown", + "metadata": {} } ] } \ No newline at end of file diff --git a/pytorch/实战/6.ipynb b/pytorch/实战/6.ipynb new file mode 100644 index 00000000..53dc4ff3 --- /dev/null +++ b/pytorch/实战/6.ipynb @@ -0,0 +1,392 @@ +{ + "metadata": { + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.8.0-final" + }, + "orig_nbformat": 2, + "kernelspec": { + "name": "python3", + "display_name": "Python 3.8.0 64-bit", + "metadata": { + "interpreter": { + "hash": "38740d3277777e2cd7c6c2cc9d8addf5118fdf3f82b1b39231fd12aeac8aee8b" + } + } + } + }, + "nbformat": 4, + "nbformat_minor": 2, + "cells": [ + { + "source": [ + "## 6 训练分类器" + ], + "cell_type": "markdown", + "metadata": {} + }, + { + "cell_type": "code", + "execution_count": 38, + "metadata": {}, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "Files already downloaded and verified\n", + "Files already downloaded and verified\n" + ] + } + ], + "source": [ + "# 6.1 加载数据。数据预处理。将数据集加载到内存当中。dataloader是加载器。只在调用数据的时候进行惰性加载。\n", + "import torch\n", + "import torchvision\n", + "import torchvision.transforms as transforms\n", + "\n", + "transform = transforms.Compose(\n", + " [transforms.ToTensor(),\n", + " transforms.Normalize((0.5,0.5,0.5),(.5,.5,5))]\n", + ")\n", + "trainset = torchvision.datasets.CIFAR10(root='./data',train=True,download=True,transform=transform)\n", + "\n", + "trainloader = torch.utils.data.DataLoader(trainset,batch_size=4,shuffle=True,num_workers=2)\n", + "\n", + "testset = torchvision.datasets.CIFAR10(root='./data',train=False,download=True,transform=transform)\n", + "\n", + "testloader = torch.utils.data.DataLoader(testset,batch_size=4,num_workers=2)\n", + "\n", + "classes = ('plane', 'car', 'bird', 'cat','deer', 'dog', 'frog', 'horse', 'ship', 'truck')" + ] + }, + { + "cell_type": "code", + "execution_count": 39, + "metadata": {}, + "outputs": [ + { + "output_type": "display_data", + "data": { + "text/plain": "
", + "image/svg+xml": "\r\n\r\n\r\n\r\n \r\n \r\n \r\n \r\n 2021-03-23T13:48:01.381807\r\n image/svg+xml\r\n \r\n \r\n Matplotlib v3.3.2, https://matplotlib.org/\r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n\r\n", + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAB5CAYAAAAgYXpDAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAACeQUlEQVR4nOz9S6itW5Ymhn1jzv+1Hnvv87g3Iu6NjKosZ0mlhhsSCBnjjpAQuGGojiksgShBQbbcMHZDhTtygRvljo3BxiZBwiUwLhV+UMLIGCNUlNUxehkbqVwqZVZGZkTcuI9zzn6sx/+ac7jxjTH/f+29z41zI7K4JHV+2Gfts/Za/3POMcf4xje+IaqKj9vH7eP2cfu4/enbwvd9Ah+3j9vH7eP2cfv1to8G/OP2cfu4fdz+lG4fDfjH7eP2cfu4/SndPhrwj9vH7eP2cftTun004B+3j9vH7eP2p3T7aMA/bh+3j9vH7U/p9hsZcBH5b4vI3xeR/0pE/uqf1El93D5uH7eP28ftV2/y6/LARSQC+C8B/EsAfgbgPwLwL6vqf/End3oft4/bx+3j9nF731b9Bt/95wD8V6r6BwAgIn8TwF8E8F4Dvt1u9cWLF7/BIT9uH7eP28ftH7/tiy+++EZVP338/m9iwH8M4I9X//8ZgP/Gt33hxYsX+N3f/d3f4JAft4/bx+3j9o/f9tf+2l/76XPv/yNPYorI74rIfywi//HpdPpHfbiP28ft4/Zx+8dm+00M+M8B/GT1/9+y9y42Vf09Vf1nVfWf3W63v8HhPm4ft4/bx+3jtt5+EwjlPwLwT4jInwMN938PwL/yXXYQGkHcBSACOQMKQN6TU332feF3yqaXv78vPftBaVv59i++97h/Upsup+Gnov7LkKHDclAR4PqFYLe/PGmRZR+QZT+CR2/g+cv9kO3X/d6H3rJ/VPsXAcZR8eaNYuiXTycF+qRQBWIQBAHqCtg0PJNxUswZmCd+XzMwz4CqItaCUAEhAnXL+zvPwJyWaxEBKgGCAKrLuM9GJsgZyGqfqwCBIKsi22fnieemeXWRz12s7QMiPG4FSET5rqoiDwASECugqjmZNPFvEAECIIF/hwBpBFLScv8gQIgCCUBXA10jF8+rz4peeS6FKyEKCK8lJb6fEpBz+QAk8J5LWI7DzysvdX3dPk9sOMdaUJlVm2dd7rFdW5p4/XkCcuJzCNGuswVCxfdz5ufzYPfaHqAIgGi3pwJCWJ4rwP0AdozM15T46n+DrH63awiRP1UUXHeCtvqwkf9rG3BVnUXkvw/g/85Lwr+lqv/5d9lH3Afs/skaoQ0YZmDOvBmhDEjlgFBAbMD631QADfxEub/KH9VlQuh68Pj3HpuFS1tW3ns8M9b7Ul8g7PXx95/MqW+bbPL0T5L5E7CESRr52fzVDP1yKl+IEfiv/RMRv/MX4oWxDjYBJAAhcHIFcNCWw64NuyxXUX5734K6+tB3NbL65Jf3bL/p/p97X80wBODd24y/+3cm/PKL5RtDUvzypBgzsKuBtgJetIIfvQ4QAN/cZkxnxd1Z8eabjHkCjg+KeVZ0N4L2Bmg2ghfbgFAJ7kbFsee+A4AowFUDtFEwT4pppMEekiIpMM6KaaYB2+0EMSiGGZiSou8VD/dKQ9YDOj9zwQoOcjMYMdJwV3tB3ABpBuZRkSdgfKPIJ2C7F1zd8DvzSaEzoDWglaBqgO2NQAQ4vFMMBy2GPUSg2QmqRvDDF4KfvJZiyDKAb2bFl5Miqy6G34zlOCqOR8U8A8ezYlqGM6oKeHEjqGtbeCpgmhSHkyKnxZnTBOi0fCeIYHdNZyYl4HDgc5knGvO5B07vuBAN7xTzEahqoO0EsQV2PxLUO8E4KIYzkHvF+CWQBhRPSmogbGm8u52gaTmfogASBKHm9c2DYup5v88PijQvi6gE8HOwxTQDTSvY7IFdG/BP/1aNT6/+ERtwAFDVfw/Av/frfl8iELcB0gnCDEiy+1S8T/MVMzggAQSz1hqA7BPcX7N9VwHR1Ujy88VieJ5M8ufulzx6c7Xil/3q8vKtt3xlwJ81MGsjvlqwoCtjGc0Lf7w6C9BtBNc20fy0Q5BixEMsu0AQ3tv15cnje+l/yBf/e+/nv8v2XZirv87+AUCfuct+XBEubikt3ppvWYExA2MCmqiIWaAAYpTi0SoUcwKGCRgHxalXTJMibwCdBNoAI4AIxaBAr2qLp6ACMAPldcpAUi4YSYEhAcOsSCKoMw3DkBVTBs4zcJiUXmSvxXhdXp9wR5nXGCPHQVUpYhSk2RaNUTGdFelEw9JsaMDHM4271kCugToDsmEkcu4V5165AJrHmGpe02Te9HqoTAqckhnwmU/FI8phVjyMNOAPvWKclodTZ6CagUb8WSnGBDxMthCo0FGbFwNeJyBEBWZBrbwFx5QxzVws5hGYzorDkQZ5uFdMB0ZKm6SISZB6oK4VQw/0J0U+83PpjOJJSQ3ETAO+gaBJdn8FkKC0YwGYeh5vnoDzYWXAK0YYsea1qdm2JiumSqCSMecPnyC/kQH/jbcA5AoINb3o/CTMx2LEPNQ0Q56En1cshhx58cCLsQ3cSfGcASSRZXrLZQj0rDe+PhUzwGo3WbMsBunChX70xdWfnxowffqbMNpgeM2T0orWXB9lLvz8RRYD7kbKXz3UC+vP+/ffZ7yBxV1fndxvYrz9ex9qxH/dY5h/8/xf3IN8ZuclSgGNQ54VcyMYz4oQACQtz2UCMAkwRWCGYAIQE6Cj4vCgCBUhlBCBCoImcO3dNIKu4nOdRaEQ1BVQg+vlbAtucTQCr0UqINjiTa9wgTqA1bg3ry7PACZFhjK6ynQAJHLONVcCbYH9jeDVy4A8KR5GwaSKUQhZhEmQzopsRrveCL3IGpAoQCfI1RIdlk2BYVTcn/JyXgKExLE49MDYA3NSpEmRZ4NyIlCHxWmLAagqMcNtHuusQAInflrGsk11zDMNZpoIlUwHYDgopjMwv+P7khldtFtg81IQGgEi4Y7UK+YDkAclvLNyogQGrQgwjz5meL9EOKZEgOmkGA98nimAnpPBTeKwC4CmA2LFY09KFGLlc/7K7Xs14CoCjRwAeeWZXthBN8jBfg9LlGhjFXkda2fbgy7e+AIeL3Z99RYHtXy7sVifmyqggYZbzU7oo51ejGXhNy/MyQqK8U9fXLddb8YyeDQK9/We1PPaeD824j7I+SMX31l/dzmb1X/8xL5lcfuu25PgRp9//9fev/27vuuPr/V9z1wAiCqQhdjzqJhHGnD1EN68vCRAirQnCYZ3T8DpTAOu5rnHALRBUAWgrQRNBfRBkez4deBjrRSIhgNzrAkgalAYMXZVRRAb9wIaB4AGeh3pzYp0pNcbWxpgrVEMSdwCoRNsrgT7K6FnWgt0AibVglOn0Z0AQWyVXmTLMYqa13ix0Ns2zYxO1vc8WmQwjsA0moF0jFiBKvBe+f0PYBQRA1YGnM8BqhCVMq5LXsH3mbiIzWdgfODrfOD3JXK/VSto9wKpLao3nDz1ijzaflbjwuc/7HOP3YRgRmE6AuO9AhUQ9uZ9uxdvXxJhBFC3wKzAmBSzyneKUL9XAx5gCR1w8EMuMV/Ye8XqWmJBffLAPCnzTJEFaqvbYqltP/YaV7tc2yhPljzjOJf/PNmdLAsJ37uMIdfGQc2d0jVo7gvAap++r4DlfNyAu/u8/s7FKa6M4BqrXx9Ly9nIAtfIo+8uZ/zkBP26/yTs7HMD9U/CkD8Tz1wea31vnjuHVdIphktvWH0MegSmdD6yjUk3etOoCFnMIgtUDXoAvcnZvlsHGp5NJYhiHri5fHMGIIQPZjtGiKD7FwEEjrg8m8dt541BoQOACZCehjH1gNR82BJpRJtWUIlgsxV0rWDONKAz6AFr5nidJ8PSW0Fj3mJoOAiiQQJVeM+Y8LFnnmWa+P+xVwwnJi/p5fI++fMpUz57kpPPQoMgCfMFxaCH5VAlB2aJ5nkE5h6YT4xadAa99po4dHCcPfDzWQ1CMqcxbFDyCUEIncSNMKpqzHv2CD/zPuusXGDcplgeQlUJsfm1BWCuGR3NMzBNjEg0ffv4Xm/fuwFvhNl+BK7OAktUCr2bYhzN+M0zExlz5kCvakFXm8ciSo9lBXK7VMCaxVI/soDFgD/aSqLS/4/V/30VCGtj6aPKbPQjN5zeujzZp/9iuScLm3m9xVsWDjSI8m+rpb9EBSsmgwiQMzFwZIOnVtezsuvFO3LyAbAy5Oop32cM+frEf4PtOW/8N+/0RyP47H50xUx4/CcF0kysNXSCphHE2lcsQU4ZeeIYVDUDqhyLMQmmZNH9GcREG4HUHKeh4ZifZoM3sqKrgToIXnaCOtJI5MRw+mCJzWlWJBsYdS2EDGvQCxyVhikp5oGLC/qVAT8qkZaK11YJUL1m4vHqJqBtA24icF0JRgD3wRyjDOioSBEYhfDGZmfeaqTHKiKoQay/i3i6mTfmcE7OhBbmkQa8PxAflwpkolV44jXNxgRJCagjEBUYRjJzNIstMus5TBuQE5OIw1kx3APDOxpvNdgjVkC1EVQdEFrem+nIc5sHsosQgeqFGe/Mn9gI2p0Z8HpZUDURrjkdFPMJyEnpTMZloZjP9OzFjD2j4iUvMZwVzU64yH3g9v1i4CuP29kRoks0pjB82wxLgoVHs4deChUgKEOtDBrxtfVcG103RI8RiOcMePEE/ftrA1gsHy7s12PDXDxnXP79sRH1X3zfCt/nynSa5y0C6K+VObw8l2feuPy4Pr4n3/b5b9/Xn6bNaV/zbB61LapJAQENt9PfPDoi9ml3wAyKQy1iE19WA9FZUiKCJtJ79e/750quxT9vBtxYeNz8l+LSoSTFymdX49jP1ZlJsRIaIVne5/lpgdcLDBdotKSCRYI2NpU5gffRfCUs16vmFU+DcuExml+IdNrk0Xcda8h2fevzDyIlB1Y8cfPUlx9baLMutEtdzksizFHCRXQloKctAQgtjVRI9ixrFKpoSSj5OSi/r0nLPvyz4hCQjZ1g2KhHe74A5GkZJx+yfa8GXLBgXskedFBLINkHfGAHKMYReLjPmHrFNCimUbG/ErQdUDdAFkG2pI4KvSNLyBv3lf5sFDyxZeu5gGd+Bx45musH56DD6gtPTRrf8VDr8edhbycsC1fxLOx4EhYj/vg+Aj64v8WQPmNnL7BvuXy/vK7DhIsjrv77XT1muXh5ck6Pt+/skduJX8BYa2P2zH0EaLjv32WcR0UQgVb02poTDdXDUTGcFGlQ1AIgApsGqCs6BmEgVl5nYsbbCmhrQciATkASRVCBQLDbANc74uznXtHPwMNZcTgrskEWUQjH5BHm6qNQzzgmBLFSZAgkKyTx+LHhuQUQg2+vBfWNQLagQRKgEkUdeG6kNCr6QdGPgEag6YjP7m4CQg3gSpA7MZxYmWgcGfKnCsD+cmLFCNStGeyZ3u3pXtHf8zs5cUzXLZOXbgtiJC+9qm0BVf74VlkSNc3AOHHe9EdgEKDNQA6EfcbJYSjuU0E7I8IkbtXx+ONgRmIEwsREbbUx79k+kxM9eOfVKwh55BmETAYa3+GkSD3v9/6Fzdman59nJkcDDOePPF6IAHogH4AMXeihH7B97wbcPe+LH/s7Peq1l65IfWaG96yYh4wUA+oUUGdBjjSmKsTCPZ8JBf+fl2O+z8ytbdFju/Tsd4rXgsWLft/mHwrPf96PV5w1WXlc8ujn2dP4AC/4Pc7ys8b7W3bzhEny+MZ9wPZdfPbHx/uwIOT5JOa3HTgnRX9WnAbFdlLUWVAl4DzRaxoGxTjSs6ssKmoqQTDjiqQMtyGIUdFkwQZYkmurYzUiuG4JuxyPpNKNk2KYCJ0FH6gKIGuBANTccgEKx38hRy8JxyBAaLiI1Z2g7khxdCcghAV2zsk44skogZHc6qoRtFtCB1MrmGsa0ykpJAH1SOqcTk8XdeanAJmAlAlNjT255L6IBltkokchsno/GrlhFZUAPG/ExVvOmecDBaQWxJHXkswL9+9kZwTY/mPN39OM4j1J5nV3e4FUTP4i2P588Zw8UuOipBOgPQ14smOHCGz2hE2TJYTF4CARADWYgPXs92TP9k+TBw7hQEHE4pViOf8AM+A+KWZFNSvyrNg2zODudoKbSlFFoFfyasdRcZ743dgGBLvKYDjdY+xTgKc3bTVg3nfu5YQvrPivuuBV2PfMx8XeZBSyFN+sjbfCkr6Xp4tn/f5HUIgf+znISOTydwWeD42hT8695Br8BL+Ldf6A7TfHxFce+GLrnmwiQB0JbYh9J2eFWnVmXQHBvNtcCaoM5EkxJUEK9JSD34MMbCJw09IYhFagKpgO9ODnCJxqwZQV5xHoZyBEwX4DoBLLeQCxFYz+0CONx2xe+Jz4kw03TRNQC2mLmAE927UGvw6BzkAISh5kpVYwZnmkyjzGxn5qIFniclZLKM6MNJCIF+eZybfH93oaFOcHRstpMP76xOuQaJWuxgZxnr3OiiwspklJihFOCRjPKGwVh4haK4iJIZDl0lqErcbJnm0FNBjDKzyrlsfNmUlO368IDXizwYLhg8nJ4aileEgzMA9W2TkDOtGLrzsW9zQt96eJ+yezhZ819iM0KKaTIM/A5KwX2/eHbt8zjZAGXOPikQLL/HfvIGRAzHhXo0JHxac3gld7QbsBrhsAUXE/AufEMOZwzwe1ew3UEiA1APMeZpuMEDM4WDm2hv956k7LP7g0Sh4trsdtfmS1noMC/LWsVJeriRuXkGF0MWFS170xQaGePbvPC9hAyvuPiywuMHwsHu6FEV997Nu2S+joSZ3rB+zhw/f/Hb/57D4U6/t/uQUhR1szx5+qIidBnnldTW2whRIXnTI9pymR1zxmenGYOY72teCTLZOG7UaQZuDrd4rDgVS9B82YABxmxWBJ+RcdOYVa8RQ7BeYoxANrM47K2olR+TPPCqkz0sikaC00BtO9JWwrGl/y1Wx8T6Q6BoMFRIDcCHILaAugBbQVpIaL1azEsGVQhJ4Rx9QrdFZMozy5ncMZeHibkWcgnQ1umADMWIxtvcAlAv49ZcU5CEJlC2ficfsTPdm6ovF3iYMYgc1GyBdXSh2MCoTJFikIUBNaqmpGNs2GCeGxV0wnPu8m0FOvGkG3g8EelhM5KvqvLYoaYYlZRmJQY8lE4Ora2D2NjYMJmG7N2B/Bey9cdEUYlQRjwMy9IrdyGab9iu37hVAsXFobp8d/B8AkxMTKs5CBShWbWnC1FTSdYGPc1skM3UmB+ZShIpCoiG0mrrXjMZyosnYW19H1ynRDwDDowgAKHtvysqM1kuDXUAwsVkZRUKhoF9fsn3nu3GR1vA/YdHVO5b1f8f0LI+7nsl5gHn/5GSOoj97/dSmB32a3L9bURxf1JJjSy9/fp7cDmGFZe4Tm6bpuhgihkbBKiqmSFQEhu0EyudyVGA3R+N9NFCQompoLQRVpTCJoPCCCpibzRQFky+NUUZAatZVbzYDTiwvK8wsCaEcYsRYa8BQUaIlZr/Qp4DiuLJfABcOSlFLz9yyLtxjc67WkbNldALnuj5kBygVkPht+bNojJcnzzMMKdr9EwITsbB7pzB9MfHZ1RcZOWwl2LQ34tmNV7ZwEc6IBnzpBI5QpGJIAhju7xoonh3Pi8bI9/5wVeeaAysY5Z4LRPPCSqPLnv9xIv6/ZpEEKlXE0bNv9tWxz3AqFyj6/o7PyvWPg0ZIXc8BS8QRcZPanXjF+k5HPGe2QIVnx+Qvgd347QKIYliV4lTmw833Gf/mzhFOvuB/oJb3+ccQP/mxE1Qma1wHReKw+m/3GO5XRz0PBkM4Nqicy/asFo/bJXL633sfqfVn9Xx59b/WlxbtfyIT+874cx9qo0RBzh/ro7xffeWTgUb779Doe/+3DB9szK8mHfOVDt2/Z72Pjvd71c+X2VQR224CqZpFNGq1sPpO322wEdS0YRnpnY1YcBuqVdBWLYqICjWHkzSYAlSA2wHZjxv6TgP1emWS0KuRtUr5uBNsNtTxOZyXs0QAQWbjeSsGtOQOHGbibSJ2LW3r9UTmOp1Hx0DBB2Q9MusGMdFWJVaIKNAKp5rlV19Qg6TMLSzKAMNNg17OiTYQySf1VoONzbbfy5DEM94rDL3iPWZwDYKLRkcyFRcyIIVMM63oryBk49kYhPLEsHQrURov99Cbg5ppJ4JdXAVUFbFp65GQQsUT/sA+YJuDt24y7O8oVHGZSM+esRoYw2qBysQoiiPekCEIWRst4D0wPfC+awxmEi4GaMQ6wCEKd5054abLowxcKyJKUdc73rxtlfu8YuFOaPIF58TcrDtAEzGcFekVIikoVu1bw4oo7cKNag2XynQDpmDEcFO/eEV+sGsH1y4A6AdULNVxZLrxax8CeUFoDE6PZP7haaT2A8O3Jc9BLA+77+DavWNzemZfg342Pjv0eR+YCw8bqd1kd6MKDffR9We3jfX9b///JRX+gof62aODbFp31/Xv27/r0b/poh++DUEQYojt7yYtMklgORaRQyJxeOCdyu7uaxi+CbIhapBS6hCD06gF0G+qSFEE2pZcNAFedYLcRTJMyGQe5UBL00u7ako05UPFPALQtE5cxE94ZKy5AUyXmaVoxUZBihNwpyRHQCpAGLEAarRJT+QNLwlUzMfMovBdihTChvhzUqoQapiMhvWBqjiGDNNjVAPYxVAdgU9OD7rNh5gPIqQcNZx2BbQCuWsFVJ3i1E1SRBryKnozlPrZCYTAZAYwZ50RcfrKEbanWTKs5I4p5FExn0jydAun4tAQAzXLeYlFKib4zyMnvgd6qPpOpVvq3OIfUhuPCavm2yPB92/fvgQs9lWllwMX+CQBEFRFALYoE6iukWXE+ZxxOAUGUkpZCKck6Kl7tgN/+LODN24yf/VHCm69JkXq4zeiuBT+8rdBdCa5eBeyumeQMtSwhHBbv2j1l91TLw1oZUjeSPi6LY7+yvs954Gr7TWor8moCxNXxXV7Uy4mfqVpejuH7UXyrcTTn64kh/1XbE6OrTxeTJ5CGj05dFltdf3b9OVz+YXlLyr9OM30man/6v+dWuW+ZKHMC7s+K8wDMQsM9R4G0ijqyKGeE4P5Bcf+OxkBHRZUFuz3wah/QViiFOW3D6sUgwOFEYxYbQdeh5CZSpseWE0yKlhxtLw0PlZQEPM9R8eYeQK/oArCPhA9eXRGCcYryPCmOL6jkd/dOcTxmzBUw1cvCMYyKnIVl3Bb2q+pSJa1gNScUMfNHRTDFZbELz0AoIsB2C7x+HRAixdZCsAhXwWSscao3V6QrXreCmy5AE/ByI5gn4JtfZrw7K+oIXG24QL5qAm4gaCYg3ytS4GI1uwE32dkI5o9ebQWbEDAk4Hpg0vjQA8OkONWKOzPoaTKP+wycMgBRqwNQjAPnfoxAaFeet0U7Oq/YbZmowqYR5KAYVYpEbc4LAeCC+WX/Ph7Tv2r73jHwaIapWtH8AJ+oFn6BnFtVGmIdM44nxcMxIwZBG+nRtJHyn693wJ//ccA2Av/P+4yvf5rw5mvBH/3BjKuXAcNBcfUyIPxOhX0lkIYwTBF9EhQ5Vz/PYvSwMtyrzemPglLlTK/lUWykVu6v4pWmFEGaHu04ZhdNWjz/aAvHcwb8IlFpr8Wr0NV76xuMVUn9M9f0nJ1b7f7iA5dG3A9+aagzlhRnsaW26pTzf24rq6QbbXm0l6U8eX3yjxeaX3lxoAG/OyoOZ3pjGcBY01OrKwAN0AO4v1fcfmVCTZWgCop9FfDJXrDfCD77JKCphUp4E9APivsjPeBXrwO2W0uIzUxkzQMX6hiZBBMFNh2vkxV/UsbnOCn6MWEagS4KtFJ0reC3PonYbYjfh8h9Dz2P//VXCfe3gl6Bh6yYFLgdFP0E9JNCzPhNluAnT5wc8XDmWIyiVB6MiqlhSX5txwqPDTgoh/vJp0xQ7vbCas6GEJQ65h6AdkPmxy4EXEWBZHrvaQaqIzB9k7CpBZ9eBbQ18LIR7ESgIxUDM4AcMxUmM39ioFBVjILtVhCuA4akeDVy0b09Kk6j4r5ShJF25WzXP50ofpWFi4JCTTRLqG2yMYqjGWVJjGq8cAsqqANQNYoUBZq5yEwTLhKU6yh3PS0/1JkCvm8IBbg42wsPFlqMaBOoO1AnAHuBTgFdZ8km9+iyIo3Eu2Flt00E2gg0JtI+94rhgRNvOiu6DRMgzVawnQJizTC0qrnf+jmLbZaqRAsrb1eMwRK8Cm/WwkYomw1cFUAqMYx98cbzihEQUCAzFLjpT+CWr3FvR9jXRS5rfGJ96gz9njfi/rls+3SD7D/ZIQk7UF79zQ14XlFJH3vx7uHVUdDWFHPyghTIamFYXYef69Mb8Oh1vQkZDjEQK/UKwRDoZU4zgEExG21MwLFWVWSvtJWgCmSbTFgqDcV5yOpFM2KWh56c87KhNOgligONg//HGSMxCupKCzzTthTIqlzxDtxnZYm7rhHMHVCBAlBjJq6vWOQAggISOSYnMApEpsKilOfOSMqln2M2T/0ZBtZmL3j1Kdkhmy2jis6KnjQwpyDCIrwYgV0Q7IKxrhrS67ZdRlML2lqwbQVtY1FNTahkcp3x4sOupB9W4a4AiEJFyAhg78VDKgiJUMuppSEfLemZoBgAJGW5e55IXd5ueC3RWSsQkFkq6ExalspmzJMcj5QbPh0V5zMjnX6wJh1p5WBlPA0rf8X2Kw24iPxbAP47AL5S1f+6vfcKwL8D4LcB/CGAv6Sq777boR8dJ1gCBp7kgOkrKrYVsH0lwByQ9jSUr38UsN/L0jVjVpzOCiTFfAZ2UXHdKD65An54A9ydFLf3ivuHjPuvEkIU/OwfzHj1WcT+VcBnfz6i2wpe/iBgdyXYdQHdFR+Q2tDNItSJcG8YK9uelToLWYGzApNCJoUMSsNuk1caIV5YsaoNFv5OkaXBszMPwuLR+zN9LDWw3h57oKrPj4W1k75+d52CWhs+Bcp/1jb1sfF2tGg2Dygl8vGz8jVlYrZDIrY6mTZG6VyiWkrUL7xxsxwSCJNdbQUvrwlP3GyZuEIQqNEiymL5PuO9usbncPAY6PkqwATXpIzstjSMp7MiPQDpxKrHuhK82gq6TvDJPuDllovh3TuuRvtrikUNE0pnlsO9YjxbkjPSyFdCBktOwOmMRS8DQFUzwqxqoOt4/zYNgJ0Ytk6DstvwM7MVnUAZuTq8sm+AHARzFIyzov0m4/4IjDPQj4R3mIwU9GczMjOQxlwWmQQuPNVMiutWif236fJ+hwD8+M8EfLKvaeQqFMilCmK6P+uYjV71tQREBeokwAwcv864/WXGfiv44SeCrjFDXoMNLhzGsP0jCL3lAGQTU+JiyKrW64oT9+UWtkLxWacEnI6s9u5nxXGiIX/bK4ZZcTjSALcbwc1rQd0IXr0U7HYB11vBp9cBbS14tQ/oGinFh+MI3L3N6AfgZz9L+OWXCXf3ip/+fEbfc799z2edEyANvpMR/xAP/H8H4H8F4N9evfdXAfz7qvrXReSv2v//9Q8/7OVWjIIZxaKLYBmUWhQb0x/2UGXTLbxP11lII+mGeeJCUAdCKl0NHAGuiNbZRMFy45RZXbd9IRh3nIhVAFpRiGXWvSJOzIQJFlVDAfFCVUYBmriY6EgDHsZcIgnH1Z2SFazEtopSFuD86F5cTIoL6/l0K54rltDsKWjN99ZsGr63DOZi4GX1p0fP67mkoOO5LC7RIsI0TvQ6xsRilZQ5UTzD71hiSpctsC6scEBJdm9aQGsyNSpf5VaLydqIP9HZsgv0BeexEy5imtROo4PR2wxYTpniUpIX6K+ppEjE1tGuf1ruSUnG27XNI6CJ3kpVL/dUTKI4zXZypVqLYbgIkJIUiKO2xb+urSAmLAtrNghOoEa9Y7GOBiBXQD0JNpVgYKKJ0YYQkgkCSKWQWfispDDmymLticyYLWn6DHd5sxNsP8HCKLN7GwPVGbNhbK51shXBDsTwm5kGfGMKiE1N4922YhRMPpM1TRcil4l6XcZYsAiHjRfM66+Jz9e1MV8aesrnWbCdaLj1lNHPjMiCKLoNcL0PaFrgk1cB19cBL68CPnsd0NWCT24Ctq2USthpVLx7E9CfteSd6jbj7QOlgYeZbB+vKpWIp/P1W7ZfacBV9e+KyG8/evsvAvjn7fe/AeDv4Nc04B7qlTJ6tYUxmfc6KJqs6Cyx4kL8baShnRPYZmoCKzaVSRtWSClaAbYRGBvFtKEXGAPrGfqHjGFQPLwVnG8z2g54+DzixQvBZ78V8eIvVGg6QXtN2iFprAyd1Ut3MzgCJyCdMqvfjpmJLVVswHC/Fj7QPABpEmglyJl66NVG0DYByYwB+bdiDS7o+QOGz7/HA+fDwqXrXKyZXL73vu/6Dr7F+PtCs/biU6K3NifgPCiGiQaEvSKXw46JdLs5gaFkok5IY91JWvOyWTzB5NpsOOO5t/1NCpkEm1awqwKiCILQQ/UTLNGBGpTz+CIsyVUM5WqrArBvyaM+Z2BQnmNricT9S9OoTkBlHnUerdBkAE49n4977LGW4ulLYi3DwSr6dhtB3BPfrhpi3DqZ9okxJlQZtsdKUDfAZGp6s8miqhCKyLawQPiAJBMWGE/cR9OyiGUyPZJxAqZ3Cj1xjrROBDBm1nYnCFek8D4oGyT4gjQn6owAHPqjAu3j8SRcnOpOLpwRhx1UBaV40057Vwte1IEqi/eco00t2G0JnXhP0IeREVs/AA9HrhxNQ4x9DaEF5biakzlaVvzjTSKqls5Ta89IZ9KZNx1wY0mtP9tF5AAMI73pbgO8/CSgbQWvXxEF2LaCFztCZ5tGUFt4LoFj7NNXHLuffBrwO38+4qs3GS9/KLh7yPj7v5/wxZcJ88ykar1nW7YP3X5dDPyHqvqF/f5LAD/8dXbixltESwODoFzNJQFhUEivqMEBEoQZfIn0rmMQaFL2JBwZjlaBD9kNeCOKTQSmBpg7eoE58PXhmHH8hqv57c8SmhqYfxzx8CKgPSjm14LGVtsmSqFUueh7TgpMplExKPQhs1vIMQOjIlbAtrGwvOb5jjOYlY7ArAFaAZDApGYERgkcMJFdXtZFR2vY5smml4P34kPFmF+++vy53N8j9/w9Rnz9OiXrazgDDye23eJkV5vINH5jYsuweabWyGQ605uKFLumwSL2b5BDNhhm6BXnnsYSo2LcCD59wVBaLCwuhntllZ0GVsacOP6+QDbrLQZg13DBlUkREotuGvOWrzeUL64Dx8TYA1/+PKM/Eyrqe6UA1J785GgzTG1M66w43VETRG64CFU1ZWejsOBjHjmGT1552DIhVjXUjV4n29XGc1ZfOEmnrYULwP1dRs7AzeuA2FhvyTvyn6d7Bc6KphN0O97AbNZ2vxXstkB/VNSnzDZsFi2NApxnLRHjCAsWVmNFQCPZdhbXmZF2T7wUVKlBhSLYdoLrbUAerZ+lEqLabAiZ5WSO18hWacOoOPT0bDcAaiUU6IywaOJ1OTFCDwGYLU+w23OxqiPL8aGA1sAsXCipHQNc/yCg6jxJznN59UlA0wAvXgRs7dy6IiqzmkLG9HGd9s8+zzidFV9+k1HtgTfvMu4nxWHOGGZAB6DeglXjH7j9xklMVVWR9zMYReR3AfwuANzc3Dz6oxtwelEhcBUce4UOiultRjow1KwrXURgArAJAZ0ojicKDbkQT3QvvgLqRnC1F7x8QW9oToomsQPQlIjv5Yn0sF3DB/miBV51wE2j2AHoVNFMimoASscTo46xKo1VeTkrKtGi6Z0B1AoEVVTKh+x6z5WQd5tasIgiGlUpCc5zZsu3jRAPkyXDXzyZZ263rn9xA7023BfP7JGRf2LM9TI2ddjh0fHmiV7fNAHn0XSrlWEgI6rFCw2WvJ0thAgV702oBZVV09WtVT8Gdn1JmYN5noFmEMxZ2YcxL17gMFFhbhYtuKNggauy8vPre6SqxWN9XAwrQhxZRdE0YqXbxgARQQA98DqyCUJQRdvSkfBGuu0sqFsaDcl0ONJEBb80YinBBsdpiGASPgnGwUvG7ftYxrJr+mQlwwTgWFfhdSSLfLzxgav4+U9JKKeFzmaBSgmr1I2PGcMYeGwRPuecLE/VWs6mEuRy058My4uxqkBpm8hDEkevIhN/XW2NJSxySuU8eb+Gnt99OCt6K7DqJ1N/rGA9YBXR+duevTJj6lh8ZYVMlendzBPv/2xQnwRbvKOwEjRyDMeKhIjG2Tc2vzRxARMsc06BpfmMLSDZYJK6Al5eCwQBP/w04OEUcOgVOCg2rZRF/0O2X9eAfykin6nqFyLyGYCv3vdBVf09AL8HAJ9//vnFdBF42bIlLgMwHhXn24zxIeOrvzfj9E3Gj3ZAb8peYwA0ALsHxeYVhdBPR96cYLS9qw54vRfsAfzZn0RcbQRffpXRxIwpAzdKLYlNyOiUovovt8C2EfyTnwA/eg18dg18FjKz1EcFBma8gxH/JyPnbyri7Ehg6XNQDKKYoWiVzUprC8vblndca5tgLV8HAUaw0/nXt4oZgvAjIDSB1CXD+iJAj+U5F1xREk1PzHuZlc864k9svIL30Se3fway0PUUwOGseDAVvYcj8W73XqoK6DoaAFI0BadRoWdBNSnOkyKPgmYPdNcctF6IUqeFkQLQ65wFCDVzC+OgCBNwf2JtQBwVseex2gqlS3gASsJUVYsRcz7u8bh0S/ctRmC75eIRAr3fUElhHVQAqizY1IIX14KxE/THjFMNHM/Au9uM7U4gIaBtgW2j0AiMBzKgptEa3o40kJ1BLbMVfNy9y7h/m1FVhDGqWtBtBNsrKcnhnC0xnBgVVA2rNCdrRCANEFq1pidSFjHvbjPPfk+W4eELypztuXesKK0rYHclmJPgfGTD36oBuj1tfp9YGSxPqt9AYkI0Y+0djNZDEjTw2zZgUwle7AJe7gN6UXw5zzj3ZIXkmdDbMPJ5fXmbcXvkvkMF1DVQxVAWOxcBy1mKjXH1wXYrqCvy8DctI73jA6OUwe5pbY5eSkD3oMAEdDcG5dSCbcVFJ2RCopMAydve2fX5s1JlxJkz4ZScgG0T8Ds/qXAaqH/z6gcBv7xN+AdfzGiCoOmemd/v2X5dA/7vAvjLAP66vf7tX2svxQNfvEtVho/jABweFIe7jH0WnMxwD5GvMgBqXNFZaJ9moxBuzJsIkVzUeRAcj4JdJ5jMyE8ZOLXA0Ag2DXDVAdsWuN6ynHffAF0g7coHd078XQwD18wVuROhFxSNkmVLs7NGBKuwtyLGpRFUewvLqh0B5IHttqqZIbfnBy4ggmcxFG40vrjwwktFpa6Mtyz/9+89t6+1AfcJ71WKo02qabaqQGXoLpETqTLvMpoHG5MgRMqQqhiFEov3UqoflT8+LlwbXYFCvRon4uIQoEqCOCsTjY0ZWiFvOeuSFE2rZGlK9OKfJGjF9TiIl+ZEg+AeuCaem9/XGJgEaxrgeGJyNlYGkQBoYLrWE4rYv66LOcLlgpwzDWzh78OimWiR3XxpiBEMmlk1L0hJipyqP/DS3zItSUPitFKosOtnvG5cEa3vYayBaNWcWjFhGEWLFsu3jkr37rFqMG6FXTF4ctOK1UCveBy1QGlq0XKagWFgziVUli8DjWQyPnahOvo8sOsMgeMyVsaVt2bRXvyTEhPsMUmR102TccFXVEofjGpOGys4ZZmrWJKnzAXpEvkkSxa3nNDXV4IXZ8ExCTb3gkqZE/nQ7UNohP8HMGH5iYj8DMC/ARruvyUifwXATwH8pQ8+4nrfWLLDToJPpsd8OGT80S8SvvnjhOlHAW0d2J7qiuW7oRFoK6g3wP5VADJw+2XC6Z7eWXNSVAL85LcjfvvPRLz6acKuTZhGCuZPM/CqAu72it1G8OlrYnC/85OAT18FvHwh+ME1J9iQWP3ZgXgjgk1qAXZb4mIpWdHGrHiXBEd72J4nO9uEaCPQbdkFu7kKkEqwUSq99VlxPihOsyJuM0IGun3AfpUI8vv2eHMvS2wxc1mA4j37d8Oyj8fG+6JrCZYJBwApk5GQMnAynYqHo+LhpEWUvw6CdkfP2xcrOKSk5OhPM9CPwO2D4niijsg391qOB9jENSMWAgf98Y7en8wKGQFBxs9/aayUvWCzC2gb4OUVk6L7liF5VZE3DLHkduLrOCqOB/5/vUUAW7tvsaIBpi50QE7A/dcZwzHj/ElAzoH5jT0x5NOQoG8U5wPw8z9MqCLw6Wt66lNP46TGBmHTXhrkUFNNTwQYz4Lh7NRAPq+UFNMgmGbFMFrS2JK69bxEKzpznw+94jbR82w7Gq/zkbmilGh4QhBs97gQ61Kbj1A6T8eDommBmxdk2LR7evPDrDhMFnHNhJna5vI++mKbsi3S+TKiIwGAEEoUyzGNwMOt4u5txk//YMY3X2f84mcz3nyVEMBEqyq97rZjTmo0GOxwpIO13TiPn9FgCFxAq5qJ0P0uoK6B1njgky8sqx/KIzDhfH+fUZ0Zec4a0A0AgqKuFXXL8RUtP0IoR8oi5LCjL5zTyAjMaz+gwIurwN8b4D6TBNE9nrTfsn0IC+Vffs+f/sUPO8S3b2uKEcDBNE1kKXz9NuMXX2bsOsGnr+ndNQHETWthaNsJNtcByIq37zJ6pQD/YVDsGuC3fhhw01nXkRM9o4dbhv3bEHDTKXY7wY8+E2w6wU9+K+DVi4CrDXCzAaCWoHOM0Ao6GvMur/aCzY5siZMJBp0Ogn4AYDrGCsqMqtHepBPEVtBdBcTa2C0qaG8Tk3UDEA6KEBWxVogwg16872csOJN1i4fgyTqHVvxeF8Ouy648AcXJrBc4nu87GathnoGHAw3JwbrHNA3QbAOqhlWEjmULUAx4mRiZnrvDLwe7VsIC9Fj6nkaWBpzX7brTIQGV5S9ODxnzDFy9EFy9UGw74Pw6oGuAlzvBvlPSTQ2CmidjZowsqOj7NVTDLYB9WgUAojFODKaYJsUvHxS33yggim6raDvBy1fEiZuvSPebBuDwNpONEiKVBo12iEg8Wex+e3TQtvQKu42g7eSi5ZezL+aZ9yglxTBYaA5i7AJGNlDgfFKcTmRMbHdMpt4dgZMttq602HWk5Y6D4nzi0/Znxvuj2F8LXn4a0HRkbYQKTEA/ZKQMNLN74Iuu0HpMuuH2qIVDiwuNKOhxiuWFZuA8KR7uMr78ZcKXXyS8+Srj/jYzD7GFabnQIOdZrRExPXJRJig3nVEcjWZZmb1oGi6UdeV5DebeLvJF5byZq9ATOA8bOm0pUe+7ash9rxpL1oJzNAbTUMFi05izodMwDry/3qN0vxVIFTBA8bpnArc+CDPDH7B9/6X0FjbFTBiibgTbvWAeAl79IGLqgVefB1x/FlFvgO6TgNgJ2puAdseyXFQAkqDesvhGpozjmWHX+Qw0lkjZdmQPhKSYJ+JgNzNLbV9/GtB2wP4qoNtwsHo/vxAF0cFnC/Vq0yF2cX8Rlh7nCFSdoDLxmzwwlE4TPY55pMZDqAijwJrDRrBjynYvSJFqc8NRsdlZfz089abLtjKwXsEpYvxXC9FTMIzuUUIUQOFNl67mwIURV5sgxzONxuFMTzqpom44mL3aMGUaXz8vYNF6eTgqbu8UpzPw8EADLpH315OKmsnLn0pjV4+RQSNoUUCaWQaeZsVWpQhNTaZJMdaKMdAbj5ELgXvVXjik+rQxBYDC2Y+y8JYri7wo98ov9ca2GXpYhx7F1c7gi4aYbFPzvN1YZ4vmXGc6zTy/rDzvqgG2ez6ZIGosCja87UdSELPC9LDpGTcV4YiUFsgoJeqYa7IOVdaUQszgu9KnGNRXWbRU5FwNtvHFxJPpVWREc2WqgWNvsA4v88m4dDmKoPaZrKUiFbao58zrP54U97cJb77JuL3LdBR6sryScsEVkdKR3lt2aSZDaZ6AEOhQpAZoN/RWYiROXjcswGHxF5k7ee2omGMzJy3w02isn25QdBM97ezODtxrV+PnK6aZr77P2RyGeSYVcRi0NElW8N75WGsrzp9vh6Mut+/VgAfzqBvDt0UF2y0QfkCe5Z/9CxX2LwL+zE8CfvTnKtQbYPdpQNWIVdJIwYc1ANuX5GufvgbefD3jGIBXGyU3WxUvrgGdBamjxx7aAKmpKb5/wVV60xHTdCwXmRQuSaygDLUPbBYBVI0ANSdfHQCZBc0ZaJCRzsRZkammmBWoO8W4V0gt0FqALpQy7e464NUnAdVB8Yuj4nBI1HGwtcMZE4+fr8KgiUEvcDifjO7JCpbJ6e9fQCZWJVl2igUHvb1XfPNNJvtj5rnsbgT7a963aHod07xKGtp3R8Mx390rvnxjVKqvFIeThQFOt3KamrGKUrZGBeDgrgIxzyorciIEkxKwV9JLEY2PPrFdmWRO5qoOqCtGB4UVoE+9b79uSUAw5lCwELkOFLXadIJxT5We+zugaSi01LZMVv/gtUVKeaW5bTBHnonhTiP7MI5Wkh+CFE+82zJhqlmhI7urH440UMez4u07si4+/SEpbE1HRyRn4DDS+Myz6VBXijQRrsnWvzGIWhHacn7UXwEkMtEeItCOgnmiQyWmnEiqp6CpFftNREqKwz1rKYYE9NMjByOjNG8Idm/nyeaEeb2hsmedBN+8zfjqDzLevcn4+RcJ795m6Mn6TQqjMIgi16yfUMtHZRXc9cwtzCkgBGCzAfbXHFt1w+fWdXTWqkjnan6GhQRlFD3OWqIFCdZpZ5MRYiCVM/p3xWA487yzYvJ5Fxi1nc50SE6njL431oypNybTmKgF2HfCnp0P32I0H23fqwHXzAGdjbc9Jw5yKL25thNsdwzJqRBFq+NC6L4IUxDd9pW9jJv7OJ8UR6C0nJLA1Vgg1rxU0LSC7S6QrmVVXsG9T7d67sEKvQDxpKTPAllC2VBZAcdAQwbVcr6OOZIYwso4MZc5xIXmBOWE02wdwvFcp5tlS4YvF0gKKFoTvG3L+a/pmwsoiUJ7spyT4XccqNNE6lZK1oVGiYu7F5JtELsh8iRYzoRb5sQQ3qGLyRJS69Ci4JAOLaSFHaFOVbt07vlMRYrYk19nCEvRiOdZ/DthtbA9u1nywJ8X4rIPenK2MI3c4zRaAQxM8MquYYGlVs/88XWmJSHp5xMrsPVZoG6JwMfRcoqe2PRn7dcVAi7alK2vn5/3T5s3rkuuwWl2Ibphl0JzXH1tSfQazdAZP082c8vNzpU819orL1S8bE2PT8yBTS4HYA6QQ3Hwc2wAJKFiZFrqBqZJ0U+kXM5GLc6PrjEEobCaRYYLnVILbl8Sj6AhTubcZKV6jEKKE+D9NkUUcyJcy7oW5opco5xJUruNPo7tHolSn/05jfpv275XAz6egbc/T6g3gtNAz04TubJpVGw3gviaxvLNm8yBeZsJWXgYZokgKErHjPO7hId3mTKPdyzk2VfAvma4ebOnLOXVq4jdC/KQN9eB+gkzw1wdgPHMp5sFQJDSIbsCOGKjN082aKUTIAvanWKTiRXOowIJCA46PzOhgbIGkJ9qHGmZFWGmBnoM8n4ZVQs/39xm49TbBLww4HhqwB8tCgVSWXnxXgn5cFI8HBhailWpDQM9QxHFsbdnOhg/fOJEnNPCzuhHynjOiZ+DWv9Jg5Gc734+00OdZ5vwAHYmZBSyImZ6bGKsgZ1RwtoIXDWCJgo+eQm82Iu12jIsXQBAUdf0yKaRIffF5pFXoDd/noDNVrDZ0TjfvBQ0G8HbrzN++UUuLKS04fk1NYWP+oET1w65cLBXdnhOoLEyjy9E5lM2O8PJMxeuruP4q2ogmL74Zkdv2O85wLLzTct7MVtjhusrGlkJgrYzFofVMEQsC1xj0cams+MYtFXcdOH1HI+MajY7MSloqmXSP1rdSwUrVA+LwQeAVgV1NrmCilWLzcBnOdxm3H6dcbhn05bakoyz2jlY67Uf/STi1acBx17x7sDCsZ//POF4D9yegeHrjM1GEBtS/2IEdlvyHH1+TQPzMawnUEpsTMYt7+k1K8z4B8WLIRTnxH+cP84OP67rbSvWav4cT9z/ZBXADkcBdFLYK1StjeIzyprfsn2vBjxNitOtIp4Vp4FFGWU5TNZ2yji5p5OFNEe+hszPaOYNgGoxXMMDkzKSaMBrVbzeC+K1AB0nQd1Sh3j/IqBqBd11IOZ1YqZ4TixKKULsFhlb/qUYQQVveHBmSmY4XzUokpNeM7yGLHT51aWyy4MNwQx49mIh8Hd76M95O+NEHnIQLQY7Cp4a8EcY+Npo+3shmPYHULDaYWSpeFbjdavxib2ZruHJvRnfYQCOh4xpYnHLYEqRg0ciFgFUshTKVJWtcTOAxNB/tuRYUwNdQ3hDEq9troGcrQ2Z4cGdGfLdRrA3IxeMKsdrZ0u0UsjxyAsXv8EW0Zx7VkA6DW2zpaG7fQOcH6ivPVwHVCCToupQOpYXA14GzzJ2IPzcPPGz48woLzbAJkjxbtUWHIbyUiKRZjW+vK9i3fFamxrIHQoXXwB0iSH+MHBxUaIRi2CaADEomppjt26JFxPX5ef7nnkK199HACIEUZ+PDnUG8mBzI66gFAA1BJ0V8ETLb0xnxfmQ0Z84yYI9L29XB8sZXb0U/OCzgPtjRm6AeALkK0ES1heME2mtD4eMrILXY1gomWExnjmbt52XKGhO1EMZRi2J9xBgHrkuBlYXeFEs8gOWqBM2t3PmAuH8/TLO7IZ5b83SVFk/3HgD37MBn3rF7RcJUsE8cOPnTla5ZB61wyLAYoAqcPA4/xVAwcPT2cLzWXGyLj5tALaG00orqHaCuBWEbaBA+4Z3NJmuxejqhpn84mADVoxFME/8fLAiAQ3CH78481h6695d1Ytnx6quhX9rH+cKnhmFTAOTmMe7jNuvElkOHWGWNF4+YlWq5OldtokoxQMPZrjciLswl/dqFEExZLXhvQhS1N1C5P53I3Bzw8VUQUtf10aNA4qXWVcw/FehiQ1xQ6Chn9Xkfh1uCizeaTu56Mi0a2EsC74GAW6u6GG6UFiaFaeWIemnLwU/eMWy+pfGMthvULrcDIOWxVFkgSvWiVzfklKTQhPZNncPijRntJYMnXwiqtLjbqg2uN3xOupmoVxOsz9bz00IJKhBMPSkRzMUp0HZzPcccLhXtBVws6ERTzYvoIvxmSbDuycW8FSRC15VCfqeC2hVAcMWpiFueZIzcDzw/Ku9ScFWUhozOKyDxGgnJVB32xyJClxA2SHe4I4emJ81O7RG64XC73kTgM6iyv5A7/d4yy7245mOSwVgCECKQNUBm08E7dZeX5EBpntg1wuOs2L7UnB+4H5yZGOOMQOvToqXZ1bxns+KquLCPPQ8/75ngVBaVaf6wPCFas6kv4ZecTgYjbCxebyyS2smSzYI5eHA3BHbR/LzjpuPA48/Tkue4rnE+vu279WAD0fF4TBTgXVkefswWPiiS/jphHgBV2wmP6ntC/tMoWoIvdeQAZ3Jq84jheg7600YNoLmOqC6CohX5GTLnqt0OiVMYHn+wwOtUtd4NZdAKq7CU2NJyYb4q1qBkbo7Ay24LzXNF2pY3S5duNdesMds2USIzvcZDy3wzYYJuOsXAU0jmPunBvzhoLi3DHYdtYStbrS8hVY0tTli7YRm2tYN+8Im8e7dYX2CEoo37lWXHhpn1l9Dakv4VYQyciYUNieYuh/vAyMUhx0uo5PBtFByYhIoCPDiigJW86CYzsQoT0camB//KODHnwV0NfBqb7oZExfCeRW9ufc1TXoxXtZbznz286h4d5vx5hvF+UgPuq6MWxxplDYbQddSMna/F+pUN+bFJY8oud+qMkZKZKWdVwcOA+Glr75mgmuzyeg64PoqIPwksLTccwK22GblwpaSYuxZel/XiigBdaO4vVXc37HD/e5KEStxwgbOD6z2DKBD00UvEeeiCjXsOSqdlVmRjW7pmLsmxWAc+vOR1NypAnt3XgxMEN+VBYcnXi5og2BXUfP77TvF6UEJn7zLmCdAZjNOkR2E6r3g+rfYc3T/Y8H2M6BTwVVif1LZAfd3ii/+KOOPpoSUgDcPiuqoePki49UNH/hhl1FFwemQSdk9KU5HVmLCxr66PVlFrWNSHM8ZGYL61qV9GaWUhUmWqC6Zdz+Mire3jER3W/bxBFCE3vozj99nwix5BuUXfoXt9O37TWIqwyzNoChUUuSJuHEx4EDx7sTCSPKZFfpYRD5wqc/gBMuwQgKjKWWDKkJlWhyVQCpm2QvAjCX0cdnQ2R5qFEXIciGB6oZgzeooBQHG2JBsKojR0JQnlnu5H076H3tFf1KcDorDvaJpGQlMjWAansZZ88wsfQhA9tL7sHRxcelNiEKDLF3FfWLZRI6VlGq1UmQl9Cy7jucWArHwWDOy0HL+S1/CnICp4jOqjVHC+cx77ThrXTMjb6cGVXqThEcUaeaCs9+yGcAUBKPyfWTud9NSx6au1tg9x9GciPkqlmt6Vnd8GWr8bl5olfOkOB8zplqwUeMgZy2LY0lC2oIwp1W0s9r/mkX0OLlYVUtEczF9ZXkt/GSlVz5bcmyeYV4dqwjnibi4QpFcEzdejrk1hLaORrJ5+mkGknCh9G5BFJuRwu92fnpKHFNr7NfHs+XoCB3ZnFbRUrnoxs7huNEWz9kgKAR63/UW6PaCbi8ILce4kx0igG7HubW/EVy/sHtw4jyh96yoK3rdqaI42tAbN9vHh/Derz1wF9jJ4H5cRMtzQa7nUogCkOIkDJOiHxjFjaPZKw1o6nUkpSU6dbvyXXpjfr80QqXugLqH7dnfnniTg9p1pIcQBegq3mg1bNnxqPKaOUCSGV2LBi0RCUhF9km7C6i2gtDRmJcedwDxuIlaGbAMd2Wec9OpTXJZPH5TxKNWuUmBjkb7uucozqCh2Uw8n/hkjTWv8qx4eMj48ouEP/5pwu5dwO1dRtOgeOAxaSkhhl338aQ4nIyj7sY3LhzfddeTWpjMrTvy6Pd7htCbDfWWq0BvMawmtst6FrqfY39hOX9/UfXPWRFHWUBl6cZt1oNGTBfbonLBE3YD0xk7aDwr+mMg68ZC+xcvA646fr4/Z+QE3N2xktALKCAsX16X98+eVFptrDRF0eCooBiOwM/vMmIlePUJqWjTZGymALy7U9yfqJznOD4xZJSEdYEGDVIQYfS23QEbBZouEjNXso42W9YFhEYg5gTMAxPJ2Qy3c+b7M5BrYOgJKZ6OLIbpdoLdjUkLtMaMSoqpEwQY3h1NdkBtHg4cq+eTYgQXh9OZx8xYmh9M5mQh+rNUoL0c0xMUY84s0lFZpCWUuZpUMzIYVdEnVpC+vaP+id+r5rXg5YuA608Fn/35gG4PxC1whPGpwXn96jPBiyy4fqn44eeC40Hxx/8wU78lK375VcZxS4nnOgqOD4x4PNkuwsRsVYNiWqanIhaZTAKcJsWowPguswJ3QxiljpwfMQLbjoVT7+4z3hiX/R/8wxnHU8anLwNeXwdsd4JPP+HnjwdWGI+ghhIUaCzB/CHb91vIA/J6VfnqOBuSVwPaQ4o0OlVggip4iALztM2NciPuYdB69fdVVYJ54J7Rr4AnOq1qHqSF4UHMq40sOAppMWLc5+JNeAGNf7930auJn5vTpfNMo0VYJltSiwJdSp1zZMIuDSdO2wi2DbBrZH26mGZFP9OAp5p4a7SQsFZ6SBUWFoQEQj+VwQJ1ZRxfg0Uqw/bcPnsxhG8l+jB812/e2nvMJqC1VsPz1+TPygQrFgO+vDouLmIVuEIpUx8rU819bDouWmk2D26mFOvDgd7hYN1mcloKOhrBslCsNlWULvNubJMt5iEqdjtyyjVbvkCsm83MBcbZFRvDr50Kul74rITBErcce9ELRGZWGDaNFMbJuiHEbMVLpZvRzCRYEnLGcyZ/e5qop6N+LCv3rmpBVSlVFVcLtC+Y2WijapjwPHPRdDEshWkV9fy92sCS90/Dfn/uABBhlEhzL2cY5GjebVIh7GT6J/PEk2orQXdF6GT3glINWYBJeA3eoKHdeSKW47i7U3zzjRYo42g67aejoop8nkNPp2kcTSu8BSTqIuEcQAtp5zh5whELJDknzjcon+VcM1ncj1a49pDx9ZuEh6M12MjANAs2G5b0O4Q2BTqd38X7Br5vD1yM82qZWli1mAvBOBulNS+hjmQiRBGcsxb50tHKaSer0iPFDwyxJ1qLYWIGfhiYANNEd6AkE8HjeYOGPKt1Cqf3EQ3fFvOwCKEsrn9OgmQr+v1dxru3GW/fKb5+pxbqCfpZ8clkWLDBLQUeymRv3N1l3N5mDrBBUfVLx/IYMsZaUO0Fu9pnHvfhi0WwRSIEypzWljTd7eltX11ZSXEnpay4qmjsGVmgiPusovf3J1YeheSAN6BbJm/h2vrP6r11KO8sIg9HAXEKfjG0IZCBkS0ZmrPaJGBC73TkpLy/Wwp9vDJ0GJnkq2tF04qxKh5djgKNGe6j1SW0jWB/xYKlF6+YtJxGaxmXfWKjQGmxAnY73tc0Ee5pZkY1TFIvC8A4LZ1aAHpylUVGdTQjv+FNqWp3EMSKpBTTRA0epyDGCrh5Edg9ZksqZd0IYMJpTc1u8R5lHmbe6xD47B/OhJA2DaOeNbw2ZxrayhYVn3N5NJjj0dCIFUgQEFhzoWVMpZUN2LYC2QA/eBXw8JOIYVTcHhVzVrz8UcCLz2m8uw0X3wRFgiyJ0WUaoGqA3RUT9D/+ScDpleL8jWK4JUPlq7f01/qjFvXGnBUxKpoNCwGzAMkWt8ocNzWn0edBCkAsrDlBND76nGnAv36X8MWbjFNvTBkFDqNifpPwcBZkUTQ1abTR7k8Ol/1pP2T73kvpGzPgPVDcZXUvYAVpVIHGqGs4gEYzhFNmaJMS2QOTTSStrMfmyGx6P7KAZBzoqbk2b6FEASYwRNw7TzScaSatUAwLi7V11snEtNQeYM4meTko7m4Vb77J+PqbjK/e8mlIpdgl4DhQArU4sFDAZATGQWn432YcD8xOVzUwnjPSxEkbo2JbBeDq8l5OE3spxsCEkoshRdOB2O0p9nR1w2Rb2/BeRk9YCifkbJz6MojWUM3qucHP334P6/dwacB19eq/O6XKuehiEZYnjZya5V6iHzwGQEynG4bxHg68X+MIPDwwYXQ+kYnh1ZwA0Jx5/6qa3tZsGubrLYDNQ6ICYeY42G2BH34e0bSCjRnE89kgmlkxn7w5BCdvXQP7K3qCk/HiNQF5skq9caEZjhYdeNFNswH21my7MgPemLRD1y2daejNS+Ey6+oZRGHOoGmA62syJXqwUbE0QNgJ0gT0h1y6/8wWvXx9S0bEyxvByytWL756Zd1ufOGpeI0pK4YDF0bMQKWXeighCoJVv7I+b7nX1q4SUQT7lh70+ElEHskck9uMfla8+izgk98StDsunKGmRK5kXSI2XcZm3ZCt1HYKkYCxV/wsJxzvWF/SPyRIImUxTyjSw3UD7G64eLsqptMfY2ViVtkNrJbveY8AJvMV05nf//Jtxs+/TphVkRpGWA93GW8OFM+bkqJrBK9fCPZbwSyk4kq5T4/jmee379WAAzCutUmSTvSqvXrKYY/nEk3eI5FhzNLpuRjwxMlYG148JcUwCTU9jhndA9CcBE2/VCjqzIrQPHFiTiaWIzMA4cSvWsb4vYeQPRAHJr2GiedweMi4u2ep+Nm0UM4DmxT0AxkEseHiEGfyXDMEQ89CieNBi2GhtojBRIHe12MFPTeWOS8wjgQX8OFEZkm5lLJ9gHi0qHFXhVHAOhG73OzLX58Y8NVnfFLp5dv8/fHnwCYMkKfDVR6N4VIekBZa3bnn8z6fSA+bJmPImLztZJxc97IV5vWaomGe8eReAgZ5ZPM+TciqbkwWNi9VpFxEiHVXynvsMNP5ROEiKREaSlPjyXShyTm2xati9DQMQBXoUVeWKJ1BOKRo1bjBELAW4tGNriz0D8JFKs+KWcCqRY8cJoMrRhRtlmlGqTikc2OMG2dn2P0KWGoMMsx7fMZrDOZEZIBFdVgPHGsAYk7PZFXLdS1oBdhnoM6CzZUtYI0v6gv85OPpuYRsrIB2Q098s+XCm3pSLrORCtJkTkPlzgudgGR6/eoRcl4S6mKrRTD4REDvPEwwOqh3qMpklQTKFKgI5pNVetpcTcqKzjWpYR2Nfcj2/RbyZLUQU/FwyjgcFUergsq6cJVziwWjsxBFIhBb4oXHnoPx/p5VWagAsYz+PrI91sNJ0WqGBuCnf5hwd5fRCycHMV8ACTjfZUyHjPOBfM9xJIUoZ+A0Zpwn8n3nTPrdDRR9CkWN8HxW/PSnCX/0hxlffpnw9a3hr5Vi0yt+9CbjzTcZ4xRwdUNvf5wZir79JuOP/ijh668JoQA0Fg8HGp5mArWmp/BkjU6WrGtaKYVKVzcB11eETfZXTK5U1pIuCxUSRXl9xXNeGW4FniyeNNqX8E1pTwasEg7LCbpnWIqULDQNQsqhNyYOYYEeZf3jxtuSdqzsVLx5o8Z5dmVBU6ezxdQ1WJxr7afdj8DpnBEFuGkDmpX+sto+5llRd4J9CNhdE4IKQXD/YDCc4eRVBbx8yRZbfsnDoPijP0zICXjxQrAzgauxN+Mx+vPigsNCJXaTOZ8U91aZ6hFJ1TBR3loiOVag4FrF5Frw0N4GRa4ENeiMPLzJXMQqarnMZ8Vwzzl3PDJhyAQritZ6AM9nuxVTITTVwlPG8cSoedfx3CYoelVUj62OMPprt5S+SKtxBdDBurvP0AE4fKMY73lvNjtFG4Hm84hcAd1LoH0hpXnDMk6kjD0IioBbZZ5xrGzBTcB0DhABjreKL44cO+eBzWM2LZuj5wwcThSRq7aEr2BwIsC6gNkEwZ19NJkQWYyCyuoWvn5D+3XOinPiGHr1qdUHjILDLSu7fdFT8/JDRQkOT3B/6Pb90giB0rlimokHTpNxJLF4aWoyYQopDXX9RkKsmsoglH5gBaYoExptA0hQTEkwGk58OBqZ/6gYzmxBlSsAmRSmNK+kRycWGaQMxJ4GMCsNdcqC+gTEjp89DZTxfHhQ3D9kHM/sUyhiia4oOBtxv2mN5lbZtc/khB4PnFh+D1zdL1lrLgUTTY9vZEkQwjyfyDC3aYmnUrd4CXGJQHASlCSwexz+cHDpWfm4cuaJq/mR1mafzQsFVOxg6yKiyhkyHvWsHJByDHv4F86IXaPfj9F1MwoVbIEWslpyKT9KSNquzmcmOOso2LMi7GJMerLRxcqqhtEMQG91HJaIBxWLebpu8dSGgVj8NNPQdt1Ck8t5aVLgUaeIYC0ZTNF/LfcpTDwX3Vg1qj07X9xcW6TcO/DeJqXHOSdWraZI3HccTFRrMmGyeYGZYMnN0ogj0nhDmJQmZXC5X/QoFfkJn8cW68qL21Y4B2DJvCUSII7PiCfWQLgGtBZUOzJo1iu6O99WerDKm1w2VfD8WrdjR6N5QElIlgpMG2sqdj4igEVU5RmXKJMHc5ou92W6KkKH4XDKOBx5r1MA6kDJ4abzVn02xuyHJIgVdPhMJPNt2/fsgdOwTdYe6+6eEqGns5UGbwV1YJw2Dga11IJceWccRVSu2BrpiWQLNfNEPLiaBXNkf8paFNID7x6IQ++/ydi0Ni5MFUwHALPi/qB4d6RH/2CTcXMEtvek9L25pybE/jZge8VmxYcTPcE//lnC128yTr2iNk8lARhm4M0bxR/8fsKLF6y22+0Ex5FKbl/8ccLX32S8eUfMfko8HyrBKbYzw/TRC0QeQwxKA3l1E9B1bHJQtxzx/WiRhqJ0/CiY5BrveA6ugs8duXhvGBfNkvPZyoUtfxGEbIxSxAEUIf26Nny1oTfnla4F716dk/93tijofGakNVi0496wsyTckPej0jitDLjDD2frHt9Wz6jROQgurBKtQSPsMNUwEbppGipnNi2TwW0HW0wE40z9+CnxvWPFKOB01IIhh0DPum4CjbbfZ6vW9ZqAWYE0eGNnFPnZ5i7R8Uh8LwjF34IzsNTYTOZhjxmYApB6GnBkoN0QWqA+z/KkRTguPRTK4CITg6KxIrHJ9vkwKe5GQjw3l0OSXapkNcbEahPEDFUHICjkRgtMg8joub0RSANIiyU76gY8rMbJymiLSBHIcidPFdjfAJoDJGRsvyDzTJSywNuWkgsAk4zTmLFvg/HxGTX5/IIyJ9WavkqoBVITWm06gYzWQk4Z0VYBaCvB1T5gswXebgKaxogGI/VjvOlGiAKtwQrY/sNd8F9pwEXkJwD+bbDzvAL4PVX9X4rIKwD/DoDfBvCHAP6Sqr774CPDJtrMwf1wUtyZoewHFq1cWcdnVQ+DBePEJsE5W5IS1KDIkf/Ps7JkWwm/RFXMlaAOiiYKwgDcHtgl5OW7jH0LYFakcRGEjwE4HDNuz1TPe3tHQ9E25EnXNReBuhZsbxXdVgjhmObHF19k3N5zVY7W3CCBYePb24w//ik7zLzaAeddwMNA+OWrX2S8e0vN7GSrunuQTllLVqjxeHMDHivB7sqMysYYCEJjG4KiFvY6dI3uEnr7835uDNhrkAWLVmVJ9uFgC/B9LupvKbEYp2tWXqSQh73dUq9k19HAA7znhDcu+wr6sdW80tnwUoe2zj2PXZgBiR5lStTOHs3jdfaAqy32I424qx4+udgagAgFngIjGBWewzhyfLKwiX0vu47Gc7Bq4imJ/fA8opVrHx5oxbZ7jtmmYiI5Kz+XfeWTJdrQbMn3UTG2vDYf17Jy5aoK2F+TmpaB4hH2k+Vnkhlw88CjCLbX5P4zP8+77vaWCz+NuJprHwMhLwjJA2NSHCfgYVRcPZdLEFARc7UYV2GRX8igNy9XgMSFjRNqQXMNhEaQA3Hk9ebiZMVrleWHukH+LLnwbq/pjaccsLnJxKwzq4W3LbDbBqSkeGMNSsK1YmMY+GiYuYe3TUceuoqxbGpB6JiERbVEyR5x1hVwtaPO/6ZjrUAQjuVoeby6NVizVmAWyKPr/bbtQzzwGcD/SFX/UxG5AvCfiMj/A8C/BuDfV9W/LiJ/FcBfBfCvf/ihl9B7No/Su9PHyJXau5aQR+uiRlK0JHzi7ja8Wa3xmSUby0TWWNlifLwysW2ZAU4TvYmcUDy644kTtR9MiH2EJSoJv4yJeHQ3CtoTJ+/hoBhneuz94CGhlIfpqzrHgpaQbxwImzARZ5VtsoSIYZlLF/KhZbP9u4xoZU14ndPsGXMFk0nISyJlDZc8ty0wBulVqiaNmVmodDiZ8uCwGG/n3gYxZkWFIue69ApcoAo/B18YHO6YEtvLOTsmZyYH14JBZRwlhyWWfpcp6yUHfWWwvTP5Y7xRdVGpy5lQBgToYFiucaMdqkmZXnbKhMBG4/3vdkw2bzfMlaha4ttOm0VOdmdt3COQ1tnWYiJtpKd6gwHS2WiUJktoVoEMCgiNNceldbAPIH6tzLEgczx2nY2jQDiQUAfniqcDFEbthaCZlqRhiCZKZhrgydq4FdG3i7FjjVDEn6lYoZ3dO5iB76hqKD7GLYflpe0iy7iAjSHxiWEQx9roOeLhY8ulIepW0e2MwdQru94Ii5KSErppQbjl6qXY32zM2EnXrWB3TU49k6smobARIBpktiGslBJKaMVEteU6fKz5yVrkEa1l3J8oBq6qXwD4wn5/EJG/B+DHAP4igH/ePvY3APwdfFcDnjjoh17LYJRqJWykNoiDIBsHdrezIgmjWJ3PiioQ6ri/V4xDxjABOi6dbMjwkYJ31Q1bV716GfD5ZxHDWXEfWJn1zTcZ724VX36VcXtHudS7B7XyWeKJMGxPxDC7yml8emGYug7Ym3cZIyenBJu8ulDFDgfFL7/MePMmGyyAUh1Kr4UJrSosWjCPt2jVlk3HxFHTsoHzdKY33FQMLRH5muznMeatFkKvCQOr/yIl4HCg0X73TnH7LpeWUe41pkzPY06WrNzx/Ik9OmtIzNhyUkO0JLryRIjseFB8/VVCcq1ssey95U2YwTcJ0IH7PfWWIJxpiNeLN/VVOEM2W2EH+0feTlbSUqcRdF1mYH9FbrEKPfzzAHSuIjgC797y/Pue3nJdC370w8BuPpGL5+mkmFMuyd7CKbYFua4s3N+tut1YQwdP4EkANBBW6UfmafZ7wfaKHuS7ByZg91fsaB8qepcqwHzKmAZ6gN2GRj1lwnqhY2/ZGBjxCgQPDxkPDxTsqu06AJ7HeGYf02FmFaGamuB6EzCh2IbA5LI/C9gzhGIUQp/VS0JVPl8XUJv/EXsw2fJCWC2u7om7Zrt6CKfLPSaWr9heC17/iN7w7ZxxSkAeGDkDQLvl4vmDnwT85J9iYwjfh0cRsaLHHKJVt5qqZVsLxjPw+g3t0eEh4+5OoSO/GwIj0Ks99V/mwaCZIGQMVbRLGsgx/xaf6mL7Thi4iPw2gH8GwP8LwA/NuAPAL0GI5bnv/C6A3wWAm5ubi795aOxFOyUBUfROFtzSQ7BouhGVrVhpBtrGEooVw/HZvD/gOUhgMVDRElB5XoxlSY5Z8sfLwRUMiYfRPGMrDgmuUjdbi628YLoeNXjXbWdeLGfCf1JWa+TrFXaeuPUown5k+f/T+4yyqAVbtbyDuUCRgsneJnr/xYADK+MNuLbtWrq2DCalkRwmJvKG0e6H7csNk1fUztm8p9U+3JhmZdUgvWO9OI85mf7ImQU580rC1DuLF1hE3ai7561L6X5+dEz3wGVhD8gjd8cXoTkBOrGx7TxLSRAXr2n1eWe5OANGBAVqMwfemiIwWXmhBW/P07Fr9nG0ruiRAyBWiljxwAr3/GkUqe3DZNpsC1fpRm+e3fpcC4bsz6x4sFKqnmX1+SIkJ9bUGnyuoyX5s3uZ7/HAnS9VWhW4EbfrgBnf4n1juc96ubPFtV7t5zFrqnzW3/M5FASxVjQdE7GxFlZdgh64BCuqacnF3+wMDvF92M6dWkpBNhryGPgaTBvImykv4SXKXI5WYLhg+GZP/OK/g/cNfAcDLiJ7AP8nAP8DVb1fD3xVVfEa2Uebqv4egN8DgM8//1wv/8ZkY57oFVb2MDUu11FK1rEY8AITRJRGpSJMSGxaGr9eTIEOy75dT2UcKas5TRzs48wimMNJ8cXXGT/7ecIw0ZMKlaDe8jy+fqt4+CaTu2rJus683QwFIh+6QznXV4JPXhGX3G09PEZJVM3gOpDNE7ngz60XL9BAVCpoLAl08WzAc+2Ur3OmVzT09MjqKGhNFyVWnIyLu7N4t1QiNEaEYe5qntA8L0b7q68YKczzOpNurIjI97wwqEA1Ppbt3F2IaZyAYZYCD6kCt7emB/PLjL/3n88YByaMYxRc3whefxIKjSvnBerydm+e1HQBLS+2mCa7Hy2KDsjjcDUlkzd1mdSBi83umvev6wQvXoDqgx0N7cF0z7uN4OqaxS+7Pfff93bfDLtPzvZQRSUB2BjM1FpiMxrPPRM2SpnVwl1HTPt04t/GZNWMFVB1AkxkOcH7yu7Y3u7+wEYC90cWyDQBSJbnaRpylKNZgWyLkYCQw/UN789oTsXDSXEeea9PPc8zm1yyVepcjksFguGAjnbQL3ENI34nGzyV186JPCpnsfFZWCf2ZoEjEj/D3JFptvgqEEAZ31Zw/Tqg7RT3XyRkMHK7fUcK3w9/INhdC/Y3AfsXUqSUfeFxOmupMLZxLEqNlHnmNUSLmILN6TFR5mJyeNHmfA5cpGdvDGNGXL+DEf8gAy4iNWi8//eq+n+2t78Ukc9U9QsR+QzAVx9+WNvUStWt72GUdfiuRRt3oaUZnhuWn6oiVVAzdUKaSjBFRTQDEuzGB2EoGMQogqOUgqE5kbVw7oG3t4qvvlEWcHSCKgh1RwJwf8r0PhJwHMyDqcHGxCrwtmt1K+ga8mhvrDPKfiulVRpUjRurSCqlVdRa1dBX/cKeEDpTz0IowqikFXKEk3JAD6NxjoMli8zLEbEuIu7VmiHzUDNE7pQ0P06jaWYx0vkM3N4Rs69MR9qTSVhFCALzjBWL96qrhdlw6tlw6yCEiDSzAOrtO8JKf/AHM/qeic+6FvzwR2w6zbZYCzQyTa7OZ8ZbjWFi7l42gzjPZA24I/AEA89s1DuciLdPA1A1NOiM2OhxbTZ8rpOiMGK6DZ95a1WTMZrsa+ZC1fdc9IKh33PHByzCgqsq8vk4ROSwFOUDBPNsUJ4ZbwWAQIlkQl9asNa2FcyqOJ7JaDqemChNQSEWwTatFNlgm46mO85n2zT0untLzt73ioczW58No7UJHAFJpO4+3taG1s/NjXdJoIMLpE15ftocGPFJb/tymp0ApdUcsBjWxbvnh3w98Ug31sDuygS8rEnGODG5XLWMPrqtoLOuSCHyvq69fNJATYnRHAW1KGTOnNdiBVDuBDkRwSNDxZLjymH1YxOnmMAP2D6EhSIA/k0Af09V/+erP/27AP4ygL9ur3/7A49ZthCYABA1WcpiBJbCC3pQ5K2OA/Bwz+KDaF4EZV/5fYGiqYDRBqiq0dTC2nDbhKvIub67V5yPpH1l0KO+ehEWSlNg4gdC2ldV0duOVnLu0IinnqNhXduOeP1utyj9VZHh2jhwMn/5dUZ7r3hnWPs0ofTa5M1n1r+z0uVNRx5wHZ/ey7oWdDXZEI0tOF45OPaMLnJSzAMnnlfJ0YBLqV4Lrg/dMfrYXAvqjoyPozXXPfeEmBpFgSDcSLvkADIndhTgKKwI3HZcgBx/zeqND3jfvGLxeFLc3RFCYScc7+JOQ3o6MZHMwg4pBs+Tpw4heMFHiHYfzkvTZqedPZ4pAqAR4pDJMOcAkxiAhdANSoMOMoW0NF9urIBsGpUKdmfj9ScqDObMZxrEjOzM654qIGc2pajsJgVfjKx5w2B6L1mJv4aK+Y52x9B/80JIQasF/QScR8WhpwEfEiO+OiziZbV3jrLFEEoWFxO5NLjjDDycCe89nBXHAUBCMUCTOUBzXgymb0mZVM0KTLo0S2A+UJf5jcVRW7OcoEAIqw5T5rnQEdDijfO5rQy6R7BYec+OZZt3HBveQ6kEs31GjcIIWUFcuBwi4scST0EbXCtWQGSOgUMqFM1bDLob9WgLitR0AjXAKJfywUqEwId54P8tAP8qgP+viPy/7b3/MWi4/5aI/BUAPwXwl77DcXnwCFxtgakSzCOFf+ArtfKivOnnPCjOGfjyi7TynGjYtiZzGsX6ASYK8Sioi1BFGo1xYOn2w0NGSoKv32R88QWTZIPJvF6/CkhxwRMBWGjKELlpAURBrWR0NKYrUuhcEbi5YrLi1YuAT15bR3Sb2Hd3LPK4f1D8//5BYpHPTK/+1LPBQu2iR0pdjNc3XAR2exrntr5kT4hY38RG0O0CNlsacC8Tv3+X8cd/mDCeFfdfJ4y9mogVoQ9XLXR+ctMJdjcBTQf88LcrXL8MOBy50Aw99Y3PZ8XWxOnZNsoiHku2cdZywI8nRRWAF1dABCtrp1lQJ8U4kV8chfmLPCvevsv4xS8THu6Ifcea4kPzpNh0irt3pPFt9mKVqTQ0DqGoWnPqmpBZHYl7nh4Ms1VFDAvXd71FYUuz4PhxxTEzDMTsd6YxUtXkE2uwfp+mdrdbtQCcE/DulqqSlYlMxcAxWkeUhgKzGTJqgnPBlFkQE6CzYjjxc0ySMnra3gRsdoL9S8H+E1YCj1FIexuAu5Pi/gR8dZ/ZScaod11NL7Opqa3dtCgUTa9C1aSll/BpUHx1R0/1NHKeNBWwsfl1no2l9cgDV1gF8+SSwpfWsFAAV9/x5r/Zk+LZcgIRBfIJsswNgawivkUVU1bnACw4vYKGu1ag2QnavSB0hD8yaEilFWjUMndCMMjRHRR4BEzmkEe0LlFRRSBVluDcCGJHx88LlGKtRYqgaijipS2KJAF8n4+N5Xu2D2Gh/Iffsr9/8QOP8+wWhN6GZhoU6kisqgUt4VMZH1ewGCU7OyZXys12qg6xPcA0QDxxkHERLk4TlhLsxHBfIm+4ThTMUZ4SobRAz0XB7HxSO1Yltspr0eOuIy4MxDqJlpJh1CYkPyswQ0p/yQBbvJT72piOdWMZ78fMCd+cllg4sYH9HyFLom+atQgYpVkgwZuymodgmfe6I+6U3dsqBzFjXxKmdHecZhYDrP+hACY4FO1eBH9dnb/fF4aOVnRimiZzsojE8Xjz3OcJkECDGnTZh6ozEWxhbYEgZO7MMz351WW85x7y2WVb4KpkQl8z4QcYNVLBDuTeHIKjkbhugXVmNdYLr6FtDXt2bXOjrIqQTgsYK8chAVmue60NJELoo7YK0WC5nao2yGpauqs7hCU2pmCJSr+npXrXotlsUdCYST3sR5N6sPlR6K32falAvfJnbqgnKhfMe/nbYzrshQeui5HG6r114lLdtbbErrPWeLTHJ6PlGI4jUlba4EP3kCt6zGIsENcKgto+189hNZCWKGCZEzGqGexFEriQJuxZRJO09rm+3ueHbt9rJWaM5HDnRhADNTsE5sEpJV/9wTkmFh49+RCMGqauC81CluuX/LszP9S8mcqqH+uaUMCX35j1NyOXhdzZSZnEIu+XXkGMwOtXwtDf4reuJkVPAEQTrd9YwlJnxd1tLswQwSJWNYyKd/cmtGQdge4PnPBBUHDaFzeCH31Kw+0YeftYt9MMxikrEIHdxMFYt0CoA65eKl7+MGA4Ea4aT65BY4lGb6phvUg1ANcWDnYbq8oTegibCYgxWHLUF7SlF2i1qqys7flsan520zLiqqK1F7NjzxOf88mUH+8fXMsbaPcBlRlCLjYUJEsqaDe8H4TGWD+wN42Qzz8LePU6IE30dPseONyxGtJZA89h4FUluH4pSFtBfVB0Pb3R/pQRJ8Fuz1XqfFacDplStYHVdDnz2Y4TcHvP868b4NVLsTZpXGAn69maEzCceeem0caIaEkuO0/fF7GqEez2NAqvXgXsbwRXV8EqXjm2c+biM0cah6rmWI2tIJhUbWg5lg4DaW67TnC9Y3Xg6Z6Qyzd3ird31A7ShknGsBG0Bs2J6dHvG8V2Umyun+HUw5wwXRJ/Po/LvF551GUxBz1vn7+VsdJcZMsdIdoD7jiZgeU5KJZ/F4eGh1T276wVoQXaPXDzCdvcXX8quHoNVFtgtELB4EJoujay/KUkZcVzS9YTtgG2V4LcAM3WFtqKjUc66CJXG4FmC8zmiEaYE4YP375XA048UQqfqPWydgtR1JY7lmg/vonL5g+9qWmYWwBbPwbsoSeFzlL0pGPFkueHAwo1rWitOLaplvE3EX0R01IWFB3xOvBHwIYJgmVl14zS7cO3vqfxPp2Bt29Jwwv1olSoNjC9xdamI5slBqO2qZY+lL4prIjF+NUp03OIkVzgtmPJdIzAeDQmTk/DpgmFbugd0ttZijdRWeFUkwTdxKpWVzBcyt+llDdX5rlGAVrLP2ysZL6p+LsAlO4FSuIsZ+tMMhJfZxMGKb0zq8bCWllyBTmZpG9G0Wxva3q6r14GfPaDwLzJg+JUKWl9FkFclGOvthiZgMyRYTxAmYPe2ChOxZss+eUFRbGWwuJwLZSUgJdWar/ZkLkCAIdVU4Z58nvAcTIMimbkPS3P2TzxGK3TT02WiTNhYgDrCoLzorV4fSECIdOYe4FPsCYS48Sk/KalZwillz3NwOGo+OadIjbE1r3eIbTWwcqeYxsEkgi7Pd4UC7tQV5ey0Etoydeet29LL9eFdeXy0mtm2nph0IuDcHC58fb3vYEEu3MxebnZsTtVt2NrtmhSuWL5Fe5biw6Tn0/GZUcnnzMxMo/SVYTDHKLcbIDWSAuTkxaaZR804N/JAf9+DXhVATtjcGw646+iREIldvcON1RLW4sl8fN+wc66W/CkZWX0TicAKM4Dp3ituqUIijesysSjmmevWUyQSAsfWWRRPxMsi4VEuWBlAMsA9b6TGRwQ42xGyDRhSoGHPVXH+OvKx7yg2jxdpYdBaRiQyZ6puTiGioNwsxWG7ohII5XqZvO4fWFwudLtXvDJDwPajo0MSFFUAIGJ0E6gFvp7QOS8ca+iDUIhnyBAE8F+omHxohzNkOid1xdxqvOZVM+qXhVNNIQrQpQSRnvysq7YobzrBJ98QrmDCOB4r7i9zfjZH1Pd8csvEiljAdhfBYQKS/eV9WaTK1oBVVvTk5TAmoHhbM1wTzSU+5319QxMwA49cHxggdN+F4CtAqUJA8fd+UgP7+bKFiBdxn3fY2FtKBkufc/7sdkt0rbBxnFaycBqJuYfWkI4lTB5yeFIKGAynNVhngcFdKA2+Jt3pIiez7TmQa0pSDANkYrOgfZ8/pvAApS2enof/da6vS6cbDg0YU5T1kKZLfUb68/h6X7tVpUKUDWYYzGAi4NQHnEx5MsEi5W3r2PEGmtG4cT0qRa6hm58zAMl7YUEoFJd+vuqF0VZVKR8RoJV1GcJ8jVl8NdBUb5XA15Xghc7VmCWPoq6eDpFPCkvYVNpeLw2HrY/3lDyQBOW0EkAE9TXIjU6J+Dd24z7O+LC5543c3fFstj9XvDqJY1LlCW0cZ3qYFlop28vKW+UE3MetKoliZSl9701dUhqsqdp0UB3BoUP+qbmJG8a0iRjBHoRnNcPXkl5uz0o+oE88MqSVHVDNbn9FfP08RU96ODXskrl+wJUt0yYhkoQNhTRryousoAUsaDnbF+ZeLI6RmY7Lc26KCkuh0W07vUPRk/0ZhZdYKmyCD0kL/u2XVKGISi6TnDd8Zn9mZ8EtK3g4S7j9k3Gz36W8J/9pzOOR8XdgRzmGAUvX7PFlcZHV2AulgRCMk20aMg0ZfKkOFmV6P2Dom2BH/4gYLcT9CdWA/dnxe07hWY2RkDm5E6zWCUrG1XvWuDTFxz3Z8PDB9UiJQywecDpSK9+fy3YXwc0rXneRkudJ+Pkz4AmMrGqirh/I8yxiOPFQglaqOmB95SXfTuxOOfdHaOf8QzoBApLeYRasyIaA6BHLl67a8GuAzbV+4wOi3j8ebtc6rrzjGurc07LpRe6nldrh80WtzkvhlotF+NVo+V7ZQXBwh6xOVbVvIamo3541dKAnyddvMFVxLD26HlqXORSBmCNO2COWttw0aM0Bp8FdXXIuMsCzOJRxIKz/6nxwEuiLSxhFAkMl22FVBaDrUYPdG2QJ0mEdVIDixHPSm8PWASTvAnEOJMBAgCxFSAymbSE26v2TW7A7ZjrjLH6+bj3nRcPpJBVBaVYBuC1OCdaleOlsHB0Gb/u9TuW/njzKMU7kuekTJBkIMRV6zhv22SRAwclz6V0sDcZUU9M5iVNwM+tJtITA14eGsr1eSGNa2YIjLECP2drxmAt75z/7NRG56izq/yyegQr6HI517qmMRhB5sf9XcbtO8X9fcbptDT8mE0vJcenkJx3Ws+TebbmHc+ZEdM8AEnoZS9dmS6ff7kFukRTc6Ln7c+IyUvCUwBKy65sWuT0+JZ5UYqh1kZpde6EG7ggVSZaVteLmFg2vvI88dydlTUPgM5AHq2piksQgHmLKsAKcng+vhCr5Z1EZe3QPt38/O0zIrYvIf1uneNaD501vFXsZ/nlmWNc/vfy3tizKNpKury/XizKvVYt1eGssrvY88UCAgARiqiyjPNHl1QqXtXtAQXM3BFZFq/33MNv2b5fA44FR/UHLe7WYikCcexJATYhNmOwNtxY3uZDiasFWFE8XlWyS5BY3PDNOyadHk68u3NQ7DLQthQSovethb6UbYXNZl0vhAGdDxrpnZRqwIxSJj+nJfTyBIhTCEW4YgdwEkeBdZmxAg0Lv/TCRUExsvMM6FmREhOn3ZGMGm+AwZDBtVm8lHx5BlQKFCQB4qSQZO5bQCmg8vvs9/1i8bLn5ZPFcW7JPtkXipSfzjwqjgoc7jO+/DLjdFScj7w5VSQtL0RBnnNpuoFEWc/9lWC7YbHU9XXANCp+8fOEoQf+4e/P+OLn7C/6Rz9LmCaUJsHnk+nAZ0AfdVKfJ+q7TD2Nqc7E5O8frEP7QJnf2AhityjW9V530NIbq2pq+Cj4XM59xtdfc7J6W79QAfuXoRTwAIo394p3D1hVYvJ9T/ZO1uFpHgWpMkjOWCX7K+6/jdQ0SZk47AxWYQ6ZbdVO39C5mE+UCijCUkovXJW5ik3DxGc9MVqJFUvj08AcigiQO8UMQe58gi730hcNdjFaHLRg897HQNCl1D6bg6Ur7Xqsd60WZa/GGWyf6nPdWWmyshtAieTnmYtIE8n+ypMiRT6X3qR7C3Y/s7Jbs5ZFtDCerJioi9av1BdZIT++t0Yp1UBWSivW/zIDvVoUngC1DkwxKtUUFR/MBf9eDTiwMgLKME9k0eEoeKmgVCnFgEW/YbWS+opYVlV7+GsNC18I3GOYLTk1TIrTwO8Mo6IeWfXmT2PtVXqHcaRHjoPv2LCDoMvq6iv6UomF5ZvmHZEyZyu5H1rpSc9pRS/LgManrojauc0utWtMnsoaSrjlVMN9vTu5iENE9l4NhEyKXFAtGGMMtniujrs24B4GrhX/xMLbkHkuQZbJ5GtQnglLnM/EwE8nLc0F2F+Sxqm27irZdEkkMhxtWmLf243gmJhYPD4ovvplxi9+nnA4Uk8lZ2pcVDXleGdLyKpe3sucLTdyBpwM3ffA8YFQ2+FEI7e5Dthb4s41uUvS0Kh6rvGiSnbN4aDWLNrZVoa9RhpdgeA0ZjycAbHErifpaDi0JPQ94iJubJdQ8wY3FY1T09CDDiJsF2gl+qOJT2lvBjyg9Kj041WRhomcfrAoK5FWiaSsvhQwt2TJ5Cfbahy7NK4zlpy5UcaSeH9ZFLqjXvz90iFbvGi5nE8278TuyfrpKpZFRUB4NEAtohATq5Oy6JT5llct8RRmG1CqvCtrwSZ5bWu0aAGJwYYa6IxATbfHPkfyGHMs741k3rN9rwacA2ZpnyYmohQsU+w3koNVFyusi9F2AwSRgp07FKAAYFS54B4sgKkGsmEvnjSsLAlTHvLqRjpP2ds3CcAMMhbjqI6rCErbMOdLZ2t+rMapDuIULx7XS9nLscyjzsY8GUdGAVUEO6tYl/HLm2mMkszqNwCYztmaG1tHmQB+L5rHHst6w4GminEgk2fopVyDAKacyFVzDV1dGvCVBw69gJ2k/LE8eL4m/hweMg73WhpDlMjMzi1W1hexUoR1n88KuH2X8c1XGQ/3GX/4DxOOR5bhHwzz9oIPtaTn+ax49y5jHgU/vIqkLdk2G759PvpKK+jP7G2aM+mC7Tbg+mXAqx8ENI1gv+dicndHBToXEKPWiKBuBSerZE0JaGpWElP7noUfsITvNADJhNRGK5sXXSqK88TCn/5khicIYktT6IyWaM1/60bw4jqgjgq9z9CzLTbGqJjBqDTNDLQWj0TRz5RVqGpqlos5TBdJPZAfHmagzbjc3MhmX+CWPqM+por2iT3nZIbevew1e8SG+MVrORDKo1oqgsH5U87bFxJjXVWBVc25BXYbQWhAKY9sla92kdkqrufkbClGE1A2iInmcFoJCwYjOpwn4GDUzhgFUVg8tTHxPL/nRGh8YVOECaizIn4gEv79euDuoQqK0LtrdDhMkcx4F2EnGyhr9oMrqIlh5yHo0pnEjhOwGMrGBHZErOILRgs0l2DtKat5lbr2+MX2BVwwTjyjLGExGkFQZGgzHXrr2Uf8tq4Jufj5qukpOD1utoIfQBCFYbwIII8NuOEyaSJ1MWeYEhyNSLcRSEXKFypAopbOPGJl09NAr7yugN5ElmKhc6HoZ7vHVxYwXXI9/hPEyuax5Alc90azGZusxYs5PTCJOfQUY3Ls2w9S1RzkVSBtszG9jypSZOyLn7HP6R/8fmLSzyiJ69J5zzecT4q3bzPSFDD96PI2plnxcMg43tONU1GMPUviRYCbVwHdVvD6E8FnnwU+x5pe4C+/AH7xxwlNI3j50nSiW+pG48HErBIlY5uW2vG9G+nA+zQNhCjmpBhNZ1xglbzC5yuqOJ8y5pnQWNNZ8VDPRaqrAZGAtmY1cNdQ+nUy4z/oIvY0qRZ2FcBoSQD0UaGRtLd2Z+enNlbyMuSmpNBJsEmX3i5ghjaZ9zrx3CorlAFQIESfN3NekpIhuxYPivNUePFYxp+zVtYGnNoqC7UVBktoZm5DE8fmthVIB8xbQGoT08uLoVcz4FoMuBWajTbPI5PdqAS14fnTrDiPwFEVh5zZW9SaujSmqzRhlVfxKMGiqzg/XqC+ffvePfCUDfM1I4nAAUqWAQ2fGo4M5c0HsHTlzgu84p6BJ0w8CQgsHm5hpjz6cRgB8FCVtEE7RPE2w3pfvj9gqVb0Fd+xPPPy3atP2ara3KDrMhBdI2K9PV6H9T3vlz+uflxQPgm9PFFlpZkxakrNWgluTEdkFpzzkgBzzLyqbHKV+yaLB+UTTZbFzj1vF+Dybu+qQDIKoxvwadKiFkiKHC6SuRBZlTVzsB8eyIm/e0es+3DQ0uLNw2iPgiALBi5BVpV+j8yOfS44ZJNhFLXl7wWiU5RFlobDox1y772uYXINExvHnnxVZTQw2/EqO9+uE4j1b/Xoc7KxOFc831LPYODv2stkPsQW8cDIqWspAzBnoDLIBwpUA+sBSsd7x3kFmAOx2NkXbK/FyDARM9dTMTmGZ8ajJykDVuPC5ro7NBBd/r8eyupzRIutWA9+zpuVB+7fsXd8MRIYbdWjd0Up5qKaKYBokfx6stu1lkSrH14up9uc2UxDs9ViZMVsNozzm4WKDq158dFjO71Oen/o9r0a8Jypg50EGDKATBzIdXgHayZ8YcDtAisL6SUAMV9ys/0hlc8DcGhDBQirEu3K2mU1nlERlNZdx5MWHRWA4VJl8Exj3qhYnbIPHDcH6wVD1ZrHqqkeDuxmw+47NG6NCU/Nmaje2ni6QJTzRuXJTFkvMOQXS6LnOI+KNAk0sZCjESDU9BpcRrSwcgaXDs2Fs+w3MERWk5X86cqQhSis+vToKdJADz0/21npuOtaCBwqMm8uAacjoYtpoAxCHQOjlwTA5W2DADM57MOk+OKn1Hf56suMr36ZMRsdNFthWLQS89qoabHmubYOmTwzUUItaG9YyHO6U+SDFjmBddeXnGiYIUYDzSzQ2F0H7LbApz8IaBt6bndvlbzwiZ/fbAKurkgp/OqrjKoSvLxmIVrbCX5wBZxOwDgJeiWj5uFB2aRjYjT1o88jXr4kphotL+OU1jQApwcm5YIxXV6/Cri+YYJ6DMR7Hx7I+rn7KuPNLzLSyAgoz8AogrkiwhWDleqfFHFiXcTNFWG59sq60mx9YKw25bOVzGcAAFXmQpAADLDEpc89LAwxNa9inA2zN9jHd/vY2/c5sBhwlIi5FlZzBlAHJSiLzEIn6AKwr9htqq84ttZ6TNkXZxv0LInXcsAkbADy5R3ZCeeBi+cAJjJdVoM5Cc6TmBc7tcBWy+L4Hez39+yB68J7xkwMV4JRlXRJ2HgCZ22QXRxfLJZyT1JWv19sj562J0sLVdCqcTxUcxYAM930VJX8J/aUDIsKmR/OPYS18fa/lRZcyaha8+LdOUQBWMJvZRwfZ+K/bYH2RcsHrnrCyzjCgFrzCjGvx3buSePEv6eEUuDkSdsY1VTyHhlwx/nVvIvIDjLTRKgCCmhr6neVlga15Tkaq8S5zEwQrcLf1fNzTWkPYx/uFaeD4u4u4+EhL+MJDpctORUacBPeX/cNe3wPxVQnWxSvUoDiuZdnqkxIQgTjuIT+dWNywh1x+vlAJ8VVBANQ9HJGW+TqSjFuGI53kUZ8SiiaMWlmqb0Im3p7JFSKQtbwonvg1n3Ix1bXCRoAKbDX55wFCRmxAfoDC8WQ/PoWY5mETCtP3OdR0Zlsc11z0Yk1vtUDp6ds9zeD0svuoQKlq3tYc3Jto3aQXoz7x3D72m2/mB959exULyZkCCZPUQvQAQlUj5w93yUrg6q4KLqRsARu9MAVvXWWmpI1S8FyznTo+OUCCT2+hF81ud+zfb8euDqtTi0ly6XOO2SXm5Qv+bAA388AgpKuFQRsx2aCSY01zFWTx0zGNTbnzIS0GEJ55xIFSqg7T0xc1caA8EasujrvIEtoXUSqgCfGx8OsrCbLeiKm6jQx9vOUwn8G6PGQv0u1Ow8jYwTaWvGYiKKWXfdBG+yzWi2GAApyfkWLQXdBMadyBlhV37jKvGfSrJJ7SmVUmsGrFVUMQE2II0RADcPlPslUIGtCLha4bJzr3jng1uEnmsey3bJ0exipH3I4KN5+kTCcFW++SOjPiv5kzI5ALxsiJXnkC4yIFKEir4Z1Odr11rTAJz8ImAagbxXTFdv13T1oUVucMnDoFXinyMoqyZSpVnlzwzL3zRXP4e6BzJVhXAxZWffNOElg4VkSev5ogHRcpGj7wfTOZ+B8zmhbYLtNON4rNh27qqui3Lv9jaCOAVFYvq/wA1MXXoOPX8GxB5CJ5c+D4vANo6BBlVFxBKaeUTJGRZU4RrYmsKbmoWr91Cv2SllCcITbYnY4g/mYrCi9IArv3ee+coyW9VYAhRSI1e2A2iLm30u2E4eVshfaCCmP0eZ+typTnTJtxDgrpgBM5pQMyXJxyb3wclgk89TFFjcoSmMW90uzcKzMc0LMir7POE2kpk42Z9XcfcUC233o9r174HNiIgvJuspDS+GIdyv3zzomBtBAiC4dOZZOGJyojYntePIkZWDqF+svhg3WDYq6WlY34NSoHgZFrlFCcTfegOHaWDxJV8lTLFQyHskTFkwa9cNSMu6Nc2OUYmScQucQiuaFm5thHuSWxIkL05NpaC8MeOCAonfE6ZWtKkzTgpHCMGfHBSUBeYRxj2G0RMPu3ID7xYl5nBsa0eil1onwTc4rbDFLMWDFu5xgndLJwJitAi4GLpzdBkUOICVWJX7xi4zhpLj9OmPsrXmy7ZNeNp+ry+N68tW58EyUX7J/fKtrwcvXAZqAqVakHRtMxJhZ8JUUU1acBsE4Z9IOTxyzP/o84uoqYrMDuh3PI6ni1GeMKwkGbzPmybIQjYwjZgAqLrKnnjDeMHh0AowD+fBtLTjeW6XnK97MaeSzaeqA3ZYRU7slrBUt8vB8z5gU5ywcHzVFw6ZB0bakUN6dFPOZ42kaiCdXGYjKcdmZQFM/g4npzRP/u0ScITLiCALELNRmAVAlJvEmUHArmT48v7tIZrjz5A9r8u7a7kTY/Fcz7KVFo33EdWYAMaMriBZFkIyuQBLk0eQFomJSztfBPWrlM+L5+LNcRLaKJk7gwrgYcMVpyJhGQZ4SjkdhExAtNh8OwWqZx09u5Xu3750H7gstqx3VwiBZEoQew4KvTg/0cDZUsvSma1ZiPdWKuqcMcwbjRLs2txtaN+Ci7j1fapk40yCKJZpkYcH4EwjFe0YRdAdg/FZQalb5fgaP6Yp40cJz94g92iuLV16y63hUpVruo65Wb9USqhUsT4GQWZQgds4FgrLNWSfJYAfNYlGGFG/Gj+WbiC0WIqVxhnejj5VN1oiiIpkzBZc8SpGKsFF0Guf62nRZuFOiBz5PTFROLqkAkzswA+UwSazs97Ay1GEZTxfjarWFaJ6ZAu1g960GJgmYEileY7Z8wdllec2TSjTOeRacjrQe80yD0bYwTRGeZ4lIMp2RlKRAJcO4yD14f07vP+lORD8owtEYTSEzwjCDdbpXVCFzbmxNMtg8QxdumxKZP32vGJKit3ZfshFUUdEKsAGQJ34WAmppB7BhRCJNtB8NdpgVncrFPc2ZC0W07GUQw6B93FhRnsIjbE/8LawzH3fBvRKbOxJQjDXH/pI0dAfDnaBKaOgCluOJ579ADntQXzhQmHGOw2NZK0oFuP/fz2093+FDa+XciSiGxDmeQMefkRE/rfCF4LuhKL/SgItIB+Dvgk5fBeD/qKr/hoj8OQB/E8BrAP8JgH9VVcfvcOwC27mRjIFFGo+pgp4VF7+DgE1GQd0JNjem22Ei+8E+p1mQU0ZS4DQBtwdFY2p1VViU8xA4SbMuHrsbHIlMGnWdWDJkeXh2hwjhz0BlIWztmKBIWXDGJEgKPJyztXxiI4ggxIdjFJxPxu81DzQGgSuiiTEEgugTpgqUjB03ahd/FVswZlswfGKInZ8sRjdGk4dN5jEZtOWDcZ0YVpgXa/ekqei9FinZikqKqrYvm3xzYtHCxrnpFYBGkKa8lBKvFoqU6eF527LjAXi4U2riZPPUG5bThwqoOqHHWS/U0KIBv7ona49uvdUVcHXF5xFbRThTpvd6JhZ8CMAowNc/Tfj530+UATibkNGsCKAM8ZdHK5oZXFUy4KXJyfanjNGijTwrMgTTmQndY2QU+nAkHa2fqJNy6rVEVjkB795Rd+VdC3z1NTHsF9uApgKGe8X9L9hqrdnzHoxiaEENVFuOxduHzG49FdBbh6HNS0EVBeFesHkgPHU7UHWxvaLmi0TBm5GDwCtTf7ALuFrNCwW91+OoEGERUBD2Zq0CBbI668aVJ6MQzksEJsa1FivnL9IOFkEkFVL2TN9osiIlX7CjUOY5CNAEoA6MOueRM7aqgXYjyBWKvGsWOnpMy2hJLZi5KfbInXbHjNxhABauuOalMGnMpG9OWXHyh2g5t0rMSVLSjYOzXj5w+xAPfADwL6jqwXpj/oci8n8D8D8E8L9Q1b8pIv9bAH8FwP/mww/9aDMDXYo+gAsvvExChybcQzbvNVhYLGuFKzNevqqy3ZZcHA9Y9s9CSvNMV3PbBXJYTm4r7NoLhYdTqzDZIwQz4F6ZCSxeA/crViSzJIJ0dd2PTcxzDU8VKPkB/31J9pqHAykeihrHTuzk3fvxna9uXzlAyZqvn82jZ7Q+NbH75te5buLgnPLaMGgVLbzv9WH8mgT0sNKKi8v3BQiLgJE4TGCv7nn7WCkLw+PrW5+3kL1SRSogxmza4+Y5zhZlVI2U++zVeupJ4KzWC9OevVhbvZ0t9qMgWR/Q6IlRH6Np1SjCn61HDbYvEdi98HtEAasusqIQM3W+42hhfwR6JSsitEClnBf9mXLGQ028u2qEdLpagEYRWy78MQCwiCo29BZHG8OjEsNOz9xLRr42Bi1nFG2SB0jBlJdiFhu37rAoCj9+5YCXil5+GaVEXrPdKwVctGhNBnCqr6pBghFAIqxS/mZ/L/CMFid5eQa2Xx9ALjPhXvT6b346hdpspxZ8Tj26X3/iSUxlPfDB/mstfKEA/gUA/4q9/zcA/E/wHQ14GZCGe7ugk2tii+lKh9onOwpzwh31IN7Q1sL8SlEHYrE5EXOehqU9WlZ6NTKT0jdZVVZbc78hLiuI31D3VAGeo5+7e6NQTrxpXh6WhKUvCD1wDuaHB8XdrWHDlSXrakFdiYXbDLdqa9bcNObRV8+wKFZbSoaBXj68leHl6Jl65/VqMbLZkpg6kmc8T7xnHuZnH8+PDltgHiWMIMIKsmwFE15A4e3SmHxjefzNDeVtp55JszxJgVHmzNA89NQgkQAcj+Qrz6VknaG+NwSuWzPcDcrkLCwjXxhXE0j9IT66pgzFqLbYBUAret69GSrpgXpStL1ikwgvqDABeXxQfPlFLsYYAuyvA7ot5W5fvKSj0bXEm6eemiyFVjEpQqLXEQO1aTg/AjaNlmeQM/XGp5H4bD9xHI1TRhUF2xrYGE20OdEDPyTFoEDogHhFIzycTUgMwCiKthPsQkDYAvsG2HwScGgU4z2bgLctI9YxAw/JYLotgADo5smQtHoKjkHn5Su4gIwJGCzfs1RP6lI4lk010+FIW0S4mi/zymsslkm5RG7jyEUyR8EceJ+zhfwRbAJ+TIr7c8Z5VIxZkYXVp03Dw8w2Fnz18NNw55GvBrkqKYZOe85BCuxSHFR3SH0OyzIW3Ul8JjB87/ZBGLiIRBAm+fMA/tcAfh/AraqaWcTPAPz4Pd/9XQC/CwA3NzeP/rjyulaFLSU0gV1QWMIn/7MXTigIe8Bbg4GTN5lE6WhNjNmyysSHrAHtNDOJGk04CWLFIj5QllOBy196WAQsiw0NOD0v9xpDcKya2NxojBNnFgShcFSMNGh1XJJtAC5ab7nwVGmW+swD9nLl5b4v17BOBM/j6jMwqKiyStBEpklKzMg7nbJ8Y+01CC5+8dJ/91pyWgyTyxhsOjPcjeClvfbHjP4EnB6WNl/e+m0YlwrIftClBZmHyZF5idjQMxRLYnszA3fb1pFWeaD6NGrw5z0rcwVBiJXOKhjB84qTouqBegQay0KNoBEfzorbN7l4ZDEKwl7R1YJtJ9hdBS7YFSGgqVcMdUaa2fMzz2AbP/M825bOjWRS/3xLid5z6V5v1zJOihAUQyMYLM/TmMrmfWJPWemAOPPC04BCG51nRd4B+ZUiVNSEf7UTVAq83QiGYJTBQMmKo+VZmo4SB9rgyc0s9N9yvzk43N4O9kyiednRxn2wBSwYnTSY1czG+irP0eaaR7alwtP+Nlv0RhaVlCKiEIAIr55UHAdFP7H4RgOx+XYdaZRhbgYZK82mVWjq+i4qXPxLFXC4HGjriHQNea6jjQ/dPsiAq2oC8E+LyAsA/xcA/9SHHkBVfw/A7wHA559/fhEg5EycTDhrAEsEZfOyus7CtoqrIrB44K70l6HsVRgEbSZ/d65M52G2hNdAnrFaouZ05o079zTwECuPFVINp8mU4GaGzP1ZS5Wb322f/O4VJffG1BadvODROQOnnjhdf+a5VFGMG03P1fVSvI9m2wi6FmibxYhXDhOEpw85Z7VFCssf3XBbllzNU/Yst1ikUDlDwBaJnO1zujBtije7DiUjnw0UOMVMzDmy6IZcbYOUVKAzCxrGjpjnXea1ng6K04Pi7ZuM/rxwppNVBrpHopYszBmLPosllqN1/sbacK+cg2UsLq/vC1ezklkhmWOlBvWz60C8sqnJxJhvAtLnHC/Hk0dfNE6uWz9nUhDDQ4aCHPEY6Y56cq42Ol40zzruBHHLc99dUaxr0yjyYCJQ2bjyM0vkZ2VTXk9iw7zFDBOk2tIIDiO9bbEmH4BxldXhHwCJVZpVYrKxEsv5VLz//cjmxudM2p1GQjKIzyffxBax9fvZ5rgYG6QkAGV1DXCYhAYVUZBFCxV3ysSqNfO50JEjFLd27PxZTzNtQSWCbSXoKpa3X20F59G84cyuWAGEV3x4rOdAuS5zzILDres5Z0beKdDPRa5rOnRe71eeHZLfun0nFoqq3orIfwDgvwnghYhU5oX/FoCff8djs+LxbNjhAOikSCMwnWmktleCulXULQXXHZ8GHDYhpNFPAAJw80nEZs+wFzW9yPNBMZwz0mAaz8rEzJyAu3t6w/NMLrOI4YI9+d/jhg/24R7oT34PzGswfLW2ZgcpkeXi8I9/VsH9396Sz/twrxhPDGU3Db2sfqDhKiXKAPZb4GZPHrQ3cnC2TRUf3UjlojeP+mQAKDh4nd9O5sQS5gVBwZ+dPbLgxqSZeZgYVkbc74G3iyqt4+wz2SajBCANLOWWWdBIgIji1miM97cZ97cZxwOhJepVk1bYbRfWT57VtE20GKC6tUXHmUd+3ljCVr8J6jfDn4suE3S9Tcl0uwPbhdXmIW6MLbTd0IjffB7wo1eBJf0nVtXe3mfc3pPZ8e6dPfe7jMNR0L5T3L1h1eV+xyKfqyvBixdh0bYHSP3Y8NoRGF3WM1DNvrBysdh3GaczjfcIet+3t4RVAIoxtbXg5lVAVQPzISP3gNQCafh8xtVzyjOASdFMQDeZBkqwBLRJxT7cKoajYg7AUNGwo2XU5tDiemOl8+XiMo2wAiNrAWcRFCJMbIw0V4+4JQJa01nrJ0JGk1FcBUDnNy7SiA9J0a+cD4D89jwCu0awuwnYNYLXVwE/eBEwzIpQJ0J9tsBeNA3RJdp3aqMv/hELlu1SHBfQ3epnPR+dPrpKOxUa7GqYftD2ISyUTwFMZrw3AP4lAP8zAP8BgP8uyET5ywD+9nc4btk8+VA6tluyUbEkibxIZkkuiolcGfZsPRKnUVFPsiSTZnqB08hV3wuCUloG3Lr8viSk7DUnZqfHERfFRSzflxVepSu9k0Uu1Ol7zJJz0XFFNibyTI3Q+Lu+IAQxtb2aE8gbBVfBIZSnsf96VecZLcYpGbziBsB1LzSbmI66SNXSQ1CiLZZZzChrUXfzkNC1TSCkHq5hG488QgDG2rzNigwMAKX/5vGB1ZS9N1r26tFMepcr9PrzKNEAUGQXCmTih1954Fjdh/K7P+tnZoqCHl4GWDAighZUD6yM2dCEhe2QDDqaZmBIAcOcIQE4nQVhsmpd5bUOPY1dHQHNzHHME+GrKriD4tdFUa8QCME0Nk+SNV049/xspShUzba1PqKgB9205MNXNRlCVQKkBlALJGtRzcyCQh5wj9GxXocwJWgpYEs2LPy5FKE5xcW4XHumcvFnWY6F5Vj8k49tq1cQFMXSBWLVJbfiSeBASMeKY8uz9KNrNm3uIKijFIOrQOltWaokH31fn/lZ/8HvmayuC9BLI74ah/po/JWodvUcPnT7EA/8MwB/w3DwAOBvqer/VUT+CwB/U0T+pwD+MwD/5nc4rp054Mrl2cIwaaibECPQ7anm5o2FUVYpakbPs3NkaZhvbzMOJ75PowvomIEENFHRVmZkTQ/5ZseKNbVGo150EgFoUgxnNh8dB38YvLX0Wvle3RgMEgziWL1ycNM778/Uuq4i8MlL0hKv98S/51HRq2LbAVefESd9fSPYbdiv7/pG6AFbki9Xl1l/hS1K4+Ih8L0FNplNBTCNC8fWk+3e5mwVBZZIZ9084rFXIWFZTORiB8uZiQD9HWGCd43gqw1nzWwY+9iThjfNwGCtxKaTYjoqUqcQU1+cesVw4rm31t6stobHIVxOAj/Xchbv8byfM+ApAwdL7MkESFK8agWfXgnaSnHzIqCrBf1IDfmUFcEaSbefKF6eA8ZB8YPbjHkEjncZ/ZER0HBmBDieeF/u3gBvvxRUlWC3J7Vt80lAZ1FPey0IInhZC/am9S3gs7z5SnA+KB5GxW3PwrPumtW8m0CNj6YiTCACjLVCRhampS2QZgHmXBK0p4EVj1oD2nBBaIQ/dUXHApUiVeasjCBTZLQK1/z0XroHW2BHkG6Kitz9yimnURZ1UceTM6PXMRNyc264KFu8BSgaS9iKMFGpAtyPwGRQymzOWD0zSt40gusN4ZMpKW6PGQ/njPOg6OeVYmhGUQlcGEAr42zYem1eeBTSItcOTA5SHB5nl83WOpHjjAORlGMvRBNUkIux+6u2D2Gh/H8A/DPPvP8HAP65Dz/UezbL3nqJr4A3pIoMketGkMdF5J9CXlIkXz2RkxWYzwoZGQIfTc+5EasEbFd4WQaQyUOta8E0KU49T0eAUkzj0q2Ttd1Ztx5z6ICDgw9COlaGOuzgSdes9L7HUakJvKUB7yyr77h3XVFqtI583bZsrLzpLKQMHKSjPKVtuei8J478/FNe8HEvt19Xeq2N2/Lm8vIYlHHDvRhKfWq3/UZiMb6FMmnFG27As/GnS6eUIIS6RsuFWBFJgYjyCvIpFazLsdavF/fnsdd94UotW1YmrgZV6EBDtQdQ7VgIsekE3VagA8utvXgmK1BPwG4WzINiu+HC/FaAB804nxT9AyOMyXuzngT9QbmIz8S7sSdnOlaCqiN7Z7cVXLdLtWmaWYZ+bhTxrJhjxpQBdEy47qNgGznua8NzNwLMNX/GLRBnILWC0LM/o0s3mEUqib4ozFV4ItGTcznZ55N1bnrGgJcoUM3nFpQes9SDWSpDiwEXKc5HzoRM5rzsTJRGqw5AG60pujCyyUJRvGpeogRV88giBaXamoyznMmvH7x8fgbyo3OmN75yYFZjW5T3x9YjNMYq8X2Ikyos2hYhd12yXt4XQRF48+T3n6gB/0e5hcgwj30SFSlI4UjG4E8WgPO7gWIYY2CxRpiphZIzoGbgnPMMMAkVhBrXsbJwzQa1Jzummf0DU2K4OY40NLXdnTUcITAvu14SMItBXzTFi5cH/n23C4iR2hXbjiHt3sqtb268iIYtxKoI7DvhYGulDPQyOJ4zUBkF277ge9tidZHu/sBNypVcOtfiXYpsIGN9Tn4YD2UVS8VhXmQSvEJ2XS3hk3g4Kx7uMyQC3S8TQmQRj3PK69rgg+o990Ivf3/8U95/z0X7s5tHxXwE+gwc6ozcCTYv6KlKDbRmIHwcqYmG5RbY1YI8K7aN4Pw68Jpu6YiMPb1oV28MFdBesfAm7oTOTFxopn4f55kRlOcWmg37czSZsKE3rGg2wPU2cOoox8ODF4kE8sFVFO1W0IkgVEyydhsY/ssk+6YB+kmwM2+yrgSh4vVnGMZtzQqeY0aFQE49FEXalX02xVESPg/Du93bzZlyBQ4JuoGvhA5SK8ZnDjSeKlx0ZzCH4fK+ySDNYJIbcwaOg2JOGXP6/7d3bqG2lVUc//3nnGvvdfbZ5+Kli2mkohQm3ZAwiogK0grtoQdDyEjoJcgiCMWnHqPoBlZE94iK7CZBUVnQU5ZWmHnJ04VUNC09Xo7HvfZac/QwxjfnXGvv7dnHPGutCd8fFmvNy5rf+G5jjm+M8Y3hK5SHj9aMBVZGXKB40SiGZhoPk86K1Z0I2p2vqBU+bZLCXjhdKckxhGBF8I/gLSlKoghhoDMfdoOFM/BqJTo4DDZNRYowthQ+WlROx2QehJ/rpBblKJZMMZmqgfkuP8LQgKiwxiCWBpBCqh2PPZv7pIb9m9YkdW06LyzsST+eYjEX4VLVMC1o9WrWDoCiEvsPeGCh9TWxb28w7JAeVqqC/evuDbJn6MvL1TJ2iw5oEt+OQs++3Qu6rmMXqbUDJUnkDSPfQercHtbR580w8K6Inq6FxIK1TL1ZTtZtWyjaKD27u2XatWMe72Q0dqPvaBJeJgM1HjjDoT+s6zY2FQBt5ns2IXAjhHclctpr6TM6aoweNY6MxGHcqL33uaKioBz4izjVXRbJMupYbsdzR6e5Cmtjw/X9k03j6OPuGbVZ+0YYK2C8BhbMvA7mXa36aqwo/LmbTxlHDhtmxmAoVtfFauWSaD2COvzCh+vi1FMFtb9EJhM4PBZl5Dd9ylyFc9J+sWcdBqsFRVG3XjLmzG196DHx94Ug9fAAilXfqVtGHseqjBABbB2XyYjZbFzB1V8rpW9k2tzojJPExHHJeRQCyaB0lWqVDMmCtdKTIyRPnpRVaFSHF1ky+CamG3aacQ2Hn6ypCvjvk35uY2KMZNRpM12RBrH/NyWnT6tXV4motRGQ9Pcdxt0RoJIKE9oXVLPRD7d9VNDs7Cx2mN87YeGxUABQ+O6WHqAqTtHNn0epKcaQ8uZhLsUoKg/t0gZSA/sGk8Q0UhyVpAookMfojcmUOhxCgg4GnoLdq2iZb2NsTTdDx8jmzyqic+ra3cbKFR8oSaddDozKXE9cVJpergomtMuzWom1dpoPWBvCwQMKhhX3N37paqQZlwhmFCNdxtYw5baMaeZNx+I01YXtQ2YY+NSh2n5JRrAkiSVmOlh1yW11TexJrqSh71bRephMeft0col265IMyU0ZHZrXhi7hdlHgajcjGOrAdbSRdJzR2PcVlIVHaGyYtflW+DR2UhONQw1Qp4eXLr0XEX8jScVFGR4/USFLzNfESMZTck+TUYRWpfZxWUc7psQN9cB91/2F6UbqOlaKg4EYFMYwmMeq3Dg63OO7Q8uqdXU0nIlOzF+gg4kLF+uWjOLeF2tD95IapPRLHZQTGGz6ycTAS4midrfSMrIApVSKcUAxEeWmt2VpMVfkx43hNO2zSCvOFIZ47P75SQLH3OGgJGKRj/2/iZmoFlUS0KTG/S+NobSDuqiJmCY0euoUXyW61XlRCFmY7yWIoeHPi9VCmgci9OjxzqCGqu7wuV1gsdEIazFuKudxTLp8YLOIoC/l1sExCeaWoti7Tso/lXnDQCvpdaXJJntO3GAG1dAPqx2ksikpm5ZJH6utm84Lo0xVtHrbtJW4qGAlfH9NvlMy5QtUO9aal1k90xhlCeecXXHa88upMruMekr3u/XnjhXZVho4lohgT3/PlkszZSs8MhoJLo67THvLQ2ak623PzZRTlrB3OP2gFeC5yNVyB416PZbqla8GH37EeOyJiTNpTT9Xnd9NkZ2Xd3fru1WtCs/iRo3bek3GxpFNL+MoUGLNVnsARVb49IIoS3HyfqjNN4cdjlC+6cVY7REHV2Af4pQoIwWW2rdPnHJqESEC/GVwtBD3hrvt4IDYty7OPFjwgro1lAuaiI+rA6bUWTIYHikYbLRve0HjuVUZrIRQ0TX+pUYbpNWuOiu2mLcbct/8bv9a7cxsXw3DJHClfknS7QSq8bSRcAVjT5rb24wt6/RRmkMz2tz47b/SShCYDnthM8e0dVLnHiFPHL1LLJaBE5ITTE9OOgM7rjEjKTXXkzTG9BiYdZWe/e8sipnvE4PpjmleCGVLbzo32e7GnZ4ql74PHth9x2dsjxJ3xQ4RdSrhseE7QtnY9q/Hhy1Ma/o7pWoDY9sIcePpQSG5UT5hYzx9e1HBarXD+FjdemoCPBHG+2Ilstk0hO8O5aZL0scP7Th/LWjbLvaKaGN97PjnGWPrbpMH7x7znYMnll9lZGRkZJwwZAaekZGR0VNkBp6RkZHRU8i22452ogqTHgKOAP+ZW6EnBqfS7zr0nX7ofx36Tj/0vw59ov9FZvac2ZNzZeAAkm42swvmWuizjL7Xoe/0Q//r0Hf6of916Dv9kFUoGRkZGb1FZuAZGRkZPcUiGPgXF1Dms42+16Hv9EP/69B3+qH/deg7/fPXgWdkZGRkPDvIKpSMjIyMnmKuDFzSRZLuknRI0tXzLPuZQNILJf1a0u2S/iLpqjh/sqRfSLo7vk9aNK1PB0mlpD9K+kkcnyXppuiH70paWTSNTwdJByVdL+lOSXdIek0P++BDMYZuk/RtScNl7gdJX5H0oKTbOue2bXM5Phv1uFXSqxZHeYsd6vDxGEe3Svph5PlN166JOtwl6S0LIfo4MTcGHhl9rgMuBs4D3iXpvHmV/wwxBj5sZucBFwLvD5qvBm40s3OBG+N4mXEVcEfn+GPAp8zsHOAR4MqFULV7fAb4mZm9BHg5Xpfe9IGk04EPABeY2fl4yJXLWO5++Bpw0cy5ndr8YuDc+LwP+PycaDwWvsbWOvwCON/MXgb8FbgGIOb1ZcBL4z+fC5611JinBP5q4JCZ/d3MRnguzUvnWP5xw8zuN7M/xO/HccZxOk731+O2rwPvWAiBu4CkM4C3AV+KYwFvBK6PW5ad/gPA64mUfWY2MrPD9KgPAhWwR1IFrAH3s8T9YGa/AR6eOb1Tm18KfMMcv8UTnp82F0KfBtvVwcx+HonYAX6LJ2QHr8N3zGzDzP4BHOLZyDh2gjFPBn46cE/n+N441wtIOhNPLXcT8Dwzuz8uPQA8b1F07QKfBj5CG4ftFOBwZxAvez+cBTwEfDXUQF+StJce9YGZ3Qd8AvgXzrgfBW6hX/0AO7d5X+f2e4Gfxu9e1iEbMXcBSevA94EPmtlj3WtmNhX5dpkg6e3Ag2Z2y6Jp+T9QAa8CPm9mr8RDMUypS5a5DwBCV3wp/jJ6AbCXrUv7XmHZ2/xYkHQtriL91qJp+X8wTwZ+H/DCzvEZcW6pIWmAM+9vmdkP4vS/0xIxvh9cFH3HwGuBSyT9E1dZvRHXJx+MpTwsfz/cC9xrZjfF8fU4Q+9LHwC8GfiHmT1kZpvAD/C+6VM/wM5t3qu5Lek9wNuBy631o+5VHRLmycB/D5wblvcV3GBwwxzLP26EvvjLwB1m9snOpRuAK+L3FcCP503bbmBm15jZGWZ2Jt7evzKzy4FfA++M25aWfgAzewC4R9KL49SbgNvpSR8E/gVcKGktxlSqQ2/6IbBTm98AvDu8US4EHu2oWpYKki7CVYqXmNmTnUs3AJdJWpV0Fm6Q/d0iaDwumNncPsBbccvv34Br51n2M6T3dfgy8VbgT/F5K65HvhG4G/glcPKiad1FXd4A/CR+n40PzkPA94DVRdN3DNpfAdwc/fAj4KS+9QHwUeBO4Dbgm3genKXtB+DbuL5+E18FXblTm+NpaK6Lef1n3NtmWetwCNd1p/n8hc7910Yd7gIuXjT9u/nknZgZGRkZPUU2YmZkZGT0FJmBZ2RkZPQUmYFnZGRk9BSZgWdkZGT0FJmBZ2RkZPQUmYFnZGRk9BSZgWdkZGT0FJmBZ2RkZPQU/wN6UcLE+LlezgAAAABJRU5ErkJggg==\n" + }, + "metadata": { + "needs_background": "light" + } + }, + { + "output_type": "stream", + "name": "stdout", + "text": [ + "torch.Size([4, 3, 32, 32])\nhorse car frog horse\n" + ] + } + ], + "source": [ + "# 使用迭代器工具，显示一部分图像。\n", + "import matplotlib.pylab as plt\n", + "import numpy as np \n", + "\n", + "def imshow(img):\n", + " img = img/2+0.5\n", + " npimg = img.numpy()\n", + " plt.imshow(np.transpose(npimg,(1,2,0)))\n", + " plt.show()\n", + "\n", + "dataiter = iter(trainloader)\n", + "images,labels = dataiter.next()\n", + "\n", + "imshow(torchvision.utils.make_grid(images))\n", + "print(images.shape)\n", + "print(' '.join('%5s' % classes[labels[j]] for j in range(4)))" + ] + }, + { + "cell_type": "code", + "execution_count": 40, + "metadata": {}, + "outputs": [], + "source": [ + "# 2 定义卷积神经网络\n", + "\n", + "import torch.nn as nn\n", + "import torch.nn.functional as F \n", + "\n", + "class Net(nn.Module):\n", + " def __init__(self):\n", + " super(Net, self).__init__()\n", + " self.conv1 = nn.Conv2d(3, 6, 5)\n", + " self.pool = nn.MaxPool2d(2, 2)\n", + " self.conv2 = nn.Conv2d(6, 16, 5)\n", + " self.fc1 = nn.Linear(16 * 5 * 5, 120)\n", + " self.fc2 = nn.Linear(120, 84)\n", + " self.fc3 = nn.Linear(84, 10)\n", + "\n", + " def forward(self, x):\n", + " x = self.pool(F.relu(self.conv1(x)))\n", + " x = self.pool(F.relu(self.conv2(x)))\n", + " x = x.view(-1, 16 * 5 * 5)\n", + " x = F.relu(self.fc1(x))\n", + " x = F.relu(self.fc2(x))\n", + " x = self.fc3(x)\n", + " return x\n", + "\n", + "net = Net()" + ] + }, + { + "cell_type": "code", + "execution_count": 41, + "metadata": {}, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "[1, 2000] loss: 2.293\n", + "[1, 4000] loss: 2.053\n", + "[1, 6000] loss: 1.794\n", + "[1, 8000] loss: 1.668\n", + "[1, 10000] loss: 1.593\n", + "[1, 12000] loss: 1.529\n", + "[2, 2000] loss: 1.475\n", + "[2, 4000] loss: 1.415\n", + "[2, 6000] loss: 1.408\n", + "[2, 8000] loss: 1.363\n", + "[2, 10000] loss: 1.338\n", + "[2, 12000] loss: 1.329\n", + "Finished Trainning\n" + ] + } + ], + "source": [ + "# 3 定义损失函数\n", + "import torch.optim as optim\n", + "criterion = nn.CrossEntropyLoss()\n", + "optimizer = optim.SGD(net.parameters(),lr=0.001,momentum=0.9)\n", + "\n", + "# 训练网络\n", + "\n", + "for epoch in range(2):\n", + " running_loss = 0.0\n", + " for i,data in enumerate(trainloader,0):\n", + " inputs,labels = data\n", + " optimizer.zero_grad()\n", + " # forward\n", + " outputs = net(inputs)\n", + " # calculate loss\n", + " loss = criterion(outputs,labels)\n", + " # backward\n", + " loss.backward()\n", + " # optimize\n", + " optimizer.step()\n", + "\n", + " running_loss += loss.item()\n", + " if i%2000 == 1999:\n", + " print('[%d, %5d] loss: %.3f' %(epoch + 1, i + 1, running_loss / 2000))\n", + " running_loss = 0.0\n", + "\n", + "print('Finished Trainning')\n" + ] + }, + { + "cell_type": "code", + "execution_count": 42, + "metadata": {}, + "outputs": [], + "source": [ + "# 保存模型\n", + "path = './cifar_net.pth'\n", + "torch.save(net.state_dict(),path)" + ] + }, + { + "cell_type": "code", + "execution_count": 43, + "metadata": {}, + "outputs": [ + { + "output_type": "display_data", + "data": { + "text/plain": "
", + "image/svg+xml": "\r\n\r\n\r\n\r\n \r\n \r\n \r\n \r\n 2021-03-23T13:49:19.410200\r\n image/svg+xml\r\n \r\n \r\n Matplotlib v3.3.2, https://matplotlib.org/\r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n\r\n", + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAB5CAYAAAAgYXpDAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAACPa0lEQVR4nO39a6xt25YWhn2t9zHGfKy19uPsc8695z7qgSljYZQAQgTLUYQgVsBBrvywENhysIJ0/ySKnViKi/DDQcoPrER2HIVglQyhiBAFxjiUkJ2EECzkH+ZlHAwUBUU9b9W957Uf6zXnHGP03vLja633PuZaa5+9z7119t1idWmuOdd8jNGfrX3tLaqK+3bf7tt9u29vXwtvugP37b7dt/t23z5fuyfg9+2+3bf79pa2ewJ+3+7bfbtvb2m7J+D37b7dt/v2lrZ7An7f7tt9u29vabsn4Pftvt23+/aWtu+IgIvI7xCRnxKRnxaRH/ludeq+3bf7dt/u22c3+bx+4CISAfxDAP8CgG8C+BsAfq+q/v3vXvfu2327b/ftvt3Vuu/gt78ZwE+r6s8AgIj8OIAfBnAnAd9ut/ro0aPv4Jb37b7dt/v2T1771re+9Ymqvnf8/ndCwL8K4Beb/78J4L/3sh88evQI3/jGN76DW963+3bf7ts/ee0P/aE/9PO3vf8rbsQUkW+IyN8Ukb95fX39K327+3bf7tt9+yemfScE/JcAfL35/2v23qKp6o+q6m9S1d+03W6/g9vdt/t23+7bfWvbd6JC+RsAfkhEfhAk3L8HwL/yOhcIquiTQpr3BIDI8ntqf7T8wy/6915mhy3XuuU77X3kxot63Zddv3zntlto092mr7dd92jIZazZf9M8Sy8IQ/2FCPDwYcDpqSyuezyPb6zdMn96S9/KW3rbm9/9Nk2KZ88U41hvGDpBvxVIA21yBnKqa6cKdB3Q94CIIATOtUjtrs/98XvlNZb7uVzDPlycCd87GUj5Nb7/GeMvR0M5xvY6Epafl2fl94JgcdOLC8WL53mxp7sOiJFzGSLnys+3CMcAALmZ15x12bfj/goQIGWMYfHh7W1BN47fv+X6L73k59iPx3vj+DMI1zYnRc7A9U4xza927c9NwFV1FpH/BYD/F4AI4I+r6t97nWv0SfFoPyOqQnwjCxAjX4uNOidFyrbAtolCBGLgIvh7C4JoE+OTpxmkhqgTGaIdBNSHv6iHlZNaCOgRRc7ZNiCA+RYqfrxJcgZSUntdT1o92Pws2VhnBSa7x5T43D/uEN7pyo9iBH7ohyJ+za/p6mFcdvO71l75enrz9eKtZp6dmpU1uIOAi9zOJF+ltf32NXzxIuOv//UJH39cL9JtBA+/v0e3lsKVp1Fxfa3ICUgz98PJmeDRo4AYgc1aECP3LgmrIBqBjUbsgr0W8LWgEqwgwDDwN+UaaPam7eFpAvZ79nUYgK4jIfPzEprv+tTkO5h5C5TmGZgnftH7T+LL3ydnXkZgYqj39jX5yb8/42//rRHzXO+32Qi2J4LYC4ZtQIhAL0C0668GgQowzoo5A/PEufbfO5PzLeLz0QXOZRQgBlkS3luQkELLPJS96DTD3xKOy5mY2OTctt/Le6Ic/xHtab8n0qyTM74GvUjkwqVRMe0y9nvFz34z48XFq23s7wSBQ1X/MwD/2ef9fQAQs6LLWg5uCEDnxBcKqCCrIhgBT7YSUQRRlkR9MYk2uU7IkQWaQUZhi8QJVSMcgmZdkKFQI9ypIdLLe1QC3qKIFj0co2HJCtiBYMc5iMX+s+uJPQjDAZ3rQWqbH5aHD8MCxbT3/c6bvN61XpWA+ycqLyHg9VAfjbwQwrsp+c1++/dTCoX4eQsB6NZE4WXeAxBnAAnIAkgG4iDoN4K+F6w3JOCFABtRFSGxCSIIwn29IOBZkTMR/Grgb6JUdB3EnzmGcVR0Pfu5XpMIhgCEUAlEIcx2NpKdjYKoW4YIZwyKaeQHpd8dr58zMM/KfZV4hmIA+n5JwDcbuUE8QyAK73pgWAkJeODZ7TuOAQBkFnQZGIOWZRQnjmgIuN2vD2LzCvRG6UXqwVOn/L7eWZCP9qM2z/5RCHWeWgnKX0h7AalEvpxzXf7OHz5X0ZhX+ZEAiIAEwdxljCqFBr5q+44I+HfaFEDC8tAGe82xSfnMJy+QhZVNDgCB579MWEHFPklwYkh8G1CJu10O4Wg12z4V8Q52pqUy0SRGwAHMRvCl3RU+Fhd5VRDsGmISBPKS8Ff5etnaTXHbZyE4cfquUe3F9V/vB83LZj5K95p9rMZRXQor4y/fa5jrHX3T23Qyd/Tbv+9EYvFZAIYeGPrl3ptHRZoBZBLFLgB9BLoI9B0JlaNswIipH1onNrESIoBqGWTu+S7wezHIgoADRqQFyHY/RWUC7cPv054DNeLr9I37ZDno9jd+rxgqISlnz5YlBo65gCN773iqfW0zBNkGnoIAwdC9E1/7bQYwWX/LOWxURX79HIDZ5mf2z01Vo1lvSB1zohRxTJgdPS/UUsd7JfhnWs6XNmf7uHl/+07QD5zr2PHZ5xmo9EM6n1iBHoDc4WYnXtLeOAHPIBH3d1SBYGjMwRHRMXdgAMgNWw7pzL95z9FzS5BNQloQwrKoR4viqoyq45bSX7UNqTACLlLUHc4EpGEIAcsDF2zjGMO9gRjYAa26b39LpIjJR1++Qdy/23U6Pi+Sv3EgUNH34r3Fi+aecvMad/XvLjH2ru/fRdy7SIToBFwTcOi4B+eR/Q9GhLqGiJf9hbqvYkMQu7gkRJLtWuLfkUI4271ZCHpLVI/675/HI4LnqlSXPAMqk7HhGcLXwjSCMZFgFFvQAAyp42ntBOGWuQSk7OFsHc4mkWQB1NUfdh1VxZSwUHd6P9rLq1ASCkJiLjYuyaaidFWpXXeeKEXwelXtFAIJe0rNnOKIbrgqLFaAlDMWTGa5BnXuBhFIoOQRYyMJCipH7dh5nRU5CtQ30Su2N0rAlxNQxacCnVUhItBMdQaw3DQ+2ceitW/EOsdSUWBz0IueGyifOwrUWygHmY0gKTBOpqNOpmJJwGQqDjGdvgiZUBCgS41IjOYALvrZHEpHiLAN7+P8DMJ8lz7uO23aiIev9Tv4NC5W6Pbv5vr9sqYuXd35q9q/u96/rd93fZ/35r7zNSz7SSojNiBZCGL53zdly8xF7TM7xLntgKsPbZ87h9OGsQsWSLR03T8rG9u0cnZW0qwYD1Sp9b1UdVFQyjStjh1Hj2bM/jqj3tz3saKeodsmWZrVLw+p4AUCSDRmuAJWW4EmkPMoCVQ02pBbpBwBiYJgkoCrk1xdImgIeFTMs5R+kyBLJciuklxwCeubrZkTZX6/2ZtHaN9tIcNA9VoIjWrL1vq2uZLmGjeQzEvamyXgwHJl7X8ngj4QzURBMGs2GvThqMBRgv8qRkPMvrkarllulcteL4S7qNL8+oYUXJ84qWBKivM9ifeYgDkpXx+87/x5H4HBFtT1n0MPDB31912nJj0JVJYLJoGSSFT2Uez+d5C+XzHCvbjLK15fjvafqh71u6HK/k42FZSacbn5qksxr4NMjvvdEvHPGocTZqq8lGtlixoDkEVpiBOqPrpA45wf1pzNKKgkAMEYee9eF9Bi43CVYRHvbNBljxqxaQnrDVUf2M8MQVAS7pSo27665Pk52QJxJdYnQJyIC9WHPt8EGYog3KMQG5PYWcpcO1e1JEfMdzFEGydtVWoqFVNHCg240bx5chBgAJAA2QMyA8Os6GdBzopkZ1cjoIMAPaArqk7WnUkFhWnWNU+zICUtzAagjr84S5g+44YTxNGzA6hjj6SWVnSuQukFfe9zKou1QrPUCl2Cxdfc4987BPz4PTTIwNidc/OF6qP9mRE4vWU+JLSuR/4F/WxE2zxSpifIlAVzorg3Z8Gk9pkzH+MaBVWDqpbW9SoalXAx1b9YDyQWG+O28bbt86o4fiXaQp+vi6fyj6PLQru0GssiBBr9AxQx24nMdzLWV2FCBZU6kmvfu/HMfXnjO+Va9rks+11xqW9FhZqNpt07VSV3yxlvzo4Tp6yUBKdRMU0OKhSblZR7NLcu/cXR9dvHbfN3YwleMq/ezcJ7mguXbe4MMPJbIXL+uwmISRESoKMRuw5AVmgWU7kCIdtzmc+mj41KphBwNRWtopG62w4dzYddz9F9YQb+fVflxKpCkRsLVn/bYhzv763z+hntjRJw8ckvJ55IwA19UZSGTIdfPiniKALLjSMN8XBC2QEQwWoQrFe0ok8j3aHGUTFO/GoMDXPQehgyBKOh3/Md8OyKSGAyJDFDkGCGlUGgqphHQZoVGAEkjmkwHVowAt93wIMT6k6HDuijGMrURf+DbTRRbjrR26zUte+ONn+l0firtkKoW0QO95ShuJszME6KZy8yUqJHw2otJnkRpW1W5tpmOkW/DnATYX8nTQBD3CR8AKgSQ1WRFBc2ANH2rxjjKV4kJm53YvtYBNEoBdUHuvCgyjMA2+9+ul2Xqx3nqlUNtofe58ANgE+fZTx7qkhJMe616NeHnnPYjrVlVnVxsGAaBUQ1qFMzJ+Rlkp+jbQH3dgbdBjVQWpht3YJDMwGGwP+HHvQQe5YRPsnUY1+zU+snAf0Z9cvobBpMt9XB1w6YEiWdaHTG6Y0IEDaCMHCskpfMKndANkNBMM81f2QBxo5oPowKmQFZCcJJgHZA7oHciSFru5698nMsaPasuRI7E5LGBfhV2htG4ApRRfCRBh8gt6iLsirVR7P1NgFQZ72ZmAVyDWQAq5Xg5DQgJ8UuAGkGxpkuiiXAoPaK+m47YK733k+Ki2ub8w7QQGNMbp8zMImSsCdFOrAfq46HXhOgs2LogdgJBgugCDa+KMV8WkRBR+mh5fh3tGPi/bn01lqfPzdhbNZF7f/2uvQS0uIxNI7A1ZURIWkI/8yz2ZtfsurNCXidPn7WdwtBa6hjJWRaiDj3pqFrrfs2OJFyBiz1t47EA6r3lTMHzfY7U1EU3axg4WFzo/++J2B7NQNX14qnzzP91idFDMCTx3RZLK6u0o6rjrPofo/vgcrQWnWnQ9HbspqSFwhanOVDyVJVg46CAwQhKCKAlalLdQRwkZFHQC45H/1asO4BkTqQye7RBWAQm4uR6iNHuwFA7+ubAcwCyQoxDxU3xKaeDygQk61tBiQBKQKyMoK8V4TR9sKG8zqaft5tGkBVl4RG+ilIvJmY4jb8GuDrDRNwWb40g4FvKhVZjkUq929FlKLbvmXgwZD6yQPBex8EzCPw9JOM8aDYTYJ04EVSoTK81mEC9gfqZa8OiikpLkdgPwHSCVZ9gERAIy3rGWZsVAGCIkyKPAvSoNS/JyXiiOxPjsAuMzhDbcN1EdDOvRIEwXXbx1bvG+OsbxwjotuI8KsS5u8KqhVAXHHbqAiyBcRMMwNJdqNit1dMs6JfCbrebAexGn+9Uw1veLUu3DEOcdh31F/3sjDaVPS96hTOjJCOLN2lyHXaJfhLlXtYQX/fQhgbm4AzuMzv+v7TrJgn/jv0RIUex6D2OQ3bvFJW2mNS4t49TPxOQY7+yI0xEA3oQd07raeF5vobJ3JlOZvfvNI6oCVYVIEg1JgHJ+QZVDkWpaka11txAjPoWYIZ0AOjP5MZ+jVwrhXAZC6F3r+oQOf67oMCvZS4DIERWiHxnnuuZ04k3sGeUwekLbukVwoZFXEK6Hql/n4lhNo2d5ACsqnz9/sbQl+ohY/B6Su0N68D9+awpum8c2iFIUtUY524C5JPlCG6pUpFirvXoycBX/+nOux3jPq6ulRc7BXp2gyUJdKT/1/uFc9fUMXy6QvF/gCEQRBWgl4Em0HQDwI1WToLMEf+Ng6KeSbRzhMNSteXGdNoPsOBapfLOUNA98MxEaULzM/WQo9VdWH1vot43XWQbtsMX6S+vNDIwm3NyGdzvp+Aw4GRjhfXinlSdCtF7KlaGlYWtOEeE0fXvpN2yOLptfrr/thwNYe552laEpqCrJWEIKggQs1bwxi3mkRRVGRNn5r9qrbZ1ayFmqjiU2WQT9/x/n7v4rEjihCpihonxZiA3UFxvae00Js6J5l7nWRFl8lIY66I2BG0qhjBJmJPmfs3W7Cdu0UWe5N5iH0mEfd5QyXkhSk06jQxEDWDKpQAGvKpR+OPkwLTQZGvFPkZ5yoZ78sdQVAW4BCMeNq9ewX6RKSvnRF7BcWh5mylAZgGkwom0Fc/UcWRB2B+QKcDvVDoDugfc6JlA+gJgKFhhigaKdMmaCs0GcCxdZQqBbxqe+ME3NUVjqDda8TPvL9uRXD7YftUCNyxW49P5DwrxgMf00xEPZnnQzIk6Ia0lAXXe2A30V3wMJPAdmTaVPX4I8KUW1JD+01HmwFoEqqJoiD2TQBSZog8tHqoCIDZLNddrAMX42IvQ55fBFG+DckDde7Ld6RZDzQbuaBAGthm85QY7bXRvIIAS/TrrZ1ZPL2k05+PiEsz4YXoOPLUFjth0UlprqFH/y+J9/KeevSeeztwT1JSS4nEVIHiVuv2ABhDnGfu2f2eqpMwkAB6F30dHBj5vcp9zZVRVQphVq3G/nYuF54Ydy7U8fu8sTYMvZUuKVU0HTbmBwHE/fF6EmntAI3mehlMBdWZDvv4xt7XYN5pvV03K9UpzS1zL5AVO6N2VoskMoBIW4TPGdTZdAAsmOh4v/neWewNG1tB3XdM3We1N0rAszCiKgswJmEeBUMXIQCbXouLFnVXUnSGUIWmKstJuxEaVDHPJA4ffytjHGdMk+Lpc8VhBJ5dKC4OVJWcXzKBzPkVsD9kZEjRgR8yUVjoBWEtiGtBHPgInUDMkOLBPZ0IsrkVppkGrJMzAHYoclKMI3B5JZgnxXVQ9AKcrDjGVUc3Jxp0FHaiKpOTNvgJZcDup3rsxvfdaLdusoapHiOKItVkxW40751EYk2jJT8fJ87FPCvD1DvBnKkCAIBVj4K6nKe1hPEz+/0ZY7rt+51wzzkDcaOlE840oaDx1rgnoD3F9eRuv+CzVlfIjMa/lc29G7KagS8zydY8c79cX0lljHAmAmxFsNpwznbXRN6ffJzxzV/K2KyB95+wAwlEnBmUflSBTk1nj0qIS1Si0H2yMI2M4otOXTvvPyfUXEVHc+mM2AmgZmcEPpkVoQI1kAgmKcQESC/AmUAsyZgGAYR7RddKwgBA11y0ZCoUJECuFcF9b11XboEZugWwMrWXjXn29R6YKkEzkA+UMBKMmEdA1mYz2yswAXEriA8CpAewRlHhqDljuPuopT0pGgVHBlLQweu3N4/ATXc1Z2DMVWUQhC452cQLP7wFmCpgTkU3Ren2YNjJuL7OUAimBFxek1jvRyLr/Qxc7s0T4gK43it11QaNs/kl5sD3JPIRIspD237YQUmxrks30B94nhSzCDBT3J0SPRAmkEhMcxVRXX8chCMNqkWfdtyOXdTKFLyEkLuE81ltce3CHD/z8lClx844Kw4TbM7NA2jm4Z4mfp4TqoFLjXDE6qXi6D0c33cBCY87fvO17527KHtrg2lVBf5+VRloc9nlBT0AyxF3MWQ28PuGTaPStEJQGSWoGEfBPGuR+ur8EqX7P/MMTCOw2ymurjI0C+ZHR7l8YOoQkQWCrrpvdipn+l4vCG/jL+3qD0emt6pQtNknfo9m2Dj6LEgFQU5YNQDoBDIIwoYqSwUZHRTAZJN7GojMI5AC6B2STQXiTv3BjJORTAFrZ8L0GvGUGKFB4NgLVaGBw5cokN5WeyWQWSErgWxNQigEqu6n0D77+/6n3Zefo71RAp4ysLMkQbskGDM9EZ49zxABHj8A1ivg8Vbw7gnnxiJPF61V/B87KThROIzAOGfMCbg41CAcDUJi3QMCoYVZBaEnF4ZYqK0SDTiS6VfUz/Y90fKcgIMRp2niYcKs1EGC7oxBSKzTzM0SeurjppHMIwTg4lox9YLTlenE9Wi8t1JMD9yw/zwlpxn81H8nDZJuN44sD9VtCL7Za2XOk6mgqlqhJhlzj4g5AbsDjZSHEbg+aAl3dvWAo7hxJEGYZsXVNTD0it01w9ofPwjYrIHNGuj6aux+aTPh5Qa4acZ+4xpKnWxwAqZEg2KnO1vGvDwJkEzdMBN4IApCx/EFIzA07UgJ9VYFk5kdhYwfMxv6cyumEVCloS5EGnfb9ZIOkF4xz0q98GQujaZM3Y/cQM9e5JLBc7A0rw/PBKuVBf4YoU4WTdwFRQqUJL2vGhQSBMl09QowsdoMLLNFLTdNi/DVNmMb2tV+FgDIrIjnirhXpI8T9NMM3QhyoDheQLy5nigEYbJrmN5bJkV3pQiTMb1oF09SfXOTmD2BXLpj2CcT7JmUhQPXPwwCHQDMinC13D+SFPPBGOyK4E7XQN4wQiDbAW7jBpyThWxAyhXlr4KomvZmCbiSgOsMXGfgkAUfnSt+5hdIwL/8vuDUakA8PCHldsu/6+WIbLgZYlzmDymoWGgo248kFleTiUudkCN0QBgMKa2NWw5AvzGHRhP70ZmxxAj4esOUmEMPHA5AOjAiMx8U88jODebG2PcUp6lW4YJJR1R+GIlQVYG1AGOveLQFtoPAVeyFEt+xyHMiEygIEya2O5r390xffxcCWARINve5jdipMgrVRWGgMpysJO5TItG+3NG4drXTIzRYUxGMey0iN1TRRcF6UPQ9v3tm0VDbk4pqX2XD3wa4byXe/r4R7aLqMA8EJEUaFfOezyXRSOB3pQMkMRoy2gApNiuikuALSPQKAW/n2GA6jfU06I4jjZnTBHS9YliTeXkwDBFrph1nr0gjr9kZcNiPwKyKT54pxjnTzc5ymasGbLfVXdcjSDV7xj+TjGZ3HiBnCRDkzlJAzzYPS53erWvQZuwsk43mPVfTzEA8z4iXCv0oI32UgDOqPXRVUYgWvQYQJgWyIEGRQB/tcGlufh3MVkU0rW6HmowxZU5Ad2JzMdN9UJNCjYDjhGAPkwLnts9XBHWalHEfQpuDREAfB+gg0EiVWAuegnI/+P71TKufaUu4pb1ZFYrAXG7ofpdAop5Azjh5qHo2HahWRA2gOZl1R1D/Wgn5rNXF6jArklpmNFCvFnpeN3Ykll0H9Fn5v4lryFoChwpasIWnq5sgBbUwa+tadnGXnc52YN2Fq4qsVWpQUM0QxcY+M7Chi3WsdsYXzQ9HylioAMrMSHuPSrFv+NI30ym3vX903ZS0oH7X9RYG6mNzTwMx746OBM5XLCVD81DmpxBDtblJ/qMNoVeiNzcLtC5tC92RD/PGZBy1W6i4wN0B61xI+6wWeKE1mCfC0JRJa34oPXjLgzS4n/goOnRp5jxXxCs2Z8wnTc7oOmrmhqdUc3mtyDM9q6YJePFCcb1ThEkwaUbXCTRnXF9bCgcDFNMInJ4I1oNgu27WXmHqE6kqkmwEDQYoLLcIo9j09rkFFpKSX78FWb48CpvPpAgzGAR34HwGJ+4Tv6yBkyXZ8qZ4v03N5FGcCICah1jx7TfULSOYfqAh4IgozEjmSsApLtt5HpW+6RkWpUUAmg6cAM3GLEbbC+0e9EAu1DnwiWg9Vl6nfSYBF5E/DuB3AfhIVX+dvfcOgD8D4AcA/ByA362qz17z3tQnrQJyB6QdiewUBLPlftgnTvRuEoyzQKIiRyA3mfr8MLFffqCFblMArmcyzd0IXO+5CGElCJFGkX4tCAdgVIpNBwBhJBeNA/WEczTiawn954luTHMA4kawHXhAx+juZKj+oxHVp1eZlhQmrk6jBa6A+vQE4PJA3fiLHRCi4mRllV9srLfCSWN211OTGU/ATGjGdLIdJA2VoJY5u+WSL7Wp2G8OB+D8PCNG5naOUdBFdpDiOFVJwVCfBEG0AAkfyjxzDnISTNEItenHS1rTSEZ8mOm542e2qM1uIdzex8V0HTH/u6oCdaARU8B7xQxIYsBHVPtcgZUCg9Ju1UHQzURrApTovk4twRGIBjNIiGSqgT1o9q+a2ylmYNULYlCshVGH+4Pi/IqG4OcXiv1BcXWleHHB/XVthvira0o6hSBITX3rgWR9B3zwfsDZacBXPwj4wR+IGAbB6cbyeEzc92lWTKO56s1AnhWyFqJxATBZNOJ8k4K3WQuL/tfBT6hIXGDzOitWExCvFPo8I10oulERo4GTC0XuFKmragnfSGJqnLgRxBO+lweUlKYWnoFutGCsvVoEqBZwky+ZwkESx4UM6I7qpXgqiFvOXzrAXEMZCzLvFLtnvE43KCQK+i4zWrSj5NUqwBXAbIseDKzOuQLV1wDgr4TA/wSA/wuAP9m89yMA/rKq/mER+RH7/99+jfuyCRA6AYRijaeJ9BBdAhVpiilQ1+iH3wdakNjRQzMP/Kiew4SbajC9d+wEXc9k8rEDoiqT3CQaMZ34xSAs8OCuboWYGwMRlLzPUcjZW1jtBqml0alKCoChe1tEmav0scpmWLmNyjYtK5CUPsmdwBJkoWZ9a+arUcH5v7XpS2+zvKe5A/qchFDH7OsCoETYeaGOtj9iPcjmOqaZOlZH5kFIANs+l3E1O/2YGBsfv30sdxDv9reFMLSbzebGc24wLUJF43C1iP8eWnPwtCqTZJtbakKpKpKgJI1yL5ZozG+cKfXMM90Er3Yk5J9+qpgmxfWlGTJnMwyr2XkUJQe+zsB8UMYZZODqVOmt8l5ASsC6pxSUOimpVtNMwjWPRPpzYJBasOtp0lsTWt0FDo5VBS6FiNLfWgwFO7oOFiznEZO3ARidLaJyUCPLQI7L/e+5q9XWTRq9vTpC8vuaDStbNCdGWA4WYQBRqPsoKxkdMgOKQuI8eTi3FLR0dBaleh/dMPC+YvtMAq6qf1VEfuDo7R8G8Fvt9Y8B+C/wOQj4sBI8fBCgGbj+SLFLim4QDCtWSnn/y4IHp4J3tkC/oidKzuSIC0JtPqDlMHSCYUXd1P654so4ZuiZZP30QUA3ALFnuaeUyeWjVWdwETbnZQYz2C3TxGjOy94y0klgqakI9Bbg008WYWg63+JJYQc8BB6WGBQTLBFWAg6TYhbg/Nq+J5wDCEUzUdQN0XRKOkoWEqSKjcb1fQwtYm1pRiHkn7F7yveMeJY6fiI1i6PtwiC0D3SJevJyqGzNYgeICKvBTIBCauk6e/YgrBBqCbH1xg5Nsx5HU7E4KCXa8Jbv3tbSDFycE9GmRB3wOCp21+a7PlHsPuwV5xcZfRQcBiC6vjgrYhCs+rbfXIu4Zz8OozGoUNfIn4tx14y5RnMggZGqLy4ZVPbsnAT86pqqtqTM7dFHAD0gicCnN8DTBapPpr1iZ2u1P3BjfPxxxnadsFkLdu8INivBdi3YrBvbRKLUmSbF6akU9eF8oOHz8kpvEB9XL8Kij0uu3RsIDCUCc4yArID5vQB5oFjNAX2mXtsDc5IRdBclBUpvlEy9dNhIMQK3ZzfaGROlROc2zcmuFcxdpMtmy0pcZ81Atwa6lRS1rAYgntGtGAOwtkob4l4WDwWje7xYyltn9Irl/hWAoAUv1Ubd2j6vDvxLqvote/1tAF/6PBfpe+DhO5SFPrrMkGsS1X7FiihP3o148ljwICgGUYRM/VNuqqmoESzAQY/YNQLSpDjMGZc7hmcPa0G3EmxPA4YVAEOp02yiejDDTDL07zqsknaSBDXNihfPgCBUcWxWtOr3PTdq39FzYp4U+33N96HwQysIIhh66n2R6NM+gcbMAOBix8xx6xU32UIMvUGJTJ8/uIGLFLq4HBbCreVfBx+hOUQva+VrzTVdotAWeRu3iIE+3Cmad4zrGQUIQTCsiKwmC5YiETM87t+LLMdVVGMm5dD1tI6lmYYbiK/l9e337hpvmhVXlxnzRLuFexUd9iRUyULUx1FwdanoomLsqKoaR6ob+l5wdiLoIrCavPgxan7qZAb4IAjR7CuWPGlKVINkUM1cwvdVsBuBiyui7/NL4GpHHfjoSNUM8p6gib/knhgidd/7qJgPGZoVhwMLVDz9VNGFjO0GyHPAdi32MAJ+0ELI50lxOAgGG/NhR1XZ9e4m5ZFgKVtrQpi6Di0Ct/+zCKagDEd/EoBEl8BVsNxF5sGU3dYUCVygYrp4GiyDGb0PnhtcquSkGYsc7nMApo698vM1qzJKNVfD+tAD2tHPfI4k4N2K1Xa6tWA12LkyTUI6EUyuh7ML9yalLAi4fcXt5bcIMi9t37ERU1VV5G7sJiLfAPANAHj48OHys2CBMACGtWCzATYHwfYkYOiBzVaYmU6AHmq6Kc/01YTS22FYbwL1dyvB+iRgnID3Z8H6VIsf8WBeI31k8EPJZwzbY0Ys3OFeYahZuYlmd9NSrSXRdCkCxY7Ia07UJ6bcoAGRcphdNdD1phpKVOsg8xDv9orrPXC9F/QRWMeao+NoDfDiRcYvfyvXjAS2QT0XTKkSYon91z2w6aXQ1bImdyCAY5UD5xzFjfJGCk3vgxnhPC+II53OS2BFc8E7lhhsHdxoV32NWRtJ/BQUscAQn6Edz7x3rHpyz6W72mwIfNejuBGmmcQ5JcYITCMQQ0aIwdIekAgncxHtO6rAuggMg6I3I2QMzVhgXkKdzwkpzThz7VXI/NSRqyHww6hE3AYKIJbkC3Xu87wco9i4XfqLVgzhdEM9+2YrtUJ8rgFWovQ/311ZZsNDDbjq19xr455qm6v9zdTMRf1lOYO8gMRChWVAwDvskqMMKEbfGRYngmL/rcUlbA84BRJTQSqAsVE/ilTPILdX0ltK4PUcClE1Y7TmUleiqk6VMRuw/6MhhGFt/XDp2CnrDfTQvG9Ny5/l61dpn5eAfygiH6jqt0TkAwAf3fVFVf1RAD8KAF/5ylcWXQuBhDsE4MGjQE+OwUS/XvDuewGPHgZsc8ZJBvKo2I95IWZQlBfEIHjv/YjH70RsTwMePqZO78lXM3bXik+eKj7+JCN2wMk6IHbA5bWJgBMsnzBdp3ozKKoR/Xli5Nl4oEgVBdj05P5tXxzVrqxKuUQS8TkB1zsifTGfrQDq4qljV8w9Eds8kRBc7RQXMw1i25ViPQBPHgrWw831TQn42Z9L+LmPc1VlgLr7IFZUduCcPnksWK8F778T8P4TKVGGxZhmXh5lQ/kGbA+ZMazYC93QIm0J0UVlgAFIgQdls1YM7r8LEu5ohHaO7JeLwxAe1qzMt344kHAeDiRs86SYrMpMnirzhRHv7ZaE+53HAaenzKMS43IIhdjdcrB2e8XP/2Jq5tiCywyRufrj2QXwy59mY5SezZLr1XWCEyt0PPQk5GWeQQlEQAnDK8qIjX0yo650FkxCowoQuF9fXFGVQ48qY6Lr5UBUgDQtJRQWIeBmXa0oIX7tKwEPz6S4Ckbz+54mxXwArpUS5KefZkwTpYM5A5vngo/P2W9H5waAF41AhlJB7k31ZUEzRf2JSrREAHR05et6Ps8WUzGq4gooxBaotiigxs/kBOQRZmowrxmTBNy9U0BVkxda8D3kunnGaqC4NkKBOJNGZFAqQLACzQBOO8H2pKnbqqi1LqVszwUxF1unlnAvXCxfsX1eAv4TAH4fgD9sz3/hc13F0WHkIYuRiG69Ye7iEiiThfrpxFVe5to1lGYEZdgKVlvB+jQgJ+BsD/Sd4nqXLSUpCSVr4x0VUIWBHTNelkTvNrFuQEIA1AIqPAm9hxkDhjog6Mbqm+7I3gmHL6Aqf6/KfCkh1kAXprxlMIYIg4Vyvt3VaLen3q41tsVAZNj3VFkMA8gAMjCeWQpQ8VBqXZ4mlSbRESolaL4m7hZ4hL7blL9EmtyprQujI+PQSBRucFKlnSMlurrNmXpfVk+3MmGJPs+FgENLgE/fu8QjNw7EwrB5CwFPCbg2zwOW03IfaD57ThL6K/MCwTaKSzydpcSNAegnJxZMsdCuixNwiBgCt+yMCZCOVXEkVMllfyDxnuYqkZAZVUpwwxjWjt/2oXsqrQYy8+lAVK+odhp3cXTmOU6UKmZDz6HnGDy3PtVdywl1CczFQrqHLolUQ8MqRnAUDpSI0AzBDF2oHtxw71cS638y25UDKjEe6EhdYIzAbu5ZA8s51yr5tfOnSukjKYAsCFkhWYo6px3MQhJt/t5clPqv3vX5S9qruBH+aQC/FcC7IvJNAP8OSLj/rIj8fgA/D+B3v/IdF9emHhSBQR7nFyRiX/ogYuiBk7OAYQN0EyOq8izFeObi9jwB59eKrmdxhbgJCOsAGQSSaFTUxDJmvRUD6CI5cm+POaDUP9xsBN0guNordpfcnBIYYZiVekMdgH4VsFkLHj4KePf9UHSlqkA/VFE0XEjdNYaG+5WUIgY5m47bDrwTofmAkhr06QtgvVIMHQ9vPwCr0zqP7uUyKZhm1FFJIMoYR8Vhz8jGdceUuo/PzJPGvR2AmtjeD55LF7IkCv7c9cB6K4UotEiSCMi+GFCiQos+PtS+K6i6uNwzUvZqr9iPVFVcX1N8P1i+FNdD51wJuGfP6wfBw4eMLjzZCk5PiGRVjwj30TjaNk6Kj56RYMWoloO88rXJvDu6TtCvOHde8szD7GOgS2dxgwyWpzq2fTAPlQhAlExOKhENnSCOdElLxjz2B8XFhZakVckUvIZp6SOv3DOeyTAbU98OdOXsOgao9bZe48g9vd/Z/toQsT46o/H8xXnG83PaHEax6OkE7M85e9k2xnYFnHRHhMuYlHRMJYHA+ZtTdQ/1PeEZY1vPFbENogDUbF4IVSUiQCnq4uH+DsCKvl3EsksaqPH3lRKTKui0gKU0HfuKyAEDakGK7z+ErqPu359sj7Wbq2uYBqC1fB6W50ma50rsX42Iv4oXyu+946Pf/kp3eFkT2AbmwbneK07PBI/foQ58vaWoF1UguWblcz2mgBv0+poTnoWGvDAI0AWI0C1QOxNjAxezC2QcMagVIeWECYhKegHGpJZSE6UohBrRjZFi+rACTk4FDx8GunFdk5B24PRfm7m/Rd+xEwwDiz14Bj5uOnO1M6NMiDy8U1KcXyvGWXC2JWw4ObOEaM1UOjFmNB07UAyzAEZhpsPrjQKZyEkzikeHSxwCR5JaTqOiQXZOgME5HFxlgipdtL8p69x8Vj5v+j8nxdU1dbznV/RjHg/A1RXXYRxdZFa6rtk4c0JJuDQMnJ/1WhgZm4h2/cYN6Fr0o21TAp5fKnYHLXp0H4cqpYGsXPu1lc8yEE0/aXN16yatxjNuR6x6R5V236DVYBsoZRVm1zHtqwRL56o17e5SqtDyN1nfiNLJyGerOLXpxMLoBScrmD5eMU1M5XA48LrzLMi9Yr0WvPMOS1T0Q2ZyMUUp5H3Y877tvj45mksHAiHyfHru76woOdYX5wOwIhmGp48XR4BScNqv72N3uiBVGozSFCQWFHdfoDJ+B4S+H7kuVQJQy0fr1ebbTWO8t3jQuE7Et7rrx10ibsehzXW+k/bGIzHdT1bBA+eWf0+W06pY1KuyGCEOgdkKO5ukTz5KQCd4+FjxJcsENu4V6UDU4qWufMOsVtwBCkV34VzY6jIGYL2SkkfZubNbtTcbwcmp4PSUutb9nt4B2VBPUtd/U5fr18huONVq/MtqpZ86KV4X3UDRMVrSLAjzuURRDEdBEyLA40eC7oxqo3nk513kPPnh6Drg0QPBaiXYbgw1KpFtUb0oLH90jf4r4m+DEAuBdoRiUKr1tvDWFgFYEHgsre/FPdpUM4P9IGfBamWShQrEdJ9T0Y8TSa7WgpNtwHrdeH5Aqspn2YVbm8Kiw9U8dBqPJ6Dqr/sVDeyFYAAYVchg1DxroBg6qRXUB/7Wc4/43IoAoedeDEEqYDD32ATLVSLcCyV+QOvaqk10UFcvCnIEItj3B2eCh6eCzQA82BqQsUru14MBCBhyz9Ve03XMP5MycG37OGVdSCUiOGIqKITZ1WvBxuoSTUHFgpLAyg2FggomEIDQU/20ylSplP0EMKDIbRTqUg8/jE0fHIG7jbGkygVKJsViE7e+ouxxKXnQvW8AAYzbc7KYlFCH32od+Z4s/1/MFSpdeukGPWpvtiamSDHieGa6aWZinhikHOQegkEEIVE3Hme1oBmGsK8iDSw/+48S/vHPZnz1+zskBEYk7nka51mZUMoMbSEoTk4EJ1EQ+4zn54CO1LmqeRCcnpCAX7t+GRRdJQAPHwc8eSfgybsB774XcH6uuLhUJKVRdJoV48T8K9PkSfFJLMPETbVyGC3VsLPOFkAxC0JgHcVgxJOoVLF6BwtKFAPwta9GfPADHe9hBHzoacTjXHMDrnp+f7OWklR/OlTx3/17Y3TCQuLDnMvc5F5rkNKQFEYr0IU+PBhz9mg8eJcbcJX8oZVwSid0G4Xg7JSDzLPW1AVCY9vuim5tl5ec+80WePKOJ72iq2ZRS6DalPzetwGgrPRe2CfQn145jqF3VMfnrUleIihpEjzidk5USUABOaFtRnpBt2V/DjvuIzcQh0hXtGDpG2I0N0IrgTZq1et2A+dczSjZEqUuEP/FDuh7ztdsyPtL7wV86V3B2Vbw7iN6z8wmsV2cZzx7yjw9zz9NmCfgy+8HxCgYVooHDwNir3i+y9zLWoleWce8ZI4Ctw+huIMikJmGwHnsIzdmBnXLk+UqAlDC37uONqtOqx+2qyQKHWk2FOdDFvPizKLsv/bXDQJ3Aq6CEulZg9G0MJWIqg4UATSYYbNhLK3PdwEQLWFWlzhsT0o9N69Bv998OllxAmbE5Dg95TFHck7lTNLF05RpEd/vqTc9mHU8TgqxkO5kRlC/buyAaN4ZXS/orIAuwEPbdXRd7KxIaUuY+t48DCxIw0VhoKLs3IylEA1HGTYuN0A59/XUtNEiQiMYGScgUndxejmJHMN6TQSYAjf0MJi7XjNvQ1fnrEU9OZv1PRkBN914F3j/DFfT0F4xp6UYWOemIitnAl1sNmdYMu5WzeIHvrf5LyKvgu6G2eIigiHdJJijRR5OivWKXjqrlfkfW/8slUVR/3zWAVH/UnBmX/Wq/uMWebt0Ah+P1ut4JGXsSMgBU/dBqzucb2RpXiuKF1TJc9NctyWepb++h4yxKujSGIMbsL24N+d5TrTrTAfBZl0Xo0QaW1y3Gz3F9kC5v1ap7DZm2KoOfb8HceJsBmat85As+yKZvtkXQs08KY0669iv1f9rl2lJwG9fdXdhbSU0Zx5lbpVMwq8b2wsc0acbHTp6T8r1UNarVSHJ7d28s71hBG4LKdQfTgfFeBDMI8N1RRiRmUeqD5LVj/Toqwgahs7WwDAB5y8yrq4Fl88zXjyn14lYSfnzc/pKr9aC9VlAHwWbE8H2VICQ8fBCMOyB+ZmFpEeKtVnp45myYHcNPFsp1hvg7Czg7EGABBT/3OIyuFfsdordviHedjipo2TVH9d9Fv/cQLcrZ2Se06QDeJG9lOjQ4+bJhkqSJFQmF6WW1vIADAX7kRMRdUrA5QV1rI7AYxScPaRb5O7AkmfXO8XP/ULG+UUuudyJlPjcG4KMkQFNLml0ndkNBsEwAO++G7Be8/VghOWdB5xnAEWf6YfF9ctO1LMKxhNFSoKHO6adXa0FT55YJGwPwDx2phHFR9yLY991TgpjT1SRbCzHS2/EY3/IjAcw33CBFAPabJWeIMDKfKvPzJ2xH+hvDQGGrcUGmITm91XL4QNBiR3IZq+YZxSbDIk7d0GrHijETqk6CZ1gWPOMPDgRnG4ZnNN3ZOybFRnlOiq2A5Nh7Z5nXCXFtFc8+4RZDreDlYObgd1FZrShp1oGagqM47m0veexCbAiERLo5rg3u8Z+onvi/lpxfZURhAm2YgQePwo4fWDjMuthQo2QdknQCWMh4AuiKDfUevXguAm4YQJaz62/V67ZDq45iF5wvKhoUYGmE3nnzQAKUahqXXqzdO09XqG9cQTug3PUSoMU6/GRVpNTT3MVdcrPbMArsxrmmTrv/Y6PuYPJn3QjvNppqUoiwVLCbgXrSbDe2gReKsKB9/AK4x24UVYrs+L3JBarNTsyWdi8o6VxYqIh91goBNyukxMP2KJKOAC4bj9b+llhYeMIATJ9c4+kx9q05qMom842Fb0gpHhDhEBG4/p9Lyu3Oygur7QQ5NiR2ISeOb3PL+gp9PO/kPDp04xsm47Rp7xu7yJvrEFDmzWJ92ogUVyvGaTlKjK/38alI58uPwCoBks3RqkCUw8zXjJv/DDQJuGGR2eETmRc/ypw5Hg71fEIyc5cWaONL2ewuILpXOcZaMt/eaCHBHoTdR2w2gjW20CJzVLBxuxGSRQPk4yqq+c+qmq3ci6yoeKmuWRRgrZsPQRUn61NUvS0x31XvTLWA5l0BNNBBGSsesEYyTCvr2i8I/MitZ4PgKhSmhDX/b4iAkftY1Jea070rhktYOjqQhGDQteCvhPkLVOvtrrtGVLiNJzSus3d904h3M29b0O3WiaxOTOyPEftbxfXkPpU8rnUtyvRN8baqNbrd5t5aYPwXrW9eQJ+RHD6AGxWzMfQ9zx0lyNw/jQz89kOkBk4bQyaITL0lYYGhWYGIHQR0IlUVSJw+oBqhq5vklkNNEitN0ST6zXDu1OmR4IfKAFR5GYtRUQPptNzQlgeyR6uPvGHG0OLGsXVMoK+BLqgBBDMATUfeAa6FYBEYreYwgx8+lQxWQ5TTwLUWeRmF1imjWjYCPlgfrtuoFMgdILVlvPqYxw2wvvuzCf9ALw4Vzx7rkhg3pYgNh8CdJH2iS7yXjECD06JsM/OgO1a0IlgiIKVlY3rjQDEUFUQamP2+cgW0NHubRenYyTxDpFqNC+Q4ZkQY8e12myqVNDFGtLeNk9xkCBF9QNUw7Mj3zwrDsboPXviOJrrKKraY3cAJNK7yJOpSZRi9/Fc9sU/2hGYVhUN15sGyhgbqgFUD4tQx+b5zIcInKy59qvBXD2hSBPzBAU1VYrlAI9gYJbaHn7+nLnYT07EVHc8S2kSjNcEQ7B8HyndfcRbROyMU7OWwif7Pe1Fh72aEVWxuySAG3cZ588C+gE42TIAr18zXYYEUBVn17/JRSp6aglw+9VWtdbuq9uCagqDCMvvLge6JOClG5XPt70r/coN/bhhEH5Je7MEXOuzi/1DR0PLZkMxqu+YHvOb31ZIUmySolcgboCVh6KbsSkawUrJRLEIqGUJ22wEDx9QZO9WVI90AxNqrTZgJFUEri7FisgyF7CjagWRzMkJPTjcgq9aoy3dnW1OlBjSXF37HJ3NyQyZAZbBT7DZmkcMOB5kEu0005DXiQKWnAgzkVRLyXIGvvXtjF/8mB3Nprt0bt4HYDDVxsrQ8INHggePiQxPt/ysWwm2PYnh9sRUBysyOr3IOIw0pH7yVPHtj9QTtlE0NmLjKKKLlFj6TvDuI4rvnQSER5QCNj1wMnC93RXRA5sceVYUzZwd01hTIoiYlBJJ4LqOv9vtLOT9mkzcE4zFTjHPlJpcijrMRpSaFgLXQqP75LNTnnLBvz7PwO6aao39HovkVAqUvBtXO8VhYjj9+gBLvcs+cP0b24+jQJgnlIAbQX2vVSZfkmB5v4WSSDRPlAAS7kcn3C/blWCwaMTpQKadR3OzdElNgAdnAUOvuLzM+OhjZVxEH/gV8ymfRrr8agDTt/bAvLUFPCKGxbZjkoHbC5zhjRPVdofRJOfrjHkCLp/RmPrxin79243g3XcjVivBoycBJ2d0xx1OmrzxBgSqntzPCV/4/vRjxomr/1XPloruXUXTjicYE10Q4/LPTVDgb1DSOuIzZnCdU1XBvj0EvJlM1206enMDmMA9A2hQDDNFYs8LXdQTvnd8c0yKmKUGp8RaAs2rW8/J6mKO5uqX+PsY6a4UI2dcjaAA9eB456oRUMszgOL/2nlaTuMCNO5Vd7vWzapFKTEAGnjgeDsGOcFQe9sU9OPduX7U+uBGLw8i6czdaciCw1hLwHmuaJcqQiAaDxFAaPis9ZkEEDW6TaoetjcmOvRkdH1PonCycR2sYL2Sapw0X+XWmDpbsIf/n7JlAxwd2arlYWEffZ+kTLVETtWQHTsmEaLfvlVLMaaTbjsszboAjUTgYA7umHgkXflBbxCfi/Ah1Ejd+pDijSFB0cM9XmCRq943BvyEhIVBv438hapJDvZ7ZcDJ0FH100dZ6F61ec5N372YeDTmchh5H8806AWpVWv0M9OyoomIvb2V+QgV+fc952q1svlRQJSRoQfLG58yg9AEwOVlxjgReCnoLko3Y0pLBRk36gl/bn3Ob3aueWrnicftBrCn98+SiCtu+WI738evm/fQ7CH1a71ie7NGTFSPhdXArH7uArYaqgfCrMDefGyv9gxhHTrBaqAhs4ukpDnQHWk/KZ6+yOh6FlvoO4bXv/NeIJIxw+nTF4pPnifs9opPn7qhiFZ6R3WjJerZWy6OEtQBlDD+2TLIHUZu8q4HNoboVSxB0hWASc2rwwm5jX1FyUBMFcGgGovcMv+6AJZw64QEsN2HmoGnzxQfnediFwBM7M9EoV0kkvnKVyNOI4AdMAv96K/OqS9953HA2RlRXGcul0ktAlJ5sFdrwftfDuhXYvUsazRiDMDDM8HpieBkK3jvXQZkPT6ja992TbfALjKVrk4Wtm5eQtfm3XK143wzlJ66X0/nmrKHubsHkBTvlawo5eqmidfqekG/JgofE3B6RlfU1ZqVbNIR4RFUgpsyoFNTONuYVTDURmImxlybK/h8dMBmE4okslrz2sOAkqVPXExqRHynMppJRZIxsq4jY2yZgZrtw9ddfM8kMtPtQAmsC9ywZJQcUwpie1iL2i/2QC+CQwKenitwrvjWx5yLTz/NXPNstTOFICPPQD6Tu8lOyxRN1XZyIlj3ATkr3s1MKEUnhoz9TvHhacb+SvHsKR0SdjvFi+cZIQhOzwI2G8HpmeCddyOGFfDoMY3ifW+M2s5XkGa+fIE/o6laiL4BjAXSBizTo1Y0fQftvjENzZe0WW/xJdNbPMw+o71xBO7c0QMLuuhGMFn47SZYVrKZE3hINL7BfkuqSLg6G3pIqlh1UkqmrTcVeScF9jsaUPYHE70z9bHREwhBSmDPbHrOxfrrEjnS+Irigtj3hvhNxTBL9dAoqFU8oMJ0s2aAmwN9UR3xiKihKVQXudoNVmu5cP0zUWIyzwWiKsUmC8akmJVZ78KBBqxsAVEPTmuVGCcsagRTgRK0cHJC4u3eGCJS9O1PHgc8fCB4cCb48pcjVgPrma4HwdDRDx1AqXA0j6b7nK2g8wxcXGVc78k4xgPndbcjAXdEDrgBlCg7mqqhVb3kDMReMSTmZ39wYFphhUW5mpH5xp50VYhW+0erY5Xgsjr/D0FIHO0tJ/IhVM8bV9sEM9x6srNoqSR8T7hESTsId5tLCn1HG81CWsuWzVHNVuJEZ9ZS/9KTmtHIXamNE/KcqhGeIIJrvh9rIrGU6G3lDMOjeDWBFbI+g/K49BDsnEahSgcQGsPhla4C9ivF7oL9v7ioqsnpwAFOk+J6FTBNtGet14LNRk2KrO5+QepcHUdRvqy3y5LLN7/c6rMXgpeP9faLLi+jKKhAVRb1Qt8eFQoc8TDq8WQbaBk3rqRZoTOt0SenAYed4tlFRkpE5Adh9RYIHenjSrDaKPo19a+xo7EjdvQ2efgoUOSOnLBf/GbC5afMezE2xEG0oup5Zih3DPR22a4F6x7QUTHv6QkxReqdh56E83AggtyZT/o8a0lE1PshCRW1Um9M0TLYYZWZ/uuDKmJvGRAtJcANlyi1RP2XasZDzunQA9Hc105N/+/Z+g57xeWLjE6Ak0gxe9MBUWkkOoweQQhABHlSrHuBnABf/0rEe+/UDH1euCBGwaMHQre5jt4mQUgADvu66z3gJWdLlLSnK+Pljqj5csdQdmbGs+83aizADuVck3V5LvbO1AW9HdzYEYF3PXPchI7dmGaqzW4YMQOlDNfPiY2vM6bpVZlIJKSmsC1hepRcVuZ+uN6QgBdDazGqchDiKqBGJVJUyTZfHsPg+6UwCEGpH4qsJdjpcK0YLWvm2AFRFGNvarTIfdwanovuFZQqswgjR3vBlOnulxIZ5UkvGDIQLflVMok23uE6UeI5UIk4pRagjYHgPLo0JfjSB9zTm43g8SO68H7yScY8OTPJNmkJw4rn+eREcHYmyI/oiXRyYnPsc3ukzihPrqNu+3xMlZt95yoaA+ILo2hrl2il9TohNgf1iuX3d9zype2Nq1DcfWa9oujt+tFSokppADp9QKQyimCfgJ0K9qBKQQK1RmFFL4phI0bADd12JGIP3wnoB7oAZqXh7+JCS0X3nOntoB4CP2mxCpfIyURCqmPGvBNMXcAUAJ1R1CDzTJ3t9Q4sDmCWdkaZGQoXirWealSAUvwWSUmcJkVvPrqewraLTME6N/OooPvk1Xlm6PaWapyTDY3Bj96h2qPrzDAXgIsXiucfk4CPHZHaKgAhKbqV4GrHNLHDmgZABd3O1gONzK5OEJBIbtY8gCdby3A3ETXPE/DiOcXiaXSDn2K/qwh7NB/66wP9gXd7Eg1t3OaIUG3ubNd6cM060w2vi0z0FYu9w5KXmR86jZlErKOVHjsGjiFSUvPqLZqN6BlzzuYz5/u2iyQavYXCe8GOkhLBbAnU+Tah925bcFE61NetDpdGQxQPHHeX64yAF102iKTzrNhfZVye059/b0R633ON14MAW6LsriOx9JQPABgwBVDV0Qt0IjNNieqxkw0l3NVsUuxMFWcXbiLPBaI0e0BrrxLTaYjlq+nt/GBNw7pm4PE7GVcXAc+fUT14dZXx7BnjEcYpM1d+T4lwuxW8+y73+WpFj7Eu0qDrNoCFzh+mgroNHbd0So5e2/+tB0vxIGq+6wQ8H1+/XGzJNO783kvam0fgUtEAQ909q3Jd5L6jayGjtCiCZaFKQsSSrgsDF3oLTx8Pii6hBATM5tcaR1YUyYpaLSc5IjQU454kbkhzURlAJ6zt2AcrfJvVsuO5iOrqIEHfKTpPFSuWWwRoMuihFBumIGEifVb6gkMxBCOsgTrO9pyXpjQEjjuF9iT2moD+HealfnAqeOcRg45GY0ptDU9675jKymFA5oCjoGQaRFhmFPTXbVHjYUXm4ajOUwocrHTa6EmW1CzyAYCjZpiPtIBRd7mi7hhr2lrPJe4obrXige06ehvFKKUEm7uaFY+huXoFTeY+t9iPqIZAr78aAolwELEDoyVc28X2EFFyY0iohuoQ3UBdEfqx0b1u9uZ1XdpyqH2PCRSwnCBOfZi4imqzybIRRqFOOwbmRXFimm19Pfik5LYxiqLKfg8DbUIPHlBX/eQRQdY4A9cHEv2w45q6hHLXGS/Da4hcaMbrKYiLl0jge6uVQLNgmqjzlhBweZnhaX7niZ3f70g1Li8y1mvmzulM6lmtORYYrSlz3FDLZiorUm8QtPe9vL4xyDvecwnkaE3b5v/X78gt37q9vfFIzBiJVlZrwWYbrDqN6S+FaS/PToAcArqYaYHeC6YYsI+CWaiDEwDDSUZcKS6vFU8/TugtsGNYCS4vFT/7s0zU74f4008z0pwxHrjwxStBTY9e3Fy4op0qelGcBuBhp9j23ECHWSynB1HDel31n6pUz3iqWRTCIRijIkVBpzTGhqw8FErVSSfANgJnZpRxgpJhVUGsqTJC7vmHGV0H7DaCkxPBP/1PRXz9g4CvfC3gV//THeZZ8XO/kHB+nrHugLMNfcTPTLzeDHyOxpxioBFsvRFLn2qMxJCkZzIMoaJLRyi7PXB5lVm/8YKIKZmuFQA8LC0O1N2KCM4it64bRwGUfex5xwuClap2WA2VYG821agZgkc02jqMWrIUzrNi3FG9tdiTAeiNUeWZOVf6zlwRgwVzRSlMTiytQrB97OHyXn0mGCLn+h0fgJuPgsgAc1vUkkAqTSxhRn13TQMchCDkYOO5OM948Zw2E90I+khbEMQ8upKSQRnTZjIxKVs9K8/jo0fU1X3wZSLsL78vePwg4Gqn+PQZ1/ZbH2VcXitONrIgUj6+onLwBwCJrPYTrZ6fBwMVei6mWFCgfwA8OIs42QYgC66uFPvdjCvLg7PbgRJ4Ugw9cPFc8MlHCau14L33ItZr4PGTgIePWK3r5KRKQCGI5Rv3cVc0fuy37Qym2ZKv1Fopy/dXUSU2n7UeSq9iaPX2xgm4P0KkqB5is9FtUT0ohDpEItkMRmQBik6YryP4Ib+muA6AOirh4b24yOZ/qsVn2I2PnnDK+zVnM5JqRb0uunaOXGEVVDIlgmziZzXIGrq1CELn8swBzjBwACU5jyZALDNgjIbyA8zY45Vqbp9LT0WgifefexoMT0+IwB89FIyjG7VqQn+mOaW/fd9LSbnraXfdcNpbNF+w17QjqEW1Sgmldz9tiAc1US3iOmxPR9MeIvdQcCaQzRuobpRKkNtN7t48g+0NlwRat8zZfL1zYu3Iea6MZJpuElUTNBgBbC5vxd0zumRVpYOCzGzMxVhWiLLcOJQF3d1yUFuR2kP0XexP2RhORqn16VJUThZ3YIbl2WIIclZkg7WetsJ9m12HW9YMlWjFQGk2muqw7xjW/s4jQX8JTHNA3yteXFBleBxc5lPQ6oT9DDVTA2CpihDrhK9zNON+moHTU87l4LlcPL9RYiKuPLsbKrA6KFaDYByZkGu1orS1svq1ETARRG5Q5BsqlaO1kVvelBsfYHmRIwZ1c/nl5rVfob1xAt5bLoz1hnrqrmMQRQqCJPTGQCfooEV/vTowWvLFueJkDTx8LOiiIE2ZqoO1oB9osNzvFeFccXlhwTPqHiU0UqbEXNNevLQbqPtlMQWugBO91VZw1gV0Afj0ecbT58DTCxaUWJ8IHr9nQS924A57xfNnGeMIXF6SSajtzhh56KLpuDsPShEeyPXAyuZ9V1dJLfmQ3qJHSYmIZBgEDx4IHj0O+OrXIn7wV0U8eUI/7FUv+OBLEWenLPg8j8z6eNKRWJ+eUG8YOhqEXR/sATMePu3pbdNEN8sQFAhS5ldBvfPDdwK2k2LY0Od8mvjMdZcaECGVGBVvDyco9lyIjt4MJwcoxutEF8QSAKSKPMP8x63SfK7EIY2VaXvLapVnQALoHi5OmA+HWhtyshzZrso5PROcnHJ++hVVHMVPvUVwju4SSi52L//lLoNqTAdZS7bDPHEMgKKXmulxPdBbKAr3raZQ0hmcbmgbONk27qdC1DsrSkbK43kIkS690T1ZomA8KF684BxA6SXy6EywXoVq+Ds+46iqEREp+b6DASBAoKKVoPmLXOcqJc7/6SnPzaNHAecvKGlcX6p5NCmysHhJmqkmneeErgNevAj48EPaZh49JhI/PZWidhsGX6vqVOAo3HO3exoPl/xKOyb2DU+47f3yuvnQGVpoXr9q+0wCLiJfB/AnwcrzCuBHVfU/EJF3APwZAD8A4OcA/G5VffYa97Y8C+a7uWbeCBGKcyLAbKKURqLsblCsjODPM3AxZnQhoF+T2E1jQJoy+hU9DkQY2KFK4jmOrgP0quAVpY179ilalGNWEieg6jOHNcPC5xF4/jThsAd++eOMj58pHj8J6M3nN5pXwuEAvHjOe+/2Frptk8hgIfPtNQWw5yzpAjBsLFdFADxneQZoRDtaYVXXgXLznZ4FPHok+OCDgO/7vojtlkE1KQHvvxfw8EHNfhiFhQmimD+6eXN4pSQvF+d6SoB9cWJzGJXqjVw3PudRcPaQ4unmlH7ehwMNuxKolvGAEafH7icbHQEripFptkyJ01zX0fW1c0lDDFxfZ4t2ZSRrTiTUKETdApEGgc41kVg7l3NSTOCecrfPYAlYDqOWSu37g7nUmS+4SmBqhp7qpuJ/3Br4HK3b/CXVsteyalEfFQOugqoatRQJZsyMnZb8IMMAwFQSnkOo70jEt5YSwVNT5Mx5AS/FsmmNsdj76WkXnIAHsXJyExNcIXNvPjgTnGRhAe6jYhMLtQmWBKoQcNtX5Wd+QGyefA9IALYnAV2neHBGkDJPVd2a7XvTyPRHgOLyivd/+izTqLmhkXO1Ejx5EnB2Rt/xszO34wChgcoKqirdtuX9cuMrUBlPkaqABeE+rtJTVCbLbbfQpL1OexUEPgP4t1T1vxaRMwB/S0T+EoB/HcBfVtU/LCI/AuBHAPzbr3d7LVPgunDA0FYoe3chng4rwXqtmHcoiGpO5gfsXLS33CaG1g57Epr93lwW1/RM8bwWMdbc3Os1q9qPqsDOCb4dfuHvQibRioNivRWcJuoMPblR2vEwXl9pCZH1UlIuigcBOig6BVbCiim9KDaxJujqI1BMuoqa2OloFh3RbraCs4cB738p4MmTgGHNQI1pZsBMNn1wylL6kI1RZgELJcx26LLPud7YXU7zdns+YlQkcGwe0ZqSu2aS4Ba1xcjrOaoazbDpSBawKkyhRdJV7TVN9NtXpQ7AEZr7Cu8P1b0xe/RrsWXYrjPEpkfvw4cpphtOiilzXccDR+0ZHB21ilS3QAlVHaFmjIVQUl/6+TZo2wCFh1B7n6nn5u89OZUEUH8MY3Jo0KDtfY8XyEZ8o0mPi0hQ47TlfPkmUgBS95sacR9NHSkGS/l+pUROpG9rZevYixLMUiif1i/egeJhW7BU9+mkuBy2+7JcekFVK7g5iOLiXLHvOcb9jpWHdtdS7CfFntFRHeoIPNi8eaxGJeKVivn9GwFiGcnZjvF4Hj5n+0wCrqrfAvAte30hIj8J4KsAfhjAb7Wv/RiA/wKvScBVK3frB2Bz4iiLn8/AInd21zGfSQwBz+aM6yuqOfYToMJAkdgJNqeCRyqYRsX5M/rEXl4xv8N2K/j61yPWa48kpMFusyZR2z4IGDaC9FTxyXmG2uFiUJwgWm2/1WlAnBTvDsDpO+zvfs+yai/O6ULnTGOe+TzNWlKo9gFYgyHUDzrgnZWW1LjuYhjFmUfD6cWQE5oNI0zU9e6XI77v+yN+/W8a8Pix4OxxxJjpnXJxVQ9khl3fgh4ciV2PzV5y31g7yGKSAFD33+FAn3cnYgpGN+72lVm46K8JpaC0gigzq5YMhx6mL0JPkmFNg7AbnPd7qg9KWTH3+qm7qfnrAU+Wkc88SJzQ5sRUDHpLMiu3c6gycGSe6AZ6cI8l00t7xF/XoaQM6HpLZmXMRGEV7QPtF+6qOrnHkzGoNpPdbXnVu0a07434JrtWjJXA9D0JfRBB32vJ++7ujn5twKQarQSqEmIpn6dk0oalXJhd5SSgzSkCp6eheH4dNyfcnh8nGNcoBRlaj5BjpHr0WsSD3Hh+1hZdW+wLx/dvrjGPijQpDtfA5Tn367f7hK6j19TJSUDXM0X0agA224CTE7G0EZxPeraQTgVPBeEMRD2TpFY3RVQbQ9ljDWMphN0MRqWgyl1M7I72WjpwEfkBAL8BwF8D8CUj7gDwbVDFcttvvgHgGwDw8OHDG597JZMgqGHkDUf1hOsClAnte4q0qlqQkNcDFAAwTwleq27GcbT8C3avlKo41xnnHQbqxvreC9qKZWHjaxsTQqQlf7BZnC073TQ7IdeS4Kq4DOZ672IkFLoKDkGwitULpNBLrZkRHBndOCtCYrI9FZycCc4eCE7OaAOYEqhrnRrmL2AItKOyMtlHD1QELM2GdcQxHTinEEDMXnBl9SzdmKQK6AQga9E3UqVFYnd1xYo6auvrIewrR7lGwN2nfH9g2tHC/NEgS+F6iwDaC/rMiMsMAIG5cVqblTOnm5uSj5S0JDZzIubNC3kQrdUoS/+5Aw83NjowyZnMy9UQJa+3EyE1t0Wg5Pd2o7K7dKqiBBS16LM6BNAdk2fmphG1VLH38WszbFPQt1k1mWWR1aCSSWheOShllBqwtzW543XttH2md1yjQSrHtgTvfykC4xKQ2zl8LZRbN4NgQkD7jUuAKeeS7301CKY5I+dgHleUQpn6OVg9VEE2hwvPrb8YEypPORIGFmrw9v3j379qe2UCLiKnAP4TAP+mqp6L1OVQVRWRW2+tqj8K4EcB4Ctf+criOzkxXWQwf7Q+AsgU6YGluCWwYJY1gCx4YYQ4ZeYqmZJgZ8QPsIO1Ejx6IkgPFHNWPH2mxUiUMhfvcFDWiDwl8X70kMRvmoFn5xTXz1+wSvrVWnG1tUUw1Dl0tGjvdsCLS6Luyx1dGVtxWgMQesF6E3D2ULAS4FHIWAXB4w3weGU5QgIgaBCm/14rMT1e5BCAr3wtYvso4t33Ax48phRxfqm43LmHgontdlAkAsWvu5nj9hlo7tkSAVsYr9DDdeZYL68cgRtaUxQ5feiVBjegVFW/ulJcXdpNA4nLfq+mHzfVQracKDOR5zQuCVZJTwC6ngoYUSrWtwSYLlIR1dRsHrp+RFVSZi6dMbHAxX5XJztG4OyMDJ5ur9X3vPMENDZXBwv7dnXJrPSCWfS7Q8mN07tazf3K0Rh1wT1Rru97ODVE20GOHIEgZfCRewZVAmhfVL8WGdRhb6qoPaNnvYCxvzdOjEpm9KsgDDQeeurlG82X1u9rlLpsJx9WQ3C9VY8jKftpsuyFFxf1MZvR0n3RBebV1FW7EiLqnJa+aS3jJsDzi1Q94gLKJIowSGi9CdhuBe+/HzAM3Atry0y6WklxsQ0BEJP+FsfVxu1npVzcvpNQ1aSv2l6JgItIDxLvP6Wqf97e/lBEPlDVb4nIBwA+es17Uzd1YFIlUYolOVDU4n6zvAY22CDkkDlp0XkntbwnWYvo2XeM7AyRecNFBU+f5pKj2XWPc6rJp/qeB/P0JODsVHB5zex5AsXzBOz33MiHvaMPLlQfiJQOZkg7jMyZvTv45Nmz6TH7FYsZrANwBsEqKM5WghMLwOntcM1ZS1i/61AL4jgm4AI8eTfg0ZcEZw8CNmfs39WVmu64Zt4rATQwJN3sMIE0G0gX97v1YPk6as3XcXWlpoeu1WNKquDeGDBqbpn9jjUifY4gCuwBRHObc93w6N4GYOUhO2gSGvQa3AUQCFb8GLDxChp9MW0YuIVoaOahPsxE/btrLWHwbkg7OyMB327FAl64F+ZZS+Wgaaoql5yNgNvc9X3Nf+MZBFdWaKGLFtUoKNWIyLzdYLe0DeREl1QNUup1co3E1o8LnFGZdZmvcm0Cm3muOWeudxZJaxJlSjTEj6MWm9EwAGcPFd0gd9sTjh62s16qM1d7Ic2b4mOdyRx3O8YWeKqKeebc+frHyHMfI+hVZevf8tms5kk1Up03zlXqcOnRAQoJuODsLOBqR+Pne+8x589qJeYhI9gIID29bTyrJO8n5eCqMWEfJ5/kxjy9SnsVLxQB8McA/KSq/nvNRz8B4PcB+MP2/Bde474AyPWvr0lcZKD+OtkCqAJiRJmHVApnjWZkaB99T8+GoUcJ49YMiKX52k9A6AFEYDcCGujV0Q2C1UawPbVq5ub5MfQMCweoH8+GUC4ulCkxbaYnFUzKzfT8kgRzPwOTIefi92zRepMKdiP7MZlP6hiBMXLRZ9vBI2goyihpOWC1KWDea6Up7MAl4DBmzJmb53CwCFNH8cCyhqFzf2eQ0Oo1YONbHCL7J3g63OYwuJQQYJnvRJCPhDKJDHiCMZFs1/U6oM5MnEgnRanynmf7QWPkywGWt8b6ZVJLAJCtsIQGlOT/drtSAAHKuV30MTDkXiNwckpE5alPY7Rwb0NWc0LR54qgFuzItUq6G1RjANamdvGgoC4aIW/m0kEGtOaGnt2bJumRYZh1QLcWGh/FkGMz7YVoavX2AXiNa8vPQ+BB5tMS7b2pTDwvjdsj3HNLoXj2TLHbJ6x7Vry/jfi0KNSryJf1BtfohpTXgAYBjZDn5xm7a8X+mi6wyEqpXQVpZqBTjDWALnnKBQeIAbUWq+/zQIAk6gU8tEgxLqm7EZ2oOePTT8nA5ingxaa6QXcdcHIaMPTUrTNNblX5epK+cpYMdFTJSCpafcX2Kgj8nwfwrwH4b0Xkv7H3/rcg4f6zIvL7Afw8gN/9WncG/XOfPc1YrYCH70RsBi7m7ETGNnDouAgRnIC+12LIWK/NBXEleO89oqMPv53x4YczpgNr+6WZyf37jQBR8OJKcXUAzk5JuE8eCt55P2BtlYBomALeeRSwXimuLxVdDEgHxYcf5YJIVYGLA41/46y4PnDRDxau7n7PTIHJlKL7BDy7BsZBcboRoAd2lotEjLoqgF1mgJCKwKvBz6DHQrR8zwVFZeD584yPX5jI+PPcDE4Q6+ZA45d8s06gV5F38d4NRyVHuOmou075+wiqYtSQuxI1rqOjPd4zW+oD1epBk1ySiEA0BJcmd59zI2dF9pKWJ5pJnJg3rlXXBBtvb0h26C3lQUTJiVGqlCs9RdojEyOw6QU9BJsTLeqIaAFHw0BPpAyuLQAcgGp4jCgugK6f1UwGsDmRojMPDVr05ogvme/3bIRzv7NYhbFGtHrg2aNHgneFhTk6k/JaO4mIV42qyD2Zq+KLC0+6RiBFwzPXYBwtJe/kAW81p4jblHAALi8TVIEvvxfwfV8JNxOtNUi/qAXNkO60ytU7cCmpkTKdzu2vFR9+K+PqMuP8WcbhWqEzo4ejAC8uOD+9FbV2P/js6xG05NNxbxYnnBaOYIZ4xpgEqd5jOQNXV5zDywvmN/KqVb2tZW81Xx89Ijo/ObEo8B44e8AcTKdbpraIlpcnBACd1g405/RV26t4ofyXuPuSv/3Vb3WzpUyVg0Jw2spVjVwrqjhWVLYEFKhcd7OhKHN5YiHVwlB3TQxOGdam/nBXoRXR97DmhHZeIMAOYt8L+pmIqR9sQ6sbokwFcCDhnnPNnaIQU7jRiCYRkE4gnRVwBqNIZygmMLPikOt4cwb2CozF6EZiPGcaU1YAVu18gAfu+orwyHB18SogcdEiCVCqWYp4AFBKvIWqnug7ugmGUPXGxW0uElULtOZocbWJXQ+AZaxrXOzUdPJKYpIdYZrXihd08DFDQXWIWwTtkcU2wdHuDEqVwlIB6WOsz8f67/Id05N6DUY3JBYXvAYt+1b1JFM3FMFHaLg8WniJKn2U6Es3HhoaPljSr11JSaDVwJhQytGpHQzvk3tCeDCQ+9LPFqh2mKjXHidtgtwsW6MFunlVKSfOx+6dJa/MLUOnH38NbiugwvZ0O35Hvu0FFNUwOVk+HcBUpT1VG9HUll7hyHXM2fZNaq7vTMIlKxXuT6Ci7ZwJUIJoSc0MgAVlBEU1eAAZXAjAaFLaZqOWnZNSwRwEyYLdUrLSc6LoMqjWUoAJjm9JqvUK7Y1GYu5H4FtPmcNg+wQ47QRIjXjFJA08vMbBD6OhBtPRMesYDQo/8P0RX/1KwNe+qvi+7wu4ulT8o5+e8fx5ddmK0YxOEXjwgBXD12vB5hEDHyKoSug3gu1ZhgyCs2uFrBTjWnBYCXNvf5JxGBXne+Bq5EFJprjs1ixhtrLN4mJ5MZwFYOwFnwrTyz7fA/1c3a0AlCr3Wak/VRgaBfDlleAraKTNDHzyccI//pmMnChKqjrRsXJpXfVGcCYfqmkMABapTR2B9002vc6IetdzTMEyPYpI0dsG06PTsGZX74BFZR+txNqLOfjr9vAV+qtaKhMV/Y4AatftY42W7UzfOURD3+ZPz2RgLBAdAEh26rZsCkoHSWoEJlUuLnNXPhJAAlCySTZePdKRgcyw4gcJmPY8yHOo8+yFIEo2QkOph53i6ccZ40Fx/oIlAr2SfUtAuxjw4EyRV0x1HGMtzuBzmjMDj+bJCm7vCZ52h0qkPde66+4PIyNO/V7wNQ2mojQvqyuLMH50JjdsMy5RhEwdusLsErnIZ2Q2buzzzG7N3s5mVzhYrdM0M2I4SoQq8H1f5/u//GHGxZXi/Dzj2Qu6re5HLRkbAbp7nj6gO+CXvhzx6BEPnEQWzbi4zDgcgKdPM/b7jK4TPDmtOfxjJ1RjHUjkD3vOnwjnUMzd8MFZwIOHgsePQvUbD1QHjgcgmhjqe0vEM0KinPlXbW+UgM8JuN4D/cDczDdS7SlqdjxDJ55tzd27clYmw1oJHj8WfPnLAQ8eKB48EJxfKJ6eK5LUk9p6DTw4o6jTdYLBEHvIlKtDr+jX1MmvtwGzKsSiM2YFxgzsJqbT3CdDc0YAe8s9HbuaItNFNueyKQquSO9xOQNiBLwzCj5O3CzJ7kUdJtH1memzCwFXGg8//YR1BKe92uFGQc7FZ9aa2LVatYT7eoeiK5RqyTcR0V05JZJ4B7O6e9kuz2pY1AMCSA/zMKnr6oQlzc3rVIm2AovD7CHYMD2hBBSzvfT0BKKkUxGzu2q62sSZiqsZbkPoOLq/M+BFNGXzPWcqrU1ABKZaImrzNcqzluAeADWDoX/fiY0SEV9ekkg8f55xdUlDW/EKsrbfk4DGWIsCuFcJKxrB0vcaip8UV1aJaJprumRHnh6INU7AwQOsjsbp5dbmSYuhfDzchI8Fdefq2ucqFNpYtZ6JVnqpdHwxlmxSWVWR0Cd8nKiaW50z787TF6k4KThi1sxUHBJp93r4SPDe+5FM1LJndh1tSVdXtaLPyZb3WFkx88OBwUDTxLnVXAFnzrA8+Ix83m6lzBfg0iW/G2ZYMXYxqdslndeD4W+WgGfgfAI6CK4SVQkzaJxS1aVSXylyQLQUcN1uBQ8fBHzpvYBHj5jvg8l3BGen1b1udSp48ULx7DmTWc2OhABcT3ztQS3R0N5uBC52VPF8+1nG1TUw71nE4TABqaesFntg8EPqfruDcGajQD0031GjocsUjCDnJdLrTAc4ClFgAvPBtK5HuaXE4PX7nsbWKSh9vsumUYTBQrsFxdqdZ+YJafXM2hAXEiRtEKjrzxs1kxFwEQbJQKTUiHW0RpVLTVjlumdHiNmCRRQVEHuul3INoASyxOhiLYtax06wtXJtzAtiDCegJBwTI7UuGXSGznPg3LZAfPbKQKo4PeNhBJg+dqHa84e9pyBxdD1vNmmCqNE9pEqYDAD6E2+2nNfBvKbcDxuoe6LvLB1qS9ns3v0gBZlfXGTsOgawTUfeMIeDliRXo+UCGidYAWkthaRHK2E3jfxcyhwCu0kxgt5YezN2FgZwxxlvP1uADhuAGBEXoBQ8LpKWzUHsOE/vfSmUlLnMI8S5mWYgrgWXV4oHjzNOHjAIzHOUu9pmu6GdbL0G3v9SxDtPAqsidYKUWa3psKdXTWc66tMTKQBxtWLirv2ONoTLSway+XiGXvClDyKB4SmL0LTrxTxHUiVcA0xo5uQOTHFne6MEfMzAswNVJOez4FpBfa9b6AKpSiku6ocnwEQVwZN3BN//dYpDD84oysc1P9+eCA5B8N5O8dP/OOHjcwvMsBwWF3tugJy1cEaTynGYgauJm/ijT2no8VwUOQumFYBe0BvRavM5uO8qIqDmrub61mScOgv14H7PzohVbwTuYMQ7C6UAVyUIqp+8NwGZ1ump4HANzDtFQq2oHYNgtTE07nNveZypxzRC6qoLk3Zu3U0+NiPMnuDK9QjZ+uwqApFa1ouEk53Plu+j3ENQJ596rFIezqMM3TuoC/S4OO3pdXS2ZcbFaEZLEZQ8KCUKUEmEnSBu1sJUwEdzOU+K588U+wTEyLwZqpUZOaosBNyHkJkRj+XHWGXI88wnQ7KOxDz68cEDwWOlAbK3/DluwBOt7nDDIEW/66ot7896TSK8N+IrMERs3ioeDeu1RCvaNrXKkQqlRF3a+wQVZDz7gyKbHtqNnvO8jD5cNL35sjVZuEQlizd0ISmGRgI82cbFxVy1lbLi3Q8Yq/HJpxlf+TAWb6SyZcG9cWrxHu++S4cHCYLQs1+Pd4ppVjx+V/HkvczfDJSS1muuA2yts7K+wGGsKq0YgYcPmA9pGCx5mNmU2AcpXksOxlJESafrj9dpb5SAJwUOiQf/6qC4uKYKozO9d0ShFfR6EC2uVytzn9puBZsNCXabVN5dgPZmub/e8zHPJFhq3iJMKsX3oRYAAjKX/Uwitze/YFJUU2dAKCk4ugbglKgEJUjVl7rBRtEsltZnN7gUNHgL/dSFnLlsAlTdsyF2ZLfw8+buVRIDINnQ/0xCk4IwB7cjSBRp/ubhNKmoEJImMGYuY/VNKqbXs0qDXtTCvUtsrF6VHIqi73bVh/v2d1GwHkh8hw7YrpgKd92R8QW7fHmg4TfFOFsDf5BrkY1m2osO2aNng5hE42t29H1/c5qJyJKH3qdGwvB5VKJs3sdCqI37S/MdF+G90ImXbfN6qK6ycb11cV5QS3FsaikGQtUANrcFZbXkVLM249USnNWmsk22AZJJbbUoxs1UBMu5rMStpMSwfS5C76TCAf1s3LzIrR/I0UfB5mkwuuBM0L8rcEmGzgqQhqHOrYRLO8JqxTPd9S59SjX6BwKCfkV04Gqi4t0CKVktJTcEXLXYnZweJIDZV1N9vD0IPAGf7OnX/FO/rHgxJZytBO+e8ECeiJTCAsHczU7PBOtZcLpRzE8EH3w54v33A05PaJwAiCKu9oyM/P/9vRm//HHGL/1yxi98MxNtNNnsgOXh8kcWYA5ciINVZQ9+yOC+2WIoGUX/yyWWQoDd0djVE2KIw0Gn5Hrw3NEC9n60qiu+4PQPFhrgjpsyqX2EYj0AObFw8WwbKCvQBcXDRwGblSCPinRgPpjLKwZD7PY0LKdZMe3N22aqLmiVAPGWHQRDIAp8/D7zSVyZXWBK5m+vimRVx+esOEz1QFE6QHFR7C04atXDUKngZE2ideoBExvB4zPe98EKGEJdv5QN+XrstFBP765/qzXtE2sreDGNwIt4NI25Gs7mkW6o2lWi4zMvQCl87IUXXrxg+uA0s7iCE38ntm4IdWJLQkgXSsmAJKrvVCmRbdeCoaM3xNCbnSB7dsZqNzgcmj4ByKiENdvemZIFRplBMCfWzkxTI+WrZUJsiK7mKqmlka576VWIt5oufaTeP+wt9NzOUIC7lGpx5SzGWSz3WrmPg5oMlFA/S3pW9noHPHgcyvedMTojd5vI1TVL92XV4tLqQmfO9EzzOYXA6uNWMABQ5bWKsujfNFMnr9cELaUfgCUg08KcCfEBdIL5AEzX1Wbxqu2NI/DdzPSPzy6pT55PgO1Q84K4HhDiSMwOeQdAWY9wu6GukvpVLWlOr3eKT55m/PKHGR8/VTw/Z9UdL6h7HJpeiTiT4GvnCJv3J8rlwibUzzKsjmdzwFEvtXiv9f31JEatAljLD51rV76Sm74eN21+4+HCIVcUko3le9ABOkCtohAJgrl6gcFEOjEQxzPvtWOqemAtVZNOt0Q/cgBkUoSJAU4p17St/C0H7/ZqR/FoCHkX6cXDGo5ETqcbRj8+PAEenwn6AJwZ8h4tVTBguv0MGlkjSkTcMZrte6LfY3MCgKrHdgQNQQ5HEx9Av31UhF3c/GbFYUciVwoRByCa/7wbVMt6+vo3z6KCruM9SxrYBNo3nODYHvaqQj6frY6iqO5ydXUdDXmPB/reFy+bZiyO1LVRw2TzpvFzc6t01rQS7j9bqmP3JVcymWx7yYFLIeCNsfb4Hm3fxCkzLMLYzlM/NJtVXbKrkY6qRmQNFE2zASr7gu+V9t6tFNHadjwgzPvmWTPd3uDvkykbnQgWKepSZ1Kk0QBDY796lfZGCbgCmIT6u4/PFVej4vlGcHXJYIqvPRKcrpnT+CRyUs4vmVns4angbAucrC3znKHMlAVPX2T8zDcznr5Q/NKHio8+VZxfMV8KnUzaogjHsptysd3A4OoQ2IKjElMnzhpss9rlbtMbu168uBWhEnL/Wftcru8bEyhRZCyEUG+ioF5/d5kp4nrWOzu0uifKSCNwtc7Qg6BXxhDkZBbxZIFSgIWd62cGhTnxiFHwpXcDTs/M+6anq+WLS6bTPRxqVGG2wfjZC6Fu6iESpWwHwcqCmxyBD2sp6p94YDDVReJEWbAt59kOR+w5X/2K6YFjR08CpiNmpJxqTUDVroMnjkJWllWDIidZhKDT75tjcZUVVXv0MGI+G+YL6eywesWozoDJei1FBXL+gjr6eabefJ5r0M44mY0Gxoi1ElGRCnCcGVVVieLgft2W1phSFa/RRRKVQoAy1Wg1+ZrWKF5nMLft6zv2yYsXdIOkXycX3IkYskJyPUsZ9Z5qrwsCx5KYl3z+UqWiuQFkBXD4uQEK4+w7KUZ5sXM7u9rC6EEBFU1rmYh/5nVS23mgQ4DWBFv+W6UHVCjXN/AwCOu/zopsOYRK2cFXaG+UgGcxAq7At59zdU4HwSdr4HQNREQ8OaMRr19xI5+/yJhHxaOTgAen9DzpLY/EIQnmDHzyQvEPfibh0xeKX/gWCy6ME/XaAKoMK5Vgqqsyii8Ymg+bjQQj5FrRqKgFQWBZ+7Dl1p4VrmSuA1jXEA2xFin3QNmMWpBFLIlyjk6M0sf4+qJWJ1GlaiqBG3R/UEy94KID8k6wicxBrtkQcwY6ZR1OdWlCANx2OguqIzqLkZF4770b8P57AY8fBVzvFM+eZUxWJWe0PNqzc0CbUDcsBrE6nAKcdAw776NgszJXMLvvbq+4uFZj5kz1So8fEu31WUA0wh0Huoz1AwM31obiPXgr34LAxfrQBa5Bmul2qNGZvlRdrvXJr+H5dHJX3UGHnqHUdFezaM7OAspsr0wTsLtQ1rXcW23LpNiNdf2zVuQGwSJToRuLfV3oQkc0d3FtRlVLnewEPwDYdEAvTBORZov0tZD93CJtQ8ULtN0S7tukGAWeP1d8dMiUtFWgStVXFxpEr3V82RAxsFTRVBXZEQEPsKyknulRFwRcTPyodhRK6x5F2XWyuKef8YW7rdTn+loW74XgEkwlLbfNhwfttNfFoJAOQFKIZUps7/VZ7Y0ScLqfCaKiuF25njQGweWOObL7CGwGKQUcMohC+15qgAoEh1Gxm4jSn75QvDjntXzhy6YLbR+ceC771hLrYxHOuXVrJGkJ/mLRXZ3hodOhusYtknU5g7A+qolmnlpUhMSIQTl6Y5cEc6lq/WVL/61vKbF0XFRAoiB0asYsY0CT+dyam9YNMfaYlhuDocogY38NyChYp1pLcu6ATQ9MIsWflxBLytz5w2NlBgB9pqEoZXOntAO+H5nf3QMeMlgWLnQm0nYmqXSoapTOq8ajqRK/XLPlvmz2QPlT3ywMtvltRVn86kK0dqRp7oRIlVGw4gtF6Gz2GY+ULAgYR/0VoCQiy8tgLIV5WrmxMXlpNhIP8ahI32eoqNVzuLQPtWvC9qZPhRvU271+o2nzlJf3apPhqNYvin3HI4iJtKlucRWlp4loETjVhvyn1JpwBB4MKITaL5cqHIQpGsZgyE5szO2zz3FLaHMQBFOxFfVo+bPcMwufVbH/IyBZEZKVG+xvV+3d1t4oAY+Rvtw6s7LLNDFh0/mOIb6/+GHGs+eCq3cEaSYMmxMneFhb5r0THtqswMfPFJ88V/yjn034uz+VcLVjpOTs6o2GS5Zmk6gtx7c/RRJr3i/qDCPOJUBFpDCGariSIqqtVrIw2Alo0CmtuUdZbLVKOysyrNWK7lTbrEwx4EMQZvl78FBw2APnlpu7TfcK4YF+kRRXwRC452AwAj2qYjS1yyHVQJuqk9PCeIB6yK8vM375FxL2zzO+AuBUqJ5Zg/NzGASpp0ohGwH36Fp3b8tKW0hS+t6P5q/8Ykfj1PXMIJMJ9EVGBMJaID3QbVivNHSCfmseAz0Jd1wBgyUa8lJ7oWtQ6y06r4KA1JhhrN8tlYCAmmPdAcKs6AOXTjKlG01MTeuufLQ3oAZs+Ny6SqTdl/YsxnT8NYDCCKdc9dbzXA2c7hbrTJhb1XLoJ14rqyAFC46ba+7zkse9iJKli4U4OiEt2R+PVA6AG6aJhJNJlxFgpsgAiOn4FaGkWtBcJ6GuTK2Oc1dz4ls65y+15vcpRBcOFpgCwj1JsutyjvIc+Hl043A2x4Ra5UgW1weaPVT6wf2SzaVU7RqzuREGUXTQkrLYXRY/q71hBG5ubSAykkSxakqcw6udAgm43ADXB0FQ6mU74fe7wURKMLnQfq8Mp71iop7dQTF6LmR+rTy3qg6gIZzN/4B/xzZaA7luiFTttZukUNIQbUfhLcITe1H9yO1DQ3AeRRkt2KOLQJwNujUtdlLyL5SN4wTCiZGYzze0oFG/FUBDUFKtvuot+m7GXSfJcl3Pgt1lxrUEpCtF3DNXy1kgkxpRnHHKOP06s913BnDtiNsKBueD4nCtmBIlqzErA6A6Esh+gxI1GntD37Gibc/V0iJvMtTmRL+kufqguBOGZdUUf93mL6miGxeg1SfPpjLwTIJ+Dd9sgmUfi6RmqPtYMshmaE1zvb4/F08GR6oLYw4K0/brpGacd3mYqHWyBTT1HNycTBcYnMi3tJHvy3JPgGhWjsbqa+GpGRbM1L4T2rMJLIl5S0gb/XiRpqGLta7Ivp6lVhL1tavn/KYkcpsaJE00GhcCrsAUGNQXRZEDSyzqsTrgJe2NE/CqQzI9UmB5rVmByz0jwgRAGjPWPfDumaBbAXHFYsbSC1FjUnz704yf+yY9TsZcE09l28TuQwu0i4Eb+r0WBRGg85323LsrUUXgACwsOnoGP1mK6x7eWwiA3ZPFbWGV36uBK1rE4Hpl4mTJlXATMa5PBA+fUO+7OwizyV0rJ9IHoyhBQWKEJ/i4QOknabXoq09U29TxUPkX86R48WmGXimeR5aUywKcShuub0TOf2PX32Vgr4pRgeczpYCLmYh7VOA6WZ9XAAZG3K1M6upPGAnaDUCwNK8uFcWO7w8DihEzdDWw4phhl/FkhoUfDkTPh57ofXfJSSqqpca1Mtse8hJ9ekQE6xTqghC0+6x1N+TeYiEGKA8+UWI1Krq7mXvLABWlc09ImXd4f9y46dw00wOlpqVo1I3N+h632sdKpG8lOU4U1SowqRkxsawh6QTczyq07vE2SEibvz6vC2PnEeAoleS9o7avBShZOf2KLrl4VPDx+Nsz64bk1u5wq968IHjruRmlXDJTAaZeMFuk7WZrwAfLZHUva2+UgAPOnT1dKQ1FGVzsqz2wB31V99eKB1vBw1MBekEYBN0qQDp6W+xH4OOnim9+K+PTF5keGIBV/a7qC6AusOdZaFsB0g5YGuASDA4d6yMBIj14ClbLdNYaN7KV0fJMcqp1czoB73rBpud9PJ9wH+kXrZmeJilhkU/DO73eAqePAlQyzi/ICPd7+1zLn0KYJ1h0H2wTiOmZ4c+vlh1NQAJz8Swjd4LzGbh4RvfCJEWQQJEyxCQBISq/BnCpil0GPp4VBwWeZ+AqM0ptHNjB9RCoBtkKNo/o6dJtSJR9HVrPoWjGYs8kGV033iSluo3qZLUw8oO5vlk6Aa+44x4+xctAa7EMD/5xQl32kVQA0QIWomrzSogN2vZlc4IBRgH73smZgUKeBAyZ1xuM6aNlDn6NYteopeE0cT5mk3hcjXBjzXXJ8BzAeMDUXc2JXkl/qwyccgbqF3NQ3AYbeTpelwjqjf38NvlzjFGgAWwtcfd1cELdrotfS9XdixvPm+b7rQ1oCV+WrxaE3J6L2tX8/Ys6DcC4EaSBqQLymvESCa/e3jgBX4qc9pYpIKkXs+RRM0tdXe4o/u9G0y0GIEwsonCYFfvJHPpL0IAs3JWcG/thKUbMZmEXrTnvFX43Hzenzolja9V2/uubya3tqtV9SV0SiYz8ilZaRdWqyHue7JE3ST2AftFFhMCcDd3AwJWcaFg5PmD+bxbW9MwgwvW+mb3rVhH61taiJzOAppHXF7c5GCfIQnXKLMCFUAt0YY89FOdKKWCMQO4BdEC3sVTAW6DfMKdLLGl/paxFi7SKbjLUNYYZxRYSE4BjIq6JxYwPO12g4mieP5VI10Pt+tci7TXXayMnF/cp8+uovKXcfMpGOXmbSrBcf+7SXsmvYWNzCcBF9ZYQtns920JX97v6eSsdHE+TCEo++qJGukVSm2fzqpmBqx3B1BTYX3ph+Tjr+XSEm9KyT0WFJPVePh9AVV85MGol3BsDsP/bt1RrtGqRoJp1+qzzcPzx8too/ZVM6TRHAEEQNnx0G2Yx7ZriI6/SPpOAi8gawF8FUX0H4M+p6r8jIj8I4McBPAHwtwD8a6o6vvqtUThmybsBoORCUYtyVMU+A+nAwz18onhxBfyqc+BiBLoM7JRucs+ugefXit3MqE0NJBp+6DySqjc1RcrciMVYZ93yFJTV4MEXbbInBV/HwAxnCQ16yFT7RM/gBxQ0m0wNkjKZkqob14B+LVifMq2tgMbEcUf/bs1AsP4+ObPowGaX9ANLtaUEnJ4FxE5x8cJlhxvTTqRpmflcCOEy6A1j+fFvgaPzoIo88d7TTjEiYw6USAIAmemmOAtwCVZM+2UorgA8D8AzoTFn13Njx60grAX9Gtg8CIi9VTpZcV37NQBhKoMGZAGwSDu3NxR9cqumq//flmFwnhWXzzPOL5s9eUyghMzDbQi3MfVo9g4vvdci8QUBxZJQOYFeoMU6zUWHGyLH6pnvFFbt3l0GLV2Eu5Rqqr/1XTFrvbcTwoV+u50YKcJN0Wur0r88p0pIy5YAC0G8eJawG4GPL7jvB2M0LiyZhqf8psyHVmClqsW/v+1jCTZS+ulDl3vBJe6q0kBlBM1r/99BVnLJvGEGheA3hL+tC3wcZHcbwWc6CUA7IK9ogD99L2D1ULCycoODSKnr+yrtVRD4AcBvU9VLq435X4rIfw7gfw3g31fVHxeR/xDA7wfwR1/91rUtDA7NSxJJq7hiiHU/AjHQQHl9UPTZkJ3lOh7nxvOi3XF2n7JggWG4nmPlmCPf1lpyKM1qimCRbEuzoerMUOnFodU6trI3Ao1vEqt6xkX0aSKKQS4JDm9a48WJlpXW6lCCRxYI6mh+veXmud2Ht4EOaT4rOV0AS3UqlUEpDY4zpDCsA4AdFDuQkF8ZQb8yQjwFQM2DRNb0MnFU0q1QPEhCkFuT/rRE0t3Q2oPn6q/FgT5qqrDyXHqk97RBu2ExakGhckQogrB/pVQYWmK0lPqKmmRBwCvDb7puBJzvehpaL6Tr/VYBkI/VPMtFLXuvUTd4H29dbO8D6l4v19LmHkeNBtWaenVKACKdEATVC2uxnUWKrr/0tZkzGx5BQ0Po81HFpnY8txoWy16pybRahlFeN3p0T6q3OAROFLSdgzuMwGLqzyjAAMggiGsCN6pca6nGV22fScCV0OPS/nXhXQH8NgD/ir3/YwD+d/g8BLylZLCFNcWp2uc5UNSfQBfD/QT8w5/PyDLjwanggw8CUlZ88oL5vw8zSgRaAfcW5ecRcSLM5RGC6chdZDNDhi8ImQg75x4NQOW+5dBhiYgVqFFvaK5l44yBxkkEIudhwz7tDxaEs7ekSJcZ188zgoAZ93q5lbKGIOg7LXm7YyfLkOCMxcZwAgxoY2ypn91KvdvPm5ZB6egAxYus+CiRcG8V6KB4nhS7pHihwCcZOATgac+Mi3kt0K0gDMD2AW0b6zPBsCXaXm2lBGG4jtvBnjMoIlz3Nqnpbn28WYEmJXwh7jfUZfaTotv1RWtEknKABVBLxIbmPYDqI1cZaaZudTH3i8N+dP9jAmM/DaiMRMQkjcggtq5fSpnThJKf2xm3X6cluFmrn/lxf25jdJ7SNyfaPXJWTFZL8/b8HQKxcntBLf5A/YqV4MpiS2sFRD6tfm59XhRQtWo2tkalOEm7plqfWhDVzrP/hiCkWW5fY1lciq/Lsh9T8zpud+1yRg5hltR+Re+pzbuh7O9+JdCDYjrnBOk2LlSkL2uvpAMXkQiqSX41gD8C4B8DeK6qHvT5TQBfveO33wDwDQB4+PDh8sMFMqgUvIgmtoqO8hKA6wOTyf/Shxk5K959ErA64eJdXCkur6kDnwo3lYrCPIjDRNooPAQ5AyFSzzxlRqY5QinLIiY2LvSqdfMrKhJrkVvyz5vN6huyM8+J1YZRgvMMC59W7C55OA6Xit0LRd8BJyuxWPebh6UWXbD8G1Grntg6WJDWEeGqY2x3PJZ78qgdq2xncM4vM/A8ca6eKvfxtzM9S56p4sNMpH3VAbOwEO76RFjX9J2AfgVszgTDVoonSUF9Tfd8Lst6utok1v9L/7SuyQKV3zowgAEvTtkM+d3CtbRJeVhKhIH/J+tfzlpC7X2v3ObPXAyccek+6MS7BKXY0gSxkPxINQ0De2pputmSVLWV6n0uSqFo3K4qu414+z2j8F451ZwfXpDjeIrK9GhVmUgTGaM42m831qG+37rfOnHnxw34eAlyXaD49ntSyVCGoXtT8rvu/YbkumAS2l6qvBAn/gKoRc+GjaA/IUB59OVAidJ+N43A7poIPK/w3SXgqpoA/HoReQTgPwXwz7za5QFV/VEAPwoAX/nKVxZr7HXthiAMyU5qXhAVPYgbNG3DBRO3L3eKT54Cc85YbQAIsw9OM71SDlaswN29FoV5perHAP5WTPEdIrMAWt+BVhQPoLdJs5nM6WCRyKol4As9ph263oxwvSXx94K0avpBKCUDJ05ejizlu0suEUGofYfVPUKkbjwnQbLeVaniCHEDKKF2tmlfch6KSAmxorlQTAJcKFH2lBVd4qH9VEiwr4LgEBU5CvoTiv7bM8HJGXOTnG6IJoeByNIrHJV5bjrUenOEKCVvtCcJcsIWb4nAvA1535jMZg3L282caPMdTp0WxlAQH6QW6W3u2e493xdFivD+tQAh23Wz5aQXMKlUoO7Xo2c9gRLdC7VygGYvLqJrj4bs8+r/t8TbCTHAlLLjaMxiqtVyjil4ATeKGvkoXiOzdqAFSa23SNuHFkwt9eNL5nisJllOdH1eODD4eh7pEFuC3KpnjvdPWa7me9mstBJpfJcOWD8UrE+JuJGAfKgJxdK1Ytqxhi7VZJ+1SdleywtFVZ+LyF8B8M8BeCQinaHwrwH4pde5FsBDtVkLkJhcP8803M02kdF8qaHVup8DOffHzxWfPs3Yfiz48NMMCcCzF8wfcZgsj4Si5PrtOy2EU9UPOO/hB4jsT4tfdHXxqilgC7f2/dduILSbtfrVCsxwKpbPemMugydEmRkooc1DRzTbW56SYDk4spAxjbPeMBjB7utuiqOFUscOWG0MkY1eWNZ9cKVE1ZWx2Ly3Zezu3gs2XzYvk7KPHykjJrcAPhZ+5TIAYwfoGsAJjZKnjwL6AXh4Ini4ZVqE060lflpTD56EXisKvs5aXU7d17t47nRe6cQLQNQcKKWup+UgCZ9xNrQhcj7WEjTi7+XmIs08+UHmIafxLRtAKJJfdALSRPAFLIhneWH7KRvxzhPtNimSEM69YproCjg6QTD3QvGqELD8JnMzHu+svHSZ4doAAcr+SKNid1WDrjTfrkLxIsolt4ozH6nJnsqcKhaSE/uoZY7KvkSzh21uypqB+6NIHe18qkKlgrGqUyIzdEbZcjauiywkgSUB5xctfAWAMxcFoiD3tOOs3guIa+DsUWC1oBlI18r4iU8yri8YRh8nACtFfqcJWPmM9ipeKO8BmIx4bwD8CwD+XQB/BcC/DHqi/D4Af+GV7+rX9oc0D9SFqnNcWZtvk5TNPzwqLq9JgD2JfTZiprk+h0AvDghRbJ0icrviI950zvvkYfPOpBeM3e6X7XXppdax+DWikGF0HWtNDj0DTDybnqeB9cPviM8Pt2ZWtr7VQKJ10zt08QouUECNcWVtEmY1U1CJhthhOYJoi69LQeDqfTbReI7AaJLE3g7k1DHaLAyCuCLK7q1qyWB+2n2kdOLEDVKrgRd+Imjyqi/XqKyT1Pm66/FSUdsfDYNuP3sZej8Gfi2dAJboUESW+7+5R7lfQQt8tCg2W0EO7vc2/erNlBBOnMq+sf1RkOfR2IoQ5v1yQuf380daRqa+TitredteLm9/NgotapEACKTUQm0nXhUk3vDvoaSpzs2cUO3BiVmgeR+/33PRAZT5LB0CIB1tOGEADfBrKdWU1JhgGoH5ACSzV8SktKu8xly+CgL/AMCPmR48APizqvoXReTvA/hxEfnfA/jbAP7Yq9+WTc19b841RF5h3EwZXhpESvrXygnFxGtGW7644I7znBou0nmosRsaRWpFnqI/FVp/nXBwkCZ6dygboVj8Ta/sWeEcZc8zsDtoQ8TtNya6bwYaF082wOmWROvsAa8zzixuEaHY2zXnkb7IqizCDKXufzooxodlIupcqpZAHImK0Hl+EFiKWc5pavWftimTIz0jFgJA8vL6LQop+ytIWZdSs7IDUidIHTBb6P+pZX7r1+xT7ICN1RrcDMDGvkf3TgYA5axIaPzS7VBSjXULIbb/SwBVNLVZoyduVWh37kklg5+SewTwhFbE2pSLc+8T7xsawmyf1YAdwH3QPQS7qBAaZuGGSgWqv/cMSAI0KdKBRr4Mi9rtmcqgJDFzKcquN1uGuxvpYJ2wN3OxUPmFAt5Boy9Ltc0HSwUw1VJq0l50oeeyMbuqS1FSHHiys/Izv68RuFbVsbhe6VGd1yBWas/tCH7JZn5do1RqrJrKzWlD6cfRPPiUeY6Yxfdsz7mkiADIGpAoWD8QbB/S6WD7kJL2tFdcfpgx74Dd04w0Kua9QiazaQjdoj+bbdX2Kl4ofwfAb7jl/Z8B8Jtf4143rw3TyVnukyjUpR6XxApAEffLwgYAQv3fdODVPLCjRFI5KtXq4gagRHb55mCGPx6qFbPtlAhK/16xhJuXw2BV3t1vV0ACq6jWcm9eEqzvBatBsBr4+40lp5IREMsDHTi5Vk+Roe7dIBZgkk3PucxA53NZ0Fdw9zL2O0cSWBqwZJG+M2tlmEUv2cx/OSw2D+W+vjA+L84MexbCyJ0gbwQazUjbi5W74iFeGyIZrB6ke8yI2Ti8K25oqwS0WY8WwgILwioN4byNcN8JGrUiTITGttEQlPbaLUGW5jvhtj6G5n+/phMLXa6hS5FQQJJCZlD0NiIiPftXwIr7e+fah1KYwgfbDtiZ8R1or7jF+hiU9x4PVdJtjba3X6Q+SvGCdg3t/nLUt1sv14CHolJx0BBY+KPrqlQDuL5/GVHs+6Cz3OyqPB/Hm6HQDachSUx/X/sBmLsoWCAdgV5UYQDiqWDziFWqttuAEID5SjFeKsYrxfWzjDwZUwNK/dOY8Zl5+Nv2Zgs6GCJMiRMfhMa8zg+Cs9Fmw/vY1D6vay/lILS+r61Pp9+z3bQi/P448k0P/+46xWCVqYfBiwSwoC7TWdozmpSfAEIjLRTOj6pC8VzMAjKuDJapyvaYR8U88jWciAAWrSkoBSOPmxGSWPpN6SUl3icVBF77VSII4XMiht5kocaq6NLu1SBPR8OdHYz1QPXQuhc8XLPI9MmKpcE8rwsZIIpeGh2gQTCbgdgTXCWglNtyNOzz2AbnHBPVdn7ccAdUn/xCzG87KC1Dcr21SlnfNve258M5BontAaxgkhO9AMFljzYMy/TdWVGyAsrEycgzkEeufw5ATETgkxkUs+mjtdF3382pFp27QcgFptLTGgk8jcqQ+Cafz4KZHm9MBUvFORfOWpCYNn/biuDHc6lH/VzcwfSZFUU3tUGxnFcfn9sHM7S6K7srTvlhQ2fKNfSGx44jcKa2JlNdPRTm69ly76QEXD5jIN7184z9C0UauZ4tSAX0pUt1V3uzBNxUKClVFQpEMAS6RGmuvp0F/YkPte5NXyBfCBclj3VzLREHKkHyXA252Wh9L1gPdMnbbIlmB8CCbZgR2HMyq3XAJUJXNiWT/wNI4GKsRFxUoROQsyBPijyB4umuiqg+nti5IQxmnLpJw52IdT2w3gApCXJPL4WaPL+qlDyKjfPYINijg1I2WDmoUv2vXV0hLG0WAzMmDj1L3j1YCWJg0eEusrr5qskXI4GEKEeObzS11KRWSUiq94l7EbmO09VYHhHpSBjO6Jo1T0dqtEJ0bzktIqiJrxri5Jf0qFwHE8cEp964eT4CIIVwm7HbmYzbbJKl8p3M11pnsGJLAvLEvsxqbqgdMGy0nCV3IyxI/Za9smgtkTuah87AjTOIw0Gx3y8g6EuvL0q7k2SQG892dsOxMMD/Qivt+Xxpg+E8HsP/GENU8TwmcoOBA6gGaJ97gHlJSt/r/m86tZifJlgcgJr9x/bbAKxOib5P36ULrBtf54Pi4qOMeQ9Ml/Q2QQbrn6L4ARTV0esS8TebD1yATWBODi9PlZTpQ50g+wKVMHZnWx5sk0E/y2Yjpk4wRT8YlofYRPQWoZeySgmYIlE10zxKqWAdA7AarJJO59VUgJUd6hTY5zkCsddFkvgcgdkIzrajoW4tigFAVEG0wgedHdg1FNtIjxkMzF0QTf2iGThkQBMJ4nHromDVKyQL0iaQCFjdRzU11TEBV5Ubhxaom7r9v1UHtPpd1p2sIexDT932KjInSwyWAjfUtAMiqC6CLo40Hi1tX9oE+eVsGmVXR9apwsgkTMUgKhWJZmN+qZYzCwHY76iqWuzJCJydLA+0M676+uYhW5z9hqjdRhQq069AowTXpJrUielhWVNUnRFbBfp+S/3q5pSFnjXTdTjPlLYKAT/uQ9NcNdU+vA2deUIlxbRiv/oQMK6NGBolDKY+OtnKjft4ZaQswHYC5iRc91BBWOmjEe/QvCntXKFhzP6ZASgRZuEMplat/XDCLQVVu1po4Vfe9qTlIA0Tjh3QzcvvaeBWlBUQVvSc2nSCPlRGFESwCozL6HograSqiJWfi5AWdqKlbsCrtjdKwM86xa/ZVr03AHhB00IFFcvAk4Yz8vmmV0YVm+v/9RAuM8UB0ujJpOzktphyjNVw5bXs3DjVWv7z9mY//N4xKjPbRUE0o200EbB4zgTgyw/NgPcQi9zEcGakwNlqeSiDAA83gu0QaVR91ISB+1y1r3HLa5/aWw673PKPf/eY0BZduBjBRqMP9jkXWDFoLA+MfcdTBhS8Jbbh3RvDvjsXlKmlL/WhC9uFAMs87SAi310vN8/Ds4Df+M92LP12PN5bJ+Qlc3XXh80ty95tCUzzWsuL+hmAEqRFN0rzLGqBT3OPG31yRnLch6a1aqaSI8Sr+xxfTliEoE3XLAKcPAxYbRko995ciefLpqb9/V19k7b/ds1WSrr1Hsd7/bP2+VFrUw4sLuuAJpoba98ADQDaAfOTWFPIpuW9Ftvf6Mp689JdtGhvlID3AjzqX0dgeJ121yTc9v6rT1htt/T7laKnGjnpWKkmoJ7mc7ShYxX3eqHv7fayVb9TLD8mdq995c/ea0MPvPfkNSDQ91T73lp3Lyj9T3QLsNJUwK/E+rytO/W+3bf7dt/+iW/3BPy+3bf7dt/e0nZPwO/bfbtv9+0tbXKjksav5M1EPgZwBeCTL+ymvzLtXbzdY3jb+w+8/WN42/sPvP1jeJv6//2q+t7xm18oAQcAEfmbqvqbvtCbfpfb2z6Gt73/wNs/hre9/8DbP4a3vf/AvQrlvt23+3bf3tp2T8Dv2327b/ftLW1vgoD/6Bu453e7ve1jeNv7D7z9Y3jb+w+8/WN42/v/xevA79t9u2/37b59d9q9CuW+3bf7dt/e0vaFEnAR+R0i8lMi8tMi8iNf5L0/TxORr4vIXxGRvy8if09E/g17/x0R+Usi8o/s+fGb7uvLmohEEfnbIvIX7f8fFJG/ZuvwZ0TkcwbwfzFNRB6JyJ8TkX8gIj8pIv/cW7gG/yvbQ39XRP60iKy/l9dBRP64iHwkIn+3ee/WORe2/7ON4++IyG98cz2v7Y4x/B9sH/0dEflPrc6vf/YHbAw/JSL/ozfS6ddsXxgBt4o+fwTA7wTwawH8XhH5tV/U/T9nmwH8W6r6awH8FgD/c+vzjwD4y6r6QwD+sv3/vdz+DQA/2fz/7wL491X1VwN4BuD3v5FevXr7DwD8P1X1nwHw3wXH8tasgYh8FcD/EsBvUtVfBxZ++j343l6HPwHgdxy9d9ec/04AP2SPbwD4o19QHz+r/QncHMNfAvDrVPW/A+AfAvgDAGDn+vcA+GftN/9Xo1nf0+2LROC/GcBPq+rPqOoI1tL84S/w/q/dVPVbqvpf2+sLkHB8Fez3j9nXfgzA/+SNdPAVmoh8DcD/GMB/ZP8LgN8G4M/ZV77X+/8QwP8AVrJPVUdVfY63aA2sdQA2ItKBNZ+/he/hdVDVvwrg6dHbd835DwP4k8r2X4EFzz/4Qjr6knbbGFT1/22F2AHgvwILsgMcw4+r6kFVfxbAT+M7rDj2RbQvkoB/FcAvNv9/0957K5qI/ABYWu6vAfiSqn7LPvo2gC+9qX69Qvs/AfjfoOY+fALgebOJv9fX4QcBfAzg/2ZqoP9IRE7wFq2Bqv4SgP8jgF8ACfcLAH8Lb9c6AHfP+dt6tv9nAP5ze/1WjuHeiPkKTUROAfwnAP5NVT1vP1M9rvvzvdNE5HcB+EhV/9ab7st30DoAvxHAH1XV3wCmYlioS76X1wAATFf8wyAz+gqAE9wU7d+q9r0+55/VROQPgirSP/Wm+/KdtC+SgP8SgK83/3/N3vuebiLSg8T7T6nqn7e3P3QR0Z4/elP9+4z2zwP4l0Tk50CV1W8D9cmPTJQHvvfX4ZsAvqmqf83+/3MgQX9b1gAA/ocAflZVP1bVCcCfB9fmbVoH4O45f6vOtoj86wB+F4B/Vasf9Vs1Bm9fJAH/GwB+yCzvA2gw+Ikv8P6v3Uxf/McA/KSq/nvNRz8B4PfZ698H4C980X17laaqf0BVv6aqPwDO9/9XVf9VAH8FwL9sX/ue7T8AqOq3AfyiiPwae+u3A/j7eEvWwNovAPgtIrK1PeVjeGvWwdpdc/4TAP6n5o3yWwC8aFQt31NNRH4HqFL8l1T1uvnoJwD8HhFZicgPggbZv/4m+vhajeXAvpgHgH8RtPz+YwB/8Iu89+fs738fFBP/DoD/xh7/IqhH/ssA/hGA/w+Ad950X19hLL8VwF+0178K3Jw/DeA/BrB60/37jL7/egB/09bh/wHg8du2BgD+EIB/AODvAvi/gyVPv2fXAcCfBvX1EygF/f675hwsNfNH7Fz/t6C3zffqGH4a1HX7ef4Pm+//QRvDTwH4nW+6/6/yuI/EvG/37b7dt7e03Rsx79t9u2/37S1t9wT8vt23+3bf3tJ2T8Dv2327b/ftLW33BPy+3bf7dt/e0nZPwO/bfbtv9+0tbfcE/L7dt/t2397Sdk/A79t9u2/37S1t9wT8vt23+3bf3tL2/wemK/UYXXOEQAAAAABJRU5ErkJggg==\n" + }, + "metadata": { + "needs_background": "light" + } + }, + { + "output_type": "stream", + "name": "stdout", + "text": [ + "GroundTruth: cat ship ship plane\n" + ] + } + ], + "source": [ + "dataiter = iter(testloader)\n", + "images,labels = dataiter.next()\n", + "\n", + "imshow(torchvision.utils.make_grid(images))\n", + "print('GroundTruth: ', ' '.join('%5s' % classes[labels[j]] for j in range(4)))" + ] + }, + { + "cell_type": "code", + "execution_count": 44, + "metadata": {}, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "" + ] + }, + "metadata": {}, + "execution_count": 44 + } + ], + "source": [ + "# 加载模型\n", + "\n", + "net = Net()\n", + "net.load_state_dict(torch.load(path))" + ] + }, + { + "cell_type": "code", + "execution_count": 45, + "metadata": {}, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "Predicted: cat ship ship ship\n" + ] + } + ], + "source": [ + "# 使用模型进行简单的预测\n", + "outputs = net(images)\n", + "_,predicted = torch.max(outputs,1)\n", + "print('Predicted: ', ' '.join('%5s' % classes[predicted[j]]\n", + " for j in range(4)))" + ] + }, + { + "cell_type": "code", + "execution_count": 46, + "metadata": {}, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "('plane', 'car', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck')\n" + ] + } + ], + "source": [ + "\n", + "print(classes)" + ] + }, + { + "cell_type": "code", + "execution_count": 47, + "metadata": {}, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "53.9\n" + ] + } + ], + "source": [ + "# 测试总体的表现。\n", + "correct = 0\n", + "total =0\n", + "\n", + "with torch.no_grad():\n", + " for data in testloader:\n", + " images,labels = data\n", + " outputs = net(images)\n", + " _,predicted = torch.max(outputs.data,1)\n", + " total += labels.size(0)\n", + " correct += (predicted == labels).sum().item()\n", + "print(100*correct/total)" + ] + }, + { + "cell_type": "code", + "execution_count": 48, + "metadata": {}, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "Accuracy of plane : 52 %\nAccuracy of car : 47 %\nAccuracy of bird : 46 %\nAccuracy of cat : 36 %\nAccuracy of deer : 45 %\nAccuracy of dog : 38 %\nAccuracy of frog : 70 %\nAccuracy of horse : 56 %\nAccuracy of ship : 83 %\nAccuracy of truck : 62 %\n" + ] + } + ], + "source": [ + "class_correct = list(0. for i in range(10))\n", + "class_total = list(0. for i in range(10))\n", + "with torch.no_grad():\n", + " for data in testloader:\n", + " images, labels = data\n", + " outputs = net(images)\n", + " _, predicted = torch.max(outputs, 1)\n", + " c = (predicted == labels).squeeze()\n", + " for i in range(4):\n", + " label = labels[i]\n", + " class_correct[label] += c[i].item()\n", + " class_total[label] += 1\n", + "\n", + "for i in range(10):\n", + " print('Accuracy of %5s : %2d %%' % (\n", + " classes[i], 100 * class_correct[i] / class_total[i]))" + ] + }, + { + "cell_type": "code", + "execution_count": 49, + "metadata": {}, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "cpu\n" + ] + } + ], + "source": [ + "# 在GPU上进行训练\n", + "device = torch.device(\"cuda:0\" if torch.cuda.is_available() else 'cpu')\n", + "\n", + "print(device)\n", + "\n", + "net.to(device)\n", + "\n", + "inputs,labels = data[0].to(device),data[1].to(device)" + ] + } + ] +} \ No newline at end of file diff --git a/pytorch/实战/7.ipynb b/pytorch/实战/7.ipynb new file mode 100644 index 00000000..667a964a --- /dev/null +++ b/pytorch/实战/7.ipynb @@ -0,0 +1,897 @@ +{ + "metadata": { + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.8.0-final" + }, + "orig_nbformat": 2, + "kernelspec": { + "name": "python3", + "display_name": "Python 3.8.0 64-bit", + "metadata": { + "interpreter": { + "hash": "38740d3277777e2cd7c6c2cc9d8addf5118fdf3f82b1b39231fd12aeac8aee8b" + } + } + } + }, + "nbformat": 4, + "nbformat_minor": 2, + "cells": [ + { + "cell_type": "code", + "execution_count": 21, + "metadata": {}, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "99 2853.9273235396186\n", + "199 1920.1599361735143\n", + "299 1293.9318621259858\n", + "399 873.6293058201031\n", + "499 591.3102531671352\n", + "599 401.51736190288585\n", + "699 273.8164238221197\n", + "799 187.8170665910619\n", + "899 129.8480963577774\n", + "999 90.73635494859597\n", + "1099 64.32191622546065\n", + "1199 46.464885616046026\n", + "1299 34.3805974263392\n", + "1399 26.19431825870059\n", + "1499 20.642763997519296\n", + "1599 16.873875616831352\n", + "1699 14.312402496108426\n", + "1799 12.569591035284114\n", + "1899 11.382455473616632\n", + "1999 10.572905438674582\n", + "-0.03186837476257857 0.8284118475316333 0.005497824938589816 -0.08930083611579144\n" + ] + } + ], + "source": [ + "# 1 使用numpy实现梯度下降神经网络\n", + "\n", + "import numpy as np \n", + "import math \n", + "x = np.linspace(-math.pi,math.pi,2000)\n", + "y = np.sin(x)\n", + "\n", + "a = np.random.randn()\n", + "b = np.random.randn()\n", + "c = np.random.randn()\n", + "d = np.random.randn()\n", + "\n", + "learning_rate = 1e-6\n", + "for t in range(2000):\n", + " # forward\n", + " y_pred = a+b*x+c*x**2+d*x**3\n", + " loss = np.square(y_pred - y).sum()\n", + " if t % 100 ==99:\n", + " print(t,loss)\n", + " \n", + " # backward\n", + " grad_y_pred = 2.0 *(y_pred-y)\n", + " grad_a = grad_y_pred.sum()\n", + " grad_b = (grad_y_pred *x).sum()\n", + " grad_c = (grad_y_pred * x**2).sum()\n", + " grad_d = (grad_y_pred * x ** 3).sum()\n", + "\n", + " # optimizer\n", + " a -= learning_rate * grad_a\n", + " b -= learning_rate * grad_b\n", + " c -= learning_rate * grad_c\n", + " d -= learning_rate * grad_d\n", + "\n", + "print(a,b,c,d)\n" + ] + }, + { + "cell_type": "code", + "execution_count": 22, + "metadata": {}, + "outputs": [ + { + "output_type": "display_data", + "data": { + "text/plain": "
", + "image/svg+xml": "\r\n\r\n\r\n\r\n \r\n \r\n \r\n \r\n 2021-03-23T15:21:39.881860\r\n image/svg+xml\r\n \r\n \r\n Matplotlib v3.3.2, https://matplotlib.org/\r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n\r\n", + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAD4CAYAAADhNOGaAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAA/nElEQVR4nO3ddZhUZRvH8e+9uyzdi4B0SUhKCyoqjdIGGGAA0ohIS6qAgIGApIiEpEqngOALKEuz0iXdtSwsG8/7xzPggktunJmd+3Ndc+3MOWdmfqPs3HvOU2KMQSmllPfycTqAUkopZ2khUEopL6eFQCmlvJwWAqWU8nJaCJRSysv5OR3gUQQEBJicOXM6HUMppTzKxo0bzxpjMty53SMLQc6cOQkMDHQ6hlJKeRQRORzddr00pJRSXk4LgVJKeTktBEop5eW0ECillJfTQqCUUl4uVgqBiHwvIqdFZMdd9ouIDBORfSKyTUSeirKviYjsdd2axEYepZRSDy62zgh+AKrfY38NIJ/r1hz4DkBE0gG9gbJAGaC3iKSNpUxKKaUeQKyMIzDGrBaRnPc4pA7wo7FzXq8XkTQikhmoBCwzxpwHEJFl2ILyU2zkUkq5kZAQ2LsX9uyB06fh8mUIC4NkySBlSsiZE/LmhVy5wEevWsen+BpQlgU4EuXxUde2u23/DxFpjj2bIHv27HGTUikVeyIj4Y8/YN48WLUKNm2y2+4nbVooXx4qV4YGDUB/3+Ocx5RdY8wYY0wpY0ypDBn+M0JaKeUu/vkHunSxX+DPPQfDhtm/+rt3h+nTYcsWOHUKrl2D8HC4cgWOHIHff4exY+2X/4ED0LEj5Mhhi8KPP0JoqNOfLMGKrzOCY0C2KI+zurYdw14eirp9VTxlUkrFpp07oV8/mDnTPq5ZEwYPhpdfhhQp7v68FCnsLWtWePZZeP99u33/fpg1CyZOhCZNoFMnWxzatoXkyeP+83iR+DojmAu87eo9VA64ZIw5ASwBqopIWlcjcVXXNqWUpzh5Elq0gMKFYcEC6NDBfonPnQuNGt27CNxDaI6sHGz+KkErZ7D111FsfyY/Jz7rRtgTeWDUKIiIiN3P4cVi5YxARH7C/mUfICJHsT2BEgEYY0YBC4GawD4gBHjHte+8iPQHNrheqt/NhmOllJuLjLSXcj7+2F7madMGevaER7h0e/XGVVYcXMHaI2sJPBHI9lPbOXX11O0HFbU3MafI83dLCrfpTvkX36ZqxaYUzVgUH/GYK91uRzxx8fpSpUoZnX1UKQcdOgTvvgsrV8ILL9i/0PPle6iXCAkLYfbfs5m6YyorD64kNCKURD6JKJKxCMUzFidHmhxkSZmFVIlT4efjR3hkOGdDznLiynF2bl7KtiMb2ZPWnhVkSZmFN4u+SZNiTSiYoWAcfOCEQUQ2GmNK/We7FgKl1EOZMweaNrVnBEOG2Gv6Ig/89P3n9zN03VCmbJ/C5dDL5EqTizr56/DSEy9RIXsFkvglebAXOn+e4x81Z/nm2cysmI5Fj10iwkRQM19NulboSsXsFZGHyOUNtBAopWImPBy6doWhQ6FkSdsonCvXAz9955mdfLbmM37a8RN+Pn689uRrvFfiPZ7N8WzMvrCnTIEPPuB0Kl/GftaAb87M40zIGarmqcpX1b6iUIZCj/7aCYwWAqXUo7t8GV59FZYsgVat4MsvIXHiB3rq2ZCz9F7Zm9EbR5PELwktS7WkY/mOZE6ZOfby7dsHdevCrl2EDBvKqBIR9Pu9H8E3gmlftj2fvvApSRMljb3381B3KwQYYzzuVrJkSaOUiidHjhhTtKgxvr7GjB37wE+LiIwwI/4aYVIPSG18+/qa1gtamzNXz8RdzkuXjKlVyxgwpl07c/rySdNsbjNDH0yB4QXMhmMb4u69PQQQaKL5TtVmdqXU3e3eDeXKwcGDsHDhv33872P/+f28MPEFWi9sTekspdn6wVaG1xxOQLKAuMuaKpVtv+jQAYYNI0OrToypPoIlby4h+EYw5caVY9ifwzAeeBUkrnnkmsVKqXiwY4ed5sEYO1VE0aL3fYoxhlGBo+i0rBN+Pn6Mrz2ed4q/E3+Ntr6+9rLVY4/ZkcyXL1N1+nS2t9xOk1+b0H5xezae2MioWqP0UlEUekaglPqvzZuhUiX7xfr77w9UBK6EXqHR7Ea0WtiKitkrEtQqiHdLvBv/PXdEoFs3GDHCDmqrW5c0kpRfXvuFPs/14cetP1J5UmUuXLsQv7ncmBYCpdTtgoLsmUDy5LB6NRQocN+nbD+1nVJjSzHz75kMeHEAi95YRNZUWeMh7D20agXjxtkG7tdewyc8gt6VejPzlZkEHg/k2R+e5fiV485mdBNaCJRS/zp4EKpWtT2CVq6EPHnu+5S5u+dSbnw5roReYcXbK+hasav7jPJ97z349lvbdvDWWxARQcNCDVnYeCGHLh6i4vcVOXLpyP1fJ4Fzk/9bSinHnTwJVarY6SKWLoXcue95uDGGL9d9Sd1pdXkyw5NsarGJ53I+F09hH0KbNvDFF3bm0zZtwBhezP0iv739G+eunePFH1/kZPBJp1M6SguBUspOBV29Opw4YSeOK1z4noeHR4bTckFLPlr6EfUL1mdV01VkSpEpnsI+go8/tlNjjxplZ0QFymQpw6I3FnH8ynEq/1iZsyFnHQ7pHC0ESnm7iAho3Nj2Epo9287/fw+h4aG8OvNVRm8cTZcKXZjxygySJUoWT2Fj4PPP4fXXbUGYPh2Ap7M9zfzG89l/YT91ptXhevh1h0M6QwuBUt6uc2eYP98uIFP9XkuP21lCa0+rzS+7fuHral8zsPJA92kPuB8fH5gwASpWhLfftl1igUo5KzGp3iTWHlnLO3PeIdI8wCpqCYyH/B9USsWJsWNtv/s2bWwvm3u4dP0S1adUZ/mB5YyvPZ725drHU8hYlCSJbTjOkcOuhHb0KAANCzVk4IsDmbZjGr1X9nY4ZPzTQqCUt/rf/+yXf7Vq8NVX9zz0cuhlqkyqwvqj6/mpwU+8W+LdeAoZB9Klg19/hZAQWwyu28tBnSt05r0S7/Hpmk/5eefPzmaMZ1oIlPJGp07ZSeRy5oRp08Dv7pMMBN8IpuaUmmw+uZnZr87m1Sdfjb+ccaVQIbsO8l9/QevWYAwiwoiaIyiTpQzvzHmHvef2Op0y3sRKIRCR6iKyW0T2iUjXaPZ/JSJbXLc9InIxyr6IKPvmxkYepdQ9hIfbJSTPn7drAqdJc9dDr4Vdo/ZPtVl3dB1T60+ldv7a8ZczrtWrZ1dU+/57GD0agMR+iZnRcAZ+Pn40nNmQkLAQh0PGk+hmonuYG+AL7AdyA/7AVqDQPY5vC3wf5XHww76nzj6qVAx062Zn6Jww4Z6HXQ+7bqpPrm6kj5hJWyfFT7b4FhFhTI0axvj7G7N5863NC/csNNJHTLO5zZzLFgeIw9lHywD7jDEHjDE3gGlAnXsc3wj4KRbeVyn1sBYsgAED7CyiTZve9bBIE0mTX5uweN9ixrw8hjeLvhl/GeOTj4+9RBQQAK+9BsHBANTIV4POFTozdtNY5u2e53DIuBcbhSALEHWM9lHXtv8QkRxALmBFlM1JRCRQRNaLSN27vYmINHcdF3jmzJlYiK2Ulzl50n75Fytmp124h87LOjM9aDqDKg/i/acebOppjxUQAFOn2sVt2rS5tblvpb4Uy1iM9+e9z5mrCfs7J74bi18HZhljIqJsy2HsijmNga9FJNrJTYwxY4wxpYwxpTJkyBAfWZVKOCIjbREIDrZfeknuvi7w1+u/Zui6obQp3YaPn/44/jI66bnn4JNPYOJEmDQJsO0Fk+tP5uL1izSf3zxBr2MQG4XgGJAtyuOsrm3ReZ07LgsZY465fh4AVgElYiGTUiqq4cPtLJxDh9oeM3cxM2gmHZd0pF6Benxd/WvvWvy9Z0949llo2RL27weg8GOF+fyFz/l116/8tCPhXtGOjUKwAcgnIrlExB/7Zf+f3j8iUgBIC6yLsi2tiCR23Q8AKgB/x0ImpdRN27fb0cO1atkvubv43z//481f3uTpbE8zpf4UfH184zGkG/Dzg8mT7c+mTe3UG0CHch0om6UsHRZ34Py1885mjCMxLgTGmHCgDbAE2AnMMMYEiUg/EYna1+x1YJq5/fyqIBAoIluBlcBAY4wWAqViy/Xrdh6h1KltN8m7/IV/+OJh6k2vR47UOZjbaK73rt6VLZudauOPP24NsvP18WXMy2M4f+08Hy9NmJfKxBOve5UqVcoEBgY6HUMp99e9u+0ltGAB1KwZ7SFXb1ylwvcVOHTxEOvfX0+BgPsvRJOgGWPHGCxaBJs2wZNPAtB1eVcG/W8Qq5qscs/pth+AiGx0tcneRkcWK5VQbdxo5+F/5527FoGb3US3n97OTw1+0iIA9qxpzBhIlcpOThcWBkCv53qRK00uWsxvwY2IGw6HjF1aCJRKiG7csAUgY0Y7qdxd9P+9P7N3zuaLyl9QI1+NeAzo5h57zI423rQJPvsMgGSJkjG85nB2n9vN8L+GOxwwdmkhUCoh+vxz20g8evRdp5D4Zecv9Pm9D02KNaFj+Y7xm88T1K9v21c+/9yu4wzUzFeT6nmr0+/3fglqbIEWAqUSmq1b7V+xb7wBL70U7SF7z+2lya9NKJOlDKNeGuVd3UQfxtdf20tEzZrZsRjAl1W/JPhGMJ+s/MTZbLFIC4FSCUl4OLz7LqRPD998E+0hIWEhNJjRAH9ff2a+MpMkfncfXOb1MmSwl9bWrYPvvgOgYIaCtCnThrGbxrL15FaHA8YOLQRKJSQjRtjr2t9+a4vBHYwxtFrQih2ndzCl/hSyp87uQEgP89ZbUKUKdOt2ayGb3s/1Jm2StHRa1snhcLFDC4FSCcWxY3Z0bI0a0LBhtIeM2zSOiVsn0uu5XlTLWy2eA3ooEbvofXj4rbUL0iZNS49nerD8wHJWHFxx/9dwc1oIlEooOnSwX1bDh0c7cGzTiU20XdSWqnmq8smzCef6drzInRv69oW5c+Fnu3pZy9ItyZYqG91+6+bx8xBpIVAqIVi0yC4y07On/dK6w+XQy7wy8xUyJM/gndNHxIYPP7Qzt3boAMHBJPFLQu/nevPXsb+Yu9uz19TSQqCUp7t2zV6yKFAAOkV/zbrVglYcvniY6Q2nE5AsIJ4DJhB+frYN5ujRW2MLmhRvwhPpn6DHih5EREbc5wXclxYCpTzdZ5/BwYMwciQkTvyf3ZO2TmLK9in0fq43T2d72oGACUiFCtCkiZ3Fdfdu/Hz86P98f4LOBHn07KQ615BSnmzPHihc2K6u5ZpHP6p95/dRYnQJnsr8FCveXqGXhGLDqVOQPz+UKQNLlhCJ4anRT3E9/DpBrYLc+r+xzjWkVEL00Ud2kZnBg/+z60bEDRrPbkwin0RMrjfZrb+gPErGjNCvHyxbBj//jI/40PPZnuw+t5vZO2c7ne6RaCFQylMtXgzz59uVtTJl+s/uXit7seH4BsbVHke21NmieQH1yFq1gqJFbQPy1avUL1ifAgEF+HT1p0SaSKfTPTQtBEp5orAw+yWUNy+0a/ef3b8d+I0v/vcFzZ9qTv2C9R0ImMDdbDg+cgQGDMBHfOhesTvbT29n/p75Tqd7aFoIlPJEI0bArl12+oM7GogvXLtAk1+bkD8gP19Wu/vMoyqGKla08zkNGQKHD9OoSCNypcnFp6s/9bhxBbFSCESkuojsFpF9ItI1mv1NReSMiGxx3d6Psq+JiOx13ZrERh6lErQzZ6BPH6hWLdpJ5dotbsepq6eYXG8yyf2Tx38+bzJggB2817Urfj5+dK3YlQ3HN7DswDKnkz2UGBcCEfEFRgA1gEJAIxGJbnXs6caY4q7bONdz0wG9gbJAGaC3iKSNaSalErSePSE42C6leMcI4p93/szkbZPp+UxPSj5e0qGAXiRbNjt2Y9o0WLeOJsWakDlFZoasHeJ0socSG2cEZYB9xpgDxpgbwDSgzgM+txqwzBhz3hhzAVgGVI+FTEolTFu2wNix0KYNFCx4265TwadoMb8FJTOXpPsz3Z3J5426dIHMmeHDD0ns60/bMm1ZdmAZ209tdzrZA4uNQpAFOBLl8VHXtjs1EJFtIjJLRG52YXjQ5yIizUUkUEQCz5xJOAtCKPXAjIGOHSFdOujd+45dhhbzW3Al9Ao/1vuRRL6JHArphVKksIP6/vwTpk2jRakWJEuUjK/Wf+V0sgcWX43F84Ccxpii2L/6Jz7sCxhjxhhjShljSmXIkCHWAyrl9hYtgpUrbftA2tuvoP649Ufm7J7DZy98RqEM0V2ZVXGqSRMoUQK6dCEdSXmn+DtM2T6Fk8EnnU72QGKjEBwDonZSzuradosx5pwxJtT1cBxQ8kGfq5TCziraubPtLtq8+W27jlw6QrvF7Xgm+zN0KNfBmXzezsfH9uA6cgS+/JL2ZdsTFhHGyA0jnU72QGKjEGwA8olILhHxB14HbpuKT0QyR3lYG9jpur8EqCoiaV2NxFVd25RSUU2caNfNHTAA/P1vbTbG8O7cd4mIjOCHuj/o6GEnVaoEdevCgAHkC0tJ7fy1GblhJNfCrjmd7L5iXAiMMeFAG+wX+E5ghjEmSET6iUht12HtRCRIRLYC7YCmrueeB/pji8kGoJ9rm1LqpqtXoVcvKFcOGjS4bde4TeNYfmA5Q6oOIXfa/04/reLZF19AaCj070/H8h05d+0ck7dNdjrVfemkc0q5u88+s11G16yxg5hcjl0+RqGRhSiZuSS/vf2bLkDvLlq1grFjMUFBFF/xCr7iy8bmG93i/49OOqeUJzp9GgYNspccohQBYwwtF7QkLCKMMS+PcYsvGeXSqxf4+yOffELLUi3ZfHIzfx37y+lU96SFQCl31r8/hITYtoEopgdNZ96eefR/vj950+V1KJyKVqZMdlbYGTN4I7wgKf1TMjLQvRuNtRAo5a727LGLpjdrZlcfczkbcpa2i9pSJksZ7SXkrjp1goAAUvbox1tF32T6jumcCznndKq70kKglLvq2dNOKNenz22bOyzuwKXrlxhfe7z2EnJXqVLZ/38rVtAyrDihEaFM2DLB6VR3pYVAKXe0aRPMnGlHEmfMeGvzgj0LmLJ9Ct2f6U7hxwo7GFDd1wcfQM6cFO43imeyP8N3gd+57VoFWgiUckc9e9rRwx99dGvT5dDLtJjfgsKPFda5hDxB4sS2jWfzZlrdKMaBCwdYun+p06mipYVAKXfzxx92OomuXSF16lubuyzrwongE4yvPR5/X/97vIByG40bQ7Fi1B88n4CkAYzfPN7pRNHSQqCUOzEGune3PU/atLm1ec3hNYzaOIoOZTtQJksZBwOqh+LjAwMG4L//EG9SlDm75nA25KzTqf5DC4FS7mTpUjtwrGdPSJYMsIvQt5jfghypc9Dv+X4OB1QPrXp1qFCBd3/cTlhkGFO3T3U60X9oIVDKXRgDPXpAzpy2y6jLkLVD2Hl2JyNqjtAVxzyRCHz6KUWCzlDKNxvjN493u6UstRAo5S5++QU2brRrDbgmltt/fj/9V/enYaGG1HqilsMB1SOrVAlefJF3V1xg26ltbD652elEt9FCoJQ7iIiwl4MKFIA33wTsNBKtFrYikU8ivqn+jcMBVYz170+jdcEkwY/vN3/vdJrbaCFQyh1MnQo7d9ruhn5+gJ1GYun+pXz2wmc8nvJxhwOqGCtfnjQv1qL+Lh+mbJvM9fDrTie6RQuBUk67ccNeDipRAurXB+Di9Yt0WNyBUo+XolXpVg4HVLGmXz/e/fMGF0Mv8euuX51Oc4sWAqWc9v33cPCgnW7ax/5Kdv+tO2dCzjD6pdE6jURC8tRTPF+iHtkuC5MD3efyUKwUAhGpLiK7RWSfiHSNZn9HEfnbtXj9byKSI8q+CBHZ4rrNvfO5SiVooaG2ADz9tO1mCKw/up5RgaNoV6YdT2V+yuGAKrb59O1Ho+2GJYd+c5sxBTEuBCLiC4wAagCFgEYicufq2ZuBUq7F62cBX0TZd80YU9x1q41S3mTCBDh61F4aEiEsIowW81uQJVUWHTOQUBUuzBtZahAukcxYN87pNEDsnBGUAfYZYw4YY24A04A6UQ8wxqw0xoS4Hq7HLlKvlHe7ccOuM1CuHFSpAsA3f37DtlPb+LbGt6RMnNLhgCquFO32NYVPw5RVw5yOAsROIcgCHIny+Khr2928ByyK8jiJiASKyHoRqXu3J4lIc9dxgWfOnIlRYKXcwsSJ8M8/t84GDl88TO9VvamdvzZ1C9R1Op2KS088wRv+JVnrd4KDu9Y5nSZ+G4tF5E2gFDA4yuYcrjU0GwNfi0ie6J5rjBljjClljCmVIUOGeEirVBwKC4PPP4cyZaBaNQA+XPIhAN/W+NbJZCqeNHr3awCmTujobBBipxAcA7JFeZzVte02IlIZ6AHUNsaE3txujDnm+nkAWAWUiIVMSrm3H3+EQ4dunQ0s2beEX3b9Qs9nepI9dXan06l4kKNIRSqGZmRKyJ+YY//5yoxXsVEINgD5RCSXiPgDrwO39f4RkRLAaGwROB1le1oRSey6HwBUAP6OhUxKua+wMNtTqFQpqFGD0PBQ2i1uR750+ehY3vm/DlX8eePZNuwMMGwd+rGjOWJcCIwx4UAbYAmwE5hhjAkSkX4icrMX0GAgBTDzjm6iBYFAEdkKrAQGGmO0EKiEbfJkO26gVy8Q4av1X7Hn3B6G1RhGYr/ETqdT8eiVZ1viZ4Qpu2bAiROO5RB3mwXvQZQqVcoEBgY+9PM2ndjE2ZCzVM1TNQ5SKfUAwsPtfEKpU0NgIEcuH6XAiAJUzVOVX177xel0ygEvjXuBHUErOUh75Kuv4/S9RGSjq032Nl41srjHih60X9ze6RjKm02dCvv33zob6LSsE5Emkq+qfeV0MuWQV0o14XAa2DDnO8fOCryqEFTNXZVdZ3fxz6V/nI6ivFF4OHz6KRQrBrVr89uB35gRNIPuFbuTM01Op9Mph9TOX5tEPomYlS8MBg++/xPigHcVAtcloWX7lzmcRHmladNg717o1YsbkWG0XdSW3Glz83EFZxsKlbPSJk1L5dyVmVk6Oea7kXDyZLxn8KpCUChDIR5P+ThLDyx1OoryNhER9mygSBGoW5dhfw5j59mdfFP9G5L4JXE6nXLYK4Ve4VCiYDYGOHNW4FWFQESomqcqy/YvIyIywuk4yptMnw67d0OvXhy/epK+v/flpSde4qUnXnI6mXIDdQrUwc/Hj1kNC8J338GpU/H6/l5VCMC2E1y4foGNJzY6HUV5i5tnA4ULQ/36fLzsY8IiwnTVMXVLuqTpeDHXi8zMehkTej3ezwq8rhBUzl0ZQVi6Xy8PqXgya5ZdfeyTT/j9nzVM3T6VLhW6kDttbqeTKTfSsFBDDgQfYcs7NWHkSDh9+v5PiiVeVwgyJM/AU5mf0kKg4kdkpF1+slAhwurVoc2iNuRInYMuFbs4nUy5mboF6uIrvsysltWuUxGPZwVeVwjA9h5ad3Qdl0MvOx1FJXSzZ0NQEPTsyYiN37Hj9A6+rv41yRIlczqZcjMByQJ4PtfzzDq5AtO4UbyeFXhtIQiPDGfVoVVOR1EJWWQk9OsHBQpwsuaz9F7Vm+p5q1Mnf537P1d5pXoF6rH3/F52tW0M16/DkCHx8r5eWQjKZy1P8kTJ9fKQilu//go7dkDPnnRZ2Z3r4dcZVn0YIuJ0MuWmaue307PNubENXn8dRoyAeFh/xSsLQWK/xFTKWUkLgYo7N88GnniC/1XIxo9bf+Sj8h+RL30+p5MpN5Y1VVZKZi7JnN1zoGdPuHYNvor76Ue8shCAvTy09/xeDl446HQUlRDNnQtbtxLeoxutF7cjW6ps9Himh9OplAeok78Ofx79k5PZ0sIrr8Dw4XD+fJy+p9cWgmp57KpQi/ctdjiJSnCMsWcDefMy+onLbD21lS+rfUly/+ROJ1MeoE6BOhgM83bPs2cFV67AN3E75sRrC8ET6Z8gT9o8LNi7wOkoKqGZNw82b+ZM17b0/L03lXNXpkHBBk6nUh6iyGNFyJkmp708VKQI1KtnC8GlS3H2nl5bCESEWvlq8dvB3wgJC3E6jkoojIG+fSF3brqm30zwjWC+rfGtNhCrByYi1Mlfh+UHlhN8Ixg++cQWgW/jbi3rWCkEIlJdRHaLyD4R6RrN/sQiMt21/08RyRllXzfX9t0iUi028jyoWk/U4nr4dVYeXBmfb6sSsoULYdMm1nd6ne+3/sCH5T6kQEABp1MpD1Mnfx1CI0Jth5YSJeCll2yj8ZUrcfJ+MS4EIuILjABqAIWARiJS6I7D3gMuGGPyAl8Bg1zPLYRd4/hJoDow0vV68eK5HM+RPFFyvTykYofrbCAid07a+Czm8ZSP88mznzidSnmgZ3I8Q9okae3lIbBnBefP20FmcSA2zgjKAPuMMQeMMTeAacCdI2bqABNd92cBL4o9V64DTDPGhBpjDgL7XK8XNwYNgq7/nrAk9ktM5dyVWbB3AZ64ZKdyM4sXw4YNjGtXkY0nNzG06lBSJk7pdCrlgfx8/Kj1RC0W7FlAeGQ4lCkD1avbAWZXr8b6+8VGIcgCHIny+KhrW7THuBa7vwSkf8DnAiAizUUkUEQCzzzqAItDh+DLL+Gff1coq5WvFv9c+oegM0GP9ppKwa2zgXP5stD9+kIq5azEa0++5nQq5cHq5K/DuWvnWHdknd3wySeQJg0cjP0u7x7TWGyMGWOMKWWMKZUhQ4ZHe5Fu3ezPQYNubaqZryYAC/bo5SEVA0uXwp9/0v393FwKvaQNxCrGquSugp+PH4v2LbIbnn4adu2y05nHstgoBMeAbFEeZ3Vti/YYEfEDUgPnHvC5sSd7dnjnHRg3Do4eBSBLqiwUz1Sc+Xvnx9nbqgTOdTYQWCIjY6/9Qbuy7Sj8WOz/sirvkjpJaipkq8DCvQv/3egbN02osVEINgD5RCSXiPhjG3/n3nHMXKCJ635DYIWxF+XnAq+7ehXlAvIBf8VCprvr1s0O/49yVlArXy3WHlnL+WtxO3pPJVC//Ubk+nW0bpCUjCky0qdSH6cTqQSiZr6abD21lWOX4+7vY4iFQuC65t8GWALsBGYYY4JEpJ+I1HYdNh5ILyL7gI5AV9dzg4AZwN/AYqC1MSZu15DMmROaNIGxY+H4ccAWgkgTyZJ9S+L0rVUC5DobmPB8Gv4KP8TgKoNJlTiV06lUAlEjbw0g7mdAEE/sLVOqVCkTGBj46C9w4AA88QS0bg3ffENEZASZhmaiWp5qTK4/OfaCqoRvxQrOv/Qi+bumoEC24qxuulrbBlSsMcaQ/evslMlShtmvzo7x64nIRmNMqTu3e0xjcazKnRvefhvGjIETJ/D18aVmvpos3LuQsIgwp9MpT9K3L5+8lIzzhDC8xnAtAipWiQg18tZg2f5l3Ii4EWfv452FAKBHDwgLu7UcXN38dblw/QKrD692OJjyGKtWsXnPakY9eY3WpVtTLFMxpxOpBKhmvppcuXGFtUfWxtl7eG8hyJMH3ngDRo2CU6eolrcaSf2S8suuX5xOpjxEZL++tK6biPTJAuj3fD+n46gE6sVcL5LIJ9HtvYdimfcWArBTvLoWiU6WKBnV8lbj112/6ihjdX+rV/PjhVWsyxTGF1W+IE2SNE4nUglUysQpeSbHM1oI4ky+fNC4MXz3HZw+Td38dTl25RiBx2PQEK28wsVPe9K5mlD+8bK8Xextp+OoBK5m3poEnQnin0v/3P/gR+DdhQDsWcH16zB0KC/nfxlf8dXLQ+re1qyhl98aziWFES99h4/or5GKWzXy2W6ki/YuipPX13/B+fPfWiQ63dVInsv5HL/u+tXpVMqNbR3SiRGl4YPizSiRuYTTcZQXKBhQkOyps7N4f9yMJ9BCAPasICQEhg6lbv667Dy7k91ndzudSrkhs2YNrQP+Ip1vcj6tOtDpOMpLiAjV8lTjtwO/xUkXdy0EAAULwquvwvDh1M34HICeFahoTRr1Af/LDoOqDSZt0rROx1FepFqeagTfCGb76e2x/treObI4OkFBdn3Q7t0pnWMJvuLL+vfXx+57KI926fcl5J9fnZypsrO250FtG1Dx6lrYNa6FXyNd0nSP/Bo6svh+nnwSGjaEYcOom70qfx77M84nelKepffU5pxODiMaT9EioOJd0kRJY1QE7kX/NUf1ySdw5QoN/3cBgFl/z3I4kHIX25dMYnimf2iRqBwl81R0Oo5SsUoLQVRFikD9+uT/ZgrFAgozPWi604mUGzDG0HpJO9KECp+1mOF0HKVinRaCO/XqBZcv89rZjKw7ui7OBnAozzH1l76sSX2RAanrky4g2/2foJSH0UJwp2LFoG5dXp1g18eZGTTT4UDKSZdDL9Np4wBKn/LjvbYTnI6jVJzQQhCdXr3Ic/gKJXlcLw95uT4/teBUohuMyP4BPilSOh1HqTgRo0IgIulEZJmI7HX9/E/HahEpLiLrRCRIRLaJyGtR9v0gIgdFZIvrVjwmeWJNiRLw8su8tuYCG45v4MCFA04nUg7YcXoHww5Np1lQEkq3GeB0HKXiTEzPCLoCvxlj8gG/uR7fKQR42xjzJFAd+FpE0kTZ/7ExprjrtiWGeWJP7968GngNgBlB2kDobYwxtJn2NqmvGT4v1RlSpHA6klJxJqaFoA4w0XV/IlD3zgOMMXuMMXtd948Dp4EMMXzfuFeyJDkq1qLsCV+mb5vqdBoVzyZvm8zvFzYz4M8UpG/9sdNxlIpTMS0EGY0xJ1z3TwIZ73WwiJQB/IH9UTZ/5rpk9JWIJL7Hc5uLSKCIBJ45cyaGsR9Q3768tjWCLWe2s+fcnvh5T+W489fO89GCdpQ7Au/X6KFnAyrBu28hEJHlIrIjmludqMcZO1fFXeerEJHMwCTgHWNMpGtzN6AAUBpIB3S52/ONMWOMMaWMMaUyZIinE4qSJXklR3UApv31ffy8p3Jct+XdOH/jEqPWpcOnTVun4ygV5+5bCIwxlY0xhaO5zQFOub7gb37Rn47uNUQkFbAA6GGMWR/ltU8YKxSYAJSJjQ8Vm7L2HEylgzB5/RhducwLrDuyjjGbxtB+naFYi96QPLnTkZSKczG9NDQXaOK63wSYc+cBIuIP/AL8aIyZdce+m0VEsO0LO2KYJ/YVLsxbScuy1+cCf26Pm0UhlHsIjwzngwUfkOW6P332Pg7NmzsdSal4EdNCMBCoIiJ7gcqux4hIKREZ5zrmVeBZoGk03USniMh2YDsQAHwawzxxomGbkSQJg0nTujkdRcWhYX8OY9upbQybc4OUXXtDkiROR1IqXug01A+oUefcLPU5yImW+/HPkTte31vFvSOXjlBwREEqHfFl3tL0yK7dkCiR07GUilU6DXUMvV27F+eTwsKvWjkdRcWBDks6EBkexrfTLiO9+2gRUF5FC8EDqvL0m2SMSMqkk0vh4EGn46hYNH/PfH7e+TOfbEtDrkwF4I03nI6kVLzSQvCA/Hz8aFzsTeY9YTj/aQ+n46hYEhIWQpuFbSjo/zgf/Xoa+vYFX1+nYykVr7QQPIS3nm5JmC/M2D4Nduvi9glB/9/7c/jSYUbNF/yfLGpXqVPKy2gheAjFMxWncLoCTCgh0KeP03FUDG0/tZ0h64bQJNnTPLv2GPTvDz76K6G8j/6rfwgiwnulW/DX45FsWzkNtm51OpJ6RBGREbw/733SJknL0DGHoHRpePllp2Mp5QgtBA/praJv4e/jz9jyiaGbjivwVMP/Gs5fx/7im4iqpN93HAYNAhGnYynlCC0EDyl9svQ0KNSAycV9uLZsEaxc6XQk9ZAOXTxEjxU9qJmzCq9/sQCqV4fnn3c6llKO0ULwCJo91YyLXGPWM+mgc2fwwEF53soYwwfzPwDgu115kYuXYOBAh1Mp5SwtBI+gUs5K5E2Xl7FVAyAwEGbqusaeYur2qSzZv4QBpbqQ/esJdsxAsWJOx1LKUVoIHoGI0OypZqwJ3cOup/NB9+4QFuZ0LHUfZ0PO0mFJB8pmKUurmYcgMtL2FFLKy2kheERNijXBz8ePcW8Vhv37YcwYpyOp++i4pCMXr19kXOHu+H7/A7RqBTlzOh1LKcdpIXhEGVNkpE7+OvxweTXXnq8I/frBlStOx1J3sWTfEiZtm0S3it0oPGiCXWegh44QVwq0EMRI69KtOXftHNPaVILTp+HLL52OpKJxJfQKLea3oEBAAXokegF+/RW6dIGAAKejKeUWtBDEQKWclSj8WGG+PbMA07ABDB4Mp045HUvdofOyzvxz6R/GvzyOxF17QqZM0KGD07GUchsxKgQikk5ElonIXtfPtHc5LiLKojRzo2zPJSJ/isg+EZnuWs3MY4gIbUq3YfPJzaz9sCGEhurUE25m+YHljNo4io7lO/L0+mPwv//ZBmJdglKpW2K0MI2IfAGcN8YMFJGuQFpjzH8WoBeRYGNMimi2zwB+NsZME5FRwFZjzHf3e18nFqa5m6s3rpL1q6xUy1ONaasfgxEj7NQThQs7Hc3rXQ69TJHvipDULymbm6wjadGnIHVq2LhRZxhVXimuFqapA0x03Z+IXXf4QQMJ8AJwcx3jh3q+u0jun5z3SrzHrL9ncaxTc/tF8+GHOsjMDXy89GOOXj7KD3V/IOmI0XDokG3H0SKg1G1iWggyGmNOuO6fBDLe5bgkIhIoIutFpK5rW3rgojEm3PX4KJDlbm8kIs1drxF45syZGMaOXa1KtyLSRDJq33Q7n/3y5TB/vtOxvNrS/UsZs2kMH5X/iHJ+OeGzz6B2bXjhBaejKeV27ntpSESWA5mi2dUDmGiMSRPl2AvGmP+0E4hIFmPMMRHJDawAXgQuAeuNMXldx2QDFhlj7ntNxZ0uDd1U+6farD+6nsOt95G0ZFmIiIAdO8Dfo5o9EoRL1y9R5LsiJPdPzuYWm0nSsi1MnAhBQZAvn9PxlHLMI18aMsZUNsYUjuY2BzglIpldb5AZOH2X1zjm+nkAWAWUAM4BaUTEz3VYVuDYI3w2t/BR+Y84E3KGH4Km2MsPe/fa9gIV7zot7cSxK8f4oc4PJAnaDePHQ5s2WgSUuouYXhqaCzRx3W8CzLnzABFJKyKJXfcDgArA38aeiqwEGt7r+Z7i2RzPUiZLGYasG0JEtapQo4a9TORml7ESuoV7FzJu8zg+fvpjymYpAx07Qrp08MknTkdTym3FtBAMBKqIyF6gsusxIlJKRMa5jikIBIrIVuwX/0BjzN+ufV2AjiKyD9tmMD6GeRwjInSp0IUDFw4we+dsGDoUgoOhd2+no3mN01dP886cdyjyWBH6VOoDc+bAihW2S2/aaHs2K6WIYfdRp7hjGwHYVa8KjSxECv8UBDYLRNq3t5eHNm+GokWdjpegGWOoM60OS/cvZUOzDRRJmQcKFYKUKWHTJkiUyOmISjkurrqPqih8fXzpVL4Tm05sYsVB11+i6dJB69banTSOjdk4hnl75jGw8kCKZCwCAwbA4cO2EGsRUOqetBDEsreKvUWmFJkY9L9BtggMGgR//AE//uh0tARr99ndfLjkQyrnrky7su1g3z744gu71sCzzzodTym3p4UgliXxS8KH5T5k2YFlrD+6Hpo2haefho8/hgsXnI6X4IRFhPHmL2+SNFFSJtadiA8C7dpB4sR27iel1H1pIYgDrUq3IiBZAH1W9QEfHxg5Es6d02mP40Df3/sSeDyQMS+N4fGUj9sG4kWL7LTgmTM7HU8pj6CFIA6k8E9B56c7s2T/EtYeWWuXQmzXDkaNgg0bnI6XYKw6tIoBfwygafGmNCjUAEJCoH17O89TmzZOx1PKY2ghiCOtSrfiseSP0XuVq/to3752+uOWLe2oYxUjp6+epvHsxuRNl5dh1YfZjf37wz//2AZiP797v4BS6hYtBHEkuX9yOj/dmeUHlrPm8BpIlcqOON640Z4ZqEcWaSJ565e3OH/tPDMaziBl4pR2xtfBg+Gdd7SBWKmHpIUgDrUs3ZKMyTPSa1UvjDHw2mtQpQp07Wr/clWPZOAfA1m6fynDagyjWKZiEB4O778P6dPDkCFOx1PK42ghiEPJEiWjxzM9WHVoFYv2LQIRu8i9MfDBBzq24BGsObyGT1Z+wuuFX6fZU83sxmHDIDAQvv3WdtlVSj0ULQRxrEWpFuRNl5fOyzoTHhkOOXPC55/bni2TJzsdz6OcDTlLo9mNyJ02N6NfGo2IwIED0LMnvPwyvPKK0xGV8khaCOKYv68/A14cQNCZIH7Y8oPd2Lq1HVvQoYOucfyAIiIjaDy7MWdCzjCj4QxSJU7175mVn5/toividEylPJIWgnjQoGADymctT6+Vvbh646pdIWvcODspXdu2TsfzCD1W9GDZgWWMrDmSEplL2I3ffw/LltnpJLJmdTagUh5MC0E8EBGGVB3CieATDFnraswsWNDOTDpzJvz8s7MB3dzMoJkM+t8gWpRswXtPvWc3Hjxoz6ief952yVVKPTKdfTQevTLzFRbsWcDfrf8mZ5qcEBYG5crZHkTbt9txBuo2O07voNy4chTNWJSVTVaS2C8xREbaJSc3bbL/3XLkcDqmUh5BZx91A19W/RIR4cMlH9oNiRLZBuPgYHj3Xe1FdIeL1y9Sb3o9UiZOyaxXZ9kiAPDNN/D77/anFgGlYkwLQTzKljobvZ7txa+7fmXh3oV2Y8GCdqbMRYt0oFkU4ZHhNJrdiEMXDzHzlZl2HiGAv/+Gbt3sQvRNmzqaUamEIkaFQETSicgyEdnr+hndwvXPi8iWKLfrIlLXte8HETkYZV/xmOTxBB+W/5ACAQVou6gt18Ov242tW0O1avDRR7B7t7MB3YAxhvaL2rN432JG1hxJxewV7Y7QUHjrLbvYzJgx2ktIqVgS0zOCrsBvxph8wG+ux7cxxqw0xhQ3xhQHXgBCgKVRDvn45n5jzJYY5nF7/r7+jKg5ggMXDvDZ6s/sRh8f2wMmaVJ4803bduDFvv3rW0YGjqRT+U40K9ns3x1duth2gXHjIGNG5wIqlcDEtBDUASa67k8E6t7n+IbAImNMSAzf16O9kOsF3i72NgP+GMCmE5vsxscfh7Fj7QjZrv+pp15j/p75fLjkQ+oWqMvAygP/3TF3rm0TaNcO6tRxLqBSCVCMeg2JyEVjTBrXfQEu3Hx8l+NXAF8aY+a7Hv8AlAdCcZ1RGGNC7/Lc5kBzgOzZs5c8fPjwI+d2BxeuXeDJkU+SIXkGNjTbgL+vv93Rti0MH267lNar52zIeLbl5BYqfl+R/AH5Wd10Ncn9k9sd//wDxYtDrlywdq1ddEYp9dAeudeQiCwXkR3R3G77s8zYinLXqiIimYEiwJIom7sBBYDSQDqgy92eb4wZY4wpZYwplSFDhvvFdntpk6Zl9Euj2XZq27+XiMBOmla6tJ1F88AB5wLGswMXDlBjSg3SJk3LvEbz/i0CYWHQqJGdWG76dC0CSsWB+xYCY0xlY0zhaG5zgFOuL/ibX/Sn7/FSrwK/GGNuXQA3xpwwVigwASgTs4/jWV7O/zJvFn2Tz//4nMDjrnERiRPDjBm2IfSVV+D6dWdDxoOTwSepMqkKNyJusOTNJf/2EAK7xOfatTB6NOTN61xIpRKwmLYRzAWauO43Aebc49hGwE9RN0QpIoJtX9gRwzwe55vq35A5RWZen/U6l0Mv2405c8LEibZhtG3bBD2+4OL1i1SbXI1TwadY2HghhTIU+nfnpEm2XaB9e3tWoJSKEzEtBAOBKiKyF6jseoyIlBKRcTcPEpGcQDbg9zueP0VEtgPbgQDg0xjm8TjpkqZjaoOpHLp4iBbzW3CrzaZ2beje3faQGT7c2ZBxJCQshNo/1WbnmZ38/NrPlM1a9t+dgYHQrJmdQkIXoVcqTukUE27i8zWf02NFD8a+PJb3n3rfboyMhPr1Yd48O+CsalVnQ8aikLAQXv7pZVYeXMlPDX7itcKv/bvz9GkoWdJ2qw0MhATQJqSUO9ApJtxc14pdqZy7Mm0Xtf23vcDHx05BUbgwvPpqghlsFrUITKw78fYiEBJi1xY4dw5++UWLgFLxQAuBm/ARH6bWn0rG5BmpO60uJ66csDtSpLB96P39oVYtj1+/4M4i8Faxt/7dGREBjRvDhg0wdSo89ZRzQZXyIloI3EiG5BmY8/ocLly/QL3p9f6dgiJHDpgzB44fh5o14fJlZ4M+okvXL1FzSs3oi4AxdrDYnDl2ycm6dR3LqZS30ULgZoplKsakepP489ifNP21KRGREXZH+fIwaxZs3WoHmoVGO+7ObZ0MPslzPzzH/478j8n1J99eBMAu3zlypO0u2rq1MyGV8lJaCNxQ/YL1GVR5ENODptNuUbt/exLVrAkTJsCKFfDGGx4zJ9H+8/up8H0F9p3fx/xG82lcpPHtBwwdatcdfustGDgw+hdRSsUZP6cDqOh1rtCZsyFnGbx2MOmTpaff8/3sjrfegvPn7epcjRrBTz/ZdQ3c1Loj66g7vS4RkRGsaLKCMlnuGDM4YgR06mQbw7//3jaQK6XilRYCNzao8iDOXztP/9X98RVfej3XCxGxA6wiI6FjR3jtNZg2zTYmu5kftvxAi/ktyJ46O/MazaNAQIHbDxg+3A6Yq1PH9o7y03+OSjlBf/PcmIgw+qXRRJgI+vzehys3rjC4ymBbDD780P713KEDNGhg5+FJlszpyIBdVKbzss58tf4rKueuzPSG00mXNN2/BxgD/fvbNZtr17bZ3fisRqmETguBm/P18WV87fGkSJSCoeuGcvH6Rb6r9R2JfBPZMwN/f9u4+sILduCZw/3uD188TOOfG7P2yFral23PkKpD8POJ8s/s5pnMN99AkyZ25LSeCSjlKP0N9AA+4sOwGsNIkyQNn675lP0X9jPrlVmkT5YeWra0i943bgxPP21HIDs0OdvMoJk0m9eMSBPJ1PpTaVTkjvmBrlyxjdzz5tkzmaFDtU1AKTegv4UeQkTo/0J/JtWbxLoj6ygzrgwbj2+0O+vVsz2JLlywU1jPnx+v2U5fPU3j2Y15ddar5A/Iz5YPtvy3CBw4YLvALlxo2wa+/FKLgFJuQn8TPcybRd9kVdNV3Ii4Qbnx5Rj0xyA71qB8eTsiN1cuO0VDjx52pG4cMsYwcctECo4oyKy/Z9H7ud788c4f5E6b+/YDZ82ycwcdPw5LlthLWbresFJuQwuBByqXtRxbP9hK3QJ16fpbVypOqGiXvLy5gtd779kBWs88E2fzE/1+6HfKjitL0zlNKRhQkC0fbKFPpT627eKm4GB4/327rsITT9hC9eKLcZJHKfXotBB4qHRJ0zGj4Qx+rPsj+8/vp/TY0rRa0IpjN87ZBtgpU2DXLrvE4+DBcONGjN/TGMOaw2uoOaUmlSZW4kTwCSbUmcDqd1bfvo4A2KkiChWyYwO6d4c//oA8eWKcQSkVB4wxHncrWbKkUf+6cO2CabuwrfHt62v8+/ublvNbmt1ndxtz4oQxdeoYA8bky2fMnDnGREY+9OsHhwabyVsnm3Ljyhn6YNIPSm8GrBlgQm6E/Pfg7duNeekl+55Fihizdm3MP6BSKlYAgSaa71RdjyABOXjhIAP/GMiELRMIiwyjQrYKvFGkMTWPJiNH90H2DKFcOejSxfbfv0dj7cXrF1l+YDnz98xn9s7ZBN8IJnfa3HQs15F3SrxDskR3jFnYts1ODzFtGqRMaaeM6NBBxwco5Ubuth5BjAqBiLwC9AEKAmWMMdF+O4tIdeAbwBcYZ4y5uZJZLmAakB7YCLxljLnvNQwtBPd24soJJm2bxPebv2f3OdtGkC9dXkpeTUPRtfvIfugiGdNkIVnVWlCtGqEBaTkRfILjV46z4/QONp/cTNDpICJMBKkTp6ZBwQa8XextnsnxDD4SpXhcvGi7go4ZYy/9JEtmxzZ06gTp0kUfTinlmLgqBAWBSGA00Cm6QiAivsAeoApwFNgANDLG/C0iM4CfjTHTRGQUsNUY89393lcLwYMxxrD73G4W71vMqkOr2HJyC4cvHb7nczImCaDE4yUpnbUM1fJUo2zWsnZAWGQknDgBO3fCunWwejWsWgXh4fbaf6tW0LSpFgCl3FicFIIoL76KuxeC8kAfY0w11+Nurl0DgTNAJmNM+J3H3YsWgkd3OfQyJ66c4GTwSUKPHYbfV5EocAuZ/txB5vNhpLkO+PpCmjSQOrWdDiI0FM6e/bfBWcQ2BNesaZfSLFNGxwQo5QHuVgjiY2RxFuBIlMdHgbLYy0EXjTHhUbZnuduLiEhzoDlA9uzZ4yapF0iVOBWpEqcif0B+yAlUeNvuuHYNtm+3f/Hv3WsHp128aL/0kySB9Olt99S8ee2gtdSpHfwUSqnYdN9CICLLgUzR7OphjJkT+5GiZ4wZA4wBe0YQX+/rNZImtX/Zlylz/2OVUgnKfQuBMaZyDN/jGJAtyuOsrm3ngDQi4uc6K7i5XSmlVDyKjwu7G4B8IpJLRPyB14G5rj6tK4GGruOaAPF2hqGUUsqKUSEQkXoichQoDywQkSWu7Y+LyEIA11/7bYAlwE5ghjEmyPUSXYCOIrIP22YwPiZ5lFJKPTwdUKaUUl7ibr2GtM+fUkp5OS0ESinl5bQQKKWUl9NCoJRSXs4jG4tF5Axw70lzHk0AcDYOXje+eHp+8PzP4On5wfM/g6fnh7j7DDmMMRnu3OiRhSCuiEhgdC3qnsLT84PnfwZPzw+e/xk8PT/E/2fQS0NKKeXltBAopZSX00JwuzFOB4ghT88Pnv8ZPD0/eP5n8PT8EM+fQdsIlFLKy+kZgVJKeTktBEop5eW0ENxBRPqLyDYR2SIiS0XkcaczPQwRGSwiu1yf4RcRSeN0poclIq+ISJCIRIqIx3QDFJHqIrJbRPaJSFen8zwsEfleRE6LyA6nszwKEckmIitF5G/Xv5/2Tmd6GCKSRET+EpGtrvx94+29tY3gdiKSyhhz2XW/HVDIGPOBw7EemIhUBVa41oEeBGCM6eJwrIciIgWBSGA0d1kL292IiC+wB6iCXXZ1A9DIGPO3o8Eegog8CwQDPxpjCjud52GJSGYgszFmk4ikBDYCdT3l/4GICJDcGBMsIomAP4D2xpj1cf3eekZwh5tFwCU54FGV0hizNMo60OuxK795FGPMTmPMbqdzPKQywD5jzAFjzA1gGlDH4UwPxRizGjjvdI5HZYw5YYzZ5Lp/Bbv+yV3XQXc3xgp2PUzkusXL948WgmiIyGcicgR4A+jldJ4YeBdY5HQIL5EFOBLl8VE86EsooRGRnEAJ4E+HozwUEfEVkS3AaWCZMSZe8ntlIRCR5SKyI5pbHQBjTA9jTDZgCnZ1Nbdyv/yuY3oA4djP4HYe5DMo9ShEJAUwG+hwxxm+2zPGRBhjimPP5MuISLxcorvv4vUJkTGm8gMeOgVYCPSOwzgP7X75RaQp8BLwonHTRqCH+H/gKY4B2aI8zurapuKR69r6bGCKMeZnp/M8KmPMRRFZCVQH4rzx3ivPCO5FRPJFeVgH2OVUlkchItWBzkBtY0yI03m8yAYgn4jkEhF/4HVgrsOZvIqrsXU8sNMY86XTeR6WiGS42ctPRJJiOx7Ey/eP9hq6g4jMBvJje60cBj4wxnjMX3Yisg9IDJxzbVrvSb2eAESkHvAtkAG4CGwxxlRzNNQDEJGawNeAL/C9MeYzZxM9HBH5CaiEnQL5FNDbGDPe0VAPQUQqAmuA7djfX4DuxpiFzqV6cCJSFJiI/ffjA8wwxvSLl/fWQqCUUt5NLw0ppZSX00KglFJeTguBUkp5OS0ESinl5bQQKKWUl9NCoJRSXk4LgVJKebn/A+HLAUVq8yimAAAAAElFTkSuQmCC\n" + }, + "metadata": { + "needs_background": "light" + } + } + ], + "source": [ + "def predict_sin(x):\n", + " return a+b*x+c*x**2+d*x**3\n", + "\n", + "import matplotlib.pyplot as plt \n", + "plt.plot(x,y,c='r')\n", + "plt.plot(x,predict_sin(x),c='g')\n", + "plt.show()" + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "metadata": {}, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "99 2079.260498046875\n", + "199 1468.034912109375\n", + "299 1037.5106201171875\n", + "399 734.1796264648438\n", + "499 520.4055786132812\n", + "599 369.7096252441406\n", + "699 263.45355224609375\n", + "799 188.51531982421875\n", + "899 135.653076171875\n", + "999 98.35609436035156\n", + "1099 72.0362319946289\n", + "1199 53.459503173828125\n", + "1299 40.345767974853516\n", + "1399 31.087013244628906\n", + "1499 24.54908561706543\n", + "1599 19.931785583496094\n", + "1699 16.670467376708984\n", + "1799 14.366645812988281\n", + "1899 12.739038467407227\n", + "1999 11.589033126831055\n", + "Result: y = -0.055228542536497116 + 0.8499048948287964 x + 0.009527843445539474 x^2 + -0.09235803782939911 x^3\n" + ] + } + ], + "source": [ + "# 2 使用pytorch张量进行梯度下降\n", + "\n", + "import torch\n", + "import math\n", + "\n", + "dtype = torch.float\n", + "device = torch.device(\"cpu\")\n", + "\n", + "x = torch.linspace(-math.pi,math.pi,2000,device=device,dtype=dtype)\n", + "y = torch.sin(x)\n", + "\n", + "a = torch.randn((),device=device,dtype=dtype)\n", + "b = torch.randn((),device=device,dtype=dtype)\n", + "c = torch.randn((),device=device,dtype=dtype)\n", + "d = torch.randn((),device=device,dtype=dtype)\n", + "\n", + "learning_rate = 1e-6\n", + "\n", + "for t in range(2000):\n", + " y_pred = a+b*x+c*x**2+d*x**3\n", + " loss = (y_pred - y).pow(2).sum().item()\n", + "\n", + " if t % 100 ==99:\n", + " print(t,loss)\n", + " \n", + " # Backprop to compute gradients of a, b, c, d with respect to loss\n", + " grad_y_pred = 2.0 * (y_pred - y)\n", + " grad_a = grad_y_pred.sum()\n", + " grad_b = (grad_y_pred * x).sum()\n", + " grad_c = (grad_y_pred * x ** 2).sum()\n", + " grad_d = (grad_y_pred * x ** 3).sum()\n", + "\n", + " # Update weights using gradient descent\n", + " a -= learning_rate * grad_a\n", + " b -= learning_rate * grad_b\n", + " c -= learning_rate * grad_c\n", + " d -= learning_rate * grad_d\n", + "\n", + "\n", + "print(f'Result: y = {a.item()} + {b.item()} x + {c.item()} x^2 + {d.item()} x^3')" + ] + }, + { + "cell_type": "code", + "execution_count": 24, + "metadata": {}, + "outputs": [ + { + "output_type": "display_data", + "data": { + "text/plain": "
", + "image/svg+xml": "\r\n\r\n\r\n\r\n \r\n \r\n \r\n \r\n 2021-03-23T15:21:40.472282\r\n image/svg+xml\r\n \r\n \r\n Matplotlib v3.3.2, https://matplotlib.org/\r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n\r\n", + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAD4CAYAAADhNOGaAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAA+j0lEQVR4nO3dd3gUVdvH8e+dSpfekYBEpIggERFEuhSpgghKF1GUaqNLEx6KL6AICIKIqBRBJEjvINKCAoL0Jp3Qe0g57x+zYAgJLZud3ez9ua69sjszu/vbR56998w5c44YY1BKKeW9fOwOoJRSyl5aCJRSystpIVBKKS+nhUAppbycFgKllPJyfnYHeBSZM2c2QUFBdsdQSimPsnnz5jPGmCxxt3tkIQgKCiIsLMzuGEop5VFE5HB82/XUkFJKeTktBEop5eW0ECillJfTQqCUUl5OC4FSSnk5pxQCEflWRE6LyPYE9ouIfCki+0Rkm4g8G2tfCxHZ67i1cEYepZRSD85ZLYLvgOr32F8DCHbc2gJjAUQkI9AHeB4oBfQRkQxOyqSUUuoBOOU6AmPMahEJuschdYHvjTXn9XoRSS8iOYAKwBJjzDkAEVmCVVCmOiOXUsqNXL0Ke/dat1On4PJluHkTUqeGtGkhKAgKFLD++vrandaruOqCslzAkViPjzq2JbT9LiLSFqs1weOPP540KZVSzhMdDatXw2+/WX///BNiYu7/vMcegzJloFIlaNAA8uVL+qxezmM6i40x440xIcaYkCxZ7rpCWinlLg4cgA8/hNy5rS/zMWMgTRro2RN+/hm2bIHTp+H6dYiKgkuX4N9/rWIxYQK8/jocOgQffwz588Nzz8E331jHqyThqhbBMSBPrMe5HduOYZ0eir19pYsyKaWcacsWGDAAfv0VfHygdm1o0gReeQVSpUr4eWnTWrc8eaBcOXjrLQCu7tnBv7MncWnBHCIGtsWM+pD09ZuQqW1ncuYsiI94zO9Yt+eqQhAKtBeRaVgdwxeNMSdEZBEwKFYH8ctAdxdlUko5w+HD0Ls3/PCDdVqna1d4/33IFe9Z3nhFxUSx5vAaVh1exfqj6/nr5F+cvnra2lnx1lGXgfEwYTwp8adQ9qI8n7s0lfJVonK+ymRIqeNMHpVTCoGITMX6ZZ9ZRI5ijQTyBzDGfA3MB2oC+4BrQCvHvnMiMgDY5Hip/rc6jpVSbi46GkaOtIqAMfDJJ9CtG6RP/0BPjzExrDi4gu+3fc/c3XM5f+M8PuJD0axFqRVciwIZC5A3fV4ypMhAoF8gxhgu/LOZ8Bnfsfv0Tnbk38OU8N2MDRuLv48/NYNr0vyZ5tQpWAc/H4+cT9M24omL14eEhBidfVQpG+3cCS1awKZN1imgr76CBxzEcfXmVcZvHs/YsLHsPbeX9CnSU6dgHeoVrEfl/JVJF5ju/i8SGgrvv0/kiaNs7NSQXyrn5KedMzh55ST5M+Tnwxc+pHWJ1qTwS5HID5q8iMhmY0zIXdu1ECilHsr330O7dtZ5/6++gkaNQOS+T7t68ypjNo1h2B/DCL8WTtk8ZWkX0o4GhRs82hf25cvQvTuMHg3PPEP01J8IZTdD1g5hw7EN5Eufj6FVh9KgUAPkAfJ5g4QKgfa2KKUezPXr0Lq11RIoVQq2bbNG+NznS9YYw4/bfiR4VDCfLP2EZ3M8yx+t/+D31r/zZrE3H/1Xe9q0ViGaOxeOHsX3uVLU3x7FurfWsbjpYlIHpOa1n1+j6pSqHL4Q7zT8ykELgVLq/k6ftoaCfvcdfPopLF0KOXLc92nbT2+n/HflaTq7KbnS5WJt67UsbLqQF/K84LxstWrB1q3wzDPQqBHy2WdUzV+Fv975i9E1R7Ph2AaeHvs0k/6ahCeeAXEFLQRKqXvbuRNKl7a+bGfNgn797nvlb3RMNEPXDqXk+JL8E/4P39T+hg1tNlAmT5mkyZgrFyxbBs2aWYXqzTfxi4zmvefe4+92f1MyZ0lah7bm7blvcyPqRtJk8GData6USti6dVCzJgQGwqpV1sVd93Hw/EGa/9qc3//9nVcLvcrXr3xNltQuuAg0RQqYPBkKFYIePeDsWfjlF4LSB7G02VL6ruzLZ2s+Y+uprfz6+q/kSvfgw1uTO20RKKXit2oVVK0KmTPD+vUPVATm7p5LiXEl+PvU30ypP4WZr810TRG4RcTqQJ440Tp9Va0aXLiAr48vAyoNYPbrs9l1Zhdlvi3D7jO7XZfLzWkhUErdbckSqFHDGhK6erU1Edw9RMdE03t5b+pMq8MTGZ9gy7tbaFqsqX2jdVq3hmnTYONGq2/j/HkA6j1Vj9UtV3Mj6gYvTnqRTcc23eeFvIMWAqXUnVassK4NCA6GlSvv2yl8OeIydabV4bM1n9G6eGvWtl5LUPogl0S9p9degzlzYMcOq6hdvgxAiRwlWNt6LekC01H5+8paDNBCoJSKLSwM6tSBJ56A5csha9Z7Hn788nHKf1eeRfsWMabmGCbWneheF3HVqAEzZlifq1YtuHYNgAIZC7C65WqypM5CtR+qseXkFntz2kwLgVLKsnMnVK9u9QksXgyZMt3z8B2nd1B6Qmn2nN1DaJNQ2j3XzkVBH1LdujBlCqxZAw0bQmQkALnS5WJZ82WkCUhD1SlVvbrPQAuBUgqOHrU6hv39rU7W+0wYt/7oesp+W5bImEhWt1pNzeCaLgr6iJo0gXHjYMECeO89a24kICh9EMuaL8NHfKj5U03Cr4bbHNQeWgiU8nZXrlh9ApcuwaJF1mmhe1hzeA1Vp1Qlc6rMrHtrHc/mePaex7uNt9+21kSYMAEGD769OThTMHMaz+H45ePUm17PK68z0EKglDeLjoamTa3pIqZPh2LF7nn4sgPLqP5jdXKny82qlqvco1P4YQwYAG+8YV1n8NNPtzeXzl2aKfWn8MeRP2g9p7XXXYGshUApb9atmzWyZuRIq2P1HhbtW8QrP71C/gz5WdlipWdekCUC334L5ctbQ0w3/TdiqGHhhgyqNIip26cyauMoG0O6nhYCpbzV99/D559b58zbt7/noasPr6be9HoUylKIFS1WkC1NNheFTAKBgTBzJmTPDq++CqdO3d7V9cWu1ClYhw8Xf8i6I+tsDOlaWgiU8kZbt8I770CFCvDFF/ecQTTseBi1fqpFUPogFjddTOZUmV2XM6lkzmwtqXn2rDWNtmMkkY/48F3d78iTLg+NZjbyms5jpxQCEakuIrtFZJ+IdItn/wgR2eK47RGRC7H2RcfaF+qMPEqpe7hwARo0gIwZratv/RKecuyf8H+o/kN1MqbMyJJmS1w7XURSK17c6jhevRo++uj25gwpMzCz0UzCr4bTZm4br+gvSHQhEBFfYDRQAygMNBGRwrGPMcZ0McYUN8YUB0YBv8Taff3WPmNMncTmUUrdQ0yMtZ7A4cPWhVbZEj7Fc+jCIap8X4UA3wCWNV9G7nS5XRjURd54A7p0gS+/tP73cHg2x7MMqjyI0N2hTNoyycaAruGMFkEpYJ8x5oAx5iYwDah7j+ObAFOd8L5KqYc1bJi1zOPnn0PZsgkeduHGBWr+WJPrUddZ3GwxT2S895BSjzZkiDXN9ttvw8GDtzd3Lt2ZCkEV6LSwEwfPH7zHC3g+ZxSCXMCRWI+POrbdRUTyAvmA5bE2pxCRMBFZLyL1EnoTEWnrOC4sPNw7ztsp5VSbNkGvXtbVtR07JnjYzeibvDr9Vfad28fs12dTNGtRF4a0gb8/TJ1q9ZM0aXJXf4GP+NDi1xbEmBibgyYdV3cWNwZmGmOiY23L61hD8w1gpIjE+9PDGDPeGBNijAnJkiUZnadUyhWuXLFOg+TIAePHJ9g5bIyhTWgbVhxawbd1v6VCUAXX5rRLUJDVX7BhA/TufXtz3vR5+aL6F6z5dw3jN4+3L18Sc0YhOAbkifU4t2NbfBoT57SQMeaY4+8BYCVQwgmZlFKxdeoE+/dbc+5kyJDgYf1W9WPKtikMqDiApsWaujCgG2jY0BpJNWSINdeSQ4tnWlA5X2W6Le3GicsnbAyYdJxRCDYBwSKST0QCsL7s7xr9IyJPARmAdbG2ZRCRQMf9zEBZ4B8nZFJK3TJzpnURVffu1oVUCZi+fTr9VvWjVfFW9CzX04UB3ciIEVCkCLRsCefOASAijH1lLDeibtBpYSd78yWRRBcCY0wU0B5YBOwEZhhjdohIfxGJPQqoMTDN3DkWqxAQJiJbgRXAYGOMFgKlnOXoUasTtFQp6Ns3wcO2ntxKqzmtKJunLF/X+tq+BWXsljKl1WoKD7/jIrvgTMH0fqk3P//zM/P2zLMxYNIQTxwjGxISYsLCwuyOoZR7M8Zab3j1ausCsgIF4j3szLUzhIwPISomirC2YWRPk93FQd3QgAHw6afWkNLXXgOsTvRnxz3L1cir7Hx/p3utu/CARGSzo0/2DnplsVLJ1XffwcKF1jnvBIpAVEwUjX5uxMkrJ5n9+mwtArd0726t0dyuHZw8CUCAbwBf1viSQxcOMXL9SHvzOZkWAqWSo6NHrQulXnrJmksoAR8t/ogVh1YwvvZ4nst1/8XpvYafH0yebI22eued2+sXVMpXiboF6zJwzcBk1XGshUCp5MYY68vr5k2rk9gn/v+bT9s+jS82fEGn5zvR/JnmLg7pAQoVgkGDrAvwfvzx9ubPX/6ciKgIei5PPh3qWgiUSm6mTIH58+F//0twkZndZ3bz9ty3KZOnDMOqDnNxQA/SqZN11XGXLnDmDGCtd9zp+U58t+U7Nh/fbHNA59BCoFRycvKk9eX14ovQoUO8h1yPvE6jmY0I9A1kesPp+Pv6uzikB/H1hW++sSbq++CD25t7vdSLzKky023ZXXNseiQtBEolJ126wLVrMHFigqeEOi7oyLZT25hSf0rynEjO2YoWha5drZbWkiUAPJbiMXqU68HSA0tZfnD5fV7A/WkhUCq5WLzYmla6Rw948sl4D/lh2w9M+GsC3V/sTo3ge69IpmLp1cv63/Tdd61CC7wb8i650+Wmx7IeHj9VtRYCpZKD69etoY5PPmktPxmPneE7eee3d3gp70v0r9jfxQE9XIoU1hxNBw7cvjAvhV8K+pTvw4ZjG5i7Z669+RJJC4FSycHAgdaX1NdfW0sxxhERFUHjWY1J7Z+aqQ2m4ueT8GI0KgHly0ObNjB8OGzZAljzEAVnDKbn8p4ePTupFgKlPN3OnTB0KDRrBhUrxntIj2U92HZqG9/W/ZacaXO6OGAyMnSotbJb+/ZgDP6+/vSv2J/tp7fz846f7U73yLQQKOXJjLHOW6dJYy02E4+lB5YyfP1w2oW0o9aTtVwcMJnJkAEGD4a1a63OY6BRkUY8lfkpBq4Z6LGtAi0ESnmyH3+05hIaMgSyZr1r99lrZ2nxawueyvwUn78cf6FQD6llS3j+efjkE7h4ER/xoceLPfj79N/8tuc3u9M9Ei0ESnmqy5etL6PnnoO33rprtzGGd357h/Cr4fz06k+k8k9lQ8hkyMcHRo+G06dvdxw3eboJ+dLnY+CagR45gkgLgVKeatAgOHHCWng9nmsGJm2ZxKydsxhYaSAlcuh6T05VsqQ1jceoUfD33/j5+NG1bFc2HtvI0gNL7U730HQaaqU80b591gIqjRtbk6PF3X1uH8W/Lk6pXKVY2nwpPqK/+Zzu7FlruG7RorByJRHRN8n/ZX6CMwazsuVKu9PFK0mnoRaR6iKyW0T2ichdg5hFpKWIhIvIFsetTax9LURkr+PWwhl5lEr2PvwQAgKsjss4omOiaflrS/x9/Zlcb7IWgaSSKZM1n9Pq1TB1KoF+gXxc5mNWHV7FuiPr7v98N5LofyEi4guMBmoAhYEmIlI4nkOnG2OKO24THM/NCPQBngdKAX1EJOEFVZVSsGiRNSNmr17WYvRxfLnhS9YeWcsX1b8gz2N54nkB5TRvvWWdJuraFa5do82zbUifIj0j1o+wO9lDccZPhVLAPmPMAWPMTWAaUPcBn1sNWGKMOWeMOQ8sAao7IZNSyVNkJHTubC0007nzXbt3n9lNj+U9qP1kbZoVa+byeF7H19e6wOzoURg+nDQBaWj7bFtm7ZzFoQuH7E73wJxRCHIBR2I9PurYFlcDEdkmIjNF5NbPlAd9LiLSVkTCRCQsPDzcCbGV8kBffQW7dlmLrMe5gjg6JppWc1qR0i8l42qN8951h13tpZfg1Vet03THj9Ph+Q74iA9fbvjS7mQPzFUnD+cCQcaYYli/+u/u3boPY8x4Y0yIMSYkS5YsTg+olNs7exb69YPq1eGVV+7aPWL9CNYdXceoGqPIkfbuU0YqCQ0dai0E1Ls3udPlplGRRkz4cwKXIi7ZneyBOKMQHANin4jM7dh2mzHmrDEmwvFwAlDyQZ+rlHL47DPr2oHPP4c4v/Z3hu+k1/Je1HuqHm88/YZNAb3YE09Ax44waRL89RddSnfh8s3LTPxzot3JHogzCsEmIFhE8olIANAYCI19gIjE/nlSB9jpuL8IeFlEMjg6iV92bFNKxbZ/v3URU+vW1rDRWKJiomg5pyWpA1Iz9pWxekrILr16WfMQffghITlKUu7xcnyx4QuiYqLsTnZfiS4ExpgooD3WF/hOYIYxZoeI9BeROo7DOorIDhHZCnQEWjqeew4YgFVMNgH9HduUUrH16AH+/tD/7umj/++P/2PjsY2Mrjma7Gmy2xBOAZA+vXXqbsUKCA2lc+nOHL54mHl75tmd7L70gjKl3N369fDCC9Cnz+0pDW7Ze3YvT499mprBNZnVaJa2BuwWFQXFikFkJFF/byXvmGCezvo0C5sutDsZkMQXlCmlkogx8NFHkD279feOXYa2v7UlhV8KRtccrUXAHfj5wbBhsG8fft9+x9vPvs2i/YvYf26/3cnuSQuBUu7s11+tKY/79bOmmo7l27++ZeWhlQytOlRHCbmTmjWtRWz69ePtgm/gK76M2zzO7lT3pKeGlHJXkZFWx7C/P2zdav3adDh55SSFRhfi6axPs7LlSp1Gwt1s2AClS0O/frxacAurD6/m6AdHSeGXwtZYempIKU8zbhzs3WuNUfe7c2nJTgs7cS3yGt/U/kaLgDt6/nlo0ACGDaNdgSacvX6WWf/MsjtVgvRfkFLu6PJl63RQxYrWqYZY5u6ey4wdM+j9Um8KZi5oU0B1XwMHwvXrVP5+DQUyFmBs2Fi7EyVIC4FS7mjECDhzxlp5LFYn8KWIS7w3/z2KZi3KJ2U/sTGguq+CBeGtt/AZ+zXv5nuNtUfWsuP0DrtTxUsLgVLu5swZ6+rhV1+1Vh+Lpeeynhy7dIxvan9DgG+ATQHVA+vTB/z8aD5jN34+fkzaMsnuRPHSQqCUuxkyBK5ehQED7ti87sg6Rm8aTftS7Smdu7RN4dRDyZkTunQhy5RfqJ21HFO2TSEyOtLuVHfRQqCUOzl2zJphtFkzKPzfsh6R0ZG0/a0tudPlZmClgTYGVA/tk08gY0Zar7jA6aunmb93vt2J7qKFQCl3MmAAREffdQXxyPUj2X56O6NqjCJtYFp7sqlH89hj0LMn1X/+i+wBGd3y9JAWAqXcxb59MHGitSh6UNDtzUcuHqHvqr7UfrI2dZ960DWflFt57z38cuSi2e4UzNs7j1NXTtmd6A5aCJRyF336WOsQ9+x5x+ZOCzthjOHLGp6z0ImKI0UK6N2bVvOOExUTxQ/bfrA70R20ECjlDrZtg6lToVMna14hh3l75jF712x6v9SboPRB9uVTideqFYXS5qP0udRM2jIJd5rVQQuBUu6gVy/rXPLHH9/edD3yOh0WdOCpzE/xYZkPbQynnCIgAPr0odXaq+wI38GfJ/60O9FtWgiUstsff8DcudbokgwZbm8etGYQBy8cZEzNMXrNQHLx5pu8FlGAgGjhJzc6PaSFQCk7GWP1CWTLZi116LD7zG6G/jGUpsWaUjFfRRsDKqfy8yNDr8+ouccwNew7omOi7U4EOKkQiEh1EdktIvtEpFs8+z8QkX9EZJuILBORvLH2RYvIFsctNO5zlUrWVqyAlSutYpA6NWCtM/D+/PdJ6ZeSz6t+bm8+5XyvvcYbFx/nRPQFVh1YbncawAmFQER8gdFADaAw0ERECsc57C8gxBhTDJgJDI2177oxprjjVgelvIUx1vUCOXPC22/f3jx9x3SWHVzGoMqDyJYmm335VNLw8aFW22GkjYAff7176VE7OKNFUArYZ4w5YIy5CUwD7hjsbIxZYYy55ni4HsjthPdVyrOtWAFr1kD37tbwQuDijYt0WdSFkJwhvFPyHZsDqqSSst5rvHo6E7POreXG1Yt2x3FKIcgFHIn1+KhjW0LeAhbEepxCRMJEZL2I1EvoSSLS1nFcWHh4+CMFXX90vUcsJK28QOzWQJs2tzd/uuJTTl05xdhXxuLr42tfPpW0RHij6gdcDDQsGP/x/Y9PYi7tLBaRpkAIMCzW5ryOFXPeAEaKyBPxPdcYM94YE2KMCcmSJcsjvf+A1QP4cLEOw1NuIJ7WwNaTW/lq01e0C2lHSM67FpFSyUylhh+TLcKfH7dMgevXbc3ijEJwDMgT63Fux7Y7iEgVoCdQxxgTcWu7MeaY4+8BYCVQwgmZ4lU5X2V2n93N0UtHk+otlLq/eFoDxhg6LOhAxpQZ+azSZ/bmUy7h5+vP6/nr8NvjN7g4ZritWZxRCDYBwSKST0QCgMbAHaN/RKQEMA6rCJyOtT2DiAQ67mcGygL/OCFTvKrkrwLAsgPLkuotlLq/eFoD07ZPY82/axhUaRAZUma4zwuo5OKN6h8T4Qe/hg6Fa9fu/4QkkuhCYIyJAtoDi4CdwAxjzA4R6S8it0YBDQPSAD/HGSZaCAgTka3ACmCwMSbJCkHRrEXJkioLyw5qIVA2iac1cOXmFT5a8hElc5SkdYnW9uZTLlUqVykeD8zGzFyXrDWqbeJ3/0PuzxgzH5gfZ9unse5XSeB5fwBPOyPDg/ARHyrlq8TSA0sxxiCxlgBUyiVutQZGjbrdGhi4eiDHLx9nVqNZ2kHsZUSEhiXe5KvrI7g44n889s47kCqVy3N43ZXFlfNV5sSVE+w6s8vuKMrbGGMtSB+rNbD37F7+b93/0eKZFrrqmJdqWLghN30Mv6UPt61V4HWF4HY/gZ4eUq62ciWsXn1H30DnRZ1J4ZeCwVUG25tN2eb53M+TK20uZpbPbC1TakNfgdcVgnwZ8pEvfT6WHlhqdxTlTeLpG/htz2/M3zufvhX6kj1N9ns/XyVbPuJDg0INWJD1EpfPn7KlVeB1hQCs00MrD60kKibK7ijKW8RpDdyIukHnhZ0plLkQHUp1sDudslnDwg2JiLnJ/PpFbGkVeGchyF+ZixEX3Wo+cJWMxdMaGL5uOPvP7+fLGl/i7+tvbz5luzJ5ypA9TXZmvpQZTrm+VeCVhaBSvkoAenpIuUac1sCRi0cYuGYgrxZ69XaflfJuvj6+vPrUq8y/sImrVcq7vFXglYUga+qsFMtWTDuMVdKLpzXw8ZKPiTEx/N/L/2dvNuVWGhZuyLXIayx8p7LVKhg/3mXv7ZWFAKx+grX/ruV6pL1zfKhkLk5rYOWhlUzfMZ2uZbvqGsTqDuXyliNzqsz84rMLKlWyWgUumoPIawtBlfxViIiO4Pd/f7c7ikqu4rQGomKi6LigI3kfy0vXsl3tTqfcjJ+PH7WerMX8vfOJ7N0TTp50WV+B1xaCl/K+RIBvAIv3L7Y7ikqu4rQGxm4ay9+n/2Z4teGk9E9pdzrlhuoWrMuFGxf4Pa+4tFXgtYUgTUAayj1ejoX7F9odRSVHcVoD4VfD+XTlp1TJX4X6T9W3O51yU1XzVyXQN5A5u+dAnz4uaxV4bSEAqF6gOttPb9dpqZXzxWkN9FjWgys3r/Bl9S91jiuVoNQBqamSvwqhu0Mx5cq5rFXg9YUAYNG+RTYnUclKnNZA2PEwJv41kY6lOlIoSyG70yk3V7dgXQ5eOMiO8B0uaxV4dSEokqUIudPlZsG+Bfc/WKkHFas1EBMYQIcFHciaOit9KvSxO5nyALWerAXAnF1z4KWXXNIq8OpCICJUf6I6Sw4sITI60u44KjmI0xqYsnUK64+uZ0iVIaQLTGd3OuUBcqTNQalcpQjd41i2pW9fq1WQhNcVOKUQiEh1EdktIvtEpFs8+wNFZLpj/wYRCYq1r7tj+24RqeaMPA+jeoHqXIq4xIZjG1z91io5utUa6NaNS3KTbsu68Xyu52n2TDO7kykPUrdgXTYe28iJyyegXDmoUMFqFdy4kSTvl+hCICK+wGigBlAYaCIiheMc9hZw3hhTABgBDHE8tzDW0pZFgOrAGMfruUzl/JXxFV8W7tPRQyqRbrUGcuSAt9/ms9WfcfLKSUbVGIWPeHXjWz2kOgWtxR3n7plrbejTB06cgAkTkuT9nPGvsxSwzxhzwBhzE5gG1I1zTF1gsuP+TKCyWEMn6gLTjDERxpiDwD7H6yWNceNg8J3zvqdPkZ4X8ryghUAlXqy+gT1X/2Xk+pG0Kt6K53I9Z3cy5WGKZClCvvT5CN3tOD1UvrzVMhg8GCIinP5+zigEuYAjsR4fdWyL9xjHGscXgUwP+FwARKStiISJSFh4ePijJV23Dvr3t863xVL9iepsPrGZ01dPP9rrKgXW6mOO1kCXRV1I6Z+S/1X+n92plAcSEeoUrMOyg8usaXBErFZBZCTscv7qih7TXjXGjDfGhBhjQrJkyfJoL9KrF9y8CcOG3bH51jBSvcpYPbKVK2HVKujenXn/LmP+3vl8+tKnZEuTze5kykPVKFCDG1E3WHlopbWhUiU4dAieecbp7+WMQnAMyBPrcW7HtniPERE/4DHg7AM+13kKFICmTWHs2DtaBSVylCBr6qw6jFQ9OkffwM3WLeiyqAsFMxWkw/O64Ix6dOWDypPSL+V/30sikDJppiZxRiHYBASLSD4RCcDq/A2Nc0wo0MJxvyGw3BhjHNsbO0YV5QOCgY1OyJSweFoFPuJD9QLVWbRvEdEx0Un69ioZitUa+GLLOPae28vI6iMJ8A2wO5nyYCn8UlAxX0WX/EBNdCFwnPNvDywCdgIzjDE7RKS/iNRxHDYRyCQi+4APgG6O5+4AZgD/AAuB940xSftNnECroFZwLc5eP8u6o+uS9O1VMuRoDZxoUov+q/tT68lat083KpUYNQrUYN+5few7ty9J38cpfQTGmPnGmCeNMU8YYwY6tn1qjAl13L9hjHnNGFPAGFPKGHMg1nMHOp5X0BjjmnMz8bQKqhWohr+P/3+99Eo9iFitge6/9+Nm9E1GVBthdyqVTNQoUAOABXuT9qvRYzqLnSqeVkG6wHRUCKrw37hdpR6EozWw4ZVnmLx1Ml1Kd6FAxgJ2p1LJxBMZnyA4Y3CSnx7yzkIA8bYKaj9Zm11ndrH37F4bgymP4WgNxHTrSodlH5EjTQ56lutpdyqVzNQoUIMVh1Yk6WqK3lsI4mkV1C5YG0BbBer+jLHGdefIweTSKdl0fBNDqgwhbWBau5OpZKZGsDWMdNXhVUn2Ht5bCOCuVkFQ+iCezvq0FgJ1f46riC9160z31Z/yQu4XeLPYm3anUslQ+bzlSeGXIkn7Cby7EMTXKniyNmsOr+H89fM2h1Nu61ZrIGdOBgSf4PTV03xZ40udT0gliZT+KakYlLTDSPVfbpxWQe2CtYk20XpxmUrYihWwZg27P2nNyLCvaFW8FSE5Q+xOpZKxGgVqsPfc3iQbRqqFoEABePPN262CUrlKkTV1Vj09pOLnaA2YXDnpnGEjqfxTMajyILtTqWQuqafB0UIAVqsgIgKGDcNHfKgVXIsFexfoYjXqbsuXw++/M+/D2iw8uJg+5fvofEIqyRXIWICg9EFaCJJUcPAdfQV1CtbhYsTF/yZ7UgputwYi8uSki+9Snsr8FO1Ltbc7lfICIkK1J6qx/ODyJPmBqoXgllitgpefeJnU/qmZtXOW3amUO1m2DNauZWTH59h3fj8jq+l8Qsp1Xn7iZa7cvMLfp/92+mtrIbglVqsg5dmL1Ayuyexds3USOmVxtAaOP5mDzyKXUadgHaoVcPnKqsqL1ShQg/CPw3k2x7NOf20tBLHFahU0KNSA01dPs/bIWrtTKXewdCn88QfdWufhZvRNhr883O5Eysuk9E9JplSZkuS1tRDEFqtVUDNdSQJ9A5n1j54e8nqO1sC6klmZcmMjH5T+gCcyPmF3KqWcRgtBXI5WQdovxlKtQDV+2fULMSbG7lTKTkuWEL1+He3rB5IzbU56vqTzCankRQtBXLFaBQ1yVubopaNsOrbJ7lTKLo7WwDdVMvBn1BE+r/o5aQLS2J1KKadKVCEQkYwiskRE9jr+ZojnmOIisk5EdojINhF5Pda+70TkoIhscdyKJyaP0zhaBbXn7MbPx09HD3mzRYs4s209PV6MoHze8jQu2tjuREo5XWJbBN2AZcaYYGCZ43Fc14DmxpgiQHVgpIikj7X/Y2NMccdtSyLzOIejVZBh7CSq5CrHrJ2zsFbWVF7FGOjblx51U3OJCL6q+RUiYncqpZwusYWgLjDZcX8yUC/uAcaYPcaYvY77x4HTQJZEvm/Sc7QKGuzx48D5A2w9tdXuRMrVFi5k05ENTCh4jY7Pd6Ro1qJ2J1IqSSS2EGQzxpxw3D8J3PNaexEpBQQA+2NtHug4ZTRCRAITmcd5HK2CuuNX4yM+/LzjZ7sTKVcyhpjevXi/fgDZUmejb4W+didSKsnctxCIyFIR2R7PrW7s44x17iTB8ycikgOYArQy5vYwnO7AU8BzQEag6z2e31ZEwkQkLDw8/P6fzBn69CHL5WgqR+Rk2o5penrIm8yezUTzJ5uy3GTYy8NIF5jO7kRKJZn7FgJjTBVjTNF4bnOAU44v+Ftf9Kfjew0RSQfMA3oaY9bHeu0TxhIBTAJK3SPHeGNMiDEmJEsWF51Zyp8f2rThjYXHOXD+ABuPbXTN+yp7RUdzbkAPulfzpVyeF3nzaV1wRiVviT01FAq0cNxvAcyJe4CIBACzge+NMTPj7LtVRASrf2F7IvM4X69e1N/vT2CMDz/9/ZPdaZQrTJ1Kz1y7uRBo+OqV0dpBrJK9xBaCwUBVEdkLVHE8RkRCRGSC45hGwEtAy3iGif4oIn8DfwOZgc8Smcf5cuXisTbteWV3DDO2/aRzDyV3kZFs/qIb40Kgfan2FMtWzO5ESiU58cTz3iEhISYsLMx1b3jmDDOr5eG1OjdY2mwplfNXdt17K5eKGfc1ZcLacSgoPbs/PMRjKR6zO5JSTiMim40xdy2np1cWP4jMmXnllc6kjYCfVo2yO41KKjdu8N2MHmzIDUNfGalFQHkNLQQPKOWH3ai/P4BZB+YRERVhdxyVBM6PHU7Xkucpm64ozZ5pbnccpVxGC8GDeuwxmjzzJhf9olgwV6cgTnauXKHHHwM4lwq+ajJFO4iVV9FC8BAqtx9OluvCT0tHWNMPqGRj/ciPGFfkBp2CGlM8e3G74yjlUloIHoJ/uvQ0yfASczKGc27BL3bHUU4Sde4M75z8hpyRKejXZLzdcZRyOS0ED6ll4yHc9INpkz7UVkEy8cWI19mWJYYvXxxI2sC0dsdRyuW0EDykEnmf5xn/PEzKcBh+0VaBp/t390b6xCyn1pVc1K/exe44StlCC8EjaFmhE2G5YPvgDyAy0u44KhE6fduIGIFRzadqB7HyWloIHsGbzzTHD1++y/gvTJhw/ycotxS69Ct+TXWYvlEvEvR0ObvjKGUbLQSPIEvqLNQqWJsfSvoT2b8PXL5sdyT1kK7cvEL7FZ9Q5IwPXbrMsDuOUrbSQvCIWpVoxanASBamC4fPP7c7jnpI/X9sy5GA64zL/jb+2XLYHUcpW2kheEQ1CtQga+qsTKqVC/7v/+DEifs/SbmFbSe3MvzQVNrsSkXZLnpxoFJaCB6Rv68/zYo1Y+5jpzjpHwH9+tkdST2A6Jho3pnSiAzXYXDFQZAqld2RlLKdFoJEePvZt4kyUXzbNsTqNN650+5I6j6+Wv8F66/tYeTfOcnU+n274yjlFrQQJELBzAWplK8S47MdIzp1Suje3e5I6h4OXThEjyXdqbEX3nh3DPj52R1JKbeghSCR2oW04/DlIyz8uD7MmQNr1tgdScXDGMM7s9/C52YkX4c/j9SpY3ckpdxGogqBiGQUkSUistfxN0MCx0XHWp0sNNb2fCKyQUT2ich0x7KWHqVuwbpkT5Odr4POQK5c8MEHEBNjdywVx5RtU1j873IGLzU8Pngs6MVjSt2W2BZBN2CZMSYYWOZ4HJ/rxpjijlvsn2JDgBHGmALAeeCtROZxOX9ff94q8Rbz9i/k8ICPICwMvv/e7lgqllNXTtF5fkfKHhHaPd0aSpSwO5JSbiWxhaAuMNlxfzLWAvQPxLFgfSXg1oL2D/V8d9K2ZFtEhAmPn4HSpa2+Ar3IzG10XNiRqzcuM2FJSnw+G2h3HKXcTmILQTZjzK0B9CeBbAkcl0JEwkRkvYjUc2zLBFwwxkQ5Hh8FciX0RiLS1vEaYeHh4YmM7VyPP/Y4NYNr8s1fE4gYPgxOnoRBg+yOpYDQ3aHM2DGD3itjeKpdb8ie3e5ISrmd+xYCEVkqItvjudWNfZwxxgAJzcuc17Fg8hvASBF54mGDGmPGG2NCjDEhWbJkedinJ7kOpTpw6uoppqc6AM2bw/DhsH+/3bG82oUbF2g3rx3FLqTgk6N5oXNnuyMp5ZbuWwiMMVWMMUXjuc0BTolIDgDH39MJvMYxx98DwEqgBHAWSC8it8bw5QaOJfoT2aRq/qoUyVKEEetHYAYNAn9/+Phju2N5tc4LO3Pq8kkmzrhBwJDPIUUKuyMp5ZYSe2ooFGjhuN8CmBP3ABHJICKBjvuZgbLAP44WxAqg4b2e7ylEhM6lO7Pl5BZWRe6FHj1g9mxYtszuaF4pdHcok7dOpvumFIQ8UQ4aNLA7klJuS0wiVtkSkUzADOBx4DDQyBhzTkRCgHeNMW1EpAwwDojBKjwjjTETHc/PD0wDMgJ/AU2NMRH3e9+QkBATFhb2yLmTyvXI6zw+8nHK5CnDnHrToVAhSJMG/vpLL15yobPXzlJkTBGyX4xi4+BzBGzcrCOFlAJEZLPjNP0dEvXtZIw5C1SOZ3sY0MZx/w/g6QSefwAolZgM7iSlf0rahbTjs9WfsffqEYL/7/+sX6JjxkDHjnbH8xrvz3+fc9fOsmhiNAHvvq9FQKn70CuLney9597D39efLzZ8AfXrw8svQ69ecPy43dG8wowdM5i+Yzp9dmfnmegsMGCA3ZGUcntaCJwse5rsNCnahElbJnH2+jkYPRpu3oQPP7Q7WrJ36sop3pv3Hs/5B9F12lEYNgzSp7c7llJuTwtBEvi4zMdci7xmtQoKFLAuMJs2DZYssTtasmWMoe1vbbly8wqTJ53Hr8yL0KyZ3bGU8ghaCJJAkaxFqP9UfUZtHMWliEvQtatVEN5/H27csDtesjRpyyRCd4cy8HwJCh28YvXL6HxCSj0QLQRJpGe5nly4cYExm8ZY49fHjIG9e2HoULujJTt7z+6l44KOVMxYki4j1kOHDvB0vOMTlFLx0EKQRErmLEm1J6oxfN1wrkVeg6pV4fXXrakn9u2zO16ycTP6Jm/88gYBvgF8P/kSPrnzQP/+dsdSyqNoIUhCPcv1JPxaOBP+nGBtGD4cAgPhnXcgEddvqP/0WdGHsONhTLhRldyb98LXX0PatHbHUsqjaCFIQuXyluPFx19k6Nqh3Ii6ATlzWiNZli+3lrZUibLi4AqGrB1Cm3wNeHXgbHjjDahZ0+5YSnkcLQRJrH+F/hy7fIyxm8ZaG95+GypVsoaTHjlibzgPdu76OZrNbkZwxmBGjv8X0qWDkSPtjqWUR9JCkMQq5qtIlfxVGPT7IC5HXLZGsnzzDURHw7vv6imiR2CMoU1oG05fPc1PUXVIvXaTVQTccFZapTyBFgIXGFhpIGeunWHk+pHWhvz54X//g/nz4YcfbM3mib7Y8AWzd83mf8U/pGTvsVCtGrz5pt2xlPJYiZp0zi7uOuncvdSfXp/lB5dzsNNBMqbMaK1r/NJL8M8/1k0XTHkg64+up9ykcrxSoCazR59Btu+Av/+GPHnsjqaU20to0jltEbjIgIoDuBxxmSG/D7E2+PjAxIlw/Tq0aaOniB7A2WtnafRzI/Kky8Okf0sga/+Ar77SIqBUImkhcJGiWYvStFhTvtjwBQfPH7Q2FiwIQ4bAvHnWsEeVoBgTQ7PZzTh19RQ/Fx9Ehk//Z83sqqeElEo0LQQuNKjyIHx9fPlk6Sf/bezQAapXt0YR7dxpXzg3N/j3wSzYt4CRlT+nZMf/QYYMMHasTiOhlBMkqhCISEYRWSIiex1/M8RzTEUR2RLrduPWAvYi8p2IHIy1r3hi8ri73Oly07VsV2b+M5NVh1ZZG0Vg0iRIndr6dXvzpr0h3dDi/YvpvaI3jYs25t1fj8C2bdbIKx0lpJRTJLZF0A1YZowJBpY5Ht/BGLPCGFPcGFMcqARcAxbHOuTjW/uNMVsSmcftfVTmI/Kky0PnRZ2Jjom2NmbPbvUX/PUX9O5tb0A3s+/cPl6f+TpFshThmxSNkKHDrCuza9e2O5pSyUZiC0FdYLLj/mSg3n2ObwgsMMZcS+T7eqxU/qkYWnUoW05u4du/vv1vR5060LatdeXx4sUJv4AXuRRxiTpT6+AjPsypOI40Ldtak8mNGGF3NKWSlcQWgmzGmBOO+yeBbPc5vjEwNc62gSKyTURG3FrkPrl7vcjrlHu8HN2WdSP8avh/O0aMgCJFrKkSvPyq41udw3vO7uHnV6eR751ucO0azJgBKVPaHU+pZOW+hUBElorI9nhudWMfZ6wLEhIcAykiObDWLl4Ua3N34CngOawF7Lve4/ltRSRMRMLCw8MTOswjiAhf1/qayxGX+WDxB//tSJUKZs6EiAhrplIv7i/ot7IfobtDGV5tOJWm/A6rV1udw089ZXc0pZIfY8wj34DdQA7H/RzA7nsc2wkYf4/9FYDfHuR9S5YsaZKD3st7G/piFu1bdOeO6dONAWM6d7YnmM2mbJ1i6Itp9WsrExMaaoyIMS1a2B1LKY8HhJl4vlMTe2ooFGjhuN8CmHOPY5sQ57SQo5WAiAhW/8L2RObxKD3K9eDJTE/y7m/vWmsW3NKoEXTsaM2fM2OGbfnssPzgclrPaU3FoIp8HdwFadoUihe3FvZRSiWJxBaCwUBVEdkLVHE8RkRCROT2PMsiEgTkAVbFef6PIvI38DeQGfgskXk8Sgq/FIyvNZ6DFw7SY1mPO3cOGwYvvACtWsGff9oT0MV2nN7Bq9Nf5clMT/JLje8IaNAIAgJg9mzrtJlSKknoXENuoMP8Dny16SuWNFtClfxV/ttx8iSUKmXNS7Rxo7WeQTJ1/PJxSk8oTVRMFOtb/8HjLTtZV1wvXQoVKtgdT6lkQecacmNDqg7hqcxP0fLXlpy7fu6/Hdmzw9y5cOEC1K1rjZpJhs5fP0/NH2ty7vo55r0xj8cHfAmhodYoKi0CSiU5LQRuIJV/Kn6o/wOnrp7ivXnvcUcr7Zln4KefYPNmaNHCWscgGbly8wo1f6rJzjM7+eX1XygxY7VVADp0gPbt7Y6nlFfQQuAmSuYsSb8K/Zi+Y/p/axzfUqcOfP65NbS0Y8dkM1Ppjagb1J1Wl03HNjGtwTRe3nIZunSB+vWtYqDzCCnlEn52B1D/6Vq2K6sOr6L9gvaUyFGCkJyxTuV98IHVZzBsmDXHTt++tuV0hsjoSBr93IjlB5czpf4U6p/KAE1rQOnS8OOP4Otrd0SlvIa2CNyIr48vP736E9nTZKfhjIacvXb2zgOGDLFGEfXrB6NG2RPSCSKiImj4c0Pm7pnLmJpjaHolP9SqZa3cFhqqVw4r5WJaCNxMplSZmPnaTE5cOUHjWY2JjI78b6cIjB8P9epZp4hGj7Yt56O6HnmdetPrEbo7lNE1R9OOEKhRA3LlgmXLIHNmuyMq5XW0ELih53I9x/ha41l6YCnv/vbunZ3Hfn4wbZo1iqh9e+uiMw9x5eYVXvnpFRbtW8SE2hN4L/pZePllyJTJKgK6XKdSttA+AjfVongLDpw/QP/V/cmXIR+9Xur1387AQOuK4yZNrM7ViAj45BO37lwNvxpO7am12XR8E9/X/56mp7JBvcqQI4d1rUDu3HZHVMpraSFwY30r9OXQxUP0XtGbbKmz8XbJt//bGRBgtQyaN4du3eDoUat14IadrHvP7qXGjzU4dvkYsxrNot72KHjjFShUCBYt0paAUjbTQuDGRIRvan/DmWtnaPtbW3x9fGldovV/B/j7WyNscue2hpf++691zUHq1PaFjuOPI39QZ2odRIQVzZdT+sdV0KMHlCkDv/0G6dPbHVEpr6d9BG4uwDeAWY1m8fITL9MmtA2Tt0y+8wAfH2tI6ahR1hfriy/C/v32hI3jm83fUHFyRTKkzMC6N1dQuuso6N7dOqW1ZIkWAaXchBYCD5DCLwW/vv4rlfNXpuWclgxfN/zug9q3t6ajOHwYSpaEX391ec5bbkTdoE1oG9r+1paKQRVZX+57CtRsap3KGjwYfvhBh4gq5Ua0EHiIlP4pmdtkLg0LN+TDxR/ywaIPiDExdx5Us6Y1U2lwsHV17vvvw5UrLs2568wuyn5blol/TaRXuZ7Mu1qPTGUqw7FjVoula1e37tRWyhtpIfAgKfxSMK3BNDqW6siI9SOo9VOtuy86CwqC33+3RhONHQtFi1qjcpJYjInhq41fUWJcCf69+C+h5cYwYOA6fN9tB+XKwbZtVqFSSrkdLQQextfHl5HVRzL2lbEsO7iMZ8c/y4ajG+48KDAQhg+HNWus0UVVq1oXoe3ZkySZdp3ZRdUpVemwoAOV85Tn74tvUrtGZ2uivDFjYMECa5ioUsotaSHwQCLCuyHv8nur3xGEMt+WoeuSrlyPvH7ngWXLwtat8Nln1gVbRYrAO+84rSBcvXmVHst6UGxsMf48vplxfvWZ23kD2Qd+AQ0bwq5d0K6d1aGtlHJf8a1f+aA34DVgBxADhNzjuOpY6xvvA7rF2p4P2ODYPh0IeJD3TS5rFjvD+evnTZs5bQx9McFfBps5u+aYmJiYuw88ccKYdu2MCQy01gCuU8eYX34x5saNh37P65HXzYh1I0zWYVkNfTEtPnnSnMqc0lpn+ZVXjNm82QmfTCnlbCSwZnFiC0EhoCCwMqFCAPgC+4H8QACwFSjs2DcDaOy4/zXQ7kHeVwvB3ZbuX2oKjipo6IspO7GsWbRvUfwF4eRJY3r1MiZbNus/f/r0xrz+ujHjxhmzfbsxEREJvsfpy6fMoLmfmJyfZTD0xVRqG2j+yI0xKVIY07q1MWFhSfgJlVKJlVAhcMpSlSKyEvjIGHPX+pEi8gLQ1xhTzfG4u2PXYCAcyG6MiYp73L0kt6UqnSUqJopv//qWviv7cuLKCQpnKcy7Jd+lYeGG5Egb5xx9VJR1umjqVFi8GE6csLb7+UG+fNbkb489xnWJYlHqE8xId5RZOS9y0w8qH4Ce6/2p+FQNa3RSnTqQMaPrP7BS6qEktFSlKwpBQ6C6MaaN43Ez4HmgL7DeGFPAsT0PsMAYUzSB92gLtAV4/PHHSx4+fDjRuZOriKgIpu+Yzsj1I/nr5F9WP0KeMpTPW56yj5elcJbC5E6XGz8fx4XlxsDu3cRs3MCxXZvYe2I7GznGmpThrM50mSt+MWSM8qdxdCHez1abwmXrQbFiVke0UspjJFQI7jvFhIgsBeKbDKanMWaOM8I9CGPMeGA8WC0CV72vJwr0C6T5M81p/kxzdpzewc///My8vfMYsnYI0b9bS136+/iTMWVGUvmnwtfHl8sRlzl/4zw3A29CkPU6hTIXomne8tQvVJ+KQRXx9/W370MppZLMfQuBMaZKIt/jGJAn1uPcjm1ngfQi4meMiYq1XTlRkaxFKJK1CH0r9OXqzauEHQ9j77m97D+3n3PXz3Et6hrRMdGkDUjLYykeI3+G/ARnDKZYtmJkSZ3F7vhKKRdwxaRzm4BgEcmH9UXfGHjDGGNEZAXQEJgGtABc1sLwRqkDUlM+qDzlg8rbHUUp5UYSNcBbROqLyFHgBWCeiCxybM8pIvMBHL/22wOLgJ3ADGPMDsdLdAU+EJF9QCZgYmLyKKWUenhO6Sx2NR01pJRSDy+hzmK95FMppbycFgKllPJyWgiUUsrLaSFQSikvp4VAKaW8nBYCpZTych45fFREwoGkmGwoM3AmCV7XVTw9P3j+Z/D0/OD5n8HT80PSfYa8xpi7pgzwyEKQVEQkLL4xtp7C0/OD538GT88Pnv8ZPD0/uP4z6KkhpZTycloIlFLKy2khuNN4uwMkkqfnB8//DJ6eHzz/M3h6fnDxZ9A+AqWU8nLaIlBKKS+nhUAppbycFoI4RGSAiGwTkS0islhEctqd6WGIyDAR2eX4DLNFJL3dmR6WiLwmIjtEJEZEPGYYoIhUF5HdIrJPRLrZnedhici3InJaRLbbneVRiEgeEVkhIv84/v10sjvTwxCRFCKyUUS2OvL3c9l7ax/BnUQknTHmkuN+R6CwMeZdm2M9MBF5GVhujIkSkSEAxpiuNsd6KCJSCIgBxgEfGWPcfvEJEfEF9gBVgaNYK/M1Mcb8Y2uwhyAiLwFXgO+NMUXtzvOwRCQHkMMY86eIpAU2A/U85b+BiAiQ2hhzRUT8gd+BTsaY9Un93toiiONWEXBIDXhUpTTGLHasCgewHmstaI9ijNlpjNltd46HVArYZ4w5YIy5ibX8al2bMz0UY8xq4JzdOR6VMeaEMeZPx/3LWCsi5rI31YMzliuOh/6Om0u+f7QQxENEBorIEeBN4FO78yRCa2CB3SG8RC7gSKzHR/GgL6HkRkSCgBLABpujPBQR8RWRLcBpYIkxxiX5vbIQiMhSEdkez60ugDGmpzEmD/Aj1nrLbuV++R3H9ASisD6D23mQz6DUoxCRNMAsoHOcFr7bM8ZEG2OKY7XkS4mIS07R+bniTdyNMabKAx76IzAf6JOEcR7a/fKLSEugFlDZuGkn0EP8N/AUx4A8sR7ndmxTLuQ4tz4L+NEY84vdeR6VMeaCiKwAqgNJ3nnvlS2CexGR4FgP6wK77MryKESkOvAJUMcYc83uPF5kExAsIvlEJABoDITanMmrODpbJwI7jTHD7c7zsEQky61RfiKSEmvggUu+f3TUUBwiMgsoiDVq5TDwrjHGY37Zicg+IBA469i03pNGPQGISH1gFJAFuABsMcZUszXUAxCRmsBIwBf41hgz0N5ED0dEpgIVsKZAPgX0McZMtDXUQxCRF4E1wN9Y//8F6GGMmW9fqgcnIsWAyVj/fnyAGcaY/i55by0ESinl3fTUkFJKeTktBEop5eW0ECillJfTQqCUUl5OC4FSSnk5LQRKKeXltBAopZSX+3+na0E96noRPAAAAABJRU5ErkJggg==\n" + }, + "metadata": { + "needs_background": "light" + } + } + ], + "source": [ + "def predict_sin2(x):\n", + " return a+b*x+c*x**2+d*x**3\n", + "import matplotlib.pyplot as plt \n", + "plt.plot(x,y,c='r')\n", + "plt.plot(x,predict_sin2(x),c='g')\n", + "plt.show()" + ] + }, + { + "cell_type": "code", + "execution_count": 25, + "metadata": {}, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "99 350.6225280761719\n", + "199 246.3155517578125\n", + "299 173.9889373779297\n", + "399 123.78739929199219\n", + "499 88.9103775024414\n", + "599 64.658447265625\n", + "699 47.779937744140625\n", + "799 36.023372650146484\n", + "899 27.827756881713867\n", + "999 22.110069274902344\n", + "1099 18.118133544921875\n", + "1199 15.329056739807129\n", + "1299 13.379064559936523\n", + "1399 12.014815330505371\n", + "1499 11.059768676757812\n", + "1599 10.390778541564941\n", + "1699 9.921894073486328\n", + "1799 9.5930814743042\n", + "1899 9.362378120422363\n", + "1999 9.200424194335938\n", + "Result: y = 0.019712112843990326 + 0.8508722186088562 x + -0.003400667803362012 x^2 + -0.0924956277012825 x^3\n" + ] + } + ], + "source": [ + "# 3 使用autograd自动梯度计算\n", + "import torch\n", + "import math\n", + "\n", + "dtype = torch.float\n", + "device = torch.device(\"cpu\")\n", + "# device = torch.device(\"cuda:0\") # Uncomment this to run on GPU\n", + "\n", + "# Create Tensors to hold input and outputs.\n", + "# By default, requires_grad=False, which indicates that we do not need to\n", + "# compute gradients with respect to these Tensors during the backward pass.\n", + "x = torch.linspace(-math.pi, math.pi, 2000, device=device, dtype=dtype)\n", + "y = torch.sin(x)\n", + "\n", + "# Create random Tensors for weights. For a third order polynomial, we need\n", + "# 4 weights: y = a + b x + c x^2 + d x^3\n", + "# Setting requires_grad=True indicates that we want to compute gradients with\n", + "# respect to these Tensors during the backward pass.\n", + "a = torch.randn((), device=device, dtype=dtype, requires_grad=True)\n", + "b = torch.randn((), device=device, dtype=dtype, requires_grad=True)\n", + "c = torch.randn((), device=device, dtype=dtype, requires_grad=True)\n", + "d = torch.randn((), device=device, dtype=dtype, requires_grad=True)\n", + "\n", + "learning_rate = 1e-6\n", + "for t in range(2000):\n", + " y_pred = a + b*x +c*x**2+d*x**3\n", + " loss = (y_pred-y).pow(2).sum()\n", + " if t%100 ==99:\n", + " print(t,loss.item())\n", + " \n", + " # 执行自动梯度计算\n", + " loss.backward()\n", + "\n", + " # 进行梯度下降\n", + " with torch.no_grad():\n", + " a -= learning_rate*a.grad \n", + " b -= learning_rate*b.grad \n", + " c -= learning_rate*c.grad \n", + " d -= learning_rate*d.grad \n", + " a.grad = None \n", + " b.grad = None\n", + " c.grad = None \n", + " d.grad = None \n", + "\n", + "print(f'Result: y = {a.item()} + {b.item()} x + {c.item()} x^2 + {d.item()} x^3')\n" + ] + }, + { + "cell_type": "code", + "execution_count": 27, + "metadata": {}, + "outputs": [ + { + "output_type": "display_data", + "data": { + "text/plain": "
", + "image/svg+xml": "\r\n\r\n\r\n\r\n \r\n \r\n \r\n \r\n 2021-03-23T15:22:15.734505\r\n image/svg+xml\r\n \r\n \r\n Matplotlib v3.3.2, https://matplotlib.org/\r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n\r\n", + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAD4CAYAAADhNOGaAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAABAFklEQVR4nO3dd3xN9x/H8dcnkYgZhFJ7r6pSEfOHIvaq0V9VW9UaHVZRteePUlt1mUWLomq01C5dRmxq7y2InZDx/f1xLoKYSe7Jzf08H4/7kPs9597zvi33k3O+3/P9ijEGpZRS7svD7gBKKaXspYVAKaXcnBYCpZRyc1oIlFLKzWkhUEopN5fE7gDPIn369CZnzpx2x1BKKZeyadOm88aYDPe3u2QhyJkzJ0FBQXbHUEoplyIiR2Nq10tDSinl5rQQKKWUm9NCoJRSbk4LgVJKuTktBEop5ebipBCIyGQROSciOx+yXURkrIgcEJHtIvJytG3NRWS/49E8LvIopZR6cnF1RvAdUOMR22sC+RyP1sDXACKSDugLlAICgL4ikjaOMimllHoCcXIfgTFmrYjkfMQu9YFpxprzep2IpBGR54FKwHJjzEUAEVmOVVBmxkUupVQCdPEinDsHV65AeDgkTw6pU3Mjkx8Hrh3jUMghTlw5wfVb1wmLCCOZVzJSJ01N5lSZye+Xnzxp8+Dl6WX3p0hUnHVDWRbgeLTnJxxtD2tXSiUGoaGwZg38/jv89Rfs3g0XLgAQ7gFrc8CiAvBXNtiaCSI8H/+WPkl8KJutLK/kfIVGhRpRKEOh+P0MbsBl7iwWkdZYl5XInj27zWmUUg8VFQUrVsD06TB/Ply7Bl5e4O8PjRqxOW8Kvk6ylXnXg7gYeZVk4k0p79x0DUtP0ZNR5NlyhGy7T5HqFvjkyEPYaw25/GYTjqWKYu+FvWw+vZk1R9fQZ3Ufeq/uzUsZX6J1ida8U+wdknslt/vTuySJqxXKHJeGfjHGFIlh27fA78aYmY7ne7EuC1UCKhlj2sS038P4+/sbnWJCqQTm5k3ry3/kSOs3/zRpoFEjaNwYU748848v5/O/P2fdiXUk90pOw0INaVSoEdXyVHvwC/zUKfj1V5gzB1autNrq1YNu3aBUKQDOXDvD7F2zmbZtGptOb8IvmR8dSnWgU5lOpPBO4dzP7iJEZJMxxv+BDcaYOHkAOYGdD9lWG1gCCFAa2OBoTwccBtI6HoeBdI87VokSJYxSKoGIijJm1ixjcuUyBowpVsyY6dONCQszUVFRZtHeRab4N8UN/TB5x+Y1Y9aNMSGhIU/+/kePGtOtmzHp0lnvX7++MTt3Rjt8lFl7ZK2pM6OOoR8m84jM5rst35moqKg4/6iuDggyMX1Hx9T4tA+szt3TQDjWdf73gPeB9x3bBfgSOAjsAPyjvfZd4IDj0eJJjqeFQKkEYscOY0qXtr5KXnzRmCVLrMJgjNl7fq+pNr2aoR8mz5g8ZtrWaSY8MvzZj3XlijEDBxqTOrUxnp7GdO1qzLVr9+zyx9E/TMCEAEM/TOC0QHP00tHYfLpEJ14LgbMfWgiUstnNm8b072+Ml5cx6dMbM3myMRERxhhjwsLDTK+VvYz3QG+T+rPUZsy6MeZWxK24O/b588a0bGl9feXMaczKlfdsjoyKNF9v/NqkGJTCpBqcyszcMTPuju3itBAopeLGkSPGBARYXx9Nmxpz7tydTTvO7jAvff2SoR/mrXlvmTNXz8RfjrVrjSlQwBgRY3r0MObWvcXmcMhhU25SOUM/TMclHeO2GLmohxUCnWJCKfXkfvkFiheHPXtg7lyYMQMyZMAYw6h/RlFifAlOXzvNoqaLmPbqNDKmzBh/Wf7zH9i0Cd59FwYPhooV4fTpO5tzpsnJ6uar6VCqA6PXj6ba99W4FHYp/vK4MC0ESqnHMwYGDYK6dSFnTti82RoRBFy9eZXGcxrTaVknauStwY4PdlAnfx3n5EqRAiZOhFmzYPt2KFnSyubg5enF6Bqjmf7qdP469hcVv6vIqaunnJPNhWghUEo9Wng4vPce9OoFzZrB339DnjwA7D2/l4CJASzYs4AR1UYw/7/zeS7Fc87P+N//WjeseXpC+fLw00/3bH6z6Jv8+savHLx4kLKTynIo5JDzMyZgWgiUUg937RrUqgVTpkDfvtZ9Aj4+ACw/uJyAiQGcv3Ge5W8tp1OZToiIfVlfegk2bIBixaBJExg//p7NgXkC+f2d37l66yqVp1bm2OVj9uRMgLQQKKVidvkyVK8Oq1fDd99Bv37g+KKftm0atWbUIodvDja13sQruV6xNeodGTNadzXXrAlt2sDnn9+z2T+zP8veXMalsEtUnlqZk1dO2hQ0YdFCoJR6UEgIBAZav2H/+CM0t2aIN8Yw+I/BNJ/fnAo5KvBHiz/I7pvApnxJnhx+/hlefx0+/dQqYNGUyFyC3978jbPXz1L9++pcDrtsT84ERAuBUupely9D1aqwbRvMm3enU9gYQ+dlnem5qifNXmzGkmZL8PXxtTnsQ3h7w/ffWyOK+veHIUPu2Vw6a2kWvL6AvRf20nhOY8Ijw20KmjBoIVBK3XXjhjUyaMcOa8K4unUBiDJRtFvSjlHrRtEuoB3TXp2Gt6e3vVkfx9PT6id44w3o3h3GjLlnc+VclZlQdwIrDq3g/V/evz3TgVtymdlHlVLxLDzc6mT980+YOdO6zo5VBD745QPGbx5P5zKdGRY4zN5O4afh6QlTp0JYGHTsCKlTQ4sWdza/U+wdDoUcYuDagRTLVIx2pdrZl9VGekaglLKmjm7RAhYvhq+/toZjYhWB1otaM37zeLqX7+5aReC2JEmswlatGrRuDcuW3bO5X6V+1M1fl07LOvHP8X9sCmkvLQRKKes6+g8/wP/+Z422wdEnsLQzk7ZMotd/ejGo8iDXKwK3eXtbU1oXLgyNG1v9Hw4e4sHUBlPJ7pudJnOacO76ORuD2kMLgVLubsYMGDDAOiPo0eNO88C1Axm9fjQdSnVgwCsDXLcI3JY6tbXGQerU1r0RJ07c2ZQ2WVp+eu0nLoRe4M15bxJlomwM6nxaCJRyZ//8Y42sqVABvvnmzn0CY9ePpe/vfXmn2DuMrD7S9YvAbVmzWpe/rl6FBg2spTQdimUqxqjqo1h+aDnjNoyzL6MNtBAo5a6OH7e+DLNmtYaJelujgH7Y/gMdfuvAqwVfZULdCXhIIvuaKFrUGlq6aRO8/741j5JDmxJtqJ2vNp+u+JR/g/+1MaRzJbL/w0qpJ3LrljVC6MYNa0ZRPz8A1hxZQ4sFLaiUsxIzGs0giUciHVhYr541Zca0aTDu7m//IsLEehNJ6Z2SN+e9ya3IWzaGdJ44KQQiUkNE9orIARHpFsP2USKy1fHYJyKXom2LjLZtYVzkUUo9RpcusH69NYdQwYKANYHcqz++Sp50eZj32jx8kvjYHDKe9eljFYSPP4Y1a+40Z0qZiYl1J7LlzBaG/DnkEW+QeMR68XoR8QT2AYFYy1RuBJoaY2I8rxKRdkBxY8y7jufXjDEpn+aYuni9UrEwaxY0bWp9AY4cCUDw9WBKTyrN1ZtXWddyHbnT5rY5pJNcuQIBAdbd1Nu2wXN3Z05t+lNT5u2ex/b3t1MgfQEbQ8adhy1eHxdnBAHAAWPMIWPMLWAWUP8R+zfFWuNYKeVsu3dDy5ZQrhwMHQpAWEQYDX5swKmrp1jYdKH7FAGwRhDNnm3NrfT229b9FA6jq48muVdy2vzSJtHfdRwXhSALcDza8xOOtgeISA4gF7AqWrOPiASJyDoRafCwg4hIa8d+QcHBwXEQWyk3ExZmTcSWPLn15eflhTGG9395n7+P/830V6dTOmtpu1M6X9GiMHo0LF0KI0bcac6YMiPDAoex5ugapmydYl8+J3B2Z/HrwFxjTGS0thyOU5U3gNEikiemFxpjxhtj/I0x/hkyZHBGVqUSlx49rFW8vvsOMmcGYNyGcUzdNpW+FfvSuHBje/PZqU0b60azHj1g3bo7ze8Wf5cKOSrQZVkXzt84b2PA+BUXheAkkC3a86yOtpi8zn2XhYwxJx1/HgJ+B4rHQSalVHTLlsGoUdC2rXUzFdYIoY+Xfky9AvXoU7GPzQFtJgITJlhDaZs2tfoOsO46/rr211y5eYU+qxPvf6O4KAQbgXwikktEvLG+7B8Y/SMiBYG0wD/R2tKKSFLHz+mBcoD7DN5VyhmCg631BAoXvrNQy/HLx2kypwl50+VlWoNpie9egWeRJo11l/WxY1ZHukPhDIX5sOSHfLvpW3ac3WFfvngU6//7xpgIoC2wFNgNzDbG7BKRASJSL9qurwOzzL29LoWAIBHZBqwGhjxstJFS6hkYY3UOX7xofcklS0ZoeCgNZzckLCKM+a/PT7hrCtihTBlrMZvJk2HRojvN/Sr1I41PGjou7Zg4O46NMS73KFGihFFKPYHJk40BY0aMuNPUemFrQz/M/N3zbQyWgIWFGVO0qDEZMxoTHHyn+Yv1Xxj6YX7e/bN92WIJCDIxfKfq+aBSidXJk9Yljv/8x5qLH5i5YybjN4+nW7lu1C/4qFHebixpUpg+3TqL+vDDO1NQvO//PoUzFKbLsi6JbkUzLQRKJUbGWPPo3LwJkyaBhwf7Luyj9S+tKZetHAMrD7Q7YcJWtKg1NfecOdYNeEASjyQMqTKEgyEHE91wUi0ESiVGP/xgzSE0aBDky0dYRBivzXkNb09vZjaamXjnEIpLn3wCpUtDu3ZWhztQJ38dymQtQ/81/QkND33MG7gOLQRKJTZnzkD79lbHZ4cOAHz828dsO7uNaQ2mkc0322PeQAHWymYTJ1pDSTt1AqxJ6T6r8hmnrp7iy41f2hww7mghUCqx+egja1bRyZPB05PZu2bzzaZv+KTsJ9TOX9vudK7lhReshe+//9668xiomLMi1fNU57M/P+Ny2GWbA8YNLQRKJSYLFlhrC/TrBwULcvTSUVotakXprKUZVHmQ3elcU48e1gytbdrAtWsADKo8iIuhFxm1bpTN4eKGFgKlEovr161LQi+8AJ07ExkVydvz38YYw4yGM/Dy9LI7oWtKmtS66/joUWvqaqBE5hI0KNiAMevHcOXmFZsDxp4WAqUSiwEDrLtiv/kGvLwY8c8I1h5dyxc1vyBX2lx2p3Nt5ctbo7DGjIGNGwHo+Z+eXAq7xFcbv7I5XOxpIVAqMdixw1pb4L33oHx5tpzeQq9VvWhcuDFvv/S23ekShyFDIGNG696CyEj8M/tTI28NRv4zkhvhN+xOFytaCJRydVFR8MEH4OsLQ4cSGh5Ks3nNyJAiA9/U/ibxLDxvN19fGD4cgoKs0URAr//0IvhGMBM2TbA5XOxoIVDK1U2ZAn/9BcOGgZ8fn674lN3nd/Nd/e/wS+5nd7rEpWlTqFjR6kA+f55y2ctRKWclPv/7c25G3LQ73TPTQqCUK7twAbp2taaRaN6cpQeW8sWGL+hQqgOBeQLtTpf4iFiL3V++bBUDrLOCU1dPMXXbVJvDPTstBEq5sj594NIl+PJLLt26wrsL36VwhsJ8VuUzu5MlXkWKWDfqTZwIGzZQOVdlSjxfglHrRhFloh7/+gRIC4FSrmr7dmuE0AcfwIsv0mlpJ85eO8vUBlNJ5pXM7nSJW9++VsfxRx8hUVF0KtOJPef38NuB3+xO9ky0ECjlioyxZhRNkwYGDGDJ/iVM2TqFT8t9in9mf7vTJX6pU1vrGwcFwaRJNCnchCypsjDyn5F2J3smWgiUckXz5sHq1TBgAJeTe9JqUSsKZyisS046U9OmVt9Mr154XbtB+1LtWXl4JdvObLM72VOLk0IgIjVEZK+IHBCRbjFsf0dEgkVkq+PRMtq25iKy3/FoHhd5lErUQkOhSxfrWnWbNnRe1pnT107zXf3vSJokqd3p3IeItQ70+fMwaBCtXm5FCq8ULjntRKwLgYh4Al8CNYHCQFMRKRzDrj8aY4o5HhMdr00H9AVKAQFAXxFJG9tMSiVqI0fCkSMwZgxLj6xk0pZJdC3blZJZStqdzP2UKAFvvw1jxpD2dAjvFn+XGTtmcOrqKbuTPZW4OCMIAA4YYw4ZY24Bs4AnXfqoOrDcGHPRGBMCLAdqxEEmpRKnEydg8GBo2JDLZUvQclFLCqUvRN9Kfe1O5r4GD7amrO7alQ6lOhARFcH4TePtTvVU4qIQZAGOR3t+wtF2v0Yisl1E5orI7QnRn/S1iEhrEQkSkaBgxyIRSrmd7t0hMhKGD+eT5Z9w6uopptSfgk8SH7uTua/Mma0F73/6iTw7T1I9b3UmbJ7gUstZOquzeBGQ0xhTFOu3/qe+88IYM94Y42+M8c+QIUOcB1QqwduyxZoXv2NHVnGYCZsn0LlMZ0plLWV3MtWlC2TNCp068cHLbTh19RSL9i2yO9UTi4tCcBKIvuRRVkfbHcaYC8aY2/dfTwRKPOlrlVJYw0U/+QT8/Aj9pCNtfmlDnrR56F+pv93JFEDy5PDZZ7BpE7XXh5AtdTaXmpU0LgrBRiCfiOQSEW/gdWBh9B1E5PloT+sBux0/LwWqiUhaRydxNUebUiq6pUth5Uro3ZtB28Zx4OIBvq3zrd44lpC88QaULIlnz160KdqClYdXsu/CPrtTPZFYFwJjTATQFusLfDcw2xizS0QGiEg9x27tRWSXiGwD2gPvOF57ERiIVUw2AgMcbUqp2yIjrfmEcudmZ+MKDP1rKG+/9DZVclexO5mKzsPDGk566hTv/XOTJB5J+CboG7tTPRExxtid4an5+/uboKAgu2Mo5RxTpsC77xI1ayblr41l34V97Gm7h/TJ09udTMWkQQNYtYr/fl2ZZSfWcLLTSZJ7Jbc7FQAisskY88Ct53pnsVIJ2Y0b0Ls3BATwbe6L/HPiH0ZVH6VFICEbPBiuX+eDrUm4FHaJ2btm253osbQQKJWQjRkDJ09y8n9d6bayO1VzV+XNom/anUo9SuHC0KIFFccuJF/qnEzZOsXuRI+lhUCphCo42BqJUq8e7UNmcCvylq445ir69UM8PGlxOA1rj67lwMUDdid6JC0ESiVUAwfCjRssaF+Nebvn0bdiX/Kky2N3KvUksmaFDh14e+pWPPDgu63f2Z3okbSzWKmE6OhRyJePq+80pXChVaT1Scum1pvw8vSyO5l6UiEhkCcPtZonYXs2b452PIqnh6etkbSzWClX0r8/eHjQv7oPJ6+cZHzd8VoEXE3atNC9Oy2WBXPy6klWHFphd6KH0kKgVEKzdy9MncrOtq8xetckWr7cktJZS9udSj2Ltm2pdy0L6W55MnnLZLvTPJQWAqUSmj59MMmT8VGBA6TxSaPrD7uyZMlI2m8gzbZEMn/3PC6GJsz7ZbUQKJWQbNkCs2fz/cdVWHvqH4ZUHYJfcj+7U6nYeOst3r2Yg1smgpnbf7A7TYy0ECiVkPTqxaWMvnRJ+Q+ls5bm3eLv2p1IxVaSJBT7eChFzsKM38fZnSZGWgiUSij++gsWL6Z328KcD7vAV7W+wkP0n2ii0KQJbwRn5O+wfRw+v9/uNA/Qv2VKJQTGQI8ebC7ix1dR6/nQ/0OKP1/c7lQqrnh40LTJAABmfv/Asu6200KgVEKwfDlRf6zlw9dTkSF5BgZWHmh3IhXHcjZpRbmLKfnhyCLMzZuPf4ETaSFQym7GQM+eTK6SjvURRxgWOIw0PmnsTqXimgjNXn6Hf9OGs/3bAXanuYcWAqXsNn8+F3YF0a3CLSrkqKCTyiViTRr3IUmUMGP1WAgNtTvOHXFSCESkhojsFZEDIvLABTAR6SQi/zoWr18pIjmibYsUka2Ox8L7X6tUohYVBX370r2RL5dMKF/W+lInlUvE0qfIQLX0pZiZ6xpRXyecpSxjXQhExBP4EqgJFAaaikjh+3bbAvg7Fq+fC3webVuoMaaY41EPpdzJ/Pmsv7iDiXmv0LF0R4o8V8TuRCqeNavUjuO+8OfUgXDtmt1xgLg5IwgADhhjDhljbgGzgPrRdzDGrDbG3HA8XYe1SL1S7i0qiqgB/WnXMCmZUmaib8W+didSTlCvQD2Se/owI/tl+OILu+MAcVMIsgDHoz0/4Wh7mPeAJdGe+4hIkIisE5EGcZBHKdewcCHTZDsb/W4ytOpQUiVNZXci5QQpvVNSp2A95r3kTeSwoXD5st2RnNtZLCJvAv7AsGjNORzTor4BjBaRGCdcF5HWjoIRFBwc7IS0SsUjY7gyuC/dqntSOkspmhVtZnci5USNCzUm2OsWf/hehrFj7Y4TJ4XgJJAt2vOsjrZ7iEhVoCdQzxhzZxCtMeak489DwO9AjHfRGGPGG2P8jTH+GTJkiIPYStlo0SL+l2Y7Z5NFMrbmF3oHsZupla8WyZIkY26tnDBqlO1nBXHxt28jkE9EcomIN/A6cM/oHxEpDnyLVQTORWtPKyJJHT+nB8oB/8ZBJqUSLmPYN7w7o0tDi6LNKZmlpN2JlJOl8E5BrXy1+CnbVaIuhdjeVxDrQmCMiQDaAkuB3cBsY8wuERkgIrdHAQ0DUgJz7hsmWggIEpFtwGpgiDFGC4FK3H79lU7Z/sUniQ+DA4fYnUbZpHHhxpy5eYG//1sWRo6EK1dsy5IkLt7EGLMYWHxfW59oP1d9yOv+Bl6MiwxKuQRjWPJVJ34tBcNe6UemlJnsTqRsUjtfbZJ6JmVuzRyUn/W3dVbQs6ctWdzqwuTKQyuZ++9cu2MoN3br14V8nGc/+ZJkpH3Zj+2Oo2yUKmkqauStwU8X/iCqbh0YMcK2swK3KgRj1o+h24qEN/OfchPGMO77duxND6Mafou3p7fdiZTNGhduzIkrJ1jfvqG12P04e9YrcKtCEJg7kIMhBzl48aDdUZQbOvvLLPrnOk7NpEWoXaj+41+gEr26+evi7enN3MidUMe+swK3KgTV8lQDYPmh5TYnUW7HGHou6MANbxjVfIbdaVQC4evjS2DuQObtmYfp0wcuXrTlrMCtCkF+v/xk982uhUA53aafv2Jy1mA6pKxKged1fIS6q36B+hy5dISd2X2gdm3rrODqVadmcKtCICJUy12NlYdWEhEVYXcc5SZMVBTt13YnQ5gHvdvo2YC6V90CdQFYsHcB9O1ry1mBWxUCgMA8gVy+eZmNJzfaHUW5iRmzevJ32qt89twb+KbWu+LVvTKlzESpLKVYuHchlCwJtWo5/azA7QpBlVxVEEQvDymnuHbzKl13jKREsBfvfDje7jgqgapfoD4bT23k1NVT1lnBhQvw5ZdOO77bFQK/5H74Z/Zn2cFldkdRbmDIDx9wyucWY/O2w8Mnmd1xVAJVr4A1CcOivYsgIABq1oThw522XoHbFQKwhpGuO7GOKzftu6VbJX6HQg4x/MgMmu1PRtn3B9kdRyVghTMUJnfa3Czc55h9x8lnBW5ZCKrlqUakiWT14dV2R1GJWJcZ75Ak0jC0ZHfw8bE7jkrARIT6Beqz8tBKrt26BqVKQY0aMGyYU84K3LIQlMlWhhReKfTykIo3Kw+t5Ofzf9BjayqytPnE7jjKBdQrUI+bkTfvfi/16WOdFXzzTbwf2y0LgbenN5VyVtIOYxUvIqIi6DCvFbkvQqfAPno2oJ5I+ezlSeuT1ho9BFCmDAQGWn0FN248+sWx5JaFAKzLQ/sv7ufIpSN2R1GJzNcbv2bX9cOM2JAGnzYf2R1HuYgkHkmonb82v+z7hcioSKuxTx84exYmTIjXY7ttIaiepzoAvx34zeYkKjE5f+M8fVb0oOpBqN+kNyTTkULqydXJV4cLoRfYcHKD1VC+PFSqBEOHQlhYvB3XbQtBfr/85E6bm1/3/2p3FJWI9Fndh6vh1xm90Q95/3274ygXE5gnEA/xYMmBJXcb+/SB06dh0qR4O26cFAIRqSEie0XkgIg8MM+ziCQVkR8d29eLSM5o27o72veKSPW4yPOEmamVtxYrD60kLCL+Kq1yH9vObOPboG/5cIPhhZbdIXlyuyMpF5MuWTpKZy19byGoVMk6MxgyBG7efOhrYyPWhUBEPIEvgZpAYaCpiBS+b7f3gBBjTF5gFDDU8drCWGscvwDUAL5yvJ9T1M5fm9CIUH4/8ruzDqkSKWMMHX7rQNpwT/rv8AM9G1DPqFbeWgSdCuLstbNWg4h1VnDiBEydGi/HjIszggDggDHmkDHmFjALuH+y9frA7U8wF6giIuJon2WMuWmMOQwccLxf/Jgx455FoivmqEiyJMn4dZ9eHlKxM/ffuaw5uob/LQ0nbftPIUUKuyMpF1UzX00Alh5cerexalUoXRoGD4bw8Dg/ZlwUgizA8WjPTzjaYtzHsdj9ZcDvCV8LgIi0FpEgEQkKDg5+tqSLFkGPHtbYXCCZVzKq5K7C4gOLMcY823sqt3cj/AZdlnfhpeupaHXUDz780O5IyoUVy1SMjCky3nt5SAR697ZWMduxI86P6TKdxcaY8cYYf2OMf4YMzziDY+/ecP06jBx5p6lW3locCjnE3gt74yipcjfD/hrGscvHGDv7Kp6dP9GzARUrHuJBjbw1WHpg6b3T5desCceOwcsvx/0x4+A9TgLZoj3P6miLcR8RSQL4Ahee8LVxp3BhaNLEujx08SJg9RMAenlIPZNjl48x9K+hvHbheSpc84OP9L4BFXu18tUiJCzk7jBSsM4KfH3j5XhxUQg2AvlEJJeIeGN1/i68b5+FQHPHz42BVca6FrMQeN0xqigXkA/YQHzq3dua53vUKACy+2anyHNFWHxgcbweViVOnyz/BBMVxefTTkPnzpAypd2RVCIQmNsxjHT/ksfvHAdiXQgc1/zbAkuB3cBsY8wuERkgIvUcu00C/ETkANAJ6OZ47S5gNvAv8BvwkTEmMraZHqlIEWjcGMaOta63YV0eWnt0rc5Gqp7K2qNrmb1rNp+eyE4Oz3TQtq3dkVQikTZZWspkLeO0X1DjpI/AGLPYGJPfGJPHGDPI0dbHGLPQ8XOYMaaJMSavMSbAGHMo2msHOV5XwBjjnPLXuzdcuQKjRwPW5aGIqAhWHFrhlMMr1xcZFUn7Je3J5pORrlP2Q6dOkCqV3bFUIlIrXy02n97MmWtn4v1YLtNZHKeKFoVXX4UxY+DSJcpkLYNvUl9+2feL3cmUi5iweQLbzm5jxK6sJE+ZFtq1szuSSmRq5rWGkTpjlmT3LARg3aBx+TKMGYOXpxe18tVi0b5Fdyd7UuohQkJD6LWqFxX9StB4+ib4+GNIndruWCqReSnTS2RInsEpsyS7byEoVgzq17cuD12+TIOCDTh/4zx/H//b7mQqgev7e19CwkIY+2cqJE0aaN/e7kgqEfIQD6rkrsKKQyvi/T4n9y0EYJ0VXLoEY8dSI28NvD29mb9nvt2pVAK289xOvtr4FW1yNKLoj79Dx47xNqRPqcDcgZy5doZdwbvi9TjuXQhefhnq1oVRo0h9E6rkqsL8vfP1LmMVo9vzCaVOmpqBv1y3CkCHDnbHUolY1dxVAeJ9IIt7FwKwFokOCYFx42hQsAGHQg6x89xOu1OpBOjnPT+z6vAqBuZvg9/cxdbZQJo0dsdSiVh23+zk98sf7/0EWghKlIDatWHECOplqYwgenlIPSA0PJTOyzrz4nMv0ub7PdbZQMeOdsdSbiAwdyBrjqzhVuSteDuGFgKwzgouXiTTd3MpnbU08/fOtzuRSmBG/DOCI5eOMCZfe5LMm69nA8ppquauyvXw66w7sS7ejqGFAKBkSWtCp+HDaZCrJptPb+bY5WN2p1IJxIkrJ/jsz89oVKgRr3y9xBoqqn0DykleyfkKHuLB8oPxd3lIC8FtffvChQu8uuEqAAv2LLA5kEooui7vSpSJYni292DePOtsIG1au2MpN+Hr40tAlgBWHI6/DmMtBLeVKgXVq5Nv5HcU9iuol4cUAH8c/YOZO2fySdlPyDliknU2oH0DyskCcwey4eQGLoVdipf310IQXZ8+EBxMgyuZWXNkDedvnLc7kbJRZFQk7X9rT7bU2ejmWxt++sm6JKRnA8rJquauSpSJirdldbUQRFe2LFStSpPvtxBpIvl59892J1I2mrRlElvPbGVY4DCSfzZczwaUbUpnLU0KrxTx1k+gheB+ffvy0u4Q8kl6Zv872+40yiYhoSH0XNWTCjkq8FpUIZg71zobSJfO7mjKDXl7elMxZ0VWHVkVL++vheB+5csjlSvzWlAoqw6vIvj6M66PrFxa/zX9uRh6kTE1xiADB1pTTOvZgLJR5ZyV2XN+D6eunorz99ZCEJO+fXltw3WiTBTzds+zO41ysp3ndjJuwzhavdyKYueT6NmAShBq5qtJh1IdiDJRcf7esSoEIpJORJaLyH7Hnw/0oolIMRH5R0R2ich2EflvtG3fichhEdnqeBSLTZ44U6ECLxaqSIEQT2bvmGV3GuVExhg+WvwRvj6+DKo8CAYMsM4GPv7Y7mjKzRXOUJjRNUaTNXXWOH/v2J4RdANWGmPyASsdz+93A3jbGPMCUAMYLSJpom3/xBhTzPHYGss8cUb69uO17ZH8fmwNZ6+dtTuOcpKZO2ey9uhaPqvyGX6HTsOcOdY003o2oBKx2BaC+sBUx89TgQb372CM2WeM2e/4+RRwDsgQy+PGv0qVeM2nBFEY5m3XswJ3cOXmFTov60zJzCV5r/h7ejag3EZsC0FGY8xpx89ngIyP2llEAgBv4GC05kGOS0ajRCTpI17bWkSCRCQoONg5HbgvfDqCQsEwe9UXTjmeslf/3/tz9tpZvqz1JZ47dt49G/DzszuaUvHqsYVARFaIyM4YHvWj72esSfwfOpG/iDwPTAdaGHOnt6M7UBAoCaQDPn3Y640x440x/sYY/wwZnHNCIRUr8lpobtZEHOT0mQNOOaayx85zOxmzfgytXm5FySwlrZsL06SBLl3sjqZUvHtsITDGVDXGFInhsQA46/iCv/1Ffy6m9xCR1MCvQE9jzLpo733aWG4CU4CAuPhQcen1Zp9hBGZN0OUIEytjDG0Xt8XXx5fBVQbDhg2wcKFVBHSGUeUGYntpaCHQ3PFzc+CBmdpExBv4GZhmjJl737bbRUSw+hcS3IowBSu/hv/1NEw/vdRawEYlOjN3zmTN0TVWB3FyP+jVC9Kn17WIlduIbSEYAgSKyH6gquM5IuIvIhMd+7wGVADeiWGY6A8isgPYAaQH/hfLPPHizdKt2JIxil0ju9sdRcWxKzev0GVZF/wz+1sdxGvWwPLl0L271VGslBsQV1yf19/f3wQFBTnteOeunyPzsEx8siEJn009CU7qo1Dxr/PSzoxaN4r1LddTMrM/VKwIBw7AwYOQLJnd8ZSKUyKyyRjjf3+73ln8BJ5L8RzVM/+HHwqGE/X5ULvjqDhyu4O45cstrQ7i5cvhjz+sS0NaBJQb0ULwhN4q+wHHfWHtwrFw+vTjX6AStAc6iI2Bnj0hRw547z274ynlVFoInlC9AvVI5ZWS6YUjYNAgu+OoWPphxw+sObqGwZUHkz55emuUUFCQNWw06UNvZ1EqUdJC8ISSeyWn0QuNmVvUi9DJ38LRo3ZHUs/oYuhFOi3tRKkspWhVohVERUHv3pAvH7z9tt3xlHI6LQRP4a2ib3HF4xYLCgADB9odRz2jT5d/ysXQi4yvOx4P8bDuIN6xA/r3hyRJ7I6nlNNpIXgKlXJWIodvDibVfh6++w727bM7knpKfxz9g4lbJtKpTCeKZiwK4eHW2cALL8B///v4N1AqEdJC8BQ8xIN3i7/LCq/jHM6U1OpcVC7jVuQt3v/1fXL45qBvxb5W46RJsH8/fPYZeOg/B+We9G/+U2pRrAUe4sHkVv7WgiXr19sdST2hEX+P4N/gfxlXaxwpvFPAtWvQrx/85z9Qp47d8ZSyjRaCp5TNNxs18tZgSsoDRGTMAF27WkMPVYJ28OJBBqwdQKNCjaiT3/GlP3o0nD0LQ4eCiK35lLKTFoJn0LJ4S05eO8VvnzaCtWth8WK7I6lHuL3qmJeHF2NqjLEag4Ph88/h1VehTBl7AyplMy0Ez6BO/jo8l+I5JmY6CXnzQrduEBlpdyz1ED/u+pGlB5cyqPIgsqTOYjX+739w/ToMHmxvOKUSAC0Ez8DL04t3XnqHX/Yv5nT/T2DnTpg+3e5YKgYhoSF0/K0j/pn9+bDkh1bjoUPw9dfWHcQFC9obUKkEQAvBM3rv5feINJF8l+08BARYQxBDQ+2Ope7TeVlnzt84z/g64/H08LQae/e27hfo18/WbEolFFoInlF+v/y8kvMVvtn0LRFDBsOJEzBunN2xVDTLDy5nytYpdC3XleLPF7cat2yBGTOsdYgzZ7Y3oFIJhBaCWGgb0JZjl4+xKNMVqFXLut588aLdsRRw7dY1Wi1qRQG/AvSp2MdqNMYa5ZUunfWnUgqIZSEQkXQislxE9jv+TPuQ/SKjLUqzMFp7LhFZLyIHRORHx2pmLqNegXpkS52NcRvHwZAhcPmyTkiXQPRY2YNjl48xqd4kfJL4WI2//gorVlgTy/n62htQqQQktmcE3YCVxph8wErH85iEGmOKOR71orUPBUYZY/ICIYBLzf+bxCMJH5b8kFWHV7Erowe8+y588YV1p6qyzV/H/mLchnG0DWhLuezlrMbwcOjcGQoUgA8/tDegUglMbAtBfWCq4+epWOsOPxHHOsWVgdvrGD/V6xOKli+3JKlnUr7c+KU1JNHHx1r0XNkiLCKM9xa+R440Oax1Bm776itrbqjhw8HLy76ASiVAsS0EGY0xt1dpOQNkfMh+PiISJCLrRKSBo80PuGSMiXA8PwFkiWUep0ufPD1NX2zKtG3TuJwmmTX/0MKF1iUI5XQD1gxg74W9jK8znpTeKa3GixetmUUDA6F2bXsDKpUAPbYQiMgKEdkZw6N+9P2Mtfjxw+ZayOFYJ/MNYLSI5HnaoCLS2lFMgoKDg5/25fGqXUA7rodfZ8rWKdChA+TKZY1KiYh4/ItVnNl8ejOf//U5LYq1IDBP4N0N/ftb/TcjR+pUEkrF4LGFwBhT1RhTJIbHAuCsiDwP4Pjz3EPe46Tjz0PA70Bx4AKQRkRuTwCfFTj5iBzjjTH+xhj/DAls8fiXn3+ZctnKMXrdaCK8k1iXH3buhIkT7Y7mNsIiwnj757d5LsVzjKg24u6GPXvgyy+hVSsoUsS+gEolYLG9NLQQaO74uTmw4P4dRCStiCR1/JweKAf86ziDWA00ftTrXUXXcl05evkoc3bNseavqVjRunHp0iW7o7mFPqv7sCt4F5PrTyZtsmiD17p0gRQpYMAA+8IplcDFthAMAQJFZD9Q1fEcEfEXkdu/DhcCgkRkG9YX/xBjzL+ObZ8CnUTkAFafwaRY5rFNnfx1KJi+IJ///bl1fWz0aLhwQVcyc4I/jv7B8L+H06ZEG2rkrXF3w9Kl1pDRXr3guefsC6hUAifGBadQ9vf3N0FBQXbHeMDkLZN5b+F7LHtzmXWNulUrayWznTutYYsqzl29eZWXvnkJEWHb+9vudhDfvAkvvmjdRLZzpy5IrxQgIpsc/bX30DuL41CzF5vxfMrnGfrXUKth0CDrskTbtrpmQTzpsqwLRy4dYWqDqXeLAMCIEdb9HF98oUVAqcfQQhCHkiZJSsfSHVl5eCWbTm2yLkcMGmQNJZ092+54ic6S/UsYv3k8Xcp2oXz28nc3HD1q3dPRsCHUqPHwN1BKAXppKM5dDrtM9tHZqZanGnOazLHWKQgIgNOnrREsqVPbHTFRuBh6kSJfFcEvuR8bW228O40EWAXgt9+s/97Zs9sXUqkERi8NOYmvjy/tAtox99+57Dy3Ezw9rbnvz5zRaY/jiDGGlgtbcv7GeaY1mHZvEViyBH7+2RqxpUVAqSeihSAedCrTiVTeqei/pr/VEBAArVvD2LGwbZu94RKBbzd9y897fuazKp/dnV4arA7i9u0hf37o1Mm+gEq5GC0E8SBdsnR0KNWBuf/OZfvZ7Vbj4MGQNq014VlUlL0BXdjOczv5eOnHVM9TnY/LfHzvxiFD4MAB7SBW6ilpIYgnncp0InXS1HfPCtKlg2HD4O+/YcoUe8O5qNDwUJr+1JTUSVMztcFUPCTaX99//7U65l9/HapVsy+kUi5IC0E8SZssLR1LdWTe7nlsPbPVanz7bahQwbrb9fTpR75ePajLsi7sPLeTqQ2mkjFltPkNo6KgZUtIlQrGjLEvoFIuSgtBPPq4zMf4JvWl16peVoOHB0yYAGFh1iUiFxyxZZf5e+bzVdBXdC7T+d67h8HqjP/nHxg1Su8gVuoZaCGIR2l80tC9fHd+3f8rqw+vthrz57dmw5w/H+bOfeTrleVQyCFaLGjBy8+/fO8aAwDHj0O3btYU02+9ZU9ApVycFoJ41r5Ue7KlzkaX5V2IMo5O4k6doEQJ647jCxfsDZjAhUWE0Xi2NS/hnCZz8PaMtpqpMXc737/9VqeYVuoZaSGIZ8m8kjGo8iA2n97MzB0zrcYkSWDyZGvBlI8/fvQbuLl2i9ux5cwWpr86ndxpc9+78Ycf4JdfrIn9cuWyJ6BSiYAWAidoVrQZxTMVp8eqHoRFhFmNRYtC9+4wfbo1Q6Z6wJQtU5i4ZSI9yvegTv469248ccI6oypb1loMSCn1zLQQOIGHeDAscBjHLh9j9LrRdzf07GktltKyJZw/b1u+hGjrma18uPhDquSqwoBX7ltLICoKWrSwVoCbNs26e1sp9cy0EDhJldxVqF+gPgPXDuTY5WNWY9Kk1uWNixetKat1FBEA52+cp+GPDfFL5seMRjPw9Ljvi/7rr62J/EaMgDxPveqpUuo+WgicaEyNMRhj+HhptH6BokWtG6Hmz9cbzYBbkbdoPLsxp66e4qfXfuK5FPcNB923Dz75xJpVtHVre0IqlcjEqhCISDoRWS4i+x1/po1hn1dEZGu0R5iINHBs+05EDkfbViw2eRK6HGly0KtCL+btnsdvB367u6FTJ3jlFeta98GD9gW0mTGGdovbseboGibVm0SprKXu3SE83Lopz8fHWg9aRwkpFSdie0bQDVhpjMkHrHQ8v4cxZrUxppgxphhQGbgBLIu2yye3txtjtsYyT4LXuUxn8vvlp+3itnc7jj08YOpU61r3W29Z177d0Jcbv2T85vF0K9eNZkWbPbhDr16wfr01VDRLFucHVCqRim0hqA9Mdfw8FWjwmP0bA0uMMTdieVyXlTRJUr6s9SUHQw7S7/d+dzdky3b3DtnevW3LZ5flB5fT8beO1M1fl0FVBj24w5Il8Pnn0KYNNGni/IBKJWKxLQQZjTG3J805A2R81M7A68DM+9oGich2ERklIg+dMlJEWotIkIgEBQcHxyKy/armrkrL4i0Z9vcw1p9Yf3dD06ZWp/GQIdb4eDex7cw2Gs9pTKEMhfih4Q/3TiYHcPKkdUnoxRetaSSUUnHqsSuUicgKIFMMm3oCU40xaaLtG2KMeaCfwLHteWA7kNkYEx6t7QzgDYwHDhpjBsT0+ugS8gplT+py2GWKfF2ElN4p2dJmy93FVcLCoEwZa7nFLVsgRw57g8azo5eOUmZSGTw9PPnnvX/ImjrrvTtERkKVKrBxI2zaBAUL2hNUqUTgmVcoM8ZUNcYUieGxADjr+DK//aV+7hFv9Rrw8+0i4Hjv08ZyE5gCBDztB3NVvj6+TKw7kT3n99B3dd+7G3x8YI5jicvXXoNbt+wLGc8uhl6k5g81uRF+gyXNljxYBAB69IA1a+Crr7QIKBVPYntpaCHQ3PFzc2DBI/Ztyn2XhaIVEcHqX9gZyzwupXre6rR6uRXD/h7GqsOr7m7Im9caSrphA3TsaFu++BQaHkr9WfU5GHKQBa8voMhzRR7cadYsq1/ggw+gefMHtyul4oYx5pkfgB/WaKH9wAognaPdH5gYbb+cwEnA477XrwJ2YBWA74GUT3LcEiVKmMTi6s2rpuC4gibT8Ezm7LWz927s0sUYMOarr+wJF0/CwsNMze9rGukn5sedP8a805YtxiRLZkz58sbcvOnUfEolVkCQieE79bF9BAlRYugjiG772e0ETAigYs6KLGm25G5naWQk1K8Pv/0Gy5ZB5cr2Bo0D4ZHhNJnThAV7FzCh7gRavtzywZ3Onwd/f2sY7aZNkPFxYxCUUk/imfsIVPwrmrEoY2qMYdnBZQz+I9p8+56eMGMGFCgAjRtb6/G6sIioCJrNa8aCvQsYV3NczEUgNNQqfmfOwM8/axFQygm0ECQQrUu0ptmLzei9ujcL9kTrakmdGhYtsm46q10bXHTobERUBM3nN2fOv3MYHjicjwI+enCnyEh4803rXorvv4eSJZ0fVCk3pIUggRARJtSdQMnMJWk2rxnbz26/uzF3bmsuomPHrGJw7ZptOZ/FzYibNJnThBk7ZjC48mA6l+0c846dO8O8edZkco0bOzekUm5MC0ECkswrGfNfn4+vjy/1Ztbj9NVoC9yXLw+zZ8PmzdCwocsMK71+6zp1ZtZh/p75jK0xlu7/6R7zjp9/bi0837GjLtajlJNpIUhgMqfKzILXF3D+xnmqf1+dkNCQuxvr1oUJE2D5cutO2wQ+J9GFGxeo9n01Vh1exXf1v6NdqXYx7zhmDHz6Kbz+unU2oJRyKi0ECZB/Zn/mvz6fvRf2UntGba7fun53Y4sWMGwY/Phjgp6gbt+FfZSeVJpNpzYxp8kcmhd7yH0A33xjnQW8+qq1yIyH/pVUytn0X10CVTV3VWY2msn6k+upO7Mu125F6xfo0gWGDrVuuHrjDWt65gTk9yO/U3piaS6HXWZV81U0LNQw5h2/+ca6Wax2beuzeHk5N6hSCtBCkKA1LNSQqQ2msuboGgKnB957mahrVxg+3JqO4rXXrGGXNjPGMHb9WKpNr8bzqZ5nfcv1lM1WNqYdYfDgu0Vg7lzw9nZ+YKUUoIUgwXuz6JvMaTKHTac28crUVzh55eTdjZ07w9ixsGABVK0KFy7YlvNy2GWazGlCh986UDNfTf569y9ypc314I5RUdYZTc+e0KyZda+Aj4/zAyul7tBC4AIaFmrIL2/8wsGQg5ScUJINJzfc3diunXVWsGkTlC0Lhw45Pd+6E+vwn+DP/D3zGRY4jPn/nU8anzQP7njtGjRqBCNHQtu2Vp+AXg5SynZaCFxEtTzV+Oe9f0iaJCkVplRg0uZJ3JkepFEjWLny7tQMixc7JVNoeCifLPuEcpPLcSvyFr+/8ztdynZBYlpC8vBhq1AtXAijR1tnMtoxrFSCoP8SXUiR54qwsdVGymUvR8tFLWk8pzHnb5y3NpYrZ81Wmj27dd29d2/rTt14svTAUop/W5zh/wynZfGW7PhgB+Wzl49553nzoEQJOH7cWmmsQwddb1ipBEQLgYtJnzw9y99azrDAYSzau4giXxVh6tapRJkoyJPHmp7hnXfgf/+DChVg3744Pf7e83upM6MONX6oQURUBMveXMa3db8lddLUD+58/Tq0bm2dseTJYy0uU61anOZRSsWeFgIX5CEedCnbhQ2tNpAjTQ7eWfAO5SaXY+3RtRgfH5g8GaZPh9274aWXrJu0YjnEdOe5nbzx0xsU/qowfxz7g2GBw9j14S4C8wTG/IKFC6FwYZg4Ebp1g7/+stZZUEolPDHNTZ3QH4lpPYLYioyKNFO2TDEZh2U09MOUm1TOzPt3nrkZcdOYU6eMqVvXWtMgf35jFi40Jirqid87LDzM/LjzRxM4LdDQD5NycErTdVnXB9dNiG7nzrvHLFLEmD//jINPqZSKC+h6BInbjfAbTN4ymc//+pzjV46TPnl6/vvCf6mZpwYV94SS8tPesHev1WHbtas1XUUMnbWnr55m7dG1LNq3iMX7FxMSFkJ23+y0LN6SD0t+iF9yv5gD7Nhh3eQ2YwakTAm9ellzBumoIKUSjIetRxCrQiAiTYB+QCEgwBgT47eziNQAxgCeWCuXDXG05wJmYa10tgl4yxjz2NnUtBA8XERUBEsPLGXqtqks2reIsIgwkngkoUC6/Lxw1Yfsmw6Q/vQVUqTJQHgpf0KLFeGExzWOXT7GtrPbOHHlBGD1RdTOV5umRZpSNXdVPD08HzzYpUvWFNnjx8Off0Ly5NC+vXWfgN9DCoZSyjbxVQgKAVHAt0CXmAqBiHgC+4BA4ASwEWhqjPlXRGYD84wxs0TkG2CbMebrxx1XC8GTCYsI469jf7Hq8Cp2nNvBruBdnLxykpuRN+/ZL90tT7J5pqOwX0ECcpWn9Eu1KJmtzL1f/lFR1mIxu3dbHdJr18Lq1dZcR3nyWHcJt2gB6dI5+VMqpZ5UvBSCaG/+Ow8vBGWAfsaY6o7nt+chHgIEA5mMMRH37/coWgienTGGG+E3CI0IxevEKZL+tACfFb9bnbk3HQXC0xPSpAFfX2s6iJs3rXsUok99XbiwNUy1YUMICNB7ApRyAQ8rBEmccOwswPFoz08ApbAuB10yxkREa8/ysDcRkdZAa4Ds2bPHT1I3ICKk8E5BCu8UkD89dC8K3XtbcxXt2GH9xr9/P4SEWJd+RKwpIPz8IFcua+SPv79VKJRSicJjC4GIrAAyxbCppzFmQQzt8cIYMx4YD9YZgbOO6zaSJbN+sw8IsDuJUsrJHlsIjDFVY3mMk0C2aM+zOtouAGlEJInjrOB2u1JKKSdyxoXdjUA+EcklIt7A68BCx5jW1cDtxWmbA047w1BKKWWJVSEQkVdF5ARQBvhVRJY62jOLyGIAx2/7bYGlwG5gtjFml+MtPgU6icgBrD6DSbHJo5RS6unpDWVKKeUmHjZqSMf8KaWUm9NCoJRSbk4LgVJKuTktBEop5eZcsrNYRIKBo/Hw1umB8/Hwvs7i6vnB9T+Dq+cH1/8Mrp4f4u8z5DDGZLi/0SULQXwRkaCYetRdhavnB9f/DK6eH1z/M7h6fnD+Z9BLQ0op5ea0ECillJvTQnCv8XYHiCVXzw+u/xlcPT+4/mdw9fzg5M+gfQRKKeXm9IxAKaXcnBYCpZRyc1oI7iMiA0Vku4hsFZFlIpLZ7kxPQ0SGicgex2f4WUTS2J3paYlIExHZJSJRIuIywwBFpIaI7BWRAyLSze48T0tEJovIORHZaXeWZyEi2URktYj86/j708HuTE9DRHxEZIOIbHPk7++0Y2sfwb1EJLUx5orj5/ZAYWPM+zbHemIiUg1Y5VgHeiiAMeZTm2M9FREpBEQB3/KQtbATGhHxBPYBgVjLrm4Emhpj/rU12FMQkQrANWCaMaaI3Xmelog8DzxvjNksIqmATUADV/l/ICICpDDGXBMRL+BPoIMxZl18H1vPCO5zuwg4pABcqlIaY5ZFWwd6HdbKby7FGLPbGLPX7hxPKQA4YIw5ZIy5BcwC6tuc6akYY9YCF+3O8ayMMaeNMZsdP1/FWv/koeugJzTGcs3x1MvxcMr3jxaCGIjIIBE5DjQD+tidJxbeBZbYHcJNZAGOR3t+Ahf6EkpsRCQnUBxYb3OUpyIiniKyFTgHLDfGOCW/WxYCEVkhIjtjeNQHMMb0NMZkA37AWl0tQXlcfsc+PYEIrM+Q4DzJZ1DqWYhISuAnoON9Z/gJnjEm0hhTDOtMPkBEnHKJ7rGL1ydGxpiqT7jrD8BioG88xnlqj8svIu8AdYAqJoF2Aj3F/wNXcRLIFu15VkebciLHtfWfgB+MMfPszvOsjDGXRGQ1UAOI9857tzwjeBQRyRftaX1gj11ZnoWI1AC6AvWMMTfszuNGNgL5RCSXiHgDrwMLbc7kVhydrZOA3caYkXbneVoikuH2KD8RSYY18MAp3z86aug+IvITUABr1MpR4H1jjMv8ZiciB4CkwAVH0zpXGvUEICKvAl8AGYBLwFZjTHVbQz0BEakFjAY8gcnGmEH2Jno6IjITqIQ1BfJZoK8xZpKtoZ6CiJQH/gB2YP37BehhjFlsX6onJyJFgalYf388gNnGmAFOObYWAqWUcm96aUgppdycFgKllHJzWgiUUsrNaSFQSik3p4VAKaXcnBYCpZRyc1oIlFLKzf0fugfkY8xkxCAAAAAASUVORK5CYII=\n" + }, + "metadata": { + "needs_background": "light" + } + } + ], + "source": [ + "\n", + "def predict_sin3(x):\n", + " return a+b*x+c*x**2+d*x**3\n", + "\n", + "import matplotlib.pyplot as plt \n", + "plt.plot(x,y,c='r')\n", + "plt.plot(x,(predict_sin3(x).detach().numpy()),c='g')\n", + "plt.show()" + ] + }, + { + "cell_type": "code", + "execution_count": 33, + "metadata": {}, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "99 209.95834350585938\n", + "199 144.66018676757812\n", + "299 100.70249938964844\n", + "399 71.03519439697266\n", + "499 50.97850799560547\n", + "599 37.403133392333984\n", + "699 28.206867218017578\n", + "799 21.973188400268555\n", + "899 17.7457275390625\n", + "999 14.877889633178711\n", + "1099 12.931766510009766\n", + "1199 11.610918045043945\n", + "1299 10.714258193969727\n", + "1399 10.10548210144043\n", + "1499 9.692106246948242\n", + "1599 9.411375045776367\n", + "1699 9.220745086669922\n", + "1799 9.091285705566406\n", + "1899 9.003361701965332\n", + "1999 8.943639755249023\n", + "Result: y = -5.423830273798558e-09 + -2.208526849746704 * P3(1.3320399228078372e-09 + 0.2554861009120941 x)\n" + ] + } + ], + "source": [ + "# 4 重新定义autograd函数，执行梯度计算\n", + "# 将模型定义为y = a + b P[3](c + dx)而不是y = a + bx + cx ^ 2 + dx ^ 3，其中P[3](x) = 1/2 (5x ^ 3 - 3x)是三次的勒让德多项式。\n", + "import torch\n", + "import math \n", + "\n", + "class LegendrePolynomial3(torch.autograd.Function):\n", + " @staticmethod \n", + " def forward(ctx,input):\n", + " ctx.save_for_backward(input)\n", + " return 0.5 * (5*input**3 -3*input)\n", + "\n", + " @staticmethod \n", + " def backward(ctx,grad_output):\n", + " input, = ctx.saved_tensors\n", + " return grad_output*1.5*(5*input**2-1)\n", + " \n", + "\n", + "dtype = torch.float\n", + "device = torch.device(\"cpu\") \n", + "\n", + "x = torch.linspace(-math.pi, math.pi, 2000, device=device, dtype=dtype)\n", + "y = torch.sin(x)\n", + "\n", + "# Create random Tensors for weights. For this example, we need\n", + "# 4 weights: y = a + b * P3(c + d * x), these weights need to be initialized\n", + "# not too far from the correct result to ensure convergence.\n", + "# Setting requires_grad=True indicates that we want to compute gradients with\n", + "# respect to these Tensors during the backward pass.\n", + "a = torch.full((), 0.0, device=device, dtype=dtype, requires_grad=True)\n", + "b = torch.full((), -1.0, device=device, dtype=dtype, requires_grad=True)\n", + "c = torch.full((), 0.0, device=device, dtype=dtype, requires_grad=True)\n", + "d = torch.full((), 0.3, device=device, dtype=dtype, requires_grad=True)\n", + "\n", + "learning_rate = 5e-6\n", + "for t in range(2000):\n", + " # To apply our Function, we use Function.apply method. We alias this as 'P3'.\n", + " P3 = LegendrePolynomial3.apply\n", + "\n", + " # Forward pass: compute predicted y using operations; we compute\n", + " # P3 using our custom autograd operation.\n", + " y_pred = a + b * P3(c + d * x)\n", + "\n", + " # Compute and print loss\n", + " loss = (y_pred - y).pow(2).sum()\n", + " if t % 100 == 99:\n", + " print(t, loss.item())\n", + "\n", + " # Use autograd to compute the backward pass.\n", + " loss.backward()\n", + "\n", + " # Update weights using gradient descent\n", + " with torch.no_grad():\n", + " a -= learning_rate * a.grad\n", + " b -= learning_rate * b.grad\n", + " c -= learning_rate * c.grad\n", + " d -= learning_rate * d.grad\n", + "\n", + " # Manually zero the gradients after updating weights\n", + " a.grad = None\n", + " b.grad = None\n", + " c.grad = None\n", + " d.grad = None\n", + "\n", + "print(f'Result: y = {a.item()} + {b.item()} * P3({c.item()} + {d.item()} x)')" + ] + }, + { + "cell_type": "code", + "execution_count": 34, + "metadata": {}, + "outputs": [ + { + "output_type": "display_data", + "data": { + "text/plain": "
", + "image/svg+xml": "\r\n\r\n\r\n\r\n \r\n \r\n \r\n \r\n 2021-03-23T15:23:32.852120\r\n image/svg+xml\r\n \r\n \r\n Matplotlib v3.3.2, https://matplotlib.org/\r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n\r\n", + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXIAAAD4CAYAAADxeG0DAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAA3kklEQVR4nO3dd3hUxdvG8e+k0DuEXkILTZpEem9SBWmC2AWsgAICioqCFAUpggVQBEUpghTpPRRBCL0Teu/NQCBl5/1j4NWftITs7uxuns917QVJNufcS3lyds4zM0prjRBCCO/lZzuAEEKIxJFCLoQQXk4KuRBCeDkp5EII4eWkkAshhJcLsHHSLFmy6ODgYBunFkIIr7Vp06YLWuug/37eSiEPDg4mPDzcxqmFEMJrKaWO3uvzMrQihBBeTgq5EEJ4OSnkQgjh5aSQCyGEl5NCLoQQXk4KuRBCeDkp5EII4eWs9JGLxImKiWL72e0cu3qMU3+f4nrMdQAC/QLJkTYHOdPm5LGsj5E1dVbLSYUQ7iCF3As4tIONJzcya+8sFh5cyI6zO4jTcQ/9vlxpc1E9X3UaFW5Ew0INyZwqsxvSCiHcTQq5B7ty8woTtk7g641fc+DSAQL8AqiWtxq9q/amXI5yFMpUiJxpc5I2eVoAbsbe5EzkGY5fPc62s9sIPxXOssPLmLxzMgF+ATQNacqrZV+lQaEG+Pv5W351QghnUTZ2CAoNDdUyRf/+rkdfZ+RfI/l87edcu3WNKnmq0KlcJ5qGNCVjyowJOpZDO9h0ahPTdk1j4raJnL9xnpDMIbxf9X3al2xPoH+gi16FEMLZlFKbtNahd31eCrnn0FozY88MuizowunI0zxV5Cn61ujL4zked8rxo+OimbV3FoPWDGLrma3kz5CfL+p9QctiLVFKOeUcQgjXuV8hl64VD3E28iwtp7Wk9W+tyZE2B2teXsPstrOdVsQBkvkno02JNmzutJk/2v1BmmRpaP1ba2r/VJud53Y67TxCCPeSQu4B1hxbQ9kxZZkfMZ/BdQbzV4e/qJK3isvOp5SiSUgTNr+2mW8afcP2s9t5fMzjDFo9iFhHrMvOK4RwDSnkln3111fUnFCT1MlSs6HjBnpV7UWAn3vuQQf4BfDGE2+w9629NC/anA+Wf0DlHyqz/+J+t5xfCOEcUsgtcWgH3Rd1p+vCrjQJaUJ4x3BKZStlJUtQ6iCmtZ7G1FZTOXj5IKFjQ5mxe4aVLEKIhJNCbkFMXAzP/f4cw9YPo3P5zsxoM4P0KdLbjkWbEm3Y8toWigUVo9Vvrei2qBsxcTG2YwkhHkIKuZvFOmJpN6Mdk3dOZnCdwYxsMNKjerrzps/LqpdW8fYTbzN8/XCaTm7KtVvXbMcSQjyAFHI3inPE8fzM55mxZwbD6g+jV9VeHtn2lzwgOaMajWJc03EsPbSUaj9W48S1E7ZjCSHuQwq5m2iteX3u60zZOYXBdQbzbqV3bUd6qA6Pd2B++/kcvnyYCt9XkBZFITyUFHI3GbxmMN9v+Z4Pqn5Ar6q9bMeJt/oF67P2lbUA1JhQg82nN1tOJIT4LynkbjB151Q+WP4B7R5rx2e1P7MdJ8FKZivJ6pdXky55OmpPrM264+tsRxJC/IsUchfbfHozL856kap5qzK+2XiPHBOPjwIZC7DqpVVkTZ2Vej/XY9XRVbYjCSFuk0LuQpejLtNqWiuCUgfxe5vfSRGQwnakRMmTPg+rXl5F3vR5afxrYzac3GA7khACKeQu49AOXpr9EsevHWdaq2kEpQ6yHckpsqfJztIXlpI1dVYaTGrAjrM7bEcSIslLdCFXSuVRSq1QSu1WSu1SSnV1RjBvN2zdMObsm8PQekOplKeS7ThOlTNtTpY+v5RUgamo93M9mdIvhGXOuCKPBbprrYsDFYG3lFLFnXBcr7X97HY+WPYBLYq1oEuFLrbjuET+jPlZ+sJS4nQc9X+uz5nIM7YjCZFkJbqQa61Pa6033/7938AeIFdij+utbsXe4vmZz5MpZSbGNBnjtTc346NolqIsbL+Q8zfO0+TXJkRGR9qOJESS5NQxcqVUMFAW+OseX+uklApXSoWfP3/emaf1KJ+s/ITtZ7czruk4sqTKYjuOy5XLWY6praay5cwW2s1oJ8vgCmGB0wq5UioNMAN4R2t91+IcWuuxWutQrXVoUJBv3Pj7rw0nN/DFn1/watlXaVqkqe04btMkpAmjGo5i7v65dF3QFRu7TgmRlDll4WulVCCmiP+itf7dGcf0NrGOWF6b+xrZ02Rn2JPDbMdxuzefeJPDlw8zdN1QimQp4rP3BoTwRIku5MoMAv8A7NFaJ70Kdtuov0ax9cxWpreeTrrk6WzHseLzep8TcSmCbou6UTJrSWrlr2U7khBJgjOGVqoAzwO1lVJbbz8aOeG4XuP41eN8tOIjGhVuRItiLWzHscZP+fHT0z8RkjmENtPbcPTKUduRhEgSnNG1skZrrbTWpbTWZW4/5jsjnLfourArDu1gdMPRPt2lEh/pkqdjVttZxMTF8PTUp7kRc8N2JCF8nszsTKRlh5Yxc+9MPqr+Efkz5rcdxyOEZA7h15a/svXMVjrM6SA3P4VwMSnkiRDniKP74u7kS5/PK9YXd6dGhRsxoPYAJu+czNcbv7YdRwifJoU8ESZum8i2s9v4vO7nXr8gliv0qtqLJiFN6L64O+Gnwm3HEcJnSSF/RJHRkXy4/EMq5q5ImxJtbMfxSH7KjwnNJpAtdTba/NaGKzev2I4khE+SQv6Ihv45lNORpxn+5PAkf4PzQTKnyszUVlM5fu24jJcL4SJSyB/BhRsX+HLdl7Qu3pqKuSvajuPxKuWpxKA6g5ixZ4aMlwvhAlLIH8GQtUO4EXODT2t+ajuK1+hWqdv/j5dvPbPVdhwhfIoU8gQ6G3mW0RtH82zJZykWVMx2HK9xZ7w8c8rMtP+9PVExUbYjCeEzpJAn0OA1g7kVe4uPq39sO4rXyZwqMxOaT2D3+d30WtrLdhwhfIYU8gQ4ee0k34Z/ywulX6Bw5sK243il+gXr07VCV0ZtGMXCAwttxxHCJ0ghT4DBawYTp+P4qPpHtqN4tUF1BlEiqAQvz36ZCzcu2I4jhNeTQh5P56+f54ctP/B8qedlKn4ipQxMyS8tfuFS1CU6/tFRWhKFSCQp5PE0asMobsbepGeVnraj+ITS2UszsPZAZu2dxYStE2zHEcKrSSGPh8joSEZvGE2zos0omqWo7Tg+491K71I9X3XeXfQuJ66dsB1HCK8lhTwexm0ax+Wbl+lVRTotnMlP+TH+qfFEx0Xz2tzXZIhFiEckhfwhouOiGbZ+GDXy1ZBZnC5QMFNBBtcdzPyI+fy07SfbcYTwSlLIH2LyjsmcuHaC3lV7247is94u/zZV81al68KunLx20nYcIbyOFPIH0Foz8q+RPJb1MZ4s+KTtOD7rzhDLrbhbMsQixCOQQv4Aa4+vZcuZLXQu31lWOHSxwpkLM7D2QOZFzGPS9km24wjhVaSQP8BXf31FxhQZaV+yve0oSUKXCl2onKcyXRd25UzkGdtxhPAaUsjv4/jV4/y+53c6PN6B1MlS246TJPj7+TP+qfFcj7nOOwvfsR1HCK8hhfw+vgv/Do3mzSfetB0lSSmSpQh9qvVh6q6pLIhYYDuOEF5BCvk9RMVEMWbTGJ4q8hTBGYJtx0lyelXpRbEsxXhj3htcj75uO44QHk8K+T1M3TWVi1EX6Vy+s+0oSVLygOSMbTqWo1eP8snKT2zHEcLjSSG/h7GbxlI0S1FqBdeyHSXJqpq3Kh0f78jw9cPZcnqL7ThCeDQp5P+x69wu1p1YR4eyHaTl0LLP635OllRZ6DS3E3GOONtxhPBYUsj/Y9zmcQT6BfJC6RdsR0nyMqbMyMgGIwk/Fc7oDaNtxxHCY0kh/5ebsTf5efvPPF3saYJSB9mOI4A2JdrQsFBD+izvw/Grx23HEcIjSSH/l9/3/G42O3i8o+0o4jalFN80/gaHdtBtcTfbcYTwSFLI/2Xc5nHkz5Cf2vlr244i/iU4QzB9qvVh+u7pLD642HYcITyOUwq5Umq8UuqcUmqnM45nQ8TFCFYeWUnHxzvip+Tnm6fpUbkHhTMVpvOCztyKvWU7jhAexVkVawLQwEnHsuLHrT/ir/x5qcxLtqOIe0gekJxRDUex/+J+hq0bZjuOEB4lwBkH0VqvUkoFO+NYNji0g0nbJ1G/YH1ypM1hO464jycLPUnLYi3pv6o/7Uu1J2/6vLYjJS1aw9WrcPEiREVBXJx5aA0pU0LatP88/P1tp01SnFLIvV3YkTCOXzvOF/W+sB1FPMSwJ4ex4MAC3l30LjPazLAdx/fcvAk7d8LWrXDgABw6BAcPwvHjcOmSKdwP4+cHOXNC7tyQJw8UKgSPPWYeRYpA8uQufxme6Pz183y84mPeLv82JbKWcOqx3VbIlVKdgE4AefN61pXUz9t/Jm2ytDQr0sx2FPEQedPn5aPqH/H+svdZeGAhDQp59YiefadPw4oV5rFxI+zaBbGx5muBgZA/PxQoAE88AVmyQKZMkDkzpE5tCra/PygFN27A33+bx6VLcPKkKf7btsHMmf8cMyAAypaFqlX/eWTNau/1u9Ha42v5btN3PFfqOacfWzlrN5bbQytztdaPPey5oaGhOjw83CnnTawbMTfINjQbrYu3Znyz8bbjiHiIjoum1LeliNNx7HxjJ8kDkuYV3iOJi4O1a01xXbQI9uwxn0+fHipWhHLlTKEtWxaCg50zRBIdDRER/1zpr1sHf/1lrv4BHn8cGjeGJk0gNNT8gPBBPRb3YPSG0VztffWR/80qpTZprUP/+/kkP7Qye+9sIqMjZSanF0nmn4xRDUdRf1J9hv45lD7V+9iO5Nni4mD5cpg6FebMgfPnzfBGrVrw8stQuzaUKeO6ce1kyaBECfN45hnzueho2LzZvBOYNw8GDID+/SF7dmjbFp591hR1H1omY82xNTyR6wnXXHhorRP9ACYDp4EY4ATw6oOeX65cOe0pGk5qqPMOz6vjHHG2o4gEaj2ttU75WUp95PIR21E8U0SE1h9+qHWePFqD1mnTat22rdbTpml97ZrtdP/rwgWtJ03S+umntU6WzOQNCdG6Xz+tT52ynS7Rrkdf1wH9AnTvJb0TdRwgXN+jpjrlPYzWup3WOofWOlBrnVtr/YMzjutqZyLPsPjgYtqXbC+9417oy/pfAtBzaU/LSTyIwwFz50LdulC4MAwcaK6Ep06Fc+dg8mRo3dp0lniSzJmhfXv4/Xc4cwa+/x5y5YKPP4a8eaFNG1i50nTIeKENJzcQ64ilat6qLjl+kq5eU3ZOIU7H8Xyp521HEY8gT/o89K7am2m7prHq6CrbceyKjITRo01XSNOmsG8ffPYZHDsGCxaYQpgihe2U8ZMxI7z6qhkOioiArl1h6VIzFFSyJPz0E8TE2E6ZIGuPrQWgcp7KLjl+ki7kU3dNpUz2MhQLKmY7inhEPSr3IG/6vHRd2DVpLnV77ZoZX86XDzp3Np0lU6aYtsE+fcxVrTcrVAiGDjVdMOPHmxuhL75o3m2MHm26ZbzAmuNreCzrY2RMmdElx0+yhfzolaOsP7GeZ0o8YzuKSIRUgakYUm8IW89sZfyWJNR1dPWquTkYHAwffgiVK5tukHXrzA3FwEDbCZ0rZUpzY3bbNvjjD/MDqnNn8/pHjPinA8YDxTni+PP4n1TN45phFUjChfy33b8BZplU4d1aF29NtbzV6LO8D1duXrEdx7Vu3jRXqMHBZvy4WjUIDzfFrWJF2+lcTynTprhmDYSFmaGWd981Q0oTJsRvwpKb7Ty3k2u3rrlsfByScCGftmsaoTlDKZCxgO0oIpGUUoxsMJILNy7QP6y/7TiuobUZMilWDN57DypVMu17s2eb3u+kRimoXh2WLYMlS8ykopdfhlKlzA81D7opuubYGgAp5M526PIhNp7aSJvicjXuK8rmKEuHxzvw1Yav2Hthr+04zrV2rbnabtfOTNxZsgTmzzeTdoTp0NmwAaZPN1fkTz0FjRqZG74eYPWx1eRKm8ulawMlyUL+2y4ZVvFFn9X+jFSBqei2yEc2oDh/Hl55xUxjP3nSDB1s2mQKl/hfSkHLlrBjBwwfDn/+adZ2ee89c0PYEq01YUfDqBFcw6V7ACfJQj5111Qq5KpAvgz5bEcRTpQ1dVb61ujLggMLmB8x33acR+dwwNixZtz355+hd29zdfnii7Kq4MMEBsI775i2xRdfhC+/hJAQMyxlYbgl4lIEZyLPUCNfDZeeJ8kV8oiLEWw5s0W6VXzU2+XfJiRzCO8uepfouGjbcRJu61Yz/v3aa1C6tOnSGDTILFIl4i9rVjOpaMMGswJju3amv/7YMbfGCDsSBiCF3NnudKu0Kt7KchLhCsn8kzH8yeHsv7if0RtG244Tf9HRpgvliSfgyBGYNMlMiCle3HYy7xYaCuvXm+GWFSvMn+dXX7mtu2Xl0ZVkT5OdkMwhLj1Pkivk03dPp1LuSuRJn8d2FOEijQo3omGhhnwa9innr5+3HefhNm0yBad/f7NY1J49Zrq6Dy0YZZW/vxlu2bXLtGt27QpVqsBe194U11oTdiSMGvlcOz4OSayQH71ylC1ntvB00adtRxEuNrT+UK5HX+eTlZ/YjnJ/t26Z2ZcVKphdd/74AyZONGt+C+cLDjbdPpMmmTH0smXN1bnD4ZLTHbp8iJN/n3T5sAoksUI+e99sAJoXbW43iHC54kHFea3ca4zZNIY95/fYjnO37dvNVfjAgfD882at7iZNbKfyfUqZdzs7d5rle7t2hXr1XDJ2vvLISgBqBEshd6pZe2dRIqgEhTMXth1FuMEnNT8hdbLUvLfkPdtR/qE1jBwJ5cvDhQtmLe4ffzQLRQn3yZHDrBI5dqzZ5KJkSdMh5MTOlrCjYQSlCqJYFtev5ZRkCvnFGxdZdXSVXI0nIUGpg+hTrQ/zIuax9NBS23HM8qyNGpnx2nr1zFV5o0a2UyVdSkHHjubvoVQpeOEFs6nF1atOObw7+sfvSDKFfO7+ucTpOCnkSUyXCl0IzhBM98Xd7a6OOHeuKRYrV8I335ideoKC7OUR/yhQwPy9DBwIM2aYsfO//krUIY9cOcKxq8fcMj4OSaiQz9o3i1xpc1EuRxJclyIJSxGQgs/rfs72s9uZsHWC+wPcumVW6Wva1Owsv2kTvPGGdKR4Gn9/eP99WL3a3PysWhWGDHnkG6F3xsdrBtd0XsYHSBKF/EbMDRYdWETzos3d8jZHeJbWxVtTKXclPlzxIZHRke478eHDps1t9GgznPLXX9IX7ukqVTKTspo1g549zdDXuXMJPkzY0TAyp8xM8SD3/H0niUK+5OASomKjZFgliVJKMezJYZyJPMMXa79wz0nnzDG7wx84ALNmmQkpyV2w6a5wvgwZ4Lff4NtvzZBL6dJmMlEChB0Jo3q+6m7bQjJJFPJZ+2aRIUUGt41XCc9TMXdF2j7WlqF/DuX41eOuO1FMjFmoqVkzKFjQLDXbrJnrzidcQyl4/XXYuNEU9rp14fPP49XVcvjyYQ5fOey2YRVIAoU8zhHH3P1zaVy4MYH+PrZrikiQQXUG4dAO+izv45oTnDxpepOHDoU33zSbHxSQ9e69WsmSZr2WVq3M4mUtWjy0q2XZ4WUA1C3gvlUqfb6Qbzi5gQs3LtAkRCZbJHXBGYJ5p+I7/Lz9Z8JPhTv34EuWQJkysGUL/PorfP2192x2LB4sbVqzeuLw4ab7KDTUtCzex7LDy8iRJodb+sfv8PlCPj9iPn7Kj/oF69uOIjzA+1XfJyhVEN0Xd0c7Y/KH1qa7oUEDyJ7dbLvWrl3ijys8i1LmhvWKFXD9utno4+ef73qaQztYdmgZdQrUcWtjhc8X8nkR86icpzKZUsr6FQLSp0hPv1r9WHV0FbP2zkrcwW7cMNO9e/Y0b73Xr4eiRZ2SU3ioqlXNu64KFcwEojfeMC2mt+04u4PzN85TN797N//w6UJ+6u9TbDmzhcaFG9uOIjxIh8c7UDyoOD2X9nz0NcuPHjWthVOmwODB5ldZMzxpyJbNDKX17AnffQe1asHp08A/4+N1CtRxaySfLuQLIhYAZllTIe4I8AtgaL2hHLh0gG82fpPwA6xcacZJDx82a6X06iUTfJKagADTxTJ9uhkvL1cO1q9n6aGlFMlchNzpcrs1jk8X8vkH5pM7XW5KZi1pO4rwMA0KNaB+wfr0C+vHpahL8fsmrWHUKNOKFhRkuhkaNnRtUOHZWraEdesgZUqia1Un7OAyt3ar3OGzhTw6LpolB5fQqFAjmc0p7qKUYmi9oVy9dZX+Yf0f/g03b8Krr0KXLtC4sRkPD3Htri/CS5QsCRs3sr5RKW7oaOouO2zmE7iRzxbyNcfW8Hf03zQOkfFxcW8ls5XklTKv8PXGr4m4GHH/J548CTVqmOVm+/aFmTMhXTr3BRWeL1Mmlr3ZED+tqPnNfLO65Xn37U7ls4V8fsR8kvkno3b+2rajCA/Wv3Z/kvkno9fSXvd+wp9/mvHw3btNAf/kE/Dz2f82IhGWHl1OaO4nyPD9JLOuTmiomdnrBj77L3JexDxqBtckTbI0tqMID5Y9TXZ6V+3NzL0zWXV01f9+cdw4qFkT0qQxQynNm9uIKLzAtVvX+OvEX6btsH17M6tXa9PZ9OuvLj+/Uwq5UqqBUmqfUuqAUqq3M46ZGIcvH2bvhb3SdijipVulbuRKm4vui7vj0A6zo/2bb0KnTmbK/YYNUKKE7ZjCg4UdCSNOx/3TdliunJkcVr68KezvvQdxrlsPP9GFXCnlD3wNNASKA+2UUlbX6lx0cBFgOhOEeJhUgakYWGcg4afCmbzmO6hTx6x816uXaS+UbdjEQyw6uIhUgamonKfyP5/MmhWWLjUXBUOHmpvkly+75PzOuCIvDxzQWh/SWkcDUwDXLPc2bZrpHLh584FPW3xwMfnS56NwJtmbU8TPc6We4/F0RXl/TheitoXD5Mlmoo+/v+1owsNprVlwYAG189cmRcB/1tcJDDTr7owdC8uXmxmhe5y/GbgzCnku4N/rgp64/bn/oZTqpJQKV0qFn3/Uu7kHD8L48Wbc8tSpez4l1hHL8sPLqVegnrQdinjzm/QLw749xPE0cQwf96rZu1GIeDhw6QCHLh+iQcEHjAB07GgKeUwMREU5PYPbbnZqrcdqrUO11qFBj7pX4fvvw++/w65dZgxq3bq7nrLx5Eau3rpKvYL1EplYJAmxsdCtG7zwAjVyVaZZ/gYMOjSRs5FnbScTXmLBATODvGHhh0wOq1oV9u83G444mTMK+Ukgz78+zn37c67x9NOmgyB1atPb+/33//PlJYeWoFDUye/etQ6EF7p40axaOHy4meizeDFfNB7Jzdib9F3Z13Y64SUWHlhI4UyFKZAxHmvPB7pmTwRnFPKNQGGlVH6lVDKgLTDHCce9vxIlzM4dtWubtyxvvmk6DTCFvFzOcmROldmlEYSX27bN9PmuXm2G60aOhMBAQjKH8Gbom4zbPI6d53baTik8XFRMFCuPrKRhIbtLNSS6kGutY4G3gUXAHmCa1npXYo/7UBkzmo6Cnj1Nh0Hdulw7foB1x9dRv4CsPS4eYOpUs8luTIwp5C+//D9f/rjGx6RLno73lrxnKaDwFquOriIqNsp6h5xTxsi11vO11iFa64Ja6wHOOGa8+PubFcgmT4bwcFa2rUicjpPxcXFvcXGmpbBtWzNOeafP9z8yp8rMh9U+ZOGBhSw6sMhCUOEtFh5YSIqAFG7dn/NefGNmZ9u2sHYti3PfIlU0VAo7aDuR8DSXLkGjRvDFF2ZT3eXLzY4+9/F2+bcpkLEAPZb0IM7huokcwrstOLCAGvlqkDIwpdUcvlHIAcqWZUnlbNT8OxPJX+oA775rOhKE2LEDnnjCbNM1dqwZikuW7IHfkjwgOZ/X/Zyd53Yyfst4NwUV3uTw5cPsu7jP+vg4+FAhP3rlKPuvHKTeMx9A164wYgQ8+SRcuGA7mrBp+nQzHn7jhtkQomPHeH9ry2ItqZKnCh+t+Ii/b/3tuozCKy08sBDwjBnkPlPIlxxaAkC9wg1MEf/xR1i71lyJbdtmN5xwv7g46NMHWrc260Vv2gSVKz/8+/5FKcWX9b/k7PWzfLH2CxcFFd5qXsQ8CmQsQEhm++vS+0whX3poKTnS5KB40O1lXl56CVatMp0JlSrBL79YzSfc6MIFMx4+cKBZ0mHlSsiZ85EOVSF3Bdo91o6h64Zy/Orxh3+DSBKuR19n6aGlPBXylEfMIPeJQq61ZuWRldTKX+t//1DLlzedCeXKwXPPmX7zf+14LXzQ+vVQtiyEhZnx8HHjIHnyRB1yUJ1BaK3ps7yPk0IKb7fk0BJuxd2iaZGmtqMAPlLI917Yy9nrZ6kVXOvuL2bPbjoU3nvP3OSqWhWOHHF7RuFiWsPo0VC9utkYd+1aMx7uhKulfBny8U7Fd/h5+8+Enwp3Qljh7f7Y9wfpk6enWt5qtqMAPlLIVxxZAXDvQg5mWuwXX5gdXiIiTA/x3LluTChcKjISnn0WOnc2N7g3bzbvwpzo/arvE5QqiO6Lu6O1duqxhXeJc8Txx/4/aFS4EYH+rplyn1A+U8jzpMvz8LUOmjc3N73y5YOmTeGDD6RF0dvt2WOG0KZNM2Pis2e7ZP3w9CnS82nNT1l1dBWz9812+vGF99hwcgPnb5ynaYhnDKuADxTyO+PjNYNrxu+mQ8GCZh/Gjh1h0CCzSeqZM64PKpxLa/jpJ9OVdPEiLFliVsd04X6aHct1pFiWYvRc0pPouGiXnUd4tjn75hDgF+ARbYd3eH0h33V+FxduXLj/sMq9pExpboRNmGA2SS1VCubPd1lG4WTXrsHzz8OLL/6zwW1t12+yHeAXwJB6Q4i4FMF34d+5/HzCM83ZP4fq+aqTMaXn7Bzl9YV8xeHb4+P5E1DI73jxRdPVkj272YbpnXceuvuQsGzDBtOVMmUK9O8Py5ZBrrv2MXGZRoUbUbdAXT4N+5TLUa7Ztkt4roOXDrL7/G6PGlYBXyjkR1YQnCGY4AzBj3aA4sVNcejSxSxlWrGiS7ZiEonkcJgb1lWqmPsaYWHw4Ydu34pNKcXQekO5HHWZz1Z95tZzC/v+2P8HgBRyZ3JoB2FHwxK/8liKFKaIz51rtpArV84MvUh3gmc4edJsANGrFzRrBlu3moJuSenspXm5zMuM2jCKg5dkgbakZPa+2RQPKk7BTAVtR/kfXl3Id5zdwaWoSwkbH3+Qxo3NdP6qVeG110yXi9wItUdrmDQJHnvM9IWPGQO//eYRu9r3r92fQP9Aei/rbTuKcJNz18+x6ugqWhRtYTvKXby6kD+0f/xR5MgBCxfCsGGweLHZjWjyZLk6d7dz56BVK3NTs0QJ8wO2UyenTPBxhpxpc9KrSi+m757OmmNrbMcRbjBr7ywc2kHL4i1tR7mL1xfyghkLkid9noc/OSH8/MwyuFu3QkiImWzSqpUpLsL1Zs40V+Fz55px8bAwKFTIdqq7dK/UnZxpc9J9cXcc2mE7jnCxGXtmUDBjQUpnK207yl28tpDHOeJYdXSVa3fmKFIE1qwxxWTePHNlOHWqXJ27ypkz8Mwz0KIF5Mlj2grfe8/tNzTjK3Wy1AyoPYANJzcwdedU23GEC12KusTyw8tpWaylRyyS9V9eW8h3nNvBlZtXXL/Fkr+/KSZbtkCBAmY3osaN4dAh1543KXE4zPh30aJmZma/fmbxqxIlbCd7qBdKv0DZ7GXpvaw3UTFRtuMIF5mzbw6xjlhaFW9lO8o9eW0hvzMu6bZFa4oVMzfcRo40V+klSpgp4dEywy9Rdu0yC129/rpZA2f7dvjoI7M+jhfwU358Wf9Ljl09xsi/RtqOI1xkxp4Z5E2fl9Ccobaj3JPXFvLVx1aTJ10e8mXI576TBgSYfvM9e6BJE7NxQZkyZgsxkTBXr0KPHmZyz549ZiOQZcvMPQkvUyt/LZqGNGXg6oGcjTxrO45wsmu3rrH44GKPHVYBLy3kWmvWHFtD1bxV7QTIlcu0wc2bB1FRZnr400+blRXFg8XFmTXCCxc2nUHPPw9795qNQDz0P0l8DKk3hKjYKD5Y9oHtKMLJ5u6fS3RcNC2LeV63yh1eWcgPXznMqb9P2V8LuFEj2L0bBgyApUvNcEu3bnBZpm7f08qVZm2UTp3MjeTwcPjhBwgKsp0s0YpkKcI7Fd5h/NbxbDy50XYc4UTTd08nR5ocVMpTyXaU+/LKQn5nfNzaFfm/pUxplsONiDBrt4wYYVrlhgyB69dtp/MMGzdC/fpQqxZcumTWSVm1yoyJ+5CPanxEttTZ6LKwi7Qj+oirN68yP2I+rYq3wk95brn03GQPsProajKkyECJrB7U1ZA9uxky2LLFrI/ds6fpchk2zOzgnhTt2GFmx5Yvb/5cvvzSDKM884xXD6PcT7rk6RhcdzDrT6xn0vZJtuMIJ5i1dxa34m7xbMlnbUd5IK8s5GuOr6FKniqe+ROydGlYsMB0uJQqBd27mzXQhw0zy68mBevWmXsGpUubG8H9+pl2zW7dzDsYH/ZC6Rcon6s8vZb24u9bf9uOIxLp152/kj9DfirkqmA7ygN5YCV8sPPXz7P3wl7PGFZ5kMqVzWYHYWGmP7p7dzPJpUcPOHbMdjrnczjMTMxq1cxrv7M64eHDpp0wbVrbCd3CT/kxquEozkSekdURvdzZyLMsPbSUZ0s+67HdKnd4XSFfe3wt4Mb+8cSqXt1clW7YYG6Ojhjxz8SiZctMAfRm587B55+bLpSmTc0PqREjzK/9+kGmTLYTul35XOV5qcxLDF8/nP0X99uOIx7RtF3TcGiHxw+rgBcW8tVHV5PcP7nHNubf1xNPmMW3Dh0yG1gsWgR165obo/37w/HjthPGX0yM2VGpbVvInRt69za//vorHDgAXbtCmjS2U1o1qM4gUgSkoNuibrajiEf0685fKZ2tNMWDituO8lBeV8jXHF9D+VzlSR6Q3HaUR5M3LwwdCqdPm8KXPz98/LHZELpKFRg+HI4etZ3ybrGx5p3Fa6/9s6PSokXw5pumBTMsDNq185oZma6WPU12+tboy7yIeczbP892HJFAhy4fYv2J9V5xNQ6JLORKqdZKqV1KKYdSyuWXyNejr7P59GbPHx+PjxQpTOFbtgwOHoRPPzXtit26QXCw6bfu08cUT1vbz506ZWZctmljer1r1zbrgz/5JMyZYxa5GjHCLF8g7tK5QmeKZC7Cu4velc2avczkHZMBaPtYW8tJ4iexV+Q7gRbAKidkeai/Tv5FrCPWe8bH46tAAXNDcOtW048+eDAkT27GnmvXNuPMdeqYIYzp0+HIEeevwBgZacbxv/4a2rc3mXLlgldeMWvLtGhhZrOeO2feSTRtajKK+0rmn4wRDUYQcSmCketlHRZvobXmlx2/UCVPFfKmz2s7TrwEJOabtdZ7ALfd0V19dDUK5dEzrBKtUCGzpVmvXqZdcdUq0/2ydq1pYYyJMc9Lm9Y8984je3Zz1RwUZHbQCQz85xEba672b9wwBfv0aXO1feqU6SrZvft/O2ly5oRKleCtt8w4fqlSPtn37Q4NCjWgSUgT+q3qx7MlnyVXOvdtFC0ezYaTG9hzYQ9jm4y1HSXeElXI3S1zqsy0LtGaDCky2I7iHunSmcW5mjQxH9+6ZSbZhIebhaYiIsxV/MyZplgnVJYspiWyWjWzCXXx4mYRq7x5pXA70YgnR1DimxJ0X9ydKa2m2I4jHuLHrT+SMiAlzzz2jO0o8ab0Q96iK6WWAtnv8aU+WuvZt5+zEuihtQ5/wHE6AZ0A8ubNW+6oJ97Q81YOh5n6fv68eVy5Yq7c7zwCAiB1akiVynSTZM9utrSToRG36RfWj74r+7L4ucXUK1jPdhxxH1ExUeT4MgdNizTl56d/th3nLkqpTVrru+5HPrSQx/PgK3lIIf+30NBQHR4er6cK4RNuxt6k5LclUSi2v7GdFAEpbEcS9/Drjl9p/3t7lr2wjNr5a9uOc5f7FXKvaz8UwhulCEjB142+JuJSBEPWDrEdR9zHj1t/JDhDsOt3HnOyxLYfPq2UOgFUAuYppRY5J5YQvqd+wfq0KdGGAasHcPDSQdtxxH8cu3qMZYeW8VLplzxzHacHSFRarfVMrXVurXVyrXU2rfWTzgomhC8aVn8Ygf6BdF7QGWcMawrnmbh1IhrNi2VetB0lwbzrx44QXi5Xulz0q9mPBQcWMHPvTNtxxG0O7eDHrT9SO39tgjME246TYFLIhXCzzhU6UypbKbou7EpkdKTtOAJYfHAxh68cpuPjHW1HeSRSyIVwswC/AL5t/C0nrp3gk5Wf2I4jgG/DvyVr6qy0KNbCdpRHIoVcCAsq56lMh7IdGLF+BJtPb7YdJ0k7dvUYc/fP5dWyr5LMP5ntOI9ECrkQlnxR7wuypMpChzkdiHU8wsxc4RTjNo1Da02ncp1sR3lkUsiFsCRjyoyMbjSaLWe2MGzdMNtxkqSYuBi+3/I9jQo38sqbnHdIIRfCopbFWtK8aHP6ruzLgUsHbMdJcmbtncWZyDO8EfqG7SiJIoVcCIuUUoxuOJpk/sno9Ecn6S13s683fk2+9PloUKiB7SiJIoVcCMtypcvFkHpDWHFkBT9s+cF2nCRj8+nNhB0N460n3sLfz992nESRQi6EB+jweAeq56tOj8U9OP33adtxkoTh64eTJlkaOpbzzt7xf5NCLoQH8FN+jGs6jpuxN3l7wdu24/i8k9dOMmXnFF4t+6pP7G8ghVwIDxGSOYRPan7C73t+Z+rOqbbj+LRRG0bh0A66VuhqO4pTSCEXwoP0qNyD8rnK8+b8N2WIxUUioyMZs2kMLYq1IH/G/LbjOIUUciE8SIBfABObT+RGzA06zZUuFleYsHUCV25eoVvFbrajOI0UciE8TNEsRRlUZxBz989lwtYJtuP4lJi4GIb+OZSKuSv61CbuUsiF8EBdKnShRr4adF3YlaNXZH9bZ/llxy8cvXqUD6t9aDuKU0khF8ID+Sk/fmz2Iw7t4JU5r+DQDtuRvF6cI46BqwdSNntZGhVuZDuOU0khF8JD5c+Yn2FPDmP54eV8s/Eb23G83m+7fyPiUgQfVv8QpZTtOE4lhVwID9bx8Y40LNSQ95a8x65zu2zH8VoO7eCzVZ9RPKg4zYs2tx3H6aSQC+HBlFKMbzaedMnT0XZGW6JiomxH8kqz9s5i1/ld9KnWx+s2Vo4P33tFQviY7GmyM7H5RHae28l7S96zHcfrxDni+HD5hxTNUpQ2JdrYjuMSUsiF8AINCjWgW8VufL3xa2bvnW07jlf5efvP7LmwhwG1BxDgF2A7jktIIRfCSwysYzouXpnzCievnbQdxyvcjL1J35V9eSLnEzxd9GnbcVxGCrkQXiJ5QHKmtJrCrdhbPDfzOeIccbYjebzvwr/j2NVjDKozyOc6Vf5NCrkQXiQkcwijG41m5ZGVfLziY9txPNq1W9cYsHoAdQvUpU6BOrbjuJQUciG8zEtlXuLVsq8ycM1A5uybYzuOx/ps1WdcuHGBQXUG2Y7iclLIhfBCoxuN5vEcj/PCzBc4eOmg7TgeZ//F/YxYP4KXy7xMaM5Q23FcTgq5EF4oRUAKpreejp/yo+W0ltyIuWE7kkfptqgbKQJSMLDOQNtR3EIKuRBeKn/G/PzS4he2n93Om/PelCVvb1sQsYB5EfP4uMbHZE+T3XYct5BCLoQXa1i4IX1r9GXitokMXz/cdhzromKi6LKwC4UzFaZLhS6247iNb3bHC5GEfFTjI3ac20GPxT0IyRxCk5AmtiNZ0y+sHwcuHWDp80tJ5p/Mdhy3SdQVuVJqiFJqr1Jqu1JqplIqg5NyCSHiyU/5MbH5RMrmKEu7Ge3YeW6n7UhWbDuzjSF/DuHlMi/7fLvhfyV2aGUJ8JjWuhSwH3g/8ZGEEAmVOllq5rSdQ9pkaWnyaxPOXT9nO5JbxTpi6fBHBzKnyszQ+kNtx3G7RBVyrfVirXXs7Q/XA7kTH0kI8ShypcvFnHZzOHv9LM2mNEtSnSxD1g4h/FQ4XzX4ikwpM9mO43bOvNn5CrDgfl9USnVSSoUrpcLPnz/vxNMKIe4IzRnKLy1+YcPJDbT+rTUxcTG2I7lc+KlwPl75MW1KtPHZ1Q0f5qGFXCm1VCm18x6PZv96Th8gFvjlfsfRWo/VWodqrUODgoKck14IcZcWxVrwbeNvmR8xn1fnvOrT28Rdj75O+9/bkz1Ndr5r/J1Pr6fyIA/tWtFa133Q15VSLwFNgDpaGlmF8AidynXi3PVzfLTiI7Kmzuqz48bvLnqXiIsRLH9xORlTZrQdx5pEtR8qpRoAPYEaWuukMyAnhBfoU60P566f48t1X5I2WVr61uxrO5JTTdw6kXGbx9GrSi9qBte0HceqxPaRjwaSA0tuv6VZr7V+PdGphBCJppRiRIMRREZH8knYJwA+U8w3n97M6/Nep1ZwLT6r/ZntONYlqpBrrQs5K4gQwvn8lB/fP/U9gM8U8ws3LtBiaguCUgUxtdVUn931JyHkT0AIH/ffYh7jiKF/rf5eeWMwKiaKpyY/xZnIM6x+eTVBqaVxAqSQC5Ek3CnmAX4BDFg9gHPXz/FN42+86mo2zhHHs78/y/oT65neZjpP5HrCdiSP4T1/i0KIRPFTfoxpMoZsqbPx2erPOHv9LFNaTiFlYErb0R5Ka83b899m1t5ZfNXgK1oUa2E7kkeR1Q+FSEKUUvSv3Z/RDUfzx74/qP1TbU7/fdp2rAdyaAdvzX+L7zZ9R+8qvelcobPtSB5HCrkQSdBb5d9iepvpbD+7nXJjy7H+xHrbke7JoR28Ne8tvg3/lp6VeyaZjSISSgq5EElUi2ItWPfqOlIEpKDGhBqM3TTWozaniIqJou30tv9/JT647mCvvEHrDlLIhUjCSmUrRXincGoG1+S1ua/R6rdWXLhxwXYszl8/T52f6vDb7t8YUm8IA+sMlCL+AFLIhUjiMqXMxIL2CxhSbwh/7PuDkt+WZH7EfGt5Vh9dTZkxZdhyZgu/tf6NHpV7SBF/CCnkQgj8lB89KvdgQ8cNZE6Zmca/NqbVtFYcu3rMbRmi46L5dOWn1JxYk9SBqVn36jpaFW/ltvN7MynkQoj/VyZ7GTZ12sSA2gOYHzGfYl8Xo++Kvly5ecWl511zbA1lx5Tlk7BPaPdYOzZ12kSZ7GVcek5fIoVcCPE/kgck54NqH7DnrT00KtyIfqv6kX9kfj5d+SlnI8869Vxbz2zlqclPUe3HalyPvs68Z+cxqcUk0iZP69Tz+Dpl4y51aGioDg8Pd/t5hRAJt/XMVj5Z+Qmz980m0C+QFsVa8HKZl6mVv9YjbXB8K/YWc/bNYcymMSw7vIwMKTLQo1IP3qn4DqmTpXbBK/AdSqlNWuvQuz4vhVwIER97L+xlTPgYJmybwJWbV0iXPB1PFnySKnmqUD5XeYpmKUqGFBnuujEZGR3JjrM72HJmC8sOL2PxwcVERkeSN31eOj3eibfKv0WGFBnsvCgvI4VcCOEUUTFRLDu8jNl7Z7Pw4EJOXDvx/19LFZiKLKmyEOAXgEM7uHDjApHRkf//9Vxpc9EkpAnNizanXoF6+Pv523gJXut+hVzWWhFCJEjKwJQ0CWlCk5AmAJz6+xQbT27k0OVDHL92nItRF4lzxKGUIkvKLGRLk41iWYpRNkdZ8qTLI62ELiCFXAiRKDnT5qRZ0WYPf6JwGelaEUIILyeFXAghvJwUciGE8HJSyIUQwstJIRdCCC8nhVwIIbycFHIhhPByUsiFEMLLWZmir5Q6Dxx1waGzAPa3N3l03p4fvP81eHt+8P7X4O35wXWvIZ/WOui/n7RSyF1FKRV+r3UIvIW35wfvfw3enh+8/zV4e35w/2uQoRUhhPByUsiFEMLL+VohH2s7QCJ5e37w/tfg7fnB+1+Dt+cHN78GnxojF0KIpMjXrsiFECLJkUIuhBBezucKuVKqv1Jqu1Jqq1JqsVIqp+1MCaGUGqKU2nv7NcxUSmWwnSmhlFKtlVK7lFIOpZTXtJEppRoopfYppQ4opXrbzpNQSqnxSqlzSqmdtrM8CqVUHqXUCqXU7tv/frrazpQQSqkUSqkNSqltt/N/6rZz+9oYuVIqndb62u3fdwGKa61ftxwr3pRS9YHlWutYpdTnAFrrXpZjJYhSqhjgAMYAPbTWHr9Bq1LKH9gP1ANOABuBdlrr3VaDJYBSqjoQCfyktX7Mdp6EUkrlAHJorTcrpdICm4Dm3vJ3oMwedqm11pFKqUBgDdBVa73e1ef2uSvyO0X8ttSAV/2k0lov1lrH3v5wPZDbZp5HobXeo7XeZztHApUHDmitD2mto4EpgFftX6a1XgVcsp3jUWmtT2utN9/+/d/AHiCX3VTxp407O00H3n64pf74XCEHUEoNUEodB9oDH9vOkwivAAtsh0gicgHH//XxCbyoiPgapVQwUBb4y3KUBFFK+SultgLngCVaa7fk98pCrpRaqpTaeY9HMwCtdR+tdR7gF+Btu2nv9rD8t5/TB4jFvAaPE5/XIMSjUEqlAWYA7/znHbbH01rHaa3LYN5Jl1dKuWWIK8AdJ3E2rXXdeD71F2A+0NeFcRLsYfmVUi8BTYA62kNvYiTg78BbnATy/Ovj3Lc/J9zo9tjyDOAXrfXvtvM8Kq31FaXUCqAB4PKbz155Rf4gSqnC//qwGbDXVpZHoZRqAPQEntJa37CdJwnZCBRWSuVXSiUD2gJzLGdKUm7fLPwB2KO1HmY7T0IppYLudJkppVJibpy7pf74YtfKDKAIpmviKPC61tprrqyUUgeA5MDF259a701dNwBKqaeBUUAQcAXYqrV+0mqoeFBKNQJGAP7AeK31ALuJEkYpNRmoiVlC9SzQV2v9g9VQCaCUqgqsBnZg/v8CfKC1nm8vVfwppUoBEzH/fvyAaVrrfm45t68VciGESGp8bmhFCCGSGinkQgjh5aSQCyGEl5NCLoQQXk4KuRBCeDkp5EII4eWkkAshhJf7P/Jz6vMW3ahyAAAAAElFTkSuQmCC\n" + }, + "metadata": { + "needs_background": "light" + } + } + ], + "source": [ + "\n", + "def predict_sin4(x):\n", + " return a+b*x+c*x**2+d*x**3\n", + "import matplotlib.pyplot as plt \n", + "plt.plot(x,y,c='r')\n", + "plt.plot(x,(predict_sin4(x)).detach().numpy(),c='g')\n", + "plt.show()" + ] + }, + { + "cell_type": "code", + "execution_count": 85, + "metadata": {}, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "99 1063.9005126953125\n", + "199 706.83837890625\n", + "299 470.62939453125\n", + "399 314.3643798828125\n", + "499 210.98336791992188\n", + "599 142.58665466308594\n", + "699 97.33384704589844\n", + "799 67.39220428466797\n", + "899 47.580684661865234\n", + "999 34.47142028808594\n", + "1099 25.796506881713867\n", + "1199 20.05570411682129\n", + "1299 16.25637435913086\n", + "1399 13.741846084594727\n", + "1499 12.0775146484375\n", + "1599 10.975820541381836\n", + "1699 10.246527671813965\n", + "1799 9.763702392578125\n", + "1899 9.444038391113281\n", + "1999 9.232373237609863\n", + "Result: y = 0.003426064969971776 + 0.837184488773346 x + -0.000591053394600749 x^2 + -0.09054866433143616 x^3\n" + ] + } + ], + "source": [ + "# 5 nn 模块。封装计算过程层。构建计算图\n", + "\n", + "import torch \n", + "import math \n", + "\n", + "x = torch.linspace(-math.pi, math.pi, 2000)\n", + "y = torch.sin(x)\n", + "\n", + "p = torch.tensor([1,2,3])\n", + "xx = x.unsqueeze(-1).pow(p)\n", + "\n", + "model = torch.nn.Sequential(\n", + " torch.nn.Linear(3,1),\n", + " torch.nn.Flatten(0,1)\n", + ")\n", + "\n", + "loss_fn = torch.nn.MSELoss(reduction='sum')\n", + "\n", + "learning_rate = 1e-6\n", + "\n", + "for t in range(2000):\n", + " y_pred = model(xx)\n", + "\n", + " loss = loss_fn(y_pred,y)\n", + " if t%100 == 99:\n", + " print(t,loss.item())\n", + " \n", + " model.zero_grad()\n", + "\n", + " loss.backward()\n", + "\n", + " with torch.no_grad():\n", + " for param in model.parameters():\n", + " param -= learning_rate * param.grad\n", + " \n", + "\n", + "linear_layer = model[0]\n", + "print(f'Result: y = {linear_layer.bias.item()} + {linear_layer.weight[:, 0].item()} x + {linear_layer.weight[:, 1].item()} x^2 + {linear_layer.weight[:, 2].item()} x^3')\n" + ] + }, + { + "cell_type": "code", + "execution_count": 86, + "metadata": {}, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "[[ 8.3718449e-01 -5.9105339e-04 -9.0548664e-02]]\n0.837184488773346\n" + ] + }, + { + "output_type": "display_data", + "data": { + "text/plain": "
", + "image/svg+xml": "\r\n\r\n\r\n\r\n \r\n \r\n \r\n \r\n 2021-03-23T16:04:59.244539\r\n image/svg+xml\r\n \r\n \r\n Matplotlib v3.3.2, https://matplotlib.org/\r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n\r\n", + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAD4CAYAAADhNOGaAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAA/2ElEQVR4nO3deZxN9R/H8ddnZjCy79nHvm+ZECqy73u2bIVfiZLIEpE1yVK2CEWEUAZZQmQJGWv2fRtrdsbs398f59JgrHNnzty5n+fjcR/uPcs971vczz3n+z3frxhjUEop5b487A6glFLKXloIlFLKzWkhUEopN6eFQCml3JwWAqWUcnNedgd4HmnTpjU+Pj52x1BKKZeybdu2f40x6R5c7pKFwMfHB39/f7tjKKWUSxGRk1Et10tDSinl5rQQKKWUm9NCoJRSbk4LgVJKuTktBEop5eacUghEZJqIXBSRPY9YLyLyjYgcEZHdIvJSpHVtROSw49HGGXmUUko9PWedEfwAVH/M+hpAHsejIzARQERSA/2B0kApoL+IpHJSJqWUUk/BKfcRGGPWiYjPYzapB8ww1pjXm0UkpYhkBCoAK40xVwBEZCVWQZntjFxKqTgkMBAOH4ZDh+DiRbhxA0JD4YUXIFky8PGB3LkhRw7w0KvWsSm2bijLDJyO9PqMY9mjlj9ERDpinU2QLVu2mEmplHKeiAjYsAEWL4a1a2H7dmvZk6RKBa+8ApUrQ6NGoP/eY5zL3FlsjJkMTAbw9fXV2XSUiqtOnYLx42HWLAgIgIQJoUwZ6NMHihSBfPkgY0ZInhwSJIA7d+DaNW4d2sO2/as5fGgzJ89t4fyapQSt70Zo+jQkyVOI1AVLkjlVNgqmK0jRDEV5MemLdn/SeCO2CkEAkDXS6yyOZQFYl4ciL18bS5mUUs60fz8MHAjz5lmva9aEESOgTh1ImvShzcMjwtkSsAW/A34sP7qcPRf3EGEiIDV4pPEgfaJ0JA4Ox+vGTW6fXceVa+sIivSNlT9tfirnqEzjgo15NfureIheTnpesVUIFgGdRWQOVsPwdWPMORFZAQyN1EBcFegdS5mUUs5w/jz07w9TpkCSJNC1K3TpAtmzR7n5iWsnmLp9KtN2TuPszbN4eXjxWvbX6PtqX0pnKU2hdIXInDwzXh6OrydjYPVqzKiRXFq3nH35UuHfoiJ/pAhk2s5pjNs6jmwpstG+RHs6vdyJNC+kib3PHl8YY6L9wGrcPQeEYl3nfwd4F3jXsV6A8cBR4B/AN9K+bwNHHI92T3O8kiVLGqWUzcLDjfn2W2OSJTPGy8uYDz4w5uLFR26+49wO03BuQyMDxHh87mFqzaplZv8z21y7c+3pj7lhgzGvvWYMGFO6tLnlv8n8tPsnU/XHqoYBmBeGvGC6Le9mLgdedsIHjH8AfxPVd3hUC+P6QwuBUjY7ftyYihWtr5A33jDm0KFHbrrv4j5Td3ZdwwBMimEpzKerPzUnr518/mNHRBgza5Yx6dIZ4+lpTP/+xoSGmt3nd5tWv7QyHp97mNTDU5uxW8aasPCw5z9OPKSFQCnlHAsXGpMypTHJkxszebL1xRyFa3eumW7LuxmvgV4mxbAU5vO1n5urd646L8fly8a0amV9jb36qjEnreKy6/wuU2l6JcMAzCtTXjEH/z3ovGO6OC0ESqnoCQ015uOPra+NkiWNOXbskZv+uv9Xk2FEBiMDxHRY1MFcvPXoS0bRNnOmMUmTGpMqlTGrVxtjjImIiDCzds8yqb5IZRIPTmy+2/ZdzB3fhTyqEGgzu1LqyW7cgNq1YeRI6NQJNm60bvx6wNU7V2n9a2sazG1ApmSZ+LvD30yuM5l0SR6aFMt5WraEHTsgUyaoWhUmTkREaFGkBXs67aF8tvJ0WNyBjos7EhwWHHM5XJgWAqXU4505A6++CqtWwXffWfcIJEr00GYbTm2gyMQi/PTPT/R/vT9b2m/BN5Nv7GTMnRv++guqV7cK1YcfQkQEmZJlYlnLZfQp34fvtn/H6z+8zr+B/8ZOJheihUAp9WgHD1o3gx0/DkuXQvv2D21ijGH0ptFU+KECiRMkZnP7zQyoMIAEngliN2vy5ODnZ3Vf/eYbaNMGQkPx9PBkSKUhLHhzAbsu7KL8tPKcvBbljI1uSwuBUipqe/bA669b4wFt2GBddnnArZBbNJ3flG6/d6NOvjr4d/CPvbOAqHh6wqhRMHQozJwJjRtDUBAADQs05Pe3fufC7QuUnVaW/Zf225czjtFCoJR62I4dUKGC9cX6559QtOhDm5y9eZbXvn+NBfsXMLzycH558xdSeKeI/awPEoHeva1LWIsWQf36EGy1Dbya/VXWtV1HeEQ4lWZU4vDlw/ZmjSO0ECil7rd3rzXgW5IksG4d5M//0CZ7Lu6hzJQyHL5ymCXNl/BJuU8QERvCPkanTtbdzitWQNOm1pkNUCRDEVa3Xk1oRChvzHiD41eP2xzUfloIlFL/OX7cugSUKBGsWQO5cj20yepjqyk3rRzhJpz17dZTI08NG4I+pXfegbFjrbaDVq0gPByAQukLsarVKm6H3KbKj1XcvgFZC4FSynL+PFSpYo0G+vvvkDPnQ5ssPriYmj/VJFuKbGx+ZzPFXywe+zmfVefO8OWXMHeu9dxYgxcXe7EYS1suJeBmAPXm1CMoLMjmoPbRQqCUgps3ra6X587Bb79B4cIPbTJv7zwa/tyQYhmKsa7tOrKmyBrFG8VRPXpAz57w7bfWiKgOZbKUYUb9Gfx1+i/aLmxrjX7qhrQQKOXuwsOhRQurl9CCBdakMA+YuXsmzRY0o0yWMqxqvYpUiV1wRtmhQ6FZM6sgzJ17b3GTQk0YXnk4c/fOZci6ITYGtI8WAqXc3SefwJIlVt/76g9PPT5z90xa/9qaCj4VWN5yOckTJbchpBN4eMD330P58tC6tdUl1qFH2R60LNKS/mv7s/LoShtD2kMLgVLu7LvvrH73nTtbvWwe8Ov+X2m7sC0Vc1RkSfMlJEmYxIaQTuTtbTUcZ89uTYN55gwAIsKk2pMomK4gzRc05/T10094o/hFC4FS7mrjRuvLv1o1GD36odUrjqyg6fymvJz5Zfya+ZE4QWIbQsaA1Klh4UIIDLSKgeOGsyQJk7DgzQWEhIfw5vw3CQ0PtTdnLNJCoJQ7unAB3nwTfHxgzhzwun+ywvUn19NgbgMKpS/E0hZLSZrw4akmXVrBgjBjBvz9N7z//r2eRPnS5uO7Ot+x+cxmhq4fanPI2OOUQiAi1UXkoIgcEZFeUawfLSI7HY9DInIt0rrwSOsWOSOPUuoxwsKgeXO4cgXmz4eUKe9bvefiHmrPrk22FNlY8dYK12wYfhoNGkDfvjBtGkyadG9x08JNeavoWwxaN4i/A/62MWDsEeOohM/9BiKewCGgCtY0lVuB5saYfY/YvgtQwhjztuP1LWPMM/3c8PX1Nf7+/tHKrZTb6tMHhg2zGk7btr1v1dmbZykzpQxhEWFsbr+ZbCmy2ZMxtkREWMNrr14NW7ZA8eIAXAu6RtGJRfH28mbH/3a4ftuIg4hsM8Y8NBiUM84ISgFHjDHHjDEhwByg3mO2b441x7FSKrb99ptVBNq3f6gI3Ay+Se2fanM16Cq/tfgt/hcBsHoSzZgBadNaw1DcugVASu+UTK8/nSNXjtBndR+bQ8Y8ZxSCzEDkJvYzjmUPEZHsQA7gj0iLvUXEX0Q2i0j9Rx1ERDo6tvO/dOmSE2Ir5WbOn7e+/IsVs4ZdiCQsIow357/J7gu7+bnxz5TIWMKejHZImxZ++gmOHLF6TzlUzFGR919+n7F/j433l4hiu7G4GTDfGBMeaVl2x6lKC2CMiDw8uAlgjJlsjPE1xvimSxeDsx0pFR9FRFhF4NYt60vP2/veKmMMnZd2ZvmR5UyoNSFujx0UU15/Hfr1g+nT4ccf7y0eUmkImZJlosPiDvG6F5EzCkEAEPle8yyOZVFpxgOXhYwxAY4/jwFrATf6KaJULBk3zhqFc+RIq8dMJBP9JzJp2yR6lutJx5IdbQoYB/TtC6+9Bu+9B0ePApA8UXLG1RzH7gu7Gb354S628YUzCsFWII+I5BCRhFhf9g/1/hGR/EAqYFOkZalEJJHjeVqgHBBlI7NS6jn9849193CtWtaXXCTrTq7jw+UfUjtvbYZWcp/uklHy8rIms/Hyss6eHCOV1s9fnwb5G9B/bX+OXT1mb8YYEu1CYIwJAzoDK4D9wM/GmL0iMlBE6kbatBkwx9zfTakA4C8iu4A1wBeP6m2klHoOQUHWOEIpUljdJCPNGXDq+ika/9yYXKlyMbPBTDxEbysia1ZrqI0NG+67yW5sjbF4iic9VvawMVzMiXb3UTto91GlntLdrqK//QY1a95bHBgaSPlp5Tl69Sh/t/+bfGnz2RgyjjHGusdg2TLYvh0KFQJgyLoh9F3Tlz9a/0HFHBVtDvl8YrL7qFIqLtq2zRqHv127+4qAMYb2i9qz8/xOZjearUXgQSIweTIkT24NTueY2azbK93wSelD1xVdCYsIszmkc2khUCo+CgmxCkCGDNagcpFM2DqB2XtmM/iNwdTMU/MRb+Dm0qe37jbevh2GWENTJ06QmBFVRrD7wm6mbJ9ic0Dn0kKgVHw0dKjVSDxp0n1DSPif9eejFR9RK08tepV/aDQYFVnDhlb7ytCh1jzOQKMCjXgt+2v0/aMvN4Jv2BzQebQQKBXf7Npl/Ypt2dIaPsHh6p2rNJnXhIzJMjK9/nRtHH4aY8ZYl4g6dICICESEUVVHcfnOZUb+NdLudE6jfxOUik/CwuDttyFNGvj663uLjTG082vHmRtnmNt4LmleSGNjSBeSLp11aW3TJpg4EYCSmUrSuGBjRm0exaXb8WOUAy0ESsUn48db17XHjrWKgcPozaPxO+jHiCojKJOljI0BXVCrVlClCvTufW8im4EVBhIYGhhvhqrWQqBUfBEQYN0dW6MGNG58b/HmM5vpuaonDfI34MPSH9oY0EWJWJPeh4Xdm7ugQLoCtCnWhgn+Ezh1/ZTdCaNNC4FS8UXXrtaX1bhx924cuxF8gxYLWpAleRam1ZuGRLqhTD2DnDnh889h0SL45RcABlQYAMDAPwfaGMw5tBAoFR8sW2ZNMtO3r/Wl5dBlWRdOXj/JzAYzSemd0r588cFHH1kjt3btCrdukS1FNt4t+S4/7PyB41eP250uWrQQKOXq7tyxLlnkzw/du99bPGfPHGbsmkHfV/tSLls5GwPGE15eVhvMmTP37i34pNwneHp4MnzjcJvDRY8WAqVc3ZAhcPw4TJgAiRIB1jhC7y55lzJZytDv9X42B4xHypWDNm2sUVwPHiRz8sy0K96O73d+T8CNRw26HPdpIVDKlR06ZA0j8dZbUNEa/yY8Ipy3fnmLcBPOrIaz8PLwesKbqGcyfDi88AJ06QLG0LNcT8Ijwvnqr6/sTvbctBAo5co+/tiaZGbEiHuLhm8czvpT6xlfczw5U+V8zM7quWTIAAMHwsqV8Msv5EiVg5ZFWzJp2ySXva9AC4FSrmr5cliyxJpZ68UXAWsIif5r+9O0UFNaFW1lc8B4rFMnKFrUakC+fZve5XsTFBbEqE2jnrxvHKSFQClXFBpqfQnlzg0ffABAUFgQbRa2IX2S9EysNVG7isakuw3Hp0/DsGHkT5ufRgUbMdF/IjeDb9qd7plpIVDKFY0fDwcOWMMfOBqIB6wdwL5L+5hadyqpEqeyOaAbKF/eGs/pq6/g5Em6v9Kd68HX+X7n93Yne2ZOKQQiUl1EDorIERF5aEhDEWkrIpdEZKfj0T7SujYictjxaOOMPErFa5cuwYABUK3avUHlNp/ZzIi/RtC+RHuq565ubz53MmyYdfNer16UzlKaslnLMmbzGMIjwu1O9kyiXQhExBMYD9QACgLNRaRgFJvONcYUdzymOPZNDfQHSgOlgP4ioj9llHqcvn3h1i1rKkUR7oTeoc3CNmRJnoWR1eLPiJguIWtW696NOXNg0ya6lenG8WvH8TvoZ3eyZ+KMM4JSwBFjzDFjTAgwB6j3lPtWA1YaY64YY64CKwH9OaPUo+zcCd99B507Q4ECAPT9oy+HLh9iat2pJE+U3N587qhnT8iYET76iPr56pEjZQ6XazR2RiHIDJyO9PqMY9mDGonIbhGZLyJZn3FfRKSjiPiLiP+lS67ZRUupaDEGunWD1Kmhf38A1p9cz+jNo3nP9z0q56xsc0A3lTSpdVPfli14/jyPD0t/yMbTG9lyZovdyZ5abDUWLwZ8jDFFsX71T3/WNzDGTDbG+BpjfNOlS+f0gErFecuWwZo1VvtAqlTcDrlNO792+KT04csqX9qdzr21aQMlSkDPnrydvznJEyVnzJYxdqd6as4oBAFA1kivsziW3WOMuWyMCXa8nAKUfNp9lVJYo4p+8onVXbRjRwD6rO7D0atH+b7e9yRNmNTmgG7Ow8PqwXX6NMnGf8fbxd9mwb4FXLh1we5kT8UZhWArkEdEcohIQqAZsCjyBiKSMdLLusB+x/MVQFURSeVoJK7qWKaUimz6dGve3GHDIGFCNp/ZzNi/x/L+y+/zus/rdqdTABUqQP36MGwY7/o0IjQilKk7ptqd6qmIMSb6byJSExgDeALTjDFDRGQg4G+MWSQiw7AKQBhwBXjPGHPAse/bQB/HWw0xxjyxE66vr6/x9/ePdm6lXMLt25A3L2TLBn/9RUhEKC9NeonrwdfZ12kfyRIlszuhuuvwYShYEDp2pHLpgxy+cphjHxzD08PT7mQAiMg2Y4zvg8udMhqVMWYpsPSBZZ9Fet4b6P2IfacB05yRQ6l4acwYOHsW5s4FEYZvGM7eS3tZ3HyxFoG4Jk8ea6L7yZN5r+FoGh/vwtLDS6mTr47dyR5L7yxWKi67eNEa7bJ+fShfnv2X9jN4/WCaFW5G7by17U6novLZZ5AwIXUn/0nGpBmZ6D/R7kRPpIVAqbhs0CAIDIRhw4gwEXRY3IGkCZPydfWv7U6mHuXFF+Hjj0nw83w6ZqzN8iPLOXb1mN2pHksLgVJx1aFD1qTpHTpA/vx86/8tG09vZFTVUaRPkt7udOpxuneHtGnp8OM+PMSDSf6T7E70WFoIlIqr+va1BpQbMIDT10/Ta1UvKuesTOtire1Opp4keXLo25fMyzdSJ2Vpftj1A6HhoXaneiQtBErFRdu3w7x50K0bJn16Oi3tRFhEGJNqT9LhpV3Fu++Cjw/vLDvPxdsXWXp46ZP3sYkWAqXior59IVUq+Phj5u2bx5JDSxhUcZDOOOZKEiWCQYOo/vsxXvRMybSdcbdzpBYCpeKaDRus4SR69eJ6Ivhw+YeUzFiSD8t8aHcy9axatMCrSDFa7zT8dug3zt86b3eiKGkhUCouMQb69LF6nnTuTN8/+nLx9kUm1Z6kk9C7Ig8PGDaMdn9eJ9yEM3P3TLsTRUkLgVJxye+/w/r10Lcv/tf2MX7reDr5dqJkppJP3lfFTdWrkz9fOV45n4Bp26fijNEcnE0LgVJxhTHw6afg40P4O2/z7pJ3yZA0A4PfGGx3MhUdIjB4MG//Hcr+ywfYEhD3hqfWQqBUXPHrr7BtG/Tvz7e7p7Ht3DZGVR1FCu8UdidT0VWhAm+mfZ0XQmHa33HvngItBErFBeHhVk+h/Pk5X78yff7oQ+WclWlWuJndyZSTJB8wjEb74Od/5hAUFmR3nPtoIVAqLvjpJ9i/HwYN4uM/ehIUFsT4muP1noH45JVXaOldiusEsWzXfLvT3EcLgVJ2Cwmxpp4sUYLVJVLw0z8/0atcL/KmyWt3MuVklbqNJf0tmLVkqN1R7qOFQCm7TZsGx48TPGgAnZZ1JleqXPR+NcpR25WL8/ItRdOgXCwJ28/1s8ftjnOPUwqBiFQXkYMickREekWxvpuI7HNMXr9aRLJHWhcuIjsdj0UP7qtUvBYcbE18XrYsXybdyaHLh5hQawLeXt52J1MxpGWTgQR7wS8TOtsd5Z5oFwIR8QTGAzWAgkBzESn4wGY7AF/H5PXzgcgzbd8xxhR3POpGN49SLuX77+HMGY727MCQ9UN5s9CbVM1V1e5UKgaVerU5uUKSMitgOVyIG3MaO+OMoBRwxBhzzBgTAswB6kXewBizxhgT6Hi5GWuSeqXcW0gIDBuGKVOazjfmktAzIaOrjbY7lYphIkKLEq35I3sEZ4f3tTsO4JxCkBk4Hen1GceyR3kHWBbptbeI+IvIZhGp/6idRKSjYzv/S5cuRSuwUnHC9Olw6hR+Xaqy/OhyBlUcRKZkmexOpWJBiwpdMAJzt34PAQF2x4ndxmIReQvwBUZEWpzdMZlyC2CMiOSKal9jzGRjjK8xxjddunSxkFapGBQaCkOHcqeMLx9d+pHC6Qvzfqn37U6lYkn+tPl5KXVhZhUMt6YitZkzCkEAkDXS6yyOZfcRkcrAp0BdY0zw3eXGmADHn8eAtUAJJ2RSKm6bMQNOnGDE2/k4ce0EY2uM1UHl3EyLkm3ZlgmO/jwJzp61NYszCsFWII+I5BCRhEAz4L7ePyJSApiEVQQuRlqeSkQSOZ6nBcoB+5yQSam4KzQUhgzhxKtFGHZxAU0LNaWCTwW7U6lY1rhgYwDm5w21/awg2oXAGBMGdAZWAPuBn40xe0VkoIjc7QU0AkgKzHugm2gBwF9EdgFrgC+MMVoIVPw2cyYcP87HjZLiIR58VfUruxMpG2RPmZ2XM73M/PJpYPJkOHfOtixOORc1xiwFlj6w7LNIzys/Yr+/gCLOyKCUSwgLgyFDWFU1N79c28SQN4aQJbl2onNXjQs2pufZnpx4wQOfL7+E0fb0GnO7O4tvh9y2O4JyZz/9ROjxo3xQMYhcqXLx8Ssf251I2eje5aG3XoJvv7XtrMCtCkG9OfWoM7uO3TGUuwoLg8GDGdsgE/uDzzCm+hgSeSWyO5WyUc5UOXkp40vMzxNqtR2NGPHknWKAWxWCfGnyseHUBm6F3LI7inJHc+Zw/txhBhS7Ss08Namdt7bdiVQc0LhAY7Zc3sWpNvVh4kQ4H/vzGrtVIaiWqxqhEaH8eeJPu6ModxMeDoMH06txSoIlnDHVxtidSMURdy8PLaifz7azArcqBOWylSOxV2JWHF1hdxTlbubO5a/Ag0z3uUa3Mt3IkyaP3YlUHJEnTR6KZSjG/EtroWVL66wglscgcqtC4O3lTQWfCvx+9He7oyh3Eh5O+JBBdGnoTeZkmfn0tU/tTqTimMYFG/PX6b8I6NbeGpE2ls8K3KoQAFTNVZWDlw9y8tpJu6ModzF/PlMTH2B7qiC+qvoVSRMmtTuRimMaFmgIwKKQPdZZwYQJcPHiE/ZyHrcrBNVyVQPQswIVOyIiuPJFf/pU9eT17K/TtFBTuxOpOKhA2gLkTp2bRYcWWXNXx/JZgdsVgvxp85MleRZtJ1CxY8ECPst0kKuJIvimxjc6B7GKkohQN29d/jj+BzezZ4QWLWL1rMDtCoGIUC1XNVYdW0VYRJjdcVR8FhHBrq97M9EXOvl2omiGonYnUnFYvfz1CAkPsX6k9u0LQUHwVewMP+J2hQCsy0PXg6+zNWCr3VFUPGZ+/ZUueY+S2isZA98YZHccFceVzVqWNInT4HfQD/Llg2bNYPx4iIX5V9yyEFTKWQkP8dDLQyrmREQw+/turM8OQ6uPIFXiVHYnUnGcl4cXtfLW4rdDvxEaHmqdFdy5EyvjD7llIUidODUvZ3pZG4xVjLn161x65DtFyYQ+vF2yvd1xlIuol68eV4OusvH0RihQAJo0gXHj4MqVGD2uWxYCsLqRbgnYwtU7V+2OouIbYxj8a1fOJodxLX7E08PT7kTKRVTNVZVEnonwO+BnLejbF27ehK+/jtHjum0hqJarGhEmglXHVtkdRcUzh+Z9y6icF2mbpDxlspe3O45yIUkTJqVSzkr4HfTDGANFikCDBlYhuH49xo7rtoWgdJbSpPJOxW+Hf7M7iopHTEQEXdf2InG48EX7OXbHUS6oXr56HL92nL2X9loL+vWzisDYsTF2TKcUAhGpLiIHReSIiPSKYn0iEZnrWL9FRHwirevtWH5QRKo5I8/T8PLwokaeGiw9vJQIExFbh1Xx3JLZA1iW4QYD0r9JhpSZ7Y6jXFCdvNZQ+fcuD5UoAbVrW43GN2/GyDGjXQhExBMYD9QACgLNRaTgA5u9A1w1xuQGRgPDHfsWxJrjuBBQHZjgeL9YUStPLS4FXtJupMopgkLv0HXXlxS8loDO/5tmdxzlojImy4hvJl+WHVn238J+/awG4wkTYuSYzjgjKAUcMcYcM8aEAHOAeg9sUw+Y7ng+H6gk1i2W9YA5xphgY8xx4Ijj/WLG8OHQ678Tluq5q+MhHiw5tCTGDqncx1fT/8exJMF8k6sLCbxfsDuOcmE1ctdg05lNXLnj6C1UqhRUr27dYHbb+bMsOqMQZAZOR3p9xrEsym0ck91fB9I85b4AiEhHEfEXEf9Lz3uDxYkTMGoUnDoFWN1Iy2Ytq+0EKtpOXTvJ0FOzaHQ8MZU6DrM7jnJxNXLXIMJEsPLoyv8W9usHKVPC8eNOP57LNBYbYyYbY3yNMb7p0qV7vjfp3dv6c/jwe4tq56nNjvM7CLgR4ISUyl11/7EVREQwslRfSJjQ7jjKxZXKXIrUiVPff3mobFk4cAAKF3b68ZxRCAKArJFeZ3Esi3IbEfECUgCXn3Jf58mWDdq1gylT4MwZAGrlrQXA0sNLY+ywKn7749hq5l1ZT6/dycnevrvdcVQ84OnhSbVc1Vh2ZNn9nVk8Y6YJ1RmFYCuQR0RyiEhCrMbfRQ9sswho43jeGPjDGGMcy5s5ehXlAPIAfzsh06P17g0REffOCgqlK0T2FNlZcljbCdSzCw0P5YMF7fG5Cj2qD9SzAeU0NfPU5OLti+w4tyPGjxXtQuC45t8ZWAHsB342xuwVkYEiUtex2VQgjYgcAboBvRz77gV+BvYBy4H3jTHh0c30WD4+0KYNfPcdnD2LiFArTy1WHVtFUFhQjB5axT/j/x7H3sATjNmamsTvvGt3HBWPVMtVDUHuvzwUQ5zSRmCMWWqMyWuMyWWMGeJY9pkxZpHjeZAxpokxJrcxppQx5likfYc49stnjIn5TwzQpw+Ehd07K6idtzaBoYGsPbE2Vg6v4ocLty7Qf3U/qh2Bus0GQKJEdkdS8Ui6JOnwzeQbK5etXaax2Kly5oTWrWHyZDh3jgo+FUjslZjFBxfbnUy5kN6re3MnNJCvt6VHOnSwO46Kh2rmqcmWgC1cDrwco8dxz0IA8OmnEBoKI0aQOEFiquaqyqJDi6zxPZR6gi1ntvD9zu/pusmQ791Pwdvb7kgqHrrbjTSmR0p230KQK5c1SfS338KFCzTI34AzN87gf9bf7mQqjoswEXRe1pmMwQnpdyA96NmAiiG+mXxJ+0LaGG8ncN9CAPdNEl0nXx08xZNfD/xqdyoVx03bMQ3/s/6MWBJCsm69IXFiuyOpeOpuN9LlR5bH6Jho7l0I8uSxJomeOJHUN8N43ed1LQTqsa7euUrv1b0pfz0FLS6mh//9z+5IKp6rmqsqlwIvsfvC7hg7hnsXAvhvkuiRI2mQvwEH/j3AgX8P2J1KxVGfrfmMK4FXGDv7OtKzl54NqBhXOWdlgPuHm3AyLQSRJomun/41AH7dr2cF6mG7L+xmgv8E3j2bkeImg54NqFiRKVkmCqcvzO/HYq7BWAsBWGcFgYFkmTSblzO9rJeH1EOMMXRZ1oVUXskY9GMAfPIJvKAjjKrYUSVnFdafXM+d0Dsx8v5aCMCaJPrNN2HcOBpkq8bWs1s5c+OM3alUHDJnzxzWnVzHkP0ZSZ0sPbyrdxGr2FMlZxWCw4NZf2p9jLy/FoK7+vWD27dpsN4a4nrhgYX25lFxxq2QW3Rf2Z2XkuWl/awDejagYt1r2V8joWfCGGsn0EJwV6FC0Lgx+b/5ifyp8ujlIXXPkHVDOHvzLOM2psAznZ4NqNiXJGESymUtF2PtBFoIIuvXD27epOHl9Px54k8u3X7OCXBUvHHo8iFGbhpJ64zVeeWXrdCjByRJYncs5Yaq5KzC7gu7OX/rvNPfWwtBZEWKQMOGNJm1k3ATzi/7f7E7kbKRMYauy7vi7eXN8EV3IF06eO89u2MpN1Urby3eKfEOwWHBTn9vLQQP+uwzih29TT7SMHfvXLvTKBstObSEZUeWMcCnLS/+9qeeDShbFc1QlCl1p5A9ZXanv7cWggcVK4bUr0/Tzbf48+SfMXIapuK+oLAguq7oSoG0Begy4wCkTQudOtkdS6kYoYUgKp99RtNtwUSYCObvm293GmWDERtHcOzqMb7xeZcEy1fq2YCK16JVCEQktYisFJHDjj9TRbFNcRHZJCJ7RWS3iDSNtO4HETkuIjsdj+LRyeM0JUpQsEwdCv/rwdxdP9mdRsWyo1eOMmT9EJoUbELlccv0bEDFe9E9I+gFrDbG5AFWO14/KBBobYwpBFQHxohIykjrexhjijseO6OZx3n696fp7gg2nN2kN5e5EWMMHyz/gASeCRidvhUsX26dDSRNanc0pWJMdAtBPWC64/l0oP6DGxhjDhljDjuenwUuAumiedyYV7IkTdNWAGDethn2ZlGxxu+gH0sPL+XzCp+T+YsJejag3EJ0C0EGY8w5x/PzQIbHbSwipYCEwNFIi4c4LhmNFpFHTvoqIh1FxF9E/C9dip3+/Xn6fEWJczB33YRYOZ6y1+2Q23yw7AOKpC9Cl7CSejag3MYTC4GIrBKRPVE86kXezlhzPD5ynkcRyQj8CLQz5t4MC72B/MDLQGqg56P2N8ZMNsb4GmN806WLpROKkiVpSiG2EMDx49tj55jKNoPWDeL0jdNMqDWBBP0/hwwZ4P337Y6lVIx7YiEwxlQ2xhSO4uEHXHB8wd/9or8Y1XuISHLgN+BTY8zmSO99zliCge+BUs74UM7UrN0oAGZN7WpvEBWj9l3ax8hNI2lbvC3lj4TAmjXQp4/2FFJuIbqXhhYBbRzP2wB+D24gIgmBX4EZxpj5D6y7W0QEq31hTzTzOF320lWpEJieGTc3YM7rPQXxkTGG95e+T7KEyfiy0nBrWPIsWaBjR7ujKRUrolsIvgCqiMhhoLLjNSLiKyJTHNu8CbwGtI2im+gsEfkH+AdICwyOZp4Y0bpiVw6nNmz5qqvdUVQMmL1nNmtPrGVYpWGkW+cPmzZZ4055e9sdTalYIdalfdfi6+tr/P39Y+14N4Jv8OLQ1LTdaZgw/qT1a1HFC9eDrpNvXD6yp8zOX+024vlyKbh+HQ4cgAQJ7I6nlFOJyDZjjO+Dy/XO4qeQPFFy6ueqxZwCEQQPGWh3HOVE/db04+Lti0yoOQFPv0WwYwf0769FQLkVLQRPqXXZ97iaGJaunwrHj9sdRznBjnM7GL91PJ1e7kTJDMXhs88gf35o2dLuaErFKi0ET6lyzsq8mDg9M4oBA/WswNWFR4TTcUlH0r6QlsFvDIa5c2HvXvj8c/D0tDueUrFKC8FT8vLwomXxVvyWB/6dNx0OHrQ7koqG8VvH43/Wn6+rf01Kr6TW5aCiRaFxY7ujKRXrtBA8g9bFWhMqEcwukQAGDLA7jnpOp66fos/qPtTIXYOmhZrCjBlw5AgMGgQe+k9CuR/9W/8MimYoSsmMJfnujRSYOXNg1y67I6lnZIyh89LOGAwTak1AgoOtov7yy1Cnjt3xlLKFFoJn1LFkR/7xuMTf+ZNB7952x1HP6Jf9v7D40GIGVRyET0ofGD8eTp+G4cNBxO54StlCC8Ezal64OUkSJGHyW/lh2TJrKALlEq4FXaPLsi68lPElPij9AVy7BkOGQPXqULGi3fGUso0WgmeULFEymhVuxhzZy42cmeGTT8AFb8pzR71X9ebC7QtMrj0ZLw8v6yzg2jX44gu7oyllKy0Ez6FjyY4EhgbyU9dK4O8P8+bZHUk9wcZTG/l227d0Ld2VkplKQkAAjBlj3TNQrJjd8ZSylQ4x8RyMMRSfVBwv8WLb+FAIDIT9+/Vu1DgqJDyEEpNKcCvkFns77SVpwqTQoYPVW+jgQfDxsTuiUrFCh5hwIhGh40sd2X5+O9v6vQNHj8LkyXbHUo8wfMNw9l3ax8RaE60isH8/TJtmzTymRUApLQTPq2XRliT2Sswk7z3w+uvW3cY3b9odSz1gz8U9DFo3iGaFm1EzT01r4d15Bj791N5wSsURWgieU0rvlLQs0pKZu2dxZfCncPEijBpldywVSVhEGO382pHSOyVja4y1Fv71FyxcCD17WvMRK6W0EERHl9JduBN2h2keu6yhCUaMgAsX7I6lHEb+NRL/s/6MrzmetC+ktXp3ffIJvPgidO1qdzyl4oxoFQIRSS0iK0XksOPPVI/YLjzSpDSLIi3PISJbROSIiMx1zGbmMopmKMrr2V9n3N/jCB88CO7epapst//Sfvqv7U+jAo1oUqiJtXDePNi40RpKQqegVOqe6J4R9AJWG2PyAKsdr6NyxxhT3PGoG2n5cGC0MSY3cBV4J5p5Yt0HpT/g5PWTLDYH4L33rEbjPXFuxk23Eh4RztuL3iZpwqSMrzneWhgUZF0OKlYM2rWzN6BScUx0C0E9YLrj+XSseYefimOe4jeAu/MYP9P+cUXdfHXJliIb32z5xjobSJECPvpIbzKz0ZjNY9h8ZjPf1PiGDEkzOBaOgRMnrHYcHWZaqftEtxBkMMacczw/D2R4xHbeIuIvIptFpL5jWRrgmjEmzPH6DJD5UQcSkY6O9/C/dOlSNGM7j5eHF518O7HmxBr+CQ2wxrNftQqWLLE7mls6dPkQfdf0pW6+ujQv3NxaeP68NZRE3brwxhv2BlQqDnpiIRCRVSKyJ4pHvcjbGevOtEf9DM7uuImhBTBGRHI9a1BjzGRjjK8xxjddunTPunuMav9Se7y9vPl6y9fw7rvWLFcffwwhIXZHcyvhEeG8s+gdvL28+bbWt8jdQeT69bPab776yt6ASsVRTywExpjKxpjCUTz8gAsikhHA8efFR7xHgOPPY8BaoARwGUgpIl6OzbIAAdH+RDZI80Ia2hZry4+7f+Rc0L/W5YfDh62RLVWsGblpJBtObeDr6l+TMVlGa+GuXTB1KnTuDHny2BtQqTgqupeGFgFtHM/bAH4PbiAiqUQkkeN5WqAcsM9xBrEGaPy4/V1F97LdCYsIY8zmMVCjhvX4/HOIQ5ex4rNd53fR94++NCrQiFZFW1kLjYFu3SB1auusQCkVpegWgi+AKiJyGKjseI2I+IrIFMc2BQB/EdmF9cX/hTFmn2NdT6CbiBzBajOYGs08tsmVOhdNCjZhov9ErgVdg5Ej4dYtawpEFaOCwoJ469e3SPNCGr6tHemSkJ8f/PGH1YifKsqezUopdNA5p9pxbgcvTX6JYZWG0at8L/jgA+vy0I4d1ny4KkZ0/707IzeNZGmLpdTIU8NaGBgIBQtCsmSwfbsOCKgUOuhcrCiRsQRVc1VlzOYxBIUFWb9EU6eG99/X7qQxZM3xNYzaNIr3fN/7rwgADBsGJ09ahViLgFKPpYXAyXqV68WF2xeYvnO6VQSGD4cNG6whj5VTXQ+6TpuFbcidOjcjqoz4b8WRI/Dll9ZcA6+9Zl9ApVyEFgInq+BTgVKZS/HFxi8ICQ+Btm2hbFno0QOuXrU7XrzSeVlnzt48y48NfiRJQseQEcZYl+QSJbLGflJKPZEWAicTEQa8PoAT107ww84fwMMDJkyAy5d12GMnmrFrBjN3z6Tfa/0onaX0fyv8/Ky5pAcOhIwZ7QuolAvRxuIYYIyh7LSyBNwI4HCXwyTySmQNO/H117BlC7z8st0RXdrBfw9ScnJJfDP5srr1ajw9HENGBAZCgQKQPLnVQO/l9fg3UsrNaGNxLBIRBlYYyOkbp5m6w9Ej9vPPreGP33sPwsPtDejCgsKCaLagGd5e3sxsOPO/IgDWqKKnTlkNxFoElHpqWghiSOWclSmfrTxD1g+xehAlT27dcbxtG3z7rd3xXFaP33uw8/xOfqj/A1mSZ/lvxa5dVptAu3baQKzUM9JCEEPunhWcvXmWb/0dX/xNm0KVKtCrl/XLVT2ThQcWMm7rOD4q8xG189b+b0VYGLRvD2nS6HhCSj0HLQQxqGKOilTKUYnB6wZbdxuLWPMVGGMNTueC7TN2OXntJG/7vU3JjCUZVmnY/Su/+Qb8/WHsWKvLrlLqmWghiGFfVf2KK3euMGy948vLxweGDrV6tsycaWs2VxEUFkSjnxsRbsKZ03iO1fh+17Fj0Lcv1KkDTZrYF1IpF6aFIIYVf7E4bYq3YcyWMZy4dsJa+P771r0FXbvqHMdPocvSLmw7t40Z9WeQO3Xu/1bcPbPy8rK66N4dY0gp9Uy0EMSCQRUH4Sme9Fndx1rg6QlTpliD0nXpYm+4OG7K9ilM2TGFPuX7UC9/vftXTpsGK1daw0lkyRL1GyilnkgLQSzIkjwLH7/yMbP3zObvgL+thQUKWCOTzpsHv/xib8A4amvAVt5f+j5VclZhYMWB9688ftw6o6pY0eqSq5R6bnpDWSy5GXyTfOPykTl5Zja/s9nq/x4aCmXKWD2I/vnHus9AAfBv4L+8NOklRIRtHbeR9oW0/62MiLCmnNy+3frvlj27fUGVciF6Q5nNkiVKxqhqo/A/68/kbZOthQkSWA3Gt27B229rLyKHkPAQGs5tyMXbF1nw5oL7iwBYd2j/+af1pxYBpaJNC0EsalqoKZVyVKL36t5cuOVoJC5QwBopc9kyvdEMa3iOd5e8y/pT65lWbxq+mR748bJvH/TubU1E37atLRmVim+iVQhEJLWIrBSRw44/H5oGSkQqisjOSI8gEanvWPeDiByPtK54dPLEdSLC+JrjCQwNpMfKHv+teP99qFbNmvD+4EH7AsYBIzeN5Pud39PvtX60KNLi/pXBwdCqlTXZzOTJ2ktIKSeJ7hlBL2C1MSYPsNrx+j7GmDXGmOLGmOLAG0Ag8HukTXrcXW+M2RnNPHFevrT5+KTcJ/y4+0dWHVtlLfTwsHrAJE4Mb71ltR24oUUHF/HJyk9oUrAJAyoMeHiDnj2tdoEpUyBDhljPp1R8Fd1CUA+Y7ng+Haj/hO0bA8uMMYHRPK5L+/TVT8mbJi/vLHqHG8E3rIWZMsF331l3yPZ6qJ7GezvP76TFghaUzFSSH+r/gIc88Fdz0SKrTeCDD6BevajfRCn1XKJbCDIYY845np8HnvQzrRkw+4FlQ0Rkt4iMFpFEUe0EICIdRcRfRPwvXboUjcj2S5wgMdPrT+fMjTN0/737fysaNoTOna3B6X791b6AsezY1WNUn1md1IlT49fMjxcSvHD/BqdOWe0BL71ktacopZzqiYVARFaJyJ4oHvf9LDNWP9RHdnsRkYxAEWBFpMW9gfzAy0BqoOej9jfGTDbG+BpjfNOlS/ek2HFemSxl6P5Kd77b/h0rjkT6T/LVV9Z8Be3aWcMnxHMXb1+k2sxqhEaEsuKtFWRKlun+DUJDoXlza2C5uXOtmceUUk71xEJgjKlsjCkcxcMPuOD4gr/7RX/xMW/1JvCrMebeBXBjzDljCQa+B0pF7+O4ls8rfk7BdAV5e9Hb/Bv4r7UwUSL4+WerIbRJEwgKsjdkDLoZfJOas2oScCOAJc2XUCBdgYc36tED/voLJk2C3LkfXq+UirboXhpaBLRxPG8D+D1m2+Y8cFkoUhERrPaFPdHM41K8vbyZ2WAm/wb+S5uFbYgwEdYKHx+YPt1qGO3SJV7eX3B3ILmd53cyr8k8Xsn6ysMb/fij1S7w4YfWWYFSKkZEtxB8AVQRkcNAZcdrRMRXRKbc3UhEfICswJ8P7D9LRP4B/gHSAoOjmcfllMhYgtHVRrP08FK++ivSWPp160KfPlYPmXHj7AsYA4LDgmn0cyNWHlvJ1LpTqZW31sMb+ftDhw7WEBI6Cb1SMUqHmIgDjDE0nd+UX/b/wrp26yibtay1IiLCakBevNi64axqVXuDOkFIeAiNf27M4kOLmVx7Mh1Kdnh4o4sXoWRJq1utvz/EgzYhpeICHWIiDhMRvqvzHT4pfWj8c2PO3DhjrfDwsIagKFwY3nzT5W82Cw0Ppen8piw+tJgJNSdEXQQCA625BS5ftnpOaRFQKsZpIYgjUninYGGzhdwKuUXd2XW5HXLbWpE0qdWHPmFCqFXLZecvuBN6h0Y/N2LhgYV8U/0b3ns5ihFDw8OhRQvYuhV++snqLqqUinFaCOKQwukLM6fxHHZd2EXrha3/azzOnh38/ODsWahZE27csDfoM7oRfIMas2qw5NASxtUYR5fSUczBYIx1s5ifnzXlZP36sZ5TKXelhSCOqZmnJl9V+Ypf9v9C1+VdudeG88orMH8+7NoFDRpY4+64gIu3L1JxekU2nt7IrIazeL/U+1FvOHSoNctYjx7W2EtKqVijhSAO6lqmKx+V+Yixf4+l/9r+/62oWRO+/x7++ANatozzYxId/Pcg5aaVY/+l/fg186N5kUd0AR050pp3uFUr+OKL2A2plMLL7gDqYSLCyKojuRF8g0HrBpEsYTJ6lHOMVtqqFVy5Ys3O1bw5zJ5tzWsQx6w8upIm85qQ0DMhq1qv+q8n1IPGj4fu3a3G8GnTrAZypVSs0kIQR4kIk2pP4lbILT5Z9QmBoYF89vpniIh1g1VEBHTrBk2bwpw5VmNyHGCMYfzW8XRd3pWC6QqyuPlisqd8xOQx48ZZN8zVq2f1jvLSv45K2UH/5cVhnh6ezGw4k8QJEjPgzwFcDbrKqGqjrJE5P/rI+vXctSs0amSNw/PCC098z5h0I/gG/1vyP+bsmUOdvHWY1XAWyRIle3hDY2DQIGvO5rp1rexx8KxGKXehhSCO8/LwYmrdqaRIlIKvt3zNqeunmNFgBkkTJrXODBImtBpX33jDuvHMpn73289tp+n8phy/epyhbwylZ/meDw8lDf+dyXz9NbRpY905rWcCStlKL8i6AA/xYHS10YyqOgq/g368MvUVjl11jEz63nuwYIHVm6hsWThyJFazhYSHMPDPgZSZUoagsCDWtl1L71d7R10Ebt60uoV+/bV1JjNtmhYBpeIALQQuQkT46JWPWNZyGWdunKHEpBL8uOtHq3tpgwZWT6KrV60hrJcsiZVM/mf98Z3sS/+1/WlSqAk7/7eT8tnKR73xsWNWF9ilS622gVGjtGFYqThC/yW6mKq5qrK943aKpC9C64WteXP+m5y7ec76kt26FXLksIZo+PRT607dGBBwI4B2fu0o9V0prty5wqJmi5jVcBZpXkgT9Q7z51tjB509CytWWJeydL5hpeIMLQQuKEeqHPzZ9k+GVRrGooOLyDsuL19u/JLgrJmssfvfece6QevVV506PtHF2xfps7oPecfl5ad/fqJ72e7s7bSXOvnqRL3DrVvQvr01r0LevFahqlTJaXmUUk5ijHG5R8mSJY2yHL582NT5qY5hACbLqCxm9KbR5lbwLWNmzTImVSpjvL2N+fJLY4KDn/sYey7sMZ1/62y8B3sbGSCm6bym5tiVY4/faeFCY7JmNUbEmD59jAkJee7jK6WcA/A3UXyn6jDU8cSqY6sYvG4wf578kxSJUtC0UFNaZ61FmQFT8fRbBHnyWNNg1qnzVJdlTlw7wZJDS/hx94/8HfA3CTwS0KpoK3qU60H+tPkfveOePdC7t9VOUaSINbPYK1FMOqOUinWPGoZaC0E889fpv/jW/1sW7F9AYGggqROn5g3vApRac4giey6RK1sx0r7XneT1mxIhEBwezPlb5wm4EcC+S/vwP+vPpjOb2P/vfgAKpSvE2yXe5q2ib5E+SfpHH3j3bmt4iDlzIFkya8iIrl31/gCl4pAYKQQi0gQYABQAShljovx2FpHqwNeAJzDFGHN3JrMcwBwgDbANaGWMCXnScbUQPNnN4JssObSElcdWsurYKk7fOP1U+6VJnIaXM79M1ZxVqZW3FnlS57HuZo7KtWvWvQuTJ8OGDdYNbR9+aA0ZkTq18z6MUsopYqoQFAAigElA96gKgYh4AoeAKsAZYCvQ3BizT0R+Bn4xxswRkW+BXcaYiU86rhaCZ3c58DJ7Lu7h9NUT/LtpNde3/InX8VN4RcCLqbOSOX8p8pSsQrZiryO5cz/cvz8iAs6dg/37YdMmWLcO1q6FsDDIlQs6dYK2bbUAKBWHxeilIRFZy6MLwSvAAGNMNcfr3o5VXwCXgBeNMWEPbvc4Wgic5MQJ61LOypWwceN/Q1t7ekLKlJAihTUcRHAw/PsvhDhO1kSgYEFrNNSGDaFUKb0nQCkX8KhCEBu3dWYGIl+XOAOUxrocdM0YExZpeeZHvYmIdAQ6AmTLli1mkrobHx/o1ct63LkD//xj/eI/fNi6Oe3aNetL39sb0qSx7lHIndu6aS1FCrvTK6Wc5ImFQERWAS9GsepTY4yf8yNFzRgzGZgM1hlBbB3XbSRObP2yL1XK7iRKqVj2xEJgjKkczWMEAFkjvc7iWHYZSCkiXo6zgrvLlVJKxaLYuLC7FcgjIjlEJCHQDFjkuLlhDdDYsV0bINbOMJRSSlmiVQhEpIGInAFeAX4TkRWO5ZlEZCmA49d+Z2AFsB/42Riz1/EWPYFuInIEq81ganTyKKWUenZ6Q5lSSrmJR/Ua0j5/Sinl5rQQKKWUm9NCoJRSbk4LgVJKuTmXbCwWkUvAyRh467TAvzHwvrHF1fOD638GV88Prv8ZXD0/xNxnyG6MSffgQpcsBDFFRPyjalF3Fa6eH1z/M7h6fnD9z+Dq+SH2P4NeGlJKKTenhUAppdycFoL7TbY7QDS5en5w/c/g6vnB9T+Dq+eHWP4M2kaglFJuTs8IlFLKzWkhUEopN6eF4AEiMkhEdovIThH5XUQy2Z3pWYjICBE54PgMv4pISrszPSsRaSIie0UkQkRcphugiFQXkYMickREetmd51mJyDQRuSgie+zO8jxEJKuIrBGRfY6/Px/anelZiIi3iPwtIrsc+T+PtWNrG8H9RCS5MeaG4/kHQEFjzLs2x3pqIlIV+MMxD/RwAGNMT5tjPRMRKQBEAJN4xFzYcY2IeAKHgCpY065uBZobY/bZGuwZiMhrwC1ghjGmsN15npWIZAQyGmO2i0gyYBtQ31X+H4iIAEmMMbdEJAGwAfjQGLM5po+tZwQPuFsEHJIALlUpjTG/R5oHejPWzG8uxRiz3xhz0O4cz6gUcMQYc8wYEwLMAerZnOmZGGPWAVfszvG8jDHnjDHbHc9vYs1/8sh50OMaY7nleJnA8YiV7x8tBFEQkSEichpoCXxmd55oeBtYZncIN5EZOB3p9Rlc6EsovhERH6AEsMXmKM9ERDxFZCdwEVhpjImV/G5ZCERklYjsieJRD8AY86kxJiswC2t2tTjlSfkd23wKhGF9hjjnaT6DUs9DRJICC4CuD5zhx3nGmHBjTHGsM/lSIhIrl+ieOHl9fGSMqfyUm84ClgL9YzDOM3tSfhFpC9QGKpk42gj0DP8PXEUAkDXS6yyOZSoWOa6tLwBmGWN+sTvP8zLGXBORNUB1IMYb793yjOBxRCRPpJf1gAN2ZXkeIlId+ASoa4wJtDuPG9kK5BGRHCKSEGgGLLI5k1txNLZOBfYbY0bZnedZiUi6u738RCQxVseDWPn+0V5DDxCRBUA+rF4rJ4F3jTEu88tORI4AiYDLjkWbXanXE4CINADGAumAa8BOY0w1W0M9BRGpCYwBPIFpxpgh9iZ6NiIyG6iANQTyBaC/MWaqraGegYiUB9YD/2D9+wXoY4xZal+qpyciRYHpWH9/PICfjTEDY+XYWgiUUsq96aUhpZRyc1oIlFLKzWkhUEopN6eFQCml3JwWAqWUcnNaCJRSys1pIVBKKTf3fxeI8Sraj01FAAAAAElFTkSuQmCC\n" + }, + "metadata": { + "needs_background": "light" + } + } + ], + "source": [ + "print(linear_layer.weight.detach().numpy())\n", + "print((linear_layer.weight[0,0]).item())\n", + "def predict_sin5(x):\n", + " return linear_layer.bias.item()+x*linear_layer.weight[0,0].item()+x**2 * linear_layer.weight[0,1].item() + x**3 * linear_layer.weight[0,2].item()\n", + "\n", + "# print(predict_sin5(2))\n", + "import matplotlib.pyplot as plt \n", + "plt.plot(x,y,c='r')\n", + "plt.plot(x,predict_sin5(x),c='g')\n", + "plt.show()" + ] + }, + { + "cell_type": "code", + "execution_count": 87, + "metadata": {}, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "99 494.2001953125\n", + "199 222.24923706054688\n", + "299 131.94528198242188\n", + "399 73.64026641845703\n", + "499 37.08171844482422\n", + "599 18.956501007080078\n", + "699 10.892595291137695\n", + "799 9.057557106018066\n", + "899 8.854501724243164\n", + "999 8.906543731689453\n", + "1099 8.938582420349121\n", + "1199 8.902544975280762\n", + "1299 8.896836280822754\n", + "1399 8.908731460571289\n", + "1499 8.911303520202637\n", + "1599 8.907160758972168\n", + "1699 8.908832550048828\n", + "1799 8.916685104370117\n", + "1899 8.927263259887695\n", + "1999 8.922130584716797\n", + "Result: y = -0.0005175599944777787 + 0.8562414050102234 x + -0.0005175702390260994 x^2 + -0.09382987022399902 x^3\n" + ] + } + ], + "source": [ + "# 6 使用optimizer自动下降计算\n", + "import torch\n", + "import math\n", + "\n", + "# Create Tensors to hold input and outputs.\n", + "x = torch.linspace(-math.pi, math.pi, 2000)\n", + "y = torch.sin(x)\n", + "\n", + "# Prepare the input tensor (x, x^2, x^3).\n", + "p = torch.tensor([1, 2, 3])\n", + "xx = x.unsqueeze(-1).pow(p)\n", + "\n", + "# Use the nn package to define our model and loss function.\n", + "model = torch.nn.Sequential(\n", + " torch.nn.Linear(3, 1),\n", + " torch.nn.Flatten(0, 1)\n", + ")\n", + "loss_fn = torch.nn.MSELoss(reduction='sum')\n", + "\n", + "# Use the optim package to define an Optimizer that will update the weights of\n", + "# the model for us. Here we will use RMSprop; the optim package contains many other\n", + "# optimization algorithms. The first argument to the RMSprop constructor tells the\n", + "# optimizer which Tensors it should update.\n", + "learning_rate = 1e-3\n", + "optimizer = torch.optim.RMSprop(model.parameters(), lr=learning_rate)\n", + "for t in range(2000):\n", + " # Forward pass: compute predicted y by passing x to the model.\n", + " y_pred = model(xx)\n", + "\n", + " # Compute and print loss.\n", + " loss = loss_fn(y_pred, y)\n", + " if t % 100 == 99:\n", + " print(t, loss.item())\n", + "\n", + " # Before the backward pass, use the optimizer object to zero all of the\n", + " # gradients for the variables it will update (which are the learnable\n", + " # weights of the model). This is because by default, gradients are\n", + " # accumulated in buffers( i.e, not overwritten) whenever .backward()\n", + " # is called. Checkout docs of torch.autograd.backward for more details.\n", + " optimizer.zero_grad()\n", + "\n", + " # Backward pass: compute gradient of the loss with respect to model\n", + " # parameters\n", + " loss.backward()\n", + "\n", + " # Calling the step function on an Optimizer makes an update to its\n", + " # parameters\n", + " optimizer.step()\n", + "\n", + "linear_layer = model[0]\n", + "print(f'Result: y = {linear_layer.bias.item()} + {linear_layer.weight[:, 0].item()} x + {linear_layer.weight[:, 1].item()} x^2 + {linear_layer.weight[:, 2].item()} x^3')\n" + ] + }, + { + "cell_type": "code", + "execution_count": 88, + "metadata": {}, + "outputs": [ + { + "output_type": "display_data", + "data": { + "text/plain": "
", + "image/svg+xml": "\r\n\r\n\r\n\r\n \r\n \r\n \r\n \r\n 2021-03-23T16:15:53.888815\r\n image/svg+xml\r\n \r\n \r\n Matplotlib v3.3.2, https://matplotlib.org/\r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n\r\n", + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAD4CAYAAADhNOGaAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAABAZklEQVR4nO3dd3gUVRfH8e9JgdC7FA2EXkRqRGm+IB0UUBEF6QIKKB3p0qSKNAWkCKLSexeQonQITZpA6D0Yamhp9/1jFggYaspks+fzPPtkd8rObzXsycy9c68YY1BKKeW63OwOoJRSyl5aCJRSysVpIVBKKRenhUAppVycFgKllHJxHnYHeBFp06Y1Pj4+dsdQSimnsmPHjn+NMekeXe6UhcDHxwc/Pz+7YyillFMRkZORLddLQ0op5eK0ECillIvTQqCUUi5OC4FSSrk4LQRKKeXioqUQiMgkEQkQkX2PWS8iMkpE/EXkbxEpEmFdQxE54ng0jI48Simlnl10nRH8DFR+wvoqQE7HozkwFkBEUgO9gDeAYkAvEUkVTZmUUko9g2i5j8AY85eI+DxhkxrAL8Ya83qLiKQUkYxAGWCVMeYygIiswioo06Mjl1IqDrl1C44cgcOHISAArl+HkBBInBiSJQMfH8iRA7JmBTe9ah2bYuuGspeB0xFen3Ese9zy/xCR5lhnE2TOnDlmUiqlok94OGzYAIsXw7p1sHOntSwSp1LAtpdh70vwTyZPAjIk49+UCQhJlgSPBF4kTZAU7xTeZEmRhSIZi/DGy2+QLVU2RCR2P1M85TR3FhtjxgPjAXx9fXU2HaXiqlOnYPRomDoVzp6FBAngzTehWzd47TXInRuTIQObbxxgxuF5rPD/ncNX/AFwQ8gWkoiMV26TY/9lEoZBSNrUXPfJyN7bf7Pk8BLuhN4BIGvKrNTMU5OPXv2IYi8X06IQBbFVCM4C3hFev+JYdhbr8lDE5etiKZNSKjodPAh9+8Ls2dbrqlXh22/h3XchaVIALt++zI9+PzLpz0kcvXIULw8vyvqUpUWxVpTOXJp86fKRyDORtf/RozBnDkyZAj/uh3TpCG3Xg/0NyrE5cDdLDi9hzPYxDN8ynCIZi/DF619Qr0A9PN09bfoP4MSMMdHyAHyAfY9ZVw1YDgjwJrDNsTw1cBxI5XgcB1I/7VhFixY1Sqk44vx5Y5o3N8bNzZhkyYzp0MGYEyce2uTc9XOmzfI2Jkn/JIbemLI/lzWTd0021+9cf/r7h4cbs2qVMVWqGAPGZMhgzNixxoSGmut3rpux28ea/GPyG3pjso/Mbqb+PdWEhYfF0Id1boCfiew7OrKFz/vAatw9D4RgXef/FPgc+NyxXoDRwFFgL+AbYd8mgL/j0fhZjqeFQKk4ICzMmB9/tL78PTyMad3amICAhza5cfeG6bW2l0nSP4nx6Oth6s+rb/Zc2PPix9ywwZi33rK+ut54w5jdu40xxoSHh5vFhxabgmMLGnpjSv5U0uwP2B+VTxcvxWghiO2HFgKlbHb8uDFly1pfIW+/bczhw//ZZM7+OSbj0IyG3pgPZ31o/AP9o+fY4eHGTJ1qTLp0xri7G9OrlzEhIcYYY8LCw8zkXZNN6sGpjWdfT/PNn9+Y0LDQ6DluPKCFQCkVPRYsMCZlSmOSJzdm/HjrizmCs9fPmvdmvGfojSn8Y2Gz6dSmmMkRGGhM/frW11jp0sacPHl/VUBQgPlo9keG3phyU8qZCzcuxEwGJ/O4QqCddZVSzyY0FDp2hJo1IXt22L0bmjWDCL11Fv6zkPxj8rPcfzmDyg1iW7NtFPcuHjN5UqeGX36B336DXbugUCFYswaAdEnSMf2D6Ux4dwIbT2+k8LjC7Dy/M2ZyxANaCJRST3f9OrzzDnz3HbRsCRs3Wjd+OdwNvUub5W2oObMm2VJlY8/ne+hcqjMebrHQMfGTT6xCkCkTVKwIY8cCICI0LdKUrU234uHmwVuT32LJ4SUxn8cJaSFQSj3ZmTNQujT88QdMmGDdI5Aw4f3Vp6+dpuSkkozaNoq2b7RlY5ON5EqTK3Yz5sgBmzZB5cpWoWrT5v7NawXSF2Br063kSZuHGjNqMHnX5NjN5gSc5oYypZQNDh2CcuWsM4Jly6y/uCPYdnYbNWbU4GbwTeZ/NJ+aeWrakxMgeXJYuNC6fDViBFy+DJMmgacnGZNl5M9Gf/L+rPdpsqgJoeGhNCvazL6scYwWAqVU5Pbtg/LlwRhrqIgCBR5aPWPfDBovbEzGpBn5o/4fvPrSqzYFjcDdHYYNg5desu5kvn4dZs4ELy+SJEjCwo8X8v7M92m+pDnhJpzPfD+zO3GcoJeGlFL/tWsXlCljfbH++ed/isDQTUOpM7cOvpl82dp0a9woAveIQNeu1iWsRYusxu27dwHw8vBi/kfzqZazGi2WtmDOgTn2Zo0jtBAopR62f791JpAkCfz1F+TJc3+VMYZuq7vRaVUnar9amz/q/0G6JOlsDPsELVvCxImwYgV89JE10imQ0CMhsz+cTXHv4nwy7xPWnVhnb844QAuBUuqB48etdoCECWHtWqubqEO4CafVslYM3DCQ5kWaM+39aST0SPiEN4sDPv0Uvv/eajuoXx/CwgBI5JmIxXUWkz1VdmrMqMHBSwdtDmovLQRKKcuFC1ChAty+DStXQrZs91eFhYfReGFjxvqNpXPJzvz4zo+4u7nbGPY5fPEFDBlitRV88YXV5gGkTpSa3+v9jpeHFzVn1uTanWs2B7WPFgKlFNy4YXW9PH8eli6F/Pnvrwo34Xy25DN+2fML/cr2Y1D5Qc435HOnTtC5M/z4ozUiqkPmFJmZ8+Ecjl05Rr359Qg3kc+XEN9pIVDK1YWFQd26Vi+huXOh+IM7gY0xfLHsC37a9RM93+pJj7d62Bg0igYMgI8/tgrCzJn3F5fOUpoRlUaw5PAS+qzrY2NA+2ghUMrVffUVLFkCo0ZZZwUOxhjarWh3/3JQnzJO/iXp5gaTJ0OpUtCggdUl1qHl6y1pVKgR/f7q55KNx1oIlHJlEyZY/e6/+MLqZRNB//X9Gbl1JG3faMvAcgOd73JQZLy8rIbjLFnggw+su6axhqP4vsr35EyTk3rz6nH59mWbg8YuLQRKuaqNG60v/0qVYPjwh1ZN2jWJnmt7Ur9AfYZVGhY/isA9qVPDggVw65ZVDO5YU18mTZCUae9PI+BmAM0WN7s3X4pL0EKglCu6eBFq1wYfH5gxAzweDDKw9PBSmi9uTsXsFfmp+k/xqwjcky+fNXLptm3QqtX9nkRFMxVlQLkBzDs4j8m7XWdMomgpBCJSWUQOiYi/iHSJZP1wEdnteBwWkasR1oVFWLcoOvIopZ4gNBTq1LHG4pkzB1KmvL9q29lt1J5Tm4IZCjLnwznxe/7f996DHj2s8YjGjbu/uH3x9vwvy/9ov6I9526cszFg7IlyIRARd6xpKKsA+YA6IpIv4jbGmHbGmELGmELA98C8CKtv31tnjKke1TxKqaf4+mvrZrGxY6FgwfuLT149yTvT3iF9kvQsrbuUZAmT2RgylvTpA1WqWKOV7t4NgJu4MeHdCdwNu0vLpS1d4hJRdJwRFAP8jTHHjDHBwAygxhO2r4M1x7FSKrYtXQoDB0LTptCo0f3FQcFBVJ9RneCwYJZ9sowMSTPYlzE2ublZl4jSprWGoQgKAiBnmpz0K9uPhYcWMmv/LJtDxrzoKAQvA6cjvD7jWPYfIpIFyAqsibDYS0T8RGSLiNR83EFEpLljO79Lly5FQ2ylXMyFC9aXf8GC1rALDuEmnAbzG7AvYB8zas0gT9o8j3+P+ChtWpg2Dfz9rd5TDm3fbItvJl++XP4lV25fsTFgzIvtxuKPgTnGmLAIy7IYY3yBusAIEcke2Y7GmPHGGF9jjG+6dHF0kCul4qrwcKsIBAVZX3peXvdX9V7Xm/n/zGdohaFUzlH58e8Rn/3vf9CzJ0yZAr/+CoCHmwcT3p1A4O1Aeq3rZXPAmBUdheAs4B3h9SuOZZH5mEcuCxljzjp+HgPWAYWjIZNSKqIffrBG4fzuO6vHjMPMfTPp91c/GhdqTNs329qXLy7o0QPeegtatICjRwEolKEQnxf9nNHbR/P3xb9tDhhzoqMQbAdyikhWEUmA9WX/n94/IpIHSAVsjrAslYgkdDxPC5QEDkRDJqXUPXv3WncPV6tmfck57AvYR5NFTSjpXZKx1cbGz26iz8PDA377zfrZqNH9kUr7vd2PVF6p+HL5l/G24TjKhcAYEwp8AawADgKzjDH7RaSviETsBfQxMMM8/F8yL+AnInuAtcAgY4wWAqWiy5071jhCKVJY3SQdX/Y37t6g1qxaJEuQjNkfzo77w0nHFm9va6iNDRvu32SXOlFqBpYbyF8n/2L6vvjZz0WcscL5+voaPz8/u2MoFfd162b1Elq6FKpWBawxhD6a8xFzD85ldYPVlPEpY2/GuMYY6x6D5cth50549VXCwsN4Y+IbBNwM4PCXh/Hy8Hr6+8RBIrLD0Sb7EL2zWKn4ascOaxz+xo3vFwGA77d9z+wDsxnw9gAtApERgfHjIXlya3C6kBDc3dwZWnEop6+f5odtP9idMNppIVAqPgoOtgpA+vTWoHIOm09vpsPKDlTPXZ1OJTvZGDCOe+kl627jnTuhf38AyviUoUqOKgxYPyDedSfVQqBUfDRggNVIPG7c/SEkAm8FUntObbyTe/NzjZ9xE/3n/0Tvv2+1rwwYYM3jDAwqP4ird64yaMMgm8NFL/1NUCq+2bPH+iv2k0/gnXcAq13g00WfcjHoInNqzyFVolQ2h3QSI0ZYl4iaNYPwcAqkL0D9gvUZuXUkp6+dfuruzkILgVLxSWgoNGkCadLAyJH3F4/bMY6FhxYyqPwgimQsYmNAJ5MunXVpbfNma2wmoG+ZvhgM/f7qZ3O46KOFQKn4ZPRo67r2999bxQA4cOkA7Va0o1L2SnrT2IuoXx8qVICuXeHMGbKkzELTwk35effPnLp2yu500UILgVLxxdmz1t2xVapArVoA3Am9Q525dUiWIBk/19R2gRciYk16Hxp6f+6CLqWs0fYHrh9oc7joob8VSsUXbdtaX1Y//HD/xrEuf3Th74t/83PNn11nRNGYkC2bNWT1okUwbx7eKbxpUrgJP+36KV60FWghUCo+WL7cmmSmRw/rSwtYdmQZI7eOpM0bbaias+pT3kA9Vbt21sitbdtCUBBdS3UFYPDGwfbmigZaCJRydrdvW5cs8uSBjh0Bq6tok4VNeO2l1xhUPn51dbSNh4fVBnPmDPTvT5aUWWhUqBETdk7g7PXHjbPpHLQQKOXs+veH48dhzBhIaI0Z1GpZKy7fvsyv7/3qtMMhxEklS0LDhtYorocO0a10N8LCwxi+ZbjdyaJEC4FSzuzwYWsYiXr1oGxZwBpaeub+mfQu05uCGQo+5Q3Ucxs8GBInhi+/xCdFFj7K/xHjd4zn2p1rdid7YVoIlHJmHTpYk8x8+y0AF4Iu0HJZS4q9XIyvSn5lc7h4Kn166NsXVq2CefPoWLwjN4JvMH7HeLuTvTAtBEo5q99/hyVLrJm1MmTAGEOzxc24FXKLKTWn4OHmYXfC+KtlSyhQANq1o3DyXJTLWo6RW0cSHBZsd7IXooVAKWcUEmL1YsmRA1q3BmDKniksObyEAW8PcL15h2PbvYbj06dh4EA6lujI2RtnmbFvht3JXogWAqWc0ejR8M8/1vAHCRNy+tpp2vzehreyvEWbN9vYnc41lCpljec0dCiVPPKQ/6X8DN001ClnMYuWQiAilUXkkIj4i0iXSNY3EpFLIrLb8WgaYV1DETnieDSMjjxKxWuXLkHv3lCpErzzDsYYmi5uSlh4GJNrTNa7h2PTwIEggnTtSsfiHdkbsJc/jv1hd6rnFuXfGBFxB0YDVYB8QB0RyRfJpjONMYUcj4mOfVMDvYA3gGJALxHRYRGVepIePSAoyJpKUYRf//6VlUdXMqj8ILKlymZ3Otfi7W3duzFjBh/f9OGlJC/xw3bnm7gmOv50KAb4G2OOGWOCgRlAjWfctxKwyhhz2RhzBVgFVI6GTErFT7t3w4QJ8MUXkDcvATcDaLeiHSW8S9Dy9ZZ2p3NNnTtDxowk7NCZZoWbsvjQYk5cPWF3qucSHYXgZSDiYBtnHMse9YGI/C0ic0TE+zn3RUSai4ifiPhdunQpGmIr5WSMgfbtIXVq6NULgNbLWxMUHMTEdyfqJSG7JE1q3dS3dSufnc2Im7jxo9+Pdqd6LrH1m7MY8DHGFMD6q3/K876BMWa8McbXGOObLl26aA+oVJy3fDmsXWu1D6RKxeJDi5m5fyY9Svcgb7q8dqdzbQ0bQuHCePcYQs2c7zJx50Ruh9y2O9Uzi45CcBbwjvD6Fcey+4wxgcaYu46XE4Giz7qvUgprVNGvvrK6izZvzrU712ixtAX5X8pP51Kd7U6n3NysHlynT9PKPzWBtwOZuX+m3ameWXQUgu1AThHJKiIJgI+BRRE3EJGMEV5WBw46nq8AKopIKkcjcUXHMqVURFOmWPPmDhwICRLQdXVXzt04x8R3J5LAPYHd6RRAmTJQsyZlBs8gX6pc/LDtB6fpShrlQmCMCQW+wPoCPwjMMsbsF5G+IlLdsVlrEdkvInuA1kAjx76XgX5YxWQ70NexTCl1z82b8PXX8Oab8MEHrD+5nrF+Y2nzRhveeOUNu9OpiIYMQe4G0+pUBnac38H2c9vtTvRMxFkqVkS+vr7Gz8/P7hhKxY7+/a0uo+vXc+dNXwr+WJDgsGD2tdhHkgRJ7E6nHtWyJdd+GU/GLp7UL9iAce+OszvRfSKywxjj++hy7WagVFwWEGCNdlmzJpQqxcD1AzkceJhx74zTIhBXff01KUxCagdmYPq+6dwMvml3oqfSQqBUXNavH9y6BQOtAjBo4yDqvlaXitkr2p1MPU6GDNChA03nneBG8A1mH5htd6Kn0kKgVFx1+LA1aXqzZpjcuWm5tCWJPBLxXcXv7E6mnqZjR0reSkPuW4mZuHOi3WmeSguBUnFVjx7WjGO9ezNj3wxWH1/NgHIDdBJ6Z5A8OdKjJ59uuMXG0xs5eOng0/exkRYCpeKinTth9mxo356rKRLSfmV7fDP58lnRz+xOpp7V55/T4LI3HuEwaddPdqd5Ii0ESsVFPXpAqlTQoQM91vQg4GYAP1b7EXc3d7uTqWeVMCHpuw/g3UMwZduEOD1pjRYCpeKaDRus4SS6dMHv5hHGbB9Dq9dbUTRT0afvq+KWunX59EpWLoVdZ9nBhXaneSwtBErFJcZAt26QIQNhLVvw+ZLPSZ80Pf3K9rM7mXoRbm5UajOKdDfhtyUD7U7zWFoIlIpLVq6E9euhRw/GHpjCjvM7GFFpBCm8UtidTL0gjyrVqBOYkcW3dnHlyjm740RKC4FScYUx0L07+Phwvs47dF/TnQrZKlD71dp2J1NRIUL9mr0Idoc5E9rZnSZSWgiUiivmz4cdO6BXLzqu68bd0LuMrjoaEbE7mYqiou80J/etxPzmP9+aXS6O0UKgVFwQFmb1FMqThw3/82Ha3ml8VfIrcqbJaXcyFQ1EhPoF6vHXyyGc+D7utfdoIVAqLpg2DQ4eJKxvb9qs6sAryV+hc0mdZyA++aRaVwCmrRsF167ZnOZhWgiUsltwsDX1ZOHC/JztOjvP72RI+SE6qFw845PSh9KpC/NrzjuYEcPtjvMQLQRK2W3SJDh+nGt9utFtbQ9Kepfk4/wf251KxYB6JT7nn3Sw87ehEBhod5z7oqUQiEhlETkkIv4i0iWS9e1F5IBj8vrVIpIlwrowEdnteCx6dF+l4rW7d635BkqUoF+CzVy6eYlRVUZpA3E8VStfLTzEg1lZb8LQoXbHuS/KhUBE3IHRQBUgH1BHRPI9stkuwNcxef0cYEiEdbeNMYUcj+oo5UomT4YzZzj0VRNGbhvFp4U/pUjGInanUjEkdaLUlM9enlnFkmBGjYSLF+2OBETPGUExwN8Yc8wYEwzMAGpE3MAYs9YYc8vxcgvWJPVKubbgYGsO4jffpH3QPBJ7Juabt7+xO5WKYbXz1eaE5012pL4LgwbZHQeInkLwMnA6wuszjmWP8ymwPMJrLxHxE5EtIlLzcTuJSHPHdn6XLl2KUmCl4oQpU+DUKZa1qcIy/2V8/dbXpE+a3u5UKobVzFMTTzdPZtV+FcaOhbNn7Y4Uu43FIlIP8AW+jbA4i2MOzbrACBHJHtm+xpjxxhhfY4xvunTpYiGtUjEoJAQGDCD4DV/a/TuVXGly8eUbX9qdSsWCVIlSUSF7BWZlvIwJC7WmIrVZdBSCs4B3hNevOJY9RETKA92B6saYu/eWG2POOn4eA9YBhaMhk1Jx2y+/wIkT/NC8EIcDDzO80nASuCewO5WKJbXz1ebkzbNsb1YNxo+Hc/aOQRQdhWA7kFNEsopIAuBj4KHePyJSGBiHVQQCIixPJSIJHc/TAiWBA9GQKVJtlrehwfwGMfX2Sj2bkBDo35+AEgXpEzCLqjmrUjVnVbtTqVhUI08N6/JQ2XQQav9ZQZQLgTEmFPgCWAEcBGYZY/aLSF8RudcL6FsgKTD7kW6ieQE/EdkDrAUGGWNirBDcDbvLgn8WEBIWElOHUOrpfvsNjh+ne5303Aq5xbCKw+xOpGJZSq+UVMpRiVlnVmIaNrDOCs6fty1PtLQRGGOWGWNyGWOyG2P6O5Z9bYxZ5Hhe3hiT/tFuosaYTcaY14wxBR0/Y3Q+t/LZynMj+Abbz22PycMo9XihodC/PzvL5uGnwFW0eaMNudPmtjuVskHtfLU5ff00W5tXs84Shwx5+k4xxKXuLC7rUxZB+OPYH3ZHUa5q2jTM0aO0riqkTZyWnm/1tDuRskn13NXxdPNkzrXNUL8+/PijbWcFLlUI0iROQ+GMhVl9fLXdUZQrCg2Fb75hZtXMbLx5kAHlBuiEMy4shVcK3s76Ngv+WYDp1s06K/j226fvGANcqhAAlM9ans2nNxMUHPfGBFfx3IwZ3DxxhE4lb1IkYxEaF2psdyJls/fyvMfRK0c5kCIY6tWz7iu4cCHWc7hcISiXrRwh4SGsP7ne7ijKlYSFwTffMOS9lzgTEsjIyiNxd3O3O5WyWfXcVn+aBf8ssOajsOmswOUKQanMpUjgnkAvD6nYNXMmJy8cYsirV6iTvw6lMpeyO5GKAzImy8ibr7zJgkMLIEcO+OQT66wglscgcrlCkNgzMSW9S2qDsYo9jrOBTrWSI+4eDC5v/52kKu6ombsmfuf8OH3ttHVWcPdurJ8VuFwhACiXtRx7Lu7h0k0ds0jFgjlz+PPWQWZ7X6dLqS54p/B++j7KZdTMUxOARYcWQc6c1lnBmDEQEPDkHaORSxaC8tnKA7Dm+Bqbk6h4LzycsH59aVMzIZlTZKZTiU52J1JxTO60ucmTNo91eQhsOStwyUJQNFNRUiRMoZeHVMybO5eJXgfYk+ouQysMJZFnIrsTqTioZu6arDuxjiu3r0CuXFC3bqyeFbhkIfBw86CMTxltMFYxKzycKwN70b2iO//L/Ba18tWyO5GKo2rmqUloeCjLjiyzFvToAXfuxNosZi5ZCMC6PHT86nGOXTlmdxQVXy1YQJ/0B7mSMJwRVUbq9JPqsV5/+XUyJs3IwkMLrQW5c8PHH8Po0RAL86+4dCEAWHl0pc1JVLwUHs6B4d34oRg0K9KMQhkK2Z1IxWFu4ka1nNVYcXTFg0Exe/SA27dh+PCYP36MHyGOyp0mN1lSZOF3/9/tjqLiIbNwIe18DpHMIzH9dPpJ9Qyq5arG9bvX2Xh6o7Ugb1748EP44Qe4fDlGj+2yhUBEqJKjCquPryY4LNjuOCo+MYYl4zuyMgf0LtePdEl0Rj31dOWylsPTzfNBOwFYZwU3bsDIkTF6bJctBACVc1QmKDiIjac22h1FxSN3F86jXa5j5PXMREudflI9o2QJk/E/n/+x9MjSBwtfew3ee88qBNeuxdixXboQvJ31bTzdPPXykIo+xjByemuOpobhH0zA093T7kTKiVTNUZUDlw5w4uqJBwt79rSKwPffx9hxo6UQiEhlETkkIv4i0iWS9QlFZKZj/VYR8Ymwrqtj+SERqRQdeZ5VsoTJKJW5FL8f1UKgosf5Bb/RL/s53vUqSKXcOv2kej7VclUDYOnhCGcFhQvDO+9YjcY3bsTIcaNcCETEHRgNVAHyAXVEJN8jm30KXDHG5ACGA4Md++bDmuP4VaAyMMbxfrGmco7K/H3xb85ePxubh1XxkTF0W9qeux4wrNF0u9MoJ5QrTS5ypM7BMv9lD6/o2dNqMB4zJkaOGx1nBMUAf2PMMWNMMDADqPHINjWAKY7nc4ByYnWqrgHMMMbcNcYcB/wd7xczBg+GLg+fsFTJUQWAFUdXxNhhlWvYPmcUP3v/S7sUlciRPq/dcZSTqpqjKmuOr+FWyK0HC4sVg8qVrRvMbt6M9mNGRyF4GTgd4fUZx7JIt3FMdn8NSPOM+wIgIs1FxE9E/C696A0WJ07AsGFw6tT9Rflfys/LyV7WdgIVJeHhYbTe2IMMt9zo0Xyq3XGUE6uWqxp3Qu+w9vjah1f07AkpU8Lx49F+TKdpLDbGjDfG+BpjfNOle8HueF27Wj8HPxgGWESonKMyq46tIjQ8NBqSKlc0bVpXtqQKYmCGeiRLlsbuOMqJvZXlLRJ7Jn64GylAiRLwzz+QP3+0HzM6CsFZIOK4uq84lkW6jYh4ACmAwGfcN/pkzgyNG8PEiXDmzP3FlXNU5uqdq2w9szXGDq3ir6C7N+i8fwSvX0pAg5bj7I6jnJyXhxfls5Vn6ZGlGGMeXukeM02o0VEItgM5RSSriCTAavxd9Mg2i4CGjue1gDXG+oSLgI8dvYqyAjmBbdGQ6fG6doXw8IfOCspnK4+7uLPcf3mMHlrFTwN/+4xzXiGMzNUat4RedsdR8UCVHFU4ee0khwMPx8rxolwIHNf8vwBWAAeBWcaY/SLSV0SqOzb7CUgjIv5Ae6CLY9/9wCzgAPA70MoYExbVTE/k4wMNG8KECXDuHAApvVJSwrvEwzdyKPUMjl0+yncnZ1DvSCKKf6ZDSajoUTF7RSD2xkKLljYCY8wyY0wuY0x2Y0x/x7KvjTGLHM/vGGM+NMbkMMYUM8Yci7Bvf8d+uY0xsfMnebduEBr60FlB9dzV2X1hN6eunXrCjko9rOO0RniEGQa93g0SJrQ7joonsqXKRvZU2Vl1bFWsHM9pGoujVbZs0KABjB8P588D8G6udwFYfGixncmUE1l9bDXzAzfQbXcyXv6so91xVDxTMXtF1p5YGytjoblmIQDo3h1CQu5PB5c7bW5ypcnF4sNaCNTThYaH0nZeM7JegfYVe4GXtg2o6FUhWwWCgoPYcmZLjB/LdQtB9uzWJNE//ggXLwJQPVd11hxfw/W7120Op+K6cX7j2HfzON9tTYlX85Z2x1HxUNmsZXEXd1YdjfnLQ65bCOA/k0S/m/tdQsJDdLIa9USBtwLpuaorbx+DmrW/hkQ6D7GKfim9UvLGK2+w8ljMfx+5diHImdOaJHrsWAgIoIR3CVInSq2Xh9QT9VrXi2shNxi5PQ3y+ed2x1HxWMVsFdl+djuXb+vENDHr3iTR332Hh5sHVXNWZenhpXqXsYrU3ot7Gbt9LC22Q/5m3fVsQMWoCtkrYDCsPrY6Ro+jhSDiJNH//kv1XNUJvB3I5tOb7U6m4hhjDG1XtCVliBt996WDzz6zO5KK54q9XIzkCZPHeDdSLQRgnRXcugXffUelHJXwdPNk0aFHb45Wrm7+P/NZc3wNfVaFkrpNF0ic2O5IKp7zcPOgXNZyrDy68r/DTUQjLQRgTRJduzb88APJg0Io41OGRYe1EKgHbofcpsPKDrx2Mymfn0wH2jagYknF7BU5ee0kRy4fibFjaCG4p2dPa5zv4cOpkbsGhwMPc+DSAbtTqThi6KahnLh6gpGzg/Do1FnPBlSsqZCtAkCMdiPVQnDPq69CrVowahTvZSgDwNwDc+3NpOKE09dOM3DDQGr9+xJlb72kZwMqVmVLlY3MKTKz9sTap2/8grQQRNSzJ9y4QaaJMynpXZI5B+fYnUjFAZ1WdcKEhzH01wDo1AmSJLE7knIhIkK5rOVYe2It4SY8Ro6hhSCi116D99+HkSP5wKcKf1/8myOBMXddTsV9f538i5n7Z9L55CtkSZAOWrSwO5JyQW9nfZvLty+z58KeGHl/LQSP+vpruH6dD9b/C8Dcg3p5yFWFhYfRenlrMnul56tfjunZgLJNWZ+yAKw5viZG3l8LwaMKFoSaNck84mdeT19EC4ELm7BzAnsu7mHongwkTpEWWuqYQsoeLyd/mdxpcrP6eMzcWKaFIDJffw1Xr1IrIA1+5/w4efWk3YlULLt8+zLd13SnTKoi1Jq+R88GlO3KZS3HXyf/IiQsJNrfO0qFQERSi8gqETni+Jkqkm0KichmEdkvIn+LyEcR1v0sIsdFZLfjUSgqeaJN4cLw7rt8MNmaw1jPClzP12u/5uqdq4z8MxGSVs8GlP3ezvo2N0Nusjdgb7S/d1TPCLoAq40xOYHVjtePugU0MMa8ClQGRohIygjrOxljCjkeu6OYJ/r06kX2E9cpREYtBC5m78W9jPUbSwvv9ykwb6N1NpA0qd2xlIurkrMK/3b6lyIZi0T7e0e1ENQApjieTwFqPrqBMeawMeaI4/k5IABIF8XjxryiRaFaNT7YdIVNpzdx9vpZuxOpWGCMofXvrUnplZK+cy+Dng2oOCKxZ2LSJE4TI+8d1UKQ3hhz3vH8ApD+SRuLSDEgAXA0wuL+jktGw0XksZO+ikhzEfETEb9Lly5FMfYz6tOHWjvvADD7wOzYOaay1dyDc1l3Yh3fZGlM6qVr9GxAuQR52kBGIvIHkCGSVd2BKcaYlBG2vWKM+U87gWNdRmAd0NAYsyXCsgtYxWE8cNQY0/dpoX19fY2fn9/TNoseH3xA4QwLSPBaIbZ+viN2jqlscSvkFnlH5yWVVyp2zEqF+4GDcPSoNhKreENEdhhjfB9d/tQzAmNMeWNM/kgeC4GLji/ze1/qAY85eHJgKdD9XhFwvPd5Y7kLTAaKvdjHi0F9+lB3TzjbLu7E/7K/3WlUDBq8YTCnrp1i5EsNcV+7Drp10yKgXEJULw0tAho6njcEFj66gYgkAOYDvxhj5jyy7l4REaz2hX1RzBP98ufn4+w1AJi+ZYLNYVRM8b/sz+CNg6mTvw7/GzobXnkFmje3O5ZSsSKqhWAQUEFEjgDlHa8REV8RmejYpjbwFtAokm6iU0VkL7AXSAt8E8U8McK7+xDeOglTt0yI0THBlT2MMbRe3poE7gkY6l4FNm+2xp3y8rI7mlKxwiMqOxtjAoFykSz3A5o6nv8G/PaY/d+OyvFjTa5c1E1Wgs/dN7F7zwoKF6psdyIVjRYeWshy/+UMq/gdmVoNh2zZoHFju2MpFWv0zuJnVKvFD3iEwbRpkd0qoZzVzeCbtPm9Da+99BpfnvOGXbugVy/w9LQ7mlKxRgvBM0qTpzCVQ7MwPWwP4ceOPn0H5RQGrB/AqWunGF15FB69+kCePPDJJ3bHUipWaSF4DnUrf8XZ5LB+yBd2R1HR4NC/h/h207fUL1Cf0pvPwf790KcPuLvbHU2pWKWF4DlUf7MhSYwnv57/HQ4dsjuOigJjDF8u/5JEnon4tuxA63JQgQLWLHVKuRgtBM8hSYIkfJjnA2bmh5t9etgdR0XB3INzWXVsFd+U/Yb081aAvz/06wdu+k9CuR79rX9OjYu3ICgBzD0wB/bEzGxBKmYFBQfR9ve2FMpQiBavNYbeveH11+Hdd+2OppQttBA8p9KZS5MjRTYm+3pA1652x1EvoN+f/Th74yxjqo7BY+w4OH0aBg8GEbujKWULLQTPSURoVKQJ67xDObZlOaxda3ck9Rz2Bexj2JZhNC7UmOLJ8kL//lC5MpQta3c0pWyjheAFNCjYAEH4+a3k8NVXoHcbO4VwE07zxc1JkTAFQyoMsc4Crl6FQYPsjqaUrbQQvADvFN5UzF6Rn309CNvhB7N1iGpnMM5vHJvPbGZYpWGkvXIXRoyw7hkoWNDuaErZSgvBC2pcqDGnwy6z5m0fa5TKkOifR1RFn/M3ztNldRfKZS1H/QL1rQbi8HCrp5BSLk4LwQuqkacGqbxSMalGZmvM+vHj7Y6knqDN7224G3qXsdXGIv/8A5MmWTOP+fjYHU0p22kheEFeHl7UL1Cfede2EFC+OPTtCzdu2B1LRWLp4aXMPjCbHm/1IGeanA/mGeje3e5oSsUJWgii4HPfzwkOC2ZSk8IQEADDhtkdST0iKDiIlstaki9dPr4q+RVs2gQLFkDnztZ8xEopLQRRkTddXsr4lGFcwDLCan0A334LFy/aHUtF0GttL05dO8W4d8aRwM3T6uWVIQO0bWt3NKXijCgVAhFJLSKrROSI4+fj5isOizApzaIIy7OKyFYR8ReRmY7ZzJxKS9+WnLh6ghWtKsHdu1YjpIoTdp7fyYitI2hepDmlMpeyendt3Gg1EOsUlErd99TJ65+4s8gQ4LIxZpCIdAFSGWM6R7JdkDEmaSTLZwHzjDEzRORHYI8xZuzTjhurk9c/RUhYCJlHZKZoxqIs2ZINRo+2hp7In9/uaC4tJCyENya+wbkb5zjY6iCpJBHkzQspUsCOHTrCqHJJLzx5/VPUAKY4nk/Bmnf4WQMJ8DZwbx7j59o/rvB096Rp4aYsO7KME+2bWF807drpTWY2G7JxCLsu7GJMtTGkSpTKumfgxAmrHUeLgFIPiWohSG+MOe94fgFI/5jtvETET0S2iEhNx7I0wFVjTKjj9Rng5ccdSESaO97D79KlS1GMHb2aF22OiDDOf6Y1nv0ff8CSJXbHcln7A/bT96++1H61Nu/nfR8uXLCGkqheHd52jtlRlYpNTy0EIvKHiOyL5FEj4nbGusb0uD+DszhOR+oCI0Qk+/MGNcaMN8b4GmN806VL97y7xyjvFN5Uz12dCTsncPvThtYsVx06QHCw3dFcTmh4KE0WNSF5wuR8X+V7a2HPnlb7zdCh9oZTKo56aiEwxpQ3xuSP5LEQuCgiGQEcPwMe8x5nHT+PAeuAwkAgkFJEPBybvQKcjfInskm7N9sReDuQXw5Mty4/HDlitReoWDV883C2nd3G91W+56UkL1ntNT/9BF98ATlz2h1PqTgpqpeGFgENHc8bAgsf3UBEUolIQsfztEBJ4IDjDGItUOtJ+zuL0plLUzRjUYZvGU545UpQpYp1mSiOXcaKzw79e4iea3tSM09NPnr1I6udpn17SJ3aOitQSkUqqoVgEFBBRI4A5R2vERFfEZno2CYv4Ccie7C++AcZYw441nUG2ouIP1abwU9RzGMbEaF98fYcCjzEsiPL4LvvICjImgJRxbiw8DA+XfQpiT0TM6bqGEQEFi6ENWusLr2pIu3ZrJQiit1H7RKXuo9GFBIWQrZR2ciZOidrGq6B1q2ty0O7dlnz4aoYM3LLSNquaMuUmlNoULAB3LoF+fJBsmSwcyd4etodUSnbxVT3URWBp7snrYu1Zu2Jtew6v8v6SzR1amjVSruTxqCDlw7SZXUXquWsZo0sCjBwIJw8aRViLQJKPZEWgmjWrGgzkiZIyrAtw6wiMHgwbNgAv/xid7R4KTgsmHrz65HEMwkTq0+0Lgn5+8OQIdZcA2+9ZXdEpeI8LQTRLKVXSpoVacb0vdM5duUYNGoEJUpAp05w5Yrd8eKdfn/2Y+f5nYx/dzwZkmawzrxat4aECa2xn5RST6WFIAZ0LNERdzd3Bm0YBG5uMGYMBAbqsMfRbPPpzQzYMICGBRtaN46B1UC8fLk1LHjGjPYGVMpJaCGIAZmSZeLTwp/y8+6fOX3ttDUVYuvW8OOPsH273fHihaDgIOrPr493cm9GVh5pLbx1C9q0scZ5+uILewMq5US0EMSQziU7YzAM2TjEWtCnjzX8cYsWEBZmb7h4oOPKjhy7cowpNaeQwiuFtbBfPzh1ymog9vB48hsope7TQhBDsqTMQsOCDZmwcwLnb5yH5MmtO4537LDODNQLW3xoMeN2jKNjiY78z+d/1sI9e6w2gcaNtYFYqeekhSAGdS3VldDwUIZucoxx89FHUKECdOli/eWqntuZ62dotLARBdMXpF9Zx8TzoaHQtCmkSaPjCSn1ArQQxKDsqbNTr0A9xviN4cz1MyBiTXJvDHz+ud5b8JxCw0OpO7cud0PvMuvDWST0SGitGDUK/Pzg+++tLrtKqeeihSCG9S7Tm3ATTt8/+1oLfHxgwACrZ8tvv9mazdn0+7Mf60+tZ2y1seRKk8taeOwY9OgB774LH35ob0ClnJQWghjmk9KHFr4tmLRrEof+PWQtbNXKuregbVud4/gZrT2+ln5/9aNBwQbUL+i4e/jemZWHh9VFV8TekEo5KS0EsaBb6W4k8kxEj7U9rAXu7jBxojUo3Zdf2hvOCVy6eYlP5n1CrjS5GF01wtDekybBqlXWcBKvvGJfQKWcnBaCWPBSkpfoULwDcw7MYftZx30EefNaI5POng3z5tkbMA4LCw+j3vx6XL59mRm1ZpA0gWPq6+PHrTOqsmWtLrlKqRemhSCWtC/enrSJ09JxVUfuj/jaqRMUKQKffWZNp6j+o9e6Xqw8upJRVUZRKEMha2F4uNVNVAQmT7bu3lZKvTD9FxRLkidMzjdlv+Gvk38x+8Bsa6Gnp9VgHBQETZpoL6JHLPxnIf3X96dJoSY0K9LswYqRI+HPP62fWbLYF1CpeEILQSxqWqQphTIUouPKjtwKuWUtzJvXGilz+XK90SyCw4GHabCgAUUzFmV0tdHWqKIABw5A167WRPSNGtmaUan4IkqFQERSi8gqETni+PmfaaBEpKyI7I7wuCMiNR3rfhaR4xHWFYpKnrjO3c2dUZVHcfr6aQZvGPxgRatWUKmSNeH9oUP2BYwjgoKDeH/m+3i6eTK39ly8PLysFXfvQv361mQz48drLyGloklUzwi6AKuNMTmB1Y7XDzHGrDXGFDLGFALeBm4BKyNs0uneemPM7ijmifNKZynNx/k/ZsimIZy4esJa6OZm9YBJlAjq1YOQEFsz2inchNNoQSMO/nuQ6R9MJ0vKCJd+One2ZhubOBHSp7cvpFLxTFQLQQ1giuP5FKDmU7avBSw3xtyK4nGd2pDyQ3AXd1osbfGg4ThTJpgwwbpDtst/6qnL+Hrt18w9OJch5YdQIXuFBysWLbLaBFq3hho17AuoVDwU1UKQ3hhz3vH8AvC0P9M+BqY/sqy/iPwtIsNFJOHjdhSR5iLiJyJ+ly5dikJk+3mn8GZAuQH87v87U/dOfbDi/fet4ZOHDYP58+0LaJNf9/xK//X9aVq4Ke2Lt3+w4tQpqz2gSBGrPUUpFa2eOnm9iPwBZIhkVXdgijEmZYRtrxhj/tNO4FiXEfgbyGSMCYmw7AKQABgPHDXG9H1a6Lg6ef3zCAsPo9TkUhwJPMLBVgdJlySdteLuXShdGg4fti6DZMtmb9BYsuHUBsr9Uo6S3iX5vd7vJHBPYK0ICYEyZWDvXuu/R44ctuZUypm98OT1xpjyxpj8kTwWAhcdX+b3vtQDnvBWtYH594qA473PG8tdYDJQ7Hk/mLNyd3Nn4rsTuX73Om1XtH2wImFCmDXLagj98EO4c8e2jLHl2JVjvDfzPbKkyMKc2nMeFAGw7rXYtAnGjdMioFQMieqloUVAQ8fzhsDCJ2xbh0cuC0UoIoLVvrAvinmcyqsvvUr30t2Ztncacw/MfbDCxwemTLH+Av7yy3h9f8HFoItU/LUiYeFhLKm7hNSJIowe+uuvVrtAmzZQp459IZWK56JaCAYBFUTkCFDe8RoR8RWRifc2EhEfwBv485H9p4rIXmAvkBb4Jop5nE630t3wzeRLs8XNrKGq76leHbp1s3rI/PCDfQFj0PW716kytQrng86ztO7SByOKgtVo3qyZNYSETkKvVIx6ahtBXBQf2ggiOhJ4hMLjClPs5WKsqr8Kdzd3a0V4uNWAvHixdcNZxYr2Bo1Gd0LvUHVqVdafWs+ijxdRJWeVBysDAqBoUatbrZ8fpEtnX1Cl4pEXbiNQMS9nmpyMqjKKtSfW8u2mCH/9urlZQ1Dkzw+1a8ebm81Cw0P5ZN4nrD2xlsk1Jj9cBG7dsuYWCAy0ek5pEVAqxmkhiCMaF2pMrXy16LGmB2uPr32wImlSqw99ggRQrZrTz19wrwjMOziPEZVGUK9AvQcrw8Kgbl3Yvh2mTbO6iyqlYpwWgjhCRPip+k/kTJOT2nNqc+pahDmNs2SBhQvh3DmoWhWuX7cvaBSEhodSf359Zu2fxdAKQ2nzZpsHK42xbhZbuNCacrJmTdtyKuVqtBDEIckTJmfBRwu4G3qX92e+z+2Q2w9WFi8Oc+bAnj3w3nvW/QZOJDQ8lAbzGzBj3wyGlB9ChxIdHt5gwABrlrFOnayxl5RSsUYLQRyTO21ufnv/N3ac30HDBQ0JN+EPVlatao2/v2YNfPKJ04xJdCf0Dh/O/pDp+6YzsNxAOpXs9PAG331nzTtcvz4MGmRPSKVcmBaCOKh67up8W+FbZh+YTbvf2/FQz6769WHECJg71+pbH8eLwbU716j8W2UW/LOAUZVH0aXUI+MojR4NHTtajeGTJukkM0rZwMPuACpyHYp34Mz1M4zcOpJMyTLRuVTnByvbtLG6lrZvDx99BDNmWI3JccyFoAtUmVqFfQH7mPb+NOq89shNYT/8YN0wV6OG1TvKQ38dlbKD/suLo0SEYZWGcSHoAl1Wd8HLw+vhxtV27ay/ntu2hQ8+gJkzIXFi2/I+atf5XdSYUYPA24EsrrOYyjkqP1hpDPTrZ83ZXL26ld3T076wSrk4LQRxmJu48ct7vxAcFkzbFW0JM2EPj8rZpo11JtCqFbz9tnXjWRzodz/nwBwaLmhImkRp2NB4A4UzFn6w8t6ZzMiR0LChdee0ngkoZSu9IBvHJXBPwMxaM6mVrxYdVnag97reD7cZtGhhtRfs2QMlSoC/v21ZQ8JC6La6Gx/O/pCC6Quyrdm2h4vAjRtWt9CRI60zmUmTtAgoFQdoIXACnu6eTP9gOg0LNqTPn31ouKAhd0MjdB997z2rJ9GVK/D667BkSaxnPHn1JP/7+X8M3DCQpoWbsrbhWjIkjTB6+bFjVhfYZcustoFhw7RhWKk4Qv8lOgkPNw8m15hMv7L9+PXvXyn/a3nOXj/7YIPixa07crNmtYZo6N7dulM3hhlj+O3v3yg0rhD7L+1nxgczmFB9Agk9IswxNGeONXbQuXOwYoV1KUvnG1YqztBC4EREhB5v9WD6B9PZdX4XhcYVYtmRZQ82yJrVGrv/00+tG7RKl47R8YlOXj1J1WlVqT+/PnnT5mXXZ7v4KP9HDzYICoKmTa15FXLlsgpVuXIxlkcp9WK0EDihj/N/jF9zPzIly0S1adVotqgZl29ftlZ6eVkNsFOnwj//QKFC1jDOwcHRdvyg4CD6rOvDq2NeZf3J9YyqPIr1jdeTLVWE2dQWLoR8+ax2gG7dYMMGyJ492jIopaKRMcbpHkWLFjXKmFvBt0zHFR2Nex93k25IOjNxx0QTEhbyYIPz542pUcMYMCZnTmMWLjQmPDxKxxuzbYzJMDSDoTem1qxa5sSVEw9vtHevMe+8Yx3ztdeM2bTphY+nlIpegJ+J5DtVzwicWCLPRHxb8Vt2NN9B9tTZabq4KflG52PK7incCb0DGTJYQzkvXQru7taNWyVKwIIFVjfOZ3Qx6CL9/+qPz0gfWi5rSbZU2djUZBOzP5xNlpRZrI3+/tsaObRAAfjrL2uS+R07rLYLpVTcFll1eNYH8CGwHwgHfJ+wXWXgEOAPdImwPCuw1bF8JpDgWY6rZwT/FR4ebhb+s9AUGFvA0BuTZnAa03FFR7Pl9BYTFh5mTHCwMWPGGOPj8+AM4ZtvjDlxItL3u3Tzkpn691RTbWo1497H3dAbU+W3Kmbt8bUm/N5ZxZUrxvzyizGlSlnvmTixMV27GhMYGHsfXCn1zHjMGUGUZigTkbyOIjAO6GiM+c+0YSLiDhwGKgBngO1AHWPMARGZBcwzxswQkR+BPcaYsU87bnyboSw6hZtw1h5fy1i/sSw8tJDQ8FDSJ0lP6SyleT3T6+RLnZuXtxwg3bQFeGzZBsDlQrkJKF6AQ7lS83eiG2wN+oedF3ZhMLyS/BXqvVaPBq/VI29oSjh4EDZvtv7qX7cOQkOta/8tW0KjRpA69ZPiKaVs9LgZyqJlqkoRWcfjC0FxoLcxppLjdVfHqkHAJSCDMSb00e2eRAvBs7ly+wrL/Zez5PAStp7dyrErx566T7K7UPgClD+fiAqXU1HsUgLc7gbDv/8+aHAWsRqCq1a1ptIsVkzvCVDKCTyuEMTGbZ0vA6cjvD4DvAGkAa4aY0IjLH/5cW8iIs2B5gCZM2eOmaTxTKpEqaj7Wl3qvlYXgMBbgfhf9ufcjXP8e+tfwkwYxhhSJUpFOo/k5LgYQubjVxA3f0h0BVJfhZxi9URKk8bqnpojh3XTWooU9n44pVS0eWohEJE/gAyRrOpujFkY/ZEiZ4wZD4wH64wgto4bn6RJnIY0idM8foM8wP9iLY5SKo54aiEwxpSP4jHOAt4RXr/iWBYIpBQRD8dZwb3lSimlYlFsXNjdDuQUkawikgD4GFjkaMFeC9RybNcQiLUzDKWUUpYoFQIReU9EzgDFgaUissKxPJOILANw/LX/BbACOAjMMsbsd7xFZ6C9iPhjtRn8FJU8Simlnl+09BqKbdprSCmlnt/jeg1pnz+llHJxWgiUUsrFaSFQSikXp4VAKaVcnFM2FovIJeBkDLx1WuDfGHjf2OLs+cH5P4Oz5wfn/wzOnh9i7jNkMcake3ShUxaCmCIifpG1qDsLZ88Pzv8ZnD0/OP9ncPb8EPufQS8NKaWUi9NCoJRSLk4LwcPG2x0gipw9Pzj/Z3D2/OD8n8HZ80MsfwZtI1BKKRenZwRKKeXitBAopZSL00LwCBHpJyJ/i8huEVkpIpnszvQ8RORbEfnH8Rnmi0hKuzM9LxH5UET2i0i4iDhNN0ARqSwih0TEX0S62J3neYnIJBEJEJF9dmd5ESLiLSJrReSA4/enjd2ZnoeIeInINhHZ48jfJ9aOrW0EDxOR5MaY647nrYF8xpjPbY71zESkIrDGMQ/0YABjTGebYz0XEckLhAPjeMxc2HGNiLgDh4EKWNOubgfqGGMO2BrsOYjIW0AQ8IsxJr/deZ6XiGQEMhpjdopIMmAHUNNZ/h+IiABJjDFBIuIJbADaGGO2xPSx9YzgEfeKgEMSwKkqpTFmZYR5oLdgzfzmVIwxB40xh+zO8ZyKAf7GmGPGmGBgBlDD5kzPxRjzF3DZ7hwvyhhz3hiz0/H8Btb8J4+dBz2uMZYgx0tPxyNWvn+0EERCRPqLyGngE+Bru/NEQRNgud0hXMTLwOkIr8/gRF9C8Y2I+ACFga02R3kuIuIuIruBAGCVMSZW8rtkIRCRP0RkXySPGgDGmO7GGG9gKtbsanHK0/I7tukOhGJ9hjjnWT6DUi9CRJICc4G2j5zhx3nGmDBjTCGsM/liIhIrl+ieOnl9fGSMKf+Mm04FlgG9YjDOc3tafhFpBLwDlDNxtBHoOf4fOIuzgHeE1684lqlY5Li2PheYaoyZZ3eeF2WMuSoia4HKQIw33rvkGcGTiEjOCC9rAP/YleVFiEhl4CugujHmlt15XMh2IKeIZBWRBMDHwCKbM7kUR2PrT8BBY8wwu/M8LxFJd6+Xn4gkwup4ECvfP9pr6BEiMhfIjdVr5STwuTHGaf6yExF/ICEQ6Fi0xZl6PQGIyHvA90A64Cqw2xhTydZQz0BEqgIjAHdgkjGmv72Jno+ITAfKYA2BfBHoZYz5ydZQz0FESgHrgb1Y/34BuhljltmX6tmJSAFgCtbvjxswyxjTN1aOrYVAKaVcm14aUkopF6eFQCmlXJwWAqWUcnFaCJRSysVpIVBKKRenhUAppVycFgKllHJx/wcKAQ9FMbNJgQAAAABJRU5ErkJggg==\n" + }, + "metadata": { + "needs_background": "light" + } + } + ], + "source": [ + "def predict_sin6(x):\n", + " return linear_layer.bias.item()+x*linear_layer.weight[0,0].item()+x**2 * linear_layer.weight[0,1].item() + x**3 * linear_layer.weight[0,2].item()\n", + "\n", + "# print(predict_sin5(2))\n", + "import matplotlib.pyplot as plt \n", + "plt.plot(x,y,c='r')\n", + "plt.plot(x,predict_sin6(x),c='g')\n", + "plt.show()" + ] + }, + { + "cell_type": "code", + "execution_count": 90, + "metadata": {}, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "99 560.31494140625\n", + "199 382.79241943359375\n", + "299 262.6877746582031\n", + "399 181.34494018554688\n", + "499 126.19588470458984\n", + "599 88.7654037475586\n", + "699 63.33283615112305\n", + "799 46.033226013183594\n", + "899 34.252540588378906\n", + "999 26.221017837524414\n", + "1099 20.73921775817871\n", + "1199 16.993431091308594\n", + "1299 14.430941581726074\n", + "1399 12.675918579101562\n", + "1499 11.472537994384766\n", + "1599 10.646467208862305\n", + "1699 10.07874870300293\n", + "1799 9.688159942626953\n", + "1899 9.41912841796875\n", + "1999 9.233620643615723\n", + "y = 0.01773737370967865 + 0.8454191088676453 x + -0.0030599930323660374 x^2 + -0.09171997010707855 x^3\n" + ] + } + ], + "source": [ + "# 7 自定义nn模块。进行forward、backward、optimizer\n", + "import torch\n", + "import math \n", + "\n", + "class Polynomial3(torch.nn.Module):\n", + " def __init__(self):\n", + " super().__init__()\n", + " self.a = torch.nn.Parameter(torch.randn(()))\n", + " self.b = torch.nn.Parameter(torch.randn(()))\n", + " self.c = torch.nn.Parameter(torch.randn(()))\n", + " self.d = torch.nn.Parameter(torch.randn(()))\n", + " \n", + " def forward(self,x):\n", + " return self.a+self.b*x+self.c*x**2+self.d*x**3\n", + " \n", + " def string(self):\n", + " return f'y = {self.a.item()} + {self.b.item()} x + {self.c.item()} x^2 + {self.d.item()} x^3'\n", + "\n", + "\n", + "x = torch.linspace(-math.pi, math.pi, 2000)\n", + "y = torch.sin(x)\n", + "\n", + "model = Polynomial3()\n", + "criterion = torch.nn.MSELoss(reduction='sum')\n", + "optimizer = torch.optim.SGD(model.parameters(),lr=1e-6)\n", + "\n", + "for t in range(2000):\n", + " y_pred = model(x)\n", + " loss = criterion(y_pred,y)\n", + " if t % 100 == 99:\n", + " print(t,loss.item())\n", + " \n", + " optimizer.zero_grad()\n", + " loss.backward()\n", + " optimizer.step()\n", + "\n", + "print(model.string())\n", + "\n", + "# 主要实现了在不同等级上的抽象。本层通过model构建了神经网络模型实现了前向传播、自动梯度计算。使用torch.nn.MSELoss计算了损失。使用了torch.optim.SGD进行梯度下降。 " + ] + }, + { + "cell_type": "code", + "execution_count": 91, + "metadata": {}, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "1999 33.18156814575195\n", + "3999 20.196273803710938\n", + "5999 14.039202690124512\n", + "7999 11.19318675994873\n", + "9999 10.00029468536377\n", + "11999 9.34005355834961\n", + "13999 9.059755325317383\n", + "15999 8.731332778930664\n", + "17999 8.888145446777344\n", + "19999 8.854000091552734\n", + "21999 8.832571029663086\n", + "23999 8.844922065734863\n", + "25999 8.65311336517334\n", + "27999 8.838438034057617\n", + "29999 8.85017204284668\n", + "Result: y = -4.497888221521862e-05 + 0.8571422100067139 x + -0.0005409775767475367 x^2 + -0.09365808218717575 x^3 + 0.00013697342365048826 x^4 ? + 0.00013697342365048826 x^5 ?\n" + ] + } + ], + "source": [ + "# 8 控制流和权限共享\n", + "\n", + "# -*- coding: utf-8 -*-\n", + "import random\n", + "import torch\n", + "import math\n", + "\n", + "class DynamicNet(torch.nn.Module):\n", + " def __init__(self):\n", + " \"\"\"\n", + " In the constructor we instantiate five parameters and assign them as members.\n", + " \"\"\"\n", + " super().__init__()\n", + " self.a = torch.nn.Parameter(torch.randn(()))\n", + " self.b = torch.nn.Parameter(torch.randn(()))\n", + " self.c = torch.nn.Parameter(torch.randn(()))\n", + " self.d = torch.nn.Parameter(torch.randn(()))\n", + " self.e = torch.nn.Parameter(torch.randn(()))\n", + "\n", + " # 这里的d,e的样本值是随机的，表示这两个参数锁对应的样本值对模型不会产生正向影响。所以最终结果应该是0\n", + " # 这个实验也说明了，可以在正向和反向传播的任何波分添加任何控制流。甚至可以跳过。也就是说，构建普通神经网络，不需要像构建卷积神经网络那样，以层为单位构建。而是单纯的构建神经元和计算图。\n", + " def forward(self, x):\n", + " \"\"\"\n", + " For the forward pass of the model, we randomly choose either 4, 5\n", + " and reuse the e parameter to compute the contribution of these orders.\n", + "\n", + " Since each forward pass builds a dynamic computation graph, we can use normal\n", + " Python control-flow operators like loops or conditional statements when\n", + " defining the forward pass of the model.\n", + "\n", + " Here we also see that it is perfectly safe to reuse the same parameter many\n", + " times when defining a computational graph.\n", + " \"\"\"\n", + " y = self.a + self.b * x + self.c * x ** 2 + self.d * x ** 3\n", + " for exp in range(4, random.randint(4, 6)):\n", + " y = y + self.e * x ** exp\n", + " return y\n", + "\n", + " def string(self):\n", + " \"\"\"\n", + " Just like any class in Python, you can also define custom method on PyTorch modules\n", + " \"\"\"\n", + " return f'y = {self.a.item()} + {self.b.item()} x + {self.c.item()} x^2 + {self.d.item()} x^3 + {self.e.item()} x^4 ? + {self.e.item()} x^5 ?'\n", + "\n", + "# Create Tensors to hold input and outputs.\n", + "x = torch.linspace(-math.pi, math.pi, 2000)\n", + "y = torch.sin(x)\n", + "\n", + "# Construct our model by instantiating the class defined above\n", + "model = DynamicNet()\n", + "\n", + "# Construct our loss function and an Optimizer. Training this strange model with\n", + "# vanilla stochastic gradient descent is tough, so we use momentum\n", + "criterion = torch.nn.MSELoss(reduction='sum')\n", + "optimizer = torch.optim.SGD(model.parameters(), lr=1e-8, momentum=0.9)\n", + "for t in range(30000):\n", + " # Forward pass: Compute predicted y by passing x to the model\n", + " y_pred = model(x)\n", + "\n", + " # Compute and print loss\n", + " loss = criterion(y_pred, y)\n", + " if t % 2000 == 1999:\n", + " print(t, loss.item())\n", + "\n", + " # Zero gradients, perform a backward pass, and update the weights.\n", + " optimizer.zero_grad()\n", + " loss.backward()\n", + " optimizer.step()\n", + "\n", + "print(f'Result: {model.string()}')\n", + "\n", + "# 主要是用来控制封装好的过程。模型中参数可以自己定义。使用库中的函数，函数的参数会自动参加训练。使用自己的函数，就需要声明自己的参数。" + ] + }, + { + "cell_type": "code", + "execution_count": 95, + "metadata": {}, + "outputs": [ + { + "output_type": "display_data", + "data": { + "text/plain": "
", + "image/svg+xml": "\r\n\r\n\r\n\r\n \r\n \r\n \r\n \r\n 2021-03-23T17:47:18.760876\r\n image/svg+xml\r\n \r\n \r\n Matplotlib v3.3.2, https://matplotlib.org/\r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n\r\n", + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAD4CAYAAADhNOGaAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAABAHElEQVR4nO3dd3gUVRfH8e9JQhJ6Db2FovQioehroQmhgwhSRMACCkgRUKogHZWqdAQBUXoXBAQUAVFClU7oHaRIIELK3vePWSBgqCmTzZ7P8+xD9s7szm8V9mTm3rlXjDEopZRyXx52B1BKKWUvLQRKKeXmtBAopZSb00KglFJuTguBUkq5OS+7AzyNDBkymNy5c9sdQymlXMrWrVv/Nsb43d/ukoUgd+7cBAUF2R1DKaVciogcj65dLw0ppZSb00KglFJuTguBUkq5OS0ESinl5rQQKKWUm4uVQiAiU0TkgojsfsB2EZHRIhIsIrtE5Lko25qLyCHno3ls5FFKKfX4YuuM4Fsg8CHbqwH5nY9WwDgAEUkH9AHKAmWAPiKSNpYyKaWUegyxch+BMWa9iOR+yC51gOnGmvN6s4ikEZEsQHlgtTHmMoCIrMYqKD/ERi6lVAISGgqHDsHBg3DhAly7BuHhkCwZpEwJuXNDvnzcyJaR8/9e5MKNC4SGh+IpniRNkpSsKbOSOUVmvDxc8vanBC2+/otmA05GeX7K2fag9v8QkVZYZxPkzJkzblIqpWKPwwEbNsDSpfDLL7Btm9UWdReBXZlgVV7YmMP6+dhDrgl4iieFMxamdNbSlM9dnur5q5Muabq4/RxuwGVKqzFmIjARICAgQFfTUSqhOnECxoyBmTPh9Gnw9oZy5aBHDyhaFJ59ll3el5l+dDHf75vD2etnAXg2WU7Kmky8c85B9v2nyRh8juRhEFm4IKG1AjlTMi/Hrp9m+7ntLNq/iG+2f4OneFLBvwKtnmtF3QJ1SeKZxOYP75riqxCcBnJEeZ7d2XYa6/JQ1PZf4imTUio27dsH/frB3LnW8+rV4YsvoFYtSJGCSEckiw8s5otN77P51GaSeCShev7qvFbwNSrnqUzWlFnvfb/Dh2HePJg2DT4YAX5+8NFH8OE8HMmSEnQmiMX7FzPzr5k0nNeQLCmy8Mn/PqF1QGt8vXzj//O7MmNMrDyA3MDuB2yrAawABCgH/OlsTwccBdI6H0eBdI86VqlSpYxSKoE4e9aYVq2M8fAwJmVKYzp3NubYsTubHQ6HmbN7jnnmq2cMfTF5RuUxozaPMhdvXHy893c4jFm92phq1YwBYzJnNmbcOGMiIowxxkRERphlB5aZCt9WMPTFZB+e3Xy38zvjcDji4tO6NCDIRPcdHV3jkz6wOnfPAuFY1/nfAd4H3nduF2AMcBj4CwiI8tq3gWDno+XjHE8LgVIJQGSkMePHW1/+Xl7GtG9vzIUL9+yy6cQmU25yOUNfTJGxRcyc3XNMRGTE0x9zwwZjXn7Z+uoqW9aYHTvu2bzmyBpTemJpQ19M5emVzaFLh57+WIlQnBaC+H5oIVDKZkePGlOhgvUVUrGiMQcP3rP5n5v/mA+WfWDoi8nyZRYzeevkmBWAqBwOY2bONMbPzxhPT2P69DEmPPzO5ojICDPmzzEm1eBUJvnA5Gbajmmxc9xEQAuBUip2LFpkTJo0xqRKZczEidYXcxTLDy432YdnN9JXTMcVHU3IrZC4yXHpkjHNmllfYy+9ZMzx4/dsPvnPSfPK1FcMfTHNFjQzoWGhcZPDhTyoEOgUE0qpxxMRAV26QN26kDcv7NgB770HIgCERYbReWVnqn9fnVQ+qdj0ziZGBI4ghXeKuMmTLh1Mnw7ffQfbt0OJErB27Z3N2VNlZ81ba+j7Sl++2/UdFaZV4Pz183GTxcVpIVBKPdq1a1CzJgwbBm3awMaN4O9/Z/PRK0d5aepLDN88nLal27K11VbKZS8XP9maNrUKQdasUKUKjBt3Z5Onhyd9yvdhfsP57Dq/i7KTy7L34t74yeVCtBAopR7u1Cl46SX4+WeYNMm6R8DH587mdUfXUWpiKQ78fYB5DebxdfWv43/4Zr58sGkTBAZahapDh3tuXqtXsB7rW67nVuQtyn9bnl3nd8VvvgROC4FS6sEOHLBuBjt6FJYvh3ffvWfzxK0TqfJdFbKkzMK21tuoX6i+TUGBVKlg8WLo2BFGj4bmza0pLJwCsgawvsV6vD29qTCtAtvObrMvawKjhUApFb3du+GVV6wv0w0brMsuTpGOSDr91InWy1pTOU9lNr29iTxp89gY1snTE4YPh0GDrL6D11+HmzfvbM6fPj+/tviVFN4pqDy9sl4mctJCoJT6r+3boXx564v111+hWLE7m8Ijw3lz4ZuM/GMk7cu0Z2njpaT2TW1f1vuJQPfu1iWsJUuszu1bt+5szpsuL+uar8PHy4eq31Xl5D8nH/xebkILgVLqXnv2QOXKkDw5rF8PBQrc2RQaHkrd2XWZtXsWQyoNYVS1UQl3NtA2bWDyZFi5Et54457LRHnS5mFF0xVcu3WNqt9V5fK/l20Maj8tBEqpu44etS4B+fjAunXWMFGnf27+Q+B3gaw4tIIJNSfwyYuf2Bj0Mb3zDnz1ldV30KwZREbe2VQicwkWN1rM4SuHaTi3IRGOCBuD2ksLgVLKcu4cvPoq/PsvrFoFee5e8w+5FULgzEB+P/U7P9T/gValWtkY9Am1aweffw6zZ1s/m7uTF5fPXZ7xNcaz5ugaPl79sY0h7ZVAz+mUUvEqJMQaenn2rDVMtEiRO5tuhN2gxvc12HJ6C3MbzKVewXo2Bn1KXbvCpUswdKh1/8PHd7/0W5ZsyfZz2xmxeQQlMpfgreJv2RjUHnpGoJS7i4yEJk2sUULz58Pzz9/ZFBoeSq0farHx5Ea+r/+9axaB2wYNgkaN4JNPrLODKIZVGUaF3BVovaw1uy9Eu/R6oqaFQCl39/HHsGyZNfY+8O7S42GRYbw2+zV+OfYL0+pOo2HhhjaGjAUeHjB1Krz4Irz1ljUk1imJZxJ+qP8DqX1S02heI/4N/9fGoPFPC4FS7mzSJGvcfbt21igbJ4dx8Pbit1l5eCWTak3izWJv2hgyFvn6Wh3HuXJB/frWXdNOmVJkYnq96ey5uIfOqzrbGDL+aSFQyl1t3Gh9+VetCiNG3LOp28/dmPnXTAZUGMA7z71jU8A4ki4dLFoEoaFWMYhyw1mVvFXo8nwXxgWNY9H+RbZFjG9aCJRyR+fPQ8OGkDs3zJoFXnfHjYzaPIovNn1Bm4A29Hiph30Z41KhQtbMpX/+CW3b3jOSaGClgZTMXJLWy1pzKfSSjSHjT6wUAhEJFJEDIhIsIt2i2T5CRHY4HwdF5GqUbZFRti2JjTxKqYeIiIDGjeHyZWtN4DRp7myau2cunVZ2ol6BeoyuNhpxTjGdKNWrB716wZQpMGHCnWZvT2+m1pnK5X8v03FlR/vyxafoFil4kgfgibUEZR7AG9gJFHrI/h8CU6I8v/6kx9SFaZSKge7drcVcpk69pznodJDxHeBrnp/8vPss4hIZaa2F7O1tzPbt92zqs66PoS9m6YGl9mSLA8ThwjRlgGBjzBFjTBgwC6jzkP0bY61xrJSKbz/+CIMHW7OItmhxp/lsyFnqzKpDxuQZWfjGQpImSWpfxvjk4WFdIsqQwZqG4vr1O5t6vNSDIhmL0HpZa67dumZjyLgXG4UgGxB11qZTzrb/EJFcgD+wNkqzr4gEichmEan7oIOISCvnfkEXL16MhdhKuZlz56wv/+LFrWkXnP4N/5e6s+ty5eYVljRaQqYUmezLaIcMGeD77yE42Bo95eTt6c2U2lM4G3KWvr/0tS9fPIjvzuJGwDxjTGSUtlzGmACgCTBSRPJG90JjzERjTIAxJsDPzy8+siqVeDgcVhG4ft360vO1Fo4xxvDe0vf48/SffFfvO4pnLm5vTru88gr07g3TpsGMGXeaS2crzXvPvcfoP0Yn6hvNYqMQnAZyRHme3dkWnUbcd1nIGHPa+ecR4BegZCxkUkpF9fXX1iycw4ZZI2acvtz05Z1hoi5913Bs6NULXn4ZPvgADh++0zyo0iBS+6am7fK2t/s1E53YKARbgPwi4i8i3lhf9v8Z/SMiBYC0wO9R2tKKiI/z5wzA/wBdKUKp2PTXX9bdwzVqWF9yTr8e+5Vua7rxeqHXE+8w0Sfh5WUtZuPlZZ09OWcqTZ8sPYMqDmL98fX8sDtxdm/GuBAYYyKAdsBKYB8wxxizR0T6iUjtKLs2AmaZe0tqQSBIRHYC64AhxhgtBErFlps3rXmEUqe2hkk6h4OeDTnLG/PeIH+6/EypPSVxDxN9EjlyWFNtbNhwz0127z73LqWylKLr6q7cCLthY8C4Ia54qhMQEGCCgoLsjqFUwtejhzVK6McfoXp1wFphrNL0Smw9u5U/3/2TwhkL2xwygTHGusdgxQrYtg0KW/99fjv+Gy9/+zIDKgyg58s9bQ75dERkq7NP9h56Z7FSidXWrdY8/C1b3ikCAD3W9OC3E78xseZELQLREYGJEyFVKmtyOufKZi/leok6z9Zh6MahXLyRuEYuaiFQKjEKC7MKQKZM1qRyTgv3LeTL37+kTUAbmhZramPABC5jRutu423bYODAO81DKg8hNDyU/uv72xgu9mkhUCoxGjTI6iSeMOHOFBLHrx6n5eKWlM5amuFVhz/89Qpee83qXxk0yFrHGSiQoQDvlHyHcUHjCL4cbHPA2KOFQKnEZudO67fYpk2hZk0AIhwRNF3QFIdxMOv1Wfh4+dgc0kWMHGldInrvPeteDKBv+b54e3rTe11ve7PFIi0ESiUmERHw9tuQPj2MGnWneeD6gWw8uZHxNceTJ22eh7yBuoefn3Vp7fffYdw4ALKkzEL7Mu2ZvXs2+y7uszlg7NBCoFRiMmaMdV37q6+sYgBsOLGBfuv70axYM5oUbWJzQBfUrBm8+ip0735nIZvOL3QmWZJkiaavQAuBUonF6dPW3bHVqsHrrwNw9eZVmi5oin8af8ZUH2NzQBclAuPHW2dbzrULMiTLwIdlPmTW7lmJ4qxAC4FSiUXHjtaX1ddfgwjGGFova82ZkDN8X/97UvqktDuh68qTBz77DJYsgQULgMR1VqCFQKnEYMUKa5GZXr2sLy1g+s7pzNkzh/4V+lMmWxmbAyYCnTpZM7d27AjXryeqswItBEq5un//tS5ZFCgAXboAcPKfk7T/qT0v53qZri90tTlgIuHlZfXBnDp1596C22cFgzcMtjlczGghUMrVDRwIR4/C2LHg44MxhreXvE2kI5Jv63yLp4en3QkTj//9D5o3t2ZxPXCADMky8O5z7/LD7h84+c/JR78+gdJCoJQrO3jQmkbizTehQgUAxgeN5+cjPzO86nD80/rbHDARGjoUkiWDDz8EY+hUrhPGGEZuHml3sqemhUApV9a5s7XIzBdfAHD48mG6rO5C1bxVee+592wOl0hlygT9+sHq1bBgAbnS5KJRkUZM3DaRqzev2p3uqWghUMpV/fQTLFtmrayVOTORjkhaLG6Bt6c339T+RqeWjktt2kCxYlYH8o0bdH2hK9fDrjM+aLzdyZ6KFgKlXFF4uPUllC8ftG8PwMjNI9lwYgOjA0eTLVW0y4ar2HK74/jkSRg8mOKZi1MlbxVG/TGKWxG37E73xLQQKOWKxoyB/fut6Q98fNh7cS891/akboG6vFnsTbvTuYcXX7Tmc/rySzh+nI9f+Jhz188x86+Zdid7YrFSCEQkUEQOiEiwiHSLZnsLEbkoIjucj3ejbGsuIoecj+axkUepRO3iRejbF6pWhZo1iXRE8s6Sd0jhnYLxNcbrJaH4NHiwdedxt25U9K9I0YxF+erPr1xubeMYFwIR8QTGANWAQkBjESkUza6zjTElnI/JztemA/oAZYEyQB8RSRvTTEolar16wfXr1lKKIozZMobNpzYzMnAkmVJksjude8mRw7p3Y9YsZPNm2pVpx45zO9h4cqPdyZ5IbJwRlAGCjTFHjDFhwCygzmO+tiqw2hhz2RhzBVgNBMZCJqUSpx07YNIkaNcOChbk+NXj9FjTg2r5qtG0qC40Y4tPPoEsWaBTJ5oWaUIa3zR8/efXdqd6IrFRCLIBUe+kOOVsu199EdklIvNEJMcTvhYRaSUiQSISdPFi4lomTqnHYgx89BGkSwd9+tyZS0hEGF9TLwnZJkUK66a+P/4g+YKlvF3ibebvm8+ZkDN2J3ts8dVZvBTIbYwphvVb/7QnfQNjzERjTIAxJsDPzy/WAyqV4K1YAevWWf0DadMyY9cMVh5eyeBKg8mZOqfd6dxb8+ZQsiR88gltilp3dbvSUNLYKASngRxRnmd3tt1hjLlkjLk9pmoyUOpxX6uUwppV9OOPreGirVpx4cYFOq3sxAs5XqBN6TZ2p1MeHtYIrpMnyTt1EdXzV2fC1gkuM5Q0NgrBFiC/iPiLiDfQCFgSdQcRyRLlaW3g9lR9K4EqIpLW2UlcxdmmlIpq2jRr3dzBg8Hbm/Yr2nM97DqTa03GQ3QUeIJQvjzUrQuDB/Nh/qZcuHGBBfsW2J3qscT4b5AxJgJoh/UFvg+YY4zZIyL9RKS2c7f2IrJHRHYC7YEWztdeBvpjFZMtQD9nm1Lqths34NNPoVw5qF+fpQeWMnvPbHq/3JuCfgXtTqei+vxzuHWLV7/9Df80/kzePtnuRI9FXG28K0BAQIAJCgqyO4ZS8WPgQGvI6G+/EVK6OAXHFCRd0nQEtQrC29Pb7nTqfm3awKRJDJzzIb12jSD4w2DypstrdyoARGSrMSbg/nY9p1QqIbtwwZrtsm5dePFF+vzShzMhZ5hYa6IWgYTq00/B25sW8w7hIR58s/0buxM9khYCpRKy/v0hNBQGD2bHuR2M/mM0rUq1olz2cnYnUw+SOTN07ky275dRw+9/TN0xlfDIcLtTPZQWAqUSqoMHrUXT33sPx7PP8P6y90mXNB2DK7n2alhuoUsXyJCBd9de5dz1cyw/tNzuRA+lhUCphKpXL/Dxgb59mbR1En+c/oNhVYaRNqnOwpLgpUoFvXpRfcFfZEmSjknbJtmd6KG0ECiVEG3bBnPnwkcfcSGF0G1NN8rnLq8zi7qS99/HK2duWu5JworgFZy6dsruRA+khUCphKhXL0ibFjp3psuqLtwIu8G4GuN0GglX4uMD/fvzzorzOIyD6Tun253ogbQQKJXQbNhgTSfRrRvrLm9jxq4ZfPy/jymQoYDdydSTatKEPDmL89J5H2bsnJ5gp6fWQqBUQmIM9OgBmTNz6/33+ODHD8iTNg89X+ppdzL1NDw8YPBgmv1xi/2XDrD17Fa7E0VLC4FSCcmqVfDbb9CrF1/uGMuBSwcYU30MSZMktTuZelqBgTRIWRafCJixbardaaKlhUCphMIY6NkTcufmyOuVGPDbAF4v9DqB+XSJDpcmQpq+Q6h1AH7YNj1B3lOghUCphGLhQti6Ffr0ocsvPfAUT0ZUHWF3KhUbypfnLYpx0Vxn5e5Fdqf5Dy0ESiUEkZHWSKECBVjzv6ws3L+QHi/1IHuq7HYnU7EksMPXZLgBM5b0tzvKf2ghUCoh+P572LePiH596bD6I/zT+PPR8x/ZnUrFoiT/e4lG13OxOOwvrp4/bnece2ghUMpuYWHQpw+ULMmEnBfZc3EPw6oMw9fL1+5kKpY1q9+PW14wf3x7u6PcQwuBUnabMgWOHuVS30/o/cunVPSvSN0Cde1OpeJA6YrNyHsrOXOO/QiXLtkd545YKQQiEigiB0QkWES6RbP9IxHZ61y8fo2I5IqyLVJEdjgfS+5/rVKJ2q1b1noDL7xAH6/1/HPrH0YFjtI7iBMpEaFhscasyRnJxS/72R3njhgXAhHxBMYA1YBCQGMRKXTfbtuBAOfi9fOAz6Ns+9cYU8L5qI1S7mTqVDh1ir+6vMW4oPF8EPABRTIWsTuVikNvlG9HpAcs+HUcnD9vdxwgds4IygDBxpgjxpgwYBZQJ+oOxph1xphQ59PNWIvUK+XewsJg8GBMubJ0vDaH1D6p+az8Z3anUnGsWKZiPJvSn9nPhMOQIXbHAWKnEGQDTkZ5fsrZ9iDvACuiPPcVkSAR2SwidR/0IhFp5dwv6OLFizEKrFSCMG0anDjBog9fZe2xtfSv0J/0ydLbnUrFMRGhYck3+TW3cG76WDh92u5I8dtZLCJvAgHAF1GacznX0GwCjBSRaBf3NMZMNMYEGGMC/Pz84iGtUnEoPBwGDeJmuQA6X5xJYb/CtA5obXcqFU/eKPwGDjHMzx9hLUVqs9goBKeBHFGeZ3e23UNEKgM9gdrGmFu3240xp51/HgF+AUrGQqZoBZ0JYmXwyrh6e6Ue3/TpcOwYw98pxNGrRxkVOAovDy+7U6l4UjhjYQr7FWZ2BT+YOBHOnLE1T2wUgi1AfhHxFxFvoBFwz+gfESkJTMAqAheitKcVER/nzxmA/wF7YyFTtD5d9ykdV3aMq7dX6vGEh8PAgZx+sRiDLsynXoF6VMpTye5UKp41LNyQDb4XOJ003PazghgXAmNMBNAOWAnsA+YYY/aISD8RuT0K6AsgBTD3vmGiBYEgEdkJrAOGGGPirBBU8q/E/r/3c/qa/dfklBv77js4epTuDdMT7gjnyypf2p1I2eCNwm9gMMxrXto6Kzh71rYssXIuaoxZDiy/r+3TKD9XfsDrNgFFYyPD47j9W9fao2tpVrxZfB1WqbsiImDgQDZXepYZl9fR/cXu5Embx+5UygbPZniWohmLMj+jgw7h4fD55zDCnkkG3erO4mKZipEhWQZ+Pvqz3VGUu/r+exxHDtO+qoMsKbLQ46UedidSNqpXoB4bLgRxofnrMH68bWcFblUIPMSDCrkrsObImgS7ZJxKxCIiYMAAZtTMwZbQQwytPJQU3insTqVsVK9gPQyGpQ2KW31HX3zx6BfFAbcqBACV81TmdMhpDl46aHcU5W5mzSLk+CG6lQ2hbLayNC3W1O5EymbFMxUnd5rcLLy8Ed58E8aNg3Pn4j2H2xWCSv5WP8Gao2tsTqLcSmQkDBjAwPp+nIu4yuhqo/EQt/vnp+4jItQrUI/VR1YT8klH284K3O5vYp60eciVOhc/H9F+AhWPZs8m+OIBRhS4QvPizSmTrYzdiVQCUa9APcIiw1jhOAhNm1pnBfE8B5HbFQIRoZJ/JdYdW0ekI9LuOModOM8GOr+eEu8kvgyuNNjuRCoBeSHHC/gl82Ph/oXWKnW3bsX7WYHbFQKwhpFevXmV7ee22x1FuYN581gVto8lWUPo9VIvsqTMYncilYB4enhS+9na/HjwR27557TOCsaOhQsXHv3iWOKeheB2P8ER7SdQcczhIHxAPzrV9iFv2rx0LNfR7kQqAapXoB4hYSGsO7bOlrMCtywEmVJkokjGIno/gYp78+czLvle9qa+xbAqw/Dx8rE7kUqAKuWpRArvFCzctxCeeQaaNInXswK3LARgnRVsOLGBmxE37Y6iEiuHg7+HfEqfSh686l+Z2s/quksqer5evlTLV42lB5da9zj16gU3b8KX8TP9iFsXgpsRN9l0cpPdUVRitWgRn2bZT4g3jAgcqctPqoeq+UxNzl4/a/VdPvssNGoEY8ZAPKy/4raFoHzu8nh5eOm01CpuOBzsHNmdCaWgTek2FM5Y2O5EKoGrlq8agrDs4DKroVcv+PffeJl/yG0LQUqflLyY80V+OvyT3VFUImQWL6ZjnoOk9UrBZxUSziLlKuHyS+5HmWxl+PHQj1ZDwYLQoAF8/TVcvhynx3bbQgAQmDeQXed3cSbE3kUhVCJjDAsmd+YXf+hfZTBpk6a1O5FyETXy12DL6S2cv+68oaxXLwgJgVGj4vS4bl0IquWvBsCqw6tsTqISk38Xz6PzM0cpmiQ775V+3+44yoXUfKYmBsOKYOey7kWLQr16ViH45584O65bF4KiGYuSJUUWfgrWy0MqlhjDsFkdOJ4GRjWcqstPqidSInMJsqbMevfyEEDv3lYR+OqrODturBQCEQkUkQMiEiwi3aLZ7iMis53b/xCR3FG2dXe2HxCRqrGR5wlyE5gvkFWHV+l0EypWnFo0ncF5z1I/aSkq5It2PSalHkhEqJ6vOiuDVxIWGWY1liwJNWtancYhIXFy3BgXAhHxBMYA1YBCQGMRKXTfbu8AV4wx+YARwFDnawthrXFcGAgExjrfL94E5gvkys0rbDmzJT4PqxIjY/hkxUdEegpftPze7jTKRdV8piYhYSFsOLHhbmPv3laH8dixcXLM2DgjKAMEG2OOGGPCgFlAnfv2qQNMc/48D6gk1qDqOsAsY8wtY8xRINj5fnFj6FDodu8JS+U8lfEQD708pGJs49xhfJ/tMl3TVMff7xm74ygXVSlPJbw9vfnxYJTLQ2XKQGCgdYPZjRuxfszYKATZgJNRnp9ytkW7j3Ox+3+A9I/5WgBEpJWIBIlI0MWnvcHi2DEYPhxOnLjTlC5pOspmK6uFQMWIwxFJh819yXbDk26tZtgdR7mwFN4pKJ+7PMsOLbt3Q+/ekCYNHD0a68d0mc5iY8xEY0yAMSbAz8/v6d6ke3frz6FD72kOzBfIn6f/5FLopRimVO7q25ld2Zr6BkOzvkXyFDpcVMVMjfw1OHjpIIcvH77b+MILsH8/FCkS68eLjUJwGsgR5Xl2Z1u0+4iIF5AauPSYr409OXNCy5YweTKcOnWnOTBfIAbD6iOr4+zQKvG6dvMfuu/7mhfOe9Pkg3F2x1GJQGC+QABWHr5v5gPPuOlCjY1CsAXILyL+IuKN1fm75L59lgDNnT+/Dqw11urxS4BGzlFF/kB+4M9YyPRg3buDw3HPWUGpLKVInzQ9yw8tj9NDq8RpwHetuOATzqgCnRAfnV1UxVz+dPnJnSZ3vN3jFONC4Lzm3w5YCewD5hhj9ohIPxG5Pd3iN0B6EQkGPgK6OV+7B5gD7AV+AtoaY+J2HGfu3NC8OUyaBGesO4o9PTyplr8aK4JX6DBS9UQO/n2AkSfn0vJAMgJaf2Z3HJVIiAhV8lRh7dG1hEeGx/nxYqWPwBiz3BjzjDEmrzFmoLPtU2PMEufPN40xDYwx+YwxZYwxR6K8dqDzdc8aY1bERp5H6tEDIiLuOSuo9Uwt/g79m82nNsdLBJU4dP6hBb7hhkHleoKeDahYVDVfVULCQuLlO8llOotjVZ488NZbMHEinD0LQNW8VfHy8GLJgfuvaikVvZ+Cf2LZ5c303p6KzK0+sjuOSmQq+lfEQzzi5fKQexYCgJ49ITz8znJwqX1T80quV1h6cKnNwZQrCI8Mp9PC1uS7BO0D+4Cvr92RVCKTxjcNZbOVZdURLQRxJ29ea5Ho8ePhvDXTX61narHv7333DtlSKhpjtoxhf+gJRvyRBp9WH9gdRyVSVfNWZcvpLXE+tN19CwH8Z5HoWs/WAtCzAvVQF29cpO+a3lQNhhqNPoWkSe2OpBKpKnmrYDCsObomTo/j3oUgf35rkehx4+DCBfKkzUNhv8LaT6AeqtfaXtwIu8GILemR93WaaRV3SmcrTWqf1HHeT+DehQDuLhI9bBhgXR767cRvXL151d5cKkHacW4Hk7ZNot0fhoKte+rZgIpTXh5eVM5TmZWHV1qL2scRLQRRF4n++29qPVuLCEeEzj2k/sMYQ4efOpA+3ItP9/pB69Z2R1JuoEreKpy6dor9f++Ps2NoIQDrrCA0FIYNo2y2smRIlkH7CdR/zN07l/XH1zPgp3DSduwGyZLZHUm5gap5rWVa/jPdRCzSQgDWItENG8LXX+N55So18tdg+aHl8XJHn3INoeGhdFnVheLXU/DuKT/QvgEVT3KlyUW+dPnitMNYC8FtvXtb83yPGEHdAnW5evMqvxz7xe5UKoEYumEoJ6+d5Ks51/Hs+omeDah4Vcm/Er8e+5UIR0ScvL8WgtsKF4bXX4fRo6marjTJkyRn/r75dqdSCcCxq8f4fNPnNLqYiZduZtSzARXvKvlXIiQshC2n42YlRS0EUfXuDSEhJP16AtXzV2fh/oU6CZ2iy6ouiMPw+Yzz0LUrJE9udyTlZir4VwCIs8tDWgiiKloUXnsNRo3i9ZyBXLhx4d51Q5XbWXt0LfP3zafHsRzk8PGDD/QuYhX/MiTLQInMJbQQxJtPP4Vr16j+UzC+Xr56eciNRTgi6PBTB3L7ZqHz9GA9G1C2quRfiU0nNxEaHhrr762F4H7Fi0PduqQYOY6qOSuyYN8CHMZhdyplg/FB49l9YTfDd2YiaZoM0KaN3ZGUG6vkX4nUPqnjZC40LQTR+fRTuHqV+kd9OR1ymj9Px+2iaSrh+Tv0b3qv602ldAHU/WGHng0o21XJW4VzXc5RNFPRWH/vGBUCEUknIqtF5JDzz/+s2i0iJUTkdxHZIyK7ROSNKNu+FZGjIrLD+SgRkzyxpmRJqFWLWmPXkMQjCfP36uUhd9N7bW9CboUwap0PkkHPBpT9PD088ZC4+d09pu/aDVhjjMkPrHE+v18o8JYxpjAQCIwUkTRRtnc1xpRwPnbEME/s6dOHNOf/oZLJzfx98+N0ng+VsOw4t4MJWyfQNmd9Ci/caJ0NpEhhdyyl4kxMC0EdYJrz52lA3ft3MMYcNMYccv58BrgA+MXwuHGvVCmoUYP6q09z9OpRdpzbYXciFQ+MMbRf0Z70ydLTd94l0LMB5QZiWggyGWPOOn8+B2R62M4iUgbwBqL2dgx0XjIaISIPXPRVRFqJSJCIBF28eDGGsR/TZ59Rd1soXngwe8/s+DmmstXsPbP57cRvDMzZkrQ/rtGzAeUW5FGXPETkZyBzNJt6AtOMMWmi7HvFGPOffgLntizAL0BzY8zmKG3nsIrDROCwMabfo0IHBASYoKCgR+0WO+rXp3rKJewtloUjnY7F2TU6Zb8bYTcoMKYAfsn82DI7NZ5798Hhw9pJrBINEdlqjAm4v/2R32rGmMrGmCLRPBYD551f5re/1C884OCpgB+BnreLgPO9zxrLLWAqUObpPl4c+uwzmmyP4HjISX4/+bvdaVQcGrJhCKeunWJ0xuZ4rvsFevTQIqDcQkx/vV0CNHf+3BxYfP8OIuINLASmG2Pm3bftdhERrP6F3THME/uKFKFO0Qb4hsP3f062O42KI4cuHeLzTZ/TpEgTXvxiNmTPDq1a2R1LqXgR00IwBHhVRA4BlZ3PEZEAEbn9rdkQeBloEc0w0Zki8hfwF5ABGBDDPHEi5acDqH1QmPvXbJ2aOhEyxvDhig/x8fThS89q8Pvv1rxTvr52R1MqXjyyjyAhitc+AqdF7SpTz28NK6pOJ7Bcs3g9topbC/YtoP6c+oyoMpyObWfAP//A/v2QJInd0ZSKVU/dR6As1TqNIfVN+GFRgjxpUU/pRtgNOv7UkaIZi9LuTHbYvh369NEioNyKFoLH5JP3WV43BVngcZB/D+2zO46KJQPWD+DktZOMDfwKrz6fQYEC0LSp3bGUildaCJ5A4/p9uO4Dy0bqDUaJwf6/9zPs92E0L96cF38/DXv2wGefgaen3dGUildaCJ5A+VKvk8WRnBlXfoEDB+yOo2LAGEO75e1IliQZQ8sPtC4HFStmrVKnlJvRQvAEPD08afZcS5bnh3P9P7E7joqBuXvnsuboGgZWHEimBSshOBj69wcP/Seh3I/+rX9CLf/XlkgP+O7oYti50+446imE3Aqh08pOlMxckveLtIC+faF0aahVy+5oStlCC8ETKpChAOUyB/Dtcx6Y7tFNtqoSun6/9uNMyBnG1hiL57jxcPIkDB0KInZHU8oWWgieQsuA99iTwUHQzp9g3Tq746gnsPvCbkb+MZJ3S75LuRQFYOBACAyEChXsjqaUbbQQPIU3Cr9BUq+kTH0pOXz8MbjgTXnuyGEctF7WmtQ+qRlcebB1FnD1KgwZYnc0pWylheAppPZNzWsFX+OHwoabO4Jg7ly7I6nHMHHrRDad3MSwKsPIcOUWjBxp3TNQvLjd0ZSylRaCp9SyREuuOkJZ9GoOa5bKcJ2DKCE7G3KWbj93o6J/Rd4q/pbVQexwWCOFlHJzWgieUgX/CuRMnZMpr2aw5qyfONHuSOohOvzUgZsRNxlfYzyyfz9MmWKtPJY7t93RlLKdFoKn5CEevFvyXVZf205wYBno1w9CQuyOpaLx48Efmbt3Lr1e7kX+9PnvrjPQs6fd0ZRKELQQxMA7z72Dp3gyoXF+uHABhg+3O5K6z/Ww67RZ3oZCfoX4+H8fw6ZNsGgRfPKJtR6xUkoLQUxkTZmVugXqMvXcT9xsUA+++ALOn7c7loqiz7o+nPjnBBNqTsDbI4k1yitzZujY0e5oSiUYMSoEIpJORFaLyCHnnw9arzgyyqI0S6K0+4vIHyISLCKznauZuZQPAj7g0r+XmPvu83DrltUJqRKEbWe3MfKPkbR6rhUv5nzRGt21caPVQaxLUCp1R4wWphGRz4HLxpghItINSGuM+c8kPCJy3RiTIpr2OcACY8wsERkP7DTGjHvUce1YmOZBjDEUGFOADMkysHFHKRgzxpp6okgRu6O5tQhHBOUml+PUtVPsa7uPtJIUChaE1Klh61adYVS5pbhamKYOMM358zSsdYcfN5AAFYHb6xg/0esTChHh/VLvs+nkJna1a2B90XTqpDeZ2ezLTV+y9exWvqr2FWmTprXuGTh2zOrH0SKg1D1iWggyGWPOOn8+B2R6wH6+IhIkIptFpK6zLT1w1RgT4Xx+Csj2oAOJSCvnewRdvHgxhrFjV/MSzfH18mXcwe+t+ex//hmWLbM7ltvad3EffX7pQ/2C9WlQuAGcO2dNJVG7NlSsaHc8pRKcRxYCEflZRHZH86gTdT9jXWN60K/BuZynI02AkSKS90mDGmMmGmMCjDEBfn5+T/ryOJUuaToaF2nM9F3Tudy8obXKVefOEBZmdzS3E+mI5O0lb5PSOyVjqo+xGnv3tvpvvvzS3nBKJVCPLATGmMrGmCLRPBYD50UkC4DzzwsPeI/Tzj+PAL8AJYFLQBoR8XLulh04HeNPZJOO5ToSGh7KpF3fWpcfDh2y+gtUvBq5eSSbT21mdLXRZEqRyeqv+eYbaNcO8ue3O55SCVJMLw0tAZo7f24OLL5/BxFJKyI+zp8zAP8D9jrPINYBrz/s9a6iWKZiVM5TmdF/jiasSiWoVs26TJTALmMlZgcvHaTXul7UfrY2jYs0tvppPvoI0qWzzgqUUtGKaSEYArwqIoeAys7niEiAiEx27lMQCBKRnVhf/EOMMXud2z4BPhKRYKw+g29imMdWH5X7iDMhZ5i7Zy4MGwbXr1tLIKo45zAO3lnyjtVXU2McIgKLF8PatdaQ3rTRjmxWShHD4aN2SUjDR6NyGAdFxhYhaZKkBL0XhHToYF0e2r7dWg9XxZnRf4ymw08dmFpnKi1KtIDQUChUCFKmhG3bIEkSuyMqZbu4Gj6qovAQDzqV68S2s9tYf3y99ZtounTQtq0OJ41DBy8dpPua7gTmC6R5ceeVysGD4fhxqxBrEVDqobQQxLI3i71JhmQZGPb7MKsIDB0KGzbA9Ol2R0uUIhwRNFvYDB9PHybXmmxdEgoOhs8/t9YaePlluyMqleBpIYhlSZMkpV3pdiw9uJS/zv8FLVrACy9A165w5Yrd8RKdQb8N4s/TfzK+5niypcpmnXm1bw8+PtbcT0qpR9JCEAc+LPshKb1TMmjDIPDwgLFj4dIlnfY4lm05vYV+v/ajadGmNCzc0GpcvBhWrLCmBc+Sxd6ASrkILQRxIF3SdLQt3ZbZu2dz4O8D1lKI7dvD+PGwZYvd8RKF0PBQ3lz4JllSZuHr6l87G0OhQwdrnqd27ewNqJQL0UIQRzo93wlfL1+GbHQujP7ZZ9b0xx98AJGR9oZLBD5e/TEHLx1kWt1ppPFNYzX27w8nTlgdxF5eD329UuouLQRxJGPyjLQu1ZoZO2dw7OoxSJXKuuN461brzEA9tZ+Cf2LMljF0KteJiv7OuYN27rT6BFq21A5ipZ6Q3kcQh05fO02e0XloUbwFE2pNsDoyq1aF33+HPXsgZ067I7qcsyFnKT6+OBmTZySoVRC+Xr4QEQHPP2+dDezbZ43WUkr9h95HYINsqbLx3nPvMWXHFIIvB4OItci9MfD++3pvwROKdETy5sI3uR52nTkN5lhFAGD0aAgKgq++0iKg1FPQQhDHer3cC29Pbz5d96nVkDs3DBpkjWz57jtbs7maIRuGsPboWr6q9hWF/ApZjUeOQK9eUKsWNGhgb0ClXJQWgjiWOUVmOpbtyA+7f2DHuR1WY9u21r0FHTvqGsePacOJDXz6y6c0LtKYt0u+bTXePrPy8rKG6IrYG1IpF6WFIB50/V9X0vqmpeda530Enp4webI1Kd2HH9obzgVc/vcyjec3xj+NP+NrjrfuHgaYMgVWr7amk8ie3d6QSrkwLQTxII1vGrq92I3lh5bz2/HfrMaCBa2ZSefOhQUL7A2YgDmMgxaLWnD++nlmvT6LVD6prA1Hj1pnVBUqWENylVJPTQtBPGlXph1ZU2aly+ouOIzDauzaFZ57Dlq3tpZTVP8x+LfBLD24lC+rfElAVudgB4fDGiYqAlOnWndvK6Wemv4LiifJkiRjSKUh/Hn6T2bsnGE1JklidRhfvw5vv62jiO7zU/BP9F7XmyZFm/BhmSiX0EaNgl9/tf7Mlcu+gEolEloI4lHTYk0pm60s3dZ0I+RWiNVYsKA1U+aKFXqjWRRHrxylyfwmFM1UlIk1J97tF9i7F7p3txaib9HC1oxKJRYxKgQikk5EVovIIeef/1kGSkQqiMiOKI+bIlLXue1bETkaZVuJmORJ6DzEg9HVRnPu+jkG/jbw7oa2ba0bzTp3hgMH7AuYQISGh/LanNcwGBY0XEBy7+TWhlu3oFkza7GZiRN1lJBSsSSmZwTdgDXGmPzAGufzexhj1hljShhjSgAVgVBgVZRdut7ebozZEcM8CV6ZbGVoUaIFIzaP4NClQ1ajh4c1AiZpUnjzTQgPtzekjYwxtF7Wmh3ndvBdve/Imy7v3Y2ffGKtNjZ5MmTKZF9IpRKZmBaCOsA058/TgLqP2P91YIUxJjSGx3VpgyoOwtfLl/d/fJ87U3xkzQqTJll3yHb7Tz11G4M3DOa7Xd/Rr3w/ajxT4+6GJUusPoH27aFOHfsCKpUIxbQQZDLGnHX+fA541K9pjYAf7msbKCK7RGSEiPg86IUi0kpEgkQk6OLFizGIbL8sKbPweeXPWXt0Ld/u+Pbuhtdes6ZPHj4cFi60LZ9d5u2dR8+1PWlStAm9Xu51d8OJE1Z/wHPPWf0pSqlY9chJ50TkZyBzNJt6AtOMMWmi7HvFGPOffgLntizALiCrMSY8Sts5wBuYCBw2xvR7VGhXmXTuYRzGQflvy7P7wm72tt1L5hTO/8S3bsFLL8HBg9ZlkDx57A0aT7ac3sIr375CicwlWNt87d15hMLDoXx5+Osv679Hvny25lTKlT31pHPGmMrGmCLRPBYD551f5re/1C885K0aAgtvFwHne581llvAVKDMk34wV+UhHkysNZEb4Tdov6L93Q0+PjBnjtUR2qAB3LxpX8h4cvKfk9SeVZtMKTKxqNGiu0UArHstNm2CCRO0CCgVR2J6aWgJ0Nz5c3Ng8UP2bcx9l4WiFBHB6l/YHcM8LqVAhgL0frk3c/fOZfbu2Xc35M4N06ZZvwF/+GGivr/g8r+XqTazGjfCbrC08VIyJs94d+OMGVa/QIcO0LixfSGVSuyMMU/9ANJjjRY6BPwMpHO2BwCTo+yXGzgNeNz3+rXAX1gF4DsgxeMct1SpUiaxCIsIM2UnlTVphqQxx68ev3djjx7GgDGjR9sTLo7dCLthXvjmBePd39usObLm3o1bthjj42NMhQrGhIXZE1CpRAYIMtF8p+rCNAnA4cuHKTGhBKWylGLNW2vw9PC0NjgcVgfy0qXWDWdVqtgbNBaFR4ZTb3Y9lh9aztwGc6lfqP7djRcuQKlS1rDaoCDw87MvqFKJiC5Mk4DlTZeXr6p9xa/Hf+WLTV/c3eDhYU1BUaQINGyYaG42cxgH7y59lx8P/ci4GuPuLQKhodbaApcuWSOntAgoFee0ECQQzYs3p2HhhvRa24t1R9fd3ZAihTWG3tsbatRw+fULHMZB66Wtmb5zOv3K96N1QOu7GyMjoUkT2LIFvv/eGi6qlIpzWggSCBFhUq1J5E+fnzfmvcHJf07e3ZgrFyxeDGfOQPXqcO2afUFjwGEcvL/sfSZvn0zPl3ree6+AMdbNYosXW0tO1q1rW06l3I0WggQklU8qFr2xiJsRN3ltzmvcjIgydPT552HePNi5E+rVs+43cCEO4+CDZR8wadskerzYg/4V+t+dSA6s5TvHjrWGi7Zta19QpdyQFoIE5tkMzzKj3gyCzgTRcnHLu2sXgHU2MHUqrF0LTZu6zJxEEY4I3lnyDhO3TaT7i90ZUHHAvUVg2DBr3eFmzWDIEPuCKuWmtBAkQHUK1GFo5aHM2j2Lrqu63ruxWTMYORLmz7fG1ifwYhAaHkq92fX4dse39H2lLwMrDry3CIwZA126WJ3hU6boIjNK2cDL7gAqel1f6Mqpa6cYvnk42VJl46PnP7q7sUMHa2jpRx/BG2/ArFlWZ3ICc+XfK9SeVZuNJzYytvpYPih935KSX39t3TBXp441OspL/zoqZQf9l5dAiQgjqo7gTMgZOq/qTFKvpPd+kXbqZP323LEj1K8Ps2dDsmS25b3fwUsHqf1DbY5cOcKs12fRsHDDuxuNgf79rTWba9e2sidJYl9YpdycFoIEzNPDk5mvzSRsbhhtlrfBYRy0LROlI7VDB+tMoG1bqFjRuvEsAYy7X3V4FW/MewNP8WR1s9W8kvuVuxtvn8mMGgXNm1trC+iZgFK20guyCZyPlw/zGs6jzrN1aLeiHUM3DOWeu8E/+MDqL9i5E154AYKDbcvqMA6+3PQl1WZWI0eqHGx5b8u9RSAkxBoWOmqUdSYzZYoWAaUSAC0ELsDb05s5DebQqEgjuq3pxvvL3ifCEXF3h3r1rJFEV65A6dKwbFm8Z7xw4wI1vq9B19VdqVugLhvf3oh/Wv+7Oxw5Yg2BXb7c6hsYPlw7hpVKIPRfoovw9vRm5msz6f5idyZum0j1mdW5eCPKAj3PP2/dkevvb03R0LOndaduPFhxaAXFxxdn3dF1jK0+lnkN5pHSJ+XdHebNs+YOOnMGVq60LmXpesNKJRhaCFyIh3gwqNIgvqn9DeuPr6fEhBL8euzXuzv4+1tz97/zjnWD1ksvxen8RBdvXOTNBW9S/fvqpEuajj/f+5MPSn9wd3jo9evw7rvWugrPPGMVqkqV4iyPUurpaCFwQW+XfJs/3v2DFN4pqDi9Ip1XduZ62HVro6+v1QE7cybs3w8lSsAXX0BYWKwdPzwynLFbxlJwTEHm7JlDn1f6sK3VNoplKnZ3p8WLoVAhqx+gRw/YsAHy5n3wmyql7BPd3NQJ/ZGY1iOIiWs3r5nWS1sb+mJyjshp5u6ZayIdkXd3OHvWmDp1rDUN8uc3ZvFiYxyOpz5eRGSEmbtnrsk/Or+hL+aVqa+Y3ed337vTX38ZU7OmdcyiRY3ZtOmpj6eUil08YD0CPSNwYSl9UjK+5ng2tNxAKp9UNJjbgICJASzev5hIRyRkzmxN5fzjj+Dpad249cILsGiRNYzzMYWGhzIhaAIFxxSkwdwGJPFMwtLGS1nXfB2FMxa2dtq1y5o5tFgxWL/eWmR+61ar70IplbBFVx0e9wE0APYADiDgIfsFAgeAYKBblHZ/4A9n+2zA+3GOq2cE/xURGWGm7Zhm/Ef6G/pico3IZQb8OsAc+PuAtUNYmDFjxxqTO/fdM4QBA4w5diza97sZftOsCl5lWixqYVIOSmnoiwmYGGBm755twiPDrZ2uXDFm+nRjXnzRes9kyYzp3t2YS5fi50MrpZ4IcbFCmYgUdBaBCUAXY8x/lg0TEU/gIPAqcArYAjQ2xuwVkTnAAmPMLBEZD+w0xox71HET2wplsSk8MpzFBxYzdstY1h2z1jUokKEAr+R6hdJZS1Mo3TNkX78Tvymz8PptI2LgckBhLrxSigOFM7MrxQ3++DeY9Sd/IzQ8lJTeKWlQqAEtir/Fi155kf374fffrd/6f/kFIiKsa/9t2kCLFpAuna2fXyn1YA9aoSxWlqoUkV94cCF4HuhrjKnqfN7duWkIcBHIbIyJuH+/h9FC8HiOXz3O0oNLWXZwGZtPbeafW/888jVioMDfUPmML69eSk3lM74kDQ2Hv/++2+EsYnUEV69uLaVZpozeE6CUC3hQIYiP2zqzAVFWWeEUUBZr4furxpiIKO3ZHvQmItIKaAWQM2fOuEmayORKk4t2ZdrRrkw7HMbB4cuHOXT5EKeuneLv0L+JdERiMKT1TYtfktTku2QodOoWySJOgLkCSa9CDrFGIqVPbw1PzZfPumktdWq7P55SKpY8shCIyM9A5mg29TTGLI79SNEzxkwEJoJ1RhBfx00sPMSD/Onzkz99frujKKUSmEcWAmNM5Rge4zSQI8rz7M62S0AaEfFynhXcbldKKRWP4uPC7hYgv4j4i4g30AhY4uzBXge87tyvORBvZxhKKaUsMSoEIlJPRE4BzwM/ishKZ3tWEVkO4Pxtvx2wEtgHzDHG7HG+xSfARyISjNVn8E1M8iillHpysTJqKL7pqCGllHpyDxo1pGP+lFLKzWkhUEopN6eFQCml3JwWAqWUcnMu2VksIheB43Hw1hmAv+PgfeOLq+cH1/8Mrp4fXP8zuHp+iLvPkMsY43d/o0sWgrgiIkHR9ai7ClfPD67/GVw9P7j+Z3D1/BD/n0EvDSmllJvTQqCUUm5OC8G9JtodIIZcPT+4/mdw9fzg+p/B1fNDPH8G7SNQSik3p2cESinl5rQQKKWUm9NCcB8R6S8iu0Rkh4isEpGsdmd6EiLyhYjsd36GhSKSxu5MT0pEGojIHhFxiIjLDAMUkUAROSAiwSLSze48T0pEpojIBRHZbXeWpyEiOURknYjsdf796WB3pichIr4i8qeI7HTm/yzejq19BPcSkVTGmGvOn9sDhYwx79sc67GJSBVgrXMd6KEAxphPbI71RESkIOAAJvCAtbATGhHxBA4Cr2Itu7oFaGyM2WtrsCcgIi8D14Hpxpgidud5UiKSBchijNkmIimBrUBdV/l/ICICJDfGXBeRJMAGoIMxZnNcH1vPCO5zuwg4JQdcqlIaY1ZFWQd6M9bKby7FGLPPGHPA7hxPqAwQbIw5YowJA2YBdWzO9ESMMeuBy3bneFrGmLPGmG3On0Ow1j954DroCY2xXHc+TeJ8xMv3jxaCaIjIQBE5CTQFPrU7Twy8DaywO4SbyAacjPL8FC70JZTYiEhuoCTwh81RnoiIeIrIDuACsNoYEy/53bIQiMjPIrI7mkcdAGNMT2NMDmAm1upqCcqj8jv36QlEYH2GBOdxPoNST0NEUgDzgY73neEneMaYSGNMCawz+TIiEi+X6B65eH1iZIyp/Ji7zgSWA33iMM4Te1R+EWkB1AQqmQTaCfQE/w9cxWkgR5Tn2Z1tKh45r63PB2YaYxbYnedpGWOuisg6IBCI8857tzwjeBgRyR/laR1gv11ZnoaIBAIfA7WNMaF253EjW4D8IuIvIt5AI2CJzZncirOz9RtgnzFmuN15npSI+N0e5SciSbEGHsTL94+OGrqPiMwHnsUatXIceN8Y4zK/2YlIMOADXHI2bXalUU8AIlIP+ArwA64CO4wxVW0N9RhEpDowEvAEphhjBtqb6MmIyA9AeawpkM8DfYwx39ga6gmIyIvAb8BfWP9+AXoYY5bbl+rxiUgxYBrW3x8PYI4xpl+8HFsLgVJKuTe9NKSUUm5OC4FSSrk5LQRKKeXmtBAopZSb00KglFJuTguBUkq5OS0ESinl5v4PVO3znrV8QpwAAAAASUVORK5CYII=\n" + }, + "metadata": { + "needs_background": "light" + } + } + ], + "source": [ + "def predict_sin8(x):\n", + " return model.a.item()+x*model.b.item()+x**2 * model.c.item() + x**3 * model.d.item()+x**4 *model.e.item()\n", + "\n", + "# print(predict_sin5(2))\n", + "import matplotlib.pyplot as plt \n", + "plt.plot(x,y,c='r')\n", + "plt.plot(x,predict_sin8(x),c='g')\n", + "plt.show()" + ] + } + ] +} \ No newline at end of file diff --git a/pytorch/实战/cifar_net.pth b/pytorch/实战/cifar_net.pth new file mode 100644 index 00000000..e8b53456 Binary files /dev/null and b/pytorch/实战/cifar_net.pth differ