diff --git a/Linear_Regression/Linear_Regression_with_tf_keras_Beginners.ipynb b/Linear_Regression/Linear_Regression_with_tf_keras_Beginners.ipynb index ae9517f..2394953 100644 --- a/Linear_Regression/Linear_Regression_with_tf_keras_Beginners.ipynb +++ b/Linear_Regression/Linear_Regression_with_tf_keras_Beginners.ipynb @@ -1,1474 +1,2782 @@ { - "nbformat": 4, - "nbformat_minor": 0, - "metadata": { - "colab": { - "name": "DL_Day4: Linear_Regression with tf.keras - Beginners.ipynb", - "provenance": [], - "collapsed_sections": [], - "include_colab_link": true - }, - "kernelspec": { - "name": "python3", - "display_name": "Python 3" - } + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "tmHsf5DWsxr6" + }, + "source": [ + "# Agenda\n", + "1. About the Dataset\n", + "2. Objective\n", + "3. Loading Libraries\n", + "4. Loading Data\n", + "5. View Data\n", + "6. Separate Input Features and Output Features/Variables\n", + "7. Split The Data into Train and Test Set\n", + "8. Train the model (The five step model life cycle)\n", + " 1. Define the model.\n", + " 2. Compile the model.\n", + " 3. Fit the model.\n", + " 4. Evaluate the model\n", + " * Hyperparameter Tunning\n", + " 5. Prediction" + ] }, - "cells": [ - { - "cell_type": "markdown", - "metadata": { - "id": "view-in-github", - "colab_type": "text" - }, - "source": [ - "\"Open" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "tmHsf5DWsxr6", - "colab_type": "text" - }, - "source": [ - "# Agenda\n", - "1. About the Dataset\n", - "2. Objective\n", - "3. Loading Libraries\n", - "4. Loading Data\n", - "5. View Data\n", - "6. Separate Input Features and Output Features/Variables\n", - "7. Split The Data into Train and Test Set\n", - "8. Train the model (The five step model life cycle)\n", - " 1. Define the model.\n", - " 2. Compile the model.\n", - " 3. Fit the model.\n", - " 4. Evaluate the model\n", - " * Hyperparameter Tunning\n", - " 5. Prediction" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "NmP7x85xcxkS", - "colab_type": "text" - }, - "source": [ - "## About the Dataset\n", - "We will be working on a data set that comes from the real estate industry in Boston (US). This database contains 14 attributes. The output variable refers to the median value of owner-occupied homes in 1000 USD's.\n", - "\n", - "* CRIM: per capita crime rate by town\n", - "* ZN: proportion of residential land zoned for lots over 25,000 sq.ft.\n", - "* INDUS: proportion of non-retail business acres per town\n", - "* CHAS: Charles River dummy variable (= 1 if tract bounds river; 0 otherwise)\n", - "* NOX: nitric oxides concentration (parts per 10 million)\n", - "* RM: average number of rooms per dwelling\n", - "* AGE: proportion of owner-occupied units built prior to 1940\n", - "* DIS: weighted distances to five Boston employment centres\n", - "* RAD: index of accessibility to radial highways\n", - "* TAX: full-value property-tax rate per 10,000 USD\n", - "* PTRATIO: pupil-teacher ratio by town\n", - "* B: 1000(Bk - 0.63)^2 where Bk is the proportion of blacks by town\n", - "* LSTAT: lower status of the population (%)\n", - "* MEDV: Median value of owner-occupied homes in 1000 USD's (Output/Target)\n" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "w_c7y3fqWuDX", - "colab_type": "text" - }, - "source": [ - "## Objective\n", - "The objective is to use linear regression to find the median value of owner-occupied homes in 1000 USD's.\n", - "\n", - "We will build a Machine learning model (i.e. Linear Regression) using `tensorflow.keras` (in short `tf.keras`) API." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "hvnf8jDeTTvw", - "colab_type": "text" - }, - "source": [ - "You already have all the information about Tensorflow and Keras from earlier modules." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "PQeegjzCbXdf", - "colab_type": "text" - }, - "source": [ - "## Loading Libraries\n", - "All Python capabilities are not loaded to our working environment by default (even if they are already installed in your system). So, we import each and every library that we want to use.\n", - "\n", - "In data science, numpy and pandas are most commonly used libraries. Numpy is required for calculations like means, medians, square roots, etc. Pandas is used for data processing and working with DataFrames. Matplotlib is used for data visualization. We chose alias names for our libraries for the sake of our convenience (numpy --> np and pandas --> pd, matplotlib.pyplot as plt).\n", - "\n", - "**pyplot:** pyplot is matplotlib's plotting framework. It is the most used module of matplotlib." - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "UEWLuXd7BMrU", - "colab_type": "code", - "colab": {} - }, - "source": [ - "# importing packages\n", - "import numpy as np # to perform calculations \n", - "import pandas as pd # to read data\n", - "import matplotlib.pyplot as plt # to visualise" - ], - "execution_count": null, - "outputs": [] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "fY_eewx0dhHt", - "colab_type": "text" - }, - "source": [ - "## Loading Data\n", - "Pandas module is used for reading files. We have our data in '.csv' format. We will use 'read_csv()' function for loading the data." - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "7zlSaTd2W9rt", - "colab_type": "code", - "colab": {} - }, - "source": [ - "# In read_csv() function, we have passed the location to where the file is located at dphi official github page\n", - "boston_data = pd.read_csv(\"https://raw.githubusercontent.com/dphi-official/Datasets/master/Boston_Housing/Training_set_boston.csv\" )" - ], - "execution_count": null, - "outputs": [] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "wze8jmpKtBG4", - "colab_type": "text" - }, - "source": [ - "## View Data" - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "Nwj5kkDjjQkx", - "colab_type": "code", - "colab": { - "base_uri": "https://localhost:8080/", - "height": 204 - }, - "outputId": "0a69f714-5724-4ec9-856c-82f7e619b4a5" - }, - "source": [ - "boston_data.head()" - ], - "execution_count": null, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/html": [ - "
\n", - "\n", - "\n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - "
CRIMZNINDUSCHASNOXRMAGEDISRADTAXPTRATIOBLSTATMEDV
015.023400.018.100.00.61405.30497.32.100724.0666.020.2349.4824.9112.0
10.627390.08.140.00.53805.83456.54.49864.0307.021.0395.628.4719.9
20.0346635.06.060.00.43796.03123.36.64071.0304.016.9362.257.8319.4
37.050420.018.100.00.61406.10385.12.021824.0666.020.22.5223.2913.4
40.725800.08.140.00.53805.72769.53.79654.0307.021.0390.9511.2818.2
\n", - "
" - ], - "text/plain": [ - " CRIM ZN INDUS CHAS NOX ... TAX PTRATIO B LSTAT MEDV\n", - "0 15.02340 0.0 18.10 0.0 0.6140 ... 666.0 20.2 349.48 24.91 12.0\n", - "1 0.62739 0.0 8.14 0.0 0.5380 ... 307.0 21.0 395.62 8.47 19.9\n", - "2 0.03466 35.0 6.06 0.0 0.4379 ... 304.0 16.9 362.25 7.83 19.4\n", - "3 7.05042 0.0 18.10 0.0 0.6140 ... 666.0 20.2 2.52 23.29 13.4\n", - "4 0.72580 0.0 8.14 0.0 0.5380 ... 307.0 21.0 390.95 11.28 18.2\n", - "\n", - "[5 rows x 14 columns]" - ] - }, - "metadata": { - "tags": [] - }, - "execution_count": 34 - } - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "DBv1fy29jAzm", - "colab_type": "text" - }, - "source": [ - "# Separating Input Features and Output Features\n", - "Before building any machine learning model, we always separate the input variables and output variables. \n", - "\n", - "**Input Variables or Independent Variables** are those quantities whose values are changed naturally in an experiment\n", - "\n", - "**Output Variable or Dependent Variable** is the one whose values are dependent on the input variables. \n", - "\n", - "Like here in this data, we are trying to predict the price of a houce i.e. 'MEDV', so this is our Output Variable \n", - "\n", - "By convention input variables are represented with 'X' and output variables are represented with 'y'." - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "ErMptop0jyng", - "colab_type": "code", - "colab": {} - }, - "source": [ - "X = boston_data.drop('MEDV', axis = 1) # Input Variables/features\n", - "y = boston_data.MEDV # output variables/features" - ], - "execution_count": null, - "outputs": [] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "QgXPagiS3YKz", - "colab_type": "text" - }, - "source": [ - "# Splitting the data" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "4vOkLlB-Ukjn", - "colab_type": "text" - }, - "source": [ - "We want to check the performance of the model that we built. For this purpose, we always split the given data(both input and output data) into **training set** which will be used to train the model, and **test set** which will be used to check how accurately the model is predicting outcomes.\n", - "\n", - "For this purpose we have a class called 'train_test_split' in the 'sklearn.model_selection' module.\n", - "\n", - "We split 80% of the data to the training set while 20% of the data to test set using below code.\n", - "The test_size variable is where we actually specify the proportion of the test set.\n", - "\n", - "By passing our X and y variables into the train_test_split method, we are able to capture the splits in data by assigning 4 variables to the result." - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "1a1OFJmpUl2u", - "colab_type": "code", - "colab": {} - }, - "source": [ - "# import train_test_split\n", - "from sklearn.model_selection import train_test_split \n", - "\n", - "# Assign variables to capture train test split output\n", - "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)\n", - "\n", - "# X_train: independent/input feature data for training the model\n", - "# y_train: dependent/output feature data for training the model\n", - "# X_test: independent/input feature data for testing the model; will be used to predict the output values\n", - "# y_test: original dependent/output values of X_test; We will compare this values with our predicted values to check the performance of our built model.\n", - " \n", - "# test_size = 0.20: 20% of the data will go for test set and 70% of the data will go for train set\n", - "# random_state = 42: this will fix the split i.e. there will be same split for each time you run the code" - ], - "execution_count": null, - "outputs": [] - }, - { - "cell_type": "code", - "metadata": { - "id": "UfKNjpJqkPUf", - "colab_type": "code", - "colab": { - "base_uri": "https://localhost:8080/", - "height": 34 - }, - "outputId": "560f2f83-590b-4d7b-fda7-96d0c3ad8224" - }, - "source": [ - "# find the number of input features\n", - "n_features = X.shape[1]\n", - "print(n_features)" - ], - "execution_count": null, - "outputs": [ - { - "output_type": "stream", - "text": [ - "13\n" - ], - "name": "stdout" - } - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "CwotIN0AU3Ci", - "colab_type": "text" - }, - "source": [ - "# Training our model\n" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "5EV_0iiBMBz6", - "colab_type": "text" - }, - "source": [ - "After splitting the data into training and testing sets, it's time to train our first deep learning model. Wait! Before training the deep learning model, let's understand the **Deep Learning Model Life-Cycle**." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "j3XXW9NL6H4s", - "colab_type": "text" - }, - "source": [ - "## Neural Network: Architecture\n", - "Here we are giving you just an overview of the architecture of Neural Network. You will know more about it in future modules.\n", - "\n", - "Neural Networks consists of an input and output layer with one or more hidden layers.\n", - "\n", - "![neural network architecture](https://dphi-courses.s3.ap-south-1.amazonaws.com/Deep+Learning+Bootcamp/nn+arch.png)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "Ymbh14OgMu2Z", - "colab_type": "text" - }, - "source": [ - "## The 5 Step Model Life-Cycle\n", - "\n", - "A model has a life-cycle, and this very simple knowledge provides the backbone for both modeling a dataset and understanding the tf.keras API.\n", - "\n", - "The five steps in the life-cycle are as follows:\n", - "\n", - "1. Define the model.\n", - "2. Compile the model.\n", - "3. Fit the model.\n", - "4. Make predictions on the test data.\n", - "5. Evaluate the model.\n", - "\n", - "We will take closer look into each of the steps and parallely build the deep learning model. Also, don't worry about the code for the moment if you don't understand it here. For the time being focus on the flow of building a deep learning model." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "auQ5lPFrN7GN", - "colab_type": "text" - }, - "source": [ - "### 1. Define the model\n", - "Defining the model requires that you first select the type of model that you need and then choose the architecture or network topology.\n", - "\n", - "Models can be defined either with the Sequential API or the Functional API (you will know this in later modules). Here we will define the model with Sequential API. Now **what is Sequential API?**\n", - "\n", - "**Sequential API**\n", - "The sequential API is the simplest API to get started with Deep Learning. \n", - "You will know more about it in upcoming learning units.\n", - "\n" - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "JY303RWVPzkD", - "colab_type": "code", - "colab": {} - }, - "source": [ - "from tensorflow.keras import Sequential # import Sequential from tensorflow.keras\n", - "from tensorflow.keras.layers import Dense # import Dense from tensorflow.keras.layers\n", - "from numpy.random import seed # seed helps you to fix the randomness in the neural network. \n", - "import tensorflow" - ], - "execution_count": null, - "outputs": [] + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "NmP7x85xcxkS" + }, + "source": [ + "## About the Dataset\n", + "We will be working on a data set that comes from the real estate industry in Boston (US). This database contains 14 attributes. The output variable refers to the median value of owner-occupied homes in 1000 USD's.\n", + "\n", + "* CRIM: per capita crime rate by town\n", + "* ZN: proportion of residential land zoned for lots over 25,000 sq.ft.\n", + "* INDUS: proportion of non-retail business acres per town\n", + "* CHAS: Charles River dummy variable (= 1 if tract bounds river; 0 otherwise)\n", + "* NOX: nitric oxides concentration (parts per 10 million)\n", + "* RM: average number of rooms per dwelling\n", + "* AGE: proportion of owner-occupied units built prior to 1940\n", + "* DIS: weighted distances to five Boston employment centres\n", + "* RAD: index of accessibility to radial highways\n", + "* TAX: full-value property-tax rate per 10,000 USD\n", + "* PTRATIO: pupil-teacher ratio by town\n", + "* B: 1000(Bk - 0.63)^2 where Bk is the proportion of blacks by town\n", + "* LSTAT: lower status of the population (%)\n", + "* MEDV: Median value of owner-occupied homes in 1000 USD's (Output/Target)\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "w_c7y3fqWuDX" + }, + "source": [ + "## Objective\n", + "The objective is to use linear regression to find the median value of owner-occupied homes in 1000 USD's.\n", + "\n", + "We will build a Machine learning model (i.e. Linear Regression) using `tensorflow.keras` (in short `tf.keras`) API." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "hvnf8jDeTTvw" + }, + "source": [ + "You already have all the information about Tensorflow and Keras from earlier modules." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "PQeegjzCbXdf" + }, + "source": [ + "## Loading Libraries\n", + "All Python capabilities are not loaded to our working environment by default (even if they are already installed in your system). So, we import each and every library that we want to use.\n", + "\n", + "In data science, numpy and pandas are most commonly used libraries. Numpy is required for calculations like means, medians, square roots, etc. Pandas is used for data processing and working with DataFrames. Matplotlib is used for data visualization. We chose alias names for our libraries for the sake of our convenience (numpy --> np and pandas --> pd, matplotlib.pyplot as plt).\n", + "\n", + "**pyplot:** pyplot is matplotlib's plotting framework. It is the most used module of matplotlib." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": { + "colab": {}, + "colab_type": "code", + "id": "UEWLuXd7BMrU" + }, + "outputs": [], + "source": [ + "# importing packages\n", + "import numpy as np # to perform calculations \n", + "import pandas as pd # to read data\n", + "import matplotlib.pyplot as plt # to visualise" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "fY_eewx0dhHt" + }, + "source": [ + "## Loading Data\n", + "Pandas module is used for reading files. We have our data in '.csv' format. We will use 'read_csv()' function for loading the data." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "colab": {}, + "colab_type": "code", + "id": "7zlSaTd2W9rt" + }, + "outputs": [], + "source": [ + "# In read_csv() function, we have passed the location to where the file is located at dphi official github page\n", + "boston_data = pd.read_csv(\"https://raw.githubusercontent.com/dphi-official/Datasets/master/Boston_Housing/Training_set_boston.csv\" )" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "wze8jmpKtBG4" + }, + "source": [ + "## View Data" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 204 }, + "colab_type": "code", + "id": "Nwj5kkDjjQkx", + "outputId": "0a69f714-5724-4ec9-856c-82f7e619b4a5" + }, + "outputs": [ { - "cell_type": "code", - "metadata": { - "id": "mMDrQE9gQP6U", - "colab_type": "code", - "colab": {} - }, - "source": [ - "# define the model\n", - "model = Sequential()\n", - "model.add(Dense(10, activation='relu', input_shape=(n_features,)))\n", - "model.add(Dense(8, activation='relu'))\n", - "model.add(Dense(1))" + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
CRIMZNINDUSCHASNOXRMAGEDISRADTAXPTRATIOBLSTATMEDV
3940.071650.025.650.00.5816.00484.12.19742.0188.019.1377.6714.2720.3
3950.047410.011.930.00.5736.03080.82.50501.0273.021.0396.907.8811.9
3961.251790.08.140.00.5385.57098.13.79794.0307.021.0376.5721.0213.6
3970.1257945.03.440.00.4376.55629.14.56675.0398.015.2382.844.5629.8
3980.158760.010.810.00.4135.96117.55.28734.0305.019.2376.949.8821.7
3990.171200.08.560.00.5205.83691.92.21105.0384.020.9395.6718.6619.5
4000.2991620.06.960.00.4645.85642.14.42903.0223.018.6388.6513.0021.1
4010.0150180.02.010.00.4356.63529.78.34404.0280.017.0390.945.9924.5
40211.160400.018.100.00.7406.62994.62.124724.0666.020.2109.8523.2713.4
4030.228760.08.560.00.5206.40585.42.71475.0384.020.970.8010.6318.6
\n", + "
" ], - "execution_count": null, - "outputs": [] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "93ntHsm1QnLP", - "colab_type": "text" - }, - "source": [ - "Note that the visible layer of the network is defined by the “input_shape” argument on the first hidden layer. \n", - "\n", - "The sequential API is easy to use because you keep calling model.add() until you have added all of your layers." + "text/plain": [ + " CRIM ZN INDUS CHAS NOX RM AGE DIS RAD TAX \\\n", + "394 0.07165 0.0 25.65 0.0 0.581 6.004 84.1 2.1974 2.0 188.0 \n", + "395 0.04741 0.0 11.93 0.0 0.573 6.030 80.8 2.5050 1.0 273.0 \n", + "396 1.25179 0.0 8.14 0.0 0.538 5.570 98.1 3.7979 4.0 307.0 \n", + "397 0.12579 45.0 3.44 0.0 0.437 6.556 29.1 4.5667 5.0 398.0 \n", + "398 0.15876 0.0 10.81 0.0 0.413 5.961 17.5 5.2873 4.0 305.0 \n", + "399 0.17120 0.0 8.56 0.0 0.520 5.836 91.9 2.2110 5.0 384.0 \n", + "400 0.29916 20.0 6.96 0.0 0.464 5.856 42.1 4.4290 3.0 223.0 \n", + "401 0.01501 80.0 2.01 0.0 0.435 6.635 29.7 8.3440 4.0 280.0 \n", + "402 11.16040 0.0 18.10 0.0 0.740 6.629 94.6 2.1247 24.0 666.0 \n", + "403 0.22876 0.0 8.56 0.0 0.520 6.405 85.4 2.7147 5.0 384.0 \n", + "\n", + " PTRATIO B LSTAT MEDV \n", + "394 19.1 377.67 14.27 20.3 \n", + "395 21.0 396.90 7.88 11.9 \n", + "396 21.0 376.57 21.02 13.6 \n", + "397 15.2 382.84 4.56 29.8 \n", + "398 19.2 376.94 9.88 21.7 \n", + "399 20.9 395.67 18.66 19.5 \n", + "400 18.6 388.65 13.00 21.1 \n", + "401 17.0 390.94 5.99 24.5 \n", + "402 20.2 109.85 23.27 13.4 \n", + "403 20.9 70.80 10.63 18.6 " ] + }, + "execution_count": 3, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "boston_data.tail(10)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "DBv1fy29jAzm" + }, + "source": [ + "# Separating Input Features and Output Features\n", + "Before building any machine learning model, we always separate the input variables and output variables. \n", + "\n", + "**Input Variables or Independent Variables** are those quantities whose values are changed naturally in an experiment\n", + "\n", + "**Output Variable or Dependent Variable** is the one whose values are dependent on the input variables. \n", + "\n", + "Like here in this data, we are trying to predict the price of a houce i.e. 'MEDV', so this is our Output Variable \n", + "\n", + "By convention input variables are represented with 'X' and output variables are represented with 'y'." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": { + "colab": {}, + "colab_type": "code", + "id": "ErMptop0jyng" + }, + "outputs": [], + "source": [ + "X = boston_data.drop('MEDV', axis = 1) # Input Variables/features\n", + "y = boston_data.MEDV # output variables/features" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "QgXPagiS3YKz" + }, + "source": [ + "# Splitting the data" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "4vOkLlB-Ukjn" + }, + "source": [ + "We want to check the performance of the model that we built. For this purpose, we always split the given data(both input and output data) into **training set** which will be used to train the model, and **test set** which will be used to check how accurately the model is predicting outcomes.\n", + "\n", + "For this purpose we have a class called 'train_test_split' in the 'sklearn.model_selection' module.\n", + "\n", + "We split 80% of the data to the training set while 20% of the data to test set using below code.\n", + "The test_size variable is where we actually specify the proportion of the test set.\n", + "\n", + "By passing our X and y variables into the train_test_split method, we are able to capture the splits in data by assigning 4 variables to the result." + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": { + "colab": {}, + "colab_type": "code", + "id": "1a1OFJmpUl2u" + }, + "outputs": [], + "source": [ + "# import train_test_split\n", + "from sklearn.model_selection import train_test_split \n", + "\n", + "# Assign variables to capture train test split output\n", + "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)\n", + "\n", + "# X_train: independent/input feature data for training the model\n", + "# y_train: dependent/output feature data for training the model\n", + "# X_test: independent/input feature data for testing the model; will be used to predict the output values\n", + "# y_test: original dependent/output values of X_test; We will compare this values with our predicted values to check the performance of our built model.\n", + " \n", + "# test_size = 0.20: 20% of the data will go for test set and 70% of the data will go for train set\n", + "# random_state = 42: this will fix the split i.e. there will be same split for each time you run the code" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 34 }, + "colab_type": "code", + "id": "UfKNjpJqkPUf", + "outputId": "560f2f83-590b-4d7b-fda7-96d0c3ad8224" + }, + "outputs": [ { - "cell_type": "markdown", - "metadata": { - "id": "ABBhVYtXRlJq", - "colab_type": "text" - }, - "source": [ - "ReLU is an activation function which is defined mathematically as **F(x) = max(0,x)**. In other words, the output is x, if x is greater than 0, and the output is 0 if x is 0 or negative." - ] - }, + "name": "stdout", + "output_type": "stream", + "text": [ + "13\n" + ] + } + ], + "source": [ + "# find the number of input features\n", + "n_features = X.shape[1]\n", + "print(n_features) # this will be used to specify the input size when we create the model below" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "CwotIN0AU3Ci" + }, + "source": [ + "# Training our model\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "5EV_0iiBMBz6" + }, + "source": [ + "After splitting the data into training and testing sets, it's time to train our first deep learning model. Wait! Before training the deep learning model, let's understand the **Deep Learning Model Life-Cycle**." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "j3XXW9NL6H4s" + }, + "source": [ + "## Neural Network: Architecture\n", + "Here we are giving you just an overview of the architecture of Neural Network. You will know more about it in future modules.\n", + "\n", + "Neural Networks consists of an input and output layer with one or more hidden layers.\n", + "\n", + "![neural network architecture](https://dphi-courses.s3.ap-south-1.amazonaws.com/Deep+Learning+Bootcamp/nn+arch.png)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "Ymbh14OgMu2Z" + }, + "source": [ + "## The 5 Step Model Life-Cycle\n", + "\n", + "A model has a life-cycle, and this very simple knowledge provides the backbone for both modeling a dataset and understanding the tf.keras API.\n", + "\n", + "The five steps in the life-cycle are as follows:\n", + "\n", + "1. Define the model.\n", + "2. Compile the model.\n", + "3. Fit the model.\n", + "4. Make predictions on the test data.\n", + "5. Evaluate the model.\n", + "\n", + "We will take closer look into each of the steps and parallely build the deep learning model. Also, don't worry about the code for the moment if you don't understand it here. For the time being focus on the flow of building a deep learning model." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "auQ5lPFrN7GN" + }, + "source": [ + "### 1. Define the model\n", + "Defining the model requires that you first select the type of model that you need and then choose the architecture or network topology.\n", + "\n", + "Models can be defined either with the Sequential API or the Functional API (you will know this in later modules). Here we will define the model with Sequential API. Now **what is Sequential API?**\n", + "\n", + "**Sequential API**\n", + "The sequential API is the simplest API to get started with Deep Learning. \n", + "You will know more about it in upcoming learning units.\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": { + "colab": {}, + "colab_type": "code", + "id": "JY303RWVPzkD" + }, + "outputs": [], + "source": [ + "from tensorflow.keras import Sequential # import Sequential from tensorflow.keras\n", + "from tensorflow.keras.layers import Dense # import Dense from tensorflow.keras.layers\n", + "from numpy.random import seed # seed helps you to fix the randomness in the neural network. \n", + "import tensorflow" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": { + "colab": {}, + "colab_type": "code", + "id": "mMDrQE9gQP6U" + }, + "outputs": [], + "source": [ + "# define the model\n", + "model = Sequential()\n", + "model.add(Dense(10, activation='relu', input_shape=(n_features,)))\n", + "model.add(Dense(8, activation='relu'))\n", + "model.add(Dense(1))" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": {}, + "outputs": [ { - "cell_type": "markdown", - "metadata": { - "id": "D100gBMQSTn8", - "colab_type": "text" - }, - "source": [ - "### 2. Compile the model\n", - "Compiling the model requires that you first select a loss function that you want to optimize, such as mean squared error or cross-entropy.\n", - "\n", - "It also requires that you select an algorithm to perform the optimization procedure. We’re using **RMSprop** as our optimizer here. R \n", - "\n", - "It may also require that you select any performance metrics to keep track of during the model training process. The loss function used here is **mean squared error.**\n", - "\n", - "From an API perspective, this involves calling a function to compile the model with the chosen configuration, which will prepare the appropriate data structures required for the efficient use of the model you have defined." - ] + "name": "stdout", + "output_type": "stream", + "text": [ + "Model: \"sequential\"\n", + "_________________________________________________________________\n", + "Layer (type) Output Shape Param # \n", + "=================================================================\n", + "dense (Dense) (None, 10) 140 \n", + "_________________________________________________________________\n", + "dense_1 (Dense) (None, 8) 88 \n", + "_________________________________________________________________\n", + "dense_2 (Dense) (None, 1) 9 \n", + "=================================================================\n", + "Total params: 237\n", + "Trainable params: 237\n", + "Non-trainable params: 0\n", + "_________________________________________________________________\n" + ] + } + ], + "source": [ + "# the number of parameters will depend on the input being given.\n", + "# Usually we add 1 bias term to the initial parameters to the\n", + "# total number will be num_inputs + 1 (for bias)\n", + "model.summary()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "93ntHsm1QnLP" + }, + "source": [ + "Note that the visible layer of the network is defined by the “input_shape” argument on the first hidden layer. \n", + "\n", + "The sequential API is easy to use because you keep calling model.add() until you have added all of your layers." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "ABBhVYtXRlJq" + }, + "source": [ + "ReLU is an activation function which is defined mathematically as **F(x) = max(0,x)**. In other words, the output is x, if x is greater than 0, and the output is 0 if x is 0 or negative." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "D100gBMQSTn8" + }, + "source": [ + "### 2. Compile the model\n", + "Compiling the model requires that you first select a loss function that you want to optimize, such as mean squared error or cross-entropy.\n", + "\n", + "It also requires that you select an algorithm to perform the optimization procedure. We’re using **RMSprop** as our optimizer here. R \n", + "\n", + "It may also require that you select any performance metrics to keep track of during the model training process. The loss function used here is **mean squared error.**\n", + "\n", + "From an API perspective, this involves calling a function to compile the model with the chosen configuration, which will prepare the appropriate data structures required for the efficient use of the model you have defined." + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": { + "colab": {}, + "colab_type": "code", + "id": "doYNy1jJQa7J" + }, + "outputs": [], + "source": [ + "# import RMSprop optimizer\n", + "from tensorflow.keras.optimizers import RMSprop\n", + "optimizer = RMSprop(0.01) # 0.01 is the learning rate" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "AiOzHe6K8iaM" + }, + "source": [ + "**Why learning rate = 0.01?**\n", + "\n", + "It is important to find a good value for the learning rate for your model on your training dataset. we cannot analytically calculate the optimal learning rate for a given model on a given dataset. Instead, a good (or good enough) learning rate must be discovered via trial and error.\n", + "\n", + "The range of values to consider for the learning rate is less than 1.0 and greater than $10^{-6}$.\n", + "\n", + "A traditional default value for the learning rate is 0.1 or 0.01, and this may represent a good starting point on your problem." + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": { + "colab": {}, + "colab_type": "code", + "id": "gNxnRf2uUqev" + }, + "outputs": [], + "source": [ + "model.compile(loss='mean_squared_error', optimizer=optimizer) # compile the model" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "GQPPF22pVHMU" + }, + "source": [ + "### 3. Fitting the model\n", + "Fitting the model requires that you first select the training configuration, such as the number of epochs (loops through the training dataset) and the batch size (number of samples in an epoch used to estimate model error).\n", + "\n", + "From an API perspective, this involves calling a function to perform the training process. This function will block (not return) until the training process has finished.\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Seed everything\n", + "\n", + "seed is something that we specify to have consistent randomization in the code. The weights of the parameters are usually randomly selected initially and they play some part in determining how the model will get trained.\n", + "\n", + "When we seed the randomization, we can generate results which will be consistent on a machine for reproducability." + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": {}, + "outputs": [], + "source": [ + "import random\n", + "\n", + "seed_value = random.randint(2, 200)\n", + "seed(seed_value) # If you build the model with given parameters, set_random_seed will help you produce the same result on multiple execution\n", + "\n", + "\n", + "# Recommended by Keras -------------------------------------------------------------------------------------\n", + "# 1. Set `PYTHONHASHSEED` environment variable at a fixed value\n", + "import os\n", + "os.environ['PYTHONHASHSEED']=str(seed_value)\n", + "\n", + "# 2. Set `python` built-in pseudo-random generator at a fixed value\n", + "random.seed(seed_value)\n", + "\n", + "# 3. Set `numpy` pseudo-random generator at a fixed value\n", + "import numpy as np\n", + "np.random.seed(seed_value)\n", + "# Recommended by Keras -------------------------------------------------------------------------------------\n", + "\n", + "\n", + "# 4. Set the `tensorflow` pseudo-random generator at a fixed value\n", + "tensorflow.random.set_seed(seed_value) " + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 374 }, + "colab_type": "code", + "id": "WnG7pev1UxNs", + "outputId": "291a66d5-71bb-49db-d7e3-a19b843977e0" + }, + "outputs": [ { - "cell_type": "code", - "metadata": { - "id": "doYNy1jJQa7J", - "colab_type": "code", - "colab": {} - }, - "source": [ - "# import RMSprop optimizer\n", - "from tensorflow.keras.optimizers import RMSprop\n", - "optimizer = RMSprop(0.01) # 0.01 is the learning rate" - ], - "execution_count": null, - "outputs": [] + "name": "stdout", + "output_type": "stream", + "text": [ + "Train on 323 samples\n", + "Epoch 1/10\n", + "323/323 [==============================] - 0s 1ms/sample - loss: 3542.9283\n", + "Epoch 2/10\n", + "323/323 [==============================] - 0s 44us/sample - loss: 183.6492\n", + "Epoch 3/10\n", + "323/323 [==============================] - 0s 47us/sample - loss: 135.1724\n", + "Epoch 4/10\n", + "323/323 [==============================] - 0s 48us/sample - loss: 127.6798\n", + "Epoch 5/10\n", + "323/323 [==============================] - 0s 44us/sample - loss: 119.9629\n", + "Epoch 6/10\n", + "323/323 [==============================] - 0s 45us/sample - loss: 111.4058\n", + "Epoch 7/10\n", + "323/323 [==============================] - 0s 48us/sample - loss: 122.9407\n", + "Epoch 8/10\n", + "323/323 [==============================] - 0s 44us/sample - loss: 112.0140\n", + "Epoch 9/10\n", + "323/323 [==============================] - 0s 48us/sample - loss: 115.9269\n", + "Epoch 10/10\n", + "323/323 [==============================] - 0s 46us/sample - loss: 113.1913\n" + ] }, { - "cell_type": "markdown", - "metadata": { - "id": "AiOzHe6K8iaM", - "colab_type": "text" - }, - "source": [ - "**Why learning rate = 0.01?**\n", - "\n", - "It is important to find a good value for the learning rate for your model on your training dataset. we cannot analytically calculate the optimal learning rate for a given model on a given dataset. Instead, a good (or good enough) learning rate must be discovered via trial and error.\n", - "\n", - "The range of values to consider for the learning rate is less than 1.0 and greater than $10^{-6}$.\n", - "\n", - "A traditional default value for the learning rate is 0.1 or 0.01, and this may represent a good starting point on your problem." + "data": { + "text/plain": [ + "" ] + }, + "execution_count": 13, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "model.fit(X_train, y_train, epochs=10, batch_size=30, verbose = 1) # fit the model" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "XZzI8-Kl_Rdg" + }, + "source": [ + "What is **verbose**?\n", + "\n", + "By setting verbose 0, 1 or 2 you just say how do you want to 'see' the training progress for each epoch.\n", + "\n", + "`verbose=0` will show you nothing (silent)\n", + "\n", + "`verbose=1` will show you an animated progress bar like this:\n", + "\n", + "![progres_bar](https://dphi-courses.s3.ap-south-1.amazonaws.com/Deep+Learning+Bootcamp/progress+bar.png)\n", + "\n", + "`verbose=2` will just mention the number of epoch like this:\n", + "\n", + "![verbose = 2](https://dphi-courses.s3.ap-south-1.amazonaws.com/Deep+Learning+Bootcamp/epoch.png)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "t_tKcVVzhJf8" + }, + "source": [ + "### 4. Evaluate the model\n", + "Evaluating the model requires that you first choose a holdout dataset used to evaluate the model. This should be data not used in the training process i.e. the X_test.\n", + "\n", + "From an API perspective, this involves calling a function with the holdout dataset and getting a loss and perhaps other metrics that can be reported." + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 88 }, + "colab_type": "code", + "id": "qC63QoqchIa1", + "outputId": "646c3c44-0773-4afd-84df-3dedc464bb9d" + }, + "outputs": [ { - "cell_type": "code", - "metadata": { - "id": "gNxnRf2uUqev", - "colab_type": "code", - "colab": {} - }, - "source": [ - "model.compile(loss='mean_squared_error',optimizer=optimizer) # compile the model" - ], - "execution_count": null, - "outputs": [] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "GQPPF22pVHMU", - "colab_type": "text" - }, - "source": [ - "### 3. Fitting the model\n", - "Fitting the model requires that you first select the training configuration, such as the number of epochs (loops through the training dataset) and the batch size (number of samples in an epoch used to estimate model error).\n", - "\n", - "From an API perspective, this involves calling a function to perform the training process. This function will block (not return) until the training process has finished.\n" - ] + "name": "stdout", + "output_type": "stream", + "text": [ + "81/81 [==============================] - 0s 766us/sample - loss: 53.5493\n" + ] }, { - "cell_type": "code", - "metadata": { - "id": "WnG7pev1UxNs", - "colab_type": "code", - "colab": { - "base_uri": "https://localhost:8080/", - "height": 374 - }, - "outputId": "291a66d5-71bb-49db-d7e3-a19b843977e0" - }, - "source": [ - "seed_value = 42\n", - "seed(seed_value) # If you build the model with given parameters, set_random_seed will help you produce the same result on multiple execution\n", - "\n", - "\n", - "# Recommended by Keras -------------------------------------------------------------------------------------\n", - "# 1. Set `PYTHONHASHSEED` environment variable at a fixed value\n", - "import os\n", - "os.environ['PYTHONHASHSEED']=str(seed_value)\n", - "\n", - "# 2. Set `python` built-in pseudo-random generator at a fixed value\n", - "import random\n", - "random.seed(seed_value)\n", - "\n", - "# 3. Set `numpy` pseudo-random generator at a fixed value\n", - "import numpy as np\n", - "np.random.seed(seed_value)\n", - "# Recommended by Keras -------------------------------------------------------------------------------------\n", - "\n", - "\n", - "# 4. Set the `tensorflow` pseudo-random generator at a fixed value\n", - "tensorflow.random.set_seed(seed_value) \n", - "model.fit(X_train, y_train, epochs=10, batch_size=30, verbose = 1) # fit the model" - ], - "execution_count": null, - "outputs": [ - { - "output_type": "stream", - "text": [ - "Epoch 1/10\n", - "11/11 [==============================] - 0s 1ms/step - loss: 840.9777\n", - "Epoch 2/10\n", - "11/11 [==============================] - 0s 1ms/step - loss: 103.6628\n", - "Epoch 3/10\n", - "11/11 [==============================] - 0s 1ms/step - loss: 179.7257\n", - "Epoch 4/10\n", - "11/11 [==============================] - 0s 2ms/step - loss: 191.5660\n", - "Epoch 5/10\n", - "11/11 [==============================] - 0s 1ms/step - loss: 140.9044\n", - "Epoch 6/10\n", - "11/11 [==============================] - 0s 1ms/step - loss: 104.5120\n", - "Epoch 7/10\n", - "11/11 [==============================] - 0s 1ms/step - loss: 116.0621\n", - "Epoch 8/10\n", - "11/11 [==============================] - 0s 1ms/step - loss: 115.5886\n", - "Epoch 9/10\n", - "11/11 [==============================] - 0s 1ms/step - loss: 107.1281\n", - "Epoch 10/10\n", - "11/11 [==============================] - 0s 1ms/step - loss: 111.2766\n" - ], - "name": "stdout" - }, - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "" - ] - }, - "metadata": { - "tags": [] - }, - "execution_count": 42 - } + "data": { + "text/plain": [ + "53.549260787021964" ] - }, + }, + "execution_count": 14, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "model.evaluate(X_test, y_test)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "gHM4sYEOnD7j" + }, + "source": [ + "The mean squared error we got here is 64.8. Now, **what does it mean?**\n", + "\n", + "When you subtract the predicted values (of X_test data) from the acutal value (of X_test data), then square it and sum all the squares, and finally take a mean (i.e. average) of it, the result you will get is 64.8 in this case.\n", + "\n", + "evaluate() does this task automatically. If you want to get the prediciton for X_test you can do **`model.predict(X_test)`**" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "metadata": {}, + "outputs": [ { - "cell_type": "markdown", - "metadata": { - "id": "XZzI8-Kl_Rdg", - "colab_type": "text" - }, - "source": [ - "What is **verbose**?\n", - "\n", - "By setting verbose 0, 1 or 2 you just say how do you want to 'see' the training progress for each epoch.\n", - "\n", - "`verbose=0` will show you nothing (silent)\n", - "\n", - "`verbose=1` will show you an animated progress bar like this:\n", - "\n", - "![progres_bar](https://dphi-courses.s3.ap-south-1.amazonaws.com/Deep+Learning+Bootcamp/progress+bar.png)\n", - "\n", - "`verbose=2` will just mention the number of epoch like this:\n", - "\n", - "![verbose = 2](https://dphi-courses.s3.ap-south-1.amazonaws.com/Deep+Learning+Bootcamp/epoch.png)" + "data": { + "text/plain": [ + "(81,)" ] - }, + }, + "execution_count": 15, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "y_test.values.shape" + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "metadata": {}, + "outputs": [], + "source": [ + "y_pred = model.predict(X_test)" + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "metadata": {}, + "outputs": [ { - "cell_type": "markdown", - "metadata": { - "id": "t_tKcVVzhJf8", - "colab_type": "text" - }, - "source": [ - "### 4. Evaluate the model\n", - "Evaluating the model requires that you first choose a holdout dataset used to evaluate the model. This should be data not used in the training process i.e. the X_test.\n", - "\n", - "From an API perspective, this involves calling a function with the holdout dataset and getting a loss and perhaps other metrics that can be reported." + "data": { + "text/plain": [ + "(81,)" ] - }, + }, + "execution_count": 17, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "y_pred.flatten().shape" + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "metadata": {}, + "outputs": [], + "source": [ + "diff = np.subtract(model.predict(X_test).flatten(), y_test.values)" + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "metadata": {}, + "outputs": [ { - "cell_type": "code", - "metadata": { - "id": "qC63QoqchIa1", - "colab_type": "code", - "colab": { - "base_uri": "https://localhost:8080/", - "height": 88 - }, - "outputId": "646c3c44-0773-4afd-84df-3dedc464bb9d" - }, - "source": [ - "model.evaluate(X_test, y_test)" - ], - "execution_count": null, - "outputs": [ - { - "output_type": "stream", - "text": [ - "WARNING:tensorflow:9 out of the last 12 calls to .test_function at 0x7f833f82a840> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has experimental_relax_shapes=True option that relaxes argument shapes that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/tutorials/customization/performance#python_or_tensor_args and https://www.tensorflow.org/api_docs/python/tf/function for more details.\n", - "3/3 [==============================] - 0s 2ms/step - loss: 64.8760\n" - ], - "name": "stdout" - }, - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "64.87602233886719" - ] - }, - "metadata": { - "tags": [] - }, - "execution_count": 43 - } + "data": { + "text/plain": [ + "(81,)" ] - }, + }, + "execution_count": 19, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "diff.shape" + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "metadata": {}, + "outputs": [], + "source": [ + "diff = diff * diff" + ] + }, + { + "cell_type": "code", + "execution_count": 21, + "metadata": {}, + "outputs": [ { - "cell_type": "markdown", - "metadata": { - "id": "gHM4sYEOnD7j", - "colab_type": "text" - }, - "source": [ - "The mean squared error we got here is 64.8. Now, **what does it mean?**\n", - "\n", - "When you subtract the predicted values (of X_test data) from the acutal value (of X_test data), then square it and sum all the squares, and finally take a mean (i.e. average) of it, the result you will get is 64.8 in this case.\n", - "\n", - "evaluate() does this task automatically. If you want to get the prediciton for X_test you can do **`model.predict(X_test)`**" + "data": { + "text/plain": [ + "(81,)" ] - }, + }, + "execution_count": 21, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "diff.shape" + ] + }, + { + "cell_type": "code", + "execution_count": 22, + "metadata": {}, + "outputs": [ { - "cell_type": "markdown", - "metadata": { - "id": "li94FIlfnMBB", - "colab_type": "text" - }, - "source": [ - "#### Hyperparameter Tunning\n", - "The hyperparameters here in this notebook are:\n", - "1. Learning Rate\n", - "2. Epochs\n", - "3. Batch Size\n", - "\n", - "We can try and change the values of these parameters and see the performance of the model (evaluate the model) on X_test data" + "data": { + "text/plain": [ + "53.54925939919559" ] + }, + "execution_count": 22, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "sum(diff)/(len(diff))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Define plotting functions\n", + "\n", + "To get a better undertanding of what's happening, we'll plot the values of the loss function with respect to the number of epochs that it takes throught the dataset. Hopefully, after each epoch, the loss should decrease and move towards it's minimum value" + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "metadata": {}, + "outputs": [], + "source": [ + "def plot_the_loss_curve(epochs, rmse):\n", + " \"\"\"Plot a curve of loss vs. epoch.\"\"\"\n", + " plt.figure()\n", + " plt.xlabel(\"Epoch\")\n", + " plt.ylabel(\"Root Mean Squared Error\")\n", + "\n", + " plt.plot(epochs, rmse, label=\"Loss\")\n", + " plt.legend()\n", + " plt.show() " + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "li94FIlfnMBB" + }, + "source": [ + "### Hyperparameter Tunning\n", + "The hyperparameters here in this notebook are:\n", + "1. Learning Rate\n", + "2. Epochs\n", + "3. Batch Size\n", + "\n", + "We can try and change the values of these parameters and see the performance of the model (evaluate the model) on X_test data" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "XljOzViRoRbp" + }, + "source": [ + "**Learning Rate**\n", + "\n", + "A scalar used to train a model via gradient descent. During each iteration, the **gradient descent** algorithm multiplies the learning rate by the gradient. The resulting product is called the **gradient step**.\n", + "\n", + "Learning rate is a key **hyperparameter**." + ] + }, + { + "cell_type": "code", + "execution_count": 24, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 428 }, + "colab_type": "code", + "id": "puhKnL9KKuZX", + "outputId": "c3ecbeff-a8e1-4a57-d9f8-61750e124fa3" + }, + "outputs": [ { - "cell_type": "markdown", - "metadata": { - "id": "XljOzViRoRbp", - "colab_type": "text" - }, - "source": [ - "**Learning Rate**\n", - "\n", - "A scalar used to train a model via gradient descent. During each iteration, the **gradient descent** algorithm multiplies the learning rate by the gradient. The resulting product is called the **gradient step**.\n", - "\n", - "Learning rate is a key **hyperparameter**." - ] - }, + "name": "stdout", + "output_type": "stream", + "text": [ + "Train on 323 samples\n", + "Epoch 1/10\n", + "323/323 [==============================] - 0s 1ms/sample - loss: 3852.9918\n", + "Epoch 2/10\n", + "323/323 [==============================] - 0s 46us/sample - loss: 605.8954\n", + "Epoch 3/10\n", + "323/323 [==============================] - 0s 41us/sample - loss: 502.8046\n", + "Epoch 4/10\n", + "323/323 [==============================] - 0s 54us/sample - loss: 203.1470\n", + "Epoch 5/10\n", + "323/323 [==============================] - 0s 53us/sample - loss: 110.2034\n", + "Epoch 6/10\n", + "323/323 [==============================] - 0s 42us/sample - loss: 123.7178\n", + "Epoch 7/10\n", + "323/323 [==============================] - 0s 46us/sample - loss: 129.7439\n", + "Epoch 8/10\n", + "323/323 [==============================] - 0s 45us/sample - loss: 286.9895\n", + "Epoch 9/10\n", + "323/323 [==============================] - 0s 42us/sample - loss: 183.4514\n", + "Epoch 10/10\n", + "323/323 [==============================] - 0s 44us/sample - loss: 150.8701\n", + "81/81 [==============================] - 0s 633us/sample - loss: 147.6725\n", + "The MSE value is: 147.67254073531538\n" + ] + } + ], + "source": [ + "####################### Complete example to check the performance of the model with different learning rates #############################\n", + "# define the model\n", + "\n", + "model = Sequential()\n", + "model.add(Dense(10, activation='relu', input_shape=(n_features,)))\n", + "model.add(Dense(8, activation='relu'))\n", + "model.add(Dense(1))\n", + "\n", + "optimizer = RMSprop(0.1) # 0.1 is the learning rate\n", + "model.compile(loss='mean_squared_error',optimizer=optimizer) # compile the model\n", + "\n", + "# fit the model \n", + "model.fit(X_train, y_train, epochs=10, batch_size=30, verbose = 1)\n", + "\n", + "# evaluate the model\n", + "print('The MSE value is: ', model.evaluate(X_test, y_test))" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "VPneBHxHYxfC" + }, + "source": [ + "As you can see above, how the loss (cost) i.e. MSE has changed by just changing the learning rate." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "fhi50q4rhUiF" + }, + "source": [ + "### Exercise 1\n", + "\n", + "Test several learning rate values to see the impact of varying this value when defining your model." + ] + }, + { + "cell_type": "code", + "execution_count": 25, + "metadata": { + "colab": {}, + "colab_type": "code", + "id": "k4BqC2Cfo159" + }, + "outputs": [ { - "cell_type": "code", - "metadata": { - "id": "puhKnL9KKuZX", - "colab_type": "code", - "colab": { - "base_uri": "https://localhost:8080/", - "height": 428 - }, - "outputId": "c3ecbeff-a8e1-4a57-d9f8-61750e124fa3" - }, - "source": [ - "####################### Complete example to check the performance of the model with different learning rates #######################################\n", - "# define the model\n", - "model = Sequential()\n", - "model.add(Dense(10, activation='relu', input_shape=(n_features,)))\n", - "model.add(Dense(8, activation='relu'))\n", - "model.add(Dense(1))\n", - "\n", - "optimizer = RMSprop(0.1) # 0.1 is the learning rate\n", - "model.compile(loss='mean_squared_error',optimizer=optimizer) # compile the model\n", - "\n", - "# fit the model \n", - "model.fit(X_train, y_train, epochs=10, batch_size=30, verbose = 1)\n", - "\n", - "# evaluate the model\n", - "print('The MSE value is: ', model.evaluate(X_test, y_test))" - ], - "execution_count": null, - "outputs": [ - { - "output_type": "stream", - "text": [ - "Epoch 1/10\n", - "11/11 [==============================] - 0s 1ms/step - loss: 35268.1797\n", - "Epoch 2/10\n", - "11/11 [==============================] - 0s 1ms/step - loss: 559.4071\n", - "Epoch 3/10\n", - "11/11 [==============================] - 0s 1ms/step - loss: 212.1511\n", - "Epoch 4/10\n", - "11/11 [==============================] - 0s 1ms/step - loss: 139.3775\n", - "Epoch 5/10\n", - "11/11 [==============================] - 0s 1ms/step - loss: 129.5609\n", - "Epoch 6/10\n", - "11/11 [==============================] - 0s 1ms/step - loss: 122.8495\n", - "Epoch 7/10\n", - "11/11 [==============================] - 0s 2ms/step - loss: 174.4107\n", - "Epoch 8/10\n", - "11/11 [==============================] - 0s 1ms/step - loss: 142.8208\n", - "Epoch 9/10\n", - "11/11 [==============================] - 0s 1ms/step - loss: 201.2038\n", - "Epoch 10/10\n", - "11/11 [==============================] - 0s 1ms/step - loss: 223.9772\n", - "WARNING:tensorflow:8 out of the last 11 calls to .test_function at 0x7f833c0ed7b8> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has experimental_relax_shapes=True option that relaxes argument shapes that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/tutorials/customization/performance#python_or_tensor_args and https://www.tensorflow.org/api_docs/python/tf/function for more details.\n", - "3/3 [==============================] - 0s 1ms/step - loss: 122.8239\n", - "The MSE value is: 122.82394409179688\n" - ], - "name": "stdout" - } - ] + "name": "stdout", + "output_type": "stream", + "text": [ + "Train on 323 samples\n", + "Epoch 1/20\n", + "323/323 [==============================] - 0s 1ms/sample - loss: 343.7974\n", + "Epoch 2/20\n", + "323/323 [==============================] - 0s 46us/sample - loss: 324.8378\n", + "Epoch 3/20\n", + "323/323 [==============================] - 0s 42us/sample - loss: 125.5561\n", + "Epoch 4/20\n", + "323/323 [==============================] - 0s 57us/sample - loss: 128.8586\n", + "Epoch 5/20\n", + "323/323 [==============================] - 0s 42us/sample - loss: 144.5521\n", + "Epoch 6/20\n", + "323/323 [==============================] - 0s 60us/sample - loss: 138.0498\n", + "Epoch 7/20\n", + "323/323 [==============================] - 0s 44us/sample - loss: 142.1506\n", + "Epoch 8/20\n", + "323/323 [==============================] - 0s 45us/sample - loss: 116.8843\n", + "Epoch 9/20\n", + "323/323 [==============================] - 0s 43us/sample - loss: 138.3841\n", + "Epoch 10/20\n", + "323/323 [==============================] - 0s 42us/sample - loss: 127.0573\n", + "Epoch 11/20\n", + "323/323 [==============================] - 0s 44us/sample - loss: 136.7117\n", + "Epoch 12/20\n", + "323/323 [==============================] - 0s 43us/sample - loss: 80.5961\n", + "Epoch 13/20\n", + "323/323 [==============================] - 0s 44us/sample - loss: 105.2849\n", + "Epoch 14/20\n", + "323/323 [==============================] - 0s 43us/sample - loss: 140.3133\n", + "Epoch 15/20\n", + "323/323 [==============================] - 0s 44us/sample - loss: 110.5351\n", + "Epoch 16/20\n", + "323/323 [==============================] - 0s 48us/sample - loss: 94.8067\n", + "Epoch 17/20\n", + "323/323 [==============================] - 0s 44us/sample - loss: 140.4404\n", + "Epoch 18/20\n", + "323/323 [==============================] - 0s 43us/sample - loss: 117.0459\n", + "Epoch 19/20\n", + "323/323 [==============================] - 0s 46us/sample - loss: 97.7931\n", + "Epoch 20/20\n", + "323/323 [==============================] - 0s 39us/sample - loss: 94.0652\n", + "81/81 [==============================] - 0s 611us/sample - loss: 71.8345\n" + ] }, { - "cell_type": "markdown", - "metadata": { - "id": "VPneBHxHYxfC", - "colab_type": "text" - }, - "source": [ - "As you can see above, how the loss (cost) i.e. MSE has changed by just changing the learning rate." + "data": { + "text/plain": [ + "71.83454560644833" ] - }, + }, + "execution_count": 25, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# Play with learning rate\n", + "learning_rate = 0.1 # Replace ? with a floating-point number(decimal no.)\n", + "epochs = 20\n", + "optimizer = RMSprop(learning_rate)\n", + "model.compile(loss='mean_squared_error',optimizer=optimizer) # Compile the model\n", + "history = model.fit(X_train, y_train, epochs=epochs, batch_size=30) # Fit the model\n", + "model.evaluate(X_test, y_test) # Evaluate the model" + ] + }, + { + "cell_type": "code", + "execution_count": 26, + "metadata": {}, + "outputs": [ { - "cell_type": "markdown", - "metadata": { - "id": "fhi50q4rhUiF", - "colab_type": "text" - }, - "source": [ - "### Exercise 1\n", - "\n", - "Test several learning rate values to see the impact of varying this value when defining your model." + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEGCAYAAACKB4k+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAgAElEQVR4nO3dd3zU9f3A8dc780ImhJCEBAhbdoCIgrhw48AFglZwVKp11trWWuuq/bXWVkWtKCiCVkXcuKuIikWEgKwwZEMghB0II5Dk/fvjvokRMy7jJu/n43GPfO877t65XO59ny2qijHGGAMQ5u8AjDHGBA5LCsYYYypZUjDGGFPJkoIxxphKlhSMMcZUivB3AI3RsmVLzcrK8ncYxhgTVObPn79DVVOqOxbUSSErK4vc3Fx/h2GMMUFFRDbUdMxr1Uci4hKRuSKySETyRORBZ/9kEVknIgudW7azX0TkSRFZLSKLRaSft2IzxhhTPW+WFEqAIapaLCKRwDci8rFz7Heq+uZR558HdHZuJwDjnZ/GGGN8xGslBXUrdu5GOrfahk8PA15yrpsDJIlIurfiM8YY83NebVMQkXBgPtAJ+LeqficiNwF/FZH7gBnA3apaAmQAm6pcnu/sK/BmjMaYY9ORI0fIz8/n0KFD/g7Fa1wuF5mZmURGRnp8jVeTgqqWAdkikgS8IyI9gT8CW4EoYALwB+AhTx9TRMYCYwHatm3b5DEbY44N+fn5xMfHk5WVhYj4O5wmp6rs3LmT/Px82rdv7/F1PhmnoKp7gJnAuapa4FQRlQAvAgOc0zYDbapclunsO/qxJqhqjqrmpKRU26PKGGPqdOjQIZKTk0MyIQCICMnJyfUuCXmz91GKU0JARGKAs4AVFe0E4v5LXAwsdS6ZDox2eiGdCBSpqlUdGWO8JlQTQoWG/H7eLCmkAzNFZDEwD/hMVT8AXhGRJcASoCXwsHP+R8BaYDUwEfi1twLbtf8wD76fx6EjZd56CmOMCUpea1NQ1cVA32r2D6nhfAVu9lY8Vc1es4PJs9ezOL+I50fn0Dw2yhdPa4wxPxEXF0dxcXHdJ/rQMTn30QW9W/PvK/uxZHMRlz07m027Dvg7JGOMCQjHZFIAGNornZevG8COfSVcOn42eVuK/B2SMcawfv16hgwZQu/evTnjjDPYuHEjAG+88QY9e/akT58+nHLKKQDk5eUxYMAAsrOz6d27N6tWrWr080swL8eZk5OjjZ376IfCfVwzaS57D5Uy/hf9OLmz9Wgy5liwfPlyunXrBsCD7+exbMveJn387q0TuP/CHrWeU1310YUXXsjll1/OmDFjmDRpEtOnT+fdd9+lV69efPLJJ2RkZLBnzx6SkpK49dZbOfHEE7nqqqs4fPgwZWVlxMTE1Ph7VhCR+aqaU11Mx2xJoUKX1Hje/vVJZDaP4doX5/H2gnx/h2SMOYZ9++23XHnllQBcffXVfPPNNwCcdNJJXHPNNUycOJGyMncnmYEDB/J///d/PPLII2zYsOFnCaEhgnqW1KaSluhi2o0D+dVL87lz2iK27j3ETad2DPnuasYYt7q+0QeCZ599lu+++44PP/yQ/v37M3/+fK688kpOOOEEPvzwQ4YOHcpzzz3HkCHV9uXx2DFfUqiQ4Ipk8nXHc1Gf1vzjk5XcPz2PsvLgrVozxgSnQYMGMXXqVABeeeUVTj75ZADWrFnDCSecwEMPPURKSgqbNm1i7dq1dOjQgdtuu41hw4axePHiRj+/lRSqiI4I54krsklLdDHh67UU7j3EuJF9cUWG+zs0Y0wIOnDgAJmZmZX377zzTp566imuvfZaHn30UVJSUnjxxRcB+N3vfseqVatQVc444wz69OnDI488wssvv0xkZCRpaWncc889jY7pmG9orsmkb9bxlw+X0a9tcxvLYEwIqq4BNhRZQ3MTuW5we54e1Y8l+TaWwRhz7LCkUIvze6fz0vU2lsEYc+ywpFCHEzsk8+ZNg4gIE654bg6zVm33d0jGmCYSzNXnnmjI72dJwQPusQyDbCyDMSHE5XKxc+fOkE0MFespuFyuel1nvY88lJ4Y85OxDIV7S7jptI7+DssY00CZmZnk5+ezfXvolv4rVl6rD0sK9VAxluHOaYt45JMVnNmtFZ1T4/0dljGmASIjI+u1ItmxwqqP6ik6IpwbTu4AwIad1iPJGBNaLCk0QFqCu45u697QXfDbGHNssqTQACnx0YSHCVuLLCkYY0KLJYUGCA8TUuKiraRgjAk5lhQaKC3RRaElBWNMiLGk0EBpCS4KrPrIGBNiLCk0UFqii0JLCsaYEGNJoYHSEl3sKymluKTU36EYY0yTsaTQQJXdUq20YIwJIZYUGigt0Z0UrLHZGBNKLCk0UEVJwRqbjTGhxJJCA1lJwRgTiiwpNJArMpykZpEUFB30dyjGGNNkLCk0QlqCi61FJf4OwxhjmowlhUawUc3GmFDjtaQgIi4RmSsii0QkT0QedPa3F5HvRGS1iLwuIlHO/mjn/mrneJa3YmsqNqrZGBNqvFlSKAGGqGofIBs4V0ROBB4BHlfVTsBu4Hrn/OuB3c7+x53zAlpqgoud+0s4XFru71CMMaZJeC0pqFuxczfSuSkwBHjT2T8FuNjZHubcxzl+hoiIt+JrCumJLlRh2z4rLRhjQoNX2xREJFxEFgLbgM+ANcAeVa2YGyIfyHC2M4BNAM7xIiC5msccKyK5IpLr77VVU61bqjEmxHg1KahqmapmA5nAAOC4JnjMCaqao6o5KSkpjY6xMdITK6a6sB5IxpjQ4JPeR6q6B5gJDASSRCTCOZQJbHa2NwNtAJzjicBOX8TXUD+OaraxCsaY0ODN3kcpIpLkbMcAZwHLcSeHy53TxgDvOdvTnfs4x79QVfVWfE0hMSaS6Igwqz4yxoSMiLpPabB0YIqIhONOPtNU9QMRWQZMFZGHge+BF5zzXwBeFpHVwC5gpBdjaxIiQnqidUs1xoSOWpOC84H+kqpeVd8HVtXFQN9q9q/F3b5w9P5DwPD6Po+/pSbYADZjTOiotfpIVcuAdhUDzMzPpSe62GpJwRgTIjypPloL/E9EpgP7K3aq6mNeiyqIpCa6KCwqQVUJ8GEVxhhTJ0+SwhrnFgbEezec4JOW4OJwWTm79h8mOS7a3+EYY0yj1JkUVLVizqI4535x7VccWyrGKhQUHbKkYIwJenV2SRWRniLyPZAH5InIfBHp4f3QgkNqgo1qNsaEDk/GKUwA7lTVdqraDvgtMNG7YQWP9MQYAGtsNsaEBE+SQqyqzqy4o6pfArFeiyjItIyLIkxgq41VMMaEAI96H4nIn4GXnfu/wN0jyQAR4WGkxEdbUjDGhARPSgrXASnA28BbQEtnn3GkJcZY9ZExJiR4MqL5T6p6m4/iCUppCdGs3b6/7hONMSbAeTKiebCPYgla6VZSMMaECE/aFL53RjO/wU9HNL/ttaiCTGqCi32HStlfUkpstDfnGDTGGO/y5BPMhXtdgyFV9inuNgYDpCW6B61t3XuIjilxfo7GGGMazpM2hcWq+riP4glKaQnOWIUiSwrGmODmSZvCKB/FErTSKpfltHYFY0xw86T66H8i8jTwOj9tU1jgtaiCTMWynNbYbIwJdp4khWzn50NV9ik/bWM4psVEhZMYE2klBWNM0PNkltTTfRFIsEtLsMV2jDHBr8Y2BRF5osr27Ucdm+zFmIJSWqLLSgrGmKBXW0PzKVW2xxx1rLcXYglqVlIwxoSC2pKC1LBtqpGW6GJHcQlHysr9HYoxxjRYbW0KYSLSHHfiqNiuSA7hXo8syKQlulCFbftKyEiK8Xc4xhjTILUlhURgPj8mgqpdUNVrEQWpym6pRYcsKRhjglaNSUFVs3wYR9CzAWzGmFDgyXoKxgM2gM0YEwosKTSRpGaRREeEUWhJwRgTxCwpNBERIS3RRYFVHxljgliNbQoi0qK2C1V1V9OHE9xSE1wUWlIwxgSx2nofzcfdy0iAtsBuZzsJ2Ai093p0QSY90cWCjbv9HYYxxjRYjdVHqtpeVTsAnwMXqmpLVU0GLgD+W9cDi0gbEZkpIstEJK9iqgwReUBENovIQuc2tMo1fxSR1SKyUkTOafyv51tpCS4K95agaj12jTHByZNZUk9U1Rsq7qjqxyLyDw+uKwV+q6oLRCQemC8inznHHlfVf1Y9WUS6AyOBHkBr4HMR6eKs6RAU0hJdHC4tZ/eBI7SIjfJ3OMYYU2+eNDRvEZF7RSTLuf0J2FLXRapaULHmgqruA5YDGbVcMgyYqqolqroOWA0M8CC+gFHRLbWg6KCfIzHGmIbxJCmMAlKAd3Cvy5xCPVdjE5EsoC/wnbPrFhFZLCKTnOkzwJ0wNlW5LJ9qkoiIjBWRXBHJ3b59e33C8LpUZwCbdUs1xgSrOpOCqu5S1duBwaraT1XvqE/PIxGJA94C7lDVvcB4oCPuxXsKgH/VJ2BVnaCqOaqak5KSUp9LvS69clRziZ8jMcaYhqkzKYjIIBFZhrv6BxHpIyLPePLgIhKJOyG8oqpvA6hqoaqWqWo5MJEfq4g2A22qXJ7p7AsaKXHRhAlsteojY0yQ8qT66HHgHGAngKou4qdrLVRLRAR4AViuqo9V2Z9e5bRLgKXO9nRgpIhEi0h7oDMw15NfIlBEhIeREh9tU10YY4KWJ72PUNVN7s/4Sp70CDoJuBpYIiILnX33AKNEJBv3GIj1wK+c58gTkWnAMtw9l24Opp5HFdISbFSzMSZ4eZIUNonIIECd6qDbcaqSaqOq31D94jwf1XLNX4G/ehBTwEpNcLF+535/h2GMMQ3iSfXRjcDNuHsCbcbdQHyzN4MKZum2VrMxJojVWlIQkXDgalW9ykfxBL3URBd7D5Vy4HApzaI8qp0zxpiAUWtJwanTv9JHsYSEdFtsxxgTxDz5KvuNiDwNvA5UVpZXjFY2P5VaZVnODilxfo7GGGPqx5OkkO38fKjKPgWGNH04wc9WYDPGBLM6k4Kqnu6LQEJF5VrNlhSMMUHIo5ZQETkf9+ylrop9qvpQzVccu5pFRZDgirA2BWNMUPJkmotngSuAW3GPOxgOtPNyXEEtPTHGkoIxJih5Mk5hkKqOBnar6oPAQKCLd8MKbqmJLqs+MsYEJU+SQsXsbgdEpDVwBEiv5fxjXlpCtJUUjDFByZM2hQ9EJAl4FFiAu+fR816NKsilJcawvbiEI2XlRIZ7kneNMSYweNL76C/O5lsi8gHgUtUi74YV3NISXKjC9n0ltE6K8Xc4xhjjsTqTgoiMrmYfqvqSd0IKfulVuqVaUjDGBBNPqo+Or7LtAs7AXY1kSaEGVUc1G2NMMPGk+ujWqved9oWpXosoBKTZ/EfGmCDVkFbQ/UD7pg4klDRvFklURBiF1i3VGBNkPGlTeB93jyNwJ5HuwDRvBhXsRMRWYDPGBCVP2hT+WWW7FNigqvleiidkpNkANmNMEPKkTeErXwQSatISXCzctMffYRhjTL14Un20jx+rj35yCFBVTWjyqEJAWqKLrXmHUFVEqluq2hhjAo8n1UdPAAXAy7gTwVVAuqre583Agl1agovDpeXsOXCE5rFR/g7HGGM84knvo4tU9RlV3aeqe1V1PDDM24EFu4puqdbYbIwJJp4khf0icpWIhItImIhcRZVlOU31KpKCdUs1xgQTT5LClcAIoBDYhns9hSu9GVQoqFiW00oKxphg4knvo/VYdVG9pcRHI2LLchpjgkuNJQURuUFEOjvbIiKTRKRIRBaLSD/fhRicIsPDSImLptBKCsaYIFJb9dHtwHpnexTQB+gA3AmM825YoSEt0UWBlRSMMUGktqRQqqpHnO0LgJdUdaeqfg7Eej+04JeW4LKSgjEmqNSWFMpFJF1EKqbL/rzKsToXCRCRNiIyU0SWiUieiNzu7G8hIp+JyCrnZ3Nnv4jIkyKyOlSqqNISXRQUHaz7RGOMCRC1JYX7gFzcVUjTVTUPQEROBdZ68NilwG9VtTtwInCziHQH7gZmqGpnYIZzH+A8oLNzGwuMr/dvE2BSE1zsPVTKwcNl/g7FGGM8UmNSUNUPgHZAN1W9ocqhXOCKuh5YVQtUdYGzvQ9YDmTg7sk0xTltCnCxsz0MdxWVquocIElE0uv5+wSUqiuwGWNMMKh1nIKqlqrq7qP27VfV4vo8iYhkAX2B74BUVS1wDm0FUp3tDGBTlcvynX1HP9ZYEckVkdzt27fXJwyf+3GsglUhGWOCQ0MW2akXEYkD3gLuUNW9VY+pqlL9ZHs1UtUJqpqjqjkpKSlNGGnTs1HNxphg49WkICKRuBPCK6r6trO7sKJayPm5zdm/GWhT5fJMZ1/QsvmPjDHBxqOkICIZIjJIRE6puHlwjQAvAMtV9bEqh6YDY5ztMcB7VfaPdnohnQgUValmCkrNoiKId0VYt1RjTNDwZD2FR3A3LC8DKrrRKPB1HZeeBFwNLBGRhc6+e4C/A9NE5HpgA+55lQA+AoYCq4EDwLWe/xqBK91WYDPGBBFP1lO4GOiqqiX1eWBV/Qb3+gvVOaOa8xW4uT7PEQxSE1xstZKCMSZIeFJ9tBaI9HYgocpKCsaYYOJJSeEAsFBEZgCVpQVVvc1rUYWQtAQX2/eVUFpWTkS41zt7GWNMo3iSFKY7N9MAqYkuyhW2F5eQnljn7CDGGONXnqynMKWuc0zNKkc1Fx2ypGCMCXie9D7qDPwN6A64KvaragcvxhUyUhN+TArGGBPoPKnkfhH35HSlwOnAS8B/vBlUKKkoHVhjszEmGHiSFGJUdQYgqrpBVR8AzvduWKGjebNIoiLCrKRgjAkKnjQ0l4hIGLBKRG7BPfVEnHfDCh0iQmpCtJUUjDFBwZOSwu1AM+A2oD/wC36cpsJ4ID0hxkoKxpig4Envo3kAIlKuqiEx9YSvpSa6WJy/x99hGGNMneosKYjIQBFZBqxw7vcRkWe8HlkISU90T3XhnsnDGGMClyfVR08A5wA7AVR1EVDnLKnmR6kJLkpKy9lz4Ii/QzHGmFp5NO+Cqm46apctOlwPFSuwWWOzMSbQeZIUNonIIEBFJFJE7sK93rLxUJqt1WyMCRKeJIUbcU9pnYG7O2o2ITjFtTelJdqoZmNMcPCk99EO4CofxBKyWsVHI2JJwRgT+GpMCiLyZG0X2tTZnosMD6NlXDSFVn1kjAlwtZUUbgSWAtOALdS8iprxQFqCiwIrKRhjAlxtSSEdGI57feZS4HXgTVW1UVgNkJboYtOuA/4OwxhjalVjQ7Oq7lTVZ1X1dOBaIAlYJiJX+yy6EGIlBWNMMPBkPYV+wCjgLOBjYL63gwpFaYkuig4e4dCRMlyR4f4OxxhjqlVbQ/NDuKfIXg5MBf6oqqW+CizUpFVZbCerZayfozHGmOrVVlK4F1gH9HFu/yci4G5wVlXt7f3wQkfFWIUCSwrGmABWW1Jo77MojgEVScG6pRpjAlmNSUFVN/gykFBXUX1kjc3GmEDm0YR4pvFioyOId0VYScEYE9AsKfhQWoLLprowxgQ0TxbZud2TfaZuaYkuCqykYIwJYJ6UFKpbj/maJo7jmJCW4KLQSgrGmABWY1IQkVEi8j7QXkSmV7nNBHbV9cAiMklEtonI0ir7HhCRzSKy0LkNrXLsjyKyWkRWisg5jf3FAlFaoott+w5RWlbu71CMMaZatXVJnQ0UAC2Bf1XZvw9Y7MFjTwaeBl46av/jqvrPqjtEpDswEugBtAY+F5EuqhpSK7ylJbooV9hRfLiyi6oxxgSS2uY+2qCqX6rqQGAFEO/c8j0Z2ayqX+NBicIxDJiqqiWqug5YDQzw8NqgYctyGmMCnScNzcOBubhnTB0BfCcilzfiOW8RkcVO9VJzZ18GUHUd6HxnX3XxjBWRXBHJ3b59eyPC8L3UyqkuDvo5EmOMqZ4nDc33Aser6hhVHY37G/yfG/h844GOuJf0LOCn1VIeUdUJqpqjqjkpKSkNDMM/0m1ZTmNMgPMkKYSp6rYq93d6eN3PqGqhqpapajkwkR+riDYDbaqcmunsCyktYqOICg+zbqnGmIBV59TZwCci8inwmnP/CuCjhjyZiKSraoFz9xLcK7sBTAdeFZHHcDc0d8ZdZRVSRITUxGi/d0v9eEkBz329lm7pCZzVvRWDOra06byNMYAHSUFVfycilwKDnV0TVPWduq4TkdeA04CWIpIP3A+cJiLZgALrgV85z5EnItOAZbhXebs51HoeVUhLcPmtofnQkTIe/nAZ/5mzkXbJzZi+cDOvzd1ITGQ4J3duyZndUxlyXCtaxkV7PZayciU8zFZ4NSbQeFJSAPgfcAT3h7lH3+BVdVQ1u1+o5fy/An/1MJ6glZrgYunmIp8/7+pt+7jl1e9ZsXUfY0/pwF1nd0VR5qzdxefLCvl8eSH/XVaICPRv25wzu6dyZrdUOqbE4kyZ3iCqSkHRIfK27GXp5iLytuxl2ZYidh04zN8v7c3FfavtT2CM8RNR1dpPEBkBPAp8iXsthZOB36nqm16Prg45OTmam5vr7zDq5a8fLuPlORtY/tC5jfqw9ZSq8sb8fO5/L4+YqHD+NbwPpx/Xqtrz8rbs5fPl7gSxdPNeANq3jOXMbq04s1sq/ds1JyK85uak8nJl3c795G3ZS96WIvI2u3/uPnAEABH34/VsnciGXQdYvmUvr95wAjlZLbzzyxuvKCkt47HPfuDSvpl0TYv3dzj1VlpWzr3vLuXsHqkMOS7V3+H4hYjMV9Wcao95kBQWAWdVNDaLSArwuar2afJI6ykYk8Lzs9by8IfLWXjfWSQ1i/LqcxWXlHLvO0t4d+EWBnZI5omR2ZXdYuuyZc9BZqzYxufLCvl2zU4Ol5WT1CySIV1bcWb3VAZ2SGZL0cHKD/68LXtZVrCXA4fdtX6R4UKX1Hh6tk6kR0YCPVoncFxaArHR7sLpngOHueSZ2RQdPMK7vz6JtsnNvPY6VGft9mLaJcdaFVYD/Hvmah79dCVZyc348LaTK/+mweK9hZu5fepCosLDmHzd8Qzq2NLfIflcY5PCElXtVeV+GLCo6j5/Ccak8OHiAm5+dQGf3HEyx6UleO15lm4u4pZXF7Bx1wHuOLMLN5/eqcEfgMUlpcz6YTufLS/kixXb2ON8868QGxVOt/QEemYk0r21OwF0bhVPVETtndTWbi/mkmdm0yo+mrd+PYgEV2SD4quvig+1zq3iuOucrpzdPdUnpbZQsGnXAc587Cu6psWzdHMRl/XL5NHhfv9+6DFV5bxxszhSVk6YCAVFh5g69kR6ZiT6OzSfqi0pNLT30cdNFdyxJi3R3Yi7teiQV5KCqvLi/9bzt4+X0zIumqljBzKgfeOqZ+KiIzivVzrn9UqntKycBRv3MG/9Ltq2aEaP1glkJccS1oCE0yEljvFX9WP0pLnc8ur3TBqTU2v1VFMY/+UaHv10JUOOa8X6nfv51cvzyW6TxB/OPY6BHZO9+twAm/cc5MVv1vHB4gIeu6JP0H1LffD9PMLDhGd/0Z9Xv9vI0zNXc2rXFC7o3drfoXnkyx+2s2LrPv41vA+DOiVz2TOzuebFubx54yBbJtdR53+gqv4OeA7o7dwmqOrvvR1YqPpxVHPT90Davf8wN7yUy0MfLOPULq346LaTG50QjhYRHsaA9i24+fROXNinNR1S4hqUECoM6tSShy/uydc/bOcvHyxrwkh/7rmv1vDIJysYlt2aiaNz+O8dp/DIZb0o3HuIURPnMHrSXK91Ali6uYjbp37PKf+YyYuz17Pn4GEmfbPeK8/lLZ8tK+Tz5du448zOtE6K4fYzO5PdJok/vr2EzXuCY5T++C/X0DrRxUXZrUlPjOGl60+grFwZPWku2/bZ+CHwcBCaqr6tqneq6p3AeyJylZfjClmt4l2INP38R9+t3cl542bx9Q87uP/C7kwc3Z/msd5ts2gqIwe05YaT2zPl2w1Mmb3eK88x8eu1/O3jFVzYpzX/Gt6H8DAhIjyMK45vy8y7TuNPQ7uxOH8PFzz1Dbe8uoB1O/Y3+jnLy5WZK7YxasIcLnjqG2Ys38Z1J2Xx9e9PZ8zALGau3Mb2fSVN8Nt534HDpTwwPY8uqXFce5J7+fbI8DDGjcymvFz5zdSFlJXXXhXtb/M37Gbuul388uQORDol0k6t4ph0zfFs31fCmEnz2HvoSB2PEvpqmzo7wZnO+mkROVvcbgHW4p4DyTRAVEQYybHRTVZSKCtXxn2+ilET5+CKDOPtXw/i2pPaB10d+d3ndePMbqk8+H4eX67cVvcF9fD8rLX89aPlnN87ncdH9PlZFZUrMpwbTunA178/nVuHdGLG8m2c+dhX3PPOkgYtn1pSWsa0eZs454mvuXbyPNbt2M89Q49j9h+H8Kfzu5ORFMPwnEzKypV3vw+OgftPf7GazXsO8vDFvSo/UAHaJcfy0LCezF2/i/FfrvZjhHV79qs1JDWLZOSANj/Z37dtc569uj+rCvdxw5RcDh0JySFSHqutpPAy0BVYAvwSmIl7UryLVXWYD2ILWemJTTOArXDvIa56fg6Pf/4DF/VpzQe3nRy0DWbhYcK4kdl0TUvglle/Z+XWfU3yuJO+WcfDHy7n/F7pjLsiu9Y2iwRXJL89uytf//50fnFCW97I3cSpj87k7x+voOhA3d8g9xw4zL9nrmbwIzP5/VuLiQgP4/Er+vD1709n7Ckdf9KQ3qlVPNltkpiWu4m6Onv42+ptxUyctZZL+2VUWx15ab8MLurTmsc/X8WCjbv9EGHdVhXu47NlhVwzKItmUT9vSj21Swr/GtGH79bt4vap3wd8qcebaksKHVT1GlV9DhgFdAfOUdWFvgktdKU2cq1mVXe1xHnjZrFoUxGPXt6bx6/IJi7IugYeLTY6ghfG5BATFc71U+axo7hxVSuT/7eOhz5Yxnk903hiZO0JoaqU+GgeHNaTGXeexnk903nu6zWc/I8veObL1Rw8/PNvkRt3HuD+95Yy8G9f8OinK+mensArvzyBj24bzCV9M2vshTUipw2rthWzKN/3gxk9parc995SYiLD+eN53ao9R0R4+JKepCW4uGPqQvYFYIwfpmcAABaRSURBVBXMs1+tJSYynDEDs2o8Z1h2Bvdd0J1P8wq5990lAZ+svaW2T5HKv6yqlolIvqpaS0wTSEuMJndD3UtNlJSWsWHnAdZuL2bN9v2s2VbMmh37WbutmH0lpRyXFs/TV/ajU6s4H0TtG62TYnh+dA4jnvuWX708n1d+eUKD5mV66dv1PPD+Ms7pkcqTo/r+pMrDU22Tm/H4FdmMPaUD//x0Jf/4ZCWT/7ee287ozBXHt2Hp5iImzlrLJ0u3Eh4mDMvO4Jcnt/e4V9kFfdJ56IM83sjdRHabpHrH5wvTF21h9pqd/OXinqTE1zz9SYIrknEjsxnx3LfcPz2Px0Zk+zDK2m3Zc5D3Fm7m6oHt6mxnu25we3buL+HfM9eQHBvNXed09VGUgaO2pNBHRPY62wLEOPcFUFX1Xif7EJeeGMOeA0c4dKSM6Igwdu0/7P7Q315cmQDWbi9m464DVC3Fpie66JgSxyX9MuiaFs9l/TJDciK7Pm2SeGxENje/uoC731rM41dk16uN5OVv13Pfe3mc3T2Vp0b1a1BCqKpbegIvXHM8uet38cgnK7j33aU8+ulKig4eIcEVwa9O7cg1g7I8HhhYIcEVybk90pi+aAt/vqB7wP0t9x46wsMfLqd3ZiJXDmhb5/k5WS24dUhnxs1YxaldUhiWHRhTmDw/ax0Avzy5g0fn33V2V3YWH+bpmatJjouqbFg/VtSYFFQ1sN6hIaTiw+Oy8bPJ332QooM/FrejI8Jo3zKWHhmJXNSnNR1bxdExJY72LWODbuRoY5zfO511O7rwz//+QMeUOG49o7NH1/1nzgb+/F4eZ3ZL5ekr+9U5gK4+crJaMO1XA/ly5XamztvICe2TGXF8m0ZV2w3PacO7C7fwad7WgPkQrfDYf39gR3EJL4zJ8Xjg461DOvHN6h3c+85S+rVtTpsWvh2pfrTd+w/z2tyNXJTdmoykGI+uEREevrgnu/Yf5sH3l9EiNirg/jbedOx8ygSQ/u2a06lVHPGuCC7onU6HlDg6psTSMSWOjKSYRvX7DyU3n96Jtdv386/PfqB9SmydA6Re/W4j9767lDOOa8UzVzVtQqggIpx+XKtq549qiIEdkslIiuGN3PyA+uBZurmIl75dzy9OaEfvTM+rtiLCw3jiimyGjpvFb15fyNSxJ3p9QGJtXvp2AwePlHHjqR3rdV1EeBhPjurL6Elz+e20RSQ1i+LULsG1qFdD+e+vdQxr3zKWz+88laljB/LXS3px/eD2nNa1FW1aNLOEUIWI8LfLepHTrjm/nbaIhZv21Hju1LkbueedJZzeNYVnfuGdhOANYWHC5f0z+d+aHeTvPuDvcAD3+Ip7311Ki9go7jq7/nXqbVo04+FLepK7YTdPz/RfN9UDh0uZPHsdZ3ZrRZfU+k/c54oMZ+LoHDq1iuOm/8yv9f0XSoLjP8ccs6Ijwnnu6v60Sojml1Nyqx05O23eJu5+ewmndU1h/C/6Ex0RXDWfl/fPRBXemh8YYxZez93Ewk17uGdoNxKbNWw+qmHZGVzSN4MnZ6xivgedKrzh9Xmb2H3gCDedVr9SQlWJMZG8dN0Ad9vCi3NZva24CSMMTJYUTMBLjotm0pjjKTlSxvWT51FcUlp5bFruJv7w9mJO6ZLCs7/oH3CNtZ5o06IZgzom8+aCTZT7uX/8zuIS/v7xCga0b8EljVzr4qFhPchoHsPtUxf6fKTwkbJynp+1jgFZLejfrnFTvbRKcPHydScQHiaMmTSXgqLgmNKjoSwpmKDQOTWep6/qx6ptxdz+mntw0Zvz8/nDW4sZ3KklE64OzoRQYXhOJpt2HeS7df75Vl3hkU9WsL+klL8M69noUfHxrkjGjexLQdEh/vzu0rovaELvL9rC5j0HufE0z3oc1SWrZSyTrx1A0cEjjH5hLnsOHG6Sxw1ElhRM0Di1SwoPXNidGSu2MWbSXH735iJO6tiSiaNzgjohAJzbI5346AjemL/JbzHM37CLabn5XD+4fZMtntOvbXPuOKMz7y3cwjvf5zfJY9alvFx59qs1dE2N5/SuTdMhAKBnRiITRvdnw84DXDd5XrUDGUOBJQUTVK4emMU1g7L4ZvUOBnVMDomEABATFc4FfVrz0ZICv4wILi0r50/vLCU90cVtHnb/9dSvT+/EgKwW/PndPDbu9H5j+syV2/ihsJibTuvY5HOADerYknEjs/l+0x7GTJr7k+7kocKSggk6957fjYmjc3h+9PHERAV/QqgwPCeTQ0fK+XBxgc+fe8q3G1ixdR/3X9i9ycfDhIcJj4/MRgRuf/17jpSVN+njH238l2vISIrhgt7pXnn883ql8+TIvny/aTcjnv3WK9Pg+5MlBRN0IsLDOKt7akglBIC+bZLomBLLG/N9U81SYWvRIR7770pO65rCOT3SvPIcGUkx/O3SXny/cQ9PzVjllecAmLd+F7kbdjP2lA5eHR9xYZ/WTL52AJv3HOSy8bNDqleSJQVjAoSIMDynDfM37Pbph8zDHy7jSLny4EU9vDrl+gW9W3N5/0yenrmauV5qUH/2yzW0iI1iRE6buk9upJM6tWTq2BMpKS3n8mdnB+wMsfVlScGYAHJp3wzCw4Q3fVRamLVqOx8sLuDm0zrRLtn7y1E+cFEP2rRoxm9eX+jRdOT1sWLrXmas2MY1g7J8VorsmZHI2zcNIjEmkisnzuGLFYU+eV5vsqRgTABpleDitC4pvL0gn1Iv172XlJZx33t5ZCU341enNk3XzbrERUcwbmRfCvce4uJn/teky58+99VamkWFM3pguyZ7TE+0TW7GWzcNonOreG54aT7Tcv3Xg6wpWFIwJsAMz8lk274SZq3a4dXnmfDVWtbt2M+Dw3r6tAdXdpskXvnlCRw8XMalz8xm8v/WNXrtgvzdB5i+aAtXDmhLUjPfL0PbMi6a18aeyKCOyfz+zcU88+XqoF2PwZKCMQFmyHGptIiN8uo3zo07D/D0zNUM7ZXml4neTuiQzEe3n8zJnVvywPvLGPvy/EYNCHt+1jrCBK4/2X/TXMdFR/DCmOO5qE9r/vHJSh58f5nfR6g3hCUFYwJMVEQYF2dn8PnyQnbtb/qRs6rKA+/nER4m/PmC7k3++J5qERvF82Ny+PMF3fly5TaGjptF7vr6N0DvLC5h6ryNXJydQXqiZ9Nje0tUhHuW2OsHt2fy7PXcNvV7SkqDa5CbJQVjAtDwnEyOlCnvLWz6SfImfL2WL1Zs486zuvj9Q1REuH5we966aRCREWFcMWEO/565ul5rJE/5dgMlpeU+axepS5iTbO8ZehwfLC7gusnzAnKJ0pp4LSmIyCQR2SYiS6vsayEin4nIKudnc2e/iMiTIrJaRBaLSD9vxWVMMOiWnkCvjESm5TZtL6SPlhTwt49XcEHvdK4LoBXFemcm8cGtgxnaK51HP13J6EnfsW1f3YPC9peUMmX2es7qlkqnVk0zNUdTGXtKRx4b0Yfv1u5i5IQ5Hv0+gcCbJYXJwLlH7bsbmKGqnYEZzn2A84DOzm0sMN6LcRkTFIbnZLK8YG+T9dBZsHE3v3l9If3bNeefw/sE3Nod8a5InhyZzd8v7cX8DbsZOm4WX/+wvdZrXpu7kaKDR7ixEdNje9Ol/TJ5fkwOa7fv57Lxs1m/Y7+/Q6qT15KCqn4NHF1BOAyY4mxPAS6usv8ldZsDJImId8aoGxMkLurTmqjwsCYZs7Bh535umJJLWqIroOeLEhFGDmjL9FsG0yI2itGT5vLIJyuqnRrjcGk5L3yzjhM7tKBf2+Z+iNYzp3VtxWtjT2R/SRmXjZ/N4vzAXqzH120KqapaMbHLViDV2c4Aqna1yHf2/YyIjBWRXBHJ3b699m8RxgSzpGZRnNUjlXcXbm5UY+WeA4e5dvI8ylR58ZrjaRHr+y6b9dUlNZ73bh7MqAFtGf/lGq547tufrUz33sLNFBQdqvdSm/6Q3SaJN28ciCsynJET5jBrVeB+dvmtoVndnXjr3V9LVSeoao6q5qSkHBtrpppj14icNuw5cITPl21r0PUlpWWMfXk++bsOMuHqHDqkxDVxhN4TExXO3y7txVOj+rKqsJih42bxydKtgHt67Oe+Xku39ISgWTu5Q0ocb/96EG1bNOPaF+fxm9cX8txXa5i5chsFRQcDZlxD006HWLdCEUlX1QKneqjinb4ZqDpZSaazz5hj2uBOLUlPdPHG/E2cX89ZP1WVu99awtx1uxg3MpsB7Ru3Apm/XNinNX0yk7jltQXc+J/5jB7YjuOzWrB6WzFPjurr1fmamlpqgotpNw7kvneX8u2anbzz/Y8fcwmuCI5LS6BLWhxd0xI4Li2eLqnxJMY0bEnUhvJ1UpgOjAH+7vx8r8r+W0RkKnACUFSlmsmYY1Z4mHBpvwzGf7mGrUWHSEt0eXzt45+v4p3vN3PX2V0Ylt24pTX9rW1yM968cRD/+GQFz3+zjpfnbKBNixiG9vTOrK7elOCK5ImRfQEoOnCElYX7WLl1Lyu27mPl1n28t3AL+w5trDw/PdFF17R49y3V/bNTqzivrUXutaQgIq8BpwEtRSQfuB93MpgmItcDG4ARzukfAUOB1cAB4FpvxWVMsBnevw3/nrmGtxbkc/PpnTy65s35+Tw5YxUjcjI9vibQRUWEce8F3RnUKZn73svjrrO7enV6bF9IbBbJgPYtflKKU1UKig6xcus+Vmzdxw+F7p+zV+/ksNPgHh4m3Hx6J+48q0uTxySBUo/VEDk5OZqbm+vvMIzxuhHPfsv24hK++O2pdVaXzF6zgzGT5jKgfQsmXzuAyCD/4DRuR8rKWb9jv1Oy2Ee/ds0bvNyoiMxX1Zzqjvm6+sgY0wCX52Ty+zcXM3/DbnKyam4bWL1tH796eT5ZybE8c1V/SwghJDI8jM6p8XROjeeC3t57HnvHGBMEzu+VTrOocN6oZYTz9n0lXPPiPFyR4bx47fE+b6A0ocGSgjFBIDY6gvN7pfPB4i0cOFz6s+MHD5fxyynz2Fl8mBfG5JDZvJkfojShwJKCMUFieE4b9h8u46MlW3+yv6xcueP171m8uYhxI7PpnZnkpwhNKLCkYEyQOD6rOVnJzXjjqHUW/vbRcj7NK+TP53fn7B7B10XTBBZLCsYECRFheE4bvlu3iw073ROrvfztep7/Zh3XDMriusGBM+upCV6WFIwJIpf2yyBM3OMQvlhRyP3T8zizWyu/LpZjQot1STUmiKQnxjC4cwqvzd3IC9+so3vrBMaN7Et4gE2DbYKXlRSMCTIjcjLZUXyYpJhIJo05ntho+25nmo69m4wJMmd3T+O2IZ24KLs1rRI8nwvJGE9YUjAmyERFhHHn2V39HYYJUVZ9ZIwxppIlBWOMMZUsKRhjjKlkScEYY0wlSwrGGGMqWVIwxhhTyZKCMcaYSpYUjDHGVArqNZpFZDuwoYGXtwR2NGE4TS3Q44PAj9HiaxyLr3ECOb52qppS3YGgTgqNISK5NS1cHQgCPT4I/Bgtvsax+Bon0OOriVUfGWOMqWRJwRhjTKVjOSlM8HcAdQj0+CDwY7T4Gsfia5xAj69ax2ybgjHGmJ87lksKxhhjjmJJwRhjTKWQTwoicq6IrBSR1SJydzXHo0Xkdef4dyKS5cPY2ojITBFZJiJ5InJ7NeecJiJFIrLQud3nq/ic518vIkuc586t5riIyJPO67dYRPr5MLauVV6XhSKyV0TuOOocn79+IjJJRLaJyNIq+1qIyGcissr52byGa8c456wSkTE+jO9REVnh/A3fEZGkGq6t9f3gxfgeEJHNVf6OQ2u4ttb/dy/G93qV2NaLyMIarvX669doqhqyNyAcWAN0AKKARUD3o875NfCssz0SeN2H8aUD/ZzteOCHauI7DfjAj6/heqBlLceHAh8DApwIfOfHv/VW3INy/Pr6AacA/YClVfb9A7jb2b4beKSa61oAa52fzZ3t5j6K72wgwtl+pLr4PHk/eDG+B4C7PHgP1Pr/7q34jjr+L+A+f71+jb2FeklhALBaVdeq6mFgKjDsqHOGAVOc7TeBM0REfBGcqhao6gJnex+wHMjwxXM3oWHAS+o2B0gSkXQ/xHEGsEZVGzrCvcmo6tfArqN2V32fTQEurubSc4DPVHWXqu4GPgPO9UV8qvpfVS117s4BMpv6eT1Vw+vnCU/+3xuttvicz44RwGtN/by+EupJIQPYVOV+Pj//0K08x/mnKAKSfRJdFU61VV/gu2oODxSRRSLysYj08GlgoMB/RWS+iIyt5rgnr7EvjKTmf0R/vn4VUlW1wNneCqRWc06gvJbX4S79Vaeu94M33eJUb02qofotEF6/k4FCVV1Vw3F/vn4eCfWkEBREJA54C7hDVfcedXgB7iqRPsBTwLs+Dm+wqvYDzgNuFpFTfPz8dRKRKOAi4I1qDvv79fsZddcjBGRfcBH5E1AKvFLDKf56P4wHOgLZQAHuKppANIraSwkB//8U6klhM9Cmyv1MZ1+154hIBJAI7PRJdO7njMSdEF5R1bePPq6qe1W12Nn+CIgUkZa+ik9VNzs/twHv4C6iV+XJa+xt5wELVLXw6AP+fv2qKKyoVnN+bqvmHL++liJyDXABcJWTuH7Gg/eDV6hqoaqWqWo5MLGG5/X36xcBXAq8XtM5/nr96iPUk8I8oLOItHe+TY4Eph91znSgopfH5cAXNf1DNDWn/vEFYLmqPlbDOWkVbRwiMgD338wnSUtEYkUkvmIbd2Pk0qNOmw6MdnohnQgUVakm8ZUav5358/U7StX32RjgvWrO+RQ4W0SaO9UjZzv7vE5EzgV+D1ykqgdqOMeT94O34qvaTnVJDc/ryf+7N50JrFDV/OoO+vP1qxd/t3R7+4a7d8wPuHsl/MnZ9xDuNz+AC3e1w2pgLtDBh7ENxl2NsBhY6NyGAjcCNzrn3ALk4e5JMQcY5MP4OjjPu8iJoeL1qxqfAP92Xt8lQI6P/76xuD/kE6vs8+vrhztBFQBHcNdrX4+7nWoGsAr4HGjhnJsDPF/l2uuc9+Jq4Fofxrcad318xfuwokdea+Cj2t4PPorvZef9tRj3B3360fE593/2/+6L+Jz9kyved1XO9fnr19ibTXNhjDGmUqhXHxljjKkHSwrGGGMqWVIwxhhTyZKCMcaYSpYUjDHGVLKkYEwtRKTsqJlYm2zmTRHJqjrTpjGBIMLfARgT4A6qara/gzDGV6ykYEwDOPPi/8OZG3+uiHRy9meJyBfOxG0zRKStsz/VWadgkXMb5DxUuIhMFPd6Gv8VkRi//VLGYEnBmLrEHFV9dEWVY0Wq2gt4GnjC2fcUMEVVe+OeVO5JZ/+TwFfqnpivH+4RrQCdgX+rag9gD3CZl38fY2plI5qNqYWIFKtqXDX71wNDVHWtM6nhVlVNFpEduKdgOOLsL1DVliKyHchU1ZIqj5GFe/2Ezs79PwCRqvqw938zY6pnJQVjGk5r2K6PkirbZVg7n/EzSwrGNNwVVX5+62zPxj07J8BVwCxnewZwE4CIhItIoq+CNKY+7FuJMbWLOWoR9k9UtaJbanMRWYz72/4oZ9+twIsi8jtgO3Cts/92YIKIXI+7RHAT7pk2jQko1qZgTAM4bQo5qrrD37EY05Ss+sgYY0wlKykYY4ypZCUFY4wxlSwpGGOMqWRJwRhjTCVLCsYYYypZUjDGGFPp/wGOhbScg7Y/IgAAAABJRU5ErkJggg==\n", + "text/plain": [ + "
" ] - }, - { - "cell_type": "code", - "metadata": { - "id": "k4BqC2Cfo159", - "colab_type": "code", - "colab": {} - }, - "source": [ - "# Play with learning rate\n", - "learning_rate = ? # Replace ? with a floating-point number(decimal no.)\n", - "epochs = 10\n", - "optimizer = RMSprop(learning_rate)\n", - "model.compile(loss='mean_squared_error',optimizer=optimizer) # Compile the model\n", - "model.fit(X_train, y_train, epochs=epochs, batch_size=30) # Fit the model\n", - "model.evaluate(X_test, y_test) # Evaluate the model" - ], - "execution_count": null, - "outputs": [] - }, + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "plot_the_loss_curve(history.epoch, history.history['loss'])" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "Mov9-uVwpmkd" + }, + "source": [ + "**Epochs**\n", + "\n", + "A full training pass over the entire dataset such that each example has been seen once. Thus, an epoch represents N/batch size training iterations, where N is the total number of examples." + ] + }, + { + "cell_type": "code", + "execution_count": 27, + "metadata": {}, + "outputs": [ { - "cell_type": "markdown", - "metadata": { - "id": "Mov9-uVwpmkd", - "colab_type": "text" - }, - "source": [ - "**Epochs**\n", - "\n", - "A full training pass over the entire dataset such that each example has been seen once. Thus, an epoch represents N/batch size training iterations, where N is the total number of examples." + "data": { + "text/plain": [ + "(323, 13)" ] + }, + "execution_count": 27, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "X_train.shape # total number of samples = 323" + ] + }, + { + "cell_type": "code", + "execution_count": 28, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 1000 }, - { - "cell_type": "code", - "metadata": { - "id": "zlZv50iKZPqo", - "colab_type": "code", - "colab": { - "base_uri": "https://localhost:8080/", - "height": 1000 - }, - "outputId": "36ef904c-7b07-4f63-954f-98b1a20b55f1" - }, - "source": [ - "####################### Complete example to check the performance of the model with different epochs and learning rate = 0.01 #######################################\n", - "# define the model\n", - "model = Sequential()\n", - "model.add(Dense(10, activation='relu', input_shape=(n_features,)))\n", - "model.add(Dense(8, activation='relu'))\n", - "model.add(Dense(1))\n", - "\n", - "optimizer = RMSprop(0.1) # 0.1 is the learning rate\n", - "model.compile(loss='mean_squared_error',optimizer=optimizer) # Compile the model\n", - "\n", - "# fit the model \n", - "model.fit(X_train, y_train, epochs=100, batch_size=30, verbose = 1)\n", - "\n", - "# evaluate the model\n", - "print('The MSE value is: ', model.evaluate(X_test, y_test))" - ], - "execution_count": null, - "outputs": [ - { - "output_type": "stream", - "text": [ - "Epoch 1/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 9724.5732\n", - "Epoch 2/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 582.0069\n", - "Epoch 3/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 555.8216\n", - "Epoch 4/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 521.9396\n", - "Epoch 5/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 483.8689\n", - "Epoch 6/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 444.7491\n", - "Epoch 7/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 406.9054\n", - "Epoch 8/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 371.2612\n", - "Epoch 9/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 337.6587\n", - "Epoch 10/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 306.2268\n", - "Epoch 11/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 277.0800\n", - "Epoch 12/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 250.4167\n", - "Epoch 13/100\n", - "11/11 [==============================] - 0s 2ms/step - loss: 225.5309\n", - "Epoch 14/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 202.9859\n", - "Epoch 15/100\n", - "11/11 [==============================] - 0s 2ms/step - loss: 182.7787\n", - "Epoch 16/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 164.8227\n", - "Epoch 17/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 148.9729\n", - "Epoch 18/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 135.4988\n", - "Epoch 19/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 123.8988\n", - "Epoch 20/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 114.1056\n", - "Epoch 21/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 106.0993\n", - "Epoch 22/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 100.3444\n", - "Epoch 23/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 96.4450\n", - "Epoch 24/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 93.5845\n", - "Epoch 25/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 92.0008\n", - "Epoch 26/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 91.3616\n", - "Epoch 27/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 91.0118\n", - "Epoch 28/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 91.1522\n", - "Epoch 29/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 91.0371\n", - "Epoch 30/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 91.0573\n", - "Epoch 31/100\n", - "11/11 [==============================] - 0s 2ms/step - loss: 90.9193\n", - "Epoch 32/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 91.0018\n", - "Epoch 33/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 90.9787\n", - "Epoch 34/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 90.9684\n", - "Epoch 35/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 91.0983\n", - "Epoch 36/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 90.9732\n", - "Epoch 37/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 90.9472\n", - "Epoch 38/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 90.9128\n", - "Epoch 39/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 91.0360\n", - "Epoch 40/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 90.9132\n", - "Epoch 41/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 90.9193\n", - "Epoch 42/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 90.9364\n", - "Epoch 43/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 90.8977\n", - "Epoch 44/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 90.9577\n", - "Epoch 45/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 90.9994\n", - "Epoch 46/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 91.0100\n", - "Epoch 47/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 91.0025\n", - "Epoch 48/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 91.0074\n", - "Epoch 49/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 90.9457\n", - "Epoch 50/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 91.0306\n", - "Epoch 51/100\n", - "11/11 [==============================] - 0s 2ms/step - loss: 91.1645\n", - "Epoch 52/100\n", - "11/11 [==============================] - 0s 2ms/step - loss: 90.9211\n", - "Epoch 53/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 90.9330\n", - "Epoch 54/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 90.8901\n", - "Epoch 55/100\n", - "11/11 [==============================] - 0s 2ms/step - loss: 91.0190\n", - "Epoch 56/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 91.0044\n", - "Epoch 57/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 91.0272\n", - "Epoch 58/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 91.0088\n", - "Epoch 59/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 91.0335\n", - "Epoch 60/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 90.8989\n", - "Epoch 61/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 91.0687\n", - "Epoch 62/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 90.9590\n", - "Epoch 63/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 90.9457\n", - "Epoch 64/100\n", - "11/11 [==============================] - 0s 2ms/step - loss: 90.9054\n", - "Epoch 65/100\n", - "11/11 [==============================] - 0s 2ms/step - loss: 91.0399\n", - "Epoch 66/100\n", - "11/11 [==============================] - 0s 2ms/step - loss: 90.9739\n", - "Epoch 67/100\n", - "11/11 [==============================] - 0s 2ms/step - loss: 90.9935\n", - "Epoch 68/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 90.9428\n", - "Epoch 69/100\n", - "11/11 [==============================] - 0s 2ms/step - loss: 90.9365\n", - "Epoch 70/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 90.9752\n", - "Epoch 71/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 90.9980\n", - "Epoch 72/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 91.0044\n", - "Epoch 73/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 91.0282\n", - "Epoch 74/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 91.0151\n", - "Epoch 75/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 90.9688\n", - "Epoch 76/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 90.9369\n", - "Epoch 77/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 91.0542\n", - "Epoch 78/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 90.9409\n", - "Epoch 79/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 91.1157\n", - "Epoch 80/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 90.9322\n", - "Epoch 81/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 91.0418\n", - "Epoch 82/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 90.9078\n", - "Epoch 83/100\n", - "11/11 [==============================] - 0s 2ms/step - loss: 90.9901\n", - "Epoch 84/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 91.0322\n", - "Epoch 85/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 91.0187\n", - "Epoch 86/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 90.9214\n", - "Epoch 87/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 91.0568\n", - "Epoch 88/100\n", - "11/11 [==============================] - 0s 2ms/step - loss: 90.8874\n", - "Epoch 89/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 90.9281\n", - "Epoch 90/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 91.0325\n", - "Epoch 91/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 90.9906\n", - "Epoch 92/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 91.0670\n", - "Epoch 93/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 90.8633\n", - "Epoch 94/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 90.9740\n", - "Epoch 95/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 91.0710\n", - "Epoch 96/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 90.9303\n", - "Epoch 97/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 90.9599\n", - "Epoch 98/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 90.9306\n", - "Epoch 99/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 91.0165\n", - "Epoch 100/100\n", - "11/11 [==============================] - 0s 1ms/step - loss: 90.9525\n", - "WARNING:tensorflow:7 out of the last 11 calls to .test_function at 0x7f833f931ae8> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has experimental_relax_shapes=True option that relaxes argument shapes that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/tutorials/customization/performance#python_or_tensor_args and https://www.tensorflow.org/api_docs/python/tf/function for more details.\n", - "3/3 [==============================] - 0s 1ms/step - loss: 71.1146\n", - "The MSE value is: 71.11458587646484\n" - ], - "name": "stdout" - } - ] + "colab_type": "code", + "collapsed": true, + "id": "zlZv50iKZPqo", + "jupyter": { + "outputs_hidden": true }, + "outputId": "36ef904c-7b07-4f63-954f-98b1a20b55f1" + }, + "outputs": [ { - "cell_type": "markdown", - "metadata": { - "id": "CPakaxBaZZNA", - "colab_type": "text" - }, - "source": [ - "You can see above how the loss (cost) i.e. MSE has changed just by changing the epochs and keeping the learning rate same as 0.01 (i.e. the first model we built)" - ] - }, + "name": "stdout", + "output_type": "stream", + "text": [ + "Train on 323 samples\n", + "Epoch 1/100\n", + "323/323 [==============================] - 0s 1ms/sample - loss: 779.2674\n", + "Epoch 2/100\n", + "323/323 [==============================] - 0s 45us/sample - loss: 90.3773\n", + "Epoch 3/100\n", + "323/323 [==============================] - 0s 45us/sample - loss: 106.2122\n", + "Epoch 4/100\n", + "323/323 [==============================] - 0s 48us/sample - loss: 116.1714\n", + "Epoch 5/100\n", + "323/323 [==============================] - 0s 79us/sample - loss: 92.8123\n", + "Epoch 6/100\n", + "323/323 [==============================] - 0s 53us/sample - loss: 95.4058\n", + "Epoch 7/100\n", + "323/323 [==============================] - 0s 44us/sample - loss: 95.6812\n", + "Epoch 8/100\n", + "323/323 [==============================] - 0s 44us/sample - loss: 81.5782\n", + "Epoch 9/100\n", + "323/323 [==============================] - 0s 46us/sample - loss: 96.0180\n", + "Epoch 10/100\n", + "323/323 [==============================] - 0s 47us/sample - loss: 86.4723\n", + "Epoch 11/100\n", + "323/323 [==============================] - 0s 43us/sample - loss: 66.0415\n", + "Epoch 12/100\n", + "323/323 [==============================] - 0s 42us/sample - loss: 80.7756\n", + "Epoch 13/100\n", + "323/323 [==============================] - 0s 44us/sample - loss: 77.7597\n", + "Epoch 14/100\n", + "323/323 [==============================] - 0s 44us/sample - loss: 69.2761\n", + "Epoch 15/100\n", + "323/323 [==============================] - 0s 45us/sample - loss: 74.7446\n", + "Epoch 16/100\n", + "323/323 [==============================] - 0s 43us/sample - loss: 76.9908\n", + "Epoch 17/100\n", + "323/323 [==============================] - 0s 45us/sample - loss: 76.6107\n", + "Epoch 18/100\n", + "323/323 [==============================] - 0s 42us/sample - loss: 71.1128\n", + "Epoch 19/100\n", + "323/323 [==============================] - 0s 42us/sample - loss: 67.9994\n", + "Epoch 20/100\n", + "323/323 [==============================] - 0s 47us/sample - loss: 64.6969\n", + "Epoch 21/100\n", + "323/323 [==============================] - 0s 45us/sample - loss: 72.5926\n", + "Epoch 22/100\n", + "323/323 [==============================] - 0s 42us/sample - loss: 52.0385\n", + "Epoch 23/100\n", + "323/323 [==============================] - 0s 46us/sample - loss: 66.4203\n", + "Epoch 24/100\n", + "323/323 [==============================] - 0s 47us/sample - loss: 58.2275\n", + "Epoch 25/100\n", + "323/323 [==============================] - 0s 44us/sample - loss: 59.0364\n", + "Epoch 26/100\n", + "323/323 [==============================] - 0s 44us/sample - loss: 58.6424\n", + "Epoch 27/100\n", + "323/323 [==============================] - 0s 42us/sample - loss: 55.0437\n", + "Epoch 28/100\n", + "323/323 [==============================] - 0s 43us/sample - loss: 54.3899\n", + "Epoch 29/100\n", + "323/323 [==============================] - 0s 45us/sample - loss: 53.4645\n", + "Epoch 30/100\n", + "323/323 [==============================] - 0s 42us/sample - loss: 47.6370\n", + "Epoch 31/100\n", + "323/323 [==============================] - 0s 44us/sample - loss: 46.0004\n", + "Epoch 32/100\n", + "323/323 [==============================] - 0s 44us/sample - loss: 59.2141\n", + "Epoch 33/100\n", + "323/323 [==============================] - 0s 44us/sample - loss: 51.4808\n", + "Epoch 34/100\n", + "323/323 [==============================] - 0s 44us/sample - loss: 48.5556\n", + "Epoch 35/100\n", + "323/323 [==============================] - 0s 44us/sample - loss: 48.1779\n", + "Epoch 36/100\n", + "323/323 [==============================] - 0s 45us/sample - loss: 45.9281\n", + "Epoch 37/100\n", + "323/323 [==============================] - 0s 42us/sample - loss: 49.0529\n", + "Epoch 38/100\n", + "323/323 [==============================] - 0s 42us/sample - loss: 44.9997\n", + "Epoch 39/100\n", + "323/323 [==============================] - 0s 42us/sample - loss: 50.6981\n", + "Epoch 40/100\n", + "323/323 [==============================] - 0s 42us/sample - loss: 40.3308\n", + "Epoch 41/100\n", + "323/323 [==============================] - 0s 41us/sample - loss: 60.3304\n", + "Epoch 42/100\n", + "323/323 [==============================] - 0s 43us/sample - loss: 33.6350\n", + "Epoch 43/100\n", + "323/323 [==============================] - 0s 44us/sample - loss: 50.8828\n", + "Epoch 44/100\n", + "323/323 [==============================] - 0s 40us/sample - loss: 37.7698\n", + "Epoch 45/100\n", + "323/323 [==============================] - 0s 44us/sample - loss: 35.0152\n", + "Epoch 46/100\n", + "323/323 [==============================] - 0s 43us/sample - loss: 46.4097\n", + "Epoch 47/100\n", + "323/323 [==============================] - 0s 41us/sample - loss: 40.1406\n", + "Epoch 48/100\n", + "323/323 [==============================] - 0s 46us/sample - loss: 47.0542\n", + "Epoch 49/100\n", + "323/323 [==============================] - 0s 43us/sample - loss: 39.9722\n", + "Epoch 50/100\n", + "323/323 [==============================] - 0s 42us/sample - loss: 49.1545\n", + "Epoch 51/100\n", + "323/323 [==============================] - 0s 42us/sample - loss: 41.7313\n", + "Epoch 52/100\n", + "323/323 [==============================] - 0s 42us/sample - loss: 33.1227\n", + "Epoch 53/100\n", + "323/323 [==============================] - 0s 40us/sample - loss: 46.7185\n", + "Epoch 54/100\n", + "323/323 [==============================] - 0s 43us/sample - loss: 36.3416\n", + "Epoch 55/100\n", + "323/323 [==============================] - 0s 41us/sample - loss: 36.2967\n", + "Epoch 56/100\n", + "323/323 [==============================] - 0s 41us/sample - loss: 46.9293\n", + "Epoch 57/100\n", + "323/323 [==============================] - 0s 46us/sample - loss: 39.4794\n", + "Epoch 58/100\n", + "323/323 [==============================] - 0s 42us/sample - loss: 39.8657\n", + "Epoch 59/100\n", + "323/323 [==============================] - 0s 44us/sample - loss: 36.3635\n", + "Epoch 60/100\n", + "323/323 [==============================] - 0s 43us/sample - loss: 39.8302\n", + "Epoch 61/100\n", + "323/323 [==============================] - 0s 46us/sample - loss: 35.6971\n", + "Epoch 62/100\n", + "323/323 [==============================] - 0s 41us/sample - loss: 41.2701\n", + "Epoch 63/100\n", + "323/323 [==============================] - 0s 46us/sample - loss: 40.1464\n", + "Epoch 64/100\n", + "323/323 [==============================] - 0s 44us/sample - loss: 32.6426\n", + "Epoch 65/100\n", + "323/323 [==============================] - 0s 41us/sample - loss: 42.1818\n", + "Epoch 66/100\n", + "323/323 [==============================] - 0s 52us/sample - loss: 36.0643\n", + "Epoch 67/100\n", + "323/323 [==============================] - 0s 47us/sample - loss: 37.1797\n", + "Epoch 68/100\n", + "323/323 [==============================] - 0s 45us/sample - loss: 39.7682\n", + "Epoch 69/100\n", + "323/323 [==============================] - 0s 45us/sample - loss: 28.8304\n", + "Epoch 70/100\n", + "323/323 [==============================] - 0s 43us/sample - loss: 35.4105\n", + "Epoch 71/100\n", + "323/323 [==============================] - 0s 47us/sample - loss: 31.1776\n", + "Epoch 72/100\n", + "323/323 [==============================] - 0s 42us/sample - loss: 44.0173\n", + "Epoch 73/100\n", + "323/323 [==============================] - 0s 44us/sample - loss: 29.8790\n", + "Epoch 74/100\n", + "323/323 [==============================] - 0s 42us/sample - loss: 40.0739\n", + "Epoch 75/100\n", + "323/323 [==============================] - 0s 43us/sample - loss: 34.7984\n", + "Epoch 76/100\n", + "323/323 [==============================] - 0s 44us/sample - loss: 25.4359\n", + "Epoch 77/100\n", + "323/323 [==============================] - 0s 45us/sample - loss: 42.0926\n", + "Epoch 78/100\n", + "323/323 [==============================] - 0s 41us/sample - loss: 32.9081\n", + "Epoch 79/100\n", + "323/323 [==============================] - 0s 53us/sample - loss: 34.4714\n", + "Epoch 80/100\n", + "323/323 [==============================] - 0s 49us/sample - loss: 39.1492\n", + "Epoch 81/100\n", + "323/323 [==============================] - 0s 46us/sample - loss: 26.0022\n", + "Epoch 82/100\n", + "323/323 [==============================] - 0s 48us/sample - loss: 34.2694\n", + "Epoch 83/100\n", + "323/323 [==============================] - 0s 44us/sample - loss: 44.6201\n", + "Epoch 84/100\n", + "323/323 [==============================] - 0s 44us/sample - loss: 28.5248\n", + "Epoch 85/100\n", + "323/323 [==============================] - 0s 44us/sample - loss: 37.2560\n", + "Epoch 86/100\n", + "323/323 [==============================] - 0s 44us/sample - loss: 34.1021\n", + "Epoch 87/100\n", + "323/323 [==============================] - 0s 44us/sample - loss: 33.0781\n", + "Epoch 88/100\n", + "323/323 [==============================] - 0s 41us/sample - loss: 30.3239\n", + "Epoch 89/100\n", + "323/323 [==============================] - 0s 42us/sample - loss: 41.6006\n", + "Epoch 90/100\n", + "323/323 [==============================] - 0s 44us/sample - loss: 22.9749\n", + "Epoch 91/100\n", + "323/323 [==============================] - 0s 43us/sample - loss: 45.6504\n", + "Epoch 92/100\n", + "323/323 [==============================] - 0s 43us/sample - loss: 22.9598\n", + "Epoch 93/100\n", + "323/323 [==============================] - 0s 45us/sample - loss: 39.6132\n", + "Epoch 94/100\n", + "323/323 [==============================] - 0s 45us/sample - loss: 41.1154\n", + "Epoch 95/100\n", + "323/323 [==============================] - 0s 43us/sample - loss: 25.9910\n", + "Epoch 96/100\n", + "323/323 [==============================] - 0s 41us/sample - loss: 22.4914\n", + "Epoch 97/100\n", + "323/323 [==============================] - 0s 42us/sample - loss: 33.6054\n", + "Epoch 98/100\n", + "323/323 [==============================] - 0s 41us/sample - loss: 36.7831\n", + "Epoch 99/100\n", + "323/323 [==============================] - 0s 46us/sample - loss: 33.0215\n", + "Epoch 100/100\n", + "323/323 [==============================] - 0s 40us/sample - loss: 26.2922\n", + "81/81 [==============================] - 0s 622us/sample - loss: 37.6629\n", + "The MSE value is: 37.66293480955524\n" + ] + } + ], + "source": [ + "############# Complete example to check the performance of the model with different epochs and learning rate = 0.01 ####################\n", + "# define the model\n", + "\n", + "model = Sequential()\n", + "model.add(Dense(10, activation='relu', input_shape=(n_features,)))\n", + "model.add(Dense(8, activation='relu'))\n", + "model.add(Dense(1))\n", + "\n", + "optimizer = RMSprop(0.01) # 0.1 is the learning rate\n", + "model.compile(loss='mean_squared_error',optimizer=optimizer) # Compile the model\n", + "\n", + "# fit the model \n", + "history = model.fit(X_train, y_train, epochs=100, batch_size=30, verbose = 1)\n", + "\n", + "# evaluate the model\n", + "print('The MSE value is: ', model.evaluate(X_test, y_test))" + ] + }, + { + "cell_type": "code", + "execution_count": 29, + "metadata": {}, + "outputs": [ { - "cell_type": "markdown", - "metadata": { - "id": "6WoG362ohrQF", - "colab_type": "text" - }, - "source": [ - "### Exercise 2\n", - "\n", - "Test several epoch values to see the impact of varying this value when defining your model." + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEGCAYAAACKB4k+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAgAElEQVR4nO3deXhU5dnH8e89MyFhD0tYwyqIoixiVERcCrXuxbovVaS01NYq1dbW1vZta/u2tZtW26q4Yl+rdRf3KtJaUFEQZFUJYQtrCBAggaz3+8echMlCmACTQOb3ua65cs5zlnkOo3PPs5u7IyIiAhBq6gyIiMihQ0FBRESqKCiIiEgVBQUREamioCAiIlUiTZ2BA9G5c2fv27dvU2dDROSwMnfu3M3unlHXscM6KPTt25c5c+Y0dTZERA4rZrZqb8cSWn1kZjeb2WIzW2RmT5pZmpn1M7PZZpZtZv80sxbBuanBfnZwvG8i8yYiIrUlLCiYWU/gJiDL3Y8FwsAVwJ3AXe4+ANgKTAwumQhsDdLvCs4TEZFGlOiG5gjQ0swiQCtgPTAGeDY4PhW4MNgeF+wTHB9rZpbg/ImISIyEtSm4+1oz+wOwGtgF/AuYC2xz97LgtFygZ7DdE1gTXFtmZgVAJ2Bz7H3NbBIwCaB3796Jyr6INHOlpaXk5uaye/fups5KwqSlpZGZmUlKSkrc1yQsKJhZB6K//vsB24BngLMP9L7uPgWYApCVlaWJm0Rkv+Tm5tK2bVv69u1Lc6yUcHfy8/PJzc2lX79+cV+XyOqjLwIr3D3P3UuB54FTgPSgOgkgE1gbbK8FegEEx9sD+QnMn4gksd27d9OpU6dmGRAAzIxOnTo1uCSUyKCwGhhpZq2CtoGxwBJgBnBJcM544KVge1qwT3D8HdcUriKSQM01IFTan+dLWFBw99lEG4w/BhYG7zUF+CFwi5llE20zeDi45GGgU5B+C3BbovL20cot/PFfn1FaXpGotxAROSwldPCau/8M+FmN5BzgxDrO3Q1cmsj8VJq3eiv3vpPN9acfQUpYM32ISNNo06YNO3fubOpsVJOU34jhUPSxy8pVOyUiEispg0IkFK1nK6tQ9ZGIHFpWrlzJmDFjGDp0KGPHjmX16tUAPPPMMxx77LEMGzaM0047DYDFixdz4oknMnz4cIYOHcqyZcsO+P0P67mP9lckHA0K5RUqKYgI/OLlxSxZt/2g3nNwj3b87IJjGnzdjTfeyPjx4xk/fjyPPPIIN910Ey+++CJ33HEHb775Jj179mTbtm0A3H///UyePJmrr76akpISysvLDzjfSV1SKFVQEJFDzPvvv89VV10FwDXXXMPMmTMBOOWUU7juuut48MEHq778Tz75ZH79619z5513smrVKlq2bHnA75+cJYWgTaFcbQoiAvv1i76x3X///cyePZtXX32V448/nrlz53LVVVdx0kkn8eqrr3LuuefywAMPMGbMmAN6n+QsKYTVpiAih6ZRo0bx1FNPAfDEE09w6qmnArB8+XJOOukk7rjjDjIyMlizZg05OTn079+fm266iXHjxrFgwYIDfv+kLCmEqxqaVVIQkaZTVFREZmZm1f4tt9zCvffey4QJE/j9739PRkYGjz76KAC33nory5Ytw90ZO3Ysw4YN48477+Tvf/87KSkpdOvWjR//+McHnKekDAoRdUkVkUNAxV5qK955551aac8//3yttNtuu43bbju443yTs/oopN5HIiJ1ScqgEA5X9j5Sm4KISKykDAoplb2PVFIQSWrNfc7N/Xm+pAwKlQ3NmhBPJHmlpaWRn5/fbAND5XoKaWlpDbouORuaNaJZJOllZmaSm5tLXl5eU2clYSpXXmuI5AwK6pIqkvRSUlIatCJZskjK6iN1SRURqVtyBoWq6iO1KYiIxErOoKDqIxGROiUsKJjZIDObH/PabmbfNbOOZvaWmS0L/nYIzjczu8fMss1sgZmNSFTeqqa5UPWRiEg1iVyj+TN3H+7uw4HjgSLgBaJrL09394HAdPasxXwOMDB4TQLuS1TeKpfgVElBRKS6xqo+Ggssd/dVwDhgapA+Fbgw2B4HPO5RHwDpZtY9EZnZU1JQm4KISKzGCgpXAE8G213dfX2wvQHoGmz3BNbEXJMbpFVjZpPMbI6Zzdnf/sVqUxARqVvCg4KZtQC+DDxT85hHhxI26JvZ3ae4e5a7Z2VkZOxXniJhTXMhIlKXxigpnAN87O4bg/2NldVCwd9NQfpaoFfMdZlB2kGnaS5EROrWGEHhSvZUHQFMA8YH2+OBl2LSrw16IY0ECmKqmQ6qFE1zISJSp4ROc2FmrYEzgW/GJP8WeNrMJgKrgMuC9NeAc4Fsoj2VJiQqX1p5TUSkbgkNCu5eCHSqkZZPtDdSzXMduCGR+amkaS5EROqWlCOawyHDTNNciIjUlJRBAaLdUktVfSQiUk3SBoVwyNTQLCJSQ9IGhZRQSG0KIiI1JG1QCIeNMrUpiIhUk7RBIRIKqUuqiEgNSRwUjHJVH4mIVJO0QSEcMkpVfSQiUk3SBoWUsHofiYjUlLRBIRwytSmIiNSQtEEhEgppkR0RkRrqDQpmFjazJxorM40pouojEZFa6g0K7l4O9AkWymlWIiGjVL2PRESqiWeW1BxglplNAworE939TwnLVSOIhEMqKYiI1BBPUFgevEJA28Rmp/FEG5rVpiAiEmufQcHdfwFgZm2C/Z2JzlRjiISMkjIFBRGRWPvsfWRmx5rZPGAxsNjM5prZMYnPWmJFwprmQkSkpni6pE4BbnH3Pu7eB/ge8GA8NzezdDN71sw+NbOlZnaymXU0s7fMbFnwt0NwrpnZPWaWbWYLzGzE/j/WvkU0dbaISC3xBIXW7j6jcsfd/w20jvP+fwbecPejgGHAUuA2YLq7DwSmB/sA5wADg9ck4L4432O/hENGqcYpiIhUE09QyDGzn5pZ3+D1E6I9kuplZu2B04CHAdy9xN23AeOAqcFpU4ELg+1xwOMe9QGQbmbdG/g8cdM0FyIitcUTFL4GZADPA88BnYO0fekH5AGPmtk8M3vIzFoDXd19fXDOBqBrsN0TWBNzfW6QVo2ZTTKzOWY2Jy8vL45s1C2sqbNFRGqpt/eRmYWB2939pv289wjgRnefbWZ/Zk9VEQDu7mbWoG9md59CtJ2DrKys/f5WT1GXVBGRWuIZ0Tx6P++dC+S6++xg/1miQWJjZbVQ8HdTcHwt0Cvm+swgLSHCWk9BRKSWeKqP5pnZNDO7xswuqnzt6yJ33wCsMbNBQdJYYAkwDRgfpI0HXgq2pwHXBr2QRgIFMdVMB10kbJSq+khEpJp4RjSnAfnAmJg0J9rGsC83Ak8EcyflABOIBqKnzWwisAq4LDj3NeBcIBsoCs5NmEhI01yIiNQUT5vCAne/a39u7u7zgaw6Do2t41wHbtif99kf4ZBp6mwRkRriaVO4spHy0qgiWmRHRKSWeKqPZpnZX4B/Un2W1I8TlqtGoGkuRERqiycoDA/+3hGT5lRvYzjsaJoLEZHa4pkl9QuNkZHGFg6CgrtjZk2dHRGRQ8Je2xTM7O6Y7ck1jj2WwDw1ipRwNBCoCklEZI/6GppPi9keX+PY0ATkpVGFQ9FHL9MANhGRKvUFBdvLdrOwp6SgbqkiIpXqa1MIBWsdhGK2K4NDOOE5S7BwKPooamwWEdmjvqDQHpjLnkAQ2wX1sP8mjQRBoVTVRyIiVfYaFNy9byPmo9FFwtGaM5UURET2iGdCvGapsvpIbQoiInskbVCorD5S7yMRkT2SNygE1UcapyAissde2xTMrGN9F7r7loOfncYTUfWRiEgt9fU+mku0l5EBvYGtwXY6sJroGsyHLVUfiYjUttfqI3fv5+79gbeBC9y9s7t3As4H/tVYGUyUSFjjFEREaoqnTWGku79WuePurwOjEpelxlE1zYWqj0REqsQTFNaZ2U/MrG/wuh1YF8/NzWylmS00s/lmNidI62hmb5nZsuBvhyDdzOweM8s2swVmNmL/H2vfUlR9JCJSSzxB4UogA3iB6LrMGTRsNbYvuPtwd69clvM2YLq7DwSmB/sA5wADg9ck4L4GvEeDaZoLEZHa4llPYQsw2cxau3vhvs6PwzjgjGB7KvBv4IdB+uPBWs0fmFm6mXV39/UH4T1rqWxTKFVQEBGpss+SgpmNMrMlwNJgf5iZ/S3O+zvwLzOba2aTgrSuMV/0G4CuwXZPYE3MtblBWs38TDKzOWY2Jy8vL85s1BYJVU5zoTYFEZFK8VQf3QWcBeQDuPsnVF9roT6j3X0E0aqhG8ys2nVBqaBBP9XdfYq7Z7l7VkZGRkMurSasCfFERGqJa0Szu6+pkVQe53Vrg7+biLZJnAhsNLPuAMHfTcHpa4FeMZdnBmkJkaIJ8UREaoknKKwxs1GAm1mKmX2foCqpPmbW2szaVm4DXwIWAdPYs5LbeOClYHsacG3QC2kkUJCo9gSInRBPQUFEpNI+G5qB64E/E63fX0t04NoNcVzXFXjBzCrf5x/u/oaZfQQ8bWYTgVXAZcH5rwHnAtlAETChAc/RYHtGNKtNQUSkUr1BwczCwDXufnVDb+zuOcCwOtLzgbF1pDvxBZuDIhJWSUFEpKZ6q4/cvRy4qpHy0qj29D5SUBARqRRP9dFMM/sL8E+gapyCu3+890sOfWFVH4mI1BJPUBge/L0jJs2BMQc/O40nRdVHIiK1xDOi+QuNkZHGpmkuRERqi6ekgJmdBxwDpFWmufsde7/i0Fc5TkGD10RE9ohnmov7gcuBG4kusnMp0CfB+Uq4PSUFtSmIiFSKZ/DaKHe/Ftjq7r8ATgaOTGy2Ei+iaS5ERGqJJyjsCv4WmVkPoBTonrgsNQ4zIxwytSmIiMSIp03hFTNLB34PfEy059FDCc1VIwmHTL2PRERixNP76JfB5nNm9gqQ5u4Fic1W44iETOMURERi7DMomNm1daTh7o8nJkuNJ6KSgohINfFUH50Qs51GdN6ij4HDPyiEQ2pTEBGJEU/10Y2x+0H7wlMJy1EjipYUVH0kIlIprkV2aigE+h3sjDSFaJuCSgoiIpXiaVN4mT1LZoaAwcDTicxUYwmH1aYgIhIrnjaFP8RslwGr3D03QflpVCmhkIKCiEiMeNoU/tMYGWkK0cFralMQEakUz9xHO8xsex2vHWa2PY7rw2Y2LxjjgJn1M7PZZpZtZv80sxZBemqwnx0c73ugD7cv4ZBpmgsRkRjxNDTfDdxGdI3mTOCHwN3u3tbd28Vx/WRgacz+ncBd7j4A2ApMDNInEp1faQBwV3BeQqWoS6qISDXxBIUvu/vf3H2Hu2939/uAcfHc3MwygfMIpsUwMyO6OM+zwSlTgQuD7XHBPsHxscH5CaNpLkREqosnKBSa2dVBNVDIzK4mZlnOfbgb+AFQWXHfCdjm7mXBfi7REgjB3zUAwfGC4PxqzGySmc0xszl5eXlxZqNumuZCRKS6eILCVcBlwEZgE9H1FK7a10Vmdj6wyd3nHlAOa3D3Ke6e5e5ZGRkZB3SviLqkiohUE0/vo5XEWV1UwynAl83sXKLTY7QD/gykm1kkKA1kAmuD89cCvYBcM4sA7YH8/XjfuEVCIYpKyvZ9oohIkthrScHMvmFmA4NtM7NHzKzAzBaY2Yh93djdf+Tume7eF7gCeMfdrwZmAJcEp40HXgq2pwX7BMffcfeE/oyPhLWegohIrPqqjyYDK4PtK4FhQH/gFqK/+PfXD4FbzCybaJvBw0H6w0CnIP0Woj2eEkqzpIqIVFdf9VGZu5cG2+cDj7t7PvC2mf2uIW/i7v8G/h1s5wAn1nHObqLtFY0mrLmPRESqqa+kUGFm3c2scrrst2OOtUxsthpHJBzSLKkiIjHqKyn8DzAHCAPT3H0xgJmdDuQ0Qt4SLqI1mkVEqtlrUHD3V8ysD9DW3bfGHJoDXJ7wnDUCTXMhIlJdvV1Sg26jW2ukxTtw7ZCXEtI0FyIisfZnkZ1mQ+spiIhUl9RBIUXLcYqIVBPPIjuYWU+gT+z57v5uojLVWMKhEOVqUxARqRLPcpx3Em1YXgKUB8kOHPZBIRI2SlVSEBGpEk9J4UJgkLsXJzozjU1dUkVEqounTSEHSEl0RpqCprkQEakunpJCETDfzKYDVaUFd78pYblqJOFQCHcor3DCoYSu5yMicliIJyhMC17NTiQcDQRlFRWEQ+Emzo2ISNOLZz2Fqfs653AVCUoHalcQEYmKp/fRQOA3wGCii+UA4O79E5ivRhEJR5tUNNWFiEhUPA3NjwL3AWXAF4DHgf9LZKYai0oKIiLVxRMUWrr7dMDcfZW7/xw4L7HZahyVjctl5RqrICIC8TU0F5tZCFhmZt8hupZym8Rmq3GkVDU0q6QgIgLxlRQmA62Am4Djga+yZy3lvTKzNDP70Mw+MbPFZvaLIL2fmc02s2wz+6eZtQjSU4P97OB43/19qHiFQ9HHV/WRiEjUPoOCu3/k7juBLe4+wd0vdvcP4rh3MTDG3YcBw4GzzWwkcCdwl7sPIDot98Tg/InA1iD9ruC8hKpsUyhV9ZGICBBHUDCzk81sCfBpsD/MzP62r+s8amewmxK8HBgDPBukTyU6jQbAuGCf4PhYM0voiLLKcQoqKYiIRMVTfXQ3cBaQD+DunwCnxXNzMwub2XxgE/AWsBzYFizeA5AL9Ay2ewJrgvcoAwqATnXcc5KZzTGzOXl5efFkY68qSwpqUxARiYprPQV3X1MjqbzOE2tfV+7uw4FM4ETgqIZlr857TnH3LHfPysjIOKB7RYI2hTKNUxARAeILCmvMbBTgZpZiZt8HljbkTdx9GzADOBlIN7PKXk+ZRHszEfztBRAcb09QOkmUcMw0FyIiEl9QuB64gWj1zlqijcY37OsiM8sws/RguyVwJtFgMgO4JDhtPPBSsD2NPb2aLgHecfeE/oRX9ZGISHXxzH20Gbh6P+7dHZhqZmGiwedpd38laLR+ysx+BcwDHg7Ofxj4u5llA1uAK/bjPRtE1UciItXtNSiY2T31XbivqbPdfQFwXB3pOUTbF2qm7wYure+eB5t6H4mIVFdfSeF6YBHwNLAOaHYLDlROc6ElOUVEouoLCt2J/nK/nOhkeP8Eng0ajZuFlMoRzao+EhEB6mlodvd8d7/f3b8ATADSgSVmdk2j5S7BwmpoFhGpJp71FEYAVxLtPfQ6MDfRmWosKeqSKiJSTX0NzXcQnSJ7KfAU8KOYkcjNQljrKYiIVFNfSeEnwApgWPD6dTAVkRGd2mho4rOXWOqSKiJSXX1BoV+j5aKJRFR9JCJSzV6DgruvasyMNAWNaBYRqS6uCfGaqz3LcSooiIhAkgeFSDhoU1BJQUQEiG+RncnxpB2OIlW9j9SmICIC8ZUU6lqP+bqDnI8mUdnQXKrqIxERoP5xClcCVwH9zGxazKG2RGcxPexVdknVOAURkaj6uqS+B6wHOgN/jEnfASxIZKYaS1B7pDYFEZHAvrqkrgJONrOuwAnBoaXNZWSzmZESNsrK1aYgIgLxNTRfCnxIdMbUy4DZZnZJ/VcdPsIhU/WRiEhgnxPiEZ3u4gR33wTRZTaBt4FnE5mxxhIJhdTQLCISiKf3UagyIATy47nOzHqZ2QwzW2Jmiyu7sZpZRzN7y8yWBX87BOlmZveYWbaZLQhmZ024SNjUJVVEJBBPUHjDzN40s+vM7DrgVeC1OK4rA77n7oOBkcANZjYYuA2Y7u4DgenBPsA5wMDgNQm4r0FPsp8iIVNDs4hIYJ/VR+5+q5ldBIwOkqa4+wtxXLeeaO8l3H2HmS0FegLjgDOC06YC/wZ+GKQ/7u4OfGBm6WbWPbhPwoRDpmkuREQC8bQpAMwCSgEn2ujcIGbWFzgOmA10jfmi3wB0DbZ7AmtiLssN0hIaFCKhkEoKIiKBeNoGLiMaCC5hP3ofmVkb4Dngu+6+PfZYUCpo0DeymU0yszlmNicvL68hl9ZJbQoiInvEU1K4nf3sfWRmKUQDwhPu/nyQvLGyWsjMugOVjdhrgV4xl2cGadW4+xRgCkBWVtYB/8SPhIxSlRRERIDE9j4y4GGig93+FHNoGnvmUxoPvBSTfm3QC2kkUJDo9gSIVh+Vq01BRASIr6Twhpm9CTwZ7F8OvB7HdacA1wALzWx+kPZj4LfA02Y2keiI6cuCY68B5wLZQBEwIa4nOEBh9T4SEamSyN5HM4mu51yXsXWc78AN+7rvwZYSNi3HKSISiKv3UdAe8DyAmYXM7Gp3fyKhOWskmuZCRGSPvbYNmFk7M/uRmf3FzL4U1PV/B8hhT5XPYS86zYVKCiIiUH9J4e/AVuB94OtE2wMMuNDd59dz3WElEjYFBRGRQH1Bob+7DwEws4eIDiLr7e67GyVnjSQcMnaVqvpIRATq71paWrnh7uVAbnMLCAAp4ZCmuRARCdRXUhhmZpUjkA1oGewb0c5C7RKeu0agLqkiInvUt/JauDEz0lQiIU1zISJSKZ4Rzc1aRNVHIiJVFBRUfSQiUiXpg0J0PQVVH4mIgIJCMM2FSgoiIqCgoGkuRERiJH1Q0DQXIiJ7KCiopCAiUiXpg0JYbQoiIlWSPiikhEJ1BoWKCmfR2gKiyzyIiCSHpA8KlQ3NsV/+u0rK+fYTH3P+vTN5Zm5uE+ZORKRxJX1QiISii8NVlhY2bt/NZQ+8z5tLNtClbSr3vrNMDdEikjQSFhTM7BEz22Rmi2LSOprZW2a2LPjbIUg3M7vHzLLNbIGZjUhUvmqKhKP/BOUVzqr8Qi786yyW5+3kwWuy+O3FQ1izZRfPqbQgIkkikSWFx4Cza6TdBkx394HA9GAf4BxgYPCaBNyXwHxVU1lSKCwu49tPfExRSTnPXH8yXxzclS8M6sKwXunc+042JWUqLYhI85ewoODu7wJbaiSPA6YG21OBC2PSH/eoD4B0M+ueqLzFioSjQeGOV5aweN12/njpMI7p0R4AM+O7XxzI2m27eLaBpQU1UIvI4ai+9RQSoau7rw+2NwBdg+2ewJqY83KDtPXUYGaTiJYm6N279wFnqLKk8NL8dUwc3Y8vDu5a7fgZR2YwvFc6f52RTWl5BR+v3srC3AJ2FJdRVl5BhcPksQP52uh+VdcU7Crl8gfeJ6NtKn++4jg6tm5xwPkUEWkMTdbQ7NGf0g3+Oe3uU9w9y92zMjIyDjgf4VD0n2BYZnt+ePZRtY6bGbeceSRrt+3iZ9MW80FOPgO7tuGLR3flgmE9OKpbW3756hLeXLwBiLZN3PTkPLI37WT2ii18+S8zWbJue637iogcihq7pLDRzLq7+/qgemhTkL4W6BVzXmaQlnBHdW/L0d3bce+VI2gRqTtGnnZkBi98exRd2qXRo30aZlZ1bHdpOZc/8D43/3M+z14/ipfmr+U/n+fx668M4Zge7fjm3+dy8X3vce+Vx9UqhYiIHGoau6QwDRgfbI8HXopJvzbohTQSKIipZkqoEb078PrkU+ndqVW95x3XuwM901tWCwgAaSlhHrw2i3ZpKVz10Ac88G4O14zsw1Un9WZYr3Sm3XgKR3Rpzfee+YSthSXVrnV3TbEhIoeURHZJfRJ4HxhkZrlmNhH4LXCmmS0DvhjsA7wG5ADZwIPAtxOVr0To0i6Nh8ZnUVxawUn9OvI/Fwzec6xtGn+4dBg7dpdy99ufV6VXVDjX/99cjvnZG3z1odn8dUY2n23YUe/7uDs7dpeyq6ScCgUTEUkAO5x7yWRlZfmcOXOaOhtVNm3fTXqrFnVWQ/30xUX848PVvD75VI7s2pa73/6cu99expcGd2X1liI+3bCDkMGNYwZy45gBVeMnlq7fzrRP1rFobQFL129n8849pY22qREuGtGTr5/an14d6y/piIhUMrO57p5V5zEFhcaxpbCEM34/g2G90rn25L584/E5XDwikz9cOhQzI29HMb95bSnPz1tLVp8OfOO0/jz54Wr+/VkeKWFjYJe2HNOjHUd0aYM7FJeVsyq/iFcWrKPC4fyh3Tl3SHdG9u9E+5Yptd5/7qot3PHyEq48sTdXnHjgvbZE5PCloHCIeHTWCn7x8hJaREIc1a0tT3/zZNJSwtXOeXHeWn7y4iJ2FpfRqXULJpzSl6+O7EN6q7q7ta4v2MUjM1fw5Idr2FlcRshgSGY6pw/szOmDMjiqWzvufvtzHpq5AoA2LSLMuPUMOrdJrbrH6vwiUlNCdG2XlriHF5FDhoLCIaK0vIJz//xfthSW8PKNo+mR3rLO83K3FrFobQFnDOpSK2jsTUlZBfNWb2XW8nxmLstj/pptVDiEDCocrj6pN1ec0Juv/G0Wl2Zl8puLhgKQvWknX/nrLCrc+dmXj+HS4zNrNaaLSPOioHAI2VpYQlmFk9E2dd8nH4CColJmZm9m7qqtjDmqC6MHdgbgjpeX8Oh7K3jlxtFkdmjFV/46i4JdpRzRpQ0frtjClwZ35ZunH0FhcRnbdpVSWlZBakqItEiYbu3TGNy9HaHQ/geN8grnX4s3MLJ/Jzo0wqC+uau28PIn6/mf8wcfUL5FmhMFBalSUFTKGX+YwZFd25KaEub95Zt54usjyerTgYdm5vCHNz+npJ5ZYbu2S2Xs0V2jX+qtUmiTGqFdyxQ6tmpB+5Yp9X7xuju/eHkJj723ks5tWvDLccdyzpDEzWZSXFbOWXe9y8r8Ih65LosxR2mciAjUHxQae/CaNLH2rVK45cwj+elLiwH4zUVDOLFfRwAmnXYEXxrcjexNO0lvlUJ6qxRSwiGKyyooLq1g2aYdvL10Iy/NW8s/Zq+ude+QQff2Lbn25D58dWQfWqdW/89ryrs5PPbeSi4ekcmnG7bzrSc+5twh3Zg4uh9DeqbvdfBgPAqLy9hdWk6nmLaSh2euYGV+Ea1bhHl45opqQcHdyd26S722RGpQSSEJlZVXMHHqHI7t2Y5bz6o9tce+FJeVk5NXyM7iMnbuLqNgVylbi0rYWljC3NVbmZWdT8fWLbhuVF9G9O5An06t+GjlFm55+hPOG9qde684jnJ3prybw5/fXkZJeQUtU8Ic36cD3dqnkRoJkRoJk9mhJcf2bM/R3WiaCEoAABFnSURBVNuSv7OE/2Zv5v3lm0lLCXPB0B6cMqAzRSVlPDJrJY/NWoE73H/N8ZwyoDMbCnYz5o//5pQBnTmudzq/e+MzXp98Kkd3bwfAPdOX8ae3PufnFwzmulP67fVZS8oq6gxWGwp2065lhFYt9gS+ncVl/O+rS0hLCXPzmUfSLq12LzCIVqEVlZTRdi/HITqOZe02BS1JDFUfSaOau2or90xfxn8+z6uWPrJ/R6Z+7URSI3saz7cWljB7RT4f5Gzho5Vb2FpYQnFZBbtKyykqKa917x7t09hZXMb23WWkt0qhtKyCwpJyvjS4KyvzC1mxuZDfXTKUf3+Wx+uLNvD2zafTrmWEk3/zDucP7c7vLx3GwtwCvvK3WbRsEWZncRl/uXIE5w2tXY319pKN3PCPjzmxX0d+cNZRDMlsT96OYn73xqc8MzeXru1Suf28wVwwtDvL8wq5/v/mkpO3E4DObVK5Y9wxnHVMt2oN9yVlFUx47EMWrCnggWuOZ9SAzrXed3dpOTf/cz6vL9rABcN68D/nD054G1QilJZX8PqiDYw5qgttUpuuUmJ53k5e/mQd159+RNwdNxrql68sISUc4rZzGv4jqykoKEiT2Lh9Nzl5hazKj5YqLjuh115/Pddl0/bdLF63nSXrt9OuZQqjB3Smb6dWlJY7/12WxysL1hMOGd84tT+DurWlYFcp1/99Lu/n5ANw05gB3PKlQQD85MWFPP1RLjNuPYPrHvmQ7btLmfad0Xz7iY9ZmFvA1K+dyMlHdKp679cXrufGJ+fRr3NrNu8sZmtRKacfmcHHq7ayu6ycq0/qw8ert7Igt4Dj+3Tg0/XbSUsJc++Vx9E6NcJtzy9k6frtnH1MN3578RDSW7XA3fn+Mwt47uNcerRPI29nMb+/ZBgXHtez6n0LdpUy6fE5zF6xhfOGduetxRtp2SLMLWceSff2aRSXVVDhzvF9OpDZYU8poqColDmrtnBk17bVSheL1hbwyKwVFBaXkdE2lS5t0zhjUAZDM9OrztlaWMLNT89n4/ZiJo8dUCuQQfQL/rMNOyivcIb1SmdfKiqc7z3zCS/MW8sZgzJ4ePwJhIP2JnfnX0s2ckRGGwZ0aVPn9eXBiP+QwZ0XD91rl+xdJeXsLi3fa6eFj1Zu4etT51Cwq5SvjuzNry4css+8783O4jJuf2Ehpw7M4JLjM6vSF60t4Px7Z2IGb0w+jUHd2lYdKygqZfqnG8nduos1W4ro2i6Nm888surfoqkoKEjSKC4r5/YXFrFk3Xae+9YoWraI/jJcnreTsX/8Dz3TW7J22y4em3ACZwzqwraiEi65/302FOzmvCHdOb5vB4rLKvj5tMUM75XOoxNOAOChd3OY+v4qjuudzk/PH8wRGW0or3Ce+mg1v3vjM/p1bs3frh5R1c24rLyCB/+7gj+99RkZbVK5+4rj+CAnnz+99Tnf/eJAJpzSj2/+fQ4f5GzhulF9g3m14Nm5uSzP28kfLh3GuOE9yd60kx8/v5APV9ZcmgSO6taWkf07sWT9duau2lo1j9aI3umcdUw3Zi3P593P82ibGqF7ehp5O6LBzQzGn9yX7581iNytRXzj8TlsLCimR3oaK/OLGNKzPeOG92BbUSl5O4pZsbmQBWu3sbs02gHhe2ceyXfGDKgKHJVVXZkdonODuTu/enUpD89cwakDO/PfZZuZOLofPz1/MKXlFdz+wkKenhNdn2TMUV34+uh+nHxEp2qB6K8zsvn9m58RDhnd26dx/1eP59ie7auOL15XwFMfruHFeWvB4OlvnlxVNVjp1QXrufnp+WSmt+T4Ph14Zm4u9391BGcfGy0VZm/aycMzc+jTqTWjB3Sut2ddQVEp1z76IZ+s2UZaSoi3bj69Kvhe9+iHzFu9jYoKZ9SATjxwTfS7tqSsgovum8WitdFZkju3acHmnSXVfqzsTUFRKe8t38zidds5vk8HRg3oVK2EfaAUFESArz32Ee98uomrTurNr7+y5xfjum27+MXLi/kgZwsFu0qBaFXXw+NPqNVYXpfisnJSQqE6v1AW5G7jxifnsWZLERUOF43oyR8vHYaZUVxWzo+eW8jz8/ZMCNw2LcLfrh7BqQP3TAtfUeEsWR/9YkmNhCgtd2Zlb2b6pxuZs3IrA7u2ZexRXTj5iE58kruNafPX8emGHXRu04Kvje7HV0f2qSqhbd9dyh/f/IzHP1hFt3ZpFOwqpU1qhPuvOZ5hmem8MG8td7/9OblbdxGyaDVYzw4tGd4rnRG9OzDj0008P28tE0f34/Zzj+bDlVv431eXsnBtAb06tuTC4T0pr3D+9u/lXDeqLz+7YHBVj7Ofnj+YGZ9uYmb2Zr59xhGkRsI8/v5K8gtL+OLRXfjT5cNpl5bCgtxtXPS39zjr2G5MOrU/3/q/uWwuLOGsY7qxoWAXq7cUsXF7MS0iIc49thsf5GzBcZ7/9in0TG9JaXkF905fxj3vZJPVpwMPXptF69QIl97/His2F/L6d09jdk4+P3lxEWUVXrWqYnqrFHp1aEWnNi3IaJPK4B7tyOrTka7tUhn/6Ecs37ST/7lgML99/VNG9OnA1AknMHfVVi65/31+cPYgSsoquPvtZbz8ndEMyWzPb1//lPv/s5y7Lx/O2cd2IzUS4gfPLuCZubk8PD6LsUfX7g23aG0BP31pEZ8E44wqtUmNMOaoLnzz9P5Vi4AdCAUFEeCzDTt47L0V3H7e4DrruCsqnOV5O1mxuZDTjsw4aPXPO3aX8stXllBYUs5dlw2v1XC9q6SccnfcndRIuEG9sCoqvM5gtGZLERltU/f6DHNXbeX2FxbSJjXCX68eUW00e1l5BQW7Sklv1aJWNUdFhXPHK9Ev+SMyWrM8r5Ae7dO4emQfPsjJZ1b2Ziocxg3vwV2XDScUMsrKK5jw2Ef8d9lmIiHjNxcN4dKs6Ez5u0vLefz9lfzujc/o3bEVd18xnO8+NZ9dpeW8Mfk02rdKYUthCT94dgGL1xXQq2MrendsVVWaSW/Vgs827OCS+9+jS9tU/nDpMH7+8hI+WbONi0dk8r9fObbq32BVfiHn3TOT1EiI/MISTuzXkXuuOI6QwczszXy4Ygsbtu9m885iNm4vJm9HcdVzp6WEmHJNFqcdmcFjs1bw85eX8OcrhvOP2atZnlfIuz84g/IK57TfzWBoZjrXn34EVz30AVec0KtqoGjl815833us2VLEKzdWn535wxVbmPjYR7RJi3BZVi9OHdiZwT3aMXvFFt5ctIHXF21g++5SrjihF7ecOeiA2pkUFESklsr/9xs6gt3dufedbB6dtYKvn9qfiaP7VX3xbtq+m7mrtjL26K7VglvBrlJ+9coSvnJczzob12fn5HPDPz5m884SzOCJr5/EqCNqn7c3H+Tkc+3DH1JSXkH7lin8+itD6uw88PIn6/je059w/RlHcFPMxJN1WbdtF3NWbWXx2gLOPrYbx/XuAETbOy667z0+37CDXaXl/OyCwUwIerA98J/l/Ob1T0lvlUKHVi149abR1XqoQXRamQv+MpNOrVswflRfxhzVhZzNhXzz73Pokd6SJ75+Et3b157toKColHvfWcZj760kLSXMby4awgXDesT9bxRLQUFEDjp3P6hToqwv2MUPn1vISf06csMXBjT4+ulLN/Lawg3cetYgurXf+zxepeUVpNQTDOKxdP12Lrh3Jl3apjLj1jOq6vt3lZRz2u9nsLWwhOe+NWqvjfLvZW/m9hcXsWJzIQBmcHS3djw+8cRq85LVJSdvJ79+7VMmjx3IkMz9q0pSUBAROchmfLaJjDap1RrAAeav2ca2ohLOGNRln/fIydvJO59uYtOOYm74woA6ZzhOBI1oFhE5yL6wly/94XF02a3UP6MN/TPq7pbbVBp7OU4RETmEHVJBwczONrPPzCzbzG5r6vyIiCSbQyYomFkY+CtwDjAYuNLMBtd/lYiIHEyHTFAATgSy3T3H3UuAp4BxTZwnEZGkcigFhZ7Ampj93CCtGjObZGZzzGxOXl5ezcMiInIADqWgEBd3n+LuWe6elZGRse8LREQkbodSUFgL9IrZzwzSRESkkRxKQeEjYKCZ9TOzFsAVwLQmzpOISFI5pEY0m9m5wN1AGHjE3f93H+fnAav28+06A5v389rDWTI+dzI+MyTncyfjM0PDn7uPu9dZ/35IBYXGZGZz9jbMuzlLxudOxmeG5HzuZHxmOLjPfShVH4mISBNTUBARkSrJHBSmNHUGmkgyPncyPjMk53Mn4zPDQXzupG1TEBGR2pK5pCAiIjUoKIiISJWkDArJMEW3mfUysxlmtsTMFpvZ5CC9o5m9ZWbLgr8dmjqvB5uZhc1snpm9Euz3M7PZwef9z2BwZLNiZulm9qyZfWpmS83s5CT5rG8O/vteZGZPmllac/u8zewRM9tkZoti0ur8bC3qnuDZF5jZiIa+X9IFhSSaorsM+J67DwZGAjcEz3kbMN3dBwLTg/3mZjKwNGb/TuAudx8AbAUmNkmuEuvPwBvufhQwjOjzN+vP2sx6AjcBWe5+LNFBr1fQ/D7vx4Cza6Tt7bM9BxgYvCYB9zX0zZIuKJAkU3S7+3p3/zjY3kH0S6In0WedGpw2FbiwaXKYGGaWCZwHPBTsGzAGeDY4pTk+c3vgNOBhAHcvcfdtNPPPOhABWppZBGgFrKeZfd7u/i6wpUby3j7bccDjHvUBkG5m3RvyfskYFOKaors5MbO+wHHAbKCru68PDm0AujZRthLlbuAHQEWw3wnY5u5lwX5z/Lz7AXnAo0G12UNm1ppm/lm7+1rgD8BqosGgAJhL8/+8Ye+f7QF/vyVjUEgqZtYGeA74rrtvjz3m0f7IzaZPspmdD2xy97lNnZdGFgFGAPe5+3FAITWqiprbZw0Q1KOPIxoUewCtqV3N0uwd7M82GYNC0kzRbWYpRAPCE+7+fJC8sbI4Gfzd1FT5S4BTgC+b2Uqi1YJjiNa1pwfVC9A8P+9cINfdZwf7zxINEs35swb4IrDC3fPcvRR4nuh/A83984a9f7YH/P2WjEEhKaboDurSHwaWuvufYg5NA8YH2+OBlxo7b4ni7j9y90x370v0c33H3a8GZgCXBKc1q2cGcPcNwBozGxQkjQWW0Iw/68BqYKSZtQr+e6987mb9eQf29tlOA64NeiGNBApiqpnikpQjmhs6RffhyMxGA/8FFrKnfv3HRNsVngZ6E512/DJ3r9mIddgzszOA77v7+WbWn2jJoSMwD/iquxc3Zf4ONjMbTrRxvQWQA0wg+qOvWX/WZvYL4HKive3mAV8nWofebD5vM3sSOIPo9NgbgZ8BL1LHZxsEx78QrUYrAia4+5wGvV8yBgUREalbMlYfiYjIXigoiIhIFQUFERGpoqAgIiJVFBRERKSKgoJIPcys3Mzmx7wO2qRyZtY3duZLkUNBZN+niCS1Xe4+vKkzIdJYVFIQ2Q9mttLMfmdmC83sQzMbEKT3NbN3grnsp5tZ7yC9q5m9YGafBK9Rwa3CZvZgsCbAv8ysZZM9lAgKCiL70rJG9dHlMccK3H0I0RGkdwdp9wJT3X0o8ARwT5B+D/Afdx9GdF6ixUH6QOCv7n4MsA24OMHPI1IvjWgWqYeZ7XT3NnWkrwTGuHtOMPHgBnfvZGabge7uXhqkr3f3zmaWB2TGTrcQTGn+VrBQCmb2QyDF3X+V+CcTqZtKCiL7z/ey3RCxc/KUo3Y+aWIKCiL77/KYv+8H2+8RnaEV4GqikxJCdMnEb0HVGtLtGyuTIg2hXyUi9WtpZvNj9t9w98puqR3MbAHRX/tXBmk3El0B7Vaiq6FNCNInA1PMbCLREsG3iK4WJnJIUZuCyH4I2hSy3H1zU+dF5GBS9ZGIiFRRSUFERKqopCAiIlUUFEREpIqCgoiIVFFQEBGRKgoKIiJS5f8Bjl9YEboenS8AAAAASUVORK5CYII=\n", + "text/plain": [ + "
" ] - }, + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "plot_the_loss_curve(history.epoch, history.history['loss'])" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "CPakaxBaZZNA" + }, + "source": [ + "You can see above how the loss (cost) i.e. MSE has changed just by changing the epochs and keeping the learning rate same as 0.01 (i.e. the first model we built)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "6WoG362ohrQF" + }, + "source": [ + "### Exercise 2\n", + "\n", + "Test several epoch values to see the impact of varying this value when defining your model." + ] + }, + { + "cell_type": "code", + "execution_count": 30, + "metadata": { + "colab": {}, + "colab_type": "code", + "collapsed": true, + "id": "QUmO4ehKqC2_", + "jupyter": { + "outputs_hidden": true + } + }, + "outputs": [ { - "cell_type": "code", - "metadata": { - "id": "QUmO4ehKqC2_", - "colab_type": "code", - "colab": {} - }, - "source": [ - "# Play with epochs\n", - "learning_rate = 0.01 \n", - "epochs = ? # Replace ? with an integer\n", - "optimizer = RMSprop(learning_rate)\n", - "model.compile(loss='mean_squared_error',optimizer=optimizer) # Compile the model\n", - "model.fit(X_train, y_train, epochs=epochs, batch_size=30) # Fit the model\n", - "model.evaluate(X_test, y_test) # Evaluate the model" - ], - "execution_count": null, - "outputs": [] + "name": "stdout", + "output_type": "stream", + "text": [ + "Train on 323 samples\n", + "Epoch 1/100\n", + "323/323 [==============================] - 0s 1ms/sample - loss: 494.3871\n", + "Epoch 2/100\n", + "323/323 [==============================] - 0s 43us/sample - loss: 489.0458\n", + "Epoch 3/100\n", + "323/323 [==============================] - 0s 47us/sample - loss: 445.1431\n", + "Epoch 4/100\n", + "323/323 [==============================] - 0s 48us/sample - loss: 405.9655\n", + "Epoch 5/100\n", + "323/323 [==============================] - 0s 43us/sample - loss: 369.8630\n", + "Epoch 6/100\n", + "323/323 [==============================] - 0s 46us/sample - loss: 335.7336\n", + "Epoch 7/100\n", + "323/323 [==============================] - 0s 47us/sample - loss: 304.4721\n", + "Epoch 8/100\n", + "323/323 [==============================] - 0s 42us/sample - loss: 275.6164\n", + "Epoch 9/100\n", + "323/323 [==============================] - 0s 57us/sample - loss: 248.6708\n", + "Epoch 10/100\n", + "323/323 [==============================] - 0s 50us/sample - loss: 224.3012\n", + "Epoch 11/100\n", + "323/323 [==============================] - 0s 44us/sample - loss: 202.2789\n", + "Epoch 12/100\n", + "323/323 [==============================] - 0s 45us/sample - loss: 181.8262\n", + "Epoch 13/100\n", + "323/323 [==============================] - 0s 43us/sample - loss: 163.7843\n", + "Epoch 14/100\n", + "323/323 [==============================] - 0s 42us/sample - loss: 147.9701\n", + "Epoch 15/100\n", + "323/323 [==============================] - 0s 48us/sample - loss: 134.3894\n", + "Epoch 16/100\n", + "323/323 [==============================] - 0s 60us/sample - loss: 122.8483\n", + "Epoch 17/100\n", + "323/323 [==============================] - 0s 46us/sample - loss: 113.2430\n", + "Epoch 18/100\n", + "323/323 [==============================] - 0s 44us/sample - loss: 105.8269\n", + "Epoch 19/100\n", + "323/323 [==============================] - 0s 45us/sample - loss: 100.0494\n", + "Epoch 20/100\n", + "323/323 [==============================] - 0s 50us/sample - loss: 96.1901\n", + "Epoch 21/100\n", + "323/323 [==============================] - 0s 48us/sample - loss: 93.7272\n", + "Epoch 22/100\n", + "323/323 [==============================] - 0s 46us/sample - loss: 92.0279\n", + "Epoch 23/100\n", + "323/323 [==============================] - 0s 43us/sample - loss: 91.2668\n", + "Epoch 24/100\n", + "323/323 [==============================] - 0s 49us/sample - loss: 91.0421\n", + "Epoch 25/100\n", + "323/323 [==============================] - 0s 51us/sample - loss: 91.0423\n", + "Epoch 26/100\n", + "323/323 [==============================] - 0s 53us/sample - loss: 91.0008\n", + "Epoch 27/100\n", + "323/323 [==============================] - 0s 46us/sample - loss: 91.0129\n", + "Epoch 28/100\n", + "323/323 [==============================] - 0s 44us/sample - loss: 91.0115\n", + "Epoch 29/100\n", + "323/323 [==============================] - 0s 44us/sample - loss: 91.0153\n", + "Epoch 30/100\n", + "323/323 [==============================] - 0s 44us/sample - loss: 90.9617\n", + "Epoch 31/100\n", + "323/323 [==============================] - 0s 42us/sample - loss: 91.0191\n", + "Epoch 32/100\n", + "323/323 [==============================] - 0s 44us/sample - loss: 90.9161\n", + "Epoch 33/100\n", + "323/323 [==============================] - 0s 44us/sample - loss: 90.9823\n", + "Epoch 34/100\n", + "323/323 [==============================] - 0s 42us/sample - loss: 91.1019\n", + "Epoch 35/100\n", + "323/323 [==============================] - 0s 42us/sample - loss: 90.9450\n", + "Epoch 36/100\n", + "323/323 [==============================] - 0s 44us/sample - loss: 90.9584\n", + "Epoch 37/100\n", + "323/323 [==============================] - 0s 44us/sample - loss: 90.9506\n", + "Epoch 38/100\n", + "323/323 [==============================] - 0s 42us/sample - loss: 90.9083\n", + "Epoch 39/100\n", + "323/323 [==============================] - 0s 44us/sample - loss: 91.0654\n", + "Epoch 40/100\n", + "323/323 [==============================] - 0s 44us/sample - loss: 91.0270\n", + "Epoch 41/100\n", + "323/323 [==============================] - 0s 43us/sample - loss: 91.0374\n", + "Epoch 42/100\n", + "323/323 [==============================] - 0s 43us/sample - loss: 91.0251\n", + "Epoch 43/100\n", + "323/323 [==============================] - 0s 44us/sample - loss: 90.9016\n", + "Epoch 44/100\n", + "323/323 [==============================] - 0s 42us/sample - loss: 90.9567\n", + "Epoch 45/100\n", + "323/323 [==============================] - 0s 40us/sample - loss: 90.8898\n", + "Epoch 46/100\n", + "323/323 [==============================] - 0s 46us/sample - loss: 90.8758\n", + "Epoch 47/100\n", + "323/323 [==============================] - 0s 43us/sample - loss: 91.1318\n", + "Epoch 48/100\n", + "323/323 [==============================] - 0s 44us/sample - loss: 90.9979\n", + "Epoch 49/100\n", + "323/323 [==============================] - 0s 44us/sample - loss: 90.9767\n", + "Epoch 50/100\n", + "323/323 [==============================] - 0s 45us/sample - loss: 91.0335\n", + "Epoch 51/100\n", + "323/323 [==============================] - 0s 44us/sample - loss: 91.0930\n", + "Epoch 52/100\n", + "323/323 [==============================] - 0s 45us/sample - loss: 90.8934\n", + "Epoch 53/100\n", + "323/323 [==============================] - 0s 43us/sample - loss: 91.0619\n", + "Epoch 54/100\n", + "323/323 [==============================] - 0s 42us/sample - loss: 90.8928\n", + "Epoch 55/100\n", + "323/323 [==============================] - 0s 55us/sample - loss: 90.9104\n", + "Epoch 56/100\n", + "323/323 [==============================] - 0s 47us/sample - loss: 91.0430\n", + "Epoch 57/100\n", + "323/323 [==============================] - 0s 59us/sample - loss: 90.9069\n", + "Epoch 58/100\n", + "323/323 [==============================] - 0s 41us/sample - loss: 90.9546\n", + "Epoch 59/100\n", + "323/323 [==============================] - 0s 41us/sample - loss: 91.0392\n", + "Epoch 60/100\n", + "323/323 [==============================] - 0s 45us/sample - loss: 91.0566\n", + "Epoch 61/100\n", + "323/323 [==============================] - 0s 45us/sample - loss: 90.9273\n", + "Epoch 62/100\n", + "323/323 [==============================] - 0s 44us/sample - loss: 90.9120\n", + "Epoch 63/100\n", + "323/323 [==============================] - 0s 41us/sample - loss: 91.1432\n", + "Epoch 64/100\n", + "323/323 [==============================] - 0s 44us/sample - loss: 90.9020\n", + "Epoch 65/100\n", + "323/323 [==============================] - 0s 45us/sample - loss: 90.9426\n", + "Epoch 66/100\n", + "323/323 [==============================] - 0s 42us/sample - loss: 90.9679\n", + "Epoch 67/100\n", + "323/323 [==============================] - 0s 45us/sample - loss: 91.0428\n", + "Epoch 68/100\n", + "323/323 [==============================] - 0s 40us/sample - loss: 91.0305\n", + "Epoch 69/100\n", + "323/323 [==============================] - 0s 45us/sample - loss: 90.9463\n", + "Epoch 70/100\n", + "323/323 [==============================] - 0s 44us/sample - loss: 91.0044\n", + "Epoch 71/100\n", + "323/323 [==============================] - 0s 44us/sample - loss: 90.9508\n", + "Epoch 72/100\n", + "323/323 [==============================] - 0s 43us/sample - loss: 91.0512\n", + "Epoch 73/100\n", + "323/323 [==============================] - 0s 43us/sample - loss: 90.8971\n", + "Epoch 74/100\n", + "323/323 [==============================] - 0s 50us/sample - loss: 90.9268\n", + "Epoch 75/100\n", + "323/323 [==============================] - 0s 46us/sample - loss: 90.9228\n", + "Epoch 76/100\n", + "323/323 [==============================] - 0s 46us/sample - loss: 90.8954\n", + "Epoch 77/100\n", + "323/323 [==============================] - 0s 45us/sample - loss: 90.9290\n", + "Epoch 78/100\n", + "323/323 [==============================] - 0s 48us/sample - loss: 91.0799\n", + "Epoch 79/100\n", + "323/323 [==============================] - 0s 45us/sample - loss: 90.8961\n", + "Epoch 80/100\n", + "323/323 [==============================] - 0s 46us/sample - loss: 91.0972\n", + "Epoch 81/100\n", + "323/323 [==============================] - 0s 53us/sample - loss: 90.9088\n", + "Epoch 82/100\n", + "323/323 [==============================] - 0s 47us/sample - loss: 90.9462\n", + "Epoch 83/100\n", + "323/323 [==============================] - 0s 46us/sample - loss: 90.9312\n", + "Epoch 84/100\n", + "323/323 [==============================] - 0s 44us/sample - loss: 90.9983\n", + "Epoch 85/100\n", + "323/323 [==============================] - 0s 43us/sample - loss: 91.0539\n", + "Epoch 86/100\n", + "323/323 [==============================] - 0s 43us/sample - loss: 90.9206\n", + "Epoch 87/100\n", + "323/323 [==============================] - 0s 42us/sample - loss: 91.0098\n", + "Epoch 88/100\n", + "323/323 [==============================] - 0s 44us/sample - loss: 90.9973\n", + "Epoch 89/100\n", + "323/323 [==============================] - 0s 48us/sample - loss: 90.9351\n", + "Epoch 90/100\n", + "323/323 [==============================] - 0s 44us/sample - loss: 90.8768\n", + "Epoch 91/100\n", + "323/323 [==============================] - 0s 45us/sample - loss: 91.0795\n", + "Epoch 92/100\n", + "323/323 [==============================] - 0s 45us/sample - loss: 90.9424\n", + "Epoch 93/100\n", + "323/323 [==============================] - 0s 49us/sample - loss: 91.0652\n", + "Epoch 94/100\n", + "323/323 [==============================] - 0s 45us/sample - loss: 90.9541\n", + "Epoch 95/100\n", + "323/323 [==============================] - 0s 49us/sample - loss: 90.8856\n", + "Epoch 96/100\n", + "323/323 [==============================] - 0s 47us/sample - loss: 90.9977\n", + "Epoch 97/100\n", + "323/323 [==============================] - 0s 44us/sample - loss: 90.9173\n", + "Epoch 98/100\n", + "323/323 [==============================] - 0s 47us/sample - loss: 91.0772\n", + "Epoch 99/100\n", + "323/323 [==============================] - 0s 42us/sample - loss: 91.0006\n", + "Epoch 100/100\n", + "323/323 [==============================] - 0s 55us/sample - loss: 90.9921\n", + "81/81 [==============================] - 0s 593us/sample - loss: 71.1015\n" + ] }, { - "cell_type": "markdown", - "metadata": { - "id": "K6cx5vbtqdeD", - "colab_type": "text" - }, - "source": [ - "### Exercise 3\n", - "\n", - "Find the best possible combination of *learning rate* and *epochs* while testing some combinations" + "data": { + "text/plain": [ + "71.10153419588819" ] - }, - { - "cell_type": "code", - "metadata": { - "id": "n8RUGIklqxKS", - "colab_type": "code", - "colab": {} - }, - "source": [ - "# play with learning rate and epochs\n", - "learning_rate = ? # Replace ? with a floating-point number\n", - "epochs = ? # Replace ? with an integer\n", - "optimizer = RMSprop(learning_rate)\n", - "model.compile(loss='mean_squared_error',optimizer=optimizer) # Compile the model\n", - "model.fit(X_train, y_train, epochs=epochs, batch_size=30) # Fit the model\n", - "model.evaluate(X_test, y_test) # Evaluate the model" - ], - "execution_count": null, - "outputs": [] - }, + }, + "execution_count": 30, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# Play with epochs\n", + "learning_rate = 0.1 \n", + "epochs = 100 # Replace ? with an integer\n", + "optimizer = RMSprop(learning_rate)\n", + "model.compile(loss='mean_squared_error',optimizer=optimizer) # Compile the model\n", + "history = model.fit(X_train, y_train, epochs=epochs, batch_size=30) # Fit the model\n", + "model.evaluate(X_test, y_test) # Evaluate the model" + ] + }, + { + "cell_type": "code", + "execution_count": 31, + "metadata": {}, + "outputs": [ { - "cell_type": "markdown", - "metadata": { - "id": "kcn3gB3Uq7u6", - "colab_type": "text" - }, - "source": [ - "**Batch Size**\n", - "\n", - "The number of examples in a batch." + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEGCAYAAACKB4k+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAgAElEQVR4nO3deZxcdZ3v/9e7esna2ZtsnaRDEsKaBGxZFRFwAdTob0Rlkcg4F72DwvzwOqDj/c3IHebKz1EUnWEbEHAYcEONgAuy6GUngRCWBBNCVrKHhJCQpJfP/eOc6hRNp7uyVFdX1fv5eNSjzvnWOVWfU6f6fPqc7/d8v4oIzMzMADLFDsDMzHoPJwUzM2vnpGBmZu2cFMzMrJ2TgpmZtasudgD7Y8SIEdHY2FjsMMzMSsrcuXM3RER9Z6+VdFJobGxkzpw5xQ7DzKykSFq2p9d8+cjMzNo5KZiZWbuCJgVJSyU9L2mepDlp2TBJ90talD4PTcsl6VpJiyXNl3RMIWMzM7N36ok6hfdHxIac+SuAByLiW5KuSOcvB84ApqSP44Dr0mczswOuubmZlStXsmPHjmKHUjB9+/aloaGBmpqavNcpRkXzTOCUdPo24GGSpDATuD2SzpiekDRE0uiIWF2EGM2szK1cuZK6ujoaGxuRVOxwDriIYOPGjaxcuZKJEyfmvV6h6xQC+IOkuZIuSstG5hzo1wAj0+mxwIqcdVemZW8j6SJJcyTNWb9+faHiNrMyt2PHDoYPH16WCQFAEsOHD9/rM6FCnym8JyJWSToIuF/SwtwXIyIk7VU3rRFxI3AjQFNTk7t4NbN9Vq4JIWtftq+gZwoRsSp9Xgf8EjgWWCtpNED6vC5dfBUwLmf1hrTsgNu0bRff/M2L7GhuLcTbm5mVrIIlBUkDJNVlp4EPAi8As4FZ6WKzgF+n07OBC9JWSMcDWwpVn/Do4g3c+thSPn3jE6zbWr6VTGbWuw0cOLDYIbxDIc8URgKPSHoOeAq4NyJ+B3wL+ICkRcDp6TzAfcASYDFwE/C3hQrso9PHcMP57+Iva7byiX97jIVr3ijUR5mZlZSCJYWIWBIR09PHERFxVVq+MSJOi4gpEXF6RGxKyyMiLo6ISRFxVEQUtP+KDx4xip998QRa2tr45HWP89rmtwr5cWZmeVm6dCmnnnoq06ZN47TTTmP58uUA/OxnP+PII49k+vTpnHzyyQC8+OKLHHvsscyYMYNp06axaNGi/f78ku77aH8dOXYwN3y2iY//26PMXfY6Y4b0K3ZIZlYE3/zNi7z02oG9YnD4mEH840eP2Ov1vvzlLzNr1ixmzZrFLbfcwiWXXMKvfvUrrrzySn7/+98zduxYNm/eDMD111/PpZdeynnnnceuXbtobd3/etKK7+Zi6sg6AJZu2FbkSMzM4PHHH+fcc88F4LOf/SyPPPIIACeddBKf+9znuOmmm9oP/ieccAL/8i//wtVXX82yZcvo12///7Gt6DMFgH61VYwa1JelG7cXOxQzK5J9+Y++p11//fU8+eST3HvvvbzrXe9i7ty5nHvuuRx33HHce++9nHnmmdxwww2ceuqp+/U5FX+mADBheH+WbvSZgpkV34knnshdd90FwB133MF73/teAF555RWOO+44rrzySurr61mxYgVLlizh4IMP5pJLLmHmzJnMnz9/vz+/4s8UACaOGMD9L60tdhhmVmG2b99OQ0ND+/xll13GD37wAy688EK+/e1vU19fz49+9CMAvvrVr7Jo0SIigtNOO43p06dz9dVX8+Mf/5iamhpGjRrF17/+9f2OyUkBmDB8ABu37eKNHc0M6pt/x1FmZvujra2t0/IHH3zwHWV33333O8quuOIKrrjiigMaky8fARNH9Adg2QbXK5hZZXNSABpHDABwvYKZVTwnBWDCsDQpuFmqWUVJeuovX/uyfU4K7G6W+qrPFMwqRt++fdm4cWPZJobseAp9+/bdq/Vc0ZyaMLw/y3yvglnFaGhoYOXKlZTzuCzZkdf2hpNCys1SzSpLTU3NXo1IVil8+SjVOGJ3s1Qzs0rlpJBqHO5mqWZmTgqpbLNUVzabWSVzUkhlm6Uuc7NUM6tgBU8KkqokPSvpnnT+VkmvSpqXPmak5ZJ0raTFkuZLOqbQseVys1Qzs55pfXQpsAAYlFP21Yj4eYflzgCmpI/jgOvS5x7TOMLNUs2sshX0TEFSA3AW8B95LD4TuD0dlvMJYIik0YWMr6PG4QN8V7OZVbRCXz76HvD3QMeuAK9KLxFdI6lPWjYWWJGzzMq07G0kXSRpjqQ5B/qmEzdLNbNKV7CkIOkjwLqImNvhpa8BhwLvBoYBl+/N+0bEjRHRFBFN9fX1BybYlJulmlmlK+SZwknAxyQtBe4CTpX0nxGxOr1EtBP4EXBsuvwqYFzO+g1pWY+ZOGIgAEs2vNmTH2tm1msULClExNcioiEiGoHPAA9GxPnZegJJAj4OvJCuMhu4IG2FdDywJSJWFyq+zjSO6E9G8Mo6JwUzq0zF6PvoDkn1gIB5wBfT8vuAM4HFwHbgwp4OrE91FeOH9WfxeicFM6tMPZIUIuJh4OF0+tQ9LBPAxT0RT1cmHzSQxT5TMLMK5TuaO5h00EBe3bCNltbOx041MytnTgodTK4fSHNrsHyTWyCZWeVxUuhg8kFJCyRfQjKzSuSk0MGkbFJwZbOZVSAnhQ4G9a1h5KA+PlMws4rkpNCJyQcN9L0KZlaRnBQ6Mbl+IK+s30bSStbMrHI4KXRi8kEDeXNnC2ve2FHsUMzMepSTQicmuQWSmVUoJ4VOuFmqmVWqLpNCOpTmHT0VTG9RP7APg/pWOymYWcXpMilERCswQVJtD8XTK0hyH0hmVpHy6RBvCfCopNlA+1iVEfHdgkXVC0w+aCAPLlxX7DDMzHpUPnUKrwD3pMvW5TzK2uSDBrLhzV28vm1XsUMxM+sx3Z4pRMQ3ASQNTOcr4prK5JzuLt49YFiRozEz6xndnilIOlLSs8CLwIuS5ko6It8PSCurn5V0Tzo/UdKTkhZL+km2vkJSn3R+cfp6475t0oEx5aDkZGjR2orIgWZmQH6Xj24ELouICRExAfgKcNNefMalwIKc+auBayJiMvA68Pm0/PPA62n5NelyRTN2SD/611bxl7VbixmGmVmPyicpDIiIh7Iz6ShqA/J5c0kNwFnAf6TzAk4Ffp4uchvJOM0AM9N50tdPS5cvikxGTBlZ56RgZhUln6SwRNL/lNSYPr5B0iIpH98D/h7IDmM2HNgcES3p/EpgbDo9FlgBkL6+JV3+bSRdJGmOpDnr16/PM4x9c+jIOl5e46RgZpUjn6Tw10A9cDfwC2BEWtYlSR8B1kXE3P2KsIOIuDEimiKiqb6+/kC+9TscMqqOjdt2seHNnQX9HDOz3qLL1keSqoB/iIhL9uG9TwI+JulMoC8wCPg+MERSdXo20ACsSpdfBYwDVkqqBgYDG/fhcw+YqSOTyuaX12xlxOQ+xQzFzKxH5HNH83v25Y0j4msR0RARjcBngAcj4jzgIeCT6WKzgF+n07PTedLXH4wi9109ddTupGBmVgnyuaP52fRu5p/x9jua797Hz7wcuEvSPwPPAjen5TcDP5a0GNhEkkiKasTAWoYNqHVSMLOKkU9S6EtyGefUnLIgqWPIS9pi6eF0eglwbCfL7ADOzvc9e4Ikpo6s42W3QDKzCpFPncL8iLimh+LpdaaOquOnc1bQ1hZkMkVrIWtm1iPyqVM4p4di6ZWmjqpj+65WVm1+q9ihmJkVXD6Xjx6V9EPgJ7y9TuGZgkXVixyStkBauGYr44b1L3I0ZmaFlU9SmJE+X5lTFry9jqFsHTIy6RjvL2u38oHDRxY5GjOzwsqnl9T390QgvVVd3xrGDunHQrdAMrMKsMc6BUnfy5m+tMNrtxYwpl7n0FF1/MVJwcwqQFcVzSfnTM/q8Nq0AsTSax0yqo5X1r/Jrpa27hc2MythXSUF7WG64hw6qo6WtuDVDdu6X9jMrIR1lRQykoZKGp4zPUzSMKCqh+LrFbItkHwTm5mVu64qmgcDc9l9lpDbBLWofRL1tEn1A6nOiIWr3+Bj08cUOxwzs4LZY1JIO7IzoLY6w+SDBrJg9RvFDsXMrKDyGU/BgMNGD2LBal8+MrPy5qSQp8NG17HmjR1s2rar2KGYmRWMk0KeDhs9CMCXkMysrO2xTiFtZbRHEbHpwIfTe+UmhZMmjyhyNGZmhdFV66O5JK2MBIwHXk+nhwDLgYkFj64XGTGwD/V1fXjJZwpmVsb2ePkoIiZGxMHAH4GPRsSIiBgOfAT4Q3dvLKmvpKckPSfpRUnfTMtvlfSqpHnpY0ZaLknXSlosab6kYw7MJh44h7uy2czKXD51CsdHxH3ZmYj4LXBiHuvtBE6NiOkkPa1+WNLx6WtfjYgZ6WNeWnYGMCV9XARcl+9G9JTDRg9i8bqt7u7CzMpWPknhNUnfkNSYPv4BeK27lSLxZjpbkz66uultJnB7ut4TwBBJo/OIr8ccNrqO5tbglfVvdr+wmVkJyicpnAPUA78kGZe5njxHY5NUJWkesA64PyKeTF+6Kr1EdI2kPmnZWGBFzuor07KO73mRpDmS5qxfvz6fMA6Yw90CyczKXLdJISI2RcSlwHsi4piI+Lt8Wx5FRGtEzAAagGMlHQl8DTgUeDcwDLh8bwKOiBsjoikimurr6/dm1f02ccQAaqszTgpmVra6TQqSTpT0ErAgnZ8u6d/35kMiYjPwEPDhiFidXiLaCfwIODZdbBUwLme1hrSs16iuyjB1ZJ0rm82sbOVz+ega4EPARoCIeI63j7XQKUn1koak0/2ADwALs/UEkgR8HHghXWU2cEHaCul4YEtErN7L7Sm4w0bXsWD1G0RUVJ+AZlYh8hmjmYhYkRzD27Xmsdpo4DZJVSTJ56cRcY+kByXVk9zzMA/4Yrr8fcCZwGJgO3BhfpvQsw4bPYifzlnJ+q07OWhQ32KHY2Z2QOWTFFZIOhEISTXApaSXkroSEfOBozspP3UPywdwcR7xFFX2zuYXV7/hpGBmZSefy0dfJDlYjyW5xj+DEjh4F4r7QDKzctblmUJ66eezEXFeD8XT6w3uV0PD0H68uMpJwczKT5dnChHRCpzbQ7GUjGkNg3l+1ZZih2FmdsDlc/noEUk/lPReScdkHwWPrBc7cuxglm/azpbtzcUOxczsgMqnonlG+nxlTlkAnVYYV4Kjxg4G4PlVW3jPFHejbWblo9ukEBHv74lASomTgpmVq7zuU5B0FnAE0N4GMyKu3PMa5W1I/1rGDevHC65XMLMyk083F9cDnwa+THLD2dnAhALH1esdNXYw81dtLnYYZmYHVD4VzSdGxAXA6xHxTeAE4JDChtX7HTV2CCs2vcXm7buKHYqZ2QGTT1J4K33eLmkM0EzShUVFy9YrvOD7FcysjOSTFO5JO7b7NvAMsBS4s5BBlYJsUvAlJDMrJ/m0Pvpf6eQvJN0D9I2Iiq9hHdy/hvHD+ruy2czKSrdJQdIFnZQREbcXJqTScdTYwTy30mcKZlY+8mmS+u6c6b7AaSSXkZwUGgZz7/OreX3bLoYOqC12OGZm+y2fy0dfzp1P6xfuKlhEJaS9svm1Lbx3Ss8ODWpmVgj5VDR3tA2Y2N1CkvpKekrSc5JelPTNtHyipCclLZb0E0m1aXmfdH5x+nrjPsTWo44ck1Y2r3S9gpmVh3xuXvuNpNnp4x7gZeCXebz3TuDUiJhO0n/Sh9NhNq8GromIycDrwOfT5T9Pci/EZJIhQK/e+83pWYP71zBheH/mu17BzMpEPnUK/5oz3QIsi4iV3a2UjqT2Zjpbkz6yHellu+O+Dfgn4DpgZjoN8HPgh5IUvXww5KPHDeGxVzYSEXQYstTMrOR0e6YQEX/KeTyaT0LIklQlaR6wDrgfeAXYHBEt6SIrSUZ0I31ekX5mC7AFGN7Je14kaY6kOevXr883lIKZMW4I67buZPWWHcUOxcxsv+Vz+WirpDc6eWyV1OXtvBHRGhEzgAbgWODQ/Q04Im6MiKaIaKqvL37l7tHjhwLw7HJfQjKz0pdPRfP3gCtI/pNvAC4HvhcRdRExKJ8PiYjNwEMk/SYNkZS9bNVAMu4z6fM4gPT1wcDGPLejaA4bPYja6gzzVrxe7FDMzPZbPknhYxHx7xGxNSLeiIjs9f8uSapPm68iqR/wAWABSXL4ZLrYLODX6fTsdJ709Qd7e30CQG11hqPGDvaZgpmVhXySwjZJ56X1AxlJ55E0S+3OaOAhSfOBp4H7I+IekjONyyQtJqkzuDld/mZgeFp+GcnZSUmYMW4Iz6/awq6WtmKHYma2X/JpfXQu8P30AfAIu1sP7VFEzAeO7qR8CUn9QsfyHSRjNZSco8cP4eZHXmXhmjeY1jCk2OGYme2zfO5oXkoel4sqWbayed6KzU4KZlbS9nj5SNJ/kzQlnZakWyRtkTRf0jE9F2LvN2ZwX+rr+rhewcxKXld1CpeSjJ0AcA4wHTiY5Hr/9/ewTkWSxNHjhvDscrdAMrPS1lVSaImI5nT6I8DtEbExIv4IDCh8aKXl6PFDWbpxO69v8/CcZla6ukoKbZJGS8p2l/3HnNf6FTas0jNjXFKXMG+FLyGZWenqKin8f8AckktIsyPiRQBJ7wOWFD600jKtYTAZ4UtIZlbS9tj6KCLukTQBqIuI3CPdHODTBY+sxAzoU83UUYN4xpXNZlbCurx5LSJaOiQEImJbRLy5p3UqWdOEoTy7/HVaWn0Tm5mVpn0ZZMf2oKlxKNt2tbJwzdZih2Jmtk+cFA6gpsZhAMxZuqnIkZiZ7Zu8koKksZJOlHRy9lHowErR2CH9GDO4L08vc2WzmZWmbru5kHQ1ScXyS0BrWhzAnwsYV8lqahzGk696JDYzK035dIj3cWBqROwsdDDloKlxKLOfe42Vr7/FuGH9ix2Omdleyefy0RKS8ZUtD00T0nqFZa5XMLPSk8+ZwnZgnqQHgPazhYi4pGBRlbCpo+qo61PNnKWv84mjG4odjpnZXsknKcxOH5aHqow4esJQ5ix1ZbOZlZ58xlO4bV/eWNI44HZgJEnF9I0R8X1J/wT8N2B9uujXI+K+dJ2vAZ8nqdC+JCJ+vy+fXWzvnjCU79z/F7Zsb2Zwf195M7PSkU/roynA/wYOB/pmyyPi4G5WbQG+EhHPSKoD5kq6P33tmoj41w6fczjwGeAIYAzwR0mHREQrJSZ7v8Izy1/n/YceVORozMzyl09F84+A60gO8u8n+e//P7tbKSJWR8Qz6fRWYAEwtotVZgJ3RcTOiHgVWEwnw3aWghnjhlCdEU/7JjYzKzH5JIV+EfEAoIhYFhH/BJy1Nx8iqZFkvOYn06IvpSO43SJpaFo2FliRs9pKOkkiki6SNEfSnPXr13d8uVfoV1vFEWMHOymYWcnJJynslJQBFkn6kqRPAAPz/QBJA4FfAH8XEW+QnHVMAmYAq4Hv7E3AEXFjRDRFRFN9ff3erNqjjj94GPNWbGb7rpZih2Jmlrd8ksKlQH/gEuBdwPnArHzeXFINSUK4IyLuBoiItRHRGhFtwE3svkS0ChiXs3pDWlaSTpw0gubWcCskMysp3SaFiHg67Sp7U0RcGBF/FRFPdLeekj4ebgYWRMR3c8pH5yz2CeCFdHo28BlJfSRNBKYAT+3FtvQq724cSnVGPL5kY7FDMTPLWz6tj04gObgPBMZLmg58ISL+tptVTwI+CzwvaV5a9nXgHEkzSJqpLgW+ABARL0r6KUkfSy3AxaXY8iirf201R48fwmOvOCmYWenI5+a17wEfIr2BLSKey6eX1Ih4BOisR7j7uljnKuCqPGIqCSdMGsEPH1zEGzuaGdTX9yuYWe+XV9fZEbGiQ1HJ/gffk06cNJy2gKeWuBWSmZWGfJLCCkknAiGpRtL/ILnnwLpx9Pgh9KnO+BKSmZWMfJLCF4GLSe4ZWEXSlPTiQgZVLvpUV9HUOJTHXtlQ7FDMzPKST+ujDRFxXkSMjIiDIuL8iPC/vnk6cdIIFq7ZysY3PRyFmfV+e6xolnRtVyu66+z8nDBpOABPLNnEWdNGd7O0mVlxddX66Isk9xD8FHiNzlsSWTemjR3MwD7VPPbKBicFM+v1ukoKo4GzScZnbgF+Avw8Ijb3RGDloroqw7ETh/HoYtcrmFnvt8c6hYjYGBHXR8T7gQuBIcBLkj7bY9GViZOnjGDpxu0s3bCt2KGYmXWp24pmSceQ9H90PvBbYG6hgyo3p0xNxlT40196Z6+uZmZZe0wKkq6UNBe4DPgT0BQRn4+Il3osujLROGIAjcP78/DL64odiplZl7o6U/gGySWj6SQjrz2TjoHwvKT5PRJdGTll6kE8vmQjO5p9M7iZ9V5dVTRP7LEoKsD7Dqnn1seW8uSrm3jfIb13HAgzq2x7TAoRsawnAyl3xx88nNrqDA+/vM5Jwcx6rbw6xLP916+2iuMPHs6fXnZls5n1Xk4KPeiUQ+pZsmEbyzduL3YoZmadyqdJ6qX5lHWyzDhJD0l6SdKL2XUkDZN0v6RF6fPQtFySrpW0OK3QPmZfNqg3O2Vqctno4b+4FZKZ9U75nCl0Nh7z5/JYrwX4SkQcDhwPXCzpcOAK4IGImAI8kM4DnEEyBOcU4CLgujw+o6RMHDGA8cP687AvIZlZL9VVh3jnAOcCEyXNznmpDuh21JiIWA2sTqe3SlpA0v32TOCUdLHbgIeBy9Py2yMigCckDZE0On2fsiCJ90+t5ydzVvDWrlb61VYVOyQzs7fpqknqYyQH9RHAd3LKtwJ7dZ+CpEbgaOBJYGTOgX4NMDKdHgvkjvC2Mi17W1KQdBHJmQTjx4/fmzB6hdMOG8ltjy/j0cUbOP3wkd2vYGbWg7rq+2hZRDwcEScAC0nOEOqAlRHRku8HSBoI/AL4u4h4o8NnBBB7E3BE3BgRTRHRVF9fek07jz94OHV9qrn/pbXFDsXM7B3yqWg+G3iKpMfUTwFPSvpkPm8uqYYkIdwREXenxWsljU5fHw1ka11XAeNyVm9Iy8pKbXWG902t54GFa2lr26t8aGZWcPlUNH8DeHdEzIqIC4Bjgf/Z3UqSBNwMLIiI7+a8NJvdldezgF/nlF+QtkI6HthSTvUJuT5w+Eg2vLmLZ1e4F3Iz613ySQqZiMhtQ7kxz/VOAj4LnCppXvo4E/gW8AFJi4DT03mA+4AlwGLgJuBv89yGknPKIQdRnRF/XOBLSGbWu3RV0Zz1O0m/B+5M5z9NcgDvUkQ8wp5Hazutk+UDuDiPeEre4P41HDtxGPe/tJbLP3xoscMxM2vX7X/8EfFV4AZgWvq4MSIuL3Rg5e4Dh49k8bo3edUD75hZL5JvNxePAg8BD6bTtp9OPyxpjvpHt0Iys14kn9ZHnyJpffRJ9rL1ke3ZuGH9OXRUHfe7XsHMepF86hT+gaT10ToASfXAH4GfFzKwSvDBI0bxwwcXsW7rDg6q61vscMzMCtr6yLrx0WmjaQu4b35Ztrw1sxKUz8H9d5J+L+lzkj4H3Av8trBhVYYpI+s4dFQds597rdihmJkB+9766O8LHVil+NiMMTyzfDMrNnmMBTMrvrwuA0XE3RFxWURcBvxa0nkFjqtifHTaGAB+M99nC2ZWfHtMCpIGSfqapB9K+mDa/cSXSO46/lTPhVjexg3rzzHjhzB7npOCmRVfV2cKPwamAs8Df0Nyn8LZwMcjYmYPxFYxPjZ9DAvXbOUva7cWOxQzq3BdJYWDI+JzEXEDcA5wOPChiJjXM6FVjrOmjSEjfLZgZkXXVVJozk5ERCvJOAo7Ch9S5amv68OJk0Yw+7nXSLqAMjMrjq6SwnRJb6SPrcC07LSkN7pYz/bBzBljWL5pO08vfb3YoZhZBetq5LWqiBiUPuoiojpnelBPBlkJzpo2mro+1dz11PJih2JmFcx3JvcS/WurmXn0GO59fjVbtjd3v4KZWQE4KfQi5xw7np0tbfzy2ZXFDsXMKlTBkoKkWyStk/RCTtk/SVrVYSS27Gtfk7RY0suSPlSouHqzI8YMZlrDYO58aoUrnM2sKAp5pnAr8OFOyq+JiBnp4z4ASYcDnwGOSNf5d0lVBYyt1zrn2PG8vHarx282s6IoWFKIiD8Dm/JcfCZwV0TsjIhXScZpPrZQsfVmH50+hv61Vdz5pCuczaznFaNO4UuS5qeXl4amZWOBFTnLrEzL3kHSRZLmSJqzfv36Qsfa4wb2qWbmjDHcM381W95yhbOZ9ayeTgrXAZOAGcBq4Dt7+wYRcWNENEVEU319/YGOr1c4//gJvNXcyn/5bMHMeliPJoWIWBsRrRHRBtzE7ktEq4BxOYs2pGUV6Ygxg3nvlBHc8uir7GxpLXY4ZlZBejQpSBqdM/sJINsyaTbwGUl9JE0EppCMC12xvnDyJNZv3ckvn6nY3GhmRZDPGM37RNKdwCnACEkrgX8ETpE0AwhgKfAFgIh4UdJPgZeAFuDitL+linXS5OEcOXYQN/55CZ9qGkcmo2KHZGYVoGBJISLO6aT45i6Wvwq4qlDxlBpJfOHkSXz5zme5f8FaPnTEqGKHZGYVwHc092JnHDmKccP6cf2fXvHNbGbWI5wUerHqqgwXvfdgnl2+mcde2VjscMysAjgp9HJnN41j7JB+fOu3C2lr89mCmRWWk0Iv17emiq988BCeX7WF38z3yGxmVlhOCiXg4zPGctjoQfzrH172fQtmVlBOCiUgkxFXnHEoKza9xR1P+C5nMyscJ4UScfKUEbxn8gh+8OAi94lkZgXjpFAipORsYctbzVz9u4XFDsfMypSTQgk5cuxg/vqkifzXk8t53E1UzawAnBRKzFc+OJUJw/tzxd3zeWuXK53N7MByUigx/Wqr+Nb/M41lG7fznT+8XOxwzKzMOCmUoBMmDefc48Zzy6Ov8tSr+Q5uZ2bWPSeFEvW1Mw5lwvABfPE/57J84/Zih2NmZcJJoUTV9a3h5llNtLYFf33b026mamYHhJNCCTu4fiDXnX8MSzds40v/9QzNrW3FDsnMSlzBkoKkWyStk/RCTtkwSfdLWl5UIYMAAArsSURBVJQ+D03LJelaSYslzZd0TKHiKjcnThrBv3ziKP7Pog1ccuez7gbDzPZLIc8UbgU+3KHsCuCBiJgCPJDOA5xBMgTnFOAi4LoCxlV2PvXucXzjrMP47Qtr+Otbn+bNnS3FDsnMSlTBkkJE/Bno2DRmJnBbOn0b8PGc8tsj8QQwpMN4ztaNv3nvwXzn7Ok8sWQT5930BBve3FnskMysBPV0ncLIiFidTq8BRqbTY4EVOcutTMveQdJFkuZImrN+/frCRVqC/updDdxw/rtYuGYrp3/3T/zk6eUeg8HM9krRKpojGV9yr49YEXFjRDRFRFN9fX0BIittpx8+kt98+T0cclAdl//iec6+4XGeXrrJw3maWV56OimszV4WSp/XpeWrgHE5yzWkZbYPDhlZx0++cDzf/uQ0lqx/k7Ovf5zTv/snbvrzEtZs2VHs8MysF6vu4c+bDcwCvpU+/zqn/EuS7gKOA7bkXGayfSCJs5vGceZRo7n3+dXc9dRyrrpvAVfdt4BDRg7kPZPrmTF+CGOH9KNhaD9GDOxDVUbFDtvMikyFuqwg6U7gFGAEsBb4R+BXwE+B8cAy4FMRsUmSgB+StFbaDlwYEXO6+4ympqaYM6fbxSy1aO1WHly4jkcWb+CpVzexs+Xt9zXUVIm+1VXUVGfITQ9SkmREftf7MoKMREaiLSJ9QJVEVUZUV4kIaG1LXmtpC1pa22hpDSSora6itkpkMslyHSl9fwkioLm1jebWNlraAvH21zquB6I6jaE6IwKS+Nogk0lizK6frCN2trTy1q7W9g4Iq6sy1FRlqMqA2P1Z2e1si2jfNgF9aqroW5OhSmJXSxu7WttobQtqqjLUVmeorcq0f68RQXNr0NzaRltE8n1lMtRUJQG15XxvrW3JI9L1gPblq6vUnuQFNLdG+3ZA0odWn+oqaqsz7fsoYvd3lrsNud9j9ruvzuzeP9nlkn2efGfZ/dralmxDbVWyDZKIiHf8jsTbf2PZZVrbku9iV0sbAdRWZeiTfpc7W9rY2dJGS2sbtdUZ+lRXUVO1e5/ufu/ke8jGmWxb8hnJd5zEVlOVIQhaW5PY3/6bTrYtk0m+19zf2M7mVna07N6nNVXJ6y1tyW86+xnV6brZh6D9O2ppayOj3e/d0trGrvR3IKW/y0wSQ9YFJzRy8fsnv/MPJA+S5kZEU2evFexMISLO2cNLp3WybAAXFyoWS0wZWceUkXV84X2T2NHcyrKN21m1eTurXn+LTdua2dHSyo7m1rfdBBeR/SNN/oiyB9a3i5yyaD9AtrYlCaIqk/wBtbUlfwQtbW3JwTsjqpQ9QGfak8Wu9CCQ/cNWh8/LHjCyr9dWZaipTv7okniTg3NyoNm9HZA9qCZ/rC1t0Z7AyDmwZw+0pO/Vp7qKvjVV9KupQsomoaCtLYh0ezMSmUxyYMvk/BG3tQW7WtvY2ZwkrdrqJBFUSe0Hu12tbe0HRKD9wFKVEa3p+tmEmT0YZb+3bPLNao3dCbYton3fVVeJfjXJdgDsaG7lreZWmluDqpykj5LvO7vfOh6I2gLa0gNZa04iyEhvS4xVGajKJImztS3Y1ZIc4LK/lPQrb//17P6dRXss2e2srU6SsETyfbUk32XfmiQRVGXU/pvZ1dLWnphy/4mJiPbfWybdN9mk3hpBc0vyj0X2wJz9zbb/DUTy3ba1xduSvyT6Vu9OVM1tyXu1RlCT/qYz0jv2S3b95B+UZN22iPbPqE7/YahJ/3HpLEEfPGJAxz/xA6KnLx9ZL9G3poqpo+qYOqqu2KGYWS/ibi7MzKydk4KZmbVzUjAzs3ZOCmZm1s5JwczM2jkpmJlZOycFMzNr56RgZmbtCtbNRU+QtJ6ku4x9MQLYcADDKRWVuN2VuM1QmdtdidsMe7/dEyKi026mSzop7A9Jc/bU90c5q8TtrsRthsrc7krcZjiw2+3LR2Zm1s5JwczM2lVyUrix2AEUSSVudyVuM1TmdlfiNsMB3O6KrVMwM7N3quQzBTMz68BJwczM2lVkUpD0YUkvS1os6Ypix1MIksZJekjSS5JelHRpWj5M0v2SFqXPQ4sdayFIqpL0rKR70vmJkp5M9/lPJNUWO8YDSdIQST+XtFDSAkknVMK+lvT/pr/vFyTdKalvOe5rSbdIWifphZyyTvevEtem2z9f0jF781kVlxQkVQH/BpwBHA6cI+nw4kZVEC3AVyLicOB44OJ0O68AHoiIKcAD6Xw5uhRYkDN/NXBNREwGXgc+X5SoCuf7wO8i4lBgOsm2l/W+ljQWuARoiogjgSrgM5Tnvr6VZAz7XHvav2cAU9LHRcB1e/NBFZcUgGOBxRGxJCJ2AXcBM4sc0wEXEasj4pl0eivJQWIsybbeli52G/Dx4kRYOJIagLOA/0jnBZwK/DxdpKy2W9Jg4GTgZoCI2BURm6mAfU0ypHA/SdVAf2A1ZbivI+LPwKYOxXvavzOB2yPxBDBE0uh8P6sSk8JYYEXO/Mq0rGxJagSOBp4ERkbE6vSlNcDIIoVVSN8D/h5oS+eHA5sjoiWdL7d9PhFYD/wovWT2H5IGUOb7OiJWAf8KLCdJBluAuZT3vs61p/27X8e4SkwKFUXSQOAXwN9FxBu5r0XSHrms2iRL+giwLiLmFjuWHlQNHANcFxFHA9vocKmoTPf1UJL/iicCY4ABvPMSS0U4kPu3EpPCKmBcznxDWlZ2JNWQJIQ7IuLutHht9lQyfV5XrPgK5CTgY5KWklwaPJXkevuQ9BIDlN8+XwmsjIgn0/mfkySJct/XpwOvRsT6iGgG7ibZ/+W8r3Ptaf/u1zGuEpPC08CUtIVCLUnF1Owix3TApdfRbwYWRMR3c16aDcxKp2cBv+7p2AopIr4WEQ0R0Uiybx+MiPOAh4BPpouV1XZHxBpghaSpadFpwEuU+b4muWx0vKT+6e89u91lu6872NP+nQ1ckLZCOh7YknOZqVsVeUezpDNJrjtXAbdExFVFDumAk/Qe4P8Az7P72vrXSeoVfgqMJ+l2/FMR0bECqyxIOgX4HxHxEUkHk5w5DAOeBc6PiJ3FjO9AkjSDpGK9FlgCXEjyT19Z72tJ3wQ+TdLa7lngb0iun5fVvpZ0J3AKSRfZa4F/BH5FJ/s3TZA/JLmUth24MCLm5P1ZlZgUzMysc5V4+cjMzPbAScHMzNo5KZiZWTsnBTMza+ekYGZm7ZwUzLogqVXSvJzHAetUTlJjbq+XZr1BdfeLmFW0tyJiRrGDMOspPlMw2weSlkr6/yU9L+kpSZPT8kZJD6b92D8gaXxaPlLSLyU9lz5OTN+qStJN6ZgAf5DUr2gbZYaTgll3+nW4fPTpnNe2RMRRJHePfi8t+wFwW0RMA+4Ark3LrwX+FBHTSfolejEtnwL8W0QcAWwG/qrA22PWJd/RbNYFSW9GxMBOypcCp0bEkrTjwTURMVzSBmB0RDSn5asjYoSk9UBDbncLaZfm96eDpCDpcqAmIv658Ftm1jmfKZjtu9jD9N7I7ZOnFdfzWZE5KZjtu0/nPD+eTj9G0jsrwHkknRJCMlzif4f28aMH91SQZnvD/5WYda2fpHk587+LiGyz1KGS5pP8t39OWvZlkhHQvkoyGtqFafmlwI2SPk9yRvDfSUYLM+tVXKdgtg/SOoWmiNhQ7FjMDiRfPjIzs3Y+UzAzs3Y+UzAzs3ZOCmZm1s5JwczM2jkpmJlZOycFMzNr938B4vVYG+mKSuIAAAAASUVORK5CYII=\n", + "text/plain": [ + "
" ] - }, + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "plot_the_loss_curve(history.epoch, history.history['loss'])" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "K6cx5vbtqdeD" + }, + "source": [ + "### Exercise 3\n", + "\n", + "Find the best possible combination of *learning rate* and *epochs* while testing some combinations" + ] + }, + { + "cell_type": "code", + "execution_count": 32, + "metadata": { + "colab": {}, + "colab_type": "code", + "collapsed": true, + "id": "n8RUGIklqxKS", + "jupyter": { + "outputs_hidden": true + } + }, + "outputs": [ { - "cell_type": "code", - "metadata": { - "id": "DTSP63PTZ8hv", - "colab_type": "code", - "colab": { - "base_uri": "https://localhost:8080/", - "height": 428 - }, - "outputId": "d0a90bd0-9e34-49e4-b351-fa035ae6df22" - }, - "source": [ - "####################### Complete example to check the performance of the model with different batch size while keeping epochs as 30 and learning rate as 0.01 #######################################\n", - "# define the model\n", - "model = Sequential()\n", - "model.add(Dense(10, activation='relu', input_shape=(n_features,)))\n", - "model.add(Dense(8, activation='relu'))\n", - "model.add(Dense(1))\n", - "\n", - "optimizer = RMSprop(0.1) # 0.1 is the learning rate\n", - "model.compile(loss='mean_squared_error',optimizer=optimizer) # Compile the model\n", - "\n", - "# fit the model \n", - "model.fit(X_train, y_train, epochs=10, batch_size=40, verbose = 1)\n", - "\n", - "# evaluate the model\n", - "print('The MSE value is: ', model.evaluate(X_test, y_test))" - ], - "execution_count": null, - "outputs": [ - { - "output_type": "stream", - "text": [ - "Epoch 1/10\n", - "9/9 [==============================] - 0s 1ms/step - loss: 27410.0840\n", - "Epoch 2/10\n", - "9/9 [==============================] - 0s 1ms/step - loss: 211.4277\n", - "Epoch 3/10\n", - "9/9 [==============================] - 0s 1ms/step - loss: 187.7308\n", - "Epoch 4/10\n", - "9/9 [==============================] - 0s 1ms/step - loss: 157.6277\n", - "Epoch 5/10\n", - "9/9 [==============================] - 0s 1ms/step - loss: 133.5317\n", - "Epoch 6/10\n", - "9/9 [==============================] - 0s 1ms/step - loss: 206.0187\n", - "Epoch 7/10\n", - "9/9 [==============================] - 0s 1ms/step - loss: 335.1154\n", - "Epoch 8/10\n", - "9/9 [==============================] - 0s 1ms/step - loss: 166.5187\n", - "Epoch 9/10\n", - "9/9 [==============================] - 0s 1ms/step - loss: 194.7234\n", - "Epoch 10/10\n", - "9/9 [==============================] - 0s 1ms/step - loss: 181.1730\n", - "WARNING:tensorflow:5 out of the last 11 calls to .test_function at 0x7f833ec5cbf8> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has experimental_relax_shapes=True option that relaxes argument shapes that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/tutorials/customization/performance#python_or_tensor_args and https://www.tensorflow.org/api_docs/python/tf/function for more details.\n", - "3/3 [==============================] - 0s 2ms/step - loss: 441.9634\n", - "The MSE value is: 441.9634094238281\n" - ], - "name": "stdout" - } - ] + "name": "stdout", + "output_type": "stream", + "text": [ + "Train on 323 samples\n", + "Epoch 1/200\n", + "323/323 [==============================] - 0s 1ms/sample - loss: 91.0048\n", + "Epoch 2/200\n", + "323/323 [==============================] - 0s 46us/sample - loss: 90.9726\n", + "Epoch 3/200\n", + "323/323 [==============================] - 0s 44us/sample - loss: 90.9788\n", + "Epoch 4/200\n", + "323/323 [==============================] - 0s 45us/sample - loss: 91.0713\n", + "Epoch 5/200\n", + "323/323 [==============================] - 0s 45us/sample - loss: 90.8662\n", + "Epoch 6/200\n", + "323/323 [==============================] - 0s 42us/sample - loss: 90.8548\n", + "Epoch 7/200\n", + "323/323 [==============================] - 0s 46us/sample - loss: 91.0140\n", + "Epoch 8/200\n", + "323/323 [==============================] - 0s 42us/sample - loss: 91.0098\n", + "Epoch 9/200\n", + "323/323 [==============================] - 0s 46us/sample - loss: 91.0072\n", + "Epoch 10/200\n", + "323/323 [==============================] - 0s 41us/sample - loss: 91.0298\n", + "Epoch 11/200\n", + "323/323 [==============================] - 0s 45us/sample - loss: 91.0009\n", + "Epoch 12/200\n", + "323/323 [==============================] - 0s 42us/sample - loss: 90.8546\n", + "Epoch 13/200\n", + "323/323 [==============================] - 0s 43us/sample - loss: 90.9740\n", + "Epoch 14/200\n", + "323/323 [==============================] - 0s 51us/sample - loss: 91.0095\n", + "Epoch 15/200\n", + "323/323 [==============================] - 0s 43us/sample - loss: 90.9693\n", + "Epoch 16/200\n", + "323/323 [==============================] - 0s 44us/sample - loss: 90.9685\n", + "Epoch 17/200\n", + "323/323 [==============================] - 0s 49us/sample - loss: 91.0063\n", + "Epoch 18/200\n", + "323/323 [==============================] - 0s 49us/sample - loss: 91.0482\n", + "Epoch 19/200\n", + "323/323 [==============================] - 0s 47us/sample - loss: 90.9558\n", + "Epoch 20/200\n", + "323/323 [==============================] - 0s 42us/sample - loss: 91.0560\n", + "Epoch 21/200\n", + "323/323 [==============================] - 0s 40us/sample - loss: 91.0178\n", + "Epoch 22/200\n", + "323/323 [==============================] - 0s 48us/sample - loss: 90.8701\n", + "Epoch 23/200\n", + "323/323 [==============================] - 0s 40us/sample - loss: 90.9232\n", + "Epoch 24/200\n", + "323/323 [==============================] - 0s 45us/sample - loss: 90.9840\n", + "Epoch 25/200\n", + "323/323 [==============================] - 0s 43us/sample - loss: 90.9959\n", + "Epoch 26/200\n", + "323/323 [==============================] - 0s 43us/sample - loss: 90.9757\n", + "Epoch 27/200\n", + "323/323 [==============================] - 0s 44us/sample - loss: 91.0143\n", + "Epoch 28/200\n", + "323/323 [==============================] - 0s 45us/sample - loss: 91.0118\n", + "Epoch 29/200\n", + "323/323 [==============================] - 0s 47us/sample - loss: 91.0158\n", + "Epoch 30/200\n", + "323/323 [==============================] - 0s 39us/sample - loss: 90.9618\n", + "Epoch 31/200\n", + "323/323 [==============================] - 0s 49us/sample - loss: 91.0186\n", + "Epoch 32/200\n", + "323/323 [==============================] - 0s 41us/sample - loss: 90.9156\n", + "Epoch 33/200\n", + "323/323 [==============================] - 0s 44us/sample - loss: 90.9819\n", + "Epoch 34/200\n", + "323/323 [==============================] - 0s 46us/sample - loss: 91.1016\n", + "Epoch 35/200\n", + "323/323 [==============================] - 0s 46us/sample - loss: 90.9449\n", + "Epoch 36/200\n", + "323/323 [==============================] - 0s 47us/sample - loss: 90.9583\n", + "Epoch 37/200\n", + "323/323 [==============================] - 0s 45us/sample - loss: 90.9506\n", + "Epoch 38/200\n", + "323/323 [==============================] - 0s 42us/sample - loss: 90.9083\n", + "Epoch 39/200\n", + "323/323 [==============================] - 0s 45us/sample - loss: 91.0654\n", + "Epoch 40/200\n", + "323/323 [==============================] - 0s 44us/sample - loss: 91.0270\n", + "Epoch 41/200\n", + "323/323 [==============================] - 0s 44us/sample - loss: 91.0374\n", + "Epoch 42/200\n", + "323/323 [==============================] - 0s 42us/sample - loss: 91.0251\n", + "Epoch 43/200\n", + "323/323 [==============================] - 0s 48us/sample - loss: 90.9016\n", + "Epoch 44/200\n", + "323/323 [==============================] - 0s 43us/sample - loss: 90.9567\n", + "Epoch 45/200\n", + "323/323 [==============================] - 0s 68us/sample - loss: 90.8898\n", + "Epoch 46/200\n", + "323/323 [==============================] - 0s 44us/sample - loss: 90.8758\n", + "Epoch 47/200\n", + "323/323 [==============================] - 0s 39us/sample - loss: 91.1318\n", + "Epoch 48/200\n", + "323/323 [==============================] - 0s 42us/sample - loss: 90.9979\n", + "Epoch 49/200\n", + "323/323 [==============================] - 0s 40us/sample - loss: 90.9767\n", + "Epoch 50/200\n", + "323/323 [==============================] - 0s 41us/sample - loss: 91.0335\n", + "Epoch 51/200\n", + "323/323 [==============================] - 0s 43us/sample - loss: 91.0930\n", + "Epoch 52/200\n", + "323/323 [==============================] - 0s 43us/sample - loss: 90.8934\n", + "Epoch 53/200\n", + "323/323 [==============================] - 0s 44us/sample - loss: 91.0619\n", + "Epoch 54/200\n", + "323/323 [==============================] - 0s 45us/sample - loss: 90.8928\n", + "Epoch 55/200\n", + "323/323 [==============================] - 0s 46us/sample - loss: 90.9104\n", + "Epoch 56/200\n", + "323/323 [==============================] - 0s 43us/sample - loss: 91.0430\n", + "Epoch 57/200\n", + "323/323 [==============================] - 0s 46us/sample - loss: 90.9069\n", + "Epoch 58/200\n", + "323/323 [==============================] - 0s 42us/sample - loss: 90.9546\n", + "Epoch 59/200\n", + "323/323 [==============================] - 0s 43us/sample - loss: 91.0392\n", + "Epoch 60/200\n", + "323/323 [==============================] - 0s 40us/sample - loss: 91.0566\n", + "Epoch 61/200\n", + "323/323 [==============================] - 0s 42us/sample - loss: 90.9273\n", + "Epoch 62/200\n", + "323/323 [==============================] - 0s 46us/sample - loss: 90.9120\n", + "Epoch 63/200\n", + "323/323 [==============================] - 0s 42us/sample - loss: 91.1432\n", + "Epoch 64/200\n", + "323/323 [==============================] - 0s 43us/sample - loss: 90.9020\n", + "Epoch 65/200\n", + "323/323 [==============================] - 0s 41us/sample - loss: 90.9426\n", + "Epoch 66/200\n", + "323/323 [==============================] - 0s 42us/sample - loss: 90.9679\n", + "Epoch 67/200\n", + "323/323 [==============================] - 0s 42us/sample - loss: 91.0428\n", + "Epoch 68/200\n", + "323/323 [==============================] - 0s 42us/sample - loss: 91.0305\n", + "Epoch 69/200\n", + "323/323 [==============================] - 0s 44us/sample - loss: 90.9463\n", + "Epoch 70/200\n", + "323/323 [==============================] - 0s 43us/sample - loss: 91.0044\n", + "Epoch 71/200\n", + "323/323 [==============================] - 0s 46us/sample - loss: 90.9508\n", + "Epoch 72/200\n", + "323/323 [==============================] - 0s 43us/sample - loss: 91.0512\n", + "Epoch 73/200\n", + "323/323 [==============================] - 0s 43us/sample - loss: 90.8971\n", + "Epoch 74/200\n", + "323/323 [==============================] - 0s 44us/sample - loss: 90.9268\n", + "Epoch 75/200\n", + "323/323 [==============================] - 0s 46us/sample - loss: 90.9228\n", + "Epoch 76/200\n", + "323/323 [==============================] - 0s 42us/sample - loss: 90.8954\n", + "Epoch 77/200\n", + "323/323 [==============================] - 0s 40us/sample - loss: 90.9290\n", + "Epoch 78/200\n", + "323/323 [==============================] - 0s 48us/sample - loss: 91.0799\n", + "Epoch 79/200\n", + "323/323 [==============================] - 0s 46us/sample - loss: 90.8961\n", + "Epoch 80/200\n", + "323/323 [==============================] - 0s 46us/sample - loss: 91.0972\n", + "Epoch 81/200\n", + "323/323 [==============================] - 0s 45us/sample - loss: 90.9088\n", + "Epoch 82/200\n", + "323/323 [==============================] - 0s 44us/sample - loss: 90.9462\n", + "Epoch 83/200\n", + "323/323 [==============================] - 0s 46us/sample - loss: 90.9312\n", + "Epoch 84/200\n", + "323/323 [==============================] - 0s 42us/sample - loss: 90.9983\n", + "Epoch 85/200\n", + "323/323 [==============================] - 0s 43us/sample - loss: 91.0539\n", + "Epoch 86/200\n", + "323/323 [==============================] - 0s 41us/sample - loss: 90.9206\n", + "Epoch 87/200\n", + "323/323 [==============================] - 0s 41us/sample - loss: 91.0098\n", + "Epoch 88/200\n", + "323/323 [==============================] - 0s 44us/sample - loss: 90.9973\n", + "Epoch 89/200\n", + "323/323 [==============================] - 0s 45us/sample - loss: 90.9351\n", + "Epoch 90/200\n", + "323/323 [==============================] - 0s 46us/sample - loss: 90.8768\n", + "Epoch 91/200\n", + "323/323 [==============================] - 0s 42us/sample - loss: 91.0795\n", + "Epoch 92/200\n", + "323/323 [==============================] - 0s 43us/sample - loss: 90.9424\n", + "Epoch 93/200\n", + "323/323 [==============================] - 0s 41us/sample - loss: 91.0652\n", + "Epoch 94/200\n", + "323/323 [==============================] - 0s 43us/sample - loss: 90.9541\n", + "Epoch 95/200\n", + "323/323 [==============================] - 0s 42us/sample - loss: 90.8856\n", + "Epoch 96/200\n", + "323/323 [==============================] - 0s 47us/sample - loss: 90.9977\n", + "Epoch 97/200\n", + "323/323 [==============================] - 0s 43us/sample - loss: 90.9173\n", + "Epoch 98/200\n", + "323/323 [==============================] - 0s 43us/sample - loss: 91.0772\n", + "Epoch 99/200\n", + "323/323 [==============================] - 0s 41us/sample - loss: 91.0006\n", + "Epoch 100/200\n", + "323/323 [==============================] - 0s 43us/sample - loss: 90.9921\n", + "Epoch 101/200\n", + "323/323 [==============================] - 0s 40us/sample - loss: 90.8962\n", + "Epoch 102/200\n", + "323/323 [==============================] - 0s 43us/sample - loss: 91.0031\n", + "Epoch 103/200\n", + "323/323 [==============================] - 0s 42us/sample - loss: 91.0353\n", + "Epoch 104/200\n", + "323/323 [==============================] - 0s 40us/sample - loss: 90.9812\n", + "Epoch 105/200\n", + "323/323 [==============================] - 0s 44us/sample - loss: 90.8917\n", + "Epoch 106/200\n", + "323/323 [==============================] - 0s 41us/sample - loss: 91.0191\n", + "Epoch 107/200\n", + "323/323 [==============================] - 0s 45us/sample - loss: 91.0755\n", + "Epoch 108/200\n", + "323/323 [==============================] - 0s 41us/sample - loss: 90.8731\n", + "Epoch 109/200\n", + "323/323 [==============================] - 0s 45us/sample - loss: 90.9451\n", + "Epoch 110/200\n", + "323/323 [==============================] - 0s 41us/sample - loss: 91.1355\n", + "Epoch 111/200\n", + "323/323 [==============================] - 0s 46us/sample - loss: 91.0065\n", + "Epoch 112/200\n", + "323/323 [==============================] - 0s 44us/sample - loss: 90.9652\n", + "Epoch 113/200\n", + "323/323 [==============================] - 0s 45us/sample - loss: 90.9982\n", + "Epoch 114/200\n", + "323/323 [==============================] - 0s 41us/sample - loss: 90.9313\n", + "Epoch 115/200\n", + "323/323 [==============================] - 0s 45us/sample - loss: 90.9931\n", + "Epoch 116/200\n", + "323/323 [==============================] - 0s 44us/sample - loss: 90.9504\n", + "Epoch 117/200\n", + "323/323 [==============================] - 0s 45us/sample - loss: 90.9955\n", + "Epoch 118/200\n", + "323/323 [==============================] - 0s 42us/sample - loss: 90.9618\n", + "Epoch 119/200\n", + "323/323 [==============================] - 0s 45us/sample - loss: 90.8971\n", + "Epoch 120/200\n", + "323/323 [==============================] - 0s 43us/sample - loss: 90.9612\n", + "Epoch 121/200\n", + "323/323 [==============================] - 0s 43us/sample - loss: 91.0730\n", + "Epoch 122/200\n", + "323/323 [==============================] - 0s 43us/sample - loss: 90.9472\n", + "Epoch 123/200\n", + "323/323 [==============================] - 0s 45us/sample - loss: 90.9765\n", + "Epoch 124/200\n", + "323/323 [==============================] - 0s 42us/sample - loss: 90.9906\n", + "Epoch 125/200\n", + "323/323 [==============================] - 0s 46us/sample - loss: 90.9024\n", + "Epoch 126/200\n", + "323/323 [==============================] - 0s 42us/sample - loss: 90.9927\n", + "Epoch 127/200\n", + "323/323 [==============================] - 0s 45us/sample - loss: 90.8597\n", + "Epoch 128/200\n", + "323/323 [==============================] - 0s 41us/sample - loss: 90.9139\n", + "Epoch 129/200\n", + "323/323 [==============================] - 0s 43us/sample - loss: 91.0680\n", + "Epoch 130/200\n", + "323/323 [==============================] - 0s 41us/sample - loss: 90.9354\n", + "Epoch 131/200\n", + "323/323 [==============================] - 0s 44us/sample - loss: 90.9667\n", + "Epoch 132/200\n", + "323/323 [==============================] - 0s 42us/sample - loss: 91.0796\n", + "Epoch 133/200\n", + "323/323 [==============================] - 0s 43us/sample - loss: 91.0187\n", + "Epoch 134/200\n", + "323/323 [==============================] - 0s 42us/sample - loss: 90.9762\n", + "Epoch 135/200\n", + "323/323 [==============================] - 0s 44us/sample - loss: 90.9976\n", + "Epoch 136/200\n", + "323/323 [==============================] - 0s 42us/sample - loss: 90.8982\n", + "Epoch 137/200\n", + "323/323 [==============================] - 0s 46us/sample - loss: 90.8569\n", + "Epoch 138/200\n", + "323/323 [==============================] - 0s 42us/sample - loss: 91.0405\n", + "Epoch 139/200\n", + "323/323 [==============================] - 0s 44us/sample - loss: 91.0070\n", + "Epoch 140/200\n", + "323/323 [==============================] - 0s 40us/sample - loss: 90.9305\n", + "Epoch 141/200\n", + "323/323 [==============================] - 0s 43us/sample - loss: 91.0827\n", + "Epoch 142/200\n", + "323/323 [==============================] - 0s 43us/sample - loss: 91.1402\n", + "Epoch 143/200\n", + "323/323 [==============================] - 0s 42us/sample - loss: 90.9068\n", + "Epoch 144/200\n", + "323/323 [==============================] - 0s 45us/sample - loss: 90.9959\n", + "Epoch 145/200\n", + "323/323 [==============================] - 0s 44us/sample - loss: 90.9411\n", + "Epoch 146/200\n", + "323/323 [==============================] - 0s 42us/sample - loss: 91.0492\n", + "Epoch 147/200\n", + "323/323 [==============================] - 0s 44us/sample - loss: 90.9926\n", + "Epoch 148/200\n", + "323/323 [==============================] - 0s 44us/sample - loss: 90.9988\n", + "Epoch 149/200\n", + "323/323 [==============================] - 0s 42us/sample - loss: 90.9906\n", + "Epoch 150/200\n", + "323/323 [==============================] - 0s 43us/sample - loss: 90.9750\n", + "Epoch 151/200\n", + "323/323 [==============================] - 0s 44us/sample - loss: 90.9179\n", + "Epoch 152/200\n", + "323/323 [==============================] - 0s 43us/sample - loss: 91.0259\n", + "Epoch 153/200\n", + "323/323 [==============================] - 0s 43us/sample - loss: 90.9388\n", + "Epoch 154/200\n", + "323/323 [==============================] - 0s 46us/sample - loss: 91.0147\n", + "Epoch 155/200\n", + "323/323 [==============================] - 0s 40us/sample - loss: 90.9897\n", + "Epoch 156/200\n", + "323/323 [==============================] - 0s 47us/sample - loss: 91.0471\n", + "Epoch 157/200\n", + "323/323 [==============================] - 0s 41us/sample - loss: 90.9125\n", + "Epoch 158/200\n", + "323/323 [==============================] - 0s 43us/sample - loss: 90.9098\n", + "Epoch 159/200\n", + "323/323 [==============================] - 0s 41us/sample - loss: 90.9805\n", + "Epoch 160/200\n", + "323/323 [==============================] - 0s 45us/sample - loss: 90.9351\n", + "Epoch 161/200\n", + "323/323 [==============================] - 0s 41us/sample - loss: 90.9352\n", + "Epoch 162/200\n", + "323/323 [==============================] - 0s 44us/sample - loss: 90.9422\n", + "Epoch 163/200\n", + "323/323 [==============================] - 0s 40us/sample - loss: 90.9675\n", + "Epoch 164/200\n", + "323/323 [==============================] - 0s 43us/sample - loss: 90.9366\n", + "Epoch 165/200\n", + "323/323 [==============================] - 0s 43us/sample - loss: 90.9621\n", + "Epoch 166/200\n", + "323/323 [==============================] - 0s 42us/sample - loss: 90.9383\n", + "Epoch 167/200\n", + "323/323 [==============================] - 0s 45us/sample - loss: 90.9140\n", + "Epoch 168/200\n", + "323/323 [==============================] - 0s 45us/sample - loss: 91.0154\n", + "Epoch 169/200\n", + "323/323 [==============================] - 0s 46us/sample - loss: 90.9953\n", + "Epoch 170/200\n", + "323/323 [==============================] - 0s 42us/sample - loss: 91.0361\n", + "Epoch 171/200\n", + "323/323 [==============================] - 0s 44us/sample - loss: 90.8944\n", + "Epoch 172/200\n", + "323/323 [==============================] - 0s 41us/sample - loss: 90.9745\n", + "Epoch 173/200\n", + "323/323 [==============================] - 0s 53us/sample - loss: 91.0999\n", + "Epoch 174/200\n", + "323/323 [==============================] - 0s 42us/sample - loss: 90.9165\n", + "Epoch 175/200\n", + "323/323 [==============================] - 0s 40us/sample - loss: 90.9682\n", + "Epoch 176/200\n", + "323/323 [==============================] - 0s 45us/sample - loss: 90.9068\n", + "Epoch 177/200\n", + "323/323 [==============================] - 0s 39us/sample - loss: 90.9880\n", + "Epoch 178/200\n", + "323/323 [==============================] - 0s 45us/sample - loss: 90.9408\n", + "Epoch 179/200\n", + "323/323 [==============================] - 0s 44us/sample - loss: 91.0208\n", + "Epoch 180/200\n", + "323/323 [==============================] - 0s 46us/sample - loss: 91.0076\n", + "Epoch 181/200\n", + "323/323 [==============================] - 0s 42us/sample - loss: 90.9029\n", + "Epoch 182/200\n", + "323/323 [==============================] - 0s 44us/sample - loss: 90.9786\n", + "Epoch 183/200\n", + "323/323 [==============================] - 0s 41us/sample - loss: 90.9245\n", + "Epoch 184/200\n", + "323/323 [==============================] - 0s 45us/sample - loss: 91.0078\n", + "Epoch 185/200\n", + "323/323 [==============================] - 0s 42us/sample - loss: 90.9055\n", + "Epoch 186/200\n", + "323/323 [==============================] - 0s 41us/sample - loss: 91.0828\n", + "Epoch 187/200\n", + "323/323 [==============================] - 0s 45us/sample - loss: 90.9397\n", + "Epoch 188/200\n", + "323/323 [==============================] - 0s 43us/sample - loss: 90.9672\n", + "Epoch 189/200\n", + "323/323 [==============================] - 0s 44us/sample - loss: 91.0326\n", + "Epoch 190/200\n", + "323/323 [==============================] - 0s 40us/sample - loss: 90.9797\n", + "Epoch 191/200\n", + "323/323 [==============================] - 0s 42us/sample - loss: 90.9740\n", + "Epoch 192/200\n", + "323/323 [==============================] - 0s 45us/sample - loss: 90.9044\n", + "Epoch 193/200\n", + "323/323 [==============================] - 0s 40us/sample - loss: 91.0329\n", + "Epoch 194/200\n", + "323/323 [==============================] - 0s 45us/sample - loss: 90.9690\n", + "Epoch 195/200\n", + "323/323 [==============================] - 0s 40us/sample - loss: 90.9936\n", + "Epoch 196/200\n", + "323/323 [==============================] - 0s 43us/sample - loss: 90.9523\n", + "Epoch 197/200\n", + "323/323 [==============================] - 0s 42us/sample - loss: 90.9760\n", + "Epoch 198/200\n", + "323/323 [==============================] - 0s 43us/sample - loss: 91.0334\n", + "Epoch 199/200\n", + "323/323 [==============================] - 0s 44us/sample - loss: 90.9047\n", + "Epoch 200/200\n", + "323/323 [==============================] - 0s 42us/sample - loss: 90.9451\n", + "81/81 [==============================] - 0s 646us/sample - loss: 71.1000\n" + ] }, { - "cell_type": "markdown", - "metadata": { - "id": "r7xoMXoVaJy5", - "colab_type": "text" - }, - "source": [ - "You can see above the cost(loss) value i.e. MSE for batch size 40 while keeping epochs as 10 and learning rate as 0.01" + "data": { + "text/plain": [ + "71.09995378093954" ] - }, + }, + "execution_count": 32, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# play with learning rate and epochs\n", + "learning_rate = 0.1 # Replace ? with a floating-point number\n", + "epochs = 200 # Replace ? with an integer\n", + "optimizer = RMSprop(learning_rate)\n", + "model.compile(loss='mean_squared_error',optimizer=optimizer) # Compile the model\n", + "history = model.fit(X_train, y_train, epochs=epochs, batch_size=30) # Fit the model\n", + "model.evaluate(X_test, y_test) # Evaluate the model" + ] + }, + { + "cell_type": "code", + "execution_count": 33, + "metadata": {}, + "outputs": [ { - "cell_type": "markdown", - "metadata": { - "id": "y9VcjhruiIxd", - "colab_type": "text" - }, - "source": [ - "### Exercise 4\n", - "\n", - "Test several batch size values to see the impact of varying this value when defining your model." + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEGCAYAAABy53LJAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAgAElEQVR4nOy9ebhkV1ku/q69azpDz90JGUkgzCHJJTFIMHgJ+lMjCuLlpwwSfyhcnwcFLuojPlzvFRSuqFdBVOZZUEAIMglIGEIgBrohkM48dro7Sc/n9Blq2MP3+2Ptb61vrdq7atc5VX1Oddf7POc5p+pU7b32tL71ve83KCLCBBNMMMEEE5RFsNYDmGCCCSaYYLwwMRwTTDDBBBMMhInhmGCCCSaYYCBMDMcEE0wwwQQDYWI4JphgggkmGAiVtR7AicD27dvpvPPOW+thTDDBBBOMFXbt2nWYiHb4758ShuO8887Dzp0713oYE0wwwQRjBaXUnrz3R0pVKaVeo5TarZS6VSn12uy9F2avU6XUZT2++wGl1EGl1G7v/T9VSu1XSt2c/Vw9ymOYYIIJJpjAxcgMh1LqQgCvAHA5gIsBPFcpdQGA3QBeAOD6Ppv4EICfL/jf3xLRJdnPl4Y05AkmmGCCCUpglB7HkwDcRETLRBQD+BaAFxDR7UR0Z78vE9H1AI6OcHwTTDDBBBOsAKPUOHYDeLNSahuAJoCrAQxLaPhdpdTLsu39PhEdG9J2J5hgggkcRFGEffv2odVqrfVQRoZGo4Gzzz4b1Wq11OdHZjiI6Hal1FsBfBXAEoCbASRD2PQ7AfwZAMp+/18AL/c/pJR6JYBXAsC55547hN1OMMEEpyL27duHDRs24LzzzoNSaq2HM3QQEY4cOYJ9+/bh/PPPL/WdkYrjRPR+IrqUiJ4F4BiAu4awzQNElBBRCuC90BpK3ufeQ0SXEdFlO3Z0RZNNMMEEE5RCq9XCtm3bTkqjAQBKKWzbtm0gj2rUUVWnZb/PhRbEPz6EbZ4hXv4KNCU2wQQTTDAynKxGgzHo8Y06c/zTSqnbAHwewKuIaE4p9StKqX0AngHgi0qprwCAUupMpZSJkFJK/TOAGwE8QSm1Tyn1W9m//lIpdYtS6scAng3gf4z4GCZYA/x43xxu2Te/1sOYYIIJcjDSBEAiujLnvWsBXJvz/kPQAjq/flHBNn9jmGOcYH3i/3zpDgQB8LHf/sm1HsoEE6w5Zmdnsbi4uNbDMJjUqppgXaKTpIjiSZOxkxk/fPAYDh4/eSOVTmZMDMcE6xIpEdJJd8qTGr/94Z1477fvW+thjC0eeOABXHXVVbjooovwnOc8Bw8++CAA4FOf+hQuvPBCXHzxxXjWs54FALj11ltx+eWX45JLLsFFF12Eu+++e1X7PiVqVU0wfkhTgposa05qHG9FaEbDiNA/cXjj52/FbQ8dH+o2n3zmRvzvX3rKwN/7vd/7PVxzzTW45ppr8IEPfACvfvWr8dnPfhZvetOb8JWvfAVnnXUW5ubmAADvete78JrXvAYveclL0Ol0kCSrO++TR3OCdYmUgGTicBRioRWhNWaTrkScpIgSQjq5xivGjTfeiBe/+MUAgN/4jd/ADTfcAAB45jOfid/8zd/Ee9/7XmMgnvGMZ+Atb3kL3vrWt2LPnj2Ymppa1b4nHscE6xJJSjjJIyBXhZe+7yZcfv5WvOEXn7zWQ1kRWnEKQHuW44SVeAYnGu9617tw00034Ytf/CIuvfRS7Nq1Cy9+8Yvx9Kc/HV/84hdx9dVX493vfjeuuuqqFe9j4nFMsC6REiEZs0nlROLgQhsHF9prPYwVo9nRK+GJjrVyXHHFFfiXf/kXAMDHPvYxXHmlDmK999578fSnPx1vetObsGPHDuzduxf33XcfHvOYx+DVr341nve85+HHP/7xqvY9MRwTDIxWlOAtX7odS+14ZPvQ4vjINj/2SFJCPMYniGm2MT6EE4rl5WWcffbZ5udv/uZv8I53vAMf/OAHcdFFF+GjH/0o3v72twMA/vAP/xBPfepTceGFF+KKK67AxRdfjE9+8pO48MILcckll2D37t142ctetqrxTKiqCQbG7v3zeM/19+GKx27Df33CaSPZR0pAMFmNFiIlQpykaz2MFYNF8XGjqtYKaZp/rb/+9a93vfeZz3ym673Xv/71eP3rXz+08UwMxwQDgymkUVJJaUpDqYh5siJJx5vKm1BV442J4ZhgYPB8NUqqJCWCmswphUhSQjTGYWfscYzxIZzSmGgcEwwMotF7HAkRJovRYqQ02vM/ahiqakwuMo3JOFeKQY9vYjgmGBg8X0Uj5NjTVBuPCfKhPY7x1ThanfHROBqNBo4cOXLSGg/ux9FoNEp/Z0JVTTAwkhPgcUyiqnojGfNw5XHyOM4++2zs27cPhw4dWuuhjAzcAbAsJoZjgoHBD/uoNY40nWQAFiEd83Bco3GMgdNUrVZLd8Y7VTChqiYYGCdE40jHYzW6VkiIEBeEaI4DOKrqZKV/TnZMPI4JBgbPV6PMI6AJVVUIygIH4jEOSWqNEVU1QTcmHscEAyM5QVTVONAYawH29E4Kqmp8D+GUxsRwTDAwTgxVRRMaowAnIjhh1Gh29Kpgco3HExPDMcHAOBEJgEQTGqMITBWOcziuFccn13gcMdE4JhgYJqpqhBNXQgQ1vvPiSHEyeBwTjWO8MfE4JhgYJ4JjTyeZ44Xg8z/WJUdMAuAaD2SCFWFiOCYYGDyhj7bI4WQ1WgTOtk7GeNYdpwTACboxoaomGBgnKgFwjBfUI8WJiGobNWxU1fgew6mMiccxwcAw4viINY4xnhdHCvY4To48jjUeyAQrwsRwTDAwRu1xcILbOBTAWwucTOL4JBx3PDExHBMMDMuxj+ah581O+O98GHH8JNA4xtn4ncoYqeFQSr1GKbVbKXWrUuq12XsvzF6nSqnLenz3A0qpg0qp3d77W5VS/6GUujv7vWWUxzBBN2xZ9VEZDnL2M4ELthfj7JVxAuCYDv+Ux8gMh1LqQgCvAHA5gIsBPFcpdQGA3QBeAOD6Ppv4EICfz3n/9QCuI6LHAbguez1BDyx3Ylz119/E9x84OpTtpYYqGc2KV65Cx3ViHCWkoDyuXkdr0nN8rDFKj+NJAG4iomUiigF8C8ALiOh2Irqz35eJ6HoAeTPd8wB8OPv7wwCeP6wBn6w4stjBfYeXcPeBxaFsj0aucdi/J3RVN6RhHUeqh4jGPhz3+rsO4X984ua1HsaaYZSGYzeAK5VS25RS0wCuBnDOELZ7OhE9nP39CIDT8z6klHqlUmqnUmrnydyApQziIcf9JyPWOOSKegznxZFDTrbjGJIbJbYJ1biG49543xF89ub9az2MNcPIDAcR3Q7grQC+CuDLAG4GkAx5HwQg984jovcQ0WVEdNmOHTuGuduxA4fNDmuSseG4o9U4/L9HgTd+/lb82RduG+k+hg1psMcxJJe9DQBjWx1AF+E8daPCRiqOE9H7iehSInoWgGMA7hrCZg8opc4AgOz3wSFs86QGi9jD8hBsOO5o+HXJe4/acNyybx637J8f6T6GDcdwjKHG0RKGYxypNsAWmBzX8a8Wo46qOi37fS60IP7xIWz2cwCuyf6+BsC/DWGbJzV4chnWTT7qkiNys6N+LhOisRNoHapqHD2OrE6VUuOrcYw71bZajDqP49NKqdsAfB7Aq4hoTin1K0qpfQCeAeCLSqmvAIBS6kyl1Jf4i0qpfwZwI4AnKKX2KaV+K/vXXwD4WaXU3QB+Jns9QQ8wRTUsqqqo5MVCK8LFb/wqvn336jSlEyn+pimN3cM/7uI4U1UztcrYUlXsxY+hwzcUjLRWFRFdmfPetQCuzXn/IWgBnV+/qGCbRwA8Z4jDPOkRm5t8yFSVt9o9vNjBfDPC3qPNVW1f8saj5pATImDMHn65Sh/HnhzGcNTDsTR8gA000d58uLaDWQNMMsdPAQxbHOd5y+fXmYJY7Qpefn/0Hsf40Q3SVozjxNvqWI9jXKmq+BT3OCaG4xRANOTw2aKSI80hJXWdSI1jHHuby/M+jj05+D6ZrodjaziiicZRDKVUqJT62IkazASjAXscw7rJi1rHtoZUf0ganpFTVSmNXV+L9AR6ZKOA1DjGcPgALFU1jud/GOhpOIgoAfBopVTtBI1nghEgHrLHkRRoHMNqB+pMjCdA4xi3h3/cw3GZ0pypV8bu3DOMOH6KehxlxPH7AHxHKfU5AEv8JhH9zchGNcFQwRP8sEI3i0qODKvi6QmlqlKCUmq0OxkypDEdx8zxVmQNx7hOvMOunhAnKRIi1CvjIbSX0TjuBfCF7LMbxM8EYwJelQ7rIS0qcjg0cfwEFjkcR49DnpOxzOMwVFU4djk0jGEnAP71V+/Ci99701C2dSLQ1+MgojcCgFJqNns9nEp5E5wwsFs9LFqjn8axWvskdY1RrEiTlKAABIHKomLGa/Iad6qK78d6JRhjjWO4VNXD8008PLe6MPYTib4eh1LqQqXUDwHcCuBWpdQupdRTRj+0CYaFeMiro6LWpcOiqkZd5PAVH9mJN37+1mz7tOqH/08/dyu+dtuBYQytFNZLkcPDi2089x3fxr5jywN9j++fahiMLVVl6N9h6YYpjRXtWIaqeg+A1xHRo4no0QB+H8B7RzusCYaJYYvjlqryDEdnOAYqHXGewr5jy9h3rGm2v9p9fHrXPlxfMlv+1ofmVx0pJsOH15Kquu/QEnbvP447Hl4Y6Ht8P1Yr42s4uA/KMJNqx4kyLWM4ZojoG/yCiL4JYGZkI5pg6BhZdVyPJmnFw4+qGkU4bizKjAzD40ip3GrxjkeO4xf/7gbs2nNssO2nZPQjwE+QXDuqinl+We22DJKUEAYKoVJjT1UNK+ovTk4+j+M+pdSfKKXOy37+J3Sk1QRjAr4hR11yxIjjq46qGi1VJb2MYXgcCRGSEiv/+eUIAHB0qTPQ9j9783484y+uQyfuXuWu5WTDhqM1oOGIU0KoFIJAjdUqW2IUFafH6VyUMRwvB7ADwGcAfBrA9uy9CcYE0ZD5WKNxFCUADjGqahQPU5KSQ7cNg1or08KVz0tnwFT1B48uY245Mt8bVj+OXXuOYrEdr/j7vO9BDUdK2uMIsijocexpkRiqaljbo7EKdOibOQ7gDUT0aiJ6WtZb47VENJivfRJh/1xz4AdlrTHsLFfezIkpOTIaw2GjYlbv1ZQN6eXPtKPBJgg/6GAYeRzNToJfe/d/4l937l3R9/W+2eMY7HjihFAJFIIsf2acVtoMNprDoqoSGq/zUCZz/KdO0FjWPYgIv/C26/FP/7lnrYcyEE5UIydLVQ1n+8BoOsTFKZmV4jAogrIRMbyfQT0OLgrIBtnN41jZyW5FCeKUsNRZ+SIoWqHHkaQpwlAhzFyOMZovDYYecDJmUVVlMsd/mGWNfwpu5vhnRjaqdYo4JRxvxTi2PBhHvdYYdiMnI477JUeYg1+t2CypqlF5HIKqWs0eTMHHEpSR9TgGm2ibHgU4DI+DqbXVUF18Xw0qjrPGwQn74xhZxQZ7WGOP0xRE+n4KgvVfyaCM4WgAOALgKvEeQWsepxRstugaD2RADDvmXJYcIbIlO8zKeIhl1UcxqcRJ6iRwrWYXKZU/tyv1OJZH4HEYqmUAXv2PP3MLTt9Yx2t/5vEApMcx2BhY4wiVMq/HDaY52pDCofkyxCmhNu6GI9M4fkxEf3uCxrOuEcX6Jhk3Ma9fQbYkJSx1YmxsVEttzykJQkCY3efDSgCUwxzFuU4Fn7xaj8N6LiXE8RVqHH7QgZs5vrLR84QXDfD9Hz54DGdunuraxsAeh6dxjBFDYxCnvZ+pQZF3bdczymgcuZ34TkW0k+FMjCcaSR9a4jM/2Icr3/oNE+7ZD/Lwpc7RHFJ1XDeqalWbykWcao+DiJCSNlQrNVBypQgA13zge3jHdXcX7HdlHoc9r/p14pz/1VFVg9zLscfD87UflHpLUkIYKkPJjNvzBAy/VhVvZ1wiq8pQVd9RSv09gE/A1Th+MLJRrVNEQ46kOFHo18jp4EIb880I880IOzbU+25PTrJxQqhnd9Fo8jhGo3GkmdGQ71XCwSkCv8T8nY8sYPN0vufGx9IuaaAZzR5U1UrPtfE4BjBifu8SQ1XFK8zjGOtw3CFHVQ1ZbB81yhiOS7LfbxLvEVzN45RAlJOANQ7o18iJJ5GFVjnDUVQryTZy0ufoDz71I1xzxXm4+JzNA43XMRwjONdxFo7r9/0o8zD48B/4OKVCz47fL+vZMZqRu7pNPMO9EqxkxRynqdNxkO+r5oCRWSZzfIw9Dts6dtgex3icizLVcZ99IgYyDohMJMUaD2RA9BPHeSJaaJVLBpNzVZJjOFIiLLRjfOaH+/GER20Y3HCIeXXY5zpNtRju516slCGwgQKp+V20ik9W6HH01jhWNnAe4yCtZ9M0X18ZVBxPUkIlCExQxbg9T8AoIhXHy+Mo1DiUUm8Tf7/G+9+HRjimdYvOkFuwnijYmPOCCS17v6zhcD0O/V0icjSONGd1XBajpKpMjaocj2NF2/M9joSKDQeL4wNSO8sdfV18qkqpVYjjfe6J/O+kzv5WWqsqTt3M8XGLqtL3Tvb3kKmqcfE4eonjzxJ/X+P976IRjGXdI69W0DjAro6K/m+pqjLwNQ5AG1Uj3goRtUx+AwDMidyYUbaOlXTPMEqbGI1DiJuFnh2L4yvVOIw4rv+ohcGKw3F50h+E6kpScva30pIjSZqOdThu7Nw3w9lmMuDzstboZThUwd+nLIadgZ2H791/FH/+hduGus2oT8x+agxHSY8jp+x5q2PflBVny0z837rrEC7786/h4EIr+77937CFU1vw0aPEVnhNTVSVaM/b3+MYlKpyaREeay0MVh2OO8j3/bpeHJk16PFYj2M8NQ5JDw4rCirxKM/1jl6GI1BKbVFKbRN/b1VKbQUwHo1xh4zoBFBV191+AO+74f4VryTz0K+sOr9/vKTHkZe5LOkKpxZUiUlh9/55xCnh2FJkvs8Y9nOUCOOflPBsfvjgsZ7UknzgKSuvXiiOr8DjiJPUUKTSGIeBQiVUKxbHpSZT/jteOC7ncQwojqeU5XFkXNWYORyOLjRsqmpcjGgvw7EJwC4AOwFsBPCD7PUulOw5rpR6jVJqt1LqVqXUa7P3Xpi9TpVSl/X47s8rpe5USt2jlHq9eP9DSqn7lVI3Zz+XFG1j2OAHeJQ3Ot+US+3hFVLsV1Y9GdDjcKmqbp47ESJqmRUtd5DL640+dKpKUEv9orcOL7bxgnd+F/9+yyOF20vFA8/HXJRUx58dRONoCSNjJxcgVAphsHKPIzLhuKuhqrIih4OG4yauxjEukyVjFHlGRRWn1ysKo6qI6LzVbFgpdSGAVwC4HEAHwJeVUl8AsBvACwC8u8d3QwD/AOBnAewD8H2l1OeIiDmcPySif13N+FYCDscd5Y3OXs3xVoRNBfkAg6JvVNWgVJXYjI2sScT/Rb+LEhM/d+MzIY6raOT0jTsP4ilnbMRpGxu5/5fGqV9NrPlmBCJgoUfpcVlyxJahyJ9NVuJx5DVwSokQBEA1VEMoOVL+/PoeR2eFHkeSEmqVQBQ5HI/JkiHP+bD0zmEXTRw1yvTjWCmeBOAmIlomohjAtwC8gIhuJ6I7+3z3cgD3ENF9RNQB8C8AnjfCsZbCiUgAZMNRdhIvAzNZ9jUc5agqxyMooqpE9FI/7D3KHofVH+y2Sg0JgDYyr/jwTnz8ew8WfsZp4JRzHBI8IfaanOX2bIhrgZa0gnBcxyCLfWmPY+WNkPieGCQBMPU0Dj4v7TgdyMCzxlEUjpuktK4DUOI+C46VYJCaZ+sBozQcuwFcqZTappSaBnA1gHNKfvcsALJRwL7sPcablVI/Vkr9rVIqN2NNKfVKpdROpdTOQ4fK9YPuh04ynH4TPg4eb+HmvXPZPthwlJvEy6BfI6eheBydfI+j34OQpIT9c+xxdGtIg6xGeUXca2KWiVv9tJQytbfkAy9DcnvtexCPY9k5rzDjCQKF6irE8UEDPVi/cRIAxXcHMYascRRFVb3yIzvxvz93a+ntnWjI6zvskiNr2Qp4EIzMcBDR7QDeCuCrAL4M4GYAwyDu/xjAEwH8BICtAP6oYP/vIaLLiOiyHTt2DGG3tsjhsN3Jf/zmvXjlR3YCkFncw/c4isZtDEe7pMeRs+rkSbYWBo7h6GdkDxxvdU1itFLDUWIylBSa49nk7Md4HD23x/u2WdVF3QBXkgDoe3KAqC4bqFUnAJYV163RkiVHRJ2yAegqrXEEhXkcDx5dxgNHlnK+2R93PHIc7/v2fSv6blnIcz70PI5xD8flCKqinzIbJ6L3Z10DnwXgGIC7So5rP1zv5OzsPRDRw6TRBvBBaFrrhMBGtwx3u8eWO2ZlaaiqkpN4GfTrVmbzOFaSAOhSVTP10BGK+62ImaaSn5XsySAPZqfEZCizr/tSVVF5qkp7O733z5PuSjUOmV0cKoVKsIqoKhNpV24seddT7nsQgVyXHIGhqvxzH6c0sG7CuPaH+/HmL92+ou+WhZvHMSyqarjbGzV6lRzZBV2TSgE4F3riVwA2A3gQwPn9Nq6UOo2IDiqlzoUWxH+y5Li+D+BxSqnzoQ3GrwN4cbbNM4joYaXvuudDU2InBJEXFjksLLZiM+mNRuPIJpyiCW3AkiMpEZTS0WVG48ge9OlaxSlN0e9c7c2EcT3O7vM7yIK6zPWR48rTaiTKeBxyMo/7RCrxdgaKqirwOAIOx10lVVX2+3krYqcy8iAeR5qiElhx3L9cUZIOnI3OaHWSkTdEGgVVZcOjx8NwFHocRHQ+ET0GwNcA/BIRbSeibQCeC00/lcGnlVK3Afg8gFcR0ZxS6leUUvsAPAPAF5VSXwEApdSZSqkvZfuOAfwugK8AuB3AJ4mISc+PKaVuAXALgO0A/nzAY14xhl1KmbHQjhElWmCMRkFV9cnjSAbcZ0pANQycbXLY6Gy9orOyc2rvfO/+o/jvH93p0FeOx5ETVeUbgW/ddQgf+s79ueOKSqyiJZ3Vr8rsoBqHzB7P/exKoqqi7qgqK46vIgFwwA6AeXSnNJCD1KtKCT3DceOEVmw4jJc4wglYXt+heRzpcLc3apQpCPqTRPQKfkFE/66U+ssyGyeiK3PeuxbAtTnvPwQtoPPrLwH4Us7n1qwqb2dIrVF9LLVjECETH2047rDQLxps0JIjaUqohwE6cWpj+Tv9qarvP3AUX7n1AJpRgpmsFvs+x+Po1kX8c/2vu/bhutsP4GXPOK9rRTmIxpFSfxG+nMYhPY7ek/FK+nHIlTwZw6En3mqw8nBc63EMSlXlZ033oqqW2jGmqqG5XtrjsAmA/rmPkhStFXZn9isJjwLSYA6trPpJGFX1kFLqfyqlzst+3gDgoVEPbD2ik7MiHgYWszyBKElHGo5b9DDJMNEyq+GUdBy+3ravcVQKxfG8fJK9x5axdabmvC/nXX/I7SjBcifBg8JTYXRKeIRysuNgh6LvlNE4JFXFE0qRYeBzESXlw039xEreZxAgE8dXqnGsjKpKyT0O7hveKqCqOnGKK/7i67j2h/vtthJNIwUFUVWroaqssR9ddJIbjbf6uYBk3tNJFFX1IgA7oL2Ez2R/n5JdAUdFVS1mRiKKR0NVyVVxXry9nDzKeB2SqpJ5HNVQoRYGhR6H7URoH45H5ls4d+u08/9eUVUckXTHI8e7xhX1oeT87ckJfrVRVXpsvQ2NmzxXboJwEgDFdQyVDsddbR5HWaoqr5x6nKSYzTzHIo+jGSWYb0a4//CS8303HNcf28qpqpYx9iOkquR9M4R5Xh7/SeNxENFRInoNgJ8ioqcR0WuJ6OgJGNu6g23kNNztssfRcTyO4VNVQH5EmFzlfPGWh/FTb/16z/1TnsfRSdDI6IiU8sXxvOzYZpRg41TVGWevFR17RLc9vNA1rjJUlZxQZDhp3sqxjMYh/8c8v1yVS8hzUbbveDPKiarKxPFwGFRVye/nRRLFKWFDZjiand5e1nzT3k8cTmzCcXM0jlaUrmg1PyqNQ9PJOYuhIbAPw6jSfKLR13Aopa7IBO7bs9cXK6X+ceQjW4cYRZHDKEnNKnpkVJWsL5Rj9eTN+ukf7Me+Y03c+lD3ip6REqzhYI0jSjBVDREqpZPrcsTxOO1+8Dpxiulq6HxWPjtdVFW2sr3j4e7xdUp4HK6421vkLDMJSWMgV915uRzSaHH/eond++fxqZ17nfdy8ziMx7EaqmqwKB73OvL9StjQ0Ea/qLQ63wdzwnCwx2F6jnslZvg6Dlp1Fxhe+2KJ+WaEn3jz13Dd7QcBeHkcQ9iPs7ga9zwOgb8F8HMAjgAAEf0Ibq+OUwadEivaQbEk6iBFSeq0cR0WotRy0Xlj13H1+gO37NMZ7LfnTMzy811RVVGCqVqoy2AQOVVoGXkeQZSkmK6F2f+7gw/88fKkcnseVVWiX0pRHkJPqqqExgFo/SVv22Yfqfxs9zb/6T/34E1eSf3CPA7jcazsXixD60nkXcc4STHb6E1V5XkcvsYhT73vjQ4KQ1UNkRY4tNDGcifBw8d12f9+982gGKbHQUS452C3Nz5slMocJ6K93lvDK906RmCaZJg9IqRn0YlTUXJkuB5HveJqEs7/U8LmjC7if9+RQwUxUiLUQjd5q5l5HEqh0ONIcmLVO3GKKTYcJaKqeMLde7TZZVzzPBofcjwdh6rq/uwgUVWAu0LONRwF+gpjbjnqyofwi0fy70ApVIJg5ZnjQqcoA8fgSqqqwVRVb49jPs/jyFnMxKs0HM0RaBxMJSc5xnYYi8i8c0tE+LMv3Jar5b3qYz/Ap3fty93WDx48hp/5m+tx14HRGo8yhmOvUuoKAKSUqiql/gAZbXWqYRRU1aLwODojoKq4zWW94tJB/mc2i0q8tUqQe8MyKIeqakYp6lXhceRM4v5klaY698F6HJbDl/uSaMcpNmaTlf9wlImqKqSq8jyOAfI4AHeSzzMMTm2nHI9jrtlxQrJ5DHaCtccQrjIBcDVUFf8dCXG8iFbizx6XHgdlJUdywnHleUqvgdUAACAASURBVFtJ9vgoNA4OXonFcTOGYTjcfCIOx4/x/hvux9duO9D1+evvOoSde47lbuvQQhuAXoSMEmUMx+8AeBV0kcH9AC7JXp9ysFFVvT/3D9+4p4urLoJLVdmoqsV2PNTVTN0Ts/3PbJ6umde/cOGjcOeBhZ7hu7443uokmKoGCJVyoqoc2skLAeVJYqpWcd6Xu/Un9E6c4sKzNgEA7j3o1jMqU/Y+LjAcueJ4p//qVd4LcqLL8wTkPvIMy3xT3wuysGGzY3NebB93CHF8pYajv3cmkWdw40Qb/UAVT/J8Ghyqyngc3YZDHs+KDMcINI7FtttgbNhidl5jNKY987yuOCWHFpUoE0I+DPQ0HFlfjN8gopcQ0elEdBoRvZSIjox0VOsUkVgp98LHb3oQn/tRuVSXBU/jkJPZYo8+EGXBE1gjE6Dzxp4IquqszVO48nE70IrSwkJzCVFuOC4neWkvJ0fj8B48njxnauwNlQnHTbAly/vwefUyFV8dqiqnSZJEswRfPghV5Xoc3Q/+fNZ3XXouzSjBTGZYZan6UAHVE0hV5U2WcZqiEgZoVMNCcZzHp3ubkMlZCGQ4rhhC7HlbgyBNbWXkYWocC57HIa/tMHK68ioYcITeco7xjNO0OPy5M5gnuVL0NBxElCCrETVBuQRAIsLBhRaOLpVLfV0UlFQUa8PBtNEwBHKeTBvVYo8jSQn1aoDpWohLzt2MJz5KN3gs0jmIckqOsDiuNFWVJ4T7GgdP3ExV5U38eVQVh4D6yYpl2qHK/3X6ieMlqCpyqCqRXJgzIffzODjyyPc4WIB28jgChTBcRT8OUbCzTGSQmzHOlA2hGihtOIrEcbGAWOokZryVQNmADXEOZffEXoYjSbt7u/cz3CuF0Thy6NdRaRx8PvO8rjilwhIvZRY7w0AZquoGpdTfK6WuVEo9jX9GOqp1ik52MXtpHMeWI0QJleYYJVXVzkpzcyb1MHQOvrF7aRx6IgrwZ8+7EL/77AtwwWmzCANVGFnlUFWirLqTx5HjcURm4tPf4Ymfu8HJcNxKoLq+z99hQdbn1TslqCpnPOL7PamqXtsrCMctogRZr/A1jk6cGoMhJwtZnsWUHMnE8eoqMseL+moUwc3HseHjlTDAVDUszOOQc/t8MzLni6PCANf4ymvSi6r666/eiRe/9z+d96ShGaXGwccfBmooHkeeN9cu8Dh0Em9x+POJSIAEytWq4p7ebxLvEYA1qxm1VuCHrdcK7eCCDtkr7XFIwxGlSFLC1uka7sPSUAwHT+y9oqoS0pzzr156tnnvURsbeGi+2fVZgKOqcjwOEVUl+XizH696LE/01TBAJVAm94ETxOKUPP47RZySmUh9j2NQqsopVtfD4+itcUj6qbfHkaSE6VoFi+24y+OQGkAzisXfCc7IPFCZx6GNbbByjWPAQn1OVVwTjkuohAr1alDocchtzy9H2JIdi6tx5I+raHIEdHHMvUfd+zMv52UYsB6Hm21fr6w8c18iT+Ph8+kbDj4/ReemzGJnGOhrOIjo2SMdwRihTFTVweM6qqEZJWh2EhNqWgRpHHjCsB7HEKiqlKkqN+RVIk7IPMSMWqV4UkpTW93UaBxZ5niUpJqq8rwLuW8ZlcP7qgTKGJY0E0/byKd3GtUQ1VB1Tc79anL5/3OK1fX0OMrlcTgeR0Eex1Qt1IYj9g2HXWjIyaLVkRpHtp3MsOoEwJVREk4GfZpiCr3v09yoqjRFNdAeR1GtKnl+5puRuQe0x9G9bXlNelFVcdJNVfULTlgpWIf0PY5aJRhKyZE8OpcNg28gfA3Ex4moDgyU8ziglPpFAE8B0OD3iOhNxd84OdEpUXLkQJYkBOgGTVO1qZ7blFTVUltf9G2zw6OqfI8jz7XmKBeJSo/ucjqPAFkegRY8WRzniCrLbYuxeBoHU021MHAK9jEVE2a0F4PPf70SoJZV55XoxO72c89HgTjunxc+Jn0MvQyR/VtOnkUex1RmwP2eHJLa9KmqWS+qKk3t+VltAiBQjtbweXimTCphb43D8TiakVkcuD3HpeEoR1XFaXdBTjnJDlXjYKrKi0TTHkfxZDC33MGHvvsAXvXsC4wmmIe8sjyWqnLngMjzSHzYe3aNNQ6l1LsA/BqA34Nu5PRCAI8e6ajWKco0CjqYxVED2nD0g6Sq+EHZMr06j6OVeTuAFMfdXAkJrn0k0WtSMglooa6V1ElSpASTOZ6mUhzvpkSMxiE8Dt0/m6OqkGUWe5nZxnCEqFaCLrpnUKrKjcd3PydXdD1LjhREVeU1c0rI5qx0exySqnINx3Q9NN/n32GQdQBM8wtX9oNjOEpMMolHGfL3q2GARjUoXAHL7x0XGkdROG5ZjyNKqOv6j56qshQdoO/DXvbpW3cdwtu+djd++OBcz+3nie1FVJWvgfhoec/9qFBGHL+CiF4G4BgRvRG6AdPjRzqqdYoyE9NB6XEs9Z/4F9qxiRLiG5SpquMr9Dj+5LO78Tv/tMsZa0+NI8fj0BN5keHQbT/ZS+BJoyESAHPDcRP3wet4HoeMGgqU3oeTbS3E9DyPg72rlRgOX7dqlly9FiUA5k3GcUZVAd3Cfp7HQaTP7XRGVUkvThvu4mvaD0WlV4qQeLQef78SKExVw9ywUcA9r3PNjhlrGAR9w3F7aRxRtmCRRtOlqkbgcYjqzkppb6uX3snXuF8Wd17IOj9TvvGUteHyUMZLHgbKGA5WoJaVUmcCiACcMbohrV+UaeR0cKGNalaO42gJj2OpHZu8BHZLNzQqUD2SqvJwvBUZw7N/rmkoM54c61k4bp4+EyepiXBhhEG3hsAg0j2jK9lkzzfxVDVEoJTXOlbspyAclz0OE3yQrahDpZxwXKZ36pUAtUq34ShT9r5Q46AehmMlUVU5k3GaktErugyH8DiWvSQ2QzMKqirM+nH0G18R+uk7PnyqiiewShhgpl7polTytu1qHMgNx3VKjvSiqhIyzc/M5x1jP3yNQ4bjVjKPr9e565Q0HNJwWg0jPxw39v7vY10kAGb4glJqM4C/AvADAA8A+OdRDmq9oszEdHChjcfumAUAHCsRWbXYkoZDX/RqGKAaBH1Lm8wtd7DciREnKV74zhvxuk/cDEDfPH6lWBuOm5NfQOgyHNUeOQJJSmbFG6dkbu6pWoAwgCOO58X/+6UbusNxCUplVJUYA6/CjOHwHg5ThLLHeSvqieEfq9sHo4c4PkBUVZySoQx9wzEvFhm+wGm9xWw8QhzPG3sZyImlaIEg4YfjsuGphQoz9YqTjyRRrHEEueG4TsmRXh6HCAnee3QZ37nnsOfxrW7FPd+McNX//SZ27583meNS46gEAQLPI/ZR1nAk3rkF7P3RZTiMxlEgjp+gqKoy/Tj+jIjmiOjT0NrGE4noT0Y6qnWKvFpI373nMJ791980K/wDx1t43Ok6ga5MSO5iO8bWLERRGo4g6D8h/PaHd+KX//47ePf19+HOAwtGX2l2EqeCKdBb4+BWnhKVHqGeKWVUSdYPgh/wRiUrq04kVsf2e10ahx+OazLzgUCh68GUmkgvqqp3+KyYMHuI4/wANqq9Q17lNeqXx5GkqWl2ladxbGxUnPIdvA2TpS8yx4Os5ziwMiFY5pQM7HEkZCa4ShhgQ71SWOVAXr/5ZpyrcTgBFKU1DnsPvf+G+/E7/7Qrt+nVSvHQXBP3HVrCrj3HjFGUGkcly0PpRVXx/XrnIws9dag8HZCN4HKUON+VkYN5Bn/d5HEopV6W8x6I6COjGdL6BV8oeQ/cc2gR9x9ewtuvuxtvfv6FOLjQxpmbGtg0VcVcSXGcxXCOsKqGqufEzXh4voX9c0381VfuBGAnnFaUmLFGRsgrpqo4E1miEqquyB+G9ghs61JjOGo6AZBIlLSQHoevcchwXOHhJKT7TfC2GG3jcYS5HkcZj7CoVlWXx5Ed04ZGtQ9VJb5TIqoqCBRqlaA7qqoZYctMDSkJqirbeCUMHO/L9zhWEnoaJSmmqiGWOkkpIdWf3PgaVgLtcbSzqgd+9FCXxyGS5zK715Wrw+hHVQH6HlruxFhoxQ41vNoVN1+/h+abxihKL7oSKqPnFYHv12PLEQ4vdrBjQz33c/JWseK3pSs7SWoYA3lcrSjpOt/rKRz3J8TfDQDPgaasTkHD0U2F8A38ie/vxQsvPRudOMWODXVsnanhaIns8cV2jA2NCqqhcjyOMlmpC60IjzttFssd3UWPXepmlFhag+mOam9xvMtw9MhKJgJCZTlejuRgjUOfF57E7ff8PAs3HNeK8Q5V5URV6f0UeRwdQSUUIXUMRzHPbwxHNinyMckwUn97faOqMm68nqPPzC1H2DRVxVI76SobUQ2V430lqTasq9E44kQL9bIMSL/Pm2MTq91qGJhw4aV27BTLBOz12zRV7crjsP04ur3K2XqltMfB510mBK6W4+frt+fwcldQTJxm1X1VH41DNOu6+8BCD8PRrfFIKqrZSazhENehFaXYYBIkss+uF42DiH5P/LwCwNMAzI50VOsQnJ/Af8v3AT2R/v4nfwQAOH1jA1umq10ax99//W786eduNa+JCIutGDP1CqphgKUOexxBzzwK8912jJ97yqNwwx89G5ecs9mUfWh2hMeR2lU60D3JEFGuxlERYnXeuQiCTONIyFA0U1lUFZAfSGC0DU/jqFcCncwmikhySQp5rp08jh7ieC+DW6RxdFNVNlCBcwYuf8t1+OzN+53PuVFV0rvK8TgyT0p7HN1U1aapKqZrodm3M8mKhUSanf9q4GbvD4I4tavYvG6FPtzmWqmNqgqVMRx5eUd8GrbO1HC8GZmJj4Mf9Ge6F2IbGpWeHgffm1FC5j7Yd2xZHN9wPI67RVMkGVVVDbvvTx/y/ryzh86R17hMFsEsCjPOE8j5HlxzjSMHSwDOH/ZA1jv4RlLeKphXgX/6y0/BfYd1NdnT2OPwDMcN9xzGt+8+ZF63Y/0AzjYqqFUCLLddj6PXTbnUSZASR2CprF6QfnBbUdrF93ORQ5+TlUXnJLQ3kT+hMFXFxo0N1lTNehy29HY3HZSI1aI8XllWncNx5XB5sm1UteHoyhzP2aePsnkcDlWVEJY7MY4udfDde9zC0NL7lA97lHPtkkQbxDyPY74ZYfN0TRsOj26oZqGrDlUlPY4VrC6jxIYGr0Tj4HNXCQJThHEpJ7KKt71lWnsc/Oz0KzmysVHtGY4b53gc+45Zj2O1Ggc/Nw8cscbI9TiUaSFQhE6cYtNUFZunq7kC+b/f8jA+dtOe/CKHYhEiQ52lkc+jklsd11sdFcpoHJ+Hrk0FaEPzZACfHOWg1iM6YnWcR3H86qVn4dhyB2/72l04f/sMtkzXcJvXt3u+GTsiIv+9ocvjYOG5+Kbk5EDu+cwTDifk8ZrAiOMFHge/9hMA2ZvIAxFM5nIiNA7tcfD5skbA7MvvxyHDcYWmwwmJflSVzfsIUQ27V+1l8myK8he6PQ5LmcSpXdXe8Yg7AfhUlS6FQrmTeZzq2k553tLccgebpipoiJwI3ob1vpAdHzKPj4304JNklKRmMVEmqsqnU/jcVUOFSqjvraUcgZzP65bpGn68b97xolSOxhEJj+N4jwTYSNxDfC73H2sainVYHkcejRQnVCqARWsTAXZsqJtSRBKf2rUPD8018Uc//0TzHutaMtCiSPTPS7pcTxrHX4u/YwB7iCi/b+FJho/e+AC+/8Ax/N2L/ouJwNG9B9ymNICmql717Avw8meej6laiC0zta48juPNyAlbZNd+pl5BLQzMDVINg74ls/m7XCl2qhYiJZuBHKU6OcqEdBZoHL08jiIKI81WvCYcN7tZ69XArCLzKtX6GkdRdVzikiNKITdzvOqG437jzoN42jlbbNRbzwiWfKqqWByvGIES0KGVUhOSc24n0SXfoyTOz+PIjssXx9OUtMcxpT2OlvfwV0JX70mzPBqOlOu1Mi9CnNjyJ6USAH3DIaKqelNV+nvTmQFmwVhSVXklRzY0Kjiw0EIRjMeRpOZcdhLdkmBuOVo1x59X9t56HGmWuR+gExcn6bbj1OYo5TzLnSygID8B0F7T5YKgC/+6R4mlEJMVLCYGQRmN41vi5zuDGA2l1GuUUruVUrcqpV6bvffC7HWqlLqsx3d/Xil1p1LqHqXU68X75yulbsre/4RSqla0jdXigSPLuO523brRlO6ouA2R5AoKgHH/t0zX0IpSZ7VwvBU5YiR7DRsbVVRD5WgcoepdMrvLcGSTANNjRJzh62ocXYYje2j9Ioey6KCPxNSq0l6RFMf5POSt2BLvnEWcgZutniNhWEKlcqgqmwBYz8TxxXaMl3/o+/jUrr0DZ47LVb9/rpn2mxUahx6D2+DKN1KNWrFuwIlj9UroTEyLnRgpAZunq04Wto1c0m1WpcYWKtVVcWAQRGlqDE8ZWse5jonN46gGSojj3QaMvzeVLVx4spPieF447oZGtbBUO+BWWJbXkZ+H1a64fWM6VQ3dqsCZ7tRrfu5khqMWBk7ot/x/lJBTap7pYelNy+TKXh7HqMrK56FMraoFpdTxnJ8FpVRhY2ql1IUAXgHgcgAXA3iuUuoCALsBvADA9T2+GwL4BwC/AE2NvUgp9eTs328F8LdEdAGAYwB+q9SRrgCbp6pZuKKtzdOodsfU+5E2ALB1RlNI7HWkKZkHnH/Lyb8a2no/ZYS3PKoKcHNH4tS2om0UeRwirFKiUrBK0l3cZMkRkceRZzhyaAh+ANtZ+KZSbhZuSlpLCgKvH7UsOZLRPcvtGET6XEYeFZaHonDcvJIjYVZOQ3P69v+ywZX/PdOUKs7RODJRuxYGTrLgovA8p2phV1XeSuh6XyyOs7ZQlHxXhCTV15AXG2Woqu7McfaGAsxktbQ4qs/fFwBTNoWNojaG+jOOx8Eax1Slb8kR/i0n2dm6W4J+pfDPyebpqps5HiqEqvv6S7TjFLUwyHrDd5/jdqK1SFkhQHocbASLijf6GocssrkeGjm9DcDroXuOnw3gjwC8jYg2ENHGHt97EoCbiGiZiGIA3wLwAiK6nYju7LPPywHcQ0T3EVEHwL8AeJ7Ss/NVAP41+9yHATy/xDGsCJuyxLz5ZiSEWXeVFmerPx8clsiRVQvZBAdIw2Enf26MBHBUVe+2oHlUFeAajk6SOgXZ5LgZxmPy4sGLyinwMQRistfx5MoYAcD1OEwDIrNitsJmPduv7C1hoqoKqKpaaMVxNloyW56o+IF2EgBFwENXyZGOznNgOk6uau94xK6XfG2kFgZQKv/BNeG4Xv8Kk0HPTZEi3+Nwo6pYHJ9docdhF0Er8zj0gsQatQ3ZZL2Y53FkY+b7k71q6XE4IdKZwd3YqKLpJb9JSJ1MXhtuQ7zaFbdPVW2aqjolcypB/wCWTpxmEYOB021S/r+T2GjNmujv0YpSU7NuuZPvSfT0ONaaqgLwy0T0j0S0QETHieidAJ5X4nu7AVyplNqmlJoGcDWAc0qO6ywAe8Xrfdl72wDMZYZIvt8FpdQrlVI7lVI7Dx06lPeRvtiU9eGeW45EzSd9Y/L9nKTddZ4AdJWWOC5qEfEK8bjncTCqhvMvHlshVSWToESGb5HHwZORb/wqOf0u5OcDBUfj4OPl7cgHhXeZV6uKDabsLcFaQJBDVVWyMOBqRlWZYnCdxI3g6jPhABB0S5DjccSYqoXGE5MT/e3C4/D3U8nKxRTlcQSBwnnbZnDvwSUz4UVm9a4wXRNUldE4AieChzUW9ji4llIvMTnvHBiPY1CqShiOaiA8jhzPh8/rdLYv9qZcjUOOTdOXM3WtLX159yP4+h0HurbJ4/E9Dr5mq9U4/InX8TiycNxA9c616hiNI388nThBnKZmGzVRWLQdp2bx6RiOHhrHuqKqACwppV6ilAqVUoFS6iXQIbk9QUS3Q9NKXwXwZQA3AxhcxVshiOg9RHQZEV22Y8eOFW2DL9x8s2NXaV4GdpJ250AA3a1P5UPNLj1P/hsbVdNRD9C8sc6kLr75eRuWqtKTyNFFaziiHI+jKKoqtzpursCrfwdZkbcoSU33PwA2qkpMtLJUg3wdJdZwdIXjBsp0E2RIQ8PiOD8sy14GdK86W/L8cAZ8Xq2qRjUwkUv88G6frbseR+omT7Jeww/4I/MtvPR9N2FuuWM8jmc8dhuaUYIfPnjMjAPQE0dDhuOakFc3qirNos54pb/UjvHjfXO45I1fxQOH+z6aZrs2HHdAqioR1XFDbcgbVR0V+NVbH8G7vnWv+azvcViqyvYc96OqqpnnBQB/8Kkf4X/9261ue1mnb7xnOLhC85CoqpnMEM3UKk6tNY506zVBdxIhjucZjiRFFKe5Hkc7Sky3xKKqv35PjmaBgRkFyhiOFwP4fwEcAHAQuh/Hi8tsnIjeT0SXEtGzoPWIu0qOaz9c7+Ts7L0jADYrpSre+yPB5hyPw3fvizyO0DccTbsaY0+DqarZRgXVit2Gn9cgce0P92HXnmNYaMUIlHXNp2r6Uh6RVFWcCnE8v+SIbKzjjz/v4bMeh8Lm6SqOLnZM9z9+H+jOypYRXtLjYE9LGipuFOVnz7cz1x/Qk2yUEJaz1XYzir0eE8WGg41PlKQmH8I/L60oo6qy88Ji+eNPn8X+uaag3+y5BbKJVFy7Wx+axw33HMbdBxdNxvFPPmYbAgV8557DeqxCL5iuVtDJJhMZ8Sb1HhbHG1VdimSxFeP+w0tISZeh8XH7w8dzM+brJhy3jMeRmsleFzm0ehygw5YXWjE+uXMvPnrjHvM93i8bjqagqmR1gH+7eT927TmqV/OBEtRWgn3Hmtgj8ilcnj91uP6pWtizJUBZ8PGds3Uas42KUxInzrS5IOhTqyrWSZZFoe1GHBeGQ2ocW3I8jrERx4noASJ6HhFtz36eT0QPlNm4Uuq07Pe50IL4x0uO6/sAHpdFUNUA/DqAz5F+Wr8B4L9ln7sGwL+V3ObA2DxtDUcn4179RDru1+2jp8fRsuL4TNb8yKeq8jSGJCW84drdeN+378NCK8ZsvWI0hamqtqXHlovE8czgeSsRGdEhUS0Ix5VU1aM2NnBwoY2ljvQ43HBc3odDR4hQSulxWHE8q1WlujPHpccB2PO63El6htfK91lXiZL8fBFA02L1SmgKCfJDubFRBZGlINPs+vPquRq4yYmmYxtHEymFTVNVPPXszfjOvTqZMBIiOIvrzSgxFFIltOeCM/2DbOKdzQoMchi2L5juPbqMX3j7t/H1Ow7a85/tb2ogjcN6mY44np2f2XoFS+0YB463c2uA8b6WhMcBwNA9f/nlO/Hh7+5BlKSoCI+DnzeZOCu3HyVa49goKNvheBz6+48/fQN2zNZNt0vASwDsR1WFmqrKC+/txCmiVIrjofH+WnGKmXqow/Sjbj0M6KaqRtUBMQ+FhkMp9Qql1OOyv5VS6gNKqXml1I+VUk8ruf1PK6VuA/B5AK8iojml1K8opfZBN4T6olLqK9k+zlRKfQkAMg3jdwF8BcDtAD5JRFyr448AvE4pdQ+05vH+gY+6JFjjmG9GTr9rAE7dID95DpC9EnI0Duakm5GhmiRVxVm1/qphz5ElLHcSPDzfwvGW/S6QL45LqoofQH+bsuicRCUMTEivhKGqlMKjNjUQp4T9x5omDNWPqgK0V5NXXp0fLD5mmeeRr3HYMhm8ymdPTlYE5m3kIZYeR5wazybP42hUAzPB8apv41TWTyMSekw2ofLxVwJrOPiYDEWTrdCf+dhtuHnvHBZakQnVZKpKfz4216bCmePCALM+sKFRxUIrxnxWF60V6aJ/b7j2Fsw3I1Mx+WHRYCz2FhN5tMZyJ8Y377TGhj2OShA4Yd58PDOZ4XjkeMu99tl4pz2qKhCGI0n1tWVjqbsK6s//t0vPxlmbp/Dtuw+bbUoPiTPHz9oybY5JVlpeKfj7f/LcJ+MDv/kTjjHS4bj5izsJQ1UVFCyNsp4isr2AiTiMEjQqYRZllx+O244S/NaHvo9/+MY9AGzSaq86c8NCL4/jNdC9NwDgRdAhtY8B8DoAby+zcSK6koieTEQXE9F12XvXEtHZRFQnotOJ6Oey9x8ioqvFd79ERI8noscS0ZvF+/cR0eVEdAERvZCIulMyh4QNjSqU0pVLTQIg53GQvInyPA5XjJbd/KTHweJ21Y+qykkAZFH2kfkWFsV3gfxwXJ0QpHl89mh8MY+fL/8YfMPHMB5HoHD6Rl1h7YEjSyZO3yQAekJ13qTeSewkrnUBqXHoH3KoqsQYDD4eXmk3RUXgvHHLffM+O4aqCroCEdjj4ImR+eONmbGWndZk+Q/N+dtj4d/8fT4/z7xgO5KUsGvPMUdnkiJyJGhEjqoyCafZ7aI9jsjxOG7ZN4+P3fQgvnf/UZMDIBcufG1MAmDOJPOFHz2M3/zg93F4sW0+w+1qo0T247Aex1wzwuHFtjOxsyfXqLpUlfE4smscJakJcKiGCudsnUI1VPi1y87FlY/bjhvvPWJLyohry4borM36XuT2xasvOaKfm+2zNZyzddpZ2HAbgn5UVTtKTOXnXI0jm1NMn5lQRFXFKerVwAmWANxAhlac4qb7j+KWffMA7D0526iU0q1Wg16GIyYivtueC+AjRHSEiL4GYGako1onCAOFjY0q5peFOG6oKv2ZJIsAyvsuYB9KfnCVslEwC+3ITP78ALLoJqvFMm57WN8gBxdamGtGjuHI9zjIJJ0VVVLlh8H3mkzJbm+lxA8KU1WAfnh9qsqnK/Lq8XTiRHgcojqu8Dj81rE+VcUT5nKWb8OHUfTcxNmKFtATaBAohEE3VeV7HMZwZF4o0wLW47AGTWYK8yTNDzVv71GbGmb8NrQ1sFpAZBNFOSE0Sckx3ICeJBbbsekg2I5swMBSOzZlQGRPc76mfub4f9x2AFe//duIk9SEzbak0ql6qwAAIABJREFUgQyUqWggOwAC2nA8cHgJRO6iga+pn8fB9wlTcB32OBKdI/GUMzfhlj/9OTz17E34qcdtx0I7xo/36/tf5shwgMmZm6fMMQ1D4+hkIj1TwWFgE2JtHkcfqqqHOE5kqxHwOWZxnLPJ2eNYlkUOxXbmljtYbNsyRrLawVp6HKlS6gylFJdS/5r439RIR7WOsHm6irk+VBWvSiX4PalxbGhUMFurOB4HT0Q8UfPvvCKDXPsqJeC+Q4suVZWNS4rjcZZgxDHnQHcpgqKSI6ZJUB+qyuy/xuK4fi0fbin08rgA1xDIEEpuHZtPVVlxHLAaB6/Q7So633KkwuMgQiFX7Wscy0bjyJKy4nyPg1flfpFJn6Lh8bNAyu/J6COebNjjSFK3xA3AHkdiDEMrTswKVk8qer/zosS/WQR5We7fuusgbnv4OJY6SVfJGI4IYwMvo6oAbcD43pPUFxs6vi7WgAbmONKMruFj5qq//KxddNZmALo0uRyvPkZ9XOdsmcbWmRoevW16aFFVVRktF1r6x8kc924zIhLen9U4/AAEaVxboiJCnNpWzI1qmBUv7Ra9Z2oh9s/poo68EOUEwNl6dU3zOP4XgJ3QdNXnWGNQSv00gPtGOqp1BN2QKRKRKG7JEVm3SKLb44ixsVHFbKNioqk0VcWGI1uxBtbz8G/K2x9ewLYsKejwYsfxOKoswglRupNRCrw6AnKiqgz1UeRxFFBVSoem8veYwvOr4/I+45xVqCOOiwczIZiIG4eqiqzG4XscTFX1S2qL09TRk7hhVJ7HUe/hcfBrDseWhl+WpGcD5lM0fL1lQcRKqMwE2xKlaSrsFRF1BTPM1itYbAmqKrJRRotFHgfTRxxpl431noOL2Zhs1FTs3edsFGUODKA1DkZKrsEBRAJgNh7OGldK6yc8YXIBQYkzNzdQDRXuP6wjq+SkuCjymf7zj5+DX774zKFoHHGSOvSxNEZ6sRiYayLxH7cdwE++5Tqti4oEQH8hI59TNvTsccgq0NO1fMMx26iYasCLLfsMALpo6ppljhPRF6BbxT4p68PB2Ang10Y6qnUEbkDTRVV5oZE+bK+BTBxvRdg4VTVRMIAOxzUaBxsOsQKXHseRxTYeOd7CTz/B5qTMiodVjy10XnMCIIcOBiqn5EiB4agUehyWKgkDhR2zujmNL447UVWJS1UZjUOI41VBVRERQoWuqKp2khNVZaiq2DEcvcJxOQxVHke3x5H21Dhajjhuj7sSBE4yI0+w/FCHgetZRklqaK2qiKpaFmI/R1XJlryB43HE5jy0osSsWJfasaGcZFSf306Y93/vIZ0DwhnN+rOWnmHPKk5dY8fjkPBrlfnieEUskPhcLndicHc9iUoY4Jwt09iT1QiTRoGpKi58yaVwVutxdDwDJqnUKNEaR6i6E/semmuiGSU4stg21XF5IUEe7cqQDcriNDXXr14JMVWruFQVexz1Ch7KPA6uEdaMNPVbr66equuHnuG4RBQT0THvvSUiWhzpqNYRNk/XzOoBsCtrP4vXh+2VYDWOjY2K4aQBLZiz4bCir6U85MVnYfw5TzzdvCepKsA+nAzmSiWNUpQAmNcBUP6fwbaMJ67TM7qKV8pMxfjiuF+yQo+PjKHkh52yzwbKLbMBaLHRp6oMRRPpVXBR3xG5b9/jyOur0I4SN6pK8Md6f0LjUFbjqGSh1X7Pd5/b5+OOsiQwAE7iWzNKhGhuu835hn62oanPuWVLj7CRK/I4ZKY6L1DmmxEOZRFYHOKqz1dqzmcYKhO0YUKFg3zDwdefe7fwdfMNaKBsi+JmJ0EnIaObSJy3fQb3H84xHNnx1UJ771eGoHH4VFW3x8H3p/s9Pu7jLV1iSBc57H6WHKqKPY5MHDdieTXAdNWNquJjn61XzHU0GkfH3rProeTIKY3NWe9wPwHQ8Tjyoqo8jWO+aT2OhVaMdpxk8ecuVSUnIDmZcUXWSx+9xUwukqoCrADJiLLifFVhOPwJlV/nlRwBelNVAPCojdrjMOJ4XlRVSvDFcqA7HJf/lxJsfoUYbifH45ATIoBSHoesCxYodBkowHocJo/DdAR0xfHcqKrAxu37ZbL5c0bjSGySpiOOdxIn5JWTIRPh8QFZ/kQnMfk7rSgxbUcXW7FZjbpUlTVUYTbJ3HvIrgUlVWVCUFMbgsoeh85fseG4ErJKQKiUoXhl5jgfB0+U3ICsFnY/T+dtm8GeI8tOIikfI+AlYQ4pqqrqbdN6kakJkfb3wwaXa9TVMo9DbzPf42hG0uMgY0gb1bArqkovquzzBmjDkWZUn44qW2OPYwJLVUneEbBhrBy15MPXOBZaWuPY2KhisR131Zpiw8GTWuCtGngVOVMPcUa2yt/YyKeqZGRTnCVUAa67zfBFTkZFcPASPMHyhMGRVVMeVSXnYV8c50mpLcVxfrhSyqKqsuQwSVXlahxufaR+SW2J53EEQbfHkaQ64sUvOSKppKb0OAKbx8F1tNjgRr7HoTyNIyahF1iNY7lje3pUxBjZ47N5HBVzLvm3oao6vaOqjHeUEu49aA1HO7aGQzbH4hyVOElNZBFjQxFVRbYiMACT6W89DisOdxI9dl48SZy/fRrNKMHBBTfBkFfbNU+PWH0eh0tVOXkc2TMfht0UJxsEjm6sZfcDH5/dvqCqpMaRWI+jUdVRVTKxL8pySHxaeqkTo5mV/imqjTVMlDIcSqmzlFJXKKWexT8jHdU6wubpKlKyN4JfZZYnDh988/NEe7wZYeNUJfM4om7DUcmLqhKGQ0RacDRTEVXFBoULwPFqrLfG4d4KfuY7g5+T0KOq/JIj/j7yNQ5LPUlqjOmf3JIjVS+qqsDj6GU4KqHN9JYRSwzb21yUHIkSTSXVuHmSXZE7HkeguiJwgO4EwDDzqOTq3tl+7HoiQaBpQiuO67H6FJHWODiqKjEax0IrNudE7o/rat3jeBzUFVUVp6I8S+ZBVsU943scpud8dn6q2TlfzqGq5MS40IpzoxQfvU1nANx/eMlZzLDh6OVxPP8fvoP3fXuweJ5O5lHJbbJOEbM4rro9+HZ2btkDrFXC3ECTtqdxBFmulW52ZaOsZH8WIEvEDJVZwMrzwKV/hqHx9EOZ1rFvhRbDb4MtUkjo0U/jZAJnjz8y39Irruwm6JcAKDWOJCUstLXHESiFxZYVM7lQXc2jqnx3U6/E9AqRDYc/afBqdeNUFceWI8QJOZNtRSQYMfzwTgYfk79ys+K4fs0eR6NqjZOPlAgkNuNoHGwoxcPFq1Sl3EY5Mu8jbxUnx1HkqstyETHxpO9Gx9hwSBvG3Oxow8Eal00AzGg1WHG8IqrjRiaqyk0A5GOQ2f2VUAlPJEUoFhFhFs3UJY57Xqf0OBa9arkLLd3X3Nc4tMdhiyNqcdzVOGT0GJcckRM8j2PLtL73LNVl61LVQtvul+83bTjsNTzeinDO1mn4OH+7Nhx7jixhexaQoY+x2+OQhhvQXRvvfGS2a5u9EAtaFLALq5QgaLruKEU2mKw51SoBVDZrRgnh07t0H7zH7LCpcK2s9wtP+K3YXSQudxI8cHgJ522fQZTo+9f3OBZbmcfB1YHXAVX1fABPIKKrieiXsp9fHumo1hG4Qu437jyIC8/a1NXuMqECjUOs2Pnm3pRpHEsdG3dvEgBZHBcrcHcyS41hOHPTlPNdxpTxOLQxYtefJ7uBxPEcQY+PBxDi+EZXHJfbqYptyPBAmznu5nHwZ4mynuZ+OK70OCr5t24ZjyPMOuoB+VRV2/E4rKhbqwRoZMUkpTgeBjLMVjlUgfU43HBcQC8WOklqjIvuw6InJMegBDYZ0hfH/ZW+G1WVGGoIsHRV7OwvMBrH9ll9r8vMcCsIp0Zr4Wg9KWLPZqXVedKPxff4kGvC87Xn362xdLwZm/tG4szNUyYk11B7ogYUMwH6fAVd11Ou2svAp6psf/cUKdlINyLkRkuxx1GvBM4i7OPfexAf/96DXeG43N8mTlNDXTWqAZ570ZkIFPCJnbrLBOfT8DNt2va2Y1OluiKo0lGhjOG4D0C176dOUnChw+VOghf8l7O6qt6WyePgUMiNU1Uz2T+SVTH18zhqgsqQF78ZJUZg7EdVmfaZbDiEcO0nFZpcAV/jMCVTfI9D/2aN4/Gnb8C2mRouOE2v6AJvYuR9yAkvzl5rvSHM9m8FRE3/2AJ4gH4482pV+X+XoqoCm9di8jgKPA6pcdTCwDRq8qOqnATA0A3dBICm6LXNqGbFEDlZkjOVq2Ggu8OJc8Z6j5/HIbWFQHkeR5ZVzBOX6UcvcjBYD3jw6DIef/oGANwcyW0mFac2eky30iUn6ogrubJnYKgusbCqV7oXF6FSDm3TjJLcqKowUDhnqw7J5XMqg0F8jUNqTElKK2p2JY08j5kndVmNQd5reR6HzdnR0Wp+QU7uM6OpWUFLV7THcdUTT8Ondu4zJYS4jD1gzzd7HFyraz14HMsAblZKvVsp9Xf8M9JRrSMwVVUJFJ570RlmYjRUVbaC9WE9jtQ8sBsbFTOpPzSvY7B9cbwoqqodJaZ0+lVPPA0vvPRsPPY0t/KLoaoygxIxVeWEvLrjLOw5blZYvsbhUls7NtSx609+Fhedvdl5H7APs9Q4GhW9GpJtYHlsgF4N82SslB2vScCsdHscnBQpz0Fvj8M+9Eqhj8dhqSrOE5iqhk5UFa8W9XlzEzFNHkfHFYUBvWKOYjL1xMwEGwb6fRG5xLkmvaiqHRvqnsah8zjYKzQeh8jBqIYKR5Z0rxAu29ERHoftkUJGv0nSbo/jMTtm8eGXX47nXnRmdtySqgqy8+leaz4Ov8qrNEgSj946jb3Hlo3HJMPPfY1DNkQC3L7dZSB7xfA2ASvkh6IaQ16Y7VERVcXPNkdMtaKk2+MQARYc0MDe9a//xLk4vNjGdbcfMNQ4LyLPywzHUjvG0cUOtkzXuqi6UaCvxgHgc9nPKQnuyfHTj9+BbbN1Wwspuy5ptjr2IW8qdpOnazb2+uE57XF0lRzJyaQG9CqE3dMzN0/hr154cdc+pzyPI0pdjyOvOZStwFrkcfhRVfp3wbPtTYzdHke9GpqoJXncTjhuSqaGFBuqtijLALjVhLfM1PBQ5sFZjSPfVTcF6rrE8d4ax3InRjW0gQBNh6qyhqgaqCyZkakq9jhcUZjPD9NCfln9TpKY3vOArSKblznOOH1jQ3scsU0ADAOFx+6Ywf65pvU4hJcZBspU0D09C63uxDIc11JufJzLnW6NA9DPyA1ZFVu+z3WjK/1/GwjhRrV1GY4cjwMAZhtV7DliqapCwyEKhLaN9zU4VZXncbBWxT3vAVcf66KqwgCdMDX/45pcruFgjSNwxsrP+399wg5UAoUf7Zs3wR2c9X/+Nk0NLrRiHFpsm8XDqKmqvoaDiD480hGsc2ybreMXLnwUXv5T5wOQGeG9PQ4umSELwlVD2+6TPQ5+8Gs+VaW6JzNfEPNh8zsyjyP2PA5PbNbHoX8XaRy+OM5jUirfcgQ5HkdKVuPgejw2cskNx42SLI/Do6ranociY+y3Co+jDFUVZHQSH3eolGNopMdhqwrbCU17HG5UFf+Pk+Rib8XuJwACQuPwks3YE4lCa1C4EKOvMXFwBQDsmK1j/1wTzY5d4c4td3Dm5u0AjnV5HNVAr4YPGcPRyK6B7aontQ4uwZIXVSXHLvfB/dGBbu9SH4cbYQQgl6oCgKmq7k3B51bqO77GMQyPw80ct5QlH0uS2nuA4RuOWiVALZYeRzdV1coi9ipikQJALPh0+G070lUGKkFgPA6ONts310SUEE7bUMdDc8219ziynhz/B8CTAZiqdkT0mBGOa90gDBTe+dJLzWtDVaV2RZUXVQXYG1jmSsyG+pTvPbqM2XrFJoR55cLZ3SYiqCzyZKqP4eAV2Gw9RKCQlS9IzU2mqap8j6OrVhWXHCnI4yjjcfAxxWLCa3gehy+OJxmPz7y+TBYECjyO6fKGg2PweeJlfaId9/Y45FjrVdtcJ03hlBzhydjmQaTONn2PI85qVcnJkhtBye6StuSIe5653/eGegXT9UrmcdhrnJKtHGsNh+txHF3ShuO0Dfrxdj0OawCnlNVv/DwOc0yiZL0+PzZcvTRVlefCQxtsWT5fehxFGgcbjqUVaBxurSoOkshopDBAmwNlZCi3Cce1GodchHUyDUoaS11CPTTnJS/EuFEN0I4TcF+UhkdVcVb9jg11HFhorQuN44MA3gkgBvBsAB8B8E+jHNR6hhHEjMaR3zqWP8t8MKANyRmbGggDhQeOLHcVKQQgVsJ2pQuwON77ck1lYmEja5/ZSVJTOoPH4xsC2Qsi7zj9G5C98rx8Df09+zdP7mlKjkahxdV8wxElqTGWQaDM/qQXALiTy6AeR5jRYHycQeB6YjbZM3TOi/Q4mALhsvpW43Bbx/L5NiGwYpVerShTq8qhqoQ4zscZZBqHn8fBQumm6SrqlQDtKDFjY2ybqaEWBoKqknkcgbnHmKrKK3JoNI6AixymuZQSLziMpyLE8XyPo7tsRxFV1cgK/tl+4PniuKtx2EABH77WICErLvA2Adfj4PceOLKE91x/b7Y/S0vx56Q43s6qIcvChZ04NfoRoI1cLQycQJN6JTRldcJAYdtMDYECztk6hUY1wP2HdR7Ojg31rHHUGlNVAKaI6DqllCKiPQD+VCm1C7p67imHwKOqUuperTO4ZoyMmz9z8xSue91P4zM/2GeS5wBR5NDPa0hThIEWY0/bUEcvsEdiehIkbiSSn1AHiP4avsch9u98vqDVLEPlUFXS46hXQ6dopDWYQuOgLNpJWQPtu+9KKcMzu4ajdx5HXlRVqNzaVrbInOtx8Aqwl8ZRyWgwS1W550/OiWzcozh1DGEtDBDFqdEVeJx5VBWgy2hvmqqiUQ3QilPUsrBM09inXsHGqarJHZJhvtIwMlXVjlORACg1jsAuhkQOjgQnskqPhc8134dynzlsV64nA2QGW+SYTNc5sEQ518nVONjjc+u2AcArPrITZ22ewl/86kVd+4o9wxjmGA5+77M378cHv/MAfv3yc7toN013xtk5sRStXypHRuYttbsXifUKexx6kXH1U8/AEx61AadtaGC2XsX9h6zHwdFZ0tsbNsoYjrZSKgBwt1LqdwHsBzBYNs1JBL4OPP/29DgyMZofVL4Rz9s+g9f9P09wPlvLHjjZ0AmwbnAZjYNddy470I4TUzoDcFdijL4eh9+/w5QcyR+DE1UlxHGeQBuexuEnPrKR8TWOw4uaM+ZcAyATHuMU07Uwe7AsnecbSHm8bh6HW04CcD0O3xMA4EzK3HSqYgx+1n8hTbPOdu44pB5mEgA92qdWCbIaVuRUkdW927sN94ZGBZumqqhXtCfUDBW2b6hh71Gto03XK9g0VenSOEJhOKqhMpRfJBY7bskRew81o6Qrj0ieIzYcsrJCnseRV1k6TzsB7MKIqy7MZPd7zfNQwhyNA9BlOTaKEPa7DiwYw+KjIwpwAtaY8QKmXrH3EOdp5Xkw0uOQiZW+4ZDX4shS2xknb4dLwYSBXjQ96YyNAPT1506NOzbUjUFPiExi6rBRhqp6DYBpAK8GcCmAlwK4ZiSjGQN05XEk+XkcQBZSK4Thos8B3StvmxBnV0x+mQEfbFimajoJyBaAkxqHZwjSfA9ChhBKEHWveCXyNIFUtI7lqCpf7DbHm6RmstG5C3pbR7IHQ2YNV4UHII2m3k4vqsoteSENFFDscfAD2agGth+H8TjscVRD26+9y+PwDGuUaL3HN1A2Zl9oHGl+pv+Vj9uOZ16w3XgcrSjFthl7nmbroam5BnDJcGVyRgCtE5mWuo7HYSmnShCY5LL5ZmRC1SV8qipOpMfRbTjygiwKPY4aG47IeV33FlSykRZTVQCwLCKriAhHFjtGxPbR1ciJG3pJjyMbOxuydmTzXxi1LMlSf9fSZXPL3YaD76GDC21sEwskPsZ2nBqPWYJ1rnolwIZ6xWxnlBVyy0RVfR8AlFIpEf1/IxvJmMBQVSQfqD4ah9f0Jg+WqnIfLn5wuYBZL/Dk2aiGqIWB4XWlxlHWcBRXx9W/iwxHkGM4HKqqTx6HCcdVXB2XPQ5tOOQDVRMewHStgmPLUU+Ng0u2h4F96GWpB4bRU6qhkxVcq9jzazsA6nPBkwz3HOfj7vY4XEN0vKULUcp7o8Z5HGKS4LIoedTim553IQDgHdfdjSQlLLVjxzObqWmPhL02ScPweLZM17KAhEzE9TQOW+Qw8ziWO05Qgjkmj6rq63HkPDuFGofwOCrZqpvPl4SskCA9ikUviz5OybTcvefgAgBlElmLqCoTjis0CNm61S+BU6sEhk1YEoZrvukaLOlxHDrexrnnuWVXGpl+Reg2rByZuWNDHUrobXrR0nvOWCn6ehxKqWcopW4DcEf2+mKl1D+OZDRjAEsh2QeqiEfkcg6yPHYR/IfA74dRhqrakWkg22e1u8orIVk1t2zJkSJx3E5c+WOQK2ETyioiy7qiqryggIg1jsAtcnhksYN6JXDyFvicTdVCYxyZG87rBc2Hwm0/AZi+H3l5HMUeR2jadHKegtU4ArHqTrsMrx9V1cm0DHlvVCuZOJ6kZrVq+nH00JjqQt+RntlMXRsOrmAgDRIf05aZqjmnjjgualZxdnOUeRxcVUHCp6rkCtkvaKmPS4zf60njgxdOx5sRKqFCPXSvuR2D0DgkVSUMBxvRueUOiAhvuHY33nDtLeb/kU9V5Wkc7HG0i6kqXXIk6Nr/fDPq7gsT2O1J3U4fY6gLX4p7gjGbhWTz828XfaPzOMpQVW8D8HMAjgAAEf0IwLNGNqJ1Dj+qKs91lJ9NUlccL0LNo6p4YuMVeDtO+xqOJ52xEV973bPwtHM3a6rKC+vza/jw9vl/EjYcd1CPQxyTkzluNY4oJdO8qDscVxc5VErTGLy/Q4ttbJ+t54rv7HHw30C+OC4pQ+lxMKXIsBFcgbPqrIv9cciraeQU2olYPrg9PY5skvapKhbHOdlLn9esH0eOOM6Q94dvOKZEC1Jdzt4t/8HeQzXMuPTY8zgSvUCqZtFZKSGfqhL5OACQEHI8Dnus8jhslYb+GodulVzkcQTOgoux1JGGg5tW6QTdR4638MhxnURKlNGH0jMKcwyH0ThsM7EujSO04bjS45lbjpw6VjJzHHCrIQAwEXPynmCw1sTdOIuasA0TZQwHiGiv99ZgaZgnEfja8jXhGj55kE1vgMGoKnnxpVjbDxectsFw15aqst352nGCaz7wPXz3Hp3haycib+wFRQ7TfhqHeJ9XhJKuq1eD/DwOUeKEsqSxQFkP5/Bix6FfADthcN8CeaxJTjiiNZJuRnZXyZGs06BSqstD0PtwNY4gkLWqAuE92ZU7wy9yqHuOuxFKNQ7TFdnLph9HL4+jIClyph5m4ZyJOT4TMMEaR/b5ujBm8pwxJcsROwDyqSovcVRWVrBam/0830dK2fIpcqUvYTUObThkXo2Eq3FIj8NOW2w4AJ2sd2SxYxIh+ZjzEgCbeeK4pKrE/tiz5fu0y+MQOR7SawWArTNuBGU9iyD0M9oBS1WdtpE9DvY8RxeSW8Zw7FVKXQGAlFJVpdQfALh9ZCNa5+AbXVJVfTUOpoN6eBym5IiXx6Ebu9iEtLKQVJWkCO49uIRv3XUIH/veg2b8YaC6REoZ5fTRGx/Al255WB93nwRAv2w4AKeqa72SUVUFUVVcnsTvx3Ek8zicY6zYiZxXo706AEpazk702Wre0zhM1q430QPscSSgTHMIlWgdG1q9Q9OUxR5HJaN9uC+8PG+dhD0OV58ymf59PI6pWmiijmbrFdQz4RzQ9Zb849sybYttcq6BPGdS42D0pKqEuN6dx9GtHVTDwGh0RbWqrMYRmSAEIF/j4HBURxwXHseRRasxHDjewmI7xnInwVI7tkUgK93jNB5HaHN8nKiqJO1qB83XUHo8c5nhkM+79CS2zrjntuGI4x5VZTyOhjPWtaaqfgfAqwCcBR2Ke0n2+pREV1RVWhxVpTUF+xD28ji4jj7fdFLg4tDPfuK4RFVEVUmNg7d1/V2HTO+LvPFLcfzDN+4xfQQMVVVwzEXiuNU43OgUX6SNU9k61q5uDy/mRJrwRF7rjqrKC8dNEms4pMYRKpeqaone5tLrqJoVbggibWD4/BnNQHoc/TQOs7p3+2wzVcWhlzzOlISHmHMrSY9jqhpitlGBUvrvRiVEJ06zFqMyRNtGVQH6msnkNKtxZMcZ9jYcJpFTeCq8mMjTONj+1cMA09Xs3i8Qx43G4XscFfe5kOGo0uNYbHdTVQCcXiSHFtomQdJt5JTdt6x/VQNzXEvZ+WpFCaKEnHMpxyPrZXWy7pfm/leuQc3zOFpRgihNuxagUhyX+1tTqoqIDhPRS4jodCI6jYheSkRHymxcKfUapdRupdStSqnXZu9tVUr9h1Lq7uz3loLvvjX77m6l1K+J9z+klLpfKXVz9nNJ2YMdBrqiqvoYDr1K7C+OT9cq+MR/fwZ+9Wlnm+8CegK0Hkd5w1EJbK8CE1UlVqkLrRi79hxzErT87wMwWa4cRdSXqsoxHKmg6/ghN/HwVffh4pVqoPSkkmYaz5HFTpfHITWOKT8cN+eh4WumEwD1e0HQ3QHQ15Pkqljuox2luuSIUuZhDgMl6JreGkfNiONeraqK9TiqgspIC/I4GDIstVENMFOvYKZWgVK2RIXfo+X/b+/boyyryjt/3znnPurVXV2Pbpp+0N3QKNB0Q9M8RBojohFwBExECTE6A7J0mYRMJg46ZmZlEk0gk4dLkxkHYxImS4NOMKOTRMQQ1CQqmVYBSfBBFBOwgaah39VVde/95o+9v3P22fe8u+69VdX7t1atuvfc89hnn332t7/v9z3kOYtpyzRxAt0LJHNyWznUbaoz4NA4AAAgAElEQVSSgk0xU1WXxmGS45FQborGkeOOe2RW1eyw688LfEN7Nb2qkshxALHqh/uOzHbxb2abpb5J3fe6noEEWIr2ZqcRstOe1HXMj5zfFFRd5HgQaRy2RmYLjuj+e2eqSnXHzUudzsw/n/U7EW0D8HYAFwGYA3AfEf0FgFsBPMDMdxDRewC8B8Dt1rHXANgJpd00AHyRiD7HzIf0Lu9m5j/LvLMeQVaq4qaZ5Y4rHEeUbiJdcADABadFMtTkOGQAlBEc5qAPbct6kK4ea+CFo3N48Dv7UisYhqm8OyrYS17AKK138nXz0qrLqvNQGMSlhqC8hCLsfK0JdFjVM2l1GJNZgqMmPv2Ric9G5HocvfRpFQDtNN1zRtvlOcxoslIC4wA16cl+c0b517B/YpwJGRxHXOOYa6lUJMO2V5XcQ5Kpylh5N2o+xhoBjjbiLtlS6EkmYVnMmOS4OcGZpio70nxVgsYh5wxNVUn1OIy2+0a/DddEcGRrHOoaXmiialiCw0xfM9tqh+PY5jgmRup44egcnjDqrT9/eBYbVg3r8+R4VVnNlDEtfJEtOOy0J3XD48qMHAcSyHGdq0oi+E10aRwDJsffAeAyAD8CsAfA162/PJwF4CFmPsbMLQBfAvAGANcCuFvvczdUhUEbZwP4MjO3mPkogEcBvLbANXuOKDuumkSZ0802EcfRSeQRMq9j2CmPz8c1hyIwX3AzchwAtq8fx4WbJvDgt59LrZku+893Ojg21zI0DvVbIa8qgxwPJx69TVZnQ9ZkIdxHqAkwh2YFmxwP3WO1qYrIuGaCqcqMkpf2i4eVHceRrHGo/0NGFcCokFNkz5aVv7zoZlfZ7sri+mpHjs+3OUxoJ/1hahxJz8wkiZuBrzSOhtjbo1rpx+c7oZAJAwCNyc6c4OKmqrgdPsmryrwvIKpXIudWfWRqHNE9h8IsZTyagsM0VXVrHJE5ebbVQTNQ/MlRy1R1ui7fagqOfUdmjXQ43QJOTL31wOt6ByKNQ/elYYYlStA4Ai+mhcU0Dtssq8fErJWeBgAu2zqFGy/aiJeeMhZr66A4jrUA7oJyxX0LVBXAzzDz3QVTrT8GYDcRTRLRMICrAWwAsIaZ9+p9ngGwJuHYRwC8loiGiWgKKrniBuP3DxDRo0T0u0SUncBpgSETo5lwLjM7rl5R5mkbNmKDv4KpKu5GGl/pnblmFBectgrffe4w5tqdTI1pvqUEV6hxlPCqCjUOZmWbNV6OQ8fnMVz3wwkw1DhCU5V62ToM7DuszArTXRpHZJ565UtW46aLN2puJDkAULQQ0wtK4jjsXFXmKtYmYc264yJ4wySHXqRxqKR0nVgyPtM+XfN1+pWuyHFlZmwleFUV1TiG6j7OXbcS205dqX4zNQ6DHI/ccbV5xadYmdU4OR6N9bFGkMpF1Hwv4jg6psaRYaryI8GRnuQw3ke1UONI5jikcFJTu2sfNe5r/5E5nDo+hOG6j3998RgAZRrddzgSHHYdc0BpHERxzzyBROaPh6aquKDLM1WZC5QxqySwPK9jc62u665Z0cRvvOHccJ+0rA8LiVRTleYxPgLgI0S0HsCbAfwTEd3OzH+Sd2JmfpyI7gRwP4CjAB6G5cbLzExEXXfHzPcT0YUAvgJgH4CvGse+F0rg1KEE2+0AftU+BxHdCmUWw8aNG/OaWxhhzXHjJc7UOHS+orSXIfU6hrpZlRwXNGrxF3brmlHsOzwLZuURkpoyxRjsonGEKUdSbse24QNa42hz7GU7NNOKlf4UxwExVXmk+pqZsf+oRI1bgkPbmWu+h0vPmMKlZ0yFbciK4zDb4XvoIsfNnFfmPYVeXHVZvbdDjijKVUXhRDbbaqPVYYw2gnAVH/PS0uebmWtbSQ4jz7N4HEd6pD9gaRw1D++9+izju25zS5kdG6E7rhYchsZx1OI4YhH3+jmtTDFTqfZToldVcgCg1kZ8r0v77D6vp7MJIB45nsNxNPTK3tY4JkcaGB+q4dhcGyN1H8ONQAuO7timyB23rcsHU5fwlgBL0Thiiw+PuopJmaYqtaiKHBVs64Sc69hcO3cRGmkcA3THJaKdUPmqfhrA51DMTAUAYOaPMfMFzHw5gBcBfBfAs0S0Vp97LYDnUo79ADOfx8yvBkD6WDDzXlaYhUr5flHK8Xcx8y5m3jU9PV20ybkwNYG0BIECydLZ7nQyifHEY43BH5mqyggO01QVt2dvXT0WmjAOzsynCo6aTzg8q16G2bBwkfotTeMgrSkA3RxH4HsxjUNy7ADRSly8YHwvqsfx/OFkU1U98BKFaVJqFSCe2deux9HuMPY8+QIe/M5ziRyH6o9ujcN2Uw28qB70rI4KH4nVBY9zHIBy0zRX75K2Y2a+HU0sFLdbJy1WzPHRtFbhkcYRrwo5OVLH+HAtXOHa5LgU1pJ+iNx3u4lxQWCS49ydq8rsA5M4H7LGqQ0p2yvXkP7L5jhUrYuRRhAKjmNzyvV2aqyOcX0fk6MNTI82YhpH3FTlhcfKuLafwcEUchxQi46j1uKh7ntRxU+D47CJcXWPcX4nC/0IAMwix38VwDVQMRv3AHiv5ioKg4hWM/NzRLQRit+4BMBmqCSJd+j/n0k4zgcwzsz7iWg7gO1QmguIaC0z7yUlkq+DMon1DbISMKN4kyoAqu2aHE/wvc5DpHFUd8cVmC8sEXD69Cj+WXuSHMgQHL4XxYKIZ1deHIdcp82RliU8j81xmBqHDHYxVUk9jg4rDxiPuiery7dOdRGU6lzdEfJAnOMITVUehfzBh//mCfzg+aOo+RSbhANrghKTyux8R1crjDgONdlEGslcuxMrOJRU3+P4fNyrSjQ1c3UpP7cM5wEb8cI/luAIojaZXlU/87JNuPa8deG4rus4DkG7ExH8pjtukitudF9kZdVN5zjkNuKmqvTBNVT3cVSv+uV+bcER5zjUImCkHoRxFPvDTMuN8D6mRusYa9biHEdKyhGpsGkvGCNTVbfGEXgeDsxHpqznj8wpd1xjHEq/2G7n3efKXoTKO/bY0wfxy//nMfz69efios0TmceURVaSw18G8AMAO/Tfr+vBRVBWpu4k9t24l4gmAcwDeBczHyCiOwB8iohuBvBDADcAABHtAvAOZr4Fik/5W329QwB+2hBaHyeiad2Oh6FI/L7BdJON7M3J+waepFVP5xHSYNanqBIAKIPHJIxfceZ0+IKKJ8ahmflMjkY8RcwUG+q86ffjE6ENqbQXkeMxjmNmPqxMp65lcxzRZKk8YBpdK7yrzl2Lq85d23X9NI6jZcRxxLyqtGZzcGYeT714DNNjja6XHohHjgPJXlWmNiNcQYzjSBAc9mfThGWnoBHtM2lyjQUAWoIjIuw7mvyPPMTM4+yo7VYnXnVQ7jONGJd7iTSOqO1R5Hh0jVCoFDBVmfcYxDgOS+MwOI7j2lQ10vBDF1zT2WKVoXGsHKrhe88eToy7knbOtjqYStE4Ds2od2WsqSp71mPPN9pXEk6acRzmYsaO4TDv27y/NMh59h2ZxRPPHUktMXAiyBIcm0/05My8O2HbfgCvSti+B8At+vNxKM+qpHNecaLtOhGYXlWh4EgZ6L5HETle0lRlrppCwVEvrnHUjVWyTPKvOecUvOacUwBELnxpyeoANUAPSyrulqovIWMwacUr8DwA7WiCbnPkchpyHMdbOKPRPaHOtaMSqzIB7zs822WmyoIqb5qeckQ8qcLraM3m8HGVg+nZQ7OxmAhzcgOiSdn0qopWv9FxYpoQk5xHVqErM6VFghCZa3fCSbYrcjkhLYc5gSYVApI2tTqcavZsWGPZzDOmJrfIDp8GSZQox8vQt/OSAfE4juHQqypdcISmKi/dq8o08yqNQ/EXR19QJPi+w1GK/pWhxqEER5pXVRI3lcZx1PW9xExVRr+qvjsaM1WZDha2Ky4Qf7ZpFo7oWuo8B46q9ow2sqb5asgix3+44FdbBpBnFtc40k09bW2qKkuOB0mCIyguOIJwdZx8jMlxpE3KgUc4OBNZJ2dbnVyvKgCxSTkKgoxrHEdmW2FKDABhOmjTVCWT5YGZeazIWOF2Xd+LB/QJklKOSOQ4gNi9mtpdyHFYcRzH56PaIVeduxZDdR/TY41wAhGuQCrV2SZB4TKA+CSVtFKVNh6TmtcJY0EEc4e5axUubRZzSpr2ao9T5RYcPfNaAVOVSqUSmarsmuPm2An5MN8Ln7HJfdkQc1Y9ICOOI76/9LNwHKONAKP1iON46AcvoO572DI9GvIRU6OK75hvR+7fSSlHpK32NiByx5UszjGt1Xi+cp+mqSqX4zCeV1pKFrutB3Tq9qSCWyeKhT/jModZOraVkJrAhK9t7aY/flGYXlXH5zvwKNv2a8Mmcm3IYBJvmbRzSNEcQNn0ZRLJCkmJqutFZqD5TtyrCkCM4wDUPcuE4xsk+8GZeZw2Ea9PkAUxEdrohO7TnuEGjJAnMWskxMlILTiMtOqAYaoiwsqhGq49b50+Vp3wiFWprktwpJiqkias0FSl3UHTxkIz8HR24fjvIiikgFDagsJevSuOg8N+kPbkmarmEsjxJI0j+o1w5Vlr8EdvuxCnTY6knrtpaByhm3QKOS5eVZMjnibHVX6x+x57Bru3TmG0EWBcR79PjTZCz7IfHVBZcmsxr6puU6IpAM2qkI3Ax3uvPgvrV0Wm2LrR1kj4mbmqCMN1lSLmFKOktMAcj1k574BoLMmz7qvG4ZCM0B2X47bfJJiR42n7pF7HGPxSxKlMAGHocZKyshyJmYnS22ASpSqxn/qcVctY2i7ZQUV4iuts1Abb/96zOI6IDxktsWryKMUd1+Q4KLoPuRczPYi5IhfBGkWOC6ndjtnwBcpdM0pqJwKyK3W9n7wirRufzVxGgDJVmeZHG3bxqeh+VF/LKjRtQWG2aajma44j6jeZlMdzTFWyupccV+q+9L3EAgBFKCvT0ytfujr1vNImQPXX1GgDW6ZHwsA3gclxiKlqpOHj6FwL33r6IJ4+MIPbrtyq70MJwMnRelg18UcHVMldUyM025yUOmVipI6n9XH1wMPrd5ya2KZG4IUR8mbKdY8IEyN13PvOS8PYGxPNmMaRbb0INQ4RHD3QOHLtJ0R0W5FtJwtkkuh08kvChpHjVoBXEZgudUWKONnI0zhGY4IjuW2229/sfGSqyuI45DdPm58krXpg5fdJ0jhmjchxOc+h4/OlVk2BHw/oE5gch53k0EZM4/CiyQ2I4gkk35Z9PGnOQ/z2pe32MDFNUrHPGRrHsfl2oplK0Ay8xLESmqr0ZJK2oDCvPVT3Yxl+lTuucBx55Lg6pmOaqhIyDnuWUMnDkDHpDtV9/M1/+DFcvGUytk+c4xByPAAz8OlvPA3fI7z6LBV3LGahqdFGmLJj78GZ8D6icyaZqqJrrjKy2Sbdi0nkmxqHXbht58ZVKfxVN+eWBhEsB2bmtPdZubmjCIo8rbcmbHvbArdjSUEI36yEc4BoHB0dw1BW45DBr9xxywqOIEfjMIvIpJnRbFvq8VbbSDmSfm1ZDcfI8YRcRyMW2W8mZvQMU9Xx+U6pVVNeAGCMHLdyBNUtrUL2N3+TpIGS+yipLxqBHxb4GW5EsQcm0iammNnKEHCA8rSy+YvYdS0vKUEzKGiqMsbpkK7WaHIcYp9PMqeY9xKS40YuN5kkTROP3HZaDQ4bw1aOrSTYHEejFnkR/vFXnsQlWyZCs9RlW6fwa9eegws3TYSCI8lUZY4ReafM+zA9oZIm/lqotUYJOWMaR44wSONLkiDa0YvHymnqZZAVx3EjgJ8CsJmIPmv8NAbghZ60ZolA2e3Ty66G+2mSdt6qKV0EocbRVnbaMq64QHd6DBtEhNFmgAPH5jPJfROz850wwjrTHdeL7sH3KCwda5LSQEQaCwKfMNdqh+cwX0w7BUMW7NxTgojjsOI4jOucc+oKfPNfDsRWaXatFEBNqqJxJAfjeaFgEXdc26Eg5rWTon10e1W1UhcDQLdrqnl+36PIVFWA4xiu+2h1Il4r8AkXblqFz/7sy3FOgjklvK/AixWCCtOqWznTzPsqqnE0C3hema7sUpTrqm2n4MWjcwh8D685J8py1Ah8vOVlmwAAK5oB6r6HHx1MMFUleMOZbZgwNLBEwRFEiw9JHx/nOLLvuxFz1sjxqtJ9OtfqYGxlnwUHVLqPvQCmAPy2sf0wVNLBkxZEKvWGaTNPgpC07QpeVWa8SBVTlbycWZPMSF0LjjSNw2qzFC8CsjWO0FSlV/aS4dcsq6qub2scXixXldmuMqaqVI0joR6HrXHsWD+Ob/7LgUSOw5zcTI0jMf1H4Ie1qIdTkveZq+yYJ1XC6tKPaRzpY6FR81OfTTPwQs+xZoqAiQnHuh8vROYpbmX7+vHU68u9tAxTVRGOI2myTUJoqsrY3zTzKo3Dw+RoAz/3qq2Z5yYiTI81Qq7CnKAlB1qHDXLck+PizgJJbQsMnkySZMbrceRpHMlBpEmo+t6UQWrvM/MPmfmLzPwyAN+G0jTGADxVNoJ8uUG4C3MFm7yfV5kcNwe/kONlEBYdyphkxLMqS2MyoepPZAtLIFqBCzneMfrAfBmTNA6xjXtEsQlwtFncHTeP4wj8yGTme3GN4eLNE/AIWD0WmWJCU0ssMtsLye+0KO4jOl3LSIo7bjyOo3tFa15b2jgzn22q2rBqKEwLbqNZ83HwWHGNY6jmx9xx8yY3gYocT0qr3j1JmrmqimAogSexEVavbAvHUfzdmRozTE5Wm+S8Njk+XPNjMVbJHEc0hobqpsYh4zD7/mPu4TmmKvMd65XgyD0rEb0RwG8B+CJUtPaHiWhg9TAWAyQpXlbeICDKVWWXBi10DcOr6vh8u4tIzoMd5ZyEtAktOofFccxHHEe2qSpOjre0nbxe82PXsjUO36OYxuFVXDmJwLbRMiZAz9KKBNvWrcTf3X4F1q7sFhy1Lo0jy1TlY/9RNUkrj7hsd9yYqSro3i4/H5trY/VYel/8zg3pdc2aNR8vasGRpomabRqu+zhwbD7XCcRGLFdVx6wrTvhPV78Uu7dGueNCjqOo4Kgn80UmTLMekG6+S4KZgdk0VYXnbXcLwKF6EDMJ55LjBsFf1FQVN1/mCA7jve1FDAdQzB33lwFcyMzPAYBO9/HXAE5awSGraFnVpmscatI8kbTqrTZjZr6DiZGyXlWyyks/TibjrJQjJuIBgOnXtqOyQ3LcMlXZwrDmRbbxLo6jDDlOSuA+c/A4xodrUdzFXJSe3hZughXNWlfm18BYLQqGTFNVIjkeJQsMfBWs1mWqynHBBQyNo6CpKsuE06h5YeR5Gvdlm+NanU7odl50DEshKkA0jui3Wy8/PbavCF17kk5DM5x0MzQOSR45K3EVJQSHoXHY47+L5BeNox45JNR8SlxIiKCTqHL5HFjnSr8nL1yE5QnZgZqqzH1EaGjsL3jcsoXpKQRkRI5L8FsFd9xYorb5dmlyvIjGEbqJpgk+o1YAEKXYALIjx80aGz5RWDq2y6vKiuNYMRSEmXDJNlWVccfVgZfX//e/xy9+6uFwu0RNrxyuhffk25pNgoCKJzBUaOaQ441aNHlKegx7v7iASNM44oLj2Fw7k7fKgiks8kxVvqfSlrdMt/OCnoE1X01wYTr2rLFS1VSVsX+UkFDKExdfdE0bWRRsjdu3FhByX8N1H0O1bh7MhLmQM72qRAAWMQMm1TNJvpZhquqRxlHkad1HRJ8norcR0dsA/CWAv+pJa5YIhCTLs/eHcRwV3HFDO63mOMqT4yI4qmsc4p0xrok/0x03a/DaKUfEVKW+m6aQ+KA+bXIkJJRNcxJQUuPwCLOtNvYePI6/+tYzeOj7+wEoweERMFoP4qYq3aSRup94X6Gpymh7s+aHhYGShGjT8soy3Z8FSQJC9revLe3K4ziyYNfrSIIZqyKFyMpzHMpUFbpuZ/Fh+qfC5Hi92zPLhoz9E9E4an53xc7Q4UQ/W7mvIUPjSLuPmuFgYRL8ZuR4HkytJgvxBVdxbrAMcnuUmd8N4H9CpTbfDuAuZr49+6jlDY8iF1Mg26sKUKRy9chxqZ9QbrKoB3FCMgl5HIes6iS61gwAzJpD5DfbHTewzDW2xrF5Kko14VmmqrJxHMIvAMD7//JxMKvstyuGajFh4VN0nbEUAj7wqMsEodxtW+H1bMRyC/meDhpM1zjSggHNmtThuSsGdJXROGp+FLzZzjHJ2gh0AGCoqRTgw0oHAGaMa9EMjlbhOLTgSHJ5teN5QnK8iOCQ97Hm4ZxTV+B129fivA3jBodVRuPIvh8ycooNkuMAgL+HSo3OAP6hJy1ZQvCtFypV49AP73irXT2OQ2scZcnxQhpHjldVvGjP0XjKkQrkuKrHkc5xxAQHUazK4FiJlZPvUWjyOn+jcq99+sAMDs7Mh26TtlYEKFNZEswU3gIVxyEcR4LgsHJdKXt2OseRTo7HNQ517oqmKkOYpZ3D9P7x/ejZ2W3IgphfxFSXZeKSVX1RU66ZqyoNMm5FsJcRtKbG0X3euFeVXGeoFuQKjsDQOMaaNfzeT+3U30uYqkTjKPAcJO9brwRH7tMiohughMVPQtXOeIiIfrInrVkikEJF0UosbbBE3EBZU5XnqcjpVlslOSztjmt4caRhtJGcfE9gF+0xkxwWERxCjndY8TxdAYCWV9WmSUtwGJN7qVokHoVmpB065mD/kTmVQl4LDjsRI5CucTQCv6v/TYGcFgAokDxMWRpHLWaqStI4jPZU5Th0m7NyXYWpyn2VubVlJDksKjjMAlVAjsYhgqNkHEfW+2QWXQLK1bGZHlXedEkCwNaO5HkqjSOb45Dz2c+uKDkOFOc4gMg0NjB3XADvg/OqisHzAGazjGfyfqJSzrbKk+OAegHEM2eoRC0OoDuTaxLE/pnnVTVUD1APPBxvRfEkWWPXnPDFQSDUOIxVm90nm6ai+APfiwTHaCMoleDRfLFOn1bC6IVjc6GpCjA0DoMcT1ud3XzZZrz67DWxbebzSHq0dsBWPYHjSMuOG+hFA7OhvcU0joqmKv3sssaETHy1QNUXb7fLaxyR4EgPkBSEHEdJd9ys/SOOo7zGMTVWj50jft4ipqrka9n8iKAMx9EoIDQFouU5r6pFBJkMzTTdSZDBYk4Apa7jRaVb7dV5HorFceRoHKEq7qEZeGGpVKJicRwmOS4ch/xmx3AAynS1ZoUyFZDBPZQd/DHBsXoUAPDCkTkcMkxVSUkOV6RoHBsmhvHyM6Zi28zI6yTty9T0ajrRnL2qVMJRfY6XU41MY3Y9DvvcZRBV/Us/PuI4lGmt1cnXrG1Im2db6c4DgjDJYUF33ImROoiy64F0cRwlNI7heoCRup/oHmxzHPLshutBLveSlgJe+qoIf9QITWT59yP79D1XlYH7iOjzAP5Uf38TgM/1pDVLBGZsApAevGNOYGVNVYCaLKQeRllTVZTGuXrkuKjRw/UAjZqP2VYbzEHmRCDtBiJyfK4Vlc+VFyWNs9k8NYJnD83qVCDxdhaFeT9nTGvBcXQOBwzBYUZky/5lrmNGCicKDqvU51suOS3RoaDme4kaqcRCmEI4PHdFjUOOy9I4TK8qf6FMVZkah5h/it3T+lXDuP8XLsfp+rkmIeI4yntVAYrnSDId2RHwRIRVwzWsWdGIzIC57ri24Oh2fkhDGVOV9EEZbrAMct8UZn43Eb0BwGV6013M/Oc9ac0SgcQmtMOI2myNA8jPoZ94nZjGUW7y3DAxjB3rV2LbuvRkdJJ8L80GLe1Xdam9kOPIG7dyqzIpi5A1OY60Km+bp0bwte+/oDmeahqHaVKYHmug5isvqxg5Hk7IMExVxV8y00MpOVdVnKe47vx1ieepi+Cwxkc98IDZFK+qihyHHJdVSTLUOALhOKqbqkKNo4CpqkyRsq1rxjJ/74rjKClop8caYf3w2HkTJv+/um03Vg3X8ewhlVE3lRxP1TjKeFWJO27+85fzDVLjADN/GsCnAYCIPCK6iZk/3pMWLQFEXlXqe1bkuKCKxhH4Hg7rfEdlTVUrmjV85mcvy9wn8qpKG+xiqvLRDPwwjiOPb/AM/sD0QKvp2AB1P8lDTwhy04RUdvDLRLVquKZXhXX864vH0O5wZKqSOA4yyfHi1xnK0zhipqr0/qqJgEjxuEryqirKB9hohhpHhqkq1DhUzA1zVAe+6BiW/WbmirvjFiXHi8A/QY1j99ZpPHf4eMJ5uyf/tStVlb+ipipbiBVNqw5Egr8QOd5jjiMrrfoKAO8CsA7AZwF8QX//JQCPADhpBQcRocMINY6sXFWCKuS471FYfrRsAGARhAGAKRNCLTRV+WhojYM5OxIYsDgOigo5mTb9NI3jTF3NbaThQ2e3rqxxrNJV6iZG6vjBvqMAkKBxRG6/ZeqaJ9XriP9umqrSn31Synag2+VzQb2qssjxIJocZVyI1lvUXFoPTVVRivw0UGiqWjjBQURYu7IZZrkt218/n5JFN0o50t0P0qd5keO2YAkzBhcwVTVDjaMAOV7B/FoGWT36JwBeAuBbAG4B8CCANwK4jpmv7Ulrlgh8T6U7z9M4zJVoFXLc9Koqq3EUQVSZLltjatZNjaOAqcqOHG8z5nXdddKxHWkax4+dOY173/kyvPSUFeGkUp7jUMNaBMfkaB1P7o8Ljlha9ZAcL6Fx1ExTVffvcVNVhsZhkeD29iR3zepeVUKO53McNSNYs6yDRmSqihJWpkFuu2jkeFF86MbzC+VrKwObHDcRuuPmahyWO65XghwvpXF4YcaCXiDrTdnCzOcCABH9AVRtjo3M3K3DnWSIXEyzXwzT+6FskkNAIqB7JzhGcpMcaiK7FmkcHc4n8sw8UL5HePbwcTAbUbk+JXpVAWq1eMFpE+HxQBWvKvVfynlOjDRCn347AJCIQjNJGXV7V8IAABYlSURBVI3DXLUnme7M37O0zbolILq2G30ZnrtqyhExVWUcb3pVyXM8JA4aRQVHENc4skxcZUvHFsWFmybwW2/cgT/7+lMLZq5JSq8vkDrz6QGAKeR40L0wSEPDeDZ58D0q7cZeBlk9Oi8fmLlNRE85oaEgQW15qRjMlUHVOA6Jvi0bx1EEjcDD63eciou3TKReX67dDHwcnJlHu8OZ6UYAQ+PQaTqkXKnUiQg8r6sWR9Z5yubbEYE9LhrHSJS4Lozj0I/D9wg71o/j167bhpefHne5zUJM48jgODzKXiGmaRyhqaoHkeNZpirTpCLP/8jxFjwqPrlLZPPxIu64JQMAy+Da89bh2vOSnRKqICs9ChFhqOanCo4wADDNVFXg9kXwF/Kq8r2eEeNAtuDYQUSH9GcCMKS/EwBm5hU9a9UiR2i3F2+TlBWVKVCqkOPmKqSsO24REBE+dOP5qb/LKnhIaxzHhePIGbhRHqh4Ntz1qxSR+BM71+EyoyZDGuTQsi+AtG/C4DgENjnue2r/t1xyWqlrNGOmqnSOI4vfAKIcRrZXVc3SOOJeVScYAJilcZhJDvXnw8dbGK4XX71GGke+O+6O9eO49PRJTBlZaRcr0mIxBK85ew0u3py2CEs+1s5+nAUR/EWsF4FHPUtwCGQIDmZe+JlqmcCjuMaRmlY9IctpGZjHlHXHXQjI6lM0jlntVVXYVOXFiySt1xrHf712W6Hrh26yFclxCRKLCY7hODle5IVNginIs7yq8vIK1XwPHnWbKuy61gupcWRxHJIgT5IcAsDh4/OlnDNsd9ws4vfc9SvxibdfUvjcg4Qdx2Hjg29OX4Sl8S2bJkdwzfa1uOC0VbnXlzFXxHox2gh6Yt4W9HQ2IqLbALwdSkv5KDN/kIgmAHwSwCYATwK4gZlfTDj2TgDX6K+/xsyf1Ns3A7gHwCSArwN4CzPP2cf3Ep6nKpu1uVh2XPW5ileVOoaoXL6dhYJc39Q4Osy5K08iQ3DoPpgarZc2t0VZa6u648ZNVR6plOrSNvN/WeR5VYVRvnkah9+degWIVu2RgOs+d1kUcceVNtWDiONQGkcZwaFNVfPZXodLDWk8RRHUUkxVzZqP39cJD/Nw/c51OGVlM+Qms/D+64otzqqiZ7MREW2DEhoXAdgB4HVEdAaA9wB4gJm3AnhAf7ePvQbATgDnAbgYwC9p92AAuBPA7zLzGQBeBHBzr+4hDRLU1m7n1+MQVIrjEI6h5veM5MqCTDAjjQCNwMfsfDGvKllhmvb99Sl1sLMwOarSS5Q9NnTHHYlrHJJSHYgT+FUQ1zi6fy9aO6GeIjjqlpvuQnhViYkrT3u49PRJbF8/Hl778PFWKVNpqHHM52scSwlZXlV5uGTzJP79lWdiu066WQWrx5qFOZsNE8PYMFH+nSuKXi5jzwLwEDMfY+YWgC8BeAOAawHcrfe5G8B1CceeDeDLzNxi5qMAHgXwWlKz5xWIEiymHd9TeOJVxdmmKlPLqBrHAfSG3yiCK89ag9990w6cPj2iNI6WKiOaz3EoTYOMBIJVBvHp06PY874rce769Oj3tOsD8TgOIOI3AEO4VVwNmzxDYgXAgnmFxCxkw06kF/Oqqho5HuSbqgDgD956IW7YtcHQOOZLaYtlkhwuJWR5VeVhqO7jtiu3Lrjb8aDQy7t4DMBuIpokomEAVwPYAGANM+/V+zwDYE3CsY9ACYphIpoC8Ep97CSAA1oQAcBTUAGKXSCiW4loDxHt2bdv38LdFZTgkLKYSfZpQUzjOAGOoxceVUXQrPm4/vz1ICI0Ax9zrQ7azLm8gBmNHYQax1ClNkyONvJ3siDXzhIcZpLDKsj1qiqYybRmkND2dvV/Ib2qorTqRWDGcThTlRE5vsCuw0sRPeM4mPlxzVPcD+AogIcBtK19mIg44dj7iehCAF8BsA/AV+1jC1z/LgB3AcCuXbu6rnEisPMvpeFEvarMtM2DhqxyZ+bbue644qkERBPzhgqmqqoQe/IqLTDGh5XJK0njqDoH1HwKSwgnelUV9LmXnFA2IjfNbs2oqqlKxlFRRwu59pG5cqYqEVBH5qIywMsBgaeeeR5vdTKgpz3AzB9j5guY+XIoPuK7AJ4lorUAoP8/l3LsB5j5PGZ+NRS5/l2olO7jRCQjfz2Ap3t5D0lQpipVczxLcMQ1jhMwVQ3Ao8qGEKszc+1CXlU2+VxV46iC685bh996445YepFVw/VYgN/EaCRUqkD89oHs7Lh5mua68SGcOt7dNzVL8MTI8YqmqrUrm/jA9dtw9bmnFNpfrs1cTuuVgLtDMyp+Z7mYqnyfFiwKfamj115Vq5n5OSLaCMVvXAJgM4C3ArhD//9MwnE+gHFm3k9EUuv8fq2hPAhVjfCetON7Dd9DWEe7SAI34AQ1jgFxHCZksjo21yqUckT2kYmzl0SdjVNWNvGTF6yPbfv5K87AFiMV986Nq/Dld78SGyert6tZ83F0rn1CXlXv/vGXhG7dJmyNYyEix4kIN11cPF7FvK8yGkcjUOkulpvgGK75hTyaTgb0uhfuJaJJqCj0dzHzASK6A8CniOhmAD+EKkcLItoF4B3MfAuAGoC/1Z5EhwD8tMFr3A7gHiJ6P4BvAvhYj++hC6FXVQlTVdXIcWBwHIeJUOOY7+TarDdPjYQFlFTJVODU8WbP25iFt718c9e2ExEaQGSSSeoOSdmR51Wl3HG7t3clOTQu0i8bezyOqPgYJCKMNAIcDAXHgjdtILj18i24ZvvaQTdjUaCngoOZdyds2w/gVQnb90AlU4RObXJ2yjm/D+XiOzCQEQBY3FRVRePQcRSLQXDUxFTVyjVV3bJ7C27ZvQUA8KYLN+CcU1cuSxVf3JXTBGkjoTxuUUyO1MO08ECkcWTVC19omNpSs+QYHDUER1UHhMWG1SuaWL1isAugxQKnd1VAWMiJObWWBWAnOayucSwKU5VeAR+dbaemRE/C+lXDlWI4lgJEoKeZKxsJdcaL4q2XbsLrzzs1/C7CqVfZTpNgLnyGa+WmitFGgL0Hj3edx2F5YJkokf2FGQCYNTHEUo5U4TjCMquDFxynrFQrrWcOHV82K8gTRTMn6Vyz5lfWOJo1PywSZF6jap6qKjDH9lC93H2MNYOQu3HjZfnBCY4K8IjQ6aCUO26Z0pj28WXNBL3AlmlVmU/FrriJAIg0jixTVZUFQxLkEv3UOMy2l/XsM1OZO41j+cEJjgpQ/vuMTk6m2IVyxy1rJugFhusB1mm30Qq3siwRprlOEaQjjWDBov5FWPcz8tgcs2XNpaNG/XYnOJYfBj8jLUFIHe0yGkeVl0cmpMVgqgKAM1aP4ukDM07j0Ag1jpTuuPMnti+YY0OUmbV/YyHmjluBHBe48bL84NaOFSCFnMoEAFZyxzXSmi8GnKFdbAeRcHExQqLD00xV29atxOlG7MiJwPSq6hdq/okIjux6JQ5LG05wVEBUyKmTSY7HvKqWeMoRIBIcC2S2X/LI86paSAzaq6qsyc0sIlTVs8xh8cIJjgoQU1UeUWxyAXaFtyIIjHoYiwGyenamB4UwALBPE6PvUZ+9qgyOo6zGYdRQWS5JDh0iOMFRAUQqf0+7w5maxEJpHIvNVOUEh4IIjn6ZYjwaoFdVSYFlVm1cLkkOHSI4wVEBPkXZcTM1DuOnKpNLGAC4CJIcAio9+cRIPTc77smCFc1AZUvtm+Cg/gqOEyHHYxrHgjXJYZFgccxISwxiqupwdgAgkarb3OrwCRVyWiwcBwDs3Dheqv70csYbL9iAM9eM9a0/fK+/2VlPjONwGsdyhhMcFSBeVa12tlcVEEWZV3LHlQDARTRRf/jGnU7j0Fg5XMPlZ0737Xo+UeWU6lUQ5zhKBgA2XQDgcoYTHBUgXlUdztckAo/AXO1lX4wax2LhW05GnLNuBc46Zaxv1zM5jmZJgRWL43CCY9nBCY4KkMpvrQ6jWcvXOBLKLRRCsAgFh8PgcM+tL+vr9cTENFTzS8fuOFPV8oYTHBUgK6gipqrA98DoVLrOpqkRrBsfir2EDg79gqdLpVZZuDhT1fKGm5EqQFZQc63sAEBAXppqpqrXbT8Vr9t+av6ODg49QuB7lTi2kboTHMsZzlGuAkTjmGt3cmMaAo9c5KzDkkXgUSWNw/cII32MrHfoL5zgqIDp0QYA4EcHZnID+zyiyjUZHBwGDd+jyg4RYq5y5Pjyg5vRKuCcdSsAALOtTmYFQEB5pjhV3WGpouZ7lVPejDYCN/aXKZzgqICtq8dQ11pEXiYR36MFK+bj4NBv+BVNVYCqyeHMVMsTTnBUQD3w8NK1yp8+V+PwqFKCQweHxYDgRExVDd+lG1mmcI+1Is45dSUAII++8L2FKx/q4NBvrFnRxPpVw5WOHW0ETuNYpnDuuBWxTfMcRTQOZ+Z1WKr4xNsvrlT2GFA1ORwxvjzhNI6K2KY1jiJxHIHzqnJYohiuB5XrnJ+xehQbKmorDosbTuOoiJecMqb4iwK5qpxnicPJiHe8Ygve8Yotg26GQw/QU8FBRLcBeDsAAvBRZv4gEU0A+CSATQCeBHADM7+YcOxvArgGSiv6AoDbmJmJ6IsA1gKY0bu+hpmf6+V9JKFZ8/F7P7UTL8lJOvf2y7e4wkcOJyVcbfrli54JDiLaBiU0LgIwB+A+IvoLALcCeICZ7yCi9wB4D4DbrWMvBfByANv1pr8D8AoAX9Tfb2LmPb1qe1G8dtspufv8+Dn5+zg4ODgsJfTS+H4WgIeY+RgztwB8CcAbAFwL4G69z90Arks4lgE0AdQBNADUADzbw7Y6ODg4OBRELwXHYwB2E9EkEQ0DuBrABgBrmHmv3ucZAGvsA5n5qwAeBLBX/32emR83dvkjInqYiP4zpejDRHQrEe0hoj379u1bwNtycHBwOLnRM8GhJ/o7AdwP4D4ADwNoW/swlHYRAxGdAaWxrAewDsAVRLRb/3wTM58LYLf+e0vK9e9i5l3MvGt6un9V2hwcHByWO3rqJ8rMH2PmC5j5cgAvAvgugGeJaC0A6P9JxPb1AL7GzEeY+QiAzwF4mT7n0/r/YQCfgOJQHBwcHBz6hJ4KDiJarf9vhOI3PgHgswDeqnd5K4DPJBz6LwBeQUQBEdWgiPHH9fcpfc4agNdBmcQcHBwcHPqEXsdx3EtEkwDmAbyLmQ8Q0R0APkVENwP4IYAbAICIdgF4BzPfAuDPAFwB4FtQpqz7mPn/EtEIgM9roeED+GsAH+3xPTg4ODg4GCBFMyxv7Nq1i/fsGbj3roODg8OSAhF9nZl32dtdLgwHBwcHh1I4KTQOItoHZRargikAzy9gcxYKi7VdwOJtm2tXOSzWdgGLt23LrV2nMXOXW+pJIThOBES0J0lVGzQWa7uAxds2165yWKztAhZv206WdjlTlYODg4NDKTjB4eDg4OBQCk5w5OOuQTcgBYu1XcDibZtrVzks1nYBi7dtJ0W7HMfh4ODg4FAKTuNwcHBwcCgFJzgcHBwcHErBCY4MENFrieg7RPSELjo1qHZsIKIHieifiOgfdWVFENGvENHTOsX8w0R09QDa9iQRfUtff4/eNkFEXyCi7+n/q/rcppcYffIwER0iol8YVH8R0R8S0XNE9JixLbGPSOFDesw9SkQ7+9yu/0ZE39bX/nMiGtfbNxHRjNF3H+lzu1KfHRG9V/fXd4jox/vcrk8abXqSiB7W2/vZX2nzQ+/GGDO7v4Q/qFxY/wxgC1RBqUcAnD2gtqwFsFN/HoPKMnw2gF8B8EsD7qcnAUxZ234TwHv05/cAuHPAz/EZAKcNqr8AXA5gJ4DH8voIqm7N56DKLV8CVQytn+16DYBAf77TaNcmc78B9Ffis9PvwSNQBd8263fW71e7rN9/G8B/GUB/pc0PPRtjTuNIx0UAnmDm7zPzHIB7oKoX9h3MvJeZv6E/HwbwOFSdksWKIlUe+4VXAfhnZq6aOeCEwcxfBvCCtTmtj64F8L9Y4WsAxkmXIehHu5j5flYVOwHga1A1cfqKlP5Kw7UA7mHmWWb+AYAn0KNSC1ntIiKCStj6p724dhYy5oeejTEnONKxDsC/Gt+fwiKYrIloE4DzATykN/2sVjf/sN8mIQ0GcD8RfZ2IbtXbcqs89hFvRvxlHnR/CdL6aDGNu38HtTIVbCaibxLRlygqrNZPJD27xdJfuwE8y8zfM7b1vb+s+aFnY8wJjiUEIhoFcC+AX2DmQwD+B4DTAZwHVWL3twfQrMuYeSeAqwC8i4guN39kpRsPxOebiOoAXg/gf+tNi6G/ujDIPkoDEb0PQAvAx/WmvQA2MvP5AH4RwCeIaEUfm7Qon52BGxFfoPS9vxLmhxALPcac4EjH01A10gXr9baBgFQNknsBfJyZPw0AzPwsM7eZuQNVl6Tv1RA5qsj4HIA/120oUuWxH7gKwDeY+VndxoH3l4G0Phr4uCOit0EVSbtJTzjQpqD9+vPXobiEM/vVpoxntxj6K4AqVPdJ2dbv/kqaH9DDMeYERzr+H4CtRLRZr1zfDFW9sO/Q9tOPAXicmX/H2G7aJa9Hn6shEtEIEY3JZyhi9TEUq/LYD8RWgYPuLwtpffRZAD+jPV8uAXDQMDf0HET0WgD/EcDrmfmYsX2aiHz9eQuArQC+38d2pT27zwJ4MxE1iGizbtc/9KtdGlcC+DYzPyUb+tlfafMDejnG+sH6L9U/KO+D70KtFt43wHZcBqVmPgrgYf13NYA/gaqS+KgeDGv73K4tUB4tjwD4R+kjAJMAHgDwPagqjRMD6LMRAPsBrDS2DaS/oITXXqhKmE8BuDmtj6A8XX5fj7lvAdjV53Y9AWX/lnH2Eb3vT+hn/DCAbwD4N31uV+qzA/A+3V/fAXBVP9ult/8xVPVSc99+9lfa/NCzMeZSjjg4ODg4lIIzVTk4ODg4lIITHA4ODg4OpeAEh4ODg4NDKTjB4eDg4OBQCk5wODg4ODiUghMcDg4LACJqUzwj74JlU9aZVgcZc+LgEEMw6AY4OCwTzDDzeYNuhINDP+A0DgeHHkLXaPhNUjVL/oGIztDbNxHR3+ikfQ8Q0Ua9fQ2pOhiP6L9L9al8IvqorrdwPxENDeymHE56OMHh4LAwGLJMVW8yfjvIzOcC+D0AH9TbPgzgbmbeDpVI8EN6+4cAfImZd0DVfvhHvX0rgN9n5nMAHICKTHZwGAhc5LiDwwKAiI4w82jC9icBXMHM39eJ6J5h5kkieh4qbca83r6XmaeIaB+A9cw8a5xjE4AvMPNW/f12ADVmfn/v78zBoRtO43Bw6D045XMZzBqf23D8pMMA4QSHg0Pv8Sbj/1f1569AZVwGgJsA/K3+/ACAdwIAEflEtLJfjXRwKAq3anFwWBgMEdHDxvf7mFlcclcR0aNQWsONetvPAfgjIno3gH0A/q3efhuAu4joZijN4p1QGVkdHBYNHMfh4NBDaI5jFzM/P+i2ODgsFJypysHBwcGhFJzG4eDg4OBQCk7jcHBwcHAoBSc4HBwcHBxKwQkOBwcHB4dScILDwcHBwaEUnOBwcHBwcCiF/w/+uSwFPAxJswAAAABJRU5ErkJggg==\n", + "text/plain": [ + "
" ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "plot_the_loss_curve(history.epoch, history.history['loss'])" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "kcn3gB3Uq7u6" + }, + "source": [ + "**Batch Size**\n", + "\n", + "The number of examples in a batch." + ] + }, + { + "cell_type": "code", + "execution_count": 34, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 428 }, + "colab_type": "code", + "id": "DTSP63PTZ8hv", + "outputId": "d0a90bd0-9e34-49e4-b351-fa035ae6df22" + }, + "outputs": [ { - "cell_type": "code", - "metadata": { - "id": "WKik9O5grNa1", - "colab_type": "code", - "colab": {} - }, - "source": [ - "# play with batch size\n", - "learning_rate = 0.01 \n", - "epochs = 150 \n", - "batch = ? # Replace ? with an integer \n", - "optimizer = RMSprop(learning_rate)\n", - "model.compile(loss='mean_squared_error',optimizer=optimizer) # compile the model\n", - "model.fit(X_train, y_train, epochs=epochs, batch_size=batch) # fit the model\n", - "model.evaluate(X_test, y_test) # Evaluate the model" - ], - "execution_count": null, - "outputs": [] - }, + "name": "stdout", + "output_type": "stream", + "text": [ + "Train on 323 samples\n", + "Epoch 1/10\n", + "323/323 [==============================] - 0s 1ms/sample - loss: 295.7401\n", + "Epoch 2/10\n", + "323/323 [==============================] - 0s 38us/sample - loss: 110.5806\n", + "Epoch 3/10\n", + "323/323 [==============================] - 0s 37us/sample - loss: 108.5253\n", + "Epoch 4/10\n", + "323/323 [==============================] - 0s 39us/sample - loss: 106.6067\n", + "Epoch 5/10\n", + "323/323 [==============================] - 0s 50us/sample - loss: 87.7566\n", + "Epoch 6/10\n", + "323/323 [==============================] - 0s 37us/sample - loss: 113.7894\n", + "Epoch 7/10\n", + "323/323 [==============================] - 0s 37us/sample - loss: 85.4327\n", + "Epoch 8/10\n", + "323/323 [==============================] - 0s 41us/sample - loss: 85.2161\n", + "Epoch 9/10\n", + "323/323 [==============================] - 0s 36us/sample - loss: 89.5082\n", + "Epoch 10/10\n", + "323/323 [==============================] - 0s 34us/sample - loss: 104.7811\n", + "81/81 [==============================] - 0s 626us/sample - loss: 111.9399\n", + "The MSE value is: 111.93991427951389\n" + ] + } + ], + "source": [ + "####################### Complete example to check the performance of the model with different batch size while keeping epochs as 30 and learning rate as 0.01 #######################################\n", + "# define the model\n", + "model = Sequential()\n", + "model.add(Dense(10, activation='relu', input_shape=(n_features,)))\n", + "model.add(Dense(8, activation='relu'))\n", + "model.add(Dense(1))\n", + "\n", + "optimizer = RMSprop(0.01) # 0.1 is the learning rate\n", + "model.compile(loss='mean_squared_error',optimizer=optimizer) # Compile the model\n", + "\n", + "# fit the model \n", + "model.fit(X_train, y_train, epochs=10, batch_size=40, verbose = 1)\n", + "\n", + "# evaluate the model\n", + "print('The MSE value is: ', model.evaluate(X_test, y_test))" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "r7xoMXoVaJy5" + }, + "source": [ + "You can see above the cost(loss) value i.e. MSE for batch size 40 while keeping epochs as 10 and learning rate as 0.01" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "y9VcjhruiIxd" + }, + "source": [ + "### Exercise 4\n", + "\n", + "Test several batch size values to see the impact of varying this value when defining your model." + ] + }, + { + "cell_type": "code", + "execution_count": 35, + "metadata": { + "colab": {}, + "colab_type": "code", + "id": "WKik9O5grNa1" + }, + "outputs": [ { - "cell_type": "markdown", - "metadata": { - "id": "wQV6zkKXrwR0", - "colab_type": "text" - }, - "source": [ - "#### **Summary of hyperparameter tuning**\n", - "Most machine learning problems require a lot of hyperparameter tuning. Unfortunately, we can't provide concrete tuning rules for every model. Lowering the learning rate can help one model converge efficiently but make another model converge much too slowly. You must experiment to find the best set of hyperparameters for your dataset. That said, here are a few rules of thumb:\n", - "\n", - "* Training loss should steadily decrease, steeply at first, and then more slowly until the slope of the curve reaches or approaches zero.\n", - "* If the training loss does not converge, train for more epochs.\n", - "* If the training loss decreases too slowly, increase the learning rate. Note that setting the learning rate too high may also prevent training loss from converging.\n", - "* If the training loss varies wildly (that is, the training loss jumps around), decrease the learning rate.\n", - "* Lowering the learning rate while increasing the number of epochs or the batch size is often a good combination.\n", - "* Setting the batch size to a very small batch number can also cause instability. First, try large batch size values. Then, decrease the batch size until you see degradation.\n", - "* For real-world datasets consisting of a very large number of examples, the entire dataset might not fit into memory. In such cases, you'll need to reduce the batch size to enable a batch to fit into memory.\n", - "\n", - "Remember: the ideal combination of hyperparameters is data dependent, so you must always experiment and verify." - ] + "name": "stdout", + "output_type": "stream", + "text": [ + "Train on 323 samples\n", + "Epoch 1/15\n", + "323/323 [==============================] - 0s 1ms/sample - loss: 94.3939\n", + "Epoch 2/15\n", + "323/323 [==============================] - 0s 249us/sample - loss: 74.0610\n", + "Epoch 3/15\n", + "323/323 [==============================] - 0s 269us/sample - loss: 69.7922\n", + "Epoch 4/15\n", + "323/323 [==============================] - 0s 230us/sample - loss: 63.4705\n", + "Epoch 5/15\n", + "323/323 [==============================] - 0s 241us/sample - loss: 63.5209\n", + "Epoch 6/15\n", + "323/323 [==============================] - 0s 232us/sample - loss: 63.8765\n", + "Epoch 7/15\n", + "323/323 [==============================] - 0s 232us/sample - loss: 56.7328\n", + "Epoch 8/15\n", + "323/323 [==============================] - 0s 226us/sample - loss: 56.3436\n", + "Epoch 9/15\n", + "323/323 [==============================] - 0s 252us/sample - loss: 51.8511\n", + "Epoch 10/15\n", + "323/323 [==============================] - 0s 234us/sample - loss: 51.8350\n", + "Epoch 11/15\n", + "323/323 [==============================] - 0s 243us/sample - loss: 52.6305\n", + "Epoch 12/15\n", + "323/323 [==============================] - 0s 233us/sample - loss: 50.5269\n", + "Epoch 13/15\n", + "323/323 [==============================] - 0s 237us/sample - loss: 49.2241\n", + "Epoch 14/15\n", + "323/323 [==============================] - 0s 227us/sample - loss: 48.7887\n", + "Epoch 15/15\n", + "323/323 [==============================] - 0s 230us/sample - loss: 47.9197\n", + "81/81 [==============================] - 0s 596us/sample - loss: 118.9669\n" + ] }, { - "cell_type": "markdown", - "metadata": { - "id": "3wYL0qp4fTv-", - "colab_type": "text" - }, - "source": [ - "#### 5. Make a Prediction\n", - "Making a prediction is the final step in the life-cycle. It is why we wanted the model in the first place.\n", - "\n", - "It requires you have new data for which a prediction is required, e.g. where you do not have the output values.\n", - "\n", - "From an API perspective, you simply call a function to make a prediction of a class label, probability, or numerical value: whatever you designed your model to predict.\n", - "\n", - "We have our new test data located at the given github location:\n", - "\n", - "https://raw.githubusercontent.com/dphi-official/Datasets/master/Boston_Housing/Testing_set_boston.csv\n", - "\n" + "data": { + "text/plain": [ + "118.96689219533661" ] - }, - { - "cell_type": "code", - "metadata": { - "id": "s-fWHtrMe8U_", - "colab_type": "code", - "colab": {} - }, - "source": [ - "# Load new test data\n", - "new_test_data = pd.read_csv('https://raw.githubusercontent.com/dphi-official/Datasets/master/Boston_Housing/Testing_set_boston.csv')" - ], - "execution_count": null, - "outputs": [] - }, - { - "cell_type": "code", - "metadata": { - "id": "crRYu0YtiO5G", - "colab_type": "code", - "colab": {} - }, - "source": [ - "# make a prediction\n", - "model.predict(new_test_data)" - ], - "execution_count": null, - "outputs": [] - }, + }, + "execution_count": 35, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# play with batch size\n", + "learning_rate = 0.01 \n", + "epochs = 15 \n", + "batch = 5 # Replace ? with an integer \n", + "optimizer = RMSprop(learning_rate)\n", + "model.compile(loss='mean_squared_error',optimizer=optimizer) # compile the model\n", + "history = model.fit(X_train, y_train, epochs=epochs, batch_size=batch) # fit the model\n", + "model.evaluate(X_test, y_test) # Evaluate the model" + ] + }, + { + "cell_type": "code", + "execution_count": 36, + "metadata": {}, + "outputs": [ { - "cell_type": "markdown", - "metadata": { - "id": "UuoFhNRrn623", - "colab_type": "text" - }, - "source": [ - "\n", - "**Congratulations! You have successfully build your first deep learning model and predicted the output (i.e. MEDV) of new test data.**" + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEGCAYAAABiq/5QAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAgAElEQVR4nO3deXhV1b3/8fc380ggJwNImIKIAwXUCASr91Zqa0d62zoPaHtre68DrdbWDre39f5ur9YOdrBOrbOtU23FWqdaOymgAQEFFCRMYQyBMAXI9P39cXYwYIYD5GQn53xez7Ofc/Y60yc85HtW1l57bXN3REQkeaSEHUBERHqXCr+ISJJR4RcRSTIq/CIiSUaFX0QkyaSFHSAWRUVFPnLkyLBjiIj0K/Pmzdvi7sUHt/eLwj9y5EiqqqrCjiEi0q+Y2eqO2jXUIyKSZFT4RUSSjAq/iEiS6Rdj/CIih6upqYmamhr27t0bdpS4ycrKoqysjPT09Jier8IvIgmtpqaG/Px8Ro4ciZmFHafHuTt1dXXU1NQwatSomF6joR4RSWh79+4lEokkZNEHMDMikcgh/UWjwi8iCS9Ri36bQ/35ErrwP7lgHQ/O6XAaq4hI0krowv/c4o3c9tcV6JoDIhKmvLy8sCMcIKEL/5TyCOvq91CzbU/YUURE+oyELvyV5REAZq+oCzmJiMiBVq1axRlnnMH48eOZNm0aa9asAeCxxx5j3LhxTJgwgdNPPx2AxYsXM2nSJCZOnMj48eNZvnz5EX12Qk/nPLokj0huBnOq6zjnlGFhxxGRkH3vqcUsWb+jR9/z+KMG8N+fOOGQX3fVVVcxY8YMZsyYwd13383VV1/NH/7wB2644Qaee+45hg4dSn19PQC33347M2fO5MILL6SxsZGWlpYjypzQPX4zY0p5hDnVdRrnF5E+Zfbs2VxwwQUAXHzxxfzzn/8E4NRTT+XSSy/lrrvu2l/gKysr+f73v89NN93E6tWryc7OPqLPTugeP8CU0RGefmMDa7Y2MCKSG3YcEQnR4fTMe9vtt9/O3Llzefrppzn55JOZN28eF1xwAZMnT+bpp5/mox/9KHfccQdnnHHGYX9GQvf4ASrLCwGYU61xfhHpO6ZOncrDDz8MwEMPPcRpp50GwIoVK5g8eTI33HADxcXFrF27lurqasrLy7n66quZPn06ixYtOqLPTvge/+jiPIryMplTvZVzTxkedhwRSUINDQ2UlZXt37/mmmv4+c9/zmWXXcbNN99McXEx99xzDwDXXXcdy5cvx92ZNm0aEyZM4KabbuKBBx4gPT2dwYMH881vfvOI8iR84Y+O8xcye0V0nD/Rz+ATkb6ntbW1w/a//OUv72l74okn3tN2/fXXc/311/dYnoQf6oHofP6NO/ayuq4h7CgiIqFLmsIPGucXEYEkKfyji3Mpzs9ktgq/SFJK9Onch/rzJUXh13x+keSVlZVFXV3i/u63rceflZUV82sS/uBumynlhTy1cD0rt+ymvLhvLZgkIvFTVlZGTU0NtbW1YUeJm7YrcMUqaQp/5f5x/q0q/CJJJD09PeYrUyWLpBjqARhVlEtJfqYO8IpI0otr4TezmWb2ppktNrMvB22FZvaCmS0PbgfFM0O7LEwpjzBb4/wikuTiVvjNbBzwBWASMAH4uJkdDVwPvOjuY4AXg/1eUTk6Qu3OfVRv2d1bHyki0ufEs8d/HDDX3RvcvRn4G/BpYDpwX/Cc+4BPxTHDATSfX0QkvoX/TeA0M4uYWQ7wUWAYUOruG4LnbARKO3qxmV1uZlVmVtVTR+NHRnIYPCBLF2YRkaQWt8Lv7kuBm4DngWeBBUDLQc9xoMMBd3e/090r3L2iuLi4RzK1rdszp3qrxvlFJGnF9eCuu//a3U9299OBbcAyYJOZDQEIbjfHM8PBppRH2LJrHytqNc4vIskp3rN6SoLb4UTH938DzAJmBE+ZATwZzwwHqxwdXIdX4/wikqTiPY//d2a2BHgKuMLd64EbgTPNbDnwwWC/1wwvzGFIQZYO8IpI0orrmbvufloHbXXAtHh+blfa5vP/Y3mt1ucXkaSUNGfutldZHmHLrkbe2bwr7CgiIr0uKQu/5vOLSDJLysI/rDCbowqydIBXRJJSUhZ+M2PK6Ijm84tIUkrKwg/R4Z6tuxtZrnF+EUkySVv429bn1/INIpJskrbwDyvMYejAbB3gFZGkk7SFH6LDPXNXbqW1VeP8IpI8krzwF7J1dyPLNu8MO4qISK9J8sIfzOfXOL+IJJGkLvzDCnMoG5St+fwiklSSuvCDxvlFJPkkfeGvLI9Q39DE25s0zi8iyaHLwm9mqWb2UG+FCcPk8kJA8/lFJHl0WfjdvQUYYWYZvZSn15UNymFYoebzi0jyiGU9/mrgZTObBey/XqG7/zhuqXpZZXmE5xZvorXVSUnR+vwikthiGeNfAfwxeG5+uy1hTCmPsH1PE0s37gg7iohI3HXb43f37wGYWV6wn3Crmr27Pv9WTjiqIOQ0IiLx1W2P38zGmdnrwGJgsZnNM7MT4h+t9xw1MJsRkRyN84tIUohlqOdO4Bp3H+HuI4BrgbviG6v3TRkVYW51HS2azy8iCS6Wwp/r7i+17bj7X4HcuCUKSeXoCDv2NrN0g8b5RSSxxVL4q83sv8xsZLB9m+hMn4TSNp9fwz0ikuhiKfyfA4qBJ4DfAUVBW0IZUpDNSI3zi0gS6HJWj5mlAt9y96t7KU+oKkdH+OOiDbS0Oqmazy8iCSqWM3ff30tZQjelPMLOvc0sWa9xfhFJXLGcuft6cNbuYxx45u4TcUsVknfn89fxvjLN5xeRxBTLGH8WUAecAXwi2D4ez1BhKR2QRXlRrsb5RSShxTLGv8jdf9JLeUI3uTzCHxeup7mllbTUpF+1WkQSUCxj/Of3UpY+YUp5ITv3NbNE8/lFJEHF0qV92cx+YWanmdlJbVvck4Wkst04v4hIIorl4O7E4PaGdm1OdMw/4ZQMyKK8OJfZK+q4/PTRYccREelxsazO+YHeCNKXTCmPMGuBxvlFJDF1WtXM7JZ292ce9Ni9ccwUusryCLv2NbNY8/lFJAF11Z09vd39GQc9Nj6WNzezr5jZYjN708x+a2ZZZjbKzOaa2Ttm9khfvKzj/uvwapxfRBJQV4XfOrkfEzMbClwNVLj7OCAVOA+4CfiJux8NbAM+f6jvHW8l+VmMLtZ8fhFJTF0V/hQzG2RmkXb3C82skGgRj0UakG1maUAOsIHoQeHHg8fvAz51mNnjqnJ0hNdWbqWppTXsKCIiPaqrwl8AzAOqgAHA/GB/HjFcc9fd1wE/BNYQLfjbg9fWu3tz8LQaYGhHrzezy82sysyqamtrY/tpetCU8gi7G1t4c932Xv9sEZF46nRWj7uPPJI3NrNBwHRgFFBPdK2fs2J9vbvfSfTqX1RUVPT6ZbHaX4f3xOGDevvjRUTiJp5zFT8IrHT3WndvIrqe/6nAwGDoB6AMWBfHDIetKC+TMSV5OsArIgknnoV/DTDFzHLMzIBpwBLgJeCzwXNmAE/GMcMRmVIeoWqVxvlFJLHErfC7+1yiB3HnA28En3Un8HXgGjN7B4gAv45XhiNVOTpCQ2MLb2icX0QSSKdj/MHsnU65+9bu3tzd/xv474Oaq4FJMaUL2aRRwXz+FXWcpHF+EUkQXfX422b0zANqgWXA8uD+vPhHC19RXibHlOZpPr+IJJROC7+7j3L3cuDPwCfcvcjdI0QvwvJ8bwUMW2V5hKpV2zTOLyIJI5Yx/inu/qe2HXd/Bpgav0h9y5TyCHuaWlhUUx92FBGRHhFL4V9vZt82s5HB9i1gfbyD9RWT283nFxFJBLEU/vOBYuD3ROfiF5NEV+UqzM3g2MH5zF6hcX4RSQyxrMe/FZhpZrnuvrsXMvU5U8ojPPzaGhqbW8lI0/r8ItK/dVvFzGyqmS0Blgb7E8zsl3FP1odMKS9kb1OrxvlFJCHE0n39CfBhoA7A3Rdy4Fr9CW/yqOg4v4Z7RCQRxDRu4e5rD2pqiUOWPmtQMM4/Z6UKv4j0f7EU/rVmNhVwM0s3s68SDPskkynlEeat3sa+5qT6zhORBBRL4f8ScAXRdfPXAROD/aRSOTrC3qZWFq7Vuj0i0r91OavHzFKBi939wl7K02dNHlWIGcyprtu/ho+ISH/UZY/f3VuAC3opS582MCeDYwcP0Lo9ItLvxTLU808z+4WZnWZmJ7VtcU/WB1VqnF9EEkC3J3ARHdMHuKFdmxO9aHpSmVJeyN0vr2TBmvr9SzmIiPQ3sZy5+4HeCNIfTB4VwQxmV9ep8ItIvxVLjx8z+xhwApDV1ubuN3T+isRUkJPO8UM0zi8i/VssSzbcDpwLXAUYcDYwIs65+qwp5RHmr6lnb5PG+UWkf4rl4O5Ud78E2Obu3wMqgWPiG6vvqiyP0NjcyutrtG6PiPRPsRT+PcFtg5kdBTQBQ+IXqW87ZVQhKQa//me1ev0i0i/FUvj/aGYDgZuB+cAq4LfxDNWXFWSn842PHMefl27m3DvnsHnH3rAjiYgcEnP32J9slglkuXuvrltQUVHhVVVVvfmR3Xr2zY185ZEFDMxJ565LKhg3tCDsSCIiBzCzee5ecXB7t7N6zOySDtpw9/t7Klx/dNa4wZQNquQL91dx9u2z+cm5EzhrXNKOgIlIPxLLUM8p7bbTgO8Cn4xjpn5j3NACnrzyVMYOzudLD87n1pfe4VD+ghIRCUMsJ3Bd1X4/GO9/OG6J+pmS/CwevnwKX3t8ETc/9zbLN+3kxs+MJys9NexoIiIdiukEroPsBkb1dJD+LCs9lZ+eN5FjSvP44fPLWL21gTsvrqA4PzPsaCIi7xHLCVxPmdmsYPsj8Dbw+/hH61/MjCvPGMNtF57E0g07mP6Lf7Jk/Y6wY4mIvEcsPf4ftrvfDKx295o45en3PvK+IQwrzOHf76vis7e/wi3nTuRDJwwOO5aIyH7d9vjd/W/ttpdV9Ls3bmgBs648lTEleXzxwXnc9tcVOugrIn1GLEM9O81sRwfbTjPTWEYnSgZk8cgXK/nY+4Zw07Nvce1jC7WOv4j0CbEM9dwCbAAeILpI24XAEHf/TjyDJYKs9FR+fv6JjCnJ5yd/XsaaugZuv/hkivJ00FdEwhPLPP5Puvsv3X2nu+9w99uA6fEOlijMjJkfHMOtF5zEm+u3M/0XL/PWRv2hJCLhiaXw7zazC80s1cxSzOxColM65RB8bPwQHv1iJc2trXzml6/w5yWbwo4kIkkqlsJ/AXAOsAnYTHQ9fl2A/TCMLxvIk1e8n/LiPL7wQBV3/l0HfUWk98Uyq2eVu09396Jg+5S7r+rudWY21swWtNt2mNmXzazQzF4ws+XB7aAe+Un6icEFWTz6xUo+Om4I3//TW3zt8UU66CsivarTwm9mXzCzMcF9M7O7zWy7mS0ys5O6e2N3f9vdJ7r7ROBkoIHoiV/XAy+6+xjgxWA/qWRnRA/6zpw2hsfm1XDxr16lbte+sGOJSJLoqsc/k+ja+wDnAxOAcuAa4KeH+DnTgBXuvprogeH7gvb7gE8d4nslhJQU4ytnHsPPzj+RhTX1TL/1Zd7euDPsWCKSBLoq/M3u3hTc/zhwv7vXufufgdxD/JzzePfiLaXuviG4vxEo7egFZna5mVWZWVVtbe0hflz/8ckJR/HIFytpbG7lM7e9wsK1uqSjiMRXV4W/1cyGmFkW0R77n9s9lh3rB5hZBtFlnB87+DGPHtns8Oimu9/p7hXuXlFcXBzrx/VLE4cN5MkrT2VgTjpffGAem3fqql4iEj9dFf7vAFVEh3tmuftiADP7F6D6ED7jI8B8d2+bv7jJzIYE7zWE6EyhpDekIJu7Lqlg+54m/uPB+TrgKyJx02nhd/c/AiOA49z9C+0eqgLOPYTPOJ8Dr9E7C5gR3J8BPHkI75XQjhsygB+ePYF5q7fx3VmLNdVTROKiy+mc7t7s7tsOatvt7rtieXMzywXOBJ5o13wjcKaZLQc+GOxL4GPjh3DFB0bz21fX8uDcNWHHEZEEdDgXYomZu+8GIge11RE9ZiCduPbMsSzdsJPvzVrMMSV5TC6PdP8iEZEYxXLmrvSylBTjlvMmMjySw38+NJ919XvCjiQiCSSmwm9mQ81sqpmd3rbFO1iyG5CVzl2XVNDY3MoXH6hiT6MO9opIz4hlPf6bgJeBbwPXBdtX45xLgNHFedxy3kQWr9/B9U8s0sFeEekRsYzxfwoY6+5aUyAE044r5dozj+GHzy/jhKMGcPnpo8OOJCL9XCxDPdVAeryDSOeu+MDRfPR9g7nxmbf427LEPYtZRHpHLD3+BmCBmb0I7O/1u/vVcUslBzAzbv7sBKprd3PVb+Yz68r3M7LoUFfNEBGJiqXHPwv4H+AVYF67TXpRbmYad11SQUqK8YX7q9i1rznsSCLST3Xb43f3+7p7jvSOYYU53HrBSVxy96tc88gCbr/oZFJSLOxYItLPxDKrZ4yZPW5mS8ysum3rjXDyXqceXcQ3P3oczy/ZxM/+sjzsOCLSD8Uy1HMPcBvQDHwAuB94MJ6hpGufO3UknzmpjFv+vJznFm8MO46I9DOxFP5sd38RMHdf7e7fBT4W31jSFTPjf/9tHBPKCrjmkQUs26QLuIhI7GIp/PvMLAVYbmZXmtm/AXlxziXdyEpP5faLTyY7I43L769ie0NT9y8SESG2wj8TyAGuJnrt3It4d1llCdGQgmzuuPgk1tXv4crfzqelVWf2ikj3ui387v5asAzzVne/zN0/4+5zeiGbxODkEYXcMH0c/1i+hR88+1bYcUSkH4hlVk+lmS0B3gr2J5jZL+OeTGJ2/qThXDRlOHf8vZonF6wLO46I9HGxDPXcAnwYqANw94WAVufsY77z8ROYNLKQrz2+iDfXbQ87joj0YTEty+zuaw9q0hrBfUxGWgq/vOgkIrkZXH5/FVt2aU09EelYLIV/rZlNBdzM0s3sq8DSOOeSw1CUl8kdF1dQt7uR/3xoPk0trWFHEpE+KJbC/yXgCmAosA6YGOxLH/S+sgJ+8NnxvLpyKzc8tSTsOCLSB8WyVs8W4MJeyCI9ZPrEoSxev4M7/17NCUcN4LxJw8OORGurs3NvM/V7Gtm+p4n6hibq9zSxvaHx3ft7mmhobObz7x/FySMKw44skrA6Lfxm9rOuXqhlmfu2r591LEs37OC/nnyTMaV5nDyiEHen1aGl1Wn1ti267+5BO/sfi7Yf+PyWVvY/trep5YCivb2hkfqDivr2Pe8+3tUFxHIyUhmYnc7uxhZeXbmNZ798GkV5mb33DyaSRKyzy/mZWSPwJvAosB44YBnI3ly1s6Kiwquqqnrr4xLG9oYmPnnrP1ld14AZXRbenmAGBdnpDMxOpyA7nYKcDAZmpzMwJ9o2IDudge3bcqJtBdnpZKalAvD2xp184hf/5NTREe6+9BTMtPqoyOEys3nuXnFwe1dDPUOAs4FziS7Q9gjwuLvXxyei9LSCnHQe/PxkHptXg7uTYkaKGakp0fV+UlOMVDPMIDUl+lhKipFikGoH7adY9DUW3U9JMbLSUxmUkx4U+wzys9KOeJnosYPz+eZHjuW7Ty3h/tmrmTF1ZM/8Y4jIfp32+A94klkZcB5wDfB1d38g3sHaU48/ubg7n7v3NV5eUcdTV76fsYPzw44k0i911uOP5czdk4iu13MR8Ay6+pbEmZlx89kTGJCVxtW/fZ29TTptRKQndVr4zewGM5tHtJf/N6DC3T/v7pojKHFXlJfJD8+ewNubdnLjM1qDSKQnddXj/zYwEJgA/B8w38wWmdkbZraoV9JJUvvXsSVcdupI7n1lFS+9tTnsOCIJo6uDu6N6LYVIJ75+1rHMXlHHdY8v5JmZp1OcrymeIkeq0x5/cLWtTrfeDCnJKys9lZ+dfyI79zZz3eMLiWUygoh0LaZF2kTCdExpPt/62HH89e1a7n1lVdhxRPo9FX7pFy6eMoJpx5bwf396i6UbdoQdR6Rfi2U658xY2kTiycz4wWfHU5CTrimeIkcolh5/R9fXvbSHc4h0K5KXyY/OnsDyzbv4/p+0MrjI4epqHv/5ZvYUMMrMZrXbXgK2xvLmZjbQzB43s7fMbGlwGcdCM3vBzJYHt4N66oeRxHf6McV8/v2juH/2al5cuinsOCL9Ulc9/leAHxG91u6P2m3XEr0UYyx+Cjzr7scSPR9gKXA98KK7jwFeDPZFYva1s8Zy3JABXPf4Ijbv3Bt2HJF+p7vpnH9190qixT8/2Grcvbm7NzazAqLX5v118H6NwQJv04G2lT3vAz51ZD+CJJvMtFR+dt5Edu9r5quPLaK1VVM8RQ5FLAd3zwZeJbpS5znAXDP7bAzvPQqoBe4xs9fN7FdmlguUuvuG4DkbgdLDiy7JbExpPt/++PH8fVkt92iKp8ghieXg7reBU9x9hrtfAkwC/iuG16UBJwG3ufuJwG4OGtbx6Nk4HXbXzOxyM6sys6ra2toYPk6SzUWTh/PB40q56Zm3WLx+e9hxRPqNWAp/iru3XyilLsbX1RAdFpob7D9O9Itgk5kNAQhuO1yExd3vdPcKd68oLi6O4eMk2bRN8RyYk87Mhxewp1FTPEViEUsBf9bMnjOzS83sUuBp4E/dvcjdNwJrzWxs0DQNWALM4t0pojOAJw85tUigMDeDH50zgXc27+J//6SFY0ViEcvF1q8zs08D7w+a7nT338f4/lcBD5lZBlANXEb0y+ZRM/s8sJrocQORw3bamGK+cNoo7vrHSv7lmBLOPF6HjUS60m3hD7wMNBEdj3811jd39wXAe67+QrT3L9Jjvvrhsbz8Th1f/90iJpSdRsmArLAjifRZsczqOYdosf8shzarR6TXZKZFV/FsaGzm2scWaoqnSBdiGeP/Foc3q0ekVx1dksd3Pn4C/1i+hbtfXhl2HJE+K56zekR63fmThvHhE0q56dm3eHOdpniKdORwZ/U8E99YIofHzLjx0+MpzM1g5sOva4qnSAe6Lfzufh1wBzA+2O5096/FO5jI4RqUm8GPz5lI9Zbd/M/TmuIpcrCYhmzc/Ql3v8bdrwGeNLML45xL5IicenQRl59Wzm/mruG5xRvDjiPSp3S1LPMAM/uGmf3CzD5kUVcSnY+vuffS5137obGMGzqA63+3iE07tIqnSJuuevwPAGOBN4B/B14iulDbp9x9ei9kEzkiGWkp/PS8E9nb1MqXH17AnOo6lm3ayeade2lqaQ07nkhoujqBq9zd3wdgZr8CNgDD3V1dJ+k3Rhfn8b1PnsDXfreI2XfWHfBYfmYag3IzGJSTHty2be32c9MZlJNBYW4GA3PSyUxLDeknEek5XRX+prY77t5iZjUq+tIfnXPKME4ZVcj6+j1sa2hk2+5GtjU0sXV3I/UNjWwN7r+zeRf1DU3s2tf55SZyM1IZ2O6L4PghA/jKmceQla4vBOk/uir8E8xsR3DfgOxg34iuqDwg7ulEesioolxGFeXG9Nx9zS1sb2hia0Mj23Y3Rb8s2n1hRG8b2bq7kTv+Xs38Ndu465IKBuZkxPmnEOkZnRZ+d1cXRpJSZloqJQNSY1rv56mF67n20YV8+rZXuO+ySQwrzOmFhCJHRmfgihyBT0w4igf/fTJ1uxr5t1++wqKa+rAjiXRLhV/kCE0aVcjv/qOSzLQUzr1jDn95a1PYkUS6pMIv0gOOLsnn91dMZXRJLv9+XxW/mbsm7EginVLhF+khJflZPHJ5JacfU8w3f/8GNz/3FtHLSov0LSr8Ij0oNzONX11SwXmnDOPWl1ZwzaMLaWzWyWLSt8R6BS4RiVFaagr/9+n3UTYomx8+v4xNO/Zy+8UnMyArPexoIoB6/CJxYWZcecYYfnzOBF5duZWzb5vN+vo9YccSAVT4ReLq0yeVce9lk1hXv4dP//IVlm7Y0f2LROJMhV8kzt4/pojHvlQJwNm3z+Yfy2tDTiTJToVfpBccN2QAv79iKmWDsrnsntd4fF5N2JEkianwi/SSIQXZPPqlSiaXF/LVxxbysxeXa7qnhEKFX6QXDchK555LJ/HpE4fy4xeW8Y0n3tC1AaTXaTqnSC/LSEvhR+dMYOigbH7+l3fYsH0vt154EnmZ+nWU3qH/aSIhMDOu/dBYjhqYzbf/8Cbn3jGbey49JaYVQcPi7mzYvpcVtbtYsXkXK2p3U71lF3W7GvnwCYM5f9JwBhf03fzyLusPY4wVFRVeVVUVdgyRuHjprc1c8Zv5DMrJ4L7PncLRJfmh5tnb1MLKLbuDAh+9rd6yi+ra3TQ0tux/Xn5mGuUleWSlpfDqqq2kmHHmcaVcXDmCqaMjmFmIP4UAmNk8d694T7sKv0j43qjZzmX3vkZjcwt3XVLB5PJIXD/P3dmyqzFa3A8q8DXb9tC+LAwdmM3okjxGF+cyujgvupXkUpyXub+4r6lr4KFXV/NYVQ1bdzdSXpTLBZOHc/bJwyjI0RnLYVHhF+nj1m5t4NJ7XmXt1j2cc0oZmWmpGNDWcTYzDMAguIcZ+59zcBttzw/a3KFm2x6qt0SHanbsffcSk1npKZQX5b2nwI8qyiU7I/ZrMu1tauGZNzfw4Jw1zFu9jaz0FD4x/igumjKCCcMGHuG/kBwqFX6RfqC+oZGZDy9g3uptuDsO+3vfjuMOHt15bxu85zUHKx2QGRT49r33PIYMyCIlpWeHZpas38GDc1fzh9fX0dDYwviyAi6aPIJPTDjqkL5M5PCp8IskKfd3vxxSe7i4x2Ln3ib+8Po6HpizmmWbdjEgK43PnFzGRVNGMLo4r9fzJBMVfhEJlbvz2qptPDBnNc++uYGmFmfq6AgXTRnBmceXkp6q04p6mgq/iPQZtTv38WjVWn4zdw3r6vdQkp/JeZOGc/6kYQwpyA47XsJQ4ReRPqel1fnr25t5cM5q/rqslhQzPnhcCRdNGcGpo4t6/LhDsums8Mf1BC4zWwXsBFqAZnevMLNC4BFgJLAKOMfdt8Uzh4j0TakpxrTjSpl2XClrtzbw0Nw1PFq1lucWb2LowGxOPTpC5egIleVFOjmsB8W1x3dpTqgAAAlhSURBVB8U/gp339Ku7QfAVne/0cyuBwa5+9e7eh/1+EWSx77mFp55YyN/emMDc1duZfueJgBGFeUGXwIRppRHKM7PDDlp3xfKUE8nhf9t4F/dfYOZDQH+6u5ju3ofFX6R5NTS6izdsIM51XXMXlHH3JVb2bUvev7BmJI8po6O/kUweVSEQbkZIafte8Iq/CuBbURnkt3h7neaWb27DwweN2Bb2/5Br70cuBxg+PDhJ69evTpuOUWkf2huaeXN9TuYvaKO2dV1vLZyK3uaWjCDYwcPoLI8wtTRESaVF+oax4RX+Ie6+zozKwFeAK4CZrUv9Ga2zd0HdfU+6vGLSEcam1tZVFO//4tg3upt7GtuJcVg3NACKsujfxGcMrKQ3CRc/TT0WT1m9l1gF/AFNNQjInGwt6mF19fUM7u6jjkr6nh97TaaWpy0FGN8WQGVoyMcO3gAkbwMivIyieRmMDAnI5QT23pDr8/qMbNcIMXddwb3PwTcAMwCZgA3BrdPxiuDiCSXrPTU6AHg0RE4E/Y0tlC1euv+vwhu/1s1La0HdnZTDAblZBDJyyCSm3nAl0Jh0FaUl0EkL/pYfmZav195NJ5/+5QCvw/+gdKA37j7s2b2GvComX0eWA2cE8cMIpLEsjNSOW1MMaeNKQZg975m1tXvYcuufdTtaqRu1z7qdjdGt6Bt8fodbNm1j53tFrFrLyM1hcLc4IsiL5Oi4P7IolzGluYzpjSfguy+fXwhboXf3auBCR201wHT4vW5IiKdyc1M45jSfI4p7f6aB/uaW9i2uyn6JRF8MWzd3ciWg74wqmt3UbtzH/ua372E5uABWYwpzWNs8FnHDM5nTElenznO0DdSiIj0MZlpqQwuSI3pxLHWVmf99j0s27STZZt2sWzjTpZt3skDc1Yf8IVQNih7/xfPMaV5HFOaz9EleWSl9+5qpSr8IiJHKCXFKBuUQ9mgHM44tnR/e0urs3ZrA29v2snyTTt5O/hS+MfyWppaoscaUgxGRHIZU5LH2MHRoaKxpfmMKsolIy0+C9ep8IuIxElqijGyKJeRRbl8+ITB+9ubWlpZtWU3yzbtavelsJMX39q8/+BzWooxqiiX2y46maNLenb5ahV+EZFelp6awpjgQPDHGLK/fW9TC9W1u1m+eSdvb4wOGxXl9fwZySr8IiJ9RFZ6KscfNYDjjxoQ18/RlQ9ERJKMCr+ISJJR4RcRSTIq/CIiSUaFX0Qkyajwi4gkGRV+EZEko8IvIpJkeu1CLEfCzGqJLuF8OIqALd0+q+/oT3mVNX76U97+lBX6V94jzTrC3YsPbuwXhf9ImFlVR1eg6av6U15ljZ/+lLc/ZYX+lTdeWTXUIyKSZFT4RUSSTDIU/jvDDnCI+lNeZY2f/pS3P2WF/pU3LlkTfoxfREQOlAw9fhERaUeFX0QkySR04Tezs8zsbTN7x8yuDztPZ8xsmJm9ZGZLzGyxmc0MO1N3zCzVzF43sz+GnaU7ZjbQzB43s7fMbKmZVYadqTNm9pXg/8CbZvZbM+v+St+9yMzuNrPNZvZmu7ZCM3vBzJYHt4PCzNheJ3lvDv4vLDKz35vZwDAztukoa7vHrjUzN7OinvishC38ZpYK3Ap8BDgeON/Mjg83VaeagWvd/XhgCnBFH87aZiawNOwQMfop8Ky7HwtMoI/mNrOhwNVAhbuPA1KB88JN9R73Amcd1HY98KK7jwFeDPb7int5b94XgHHuPh5YBnyjt0N14l7emxUzGwZ8CFjTUx+UsIUfmAS84+7V7t4IPAxMDzlTh9x9g7vPD+7vJFqYhoabqnNmVgZ8DPhV2Fm6Y2YFwOnArwHcvdHd68NN1aU0INvM0oAcYH3IeQ7g7n8Hth7UPB24L7h/H/CpXg3VhY7yuvvz7t4c7M4Byno9WAc6+bcF+AnwNaDHZuIkcuEfCqxtt19DHy6mbcxsJHAiMDfcJF26heh/xNawg8RgFFAL3BMMTf3KzHLDDtURd18H/JBoz24DsN3dnw83VUxK3X1DcH8jUBpmmEP0OeCZsEN0xsymA+vcfWFPvm8iF/5+x8zygN8BX3b3HWHn6YiZfRzY7O7zws4SozTgJOA2dz8R2E3fGorYLxgbn070y+ooINfMLgo31aHx6PzwfjFH3My+RXSY9aGws3TEzHKAbwLf6en3TuTCvw4Y1m6/LGjrk8wsnWjRf8jdnwg7TxdOBT5pZquIDp+dYWYPhhupSzVAjbu3/QX1ONEvgr7og8BKd6919ybgCWBqyJliscnMhgAEt5tDztMtM7sU+Dhwoffdk5lGE+0ELAx+38qA+WY2+EjfOJEL/2vAGDMbZWYZRA+SzQo5U4fMzIiOQS919x+Hnacr7v4Ndy9z95FE/03/4u59tlfq7huBtWY2NmiaBiwJMVJX1gBTzCwn+D8xjT56IPogs4AZwf0ZwJMhZumWmZ1FdKjyk+7eEHaezrj7G+5e4u4jg9+3GuCk4P/0EUnYwh8cvLkSeI7oL8+j7r443FSdOhW4mGjveUGwfTTsUAnkKuAhM1sETAS+H3KeDgV/lTwOzAfeIPr72aeWFzCz3wKzgbFmVmNmnwduBM40s+VE/2q5McyM7XWS9xdAPvBC8Lt2e6ghA51kjc9n9d2/ckREJB4StscvIiIdU+EXEUkyKvwiIklGhV9EJMmo8IuIJBkVfhHAzFraTaVd0JOruZrZyI5WXBQJS1rYAUT6iD3uPjHsECK9QT1+kS6Y2Soz+4GZvWFmr5rZ0UH7SDP7S7Cm+4tmNjxoLw3WeF8YbG1LLqSa2V3BWvvPm1l2aD+UJD0VfpGo7IOGes5t99h2d38f0TM+bwnafg7cF6zp/hDws6D9Z8Df3H0C0TWB2s4WHwPc6u4nAPXAZ+L884h0SmfuigBmtsvd8zpoXwWc4e7VwUJ6G909YmZbgCHu3hS0b3D3IjOrBcrcfV+79xgJvBBcqAQz+zqQ7u7/L/4/mch7qccv0j3v5P6h2Nfufgs6viYhUuEX6d657W5nB/df4d3LIl4I/CO4/yLwH7D/usQFvRVSJFbqdYhEZZvZgnb7z7p725TOQcHKnvuA84O2q4he1es6olf4uixonwncGays2EL0S2ADIn2IxvhFuhCM8Ve4+5aws4j0FA31iIgkGfX4RUSSjHr8IiJJRoVfRCTJqPCLiCQZFX4RkSSjwi8ikmT+PytynU5gxnlDAAAAAElFTkSuQmCC\n", + "text/plain": [ + "
" ] - }, + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "plot_the_loss_curve(history.epoch, history.history['loss'])" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "wQV6zkKXrwR0" + }, + "source": [ + "#### **Summary of hyperparameter tuning**\n", + "Most machine learning problems require a lot of hyperparameter tuning. Unfortunately, we can't provide concrete tuning rules for every model. Lowering the learning rate can help one model converge efficiently but make another model converge much too slowly. You must experiment to find the best set of hyperparameters for your dataset. That said, here are a few rules of thumb:\n", + "\n", + "* Training loss should steadily decrease, steeply at first, and then more slowly until the slope of the curve reaches or approaches zero.\n", + "* If the training loss does not converge, train for more epochs.\n", + "* If the training loss decreases too slowly, increase the learning rate. Note that setting the learning rate too high may also prevent training loss from converging.\n", + "* If the training loss varies wildly (that is, the training loss jumps around), decrease the learning rate.\n", + "* Lowering the learning rate while increasing the number of epochs or the batch size is often a good combination.\n", + "* Setting the batch size to a very small batch number can also cause instability. First, try large batch size values. Then, decrease the batch size until you see degradation.\n", + "* For real-world datasets consisting of a very large number of examples, the entire dataset might not fit into memory. In such cases, you'll need to reduce the batch size to enable a batch to fit into memory.\n", + "\n", + "Remember: the ideal combination of hyperparameters is data dependent, so you must always experiment and verify." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "3wYL0qp4fTv-" + }, + "source": [ + "#### 5. Make a Prediction\n", + "Making a prediction is the final step in the life-cycle. It is why we wanted the model in the first place.\n", + "\n", + "It requires you have new data for which a prediction is required, e.g. where you do not have the output values.\n", + "\n", + "From an API perspective, you simply call a function to make a prediction of a class label, probability, or numerical value: whatever you designed your model to predict.\n", + "\n", + "We have our new test data located at the given github location:\n", + "\n", + "https://raw.githubusercontent.com/dphi-official/Datasets/master/Boston_Housing/Testing_set_boston.csv\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": 37, + "metadata": { + "colab": {}, + "colab_type": "code", + "id": "s-fWHtrMe8U_" + }, + "outputs": [], + "source": [ + "# Load new test data\n", + "new_test_data = pd.read_csv('https://raw.githubusercontent.com/dphi-official/Datasets/master/Boston_Housing/Testing_set_boston.csv')" + ] + }, + { + "cell_type": "code", + "execution_count": 38, + "metadata": { + "colab": {}, + "colab_type": "code", + "collapsed": true, + "id": "crRYu0YtiO5G", + "jupyter": { + "outputs_hidden": true + } + }, + "outputs": [ { - "cell_type": "markdown", - "metadata": { - "id": "c-SLnxZxp02g", - "colab_type": "text" - }, - "source": [ - "#### Resources\n", - "* [https://machinelearningmastery.com/tensorflow-tutorial-deep-learning-with-tf-keras/](https://machinelearningmastery.com/tensorflow-tutorial-deep-learning-with-tf-keras/)\n", - "* [https://heartbeat.fritz.ai/linear-regression-using-keras-and-python-7cee2819a60c](https://heartbeat.fritz.ai/linear-regression-using-keras-and-python-7cee2819a60c)\n", - "* Google Machine Learning Crash Course" + "data": { + "text/plain": [ + "array([[39.66324 ],\n", + " [43.31839 ],\n", + " [29.805103],\n", + " [33.429066],\n", + " [29.351913],\n", + " [31.77466 ],\n", + " [27.770433],\n", + " [22.308249],\n", + " [34.840096],\n", + " [32.91533 ],\n", + " [31.574955],\n", + " [29.515196],\n", + " [ 5.115693],\n", + " [30.366613],\n", + " [33.39492 ],\n", + " [32.128086],\n", + " [28.585308],\n", + " [16.318932],\n", + " [46.921223],\n", + " [29.528286],\n", + " [35.769264],\n", + " [38.481598],\n", + " [20.160679],\n", + " [35.7884 ],\n", + " [25.344767],\n", + " [24.63024 ],\n", + " [33.256073],\n", + " [17.512663],\n", + " [30.827238],\n", + " [32.01975 ],\n", + " [33.973213],\n", + " [37.094193],\n", + " [33.19159 ],\n", + " [36.728428],\n", + " [22.18063 ],\n", + " [19.019093],\n", + " [41.219284],\n", + " [33.28828 ],\n", + " [29.050995],\n", + " [29.733736],\n", + " [24.730371],\n", + " [42.054348],\n", + " [52.406197],\n", + " [29.199072],\n", + " [37.230362],\n", + " [30.594894],\n", + " [24.732462],\n", + " [31.43057 ],\n", + " [30.73032 ],\n", + " [43.010677],\n", + " [34.514824],\n", + " [40.435 ],\n", + " [28.227936],\n", + " [38.73451 ],\n", + " [44.685932],\n", + " [32.565456],\n", + " [31.990492],\n", + " [45.578693],\n", + " [34.317436],\n", + " [26.043503],\n", + " [34.25898 ],\n", + " [45.767868],\n", + " [37.232857],\n", + " [18.633627],\n", + " [36.220154],\n", + " [25.56301 ],\n", + " [30.540886],\n", + " [37.112633],\n", + " [42.305576],\n", + " [17.573175],\n", + " [27.817085],\n", + " [38.25915 ],\n", + " [17.583002],\n", + " [35.442856],\n", + " [32.206924],\n", + " [13.453775],\n", + " [35.237286],\n", + " [51.817265],\n", + " [25.641064],\n", + " [23.654373],\n", + " [33.467278],\n", + " [18.00766 ],\n", + " [35.221016],\n", + " [20.974365],\n", + " [36.543056],\n", + " [35.558086],\n", + " [23.818836],\n", + " [36.955254],\n", + " [37.976223],\n", + " [33.435806],\n", + " [34.25633 ],\n", + " [10.545762],\n", + " [35.710716],\n", + " [26.300406],\n", + " [35.644325],\n", + " [35.838055],\n", + " [35.85991 ],\n", + " [ 6.538336],\n", + " [17.469563],\n", + " [16.151329],\n", + " [30.548754],\n", + " [33.002274]], dtype=float32)" ] + }, + "execution_count": 38, + "metadata": {}, + "output_type": "execute_result" } - ] -} \ No newline at end of file + ], + "source": [ + "# make a prediction\n", + "model.predict(new_test_data)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "UuoFhNRrn623" + }, + "source": [ + "\n", + "**Congratulations! You have successfully build your first deep learning model and predicted the output (i.e. MEDV) of new test data.**" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "c-SLnxZxp02g" + }, + "source": [ + "#### Resources\n", + "* [https://machinelearningmastery.com/tensorflow-tutorial-deep-learning-with-tf-keras/](https://machinelearningmastery.com/tensorflow-tutorial-deep-learning-with-tf-keras/)\n", + "* [https://heartbeat.fritz.ai/linear-regression-using-keras-and-python-7cee2819a60c](https://heartbeat.fritz.ai/linear-regression-using-keras-and-python-7cee2819a60c)\n", + "* Google Machine Learning Crash Course" + ] + } + ], + "metadata": { + "colab": { + "collapsed_sections": [], + "name": "DL_Day4: Linear_Regression with tf.keras - Beginners.ipynb", + "provenance": [] + }, + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.5" + } + }, + "nbformat": 4, + "nbformat_minor": 4 +}