![\"Fashion](\"https://tensorflow.org/images/fashion-mnist-sprite.png\"\n","){kind=icon}
\n"," \n","
| \n"," \n"," |
| \n"," Figure 1. Fashion-MNIST samples (by Zalando, MIT License). \n"," |
| Label | \n","Class | \n","
|---|---|
| 0 | \n","T-shirt/top | \n","
| 1 | \n","Trouser | \n","
| 2 | \n","Pullover | \n","
| 3 | \n","Dress | \n","
| 4 | \n","Coat | \n","
| 5 | \n","Sandal | \n","
| 6 | \n","Shirt | \n","
| 7 | \n","Sneaker | \n","
| 8 | \n","Bag | \n","
| 9 | \n","Ankle boot | \n","
![\"Fully-connected](\"https://www.dropbox.com/s/6a05qtkgmlih6s4/fc_mnist.png?raw=1\")
"]},{"cell_type":"code","metadata":{"id":"UEzRx5l576Bd","executionInfo":{"status":"ok","timestamp":1730172314204,"user_tz":-420,"elapsed":599,"user":{"displayName":"Akadej Udomchaiporn","userId":"12826764406638924459"}}},"source":["from keras.models import Sequential\n","from keras.layers import *\n","\n","# Feed-forward neural network\n","model = Sequential()\n","\n","# Layer 1 - Hidden\n","# Number of neurons (or units) is 128.\n","# Activation function is rectified linear unit (ReLU).\n","model.add(Dense(128, activation='relu'))\n","\n","# Layer 2 - Hidden\n","# Number of neurons (or units) is 128.\n","# Activation function is rectified linear unit (ReLU).\n","model.add(Dense(128, activation='relu'))\n","\n","# Layer 3 - Output\n","# Number of neurons is 10 (i.e., number of output classes).\n","# Activation function is softmax.\n","# Note: the softmax function is commonly used in\n","# the classification problem to normalize a\n","# K-dimensional output vector into a probability\n","# distribution of classes.\n","num_classes = 10\n","model.add(Dense(num_classes, activation='softmax'))"],"execution_count":9,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"zQCt2mvz8ROs"},"source":["# Train a Model\n","\n","In this section, we will first define several parameters that will be used during the training.\n","\n","* `epochs`: the number of training epochs (one epoch means the model has seen the entire training samples one times).\n","* `batch_size`: the number of examples per one training step.\n","* `learning_rate`: a hyperparameter that defines the adjustment in the weights of our network with respect to the loss gradient.\n"]},{"cell_type":"code","metadata":{"id":"YjNa-rS28XqD","executionInfo":{"status":"ok","timestamp":1730172352486,"user_tz":-420,"elapsed":421,"user":{"displayName":"Akadej Udomchaiporn","userId":"12826764406638924459"}}},"source":["epochs = 20\n","batch_size = 256\n","learning_rate = 0.01"],"execution_count":10,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"UHeeybzy-taV"},"source":["## Loss Function\n","\n","Before we train a model, we need to specify the **loss function**, `loss`, that will be used to quantify the error between the predicted and the target classes. As we would like to train our model to differentiate among 10 handwritten digits in MNIST dataset, a loss function that we can use is *cross-entropy*. Cross-entropy is a measure of how different your predicted distribution is from the target distribution (see [Wikipedia](https://en.wikipedia.org/wiki/Cross_entropy) for more details).\n","\n","In this exercise, we will use the cross-entropy.\n","\n","Keras also provides many other loss functions for other problems as well. You can read more [here](https://keras.io/losses/)."]},{"cell_type":"code","metadata":{"id":"P8OGmVLq-RNm","executionInfo":{"status":"ok","timestamp":1730172357003,"user_tz":-420,"elapsed":438,"user":{"displayName":"Akadej Udomchaiporn","userId":"12826764406638924459"}}},"source":["# Cross-entropy loss\n","loss = keras.losses.categorical_crossentropy"],"execution_count":11,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"LvObkjaY-32N"},"source":["## Optimizer\n","\n","Another component that we need to specify before the training is the **optimizer**, `optimizer`. The optimizers that are commonly used to train deep learning models are Stochastic Gradient Descent (SGD), Adam, RMSProp, Adadelta, etc. The list of optimizers provided by Keras can be found [here](https://keras.io/optimizers/).\n","\n","Here we will use SGD."]},{"cell_type":"code","metadata":{"id":"VTq6EcE1-4fo","executionInfo":{"status":"ok","timestamp":1730172363034,"user_tz":-420,"elapsed":1476,"user":{"displayName":"Akadej Udomchaiporn","userId":"12826764406638924459"}}},"source":["# Stochastic gradient descent (SGD)\n","optimizer = tf.keras.optimizers.SGD(learning_rate)"],"execution_count":12,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"VzG3YUL8_UdH"},"source":["## Compile Keras Model\n","\n","Next, we configures the model for training by calling."]},{"cell_type":"code","metadata":{"id":"CmCb0PgA_TJQ","executionInfo":{"status":"ok","timestamp":1730172367293,"user_tz":-420,"elapsed":411,"user":{"displayName":"Akadej Udomchaiporn","userId":"12826764406638924459"}}},"source":["model.compile(\n"," loss=loss,\n"," optimizer=optimizer,\n"," metrics=['accuracy'])"],"execution_count":13,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"rkB6NPNuW0nE"},"source":["## Train a model"]},{"cell_type":"markdown","metadata":{"id":"QP4czTe_W7HC"},"source":["We are now ready to train our model. Let's start feeding the data to train the model and it will learn to classify digits."]},{"cell_type":"code","metadata":{"id":"LJP7WcsO_bZA","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1730172388242,"user_tz":-420,"elapsed":14444,"user":{"displayName":"Akadej Udomchaiporn","userId":"12826764406638924459"}},"outputId":"2d467ca4-03d1-4326-c6ff-102ab997ed32"},"source":["hist = model.fit(\n"," x_train, y_train,\n"," batch_size=batch_size,\n"," epochs=epochs,\n"," verbose=1)"],"execution_count":14,"outputs":[{"output_type":"stream","name":"stdout","text":["Epoch 1/20\n","\u001b[1m235/235\u001b[0m \u001b[32mββββββββββββββββββββ\u001b[0m\u001b[37m\u001b[0m \u001b[1m3s\u001b[0m 4ms/step - accuracy: 0.2782 - loss: 2.1393\n","Epoch 2/20\n","\u001b[1m235/235\u001b[0m \u001b[32mββββββββββββββββββββ\u001b[0m\u001b[37m\u001b[0m \u001b[1m1s\u001b[0m 2ms/step - accuracy: 0.7634 - loss: 1.1343\n","Epoch 3/20\n","\u001b[1m235/235\u001b[0m \u001b[32mββββββββββββββββββββ\u001b[0m\u001b[37m\u001b[0m \u001b[1m1s\u001b[0m 2ms/step - accuracy: 0.8473 - loss: 0.6442\n","Epoch 4/20\n","\u001b[1m235/235\u001b[0m \u001b[32mββββββββββββββββββββ\u001b[0m\u001b[37m\u001b[0m \u001b[1m1s\u001b[0m 2ms/step - accuracy: 0.8695 - loss: 0.4980\n","Epoch 5/20\n","\u001b[1m235/235\u001b[0m \u001b[32mββββββββββββββββββββ\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 2ms/step - accuracy: 0.8819 - loss: 0.4334\n","Epoch 6/20\n","\u001b[1m235/235\u001b[0m \u001b[32mββββββββββββββββββββ\u001b[0m\u001b[37m\u001b[0m \u001b[1m1s\u001b[0m 2ms/step - accuracy: 0.8929 - loss: 0.3934\n","Epoch 7/20\n","\u001b[1m235/235\u001b[0m \u001b[32mββββββββββββββββββββ\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 2ms/step - accuracy: 0.8980 - loss: 0.3677\n","Epoch 8/20\n","\u001b[1m235/235\u001b[0m \u001b[32mββββββββββββββββββββ\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 2ms/step - accuracy: 0.9025 - loss: 0.3499\n","Epoch 9/20\n","\u001b[1m235/235\u001b[0m \u001b[32mββββββββββββββββββββ\u001b[0m\u001b[37m\u001b[0m \u001b[1m1s\u001b[0m 2ms/step - accuracy: 0.9084 - loss: 0.3294\n","Epoch 10/20\n","\u001b[1m235/235\u001b[0m \u001b[32mββββββββββββββββββββ\u001b[0m\u001b[37m\u001b[0m \u001b[1m1s\u001b[0m 2ms/step - accuracy: 0.9112 - loss: 0.3138\n","Epoch 11/20\n","\u001b[1m235/235\u001b[0m \u001b[32mββββββββββββββββββββ\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 2ms/step - accuracy: 0.9118 - loss: 0.3105\n","Epoch 12/20\n","\u001b[1m235/235\u001b[0m \u001b[32mββββββββββββββββββββ\u001b[0m\u001b[37m\u001b[0m \u001b[1m1s\u001b[0m 2ms/step - accuracy: 0.9147 - loss: 0.3025\n","Epoch 13/20\n","\u001b[1m235/235\u001b[0m \u001b[32mββββββββββββββββββββ\u001b[0m\u001b[37m\u001b[0m \u001b[1m1s\u001b[0m 2ms/step - accuracy: 0.9195 - loss: 0.2881\n","Epoch 14/20\n","\u001b[1m235/235\u001b[0m \u001b[32mββββββββββββββββββββ\u001b[0m\u001b[37m\u001b[0m \u001b[1m1s\u001b[0m 2ms/step - accuracy: 0.9206 - loss: 0.2809\n","Epoch 15/20\n","\u001b[1m235/235\u001b[0m \u001b[32mββββββββββββββββββββ\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 2ms/step - accuracy: 0.9195 - loss: 0.2836\n","Epoch 16/20\n","\u001b[1m235/235\u001b[0m \u001b[32mββββββββββββββββββββ\u001b[0m\u001b[37m\u001b[0m \u001b[1m1s\u001b[0m 2ms/step - accuracy: 0.9237 - loss: 0.2674\n","Epoch 17/20\n","\u001b[1m235/235\u001b[0m \u001b[32mββββββββββββββββββββ\u001b[0m\u001b[37m\u001b[0m \u001b[1m1s\u001b[0m 2ms/step - accuracy: 0.9241 - loss: 0.2656\n","Epoch 18/20\n","\u001b[1m235/235\u001b[0m \u001b[32mββββββββββββββββββββ\u001b[0m\u001b[37m\u001b[0m \u001b[1m1s\u001b[0m 3ms/step - accuracy: 0.9264 - loss: 0.2599\n","Epoch 19/20\n","\u001b[1m235/235\u001b[0m \u001b[32mββββββββββββββββββββ\u001b[0m\u001b[37m\u001b[0m \u001b[1m1s\u001b[0m 2ms/step - accuracy: 0.9300 - loss: 0.2476\n","Epoch 20/20\n","\u001b[1m235/235\u001b[0m \u001b[32mββββββββββββββββββββ\u001b[0m\u001b[37m\u001b[0m \u001b[1m1s\u001b[0m 2ms/step - accuracy: 0.9326 - loss: 0.2402\n"]}]},{"cell_type":"markdown","metadata":{"id":"__m07PF3Xh5A"},"source":["# Apply the Trained Model on MNIST images\n","\n","Once we have finished the model training, we can now apply it to MNIST images."]},{"cell_type":"code","metadata":{"id":"I3ap6GsLXv41","colab":{"base_uri":"https://localhost:8080/","height":827},"executionInfo":{"status":"ok","timestamp":1730172433759,"user_tz":-420,"elapsed":1402,"user":{"displayName":"Akadej Udomchaiporn","userId":"12826764406638924459"}},"outputId":"020227a2-6819-456d-8b17-44d61e3da060"},"source":["# Sample two MNIST images from the training set\n","test_imgs = x_test[0:2]\n","\n","# Plot images\n","for idx in range(len(test_imgs)):\n"," f, ax = plt.subplots(1)\n"," f.set_figheight(5)\n"," f.set_figwidth(5)\n"," ax.imshow(test_imgs[idx].reshape((img_h, img_w)), cmap=\"gray\")\n"," ax.tick_params(\n"," axis='both',\n"," which='both',\n"," bottom=False, top=False,left=False, right=False,\n"," labelbottom=False, labeltop=False, labelleft=False, labelright=False)"],"execution_count":15,"outputs":[{"output_type":"display_data","data":{"text/plain":["| \n", + " | CI Type | \n", + "Lower Bound | \n", + "Upper Bound | \n", + "
|---|---|---|---|
| 0 | \n", + "Mean (ΟΜ1) | \n", + "5.4307 | \n", + "5.8110 | \n", + "
| 1 | \n", + "Mean (ΟΜ2) | \n", + "5.4246 | \n", + "5.8171 | \n", + "
| 2 | \n", + "Mean (ΟΜ3) | \n", + "5.3973 | \n", + "5.8444 | \n", + "
| 3 | \n", + "Median (ΟΜ1) | \n", + "5.6247 | \n", + "6.1013 | \n", + "
| 4 | \n", + "Median (ΟΜ2) | \n", + "5.6170 | \n", + "6.1090 | \n", + "
| 5 | \n", + "Median (ΟΜ3) | \n", + "5.5828 | \n", + "6.1432 | \n", + "
| \n", + " | CI Type | \n", + "Lower Bound | \n", + "Upper Bound | \n", + "
|---|---|---|---|
| 0 | \n", + "Mean (ΟΜ1) | \n", + "3.3794 | \n", + "7.8623 | \n", + "
| 1 | \n", + "Mean (ΟΜ2) | \n", + "3.3067 | \n", + "7.9350 | \n", + "
| 2 | \n", + "Mean (ΟΜ3) | \n", + "2.9852 | \n", + "8.2565 | \n", + "
| 3 | \n", + "Median (ΟΜ1) | \n", + "3.6170 | \n", + "8.1090 | \n", + "
| 4 | \n", + "Median (ΟΜ2) | \n", + "3.5441 | \n", + "8.1819 | \n", + "
| 5 | \n", + "Median (ΟΜ3) | \n", + "3.2219 | \n", + "8.5041 | \n", + "
| 6 | \n", + "IQR | \n", + "2.9573 | \n", + "8.2097 | \n", + "
| \n", + " | Country name | \n", + "year | \n", + "Life Ladder | \n", + "Log GDP per capita | \n", + "Social support | \n", + "Healthy life expectancy at birth | \n", + "Freedom to make life choices | \n", + "Generosity | \n", + "Perceptions of corruption | \n", + "Positive affect | \n", + "Negative affect | \n", + "Subregion | \n", + "Continent | \n", + "
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | \n", + "Afghanistan | \n", + "2023 | \n", + "1.446 | \n", + "NaN | \n", + "0.368 | \n", + "55.2 | \n", + "0.228 | \n", + "NaN | \n", + "0.738 | \n", + "0.261 | \n", + "0.460 | \n", + "Southern Asia | \n", + "Asia | \n", + "
| 13 | \n", + "Botswana | \n", + "2023 | \n", + "3.332 | \n", + "9.673 | \n", + "0.701 | \n", + "55.0 | \n", + "0.741 | \n", + "-0.264 | \n", + "0.814 | \n", + "0.657 | \n", + "0.247 | \n", + "Sub-Saharan Africa | \n", + "Africa | \n", + "
| 115 | \n", + "Sri Lanka | \n", + "2023 | \n", + "3.602 | \n", + "9.364 | \n", + "0.790 | \n", + "67.4 | \n", + "0.754 | \n", + "0.050 | \n", + "0.922 | \n", + "0.709 | \n", + "0.353 | \n", + "Southern Asia | \n", + "Asia | \n", + "
| \n", + " | const | \n", + "Log GDP per capita | \n", + "Social support | \n", + "Healthy life expectancy at birth | \n", + "Freedom to make life choices | \n", + "Generosity | \n", + "Perceptions of corruption | \n", + "Positive affect | \n", + "Negative affect | \n", + "
|---|---|---|---|---|---|---|---|---|---|
| count | \n", + "118.0 | \n", + "118.000000 | \n", + "118.000000 | \n", + "118.000000 | \n", + "118.000000 | \n", + "118.000000 | \n", + "118.000000 | \n", + "118.000000 | \n", + "118.000000 | \n", + "
| mean | \n", + "1.0 | \n", + "9.495669 | \n", + "0.790627 | \n", + "65.237288 | \n", + "0.796364 | \n", + "0.034373 | \n", + "0.722347 | \n", + "0.654653 | \n", + "0.293610 | \n", + "
| std | \n", + "0.0 | \n", + "1.149838 | \n", + "0.131170 | \n", + "5.492634 | \n", + "0.113688 | \n", + "0.162590 | \n", + "0.173567 | \n", + "0.106431 | \n", + "0.088618 | \n", + "
| min | \n", + "1.0 | \n", + "7.076000 | \n", + "0.398000 | \n", + "52.200000 | \n", + "0.452000 | \n", + "-0.268000 | \n", + "0.184000 | \n", + "0.344000 | \n", + "0.114000 | \n", + "
| 25% | \n", + "1.0 | \n", + "8.612500 | \n", + "0.695750 | \n", + "60.700000 | \n", + "0.735250 | \n", + "-0.072500 | \n", + "0.663250 | \n", + "0.578250 | \n", + "0.229250 | \n", + "
| 50% | \n", + "1.0 | \n", + "9.636000 | \n", + "0.837500 | \n", + "66.100000 | \n", + "0.817500 | \n", + "0.022000 | \n", + "0.767500 | \n", + "0.667000 | \n", + "0.283000 | \n", + "
| 75% | \n", + "1.0 | \n", + "10.470250 | \n", + "0.894250 | \n", + "69.650000 | \n", + "0.877000 | \n", + "0.134250 | \n", + "0.844500 | \n", + "0.738750 | \n", + "0.357500 | \n", + "
| max | \n", + "1.0 | \n", + "11.676000 | \n", + "0.979000 | \n", + "74.600000 | \n", + "0.965000 | \n", + "0.590000 | \n", + "0.948000 | \n", + "0.843000 | \n", + "0.516000 | \n", + "
| \n", + " | const | \n", + "Social support | \n", + "Freedom to make life choices | \n", + "Generosity | \n", + "Perceptions of corruption | \n", + "Positive affect | \n", + "Negative affect | \n", + "Healthy life scaled | \n", + "Log GDP scaled | \n", + "
|---|---|---|---|---|---|---|---|---|---|
| count | \n", + "118.0 | \n", + "118.000000 | \n", + "118.000000 | \n", + "118.000000 | \n", + "118.000000 | \n", + "118.000000 | \n", + "118.000000 | \n", + "118.000000 | \n", + "118.000000 | \n", + "
| mean | \n", + "1.0 | \n", + "0.790627 | \n", + "0.796364 | \n", + "0.034373 | \n", + "0.722347 | \n", + "0.654653 | \n", + "0.293610 | \n", + "0.582022 | \n", + "0.526015 | \n", + "
| std | \n", + "0.0 | \n", + "0.131170 | \n", + "0.113688 | \n", + "0.162590 | \n", + "0.173567 | \n", + "0.106431 | \n", + "0.088618 | \n", + "0.245207 | \n", + "0.249965 | \n", + "
| min | \n", + "1.0 | \n", + "0.398000 | \n", + "0.452000 | \n", + "-0.268000 | \n", + "0.184000 | \n", + "0.344000 | \n", + "0.114000 | \n", + "0.000000 | \n", + "0.000000 | \n", + "
| 25% | \n", + "1.0 | \n", + "0.695750 | \n", + "0.735250 | \n", + "-0.072500 | \n", + "0.663250 | \n", + "0.578250 | \n", + "0.229250 | \n", + "0.379464 | \n", + "0.334022 | \n", + "
| 50% | \n", + "1.0 | \n", + "0.837500 | \n", + "0.817500 | \n", + "0.022000 | \n", + "0.767500 | \n", + "0.667000 | \n", + "0.283000 | \n", + "0.620536 | \n", + "0.556522 | \n", + "
| 75% | \n", + "1.0 | \n", + "0.894250 | \n", + "0.877000 | \n", + "0.134250 | \n", + "0.844500 | \n", + "0.738750 | \n", + "0.357500 | \n", + "0.779018 | \n", + "0.737880 | \n", + "
| max | \n", + "1.0 | \n", + "0.979000 | \n", + "0.965000 | \n", + "0.590000 | \n", + "0.948000 | \n", + "0.843000 | \n", + "0.516000 | \n", + "1.000000 | \n", + "1.000000 | \n", + "