CIFAR10 image classification#
The CNN is composed of two layers of convolution with 64 kernels each, followed by two fully connected layers that have 128 and 10 units, respectively. Pooling layers are used to reduce the dimensions of the feature maps of convolution layers.
Parameter are also tuned • Increasing the number of epochs generally increases the accuracy and lowers the loss • Increasing the number of layers increases the accuracy • Increasing the number of neurons per layer also increases the accuracy
1raise SystemExit("Stop right there!");
An exception has occurred, use %tb to see the full traceback.
SystemExit: Stop right there!
Importing libraries and packages#
1# System
2import os
3
4# Mathematical operations and data manipulation
5import numpy as np
6
7# Modelling
8from tensorflow.keras.datasets import cifar10
9import tensorflow as tf
10import keras
11from tensorflow.keras.preprocessing.image import ImageDataGenerator
12from tensorflow.keras import layers
13
14# Plotting
15import matplotlib.pyplot as plt
16
17%matplotlib inline
18
19print("Tensorflow version:", tf.__version__)
20print("Keras version:", keras.__version__)
Tensorflow version: 2.4.1
Keras version: 2.4.3
1os.environ["TF_CPP_MIN_LOG_LEVEL"] = "2"
Loading dataset#
The CIFAR-10 dataset (Canadian Institute for Advanced Research) is composed of 60,000 images of 10 different classes: airplanes, cars, birds, cats, deer, dogs, frogs, horses, ships, and trucks.
1# Loading the dataset's train and test data
2(X_train, Y_train), (X_test, Y_test) = cifar10.load_data()
1print("X_train Shape: {}".format(X_train.shape))
2print("X_test Shape: {}".format(X_test.shape))
3print("Y_train Shape: {}".format(Y_train.shape))
4print("Y_test Shape: {}".format(Y_test.shape))
X_train Shape: (50000, 32, 32, 3)
X_test Shape: (10000, 32, 32, 3)
Y_train Shape: (50000, 1)
Y_test Shape: (10000, 1)
1# Reshaping 2D to 1D
2Y_train = Y_train.reshape(
3 -1,
4)
5Y_test = Y_test.reshape(
6 -1,
7)
Exploring dataset#
1classes = [
2 "airplane",
3 "automobile",
4 "bird",
5 "cat",
6 "deer",
7 "dog",
8 "frog",
9 "horse",
10 "ship",
11 "truck",
12]
1def plot_sample(x, y, index):
2 plt.figure(figsize=(15, 2))
3 plt.imshow(x[index])
4 plt.xlabel(classes[y[index]])
5
6
7plot_sample(X_train, Y_train, 6)
1for i in range(10):
2 plot_sample(X_train, Y_train, i)
Data generators#
1# Instantiating two ImageDataGenerator classes: all the image pixels
2# are in a range from 0 to 255, use data augmentation for train dataset
3data_gen_train = ImageDataGenerator(
4 rescale=1.0 / 255,
5 width_shift_range=0.1,
6 height_shift_range=0.1,
7 horizontal_flip=True,
8)
9data_gen_valid = ImageDataGenerator(rescale=1.0 / 255)
10
11batch_size = 16
12img_height = 32
13img_width = 32
14
15# Create data generators
16train_generator = data_gen_train.flow(X_train, Y_train, batch_size=batch_size)
17valid_generator = data_gen_valid.flow(X_test, Y_test, batch_size=batch_size)
Training base model#
1# Setting 8 as the seed for numpy and tensorflow
2np.random.seed(8)
3tf.random.set_seed(8)
1# Instantiate a tf.keras.Sequential() class
2model = tf.keras.Sequential(
3 [
4 layers.Conv2D(
5 32, 3, activation="relu", input_shape=(img_height, img_width, 3)
6 ),
7 layers.MaxPooling2D(pool_size=(2, 2)),
8 layers.Conv2D(64, 3, activation="relu"),
9 layers.MaxPooling2D(pool_size=(2, 2)),
10 layers.Dropout(0.5),
11 layers.Conv2D(128, 3, activation="relu"),
12 layers.MaxPooling2D(pool_size=(2, 2)),
13 layers.Dropout(0.5),
14 layers.Flatten(),
15 layers.Dense(128, activation="relu"),
16 layers.Dense(10, activation="softmax"),
17 ]
18)
1optimizer = tf.keras.optimizers.Adam(0.001)
1# Providing training parameters using the compile method
2model.compile(
3 loss="sparse_categorical_crossentropy",
4 optimizer=optimizer,
5 metrics=["accuracy"],
6)
1# Inspecting the model configuration using the summary function
2model.summary()
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 30, 30, 32) 896
_________________________________________________________________
max_pooling2d (MaxPooling2D) (None, 15, 15, 32) 0
_________________________________________________________________
conv2d_1 (Conv2D) (None, 13, 13, 64) 18496
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 6, 6, 64) 0
_________________________________________________________________
dropout (Dropout) (None, 6, 6, 64) 0
_________________________________________________________________
conv2d_2 (Conv2D) (None, 4, 4, 128) 73856
_________________________________________________________________
max_pooling2d_2 (MaxPooling2 (None, 2, 2, 128) 0
_________________________________________________________________
dropout_1 (Dropout) (None, 2, 2, 128) 0
_________________________________________________________________
flatten (Flatten) (None, 512) 0
_________________________________________________________________
dense (Dense) (None, 128) 65664
_________________________________________________________________
dense_1 (Dense) (None, 10) 1290
=================================================================
Total params: 160,202
Trainable params: 160,202
Non-trainable params: 0
_________________________________________________________________
1# Train the model by calling the fit method
2history = model.fit(
3 train_generator,
4 steps_per_epoch=len(X_train) // batch_size,
5 epochs=20,
6 validation_data=valid_generator,
7 validation_steps=len(X_test) // batch_size,
8)
Epoch 1/20
3125/3125 [==============================] - 226s 72ms/step - loss: 1.8943 - accuracy: 0.2881 - val_loss: 1.3835 - val_accuracy: 0.5015
Epoch 2/20
3125/3125 [==============================] - 231s 74ms/step - loss: 1.5147 - accuracy: 0.4474 - val_loss: 1.3152 - val_accuracy: 0.5396
Epoch 3/20
3125/3125 [==============================] - 225s 72ms/step - loss: 1.4232 - accuracy: 0.4845 - val_loss: 1.2126 - val_accuracy: 0.5869
Epoch 4/20
3125/3125 [==============================] - 233s 74ms/step - loss: 1.3581 - accuracy: 0.5071 - val_loss: 1.2017 - val_accuracy: 0.5908
Epoch 5/20
3125/3125 [==============================] - 232s 74ms/step - loss: 1.3336 - accuracy: 0.5223 - val_loss: 1.1047 - val_accuracy: 0.6105
Epoch 6/20
3125/3125 [==============================] - 225s 72ms/step - loss: 1.2782 - accuracy: 0.5466 - val_loss: 1.1320 - val_accuracy: 0.6090
Epoch 7/20
3125/3125 [==============================] - 210s 67ms/step - loss: 1.2562 - accuracy: 0.5509 - val_loss: 1.0624 - val_accuracy: 0.6397
Epoch 8/20
3125/3125 [==============================] - 212s 68ms/step - loss: 1.2436 - accuracy: 0.5578 - val_loss: 1.0339 - val_accuracy: 0.6516
Epoch 9/20
3125/3125 [==============================] - 214s 69ms/step - loss: 1.2178 - accuracy: 0.5694 - val_loss: 1.0233 - val_accuracy: 0.6498
Epoch 10/20
3125/3125 [==============================] - 216s 69ms/step - loss: 1.2072 - accuracy: 0.5698 - val_loss: 1.0720 - val_accuracy: 0.6296
Epoch 11/20
3125/3125 [==============================] - 210s 67ms/step - loss: 1.1815 - accuracy: 0.5811 - val_loss: 0.9665 - val_accuracy: 0.6697
Epoch 12/20
3125/3125 [==============================] - 217s 69ms/step - loss: 1.1762 - accuracy: 0.5846 - val_loss: 0.9531 - val_accuracy: 0.6753
Epoch 13/20
3125/3125 [==============================] - 208s 67ms/step - loss: 1.1564 - accuracy: 0.5912 - val_loss: 0.9342 - val_accuracy: 0.6794
Epoch 14/20
3125/3125 [==============================] - 237s 76ms/step - loss: 1.1562 - accuracy: 0.5945 - val_loss: 0.9413 - val_accuracy: 0.6929
Epoch 15/20
3125/3125 [==============================] - 230s 74ms/step - loss: 1.1439 - accuracy: 0.5976 - val_loss: 0.9294 - val_accuracy: 0.6883
Epoch 16/20
3125/3125 [==============================] - 236s 76ms/step - loss: 1.1401 - accuracy: 0.5958 - val_loss: 0.9286 - val_accuracy: 0.6898
Epoch 17/20
3125/3125 [==============================] - 224s 72ms/step - loss: 1.1248 - accuracy: 0.6043 - val_loss: 0.9105 - val_accuracy: 0.6912
Epoch 18/20
3125/3125 [==============================] - 224s 72ms/step - loss: 1.1183 - accuracy: 0.6038 - val_loss: 0.8786 - val_accuracy: 0.7110
Epoch 19/20
3125/3125 [==============================] - 215s 69ms/step - loss: 1.1168 - accuracy: 0.6086 - val_loss: 0.8851 - val_accuracy: 0.7119
Epoch 20/20
3125/3125 [==============================] - 210s 67ms/step - loss: 1.1166 - accuracy: 0.6069 - val_loss: 0.9053 - val_accuracy: 0.6867
Statistics base model#
1# Test the model by calling the evaluate() function
2evaluation = model.evaluate(X_test, Y_test)
313/313 [==============================] - 3s 8ms/step - loss: 168.2624 - accuracy: 0.3406
[168.26243591308594, 0.34060001373291016]
1def plotloss(history):
2 plt.plot(history.history["loss"])
3 plt.plot(history.history["val_loss"])
4 plt.plot(history.history["accuracy"])
5 plt.xlabel("Epochs")
6 plt.ylabel("Loss")
7 plt.legend(["Train", "Validation", "Accuracy"])
8 plt.show()
9
10
11plotloss(history)
Tuned model with batch normalization#
1model_b = tf.keras.Sequential(
2 [
3 layers.Conv2D(
4 64,
5 (4, 4),
6 activation="relu",
7 input_shape=(img_height, img_width, 3),
8 padding="same",
9 ),
10 layers.BatchNormalization(),
11 layers.Conv2D(
12 64,
13 (4, 4),
14 activation="relu",
15 input_shape=(img_height, img_width, 3),
16 padding="same",
17 ),
18 layers.BatchNormalization(),
19 layers.MaxPooling2D(pool_size=(2, 2)),
20 layers.Dropout(0.2),
21 layers.Conv2D(
22 128,
23 (4, 4),
24 activation="relu",
25 input_shape=(img_height, img_width, 3),
26 padding="same",
27 ),
28 layers.BatchNormalization(),
29 layers.Conv2D(
30 128,
31 (4, 4),
32 activation="relu",
33 input_shape=(img_height, img_width, 3),
34 padding="same",
35 ),
36 layers.BatchNormalization(),
37 layers.MaxPooling2D(pool_size=(2, 2)),
38 layers.Dropout(0.25),
39 layers.Conv2D(
40 128,
41 (4, 4),
42 activation="relu",
43 input_shape=(img_height, img_width, 3),
44 padding="same",
45 ),
46 layers.BatchNormalization(),
47 layers.Conv2D(
48 128,
49 (4, 4),
50 activation="relu",
51 input_shape=(img_height, img_width, 3),
52 padding="same",
53 ),
54 layers.BatchNormalization(),
55 layers.MaxPooling2D(pool_size=(2, 2)),
56 layers.Dropout(0.35),
57 layers.Flatten(),
58 layers.Dense(265, activation="relu"),
59 layers.BatchNormalization(),
60 layers.Dropout(0.5),
61 layers.Dense(10, activation="softmax"),
62 ]
63)
1model_b.compile(
2 loss="sparse_categorical_crossentropy",
3 optimizer=optimizer,
4 metrics=["accuracy"],
5)
1# Inspecting the model configuration using the summary function
2model_b.summary()
Model: "sequential_1"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d_3 (Conv2D) (None, 32, 32, 64) 3136
_________________________________________________________________
batch_normalization (BatchNo (None, 32, 32, 64) 256
_________________________________________________________________
conv2d_4 (Conv2D) (None, 32, 32, 64) 65600
_________________________________________________________________
batch_normalization_1 (Batch (None, 32, 32, 64) 256
_________________________________________________________________
max_pooling2d_3 (MaxPooling2 (None, 16, 16, 64) 0
_________________________________________________________________
dropout_2 (Dropout) (None, 16, 16, 64) 0
_________________________________________________________________
conv2d_5 (Conv2D) (None, 16, 16, 128) 131200
_________________________________________________________________
batch_normalization_2 (Batch (None, 16, 16, 128) 512
_________________________________________________________________
conv2d_6 (Conv2D) (None, 16, 16, 128) 262272
_________________________________________________________________
batch_normalization_3 (Batch (None, 16, 16, 128) 512
_________________________________________________________________
max_pooling2d_4 (MaxPooling2 (None, 8, 8, 128) 0
_________________________________________________________________
dropout_3 (Dropout) (None, 8, 8, 128) 0
_________________________________________________________________
conv2d_7 (Conv2D) (None, 8, 8, 128) 262272
_________________________________________________________________
batch_normalization_4 (Batch (None, 8, 8, 128) 512
_________________________________________________________________
conv2d_8 (Conv2D) (None, 8, 8, 128) 262272
_________________________________________________________________
batch_normalization_5 (Batch (None, 8, 8, 128) 512
_________________________________________________________________
max_pooling2d_5 (MaxPooling2 (None, 4, 4, 128) 0
_________________________________________________________________
dropout_4 (Dropout) (None, 4, 4, 128) 0
_________________________________________________________________
flatten_1 (Flatten) (None, 2048) 0
_________________________________________________________________
dense_2 (Dense) (None, 265) 542985
_________________________________________________________________
batch_normalization_6 (Batch (None, 265) 1060
_________________________________________________________________
dropout_5 (Dropout) (None, 265) 0
_________________________________________________________________
dense_3 (Dense) (None, 10) 2660
=================================================================
Total params: 1,536,017
Trainable params: 1,534,207
Non-trainable params: 1,810
_________________________________________________________________
1# Train the model by calling the fit method
2history_b = model_b.fit(
3 train_generator,
4 steps_per_epoch=len(X_train) // batch_size,
5 epochs=10,
6 validation_data=valid_generator,
7 validation_steps=len(X_test) // batch_size,
8)
Epoch 1/10
3125/3125 [==============================] - 818s 261ms/step - loss: 2.0562 - accuracy: 0.2889 - val_loss: 1.5065 - val_accuracy: 0.4551
Epoch 2/10
3125/3125 [==============================] - 853s 273ms/step - loss: 1.3733 - accuracy: 0.5116 - val_loss: 1.0796 - val_accuracy: 0.6221
Epoch 3/10
3125/3125 [==============================] - 866s 277ms/step - loss: 1.1067 - accuracy: 0.6141 - val_loss: 0.8818 - val_accuracy: 0.6986
Epoch 4/10
3125/3125 [==============================] - 787s 252ms/step - loss: 0.9181 - accuracy: 0.6862 - val_loss: 0.7880 - val_accuracy: 0.7316
Epoch 5/10
3125/3125 [==============================] - 725s 232ms/step - loss: 0.8071 - accuracy: 0.7275 - val_loss: 0.7703 - val_accuracy: 0.7391
Epoch 6/10
3125/3125 [==============================] - 746s 239ms/step - loss: 0.7414 - accuracy: 0.7481 - val_loss: 0.5918 - val_accuracy: 0.8030
Epoch 7/10
3125/3125 [==============================] - 810s 259ms/step - loss: 0.6739 - accuracy: 0.7743 - val_loss: 0.6155 - val_accuracy: 0.7929
Epoch 8/10
3125/3125 [==============================] - 828s 265ms/step - loss: 0.6351 - accuracy: 0.7880 - val_loss: 0.5448 - val_accuracy: 0.8157
Epoch 9/10
3125/3125 [==============================] - 866s 277ms/step - loss: 0.6016 - accuracy: 0.7979 - val_loss: 0.5519 - val_accuracy: 0.8145
Epoch 10/10
3125/3125 [==============================] - 843s 270ms/step - loss: 0.5680 - accuracy: 0.8099 - val_loss: 0.4912 - val_accuracy: 0.8332
Statistics tuned model with batch normalization#
1plotloss(history_b)
Saving#
1tf.keras.models.save_model(
2 model_b,
3 "./assets/cifar_tuned.model",
4 overwrite=True,
5 include_optimizer=True,
6)