Short summary:
In this article, I will explain how to create a solution for image classification for the 5 classes with the best result : loss: 0.1172 — accuracy: 0.9570 — val_loss: 0.2223 — val_accuracy: 0.9125. Code for this article available here
So Let’s start :)
The steps:
###Data preparation###
###Transfer learning###
### Fine tuning###
###Show Result###
If you want just to look at the notebook or just run code please click here
First, I want to run this code (like below) in google colab, but you can run this code anywhere and it should work.
1 Go to the colab
2 Download this dataset to your PC (like on screenshot)
3 Upload file which you download (above) to colab (like on screenshot)
4 Unzip this file (just add code like on screenshot)
!jar xvf /content/flowers-recognition.zip5 Sort this dataset (prepare data)
Why we need it, if dataset is already prepared?
This dataset does not have the same number of pictures for each class.
This will cause problems during training of the model.
Therefore, it is necessary to make the same number of pictures for the train dataset and for the test dataset.
The train folder will have 80 percent of the total number of pictures. The test folder will have 20 percent of the total number of pictures.
So, you can run these commands to sort this dataset.
!git clone https://raw.githubusercontent.com/oleksandr-g-rock/flower_classification/main/flowers_sort.sh
!chmod 777 flowers_sort.sh
!cat flowers_sort.sh
!bash flowers_sort.sh6 Ok. When our data is prepared, we can run modules.
from tensorflow.keras.applications.inception_v3 import InceptionV3
from tensorflow.keras.preprocessing import image
from tensorflow.keras.models import Model, load_model
from tensorflow.keras.callbacks import ModelCheckpoint, ReduceLROnPlateau
from tensorflow.keras.layers import Dense, GlobalAveragePooling2D, Dropout
from tensorflow.keras.layers import BatchNormalization, Activation
from tensorflow.python.keras.optimizers import Adam, RMSprop
from tensorflow.python.keras.preprocessing.image import ImageDataGenerator
import numpy as np
import matplotlib.pyplot as plt
import tensorflow as tf
from numpy import loadtxt
import tensorflow.keras
from PIL import Image, ImageOps
%matplotlib inline7 Set image size & batch size
image_size = 256
batch_size = 328 Create image generator
Why we need this image generator?
Because we don’t have many images. This generator will create new images from one image like this:
# folders with train dir & val dir
train_dir = '/content/flowers/train/'
test_dir = '/content/flowers/val/'
input_shape = (image_size, image_size, 3)train_datagen = ImageDataGenerator(rescale=1. / 255,
samplewise_center=True,
samplewise_std_normalization=True,
horizontal_flip = True,
vertical_flip = True,
height_shift_range= 0.05,
width_shift_range=0.2,
rotation_range=5,
shear_range = 0.1,
fill_mode = 'reflect',
zoom_range=0.2)
9 Check our generator for one image
img_path = '/content/flowers/train/daisy/174131220_c853df1287.jpg'
img = image.load_img(img_path, target_size=(image_size, image_size))
x = image.img_to_array(img)
x = x.reshape((1,) + x.shape)
i = 0
for batch in train_datagen.flow(x, batch_size=1):
plt.figure(i)
imgplot = plt.imshow(image.array_to_img(batch[0]))
i += 1
if i % 6 == 0:
break
plt.show()10 Set settings for train and test generator. It’s mean that we will use dataset with images from “image generator” to train our model, but for test we will use images from dataset without “image generator”.
#Setting train generator
train_generator = train_datagen.flow_from_directory(
train_dir,
target_size=(image_size, image_size),
batch_size=batch_size,
class_mode='categorical')#Setting test generator without data augmentation
test_datagen = ImageDataGenerator(rescale=1. / 255)
test_generator = test_datagen.flow_from_directory(
test_dir,
target_size=(image_size, image_size),
batch_size=batch_size,
class_mode='categorical')#Print count classes
train_generator.class_indices
11 Data Visualization
source for graph in “train” folder:
from sklearn.utils.class_weight import compute_class_weight
cls_train = train_generator.classes
from collections import OrderedDict
classes = list(train_generator.class_indices.keys())
num_values = []
unique, counts = np.unique(cls_train, return_counts=True)
valdict=OrderedDict(zip(unique, counts))
for i in range(5):
num_values.append(valdict[i])
plt.figure(figsize=(30,10))
x = np.arange(len(num_values))
xlabel = classes
plt.bar(x, num_values)
plt.xticks(x, xlabel)
plt.show()source for graph in “test” folder:
from sklearn.utils.class_weight import compute_class_weight
cls_test = test_generator.classes
from collections import OrderedDict
classes = list(test_generator.class_indices.keys())
num_values = []
unique, counts = np.unique(cls_test, return_counts=True)
valdict=OrderedDict(zip(unique, counts))
for i in range(5):
num_values.append(valdict[i])
plt.figure(figsize=(30,10))
x = np.arange(len(num_values))
xlabel = classes
plt.bar(x, num_values)
plt.xticks(x, xlabel)
plt.show()source for display of a round graph in “train” folder:
import matplotlib.pyplot as plt
goal_types = list(train_generator.class_indices.keys())
unique, counts = np.unique(cls_train, return_counts=True)
goals = [(counts)]
colors = ['y','r','b','g','b']
plt.pie(goals, labels = goal_types, colors=colors ,shadow = True, explode = (0.05, 0.05, 0.05, 0.05, 0.05), autopct = '%1.1f%%')
plt.axis('equal')
plt.show()source for display of a round graph in “test” folder:
import matplotlib.pyplot as plt
goal_types = list(test_generator.class_indices.keys())
unique, counts = np.unique(cls_test, return_counts=True)
goals = [(counts)]
colors = ['y','r','b','g','b']
plt.pie(goals, labels = goal_types, colors=colors ,shadow = True, explode = (0.05, 0.05, 0.05, 0.05, 0.05), autopct = '%1.1f%%')
plt.axis('equal')
plt.show()###Transfer learning###
We will use pretrained model Inception V3, because this model is well suited for this task.
12 Download pretrained model Inception V3
pretrained_model = InceptionV3(weights='imagenet', include_top=False)pretrained_model.summary()
13 “Froze” all weights of pretrained models. We do this to add new layers to our model.
pretrained_model.trainable = Falsepretrained_model.summary()
14 Create main layers in the end of inceptionV3 model.
x = pretrained_model.outputx = GlobalAveragePooling2D()(x)x = Dense(1024)(x)x = BatchNormalization()(x)x = Activation('relu')(x)x = Dropout(0.5)(x)x = Dense(512)(x)x = Activation('relu')(x)predictions = Dense(5, activation='softmax')(x) # because we have 5 classes imagesmodel = Model(inputs=pretrained_model.input, outputs=predictions)
15 View all model before training
model.summary()16 Train a composite neural network based on InceptionV3. We will train our model for 240 epochs and save only the best epoch to file “inceptionv3_best.h5”.
Also, we will use “learning_rate_reduction” and optimizer=’adam’ and metrics=[‘accuracy’].
model.compile(loss='categorical_crossentropy',
optimizer='adam',
metrics=['accuracy'])filepath="inceptionv3_best.h5"
checkpoint = ModelCheckpoint(filepath, monitor='val_loss', verbose=1,
save_best_only=True, mode='min')
learning_rate_reduction = ReduceLROnPlateau(monitor='val_accuracy',
patience=10,
verbose=1,
factor=0.5,
min_lr=0.00001)
callbacks_list = [checkpoint, learning_rate_reduction]steps_per_epoch = 40
history = model.fit_generator(
train_generator,
steps_per_epoch=steps_per_epoch,
validation_data=test_generator,
validation_steps=5,
callbacks=callbacks_list,
epochs=240,
verbose=2)
17 Visualize the history of network learning: accuracy
plt.plot(history.history['accuracy'], label='The score of correct answers on the training set')
plt.plot(history.history['val_accuracy'], label='The score of correct answers on the val set')
plt.xlabel('Epoch')
plt.ylabel('Score correct answers')
plt.legend()
plt.show()
18 Visualize the history of network learning: loss
plt.plot(history.history['loss'], label='The score of loss on the training set')
plt.plot(history.history['val_loss'], label='The score of val_loss on the val set')
plt.xlabel('Epoch')
plt.ylabel('Score loss')
plt.legend()
plt.show()###Fine tuning convolutional layers Inception V3###
19 Download the best weights and Defrost the last 2 blocks of Inception V3
model.load_weights("inceptionv3_best.h5")pretrained_model.trainable = False
for layer in model.layers[:290]:
layer.trainable = False
for layer in model.layers[290:]:
layer.trainable = True
20 Compile a network with defrosted layers. Then, start learning
model.compile(loss='categorical_crossentropy',
optimizer='adam',
metrics=['accuracy'])filepath="inceptionv3_fine_tuned.h5"
checkpoint = ModelCheckpoint(filepath, monitor='val_loss', verbose=1, save_best_only=True, mode='min')
callbacks_list = [checkpoint, learning_rate_reduction]steps_per_epoch = 32
history = model.fit_generator(
train_generator,
steps_per_epoch=steps_per_epoch,
validation_data=test_generator,
validation_steps=5,
callbacks=callbacks_list,
epochs=200,
verbose=2)
21 Again Visualize the history of network learning: accuracy and loss
plt.plot(history.history['accuracy'], label='The score of correct answers on the training set')
plt.plot(history.history['val_accuracy'], label='The score of correct answers on the val set')
plt.xlabel('Epoch')
plt.ylabel('Score correct answers')
plt.legend()
plt.show()plt.plot(history.history['loss'], label='The score of loss on the training set')
plt.plot(history.history['val_loss'], label='The score of val_loss on the val set')
plt.xlabel('Epoch')
plt.ylabel('Score loss')
plt.legend()
plt.show()
###Testing of the model and download of the best weights###
Congratulations!
You have just created your first model. Now we will evaluate the quality of its work.
22 Load the best model
model = tensorflow.keras.models.load_model(‘/content/inceptionv3_fine_tuned.h5’)23 Upload a random photo from the internet with flowers and prepare it for our model
!wget https://upload.wikimedia.org/wikipedia/commons/thumb/a/a3/Iglesia_de_Nuestra_Se%C3%B1ora_de_La_Blanca%2C_Cardej%C3%B3n%2C_Espa%C3%B1a%2C_2012-09-01%2C_DD_02_cropped.JPG/2880px-Iglesia_de_Nuestra_Se%C3%B1ora_de_La_Blanca%2C_Cardej%C3%B3n%2C_Espa%C3%B1a%2C_2012-09-01%2C_DD_02_cropped.JPGimg_path = '/content/2880px-Iglesia_de_Nuestra_Señora_de_La_Blanca,_Cardejón,_España,_2012-09-01,_DD_02_cropped.JPG'img = image.load_img(img_path, target_size=(image_size, image_size))
plt.imshow(img)
plt.show()x = image.img_to_array(img)
x /= 255
x = np.expand_dims(x, axis=0)
24 Prediction
prediction = model.predict(x)predictionclasses = np.argmax(prediction, axis = 1)print(classes)
25 Show Result
‘sunflower’: 3
Result:
We created image classification for 5 classes with the best result.
