Imagery data must be prepped prior to it being leveraged as the foundation for modelling in image classification tasks.

One aspect of prepping image data is the scaling of pixel values, like normalizing the values to the range 0-1, centring, standardization, and more.

How do you select a good, or even ideal, pixel scaling method for your image classification or computer vision modelling task?

In this guide, you will find out how to select a pixel scaling strategy for image classification with deep learning strategies.

After going through this guide, you will be aware of:

• A process for selecting a pixel scaling strategy leveraging experimentation and empirical outcomes on a particular dataset.
• How to implement standard pixel scaling strategies for prepping image data for modelling.
• How to work through a case study for selecting a pixel scaling strategy for a standard image classification problem.

Tutorial Summarization

This guide is subdivided into six portions, which are:

1. Process for choosing a pixel scaling strategy
2. Select dataset: MNIST Image classification
3. Select model: Convolutional neural network
4. Select pixel scaling strategies
5. Run experiment
6. Analyse outcomes

Process for selecting a pixel scaling strategy

Provided a new image classification activity, what pixel scaling strategies should be leveraged?

There are several ways to find a solution to this question, for instance:

• Leverage strategies reportedly leveraged for similar problems in research papers.
• Leverage heuristics from blog posts, courses, or books.
• Leverage your favourite technique.
• Leverage the simplest technique.

Rather, we recommend leveraging experimentation in order to find out what functions best for your particular dataset.

This can be accomplished leveraging the following procedure:

1. Step 1: Select Dataset – This might be the complete training dataset or a small subset. The idea is to finish the experiments swiftly and obtain an outcome.
2. Step 2: Select Model – Develop a model that is skilful, but not necessarily the ideal model for the platform. Some parallel prototyping of models might be needed.
3. Step 3: Select Pixel Scaling Methods – Detail 3-5 data preparation schemes for evaluation of your problem.
4. Step 4: Execute Experiment – Run the experiments in such a way that the outcomes are robust and representative, ideally repeat every experiment several times.
5. Step 5: Analyse outcomes – Contrast strategies both in terms of the rate of learning and mean performance across repeated experiments.

The experimental strategy will leverage a non-optimized model and probably a subset of training data, both of which might add noise to the decision you must take.

Therefore, you are searching for a signal that one data prep scheme for your imagery is obviously better than the others; if this is not the scenario for your dataset, then the simplest (least computationally complex) strategy should be leveraged, like pixel normalization.

An obvious signal of a superior pixel scaling strategy might be observed in one of two ways:

• Quicker learning: Learning curves obviously display that a model learns quicker with a provided data preparation scheme.
• Better precision: Mean model performance obviously displays better precision with a provided data prep scheme.

Now that we have a process for selecting a pixel scaling strategy for imagery data, let’s observe an instance. We will leverage the MNIST image classification task fit with a CNN and assess an array of traditional pixel scaling strategies.

Step 1: Select Dataset: MNIST Image Classification

The MNIST problem, or MNIST in summary, is an image classification problem consisted of 70,000 images of handwritten digits.

The objective of the problem is to categorize a provided image of a handwritten digit as an integer from 0 to 9. As such, it is a multiclass image classification problem.

It is a conventional dataset for assessing machine learning and deep learning algorithms. Best outcomes for the dataset are approximately 99.79% precise, or an error rate of approximately 0.21% (for instance, lesser than 1%)

This dataset is furnished as part of the Keras library and can be automatically downloaded (if required) and loaded into memory by a call to the keras.datasets.mnist.load_data() function.

The function gives back two tuples: one for the training inputs and outputs and one for the evaluation inputs and outputs. For instance:

We can load the MNIST dataset and summarize it.

The complete instance is detailed below:

Running the instance first loads the dataset into memory. Then the shape of the training and evaluation datasets is reported.

We can observe that all images are 28 by 28 pixels with a singular channel for grayscale imagery. There are 60,000 images for the training dataset and 10,000 for the evaluation dataset.

We can additionally observe that pixel values are integer values between 0 and 255 and that the mean and standard deviation of the pixel values are similar amongst the dual datasets.

The dataset is comparatively minimal, we will leverage the complete train and evaluation dataset.

Now that we are acquainted with MNIST and how to load the dataset, let’s review some pixel scaling strategies.

Step 2: Select Model – Convolutional Neural Network

We will leverage a convolutional neural network to assess the differing pixel scaling strategies.

A CNN is predicted/expected to have very good performance on this problem, even though the model selected for this experiment does not have to feature good performance or best for the problem. Rather, it ought to be skilful (better than random) and must facilitate the impact of differing data prep schemes to be differentiated in terms of rate of learning and/or model performance.

As such, the model must have adequate capacity to go about learning the problem.

We will illustrate the baseline model on the MNIST problem.

To start with, the dataset ought to be loaded and the shape of the train and evaluation test dataset expanded to include a channel dimension, set to one as we just have a singular black and white channel.

Then, we will normalize the pixel values for this instance and one hot encode the target values, needed for multiclass classification.

The model is defined as a convolutional layer followed by a maximum pooling layer; this combo is repeated again, then the filter maps are flattened, interpreted by a completely connected layer and followed by an output layer.

The ReLU activation function is leveraged for hidden layers and the softmax activation function is leveraged for the output layer. Adequate filter maps and nodes are mentioned to furnish sufficient capacity to learn the problem.

The Adam variation of stochastic gradient descent is leveraged to identify the model weights. The categorical cross entropy loss function is leveraged, needed for multi-class classification, and classification precision is monitored during the course of training.

# compile model

model.compile(optimizer=’adam’, loss=’categorical_crossentropy’, metrics=[‘accuracy’])

The model is fit for five training epochs and a large batch size of 128 images is leveraged.

# fit model

model.fit(trainX, trainY, epochs=5, batch_size=128)

Upon fitting, the model is assessed on the test dataset.

The complete instance is detailed below and will easily run on the CPU in approximately a minute.

# baseline cnn model for the mnist problem

from keras.datasets import mnist

from keras.utils import to_categorical

from keras.models import Sequential

from keras.layers import Conv2D

from keras.layers import MaxPooling2D

from keras.layers import Dense

from keras.layers import Flatten

# load dataset

(trainX, trainY), (testX, testY) = mnist.load_data()

# reshape dataset to have a single channel

width, height, channels = trainX.shape[1], trainX.shape[2], 1

trainX = trainX.reshape((trainX.shape[0], width, height, channels))

testX = testX.reshape((testX.shape[0], width, height, channels))

# normalize pixel values

trainX = trainX.astype(‘float32’) / 255

testX = testX.astype(‘float32′) / 255

# one hot encode target values

trainY = to_categorical(trainY)

testY = to_categorical(testY)

# define model

model = Sequential()

model.add(Conv2D(32, (3, 3), activation=’relu’, input_shape=(width, height, channels)))

model.add(MaxPooling2D((2, 2)))

model.add(Conv2D(64, (3, 3), activation=’relu’))

model.add(MaxPooling2D((2, 2)))

model.add(Flatten())

model.add(Dense(64, activation=’relu’))

model.add(Dense(10, activation=’softmax’))

# compile model

model.compile(optimizer=’adam’, loss=’categorical_crossentropy’, metrics=[‘accuracy’])

# fit model

model.fit(trainX, trainY, epochs=5, batch_size=128)

# evaluate model

_, acc = model.evaluate(testX, testY, verbose=0)

print(acc)

Running the instance  displays that the model is capable of learning the problem well and swiftly.

Your outcomes might demonstrate variance provided the stochastic nature of the algorithm or evaluation procedure, or variations in numerical accuracy. Consider running the instance a few times and contrast the average outcome.

As a matter of fact, the performance of the model on the evaluation dataset on this run is 99%, or 1% error rate. This is not state-of-the-art (by design), but is not awfully far from state of the art either.

Step 3: Select Pixel Scaling Strategies

Neural network models usually cannot be trained on raw pixel values, like pixel values in the range of 0 to 255.

The reason is that the network leverages a weighted sum of inputs, and for the network to both be stable and train efficiently, weights should be kept minimal.

Rather, the pixel values must be scaled before training. There are probably three primary strategies to scaling pixel values, which are:

• Normalization: pixel values are scaled to the range 0-1.
• Centring: the mean pixel value is removed from every pixel value have the outcome of a distribution of pixel values centred on a mean of zero.
• Standardization: the pixel values are scaled to a traditional Gaussian with a mean of zero and a standard deviation of one.

Conventionally, sigmoid activation functions were leveraged that sum to 0 (zero mean) were given preference. This might or might not still be the scenario with the wide adoption of ReLU and similar activation functions.

Further, in centring and standardization, the mean or mean and standard deviation can be calculated across a channel, an image, a mini-batch, or the complete training dataset. This might include additional variations on a selected scaling strategy that might be assessed.

Normalization is typically the default strategy as we can assume pixel values are always in the range of 0-255, making the process very simple and effective to implement.

Centring is typically promoted as the preferred strategy as it was leveraged in several popular papers, even though the mean can be calculated per image (global) or channel (local) and across the batch of images or the complete training dataset, and typically the process detailed in a paper does not mention precisely which variation was leveraged.

We will experiment with the three strategies detailed above, namely normalization, centring, and standardization. The mean for centring and the mean and standard deviation for standardization will be calculated throughout the entire training dataset.

Other variations you could look into include:

• Calculating stats for every channel (for colour images)
• Calculating stats for each image.
• Calculating stats for each batch.
• Normalizing upon centring or standardizing.

The instance below implements the three selected pixel scaling strategies and illustrate their impact on the MNIST dataset.

Running the instance first normalizes the dataset and reports the min, max, mean, and standard deviation for the train and evaluation dataset.

Your outcomes might demonstrate variance provided the stochastic nature of the algorithm or evaluation process, or variations in numerical accuracy. Consider running the instance a few times and contrast the average outcome.

This is subsequently repeated for the centring and standardization data prep schemes. The outcomes furnish evidence that the scaling processes are indeed implemented rightly.

Step 4: Run Experiment

After we have designed the dataset, the model, and the data prep schemes to assess, we are ready to define and execute the experiment.

Every model takes approximately a single minute to run on the CPU, so we don’t want for the experiment to take too long. We will assess each of the three data prep schemes and every scheme will be assessed 10 times, implying that approximately half an hour will be needed to finish the experiment on advanced hardware.

We can define a function to load the dataset afresh when required.

We can also give definition to a function to define and compile our model ready to fit on the problem.

We already have functions for prepping the pixel data for the train and test datasets.

Lastly, we can defined a function referred to as repeated_evaluation() that takes the name of the data prep function to call to prep the data and will load the dataset and repeatedly define the model, prep the dataset, fit and assess the model. It will give back a listing of precision scores that can be leveraged to summarize the performance of the model under the selected data prep scheme.

The repeated_evaluation() function can then be called for each of the three data prep schemes and the mean and standard deviation of model performance under the scheme can be reported.

We can additionally develop a box and whisker plot to summarize and contrast the distribution of precision scores for every scheme.

all_scores = list()

# normalization

scores = repeated_evaluation(prep_normalize)

print(‘Normalization: %.3f (%.3f)’ % (mean(scores), std(scores)))

all_scores.append(scores)

# center

scores = repeated_evaluation(prep_center)

print(‘Centered: %.3f (%.3f)’ % (mean(scores), std(scores)))

all_scores.append(scores)

# standardize

scores = repeated_evaluation(prep_standardize)

print(‘Standardized: %.3f (%.3f)’ % (mean(scores), std(scores)))

all_scores.append(scores)

# box and whisker plots of results

pyplot.boxplot(all_scores, labels=[‘norm’, ‘cent’, ‘stan’])

pyplot.show()

Connecting all of this together, the complete instance of running the experiment to contrast pixel scaling strategies on the MNIST dataset is detailed below.

Running the instance might take approximately half an hour on the CPU.

Your outcomes might demonstrate variance provided the stochastic nature of the algorithm or assessment procedure, or variations in numerical accuracy. Consider executing the instance a few times and contrast the average outcome.

The precision is reported for every repeated evaluation of the model and the mean and standard deviation of precision scores are repeated at the conclusion of every run.

> 0: 98.930

> 1: 98.960

> 2: 98.910

> 3: 99.050

> 4: 99.040

> 5: 98.800

> 6: 98.880

> 7: 99.020

> 8: 99.100

> 9: 99.050

Normalization: 0.990 (0.001)

> 0: 98.570

> 1: 98.530

> 2: 98.230

> 3: 98.110

> 4: 98.840

> 5: 98.720

> 6: 9.800

> 7: 98.170

> 8: 98.710

> 9: 10.320

Centered: 0.808 (0.354)

> 0: 99.150

> 1: 98.820

> 2: 99.000

> 3: 98.850

> 4: 99.140

> 5: 99.050

> 6: 99.120

> 7: 99.100

> 8: 98.940

> 9: 99.110

Standardized: 0.990 (0.001

Step 5: Analyse outcomes

For brevity, we will only observe model performance in the comparison of data prep schemes. An extension to this study would also look at learning rates under every pixel scaling strategy.

The outcomes of the experiments display that there is minimal or no difference (at the selected accuracy) between pixel normalization and standardization with the selected model on the MNIST dataset.

From these outcomes, we would leverage normalization over standardization on this dataset with this model due to the good outcomes and due to the simplicity of normalization contrasted to standardization.

These are good outcomes in that they display that the default heuristic to centre pixel values before modelling would not be good advice for this dataset.

Unfortunately, the box and whisker plot does not make a comparison amongst the spread of precision scores simple as some terrible outlier scores for the centring scaling strategy squash the distributions.

Extensions

This section details some ideas for extension of the tutorial that you might desire to explore.

• Batch-wise scaling: Update the study to calculate scaling stats per batch rather than across the complete training dataset and observe if that makes a difference to the selection of scaling strategy.
• Learning Curves: Update the study to gather a few learning curves for every data scaling strategy and contrast the rate of learning.
• CIFAR: Repeat the study on the CIFAR-10 dataset and include pixel scaling strategies that assist global (scale across all channels) and local (scaling per channel) strategies.

Further Reading

This section furnishes additional resources on the subject if you are seeking to delve deeper.

• MNIST database, Wikipedia
• MNIST, Classification datasets results
• MNIST database of handwritten digits, Keras API

Conclusion

In this guide, you found out how to select a pixel scaling strategy for image classification with deep learning strategies.

Particularly, you learned:

• A process for selecting a pixel scaling strategy leveraging experimentation and empirical outcomes on a particular dataset.
• How to implement standard pixel scaling strategies for prepping image data for modelling.
• How to work through a case study for selecting a pixel scaling strategy for a traditional image classification problem.