Keras Image Classifier on AWS Sage-Maker

I still remember I used to not like Neural Networks subject in my engineering course but after spending 7 years into the software industry and seeing various use cases I decided to give it a go again. Yeah, I know you may call it a desire to solve business problems or just an interest in technology.

Recently after spending nearly 6 months, I got my Professional certification on AI & Machine Learning. I really like how the course was designed and I got a chance to know more deeply about ‘Deep Learning & Neural Networks’ with libraries such as PyTorch, Google’s TensorFlow, and Keras. Among all these Keras is the one which interests me more and I guess why because their tag line is “Keras is an API designed for human beings, not machines.”

Today, we will build a simple fully custom image classifier, using a transfer learning mechanism that you can very easily lift and try using yourself for your own projects. Whether you’re totally new to the platform or have played with the Sage-Maker before, you should walk away from this article equipped with the knowledge necessary to run your own custom machine learning jobs and endpoints on top of AWS (Because I really like AWS!)

The objective of this article is to show how to create and train a simple Keras model to classify an image between two classes of X-Ray: Normal and Pneumonia.

Data

An Amazon S3 bucket is a public cloud storage resource available in AWS. Amazon S3 buckets, which are similar to file folders, store objects, which consist of data and its descriptive metadata. Once data is stored in buckets it can be used by any other AWS service.

We are using a medical dataset of Chest X-rays images (Pneumonia). You can download it from here. This dataset has approximately 6000 images. Here we are just unzipping and structure a bit our dataset, ready to be uploaded to Amazon S3. We have created a bucket and sent everything on it.

The bucket now contains the data in the respective folder and it is ready for training.

The Model

Transfer learning is a machine learning method where a model developed for a task is reused as the starting point for a model on the second one. This means reducing the time required for training a new model. Instead of training your model from scratch, you can use a modified pre-trained model and continue training it with your dataset. That’s why it’s called transfer learning. For our purpose, I have used a ResNet-50 convolutional neural network which will serve as a transfer learning mechanism. You can also use other models such as VGG, InceptionV3, etc.

The model now is ready for training.

Training & Deployment

SageMaker provides high-level abstractions aiming to simplify training and validation. Some of these implementations like Estimators, models are for Tensorflow, Chainer, Pytorch. The training starts with an estimator which is the main object to start a training job. It uses an entry script that we created above with a bunch of other parameters. When you create a Jupyter notebook instance on Sagemaker you also create an IAM role, so there is no need to bother from a role perspective.

When the estimator.fit() method is executed, Sagemaker will create a training job based on the instance specified.

Once the training is done, it will be automatically registered in AWS SageMaker’s Traning jobs list. Now that the model is prepared and trained, the last remaining step is to serve it to deploy and use for predictions.

Deployment is pretty straightforward with the estimator.deploy() method which will host the model using the Amazon SageMaker hosting services. The endpoint will be available to use and is visible under the ‘Endpoint tab’.

Inference

Now that we have got the Inference endpoint, the final step that is remaining is the inference using a certain image(s). For this first, the endpoint has to be invoked using the Predictor method then the required image to test is fed into Predictor.predict() method which will in turn tell us the image that was fed.

Since labels and class associations were generated by the ImageDataGenerator which takes folders in sorted order i.e two classes Normal and Pneumonia. The ‘predictions’ field gives the result which indicates that the X-ray image we gave is a Pneumonia at 95.6% accuracy.

When the endpoint is not needed anymore, it can be deleted either with delete_endpoint() method or using SageMaker Interface on AWS.

Conclusion

We saw how to create, train, and deploy a Keras model with Amazon SageMaker then run that model online as an endpoint You are now hopefully well-equipped for running your own machine learning model builds through AWS!

Want the complete code? It’s all available on GitHub here