Forem: Oluwafemi Tairu

AWS CloudFormation UPDATE_ROLLBACK_FAILED fix in production

Oluwafemi Tairu — Mon, 01 Nov 2021 12:37:34 +0000

Recently, while trying to deploy a serverless application, the pipeline failed thus putting the cloud formation stack in "UPDATE_ROLLBACK_FAILED" state. This is due to an error in your pipeline. In my own case, I was trying to use a base layer - "arn:aws:lambda:us-east-1:770693421928:layer:klayers-python38-pandas:35" which at the point of writing this article does not exist.

If you are familiar with AWS SAM Cli or cloud formation generally, you will know that you wont be able to deploy a new update until this status changes to "UPDATE_COMPLETE".

Now, you have two options facing this situation (at least that I am aware of):

Attempt to complete the update rollback process.
Delete the stack and create another.

The second option is almost perfect and simple except for the fact that you might not want to do this in production has is basically shutting down your application.

The first option however is much more advisable. Now, how do you complete the update rollback.

Using AWS CLI

You can make use of the aws cloudformation continue-update-rollback command to complete your update rollback.



aws cloudformation continue-update-rollback \
--stack-name STACK_NAME \
--resources-to-skip LIST_OF_RESOURCES

Where STACK_NAME is the name of your stack while LIST_OF_RESOURCES is the list of the logical IDs of resources you will like to skip. Please note that for LIST_OF_RESOURCES you have to specify resources that are in the UPDATE_FAILED state only.

Sample



aws cloudformation continue-update-rollback \
--stack-name eazido-app-stack \
--resources-to-skip CustomerApi PaymentApi

Using AWS Console

Visit your cloud formation area of the AWS console - https://console.aws.amazon.com/cloudformation/
Select the stack that requires rollback.
Under the "Stack Action" select "Continue update rollback" Note If this update rollback still fails or you want to skip some resources, then select "Advanced troubleshooting" on the "Continue update rollback" dialog and tick the resources you will like to skip.

Once this is done, your stack should now carry the "** UPDATE_ROLLBACK_COMPLETE**" status. If you try to deploy your updates again, it should work just fine.

However, remember how the update failed because of an error, you will need to identify and fix this before you deploy. In my own case, I was using a layer whose version does not exist. I had to update the layer by updating the stack template (we will talk about this in a bit). You can tell why your stack failed by checking the Status reason column under the Events tab of the failed stack.

Updating CloudFormation Template

Select the stack you want to update its template and select "View in designer"" under the "Template" tab.

Once the "View in designer"'s page loads, you should see a page similar to this.

You can get the template for a particular stack using AWS CLI's (get-template)[https://docs.aws.amazon.com/cli/latest/reference/cloudformation/get-template.html] command.



aws cloudformation get-template --stack-name STACK_NAME

You can either edit your template as json or Yaml file. Once you are done making changes, click on the file icon on the top left corner of the page and click "save". You have the option of saving on your laptop or in an s3 bucket. Once that is done, you can exit the page.

Creating and executing a Change Set

Select the stack you want to update it's template.
Under "Stack Actions" select "Create change set for current stack". You should see a page like this
Click on "Replace current template", select a template source (local or s3) based on where you saved the edited template.
Review the change set and Execute. This is will starts updating the AWS cloud formation stack and you can see the progress on the event tab.

You can update a particular stack using AWS CLI's (update-stack)[https://docs.aws.amazon.com/cli/latest/reference/cloudformation/update-stack.html] command.



aws cloudformation update-stack --stack-name STACK_NAME --template-url https://s3.amazonaws.com/sample/updated_template.template

Note: If you are getting AWS cli errors, you might want to take a look at the - AWS cli troubleshooting guide.

Deploying Machine learning models on Mobile with Tensorflow Lite and Firebase M.L Kit

Oluwafemi Tairu — Thu, 02 Apr 2020 19:56:37 +0000

In my last article, I shared how to deploy Machine learning models via an A.P.I.

In this article, I will share with you on how to deploy models using Tensorflow Lite and Firebase M.L Kit with Mobile Apps.

Deploying models via A.P.I is fine but there are multiple reasons why that might not suit your need or that of your organisation. Some of them include:

Poor internet connection or no access to an internet connection for users of your app.
Data Privacy.
Need for faster inference.

Whatever your reason might be, changing your model availability from A.P.I's to on-device won't cost you much time. Your previous codes can stay the same with only an addition of some few lines.

Convert your previously saved model

import tensorflow as tf

# Convert the model.
converter = tf.lite.TFLiteConverter.from_saved_model(MODEL_DIR)
tflite_model = converter.convert()
open("plant_ai.tflite", "wb").write(tflite_model)

Convert new models

import tensorflow as tf

# saving your deep learning model
model.save('plant_ai_model.h5')
# Convert the model.
converter = tf.lite.TFLiteConverter.from_keras_model(model)
tflite_model = converter.convert()
open("plant_ai.tflite", "wb").write(tflite_model)

For this article, I will be using a deep learning model for plant disease detection. I wrote about it here.
The modified code can be found here

Integrating with a mobile app

Our model is now ready for deployment to a mobile app. Bundling a model (in a .tflite format ) means you can perform prediction without the use of the internet which is good for some solutions. However, bundling your .tflite models with your apps this will increase app size.

For this article, I will be using Flutter but you can do the same using Java or Kotlin following Google's Documentation or Swift or Objective-C with Google's Documentation. Flutter is Google’s UI toolkit for building beautiful, natively compiled applications for mobile, web, and desktop from a single codebase - Flutter.

If you are new to Flutter, you can follow this link to get started with your first Flutter application. Your Flutter application structure should look like this after creation.

Next step is to install the necessary packages. This packages can be simply installed in Flutter by putting them in the pubspec.yaml file.

The following packages were used for this app:

mlkit: A Flutter plugin to use the Firebase ML Kit.
image_picker: A Flutter plugin for iOS and Android for picking images from the image library, and taking new pictures with the camera.
toast (Optional): A Flutter Toast plugin.

Next, you will have to add Firebase to your android or iOS project following the guide here for android and the guide here for iOS. Do make sure you add the "google-services.json" to your project folder.

Next, we would create a folder called assets in our project's root directory and then copy our .tflite model and label.text to that folder.
We would need to make this accessible in our Flutter project by modifying our pubspec.yaml file as shown below.

I will skip the interface design aspect has you can have any interface for your app.

Let's import the necessary packages (which we installed earlier)

import 'package:flutter/material.dart';
import 'dart:io';
import 'dart:typed_data';
import 'package:flutter/services.dart';
import 'package:image_picker/image_picker.dart';
import 'package:mlkit/mlkit.dart';
import 'package:toast/toast.dart';
import 'package:image/image.dart' as img;

We will declare some variables and constants first at the beginning of our code.

FirebaseModelInterpreter interpreter = FirebaseModelInterpreter.instance;
FirebaseModelManager manager = FirebaseModelManager.instance;

File selectedImageFile;
List<String> modelLabels = [];
Map<String, double> predictions = Map<String, double>();
int imageDim = 256;
List<int> inputDims = [1, 256, 256, 3];
List<int> outputDims = [1, 15];

The first two lines show the instantiation of the FirebaseModel Interpreter and Manager. The imageDim (height and width 256x256) is the size of the image required by the model.
Then the inputDims is the required input shape of our model while outputDims is the output shape. This value may vary depending on the model used. If someone else built the model, you can easily check for these by running the Python code below.

Load TFLite model and allocate tensors.

import tensorflow as tf

interpreter = tf.lite.Interpreter(model_path='plant_ai_lite_model.tflite')
# OR interpreter = tf.lite.Interpreter(model_content=tflite_model)
interpreter.allocate_tensors()

Get input and output tensors.

input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()

The value for input_details in this case is:

[{
    'name': 'conv2d_input',
    'index': 1,
    'shape': array([1, 256, 256, 3], dtype = int32),
    'dtype': <class 'numpy.float32'> ,
    'quantization': (0.0, 0)
}]

while that of the output_details is:

[{
    'name': 'Identity',
    'index': 0,
    'shape': array([1, 15], dtype = int32),
    'dtype': <class 'numpy.float32'> ,
    'quantization': (0.0, 0)
}]

Do take note of the value of 'shape' in the two results above. That is what determines the value for 'inputDims' and 'outputDims' above.

Next, we would load the model and labels using the FirebaseModel manager. Firebase M.L Kit gives us the option of either loading local model on the device or using one hosted in the Cloud. You can find samples and more information about this here or you can check my GitHub repo for a simple example of Firebase Vision API to recognise text.

For this example, I have used an on-device model loaded using FirebaseLocalModelSource.

manager.registerLocalModelSource(
    FirebaseLocalModelSource(
        assetFilePath: "assets/models/plant_ai_lite_model.tflite",
        modelName: "plant_ai_model"
    )
);
//     rootBundle.loadString('assets/models/labels_plant_ai.txt').then((string) {
    var _labels = string.split('\n');
    _labels.removeLast();
    modelLabels = _labels;
});

Let's take a picture !!
Using the image_picker Flutter package, the function below allows us to pick a picture from the phone's gallery

_pickPicture() async{
    var image = await ImagePicker.pickImage(source: ImageSource.gallery);
    if(image != null){
      setState(() {
        selectedImageFile = image;
      });
      predict(selectedImageFile);
    }
}

The predict function is where we convert our image file into a byte and run on the interpreter using the input and output options. The result will be a prediction has you will have it when running on your Jupyter notebook or any other platform.

var bytes = await imageToByteListFloat(imageFile, imageDim);
var results = await interpreter.run(
    localModelName: "plant_ai_model",
    inputOutputOptions: FirebaseModelInputOutputOptions(
        [
            FirebaseModelIOOption(
                FirebaseModelDataType.FLOAT32,
                inputDims
            )
        ],
        [
            FirebaseModelIOOption(
                FirebaseModelDataType.FLOAT32,
                outputDims
            )
        ]
    ),
    inputBytes: bytes
);

The results of the prediction can then be converted to label text has specified in the 'labels.txt'.

if(results != null && results.length > 0){
    for (var i = 0; i < results[0][0].length; i++) {
        if (results[0][0][i] > 0) {
            var confidenceLevel = results[0][0][i] / 2.55 * 100;
            if(confidenceLevel > 0){
              predictions[modelLabels[i]] = confidenceLevel;
            }
        }
    }
    var predictionKeys = predictions.entries.toList();
    predictionKeys.sort((b,a)=>a.value.compareTo(b.value));
    predictions = Map<String, double>.fromEntries(predictionKeys);
}

The prediction result are now avaialble for further use and/or display on your app. All without the use of an internet connection and without sending data from the client to a particular server.

The codes for this article can be found here.

Thanks for reading. 🤝

A thousand ways to deploy Machine learning models - A.P.I

Oluwafemi Tairu — Fri, 24 Jan 2020 11:22:21 +0000

Originally posted here

“What use is a machine learning model if you don’t deploy to production “ — Anonymous

You have done a great work building that awesome 99% accurate machine learning model but your work most of the time is not done without deploying. Most times our models will be integrated with existing web apps, mobile apps or other systems. How then do we make this happen?

I said a thousand, I guess I have just a few. I am guessing you would have found the right one for you before you get past the first two or three. Or do you think there are more? Do let me know, let’s see if we can get a thousand 😄.
Let’s start, how do we deploy Machine learning models or integrate with other systems?

1. Via an A.P.I

This involves making your models accessible via an Application Programming Interface (A.P.I).
First, I will be deploying a deep learning model built by Rising Odegua to classify malaria cells. The notebook can be found here.

Then I will also deploy a simple Nigerian movie review classifier model built by Aminu Israel. The notebook can be found here.

Deployment of a deep learning model via an A.P.I

After building and testing your deep learning model the next thing to do is to save your model. This can simply be done by adding this line of code.

cnn_model.save('model_name.h5')

You can move your saved model to the folder accessible to your A.P.I code. I will be using Flask for deployment but Django, Starlette or any other python frameworks can be used. Here is what my folder structure looks like -

Coming over to our A.P.I folder (powered by Flask) first thing you will want to do is install the requirements. I saved the requirements in requirements.txt. Here is what it looks like

You can install these requirements simply by running this in your terminal

pip install -r requirements.txt

Just as we preprocessed images before passing them to our neural network for training, we would also preprocess all input images we collect via our A.P.I endpoint.
On line 8 we converted the image collect via the A.P.I endpoint to an array. You will notice a little difference in what you would have done on a norm and that’s simply because of Flask’s data storage. Using the code on Line 9 would do the same on a Framework like Django or if you are loading from a path on your machine.

From lines 10–15 we get an RBG version of our image, resized the image to 100x100 and convert the image to a numpy array. We also scaled our image to the range [0, 1] and return a tuple containing True if no error occurs and an array with our image in it.

The image above shows the function that performs the magic.
Line 26 simply means that the function “classify_malaria_cells” would be executed when the “classify” endpoint is called.

On line 29 we are checking to see if the request contains an image file. Then we preprocess that image using the helper function we created.
The saved model can be loaded using

from keras.models import load_model
# OR
# from tensorflow.keras.models import load_model
malaria_model = load_model(MODEL_PATH)

From lines 34 to 39, we loaded the saved model, performed prediction to determine the class for the image and get an accuracy score for the prediction. On line 40 the result from the model is saved in a python dictionary that will be sent back as a JSON response on line 60.

We end our A.P.I with:

if __name__ == “__main__”:
    flask_app.run(port=8080, debug=False, threaded=False)

With this, we have successfully built our A.P.I and it is ready for deployment to any cloud platform.

Deploying to Google Cloud

Create a Google Cloud Account — https://cloud.google.com/
Create a new Google Cloud Platform project. You can follow the steps here
Go to the root of this flask project in your terminal and run: python gcloud app deploy

** I have also added a Procfile for deployment to Heroku, just follow the steps here to deploy.

Deployment of a machine learning model via an A.P.I

This is almost the same as with the deep learning model. However, saving your models here is quite different. You could save your model using —

import pickle
FILENAME = 'filename.pkl'
pickle.dump(trained_model, open(FILENAME, 'wb'))

import joblib
FILENAME = 'filename.joblib'
joblib.dump(trained_model, FILENAME)

It is advised that joblib be used to save models rather than using pickle because of its efficiency on objects that carry large numpy arrays internally.
Just as we did with the deep learning model, save your model in a folder accessible to your Flask A.P.I code. Here is what my file structure looks like

Our requirements are a little different here.

You can install these requirements simply by running this in your terminal again.

pip install -r requirements.txt

Next, we would import the required modules and initialise some variables.

In the helper function above, we split the sentence into words, removed the stop words loaded from the pickled file on Line 10. On line 18, a HashingVectorizer is used to the tokenized words into a matrix. The output of this is a scipy.sparse matrix.

Our movie review will be in text format. In Line 24 we check to see if the required form data is been sent and then we assign a variable name to the form data on Line 25. On line 28, we opened and loaded our movie review classifier with pickle(you can achieve the same with joblib). On lines, 29–36 we passed the vectorized movie review into our movie review classifier for prediction, calculated a prediction probability score and created a python dictionary to pass out the results of our prediction. The output is then sent back as a JSON response on Lines 47.

We have successfully built a movie review classification A.P.I and it is ready for deployment to any cloud platform. I have also added a config file for deployment to both Google Cloud and Heroku.

That’s all on model deployment via an A.P.I. You can find the codes used in this article here.

In my next article, I will share how to deploy machine learning models to Apps with TensorFlow Lite.

Thanks for reading 👍