MLOps #03: Continuous Deployment modes that you should know

Mastering an important step in MLOps pipelines.

Photo by Qingbao Meng on Unsplash

In these series about MLOps, I will be covering in each blog post a step of the MLOps pipeline and deep dive into it. Today, you will learn about Continuous Deployment modes and how you can choose the strategy that works best for you.

Continuous Deployment

The real motivations behind Continuous Deployment are:

• Being up to date
• Automate frequent updates

Continuous Deployment is a process of machine learning operations that automatically and continuously deploys re-trained models resulting from Continuous Training into production.

In practice, after the Continuous Training, you will end up having new versions of your trained model. So if you want that your models are used in production, you need to deploy them. Therefore Continuous Deployment will insure that your production system is always up to date leveraging the newest version of your trained model.

Assuming that you need to deploy 1000 models per day, it will be almost impossible to deploy them manually, for this reason the automation of the deployment process is important to insure the business value.

Continuous Deployment Modes

There are three modes of Continuous Deployment:

Manual Deployment

Image by the author

This is the safest and one of the most common deployment modes and it is likely to be adopted when more control over the model or service that is deployed is required. But if a frequent update of the model used in production is required, then this method is not scalable.

Automatic Deployment:

Image by the author

Unlike the previous manual mode, automatic deployment, as its name tells, is automated and straightforward. But this is the most dangerous mode of continuous deployment, because it is equivalent to deploying a model to production without doing any tests of its performance. So it is definitely not recommended.

Canary Deployment

“A canary deployment is often a synonym of A/B testing.”

Image by the author

During A/B testing, a different product version is shown to a small percentage of users. In Canary deployment, a similar philosophy is adopted. It consists of releasing two versions of the ML model in production and forwarding part of the inference traffic to the canary model.

The monitoring of the canary model goes on, as long as there are no problems detected from the metrics or the users feedback, more percentage of the inference requests traffic is directed to the canary model until the whole traffic is forwarded to it.

If any failure with the canary model is detected, the prediction requests are no longer forward it to it.

Shadow Deployment

“Shadow deployment allowed you to have a staging environment for your models”

Image by the author

Shadow deployment consists of releasing two parallel versions of the model and transferring the all production traffic/inference requests to the shadow model. With no impact on traffic, this method enables testing of how performant the shadow model is compared to the model in production. It is safe to deploy the shadow model whenever it satisfies the performance and robustness specifications.

Unlike Canary deployment, shadow deployment can be more cost effective due to duplicating the traffic.

Which mode is the best?

The answer is: It depends.

Every strategy has its pros and cons, so your choice depends on the benefits you will get from using it and the drawbacks you will have to deal with.

So here are some final thoughts:

• Shadow deployment is a method of testing a candidate model for production (resulting from CT) where production data runs through the model without the model actually returning predictions to the service or customers (not deployed in production yet).
• Shadow & canary deployment enables you to simulating how the model would perform in production environment, this enables catching production problems and fixing them before releasing the new model.
• Unlike canary deployment, shadow deployment does not return predictions to users in production. That can cause problems for canary users, if predictions are not accurate.

Conclusion

If you reached this stage, you have learned the different continuous deployment modes. These modes are not the totality of the deployment strategies, there are others as well. But mainly, these are the most common.

Thanks for reading and see you in the nest blog post 👋