What MLOps is: a complete guide to operating machine learning models continuously

MLOps (Machine Learning Operations) is the operating methodology that systematically manages the whole course of a machine learning model, from development and training through validation, deployment, monitoring and updates. Its core is not stopping at building and deploying a model once, but holding performance in the real environment and improving it continuously as conditions change.

A model has to be managed after it is deployed

Building a good model and keeping a good model running are different problems.

In a machine learning project, a model that scored well at the development stage cannot be guaranteed to hold the same performance after it is applied to a real service.

Over time the character of incoming data can change, new types of data can be added, and business criteria can be revised. Patterns that did not exist at training time appear repeatedly in the real environment.

As these changes accumulate, a model can appear to run normally while its real accuracy gradually declines.

The problem grows when several models are in operation rather than one. Which data each was trained on, which model is currently deployed to the service, whether performance improved against the previous version, and which version to return to when something goes wrong all have to be managed.

MLOps is the framework that connects machine learning development to real operation.

What MLOps actually is: how it differs from DevOps

DevOps and MLOps manage different things

DevOps and MLOps are alike in narrowing the gap between development and operation and in automating repetitive work.

A machine learning system, however, has more to manage than ordinary software.

First, managing code is not enough. A machine learning model produces different results depending on the data it was trained on, so training data and data versions have to be managed alongside source code.

Second, the model itself is an object of management. Which model was trained when, what performance it recorded, and which version is currently applied to the service have to be tracked.

Third, performance has to be evaluated during operation too. It is not enough to confirm that the system is running; whether the model's performance is holding on real input data has to be confirmed as well.

Fourth, there is a retraining process. If the character of the data shifts substantially or model performance falls below a standard, the model may need to be trained again on new data, validated and then deployed.

In short, where DevOps builds an operating framework around software code and deployment, MLOps is an operating framework that adds data, models, training and evaluation.

The four functions that make up MLOps

Managing everything from data to post-deployment

First, data management.

Which data was used for training and evaluation is recorded and versioned. When data changes, it has to be possible to trace which model's results were affected.

Second, model and experiment management.

The model, parameters, data and performance metrics used in training are recorded together, so that it is clear which of many experiments produced a model that passed the deployment standard.

Third, deployment and version management.

Validated models are deployed to the real service environment and the versions in operation are managed. A structure for returning to the previous version when a new model has a problem is needed as well.

Fourth, monitoring and retraining.

Changes in input data and model performance are checked continuously in the operating environment. When drift in the data distribution or a fall in performance is confirmed, whether retraining and redeployment are needed is judged.

Only when these four processes are connected can the course from model development to real operation be managed as a single repeatable flow.

How MLOps is applied in practice

Record the relationship between training data and models

Reproducing a model's results requires knowing which model was built from which data.

Linking dataset versions to model versions makes it easier to tell, when a new model performs differently, whether the change came from the data or from the model itself.

Recording training conditions and evaluation results with each model also matters.

Set the criteria a model must pass before deployment

A new model is not applied to the operating environment as soon as it is built.

Its performance is compared with the existing model on the same evaluation data, and metrics that matter in the real work, such as accuracy, processing speed and error rate, are checked against the standard.

Even if some of the new model's metrics improved, replacing the existing model may be difficult if errors increased in a particular process.

It is therefore better to decide in advance which conditions a model must meet to be deployed.

Monitor changes in the model in operation

After deployment, the question is not only whether the model runs normally but also whether the quality of its results holds.

One representative change is data drift, the phenomenon where the distribution of real input data diverges from what it was when the model was trained.

If a particular document form or data type increases sharply, for example, the existing model may not have learned that pattern sufficiently.

Checking for such changes regularly allows problems to be found before performance falls far.

It has to be possible to return to the previous state

In MLOps, version management is not simply a feature for keeping records.

When an unforeseen problem arises after a new model is deployed, it has to be possible to return to the previous model. Tracking the data and settings that model used, not only the model itself, makes the cause easier to identify.

Recording who deployed which model when, and what changed, also makes the change history of the operating process easier to confirm.

MLOps and VLMOps

MLOps is the concept covering training, deployment and monitoring across machine learning models in general.

VLMOps applies that operating concept more specifically to vision language models and the document AI environment.

In document AI, operations are affected not only by the model itself but by the schema that defines which fields to extract, natural language instructions, validation criteria and changes to document forms.

In particular, a new document form or extraction field does not necessarily require training the model again, because some areas can be handled by adjusting extraction criteria or schema without changing model parameters.

Where MLOps manages operations around model training and deployment, VLMOps extends toward managing operating criteria, evaluation and change history together so that document AI keeps responding to real changes in the work.

MLOps in the Korean enterprise environment

Applying MLOps in an enterprise means considering the operating environment alongside model performance.

In environments such as finance and the public sector, where external cloud use or data export is restricted, training data, models and operating logs may have to be managed on internal infrastructure.

In that case a structure that can carry out model development, deployment and monitoring inside the internal environment is needed.

Being able to record who reviewed and approved which version whenever a model changes also makes the change history easier to trace during operation.

Frequently asked questions