Model Operators
The model operators are used to implement various system functions in the AI Pipelines, such as new model creation (analogous to the model registration in the AI Hub), model version staging, model instance creation, single model version online deployment, and prediction based s well as system functions based on the deployed model services or model files. It helps to achieve a closed loop of business from model training to service deployment and prediction based on the trained model.
The AI Pipelines provides the following operators related to registering and deploying machine learning models:
- Model (model registration)
- Mlflow Model Version Register (model version staging)
- Docker Model Version Register (model version staging)
- S2i Model Version Register (model version staging)
- Thirdparty Model Version Register (model version staging)
- Latest Model Version
- Mlflow Model Version Prediction (prediction based on the machine learning image)
- Model Instance (creating a model deployment instance)
- Model Test (model testing)
- Single Model Deployment (single model version deployment)
- Model Version Info (Getting Model Version Information in multi-deployment scenarios)
- Model Deployment (multiple model version deployment)
- Service Prediction (prediction based on the model deployment service)
- External Mlflow Prediction (prediction with the model file recorded by Mlflow in an external environment)
- Internal Mlflow Prediction (prediction with the model file recorded by Mlflow in an internal environment)
- Model Monitor (model performance monitoring)
- Model Log (Getting model logs by time or key words)
Model Operator
The Model operator is used to create a new model (similar to the model registration function in the AI Hub).
Output Parameters Description
Name |
Type |
Description |
model_name_output |
String |
Output the created model name, which is used as the input of Mlflow Model Version Register and other operators. |
Mlflow Model Version Register Operator
The Mlflow Model Version Register operator is used to create the model version file and stage the model version by means of MLflow import.
Input Parameters Description
Name |
Required/Optional |
Type |
Description |
input_data |
Required |
String |
Model version parameter input. See the input_data sample. |
alias |
Optional |
String |
Alias of the model. |
version_rule |
Required |
String |
Model version naming rule, where the time basis is taken by default to name the model version according to the current timestamp. |
annotation |
Optional |
String |
Model version description information. |
architecture |
Required |
String |
Basic hardware for running model version; options: x86 or arm. |
coprocessor |
Optional |
String |
Select the coprocessor for running the model version; options: GPU, TPV, VPU or None. |
env_param |
Optional |
List |
Environment parameter. See the env_param sample. |
framework |
Required |
String |
Calculation framework for running the model version; options: sklearn, tensorflow, pytorch, h2o, spark, etc. |
language |
Required |
String |
The language of the development model version; options: python3 or java8. |
model_reference |
Required |
String |
Name of the model that the model version belongs to, which is from the model_name_output outputted by the Model operator. |
publisher |
Required |
String |
Model creator name. |
minio_paths |
Optional |
List |
minio path. See minio_path sample. |
env_param sample
[
{
"name": "string",
"value": "string",
"annotations": "string"
}
]
minio_paths sample
[
{
"bucket": "xxx",
"path": "xxx",
"destination": "xxx"
}
]
Output Parameters Description
Parameter |
Description |
create_model_revision |
Whether the model version is built successfully. |
model_revision_name |
Name of the staged model version. |
model_builder_name |
Model build name. |
Docker Model Version Register Operator
The Docker Model Version Register operator is used to create the model version file and stage the model version by means of container image import. You can select the target files needed to build the model from multiple hierarchical directories of Artifacts source and Git source. When deploying the model, the system will deploy the specified image file as a service.
Input Parameters Description
Name |
Required/Optional |
Type |
Description |
input_data |
Required |
String |
Model version parameter input. See the input_data sample. |
alias |
Optional |
String |
Alias of the model. |
version_rule |
Required |
String |
Model version naming rule, where the time basis is taken by default to name the model version according to the current timestamp. |
annotation |
Optional |
String |
Model version description information. |
architecture |
Required |
String |
Basic hardware for running model version; options: x86 or arm. |
coprocessor |
Optional |
String |
Select the coprocessor for running the model version; options: GPU, TPV, VPU or None. |
env_param |
Optional |
List |
Environment parameter. See the env_param sample. |
framework |
Required |
String |
Calculation framework for running the model version; options: sklearn, tensorflow, pytorch, h2o, spark, etc. |
language |
Required |
String |
Language used for model version development; options: python3 or java8. |
model_reference |
Required |
String |
Name of the model that the model version belongs to, which is from the model_name_output outputted by the Model operator. |
publisher |
Required |
String |
Model creator name. |
minio_paths |
Optional |
List |
minio path. See minio_path sample. |
git_setting |
Optional |
List |
Git source setting. See git_setting sample. |
docker_dockerfile |
Optional |
String |
Image file name. |
git_setting sample
[
{
"url": "xxx",
"user": "xxx",
"token": "xxx",
"branch": "xxx",
"paths": [
{
"path": "xxx",
"destination": "xxx"
}
]
}
]
Output Parameters Description
Parameter |
Description |
create_model_revision |
Whether the model version is built successfully. |
model_revision_name |
Name of the staged model version. |
model_builder_name |
Model build name. |
S2i Model Version Register Operator
The S2i Model Version Register operator is used to create the Docker image of model version and stage the model version by using s2i build. You can select the target files needed to build the model from multiple hierarchical directories of Artifacts source and Git source. When deploying the model, the system will deploy the specified image file as a service.
Input Parameters Description
Name |
Required/Optional |
Type |
Description |
input_data |
Required |
String |
Model version parameter input. See the input_data sample. |
alias |
Optional |
String |
Alias of the model. |
version_rule |
Required |
String |
Model version naming rule, where the time basis is taken by default to name the model version according to the current timestamp. |
annotation |
Optional |
String |
Model version description information. |
architecture |
Required |
String |
Basic hardware for running model version; options: x86 or arm. |
coprocessor |
Optional |
String |
Select the coprocessor for running the model version; options: GPU, TPV, VPU or None. |
env_param |
Optional |
List |
Environment parameter. See the env_param sample. |
framework |
Required |
String |
Calculation framework for running the model version; options: sklearn, tensorflow, pytorch, h2o, spark, etc. |
language |
Required |
String |
Language used for model version development; options: python3 or java8. |
model_reference |
Required |
String |
Name of the model that the model version belongs to, which is from the model_name_output outputted by the Model operator. |
publisher |
Required |
String |
Model creator name. |
minio_paths |
Optional |
List |
minio path. See minio_path sample. |
git_setting |
Optional |
List |
Git source setting. See git_setting sample. |
s2i_model |
Optional |
String |
Name of s2i model file. |
Output Parameters Description
Parameter |
Description |
create_model_revision |
Whether the model version is built successfully. |
model_revision_name |
Name of the staged model version. |
model_builder_name |
Model build name. |
Thirdparty Model Version Register Operator
The Thirdparty Model Version Register operator is used to stage the model version by packaging the model as the Docker image file in the third-party custom system.
Input Parameters Description
Name |
Required/Optional |
Type |
Description |
input_data |
Required |
String |
Model version parameter input. See the input_data sample. |
alias |
Optional |
String |
Alias of the model. |
version_rule |
Required |
String |
Model version naming rule, where the time basis is taken by default to name the model version according to the current timestamp. |
annotation |
Optional |
String |
Model version description information. |
architecture |
Required |
String |
Basic hardware for running model version; options: x86 or arm. |
coprocessor |
Optional |
String |
Select the coprocessor for running the model version; options: GPU, TPV, VPU or None. |
env_param |
Optional |
List |
Environment parameter. See the env_param sample. |
framework |
Required |
String |
Calculation framework for running the model version; options: sklearn, tensorflow, pytorch, h2o, spark, etc. |
language |
Required |
String |
Language used for model version development; options: python3 or java8. |
model_reference |
Required |
String |
Name of the model that the model version belongs to, which is from the model_name_output outputted by the Model operator. |
publisher |
Required |
String |
Model creator name. |
thirdparty_url |
Optional |
String |
Load the address of the model version file. |
Output Parameters Description
Parameter |
Description |
create_model_revision |
Whether the model version is built successfully. |
model_revision_name |
Name of the staged model version. |
model_builder_name |
Model build name. |
Latest Model Version Operator
The Latest Model Version Operator can be used to get the latest model version of a model.
Input Parameters Description
Name |
Required/Optional |
Type |
Description |
model_name |
Required |
model_name |
Specify the name of the target model. |
serve_as_file |
Required |
boolean |
Choose whether to provide services in the form of model files. |
Output Parameters Description
Name |
Type |
Description |
lastest_version |
model_version |
The latest model version. |
Mlflow Model Version Prediction Operator
The Mlflow model version prediction operator is used to perform the prediction based on the machine learning model images.
Input Parameters Description
Name |
Required/Optional |
Type |
Description |
model_name |
Required |
model_name |
Model name |
model_version_name |
Required |
model_version |
Select the model version. |
data |
Required |
File |
Data file. |
data_type |
Required |
data_type |
Select the data type, options are csv and json . |
Output Parameters Description
Name |
Type |
Description |
predictions |
File |
Prediction results. |
Model Instance Operator
The model instance operator is used to create model deployment instances.
Input Parameters Description
Name |
Required/Optional |
Type |
Description |
name |
Required |
String |
Deployment instance name. |
resource_pool |
Required |
String |
Name of the resource pool, which comes from the resource name requested through Resource Management. |
model_name |
Required |
String |
Model name, which is from the model_name_output outputted by the Model operator. |
labels |
Optional |
List |
Tag list. |
description |
Optional |
String |
Deployment instance description. |
deploy_mode |
Required |
String |
Deployment mode, where only ONLINE (that is, current environment deployment) is supported. |
error_on_exist |
Required |
String |
Specify whether to skip without reporting an error when the model name exists. If false is selected, no error will be reported; if `true`is selected and the name of model already exists, an error will be reported directly. |
Output Parameters Description
Name |
Type |
Description |
instance_name_output |
String |
Deployment instance name. |
Model Test Operator
The model testing operator is used to test the staged model version.
Input Parameters Description
Name |
Required/Optional |
Type |
Description |
input_data |
Required |
String |
Enter the model testing data in JSON format. |
model_builder |
Required |
String |
Model builder, which comes from the model_builder_name outputted by any Model Version Register operator. |
Output Parameters Description
Name |
Type |
Description |
create_model_test |
String |
Model test name. |
model_test_output |
String |
Model test return results in JSON format. |
Single Model Deployment Operator
The model deployment operator is used to deploy a single model version online.
Input Parameters Description
Name |
Required/Optional |
Type |
Description |
model_revision |
Required |
String |
Name of the model version to be deployed, which comes from the model_revision_name outputted by any Model Version Register operator. |
instance_name |
Required |
String |
Deployment instance name, which comes from the instance_name_output outputted by the Model Instance operator. |
request_cpu |
Required |
Number |
CPU request value required to deploy the model version (the minimum value is 0.01 core). |
request_memory |
Required |
Number |
Memory request value required to deploy the model version (the minimum value is 0.1 GB). |
limit_cpu |
Required |
Number |
Upper limit of CPU required to deploy the model version (the maximum value is 32 core). |
limit_memory |
Required |
Number |
Upper limit of memory required to deploy the model version (the maximum value is 64 GB). |
timeout |
Required |
Number |
Timeout setting. |
Output Parameters Description
Name |
Type |
Description |
create_model_deployment |
String |
Name of the deployed model version. |
Model Version Info Operator
The Model Version Info operator is used to get the information on model versions in multi-deployment scenarios.
Note
You can only add this operator in the sub-canvas of the Model Deployment operator.
Input Parameters Description
Name |
Required/Optional |
Type |
Description |
model_name |
Required |
model_name |
Automatically select the name of the model specified in the model_name parameter of the Model Deployment operator. |
model_revision |
Required |
model_version |
Select a model version from the dropdown list. |
request_cpu |
Required |
request_cpu |
Specify the value of CPU request. |
request_memory |
Required |
request_memory |
Specify the value of memory request. |
limit_cpu |
Required |
limit_cpu |
Specify the value of CPU limit. |
limit_memory |
Required |
limit_memory |
Specify the value of memory limit. |
canary_rate |
Optional |
number |
- Specify the percentage of traffic to the model version.
- It only takes effect in the Canary Deployment.
|
bg_rate |
Optional |
number |
- Specify the percentage of weighted combination optimization to the model version.
- It only takes effect in the Blue/Green Deployment.
|
Output Parameters Description
Name |
Type |
Description |
version_deploy_info |
version_deploy_info |
The information of the model version to be deployed. |
Model Deployment Operator
The Model Deployment operator is used to deploy multiple model versions.
Note
To run this operator, you need to add at least 2 Model Version Info operators to the sub-canvas of the Model Deployment operator.
Input Parameters Description
br />
Name |
Required/Optional |
Type |
Description |
model_name |
Required |
model_name |
Select the model to deploy. |
instance_name |
Required |
instance_name |
Select the model version instance to deploy. |
request_timeout |
Required |
request_timeout |
Enter the timeout threshold for requesting. |
deploy_timeout |
Optional |
deploy_timeout |
Enter the timeout threshold for deploying. |
using_token |
Required |
boolean |
Choose whether using token. |
upgrade_method |
Required |
upgrade_method |
The method to update model versions, supporting Blue/Green, Canary, and Multi-Armed. |
combiner_strategy |
Required |
combiner_strategy |
- The strategy for model combination optimization, supporting Weighted Average and Specify Only.
- It only takes effect in the Blue/Green Deployment.
|
specify_version |
Required |
string |
- Specify the model version for the Specify Only optimization strategy.
- It only takes effect for the Specify Only optimization strategy.
|
Output Parameters Description
Name |
Type |
Description |
create_model_deployment |
string |
The name of the deployment instance. |
Service Prediction Operator
Achieve the prediction function based on model deployment service.
Input Parameters Description
Name |
Required/Optional |
Type |
Description |
model |
Required |
String |
Model name. |
instance |
Required |
String |
Deployment instance name. |
namespace |
Required |
String |
Name of the resource pool, which comes from the resource name requested through Resource Management. |
data_type |
Required |
String |
- Data type. Options: csv or json.
-
|
data |
Required |
File |
Input data. |
Output Parameters Description
Name |
Type |
Description |
predictions |
File |
Output prediction results. |
External Mlflow Prediction Operator
The external Mlflow prediction operator is used to perform the prediction based on the external environment through the model file recorded by Mlflow. The external environment refers to the Jupyter Lab environment in the third-party Lab.
Input Parameters Description
Name |
Required/Optional |
Type |
Description |
model_path |
Required |
Directory |
Model file path, which can be derived from the output of the Git Directory operator. |
data_type |
Required |
String |
Select the data type, options are csv and json . |
data |
Required |
File |
Input data; options: json and csv. |
requirements |
Optional |
List |
Content of the requirements.txt file, which is outputted in List format, such as: pandas==1.0.0. |
Output Parameters Description
Name |
Type |
Description |
predictions |
File |
Output prediction results. |
Internal Mlflow Prediction Operator
The internal Mlflow prediction operator is used to perform the prediction based on the internal environment through the model file recorded by Mlflow. The internal environment refers to the Jupyter Lab environment in the AI Lab.
Input Parameters Description
Name |
Required/Optional |
Type |
Description |
minio_paths |
Required |
List |
Model file path, which is the path of the model file on minio. |
data_type |
Required |
String |
Select the data type, options are csv and json . |
data |
Required |
File |
Input data; options: json and csv. |
requirements |
Optional |
List |
Content of the requirements.txt file, which is outputted in List format, such as: pandas==1.0.0. |
Output Parameters Description
Name |
Type |
Description |
predictions |
File |
Output prediction results. |
Model Monitor Operator
The model monitor operator is used to monitor the performance of a specified model in specified period of time. Use the returned metrics result as input of business decisions, such as whether to train the model again or use other algorithms to train new models for replacing the current model version. A model deployment instance can have multiple monitoring indicator, and each model monitor operator can monitor one indicator only.
Input Parameters Description
Name |
Required/Optional |
Type |
Description |
model_name |
Required |
String |
Model name. |
deployment_instance |
Required |
String |
Model deployment instance name. |
start_time |
Required |
Timestamp |
Start time for the model monitoring (selected using the data/time widget, in second unit). |
end_time |
Required |
Timestamp |
End time for the model monitoring (selected using the data/time widget, in second unit). |
metric_name |
Required |
String |
Monitoring metric name (customized indicator added on the Indicator Monitoring page of AI Hub > Deployment Instance). |
Output Parameters Description
Name |
Type |
Description |
model_indicator_data |
List |
Output model metric monitoring results. |
Model Log Operator
The Model Log operator can be used to get up to 5000 logs of a model instance in the last 7 days.
Input Parameters Description
Name |
Required/Optional |
Type |
Description |
model_name |
Required |
String |
Specify the Model. |
instance_name |
Required |
model_instance |
Specify the model instance. |
keyword |
Optional |
String |
Specify the key words to filter model logs. |
time_duration |
Required |
time_duration |
Specify the time duration to get model logs. |
Output Parameters Description
Name |
Type |
Description |
matched_times |
number |
The time information of model logs. |
log |
file |
Model log file. |