Model Operators¶

The model operators are used to implement various system functions in the AI Pipelines, such as new model creation (analogous to the model registration in the AI Hub), model version staging, model instance creation, single model version online deployment, and prediction based s well as system functions based on the deployed model services or model files. It helps to achieve a closed loop of business from model training to service deployment and prediction based on the trained model.

The AI Pipelines provides the following operators related to registering and deploying machine learning models:

Model (model registration)
Mlflow Model Version Register (model version staging)
Docker Model Version Register (model version staging)
S2i Model Version Register (model version staging)
Thirdparty Model Version Register (model version staging)
Latest Model Version
Mlflow Model Version Prediction (prediction based on the machine learning image)
Model Instance (creating a model deployment instance)
Model Test (model testing)
Single Model Deployment (single model version deployment)
Model Version Info (Getting Model Version Information in multi-deployment scenarios)
Model Deployment (multiple model version deployment)
Service Prediction (prediction based on the model deployment service)
External Mlflow Prediction (prediction with the model file recorded by Mlflow in an external environment)
Internal Mlflow Prediction (prediction with the model file recorded by Mlflow in an internal environment)
Model Monitor (model performance monitoring)
Model Log (Getting model logs by time or key words)

Model Operator¶

The Model operator is used to create a new model (similar to the model registration function in the AI Hub).

Input Parameters Description¶

Name	Required/Optional	Type	Description
category	Required	String	Model category; options: Predictor Combiner Data Transformer
model_name	Required	String	Model name, where lower cases, numbers and dash are supported with the length limited within 32 characters.
alias	Optional	String	Alias of the model.
input_data_type	Required	String	Input data type; options: Text, Tabular or Image.
scope	Required	String	Model application scope; options: Private or Public.
technique	Required	String	Model technology attribution; options: Classification Regression Clustering Other
usecase	Required	String	Model application field; options: Wind Solar Power Storage Smart Factory Smart City Other
publisher	Required	String	Model creator name
input_format	Required	String	Input parameters of the model feature in JSON format. See input_format sample.
output_format	Required	String	Model target output in JSON format. See output_format sample.
interface	Required	String	API type; options: REST or GRPC (that is, the calling method of model service API).
error_on_exist	Optional	Boolean	Specify whether to skip without reporting an error when the model name exists. If false is selected, no error will be reported; if `true`is selected and the name of the model to be created already exists, an error will be reported directly.

input_format sample¶

[
  {
       "name":"X-basic.hour",
       "dtype":"int",
       "ftype":"continuous",
       "range": [
           0,
           23
       ],
       "annotations":"",
       "repeat":null,
       "defaultValue":10
   },
    {
       "name":"X-basic.horizon",
       "dtype":"int",
       "ftype":"continuous",
       "range": [
           0,
           50
       ],
       "annotations":"",
       "repeat":null,
       "defaultValue":8
   }
]

output_format sample¶

[
  {
    "name": "test",
    "dtype": "float",
    "ftype": "continuous",
    "range": [
      1,
      11
    ],
    "annotations": null,
    "repeat": 4,
    "defaultValue": 1
  }
]

Output Parameters Description¶

Name	Type	Description
model_name_output	String	Output the created model name, which is used as the input of Mlflow Model Version Register and other operators.

Mlflow Model Version Register Operator¶

The Mlflow Model Version Register operator is used to create the model version file and stage the model version by means of MLflow import.

Input Parameters Description¶

Name	Required/Optional	Type	Description
input_data	Required	String	Model version parameter input. See the input_data sample.
alias	Optional	String	Alias of the model.
version_rule	Required	String	Model version naming rule, where the time basis is taken by default to name the model version according to the current timestamp.
annotation	Optional	String	Model version description information.
architecture	Required	String	Basic hardware for running model version; options: x86 or arm.
coprocessor	Optional	String	Select the coprocessor for running the model version; options: GPU, TPV, VPU or None.
env_param	Optional	List	Environment parameter. See the env_param sample.
framework	Required	String	Calculation framework for running the model version; options: sklearn, tensorflow, pytorch, h2o, spark, etc.
language	Required	String	The language of the development model version; options: python3 or java8.
model_reference	Required	String	Name of the model that the model version belongs to, which is from the model_name_output outputted by the Model operator.
publisher	Required	String	Model creator name.
minio_paths	Optional	List	minio path. See minio_path sample.

Input_data sample¶

{
  "data": {
    "names": [
      "AGE",
      "RACE",
      "DCAPS",
      "VOL"
    ],
    "ndarray": [
      [
        68,
        2,
        2,
        0
      ]
    ]
  }
}

env_param sample¶

[
  {
    "name": "string",
    "value": "string",
    "annotations": "string"
  }
]

minio_paths sample¶

[
  {
    "bucket": "xxx",
    "path": "xxx",
    "destination": "xxx"
  }
]

Output Parameters Description¶

Parameter	Description
create_model_revision	Whether the model version is built successfully.
model_revision_name	Name of the staged model version.
model_builder_name	Model build name.

Docker Model Version Register Operator¶

The Docker Model Version Register operator is used to create the model version file and stage the model version by means of container image import. You can select the target files needed to build the model from multiple hierarchical directories of Artifacts source and Git source. When deploying the model, the system will deploy the specified image file as a service.

Input Parameters Description¶

Name	Required/Optional	Type	Description
input_data	Required	String	Model version parameter input. See the input_data sample.
alias	Optional	String	Alias of the model.
version_rule	Required	String	Model version naming rule, where the time basis is taken by default to name the model version according to the current timestamp.
annotation	Optional	String	Model version description information.
architecture	Required	String	Basic hardware for running model version; options: x86 or arm.
coprocessor	Optional	String	Select the coprocessor for running the model version; options: GPU, TPV, VPU or None.
env_param	Optional	List	Environment parameter. See the env_param sample.
framework	Required	String	Calculation framework for running the model version; options: sklearn, tensorflow, pytorch, h2o, spark, etc.
language	Required	String	Language used for model version development; options: python3 or java8.
model_reference	Required	String	Name of the model that the model version belongs to, which is from the model_name_output outputted by the Model operator.
publisher	Required	String	Model creator name.
minio_paths	Optional	List	minio path. See minio_path sample.
git_setting	Optional	List	Git source setting. See git_setting sample.
docker_dockerfile	Optional	String	Image file name.

git_setting sample¶

[
  {
    "url": "xxx",
    "user": "xxx",
    "token": "xxx",
    "branch": "xxx",
    "paths": [
      {
        "path": "xxx",
        "destination": "xxx"
      }
    ]
  }
]

Output Parameters Description¶

Parameter	Description
create_model_revision	Whether the model version is built successfully.
model_revision_name	Name of the staged model version.
model_builder_name	Model build name.

S2i Model Version Register Operator¶

The S2i Model Version Register operator is used to create the Docker image of model version and stage the model version by using s2i build. You can select the target files needed to build the model from multiple hierarchical directories of Artifacts source and Git source. When deploying the model, the system will deploy the specified image file as a service.

Input Parameters Description¶

Name	Required/Optional	Type	Description
input_data	Required	String	Model version parameter input. See the input_data sample.
alias	Optional	String	Alias of the model.
version_rule	Required	String	Model version naming rule, where the time basis is taken by default to name the model version according to the current timestamp.
annotation	Optional	String	Model version description information.
architecture	Required	String	Basic hardware for running model version; options: x86 or arm.
coprocessor	Optional	String	Select the coprocessor for running the model version; options: GPU, TPV, VPU or None.
env_param	Optional	List	Environment parameter. See the env_param sample.
framework	Required	String	Calculation framework for running the model version; options: sklearn, tensorflow, pytorch, h2o, spark, etc.
language	Required	String	Language used for model version development; options: python3 or java8.
model_reference	Required	String	Name of the model that the model version belongs to, which is from the model_name_output outputted by the Model operator.
publisher	Required	String	Model creator name.
minio_paths	Optional	List	minio path. See minio_path sample.
git_setting	Optional	List	Git source setting. See git_setting sample.
s2i_model	Optional	String	Name of s2i model file.

Output Parameters Description¶

Parameter	Description
create_model_revision	Whether the model version is built successfully.
model_revision_name	Name of the staged model version.
model_builder_name	Model build name.

Thirdparty Model Version Register Operator¶

The Thirdparty Model Version Register operator is used to stage the model version by packaging the model as the Docker image file in the third-party custom system.

Input Parameters Description¶

Name	Required/Optional	Type	Description
input_data	Required	String	Model version parameter input. See the input_data sample.
alias	Optional	String	Alias of the model.
version_rule	Required	String	Model version naming rule, where the time basis is taken by default to name the model version according to the current timestamp.
annotation	Optional	String	Model version description information.
architecture	Required	String	Basic hardware for running model version; options: x86 or arm.
coprocessor	Optional	String	Select the coprocessor for running the model version; options: GPU, TPV, VPU or None.
env_param	Optional	List	Environment parameter. See the env_param sample.
framework	Required	String	Calculation framework for running the model version; options: sklearn, tensorflow, pytorch, h2o, spark, etc.
language	Required	String	Language used for model version development; options: python3 or java8.
model_reference	Required	String	Name of the model that the model version belongs to, which is from the model_name_output outputted by the Model operator.
publisher	Required	String	Model creator name.
thirdparty_url	Optional	String	Load the address of the model version file.

Output Parameters Description¶

Parameter	Description
create_model_revision	Whether the model version is built successfully.
model_revision_name	Name of the staged model version.
model_builder_name	Model build name.

Latest Model Version Operator¶

The Latest Model Version Operator can be used to get the latest model version of a model.

Input Parameters Description¶

Name	Required/Optional	Type	Description
model_name	Required	model_name	Specify the name of the target model.
serve_as_file	Required	boolean	Choose whether to provide services in the form of model files.

Output Parameters Description¶

Name	Type	Description
lastest_version	model_version	The latest model version.

Mlflow Model Version Prediction Operator¶

The Mlflow model version prediction operator is used to perform the prediction based on the machine learning model images.

Input Parameters Description¶

Name	Required/Optional	Type	Description
model_name	Required	model_name	Model name
model_version_name	Required	model_version	Select the model version.
data	Required	File	Data file.
data_type	Required	data_type	Select the data type, options are `csv` and `json`.

Output Parameters Description¶

Name	Type	Description
predictions	File	Prediction results.

Model Instance Operator¶

The model instance operator is used to create model deployment instances.

Input Parameters Description¶

Name	Required/Optional	Type	Description
name	Required	String	Deployment instance name.
resource_pool	Required	String	Name of the resource pool, which comes from the resource name requested through Resource Management.
model_name	Required	String	Model name, which is from the model_name_output outputted by the Model operator.
labels	Optional	List	Tag list.
description	Optional	String	Deployment instance description.
deploy_mode	Required	String	Deployment mode, where only ONLINE (that is, current environment deployment) is supported.
error_on_exist	Required	String	Specify whether to skip without reporting an error when the model name exists. If false is selected, no error will be reported; if `true`is selected and the name of model already exists, an error will be reported directly.

Output Parameters Description¶

Name	Type	Description
instance_name_output	String	Deployment instance name.

Model Test Operator¶

The model testing operator is used to test the staged model version.

Input Parameters Description¶

Name	Required/Optional	Type	Description
input_data	Required	String	Enter the model testing data in JSON format.
model_builder	Required	String	Model builder, which comes from the model_builder_name outputted by any Model Version Register operator.

Output Parameters Description¶

Name	Type	Description
create_model_test	String	Model test name.
model_test_output	String	Model test return results in JSON format.

Single Model Deployment Operator¶

The model deployment operator is used to deploy a single model version online.

Input Parameters Description¶

Name	Required/Optional	Type	Description
model_revision	Required	String	Name of the model version to be deployed, which comes from the model_revision_name outputted by any Model Version Register operator.
instance_name	Required	String	Deployment instance name, which comes from the instance_name_output outputted by the Model Instance operator.
request_cpu	Required	Number	CPU request value required to deploy the model version (the minimum value is 0.01 core).
request_memory	Required	Number	Memory request value required to deploy the model version (the minimum value is 0.1 GB).
limit_cpu	Required	Number	Upper limit of CPU required to deploy the model version (the maximum value is 32 core).
limit_memory	Required	Number	Upper limit of memory required to deploy the model version (the maximum value is 64 GB).
timeout	Required	Number	Timeout setting.

Output Parameters Description¶

Name	Type	Description
create_model_deployment	String	Name of the deployed model version.

Model Version Info Operator¶

The Model Version Info operator is used to get the information on model versions in multi-deployment scenarios.

Note

You can only add this operator in the sub-canvas of the Model Deployment operator.

Input Parameters Description¶

Name	Required/Optional	Type	Description
model_name	Required	model_name	Automatically select the name of the model specified in the model_name parameter of the Model Deployment operator.
model_revision	Required	model_version	Select a model version from the dropdown list.
request_cpu	Required	request_cpu	Specify the value of CPU request.
request_memory	Required	request_memory	Specify the value of memory request.
limit_cpu	Required	limit_cpu	Specify the value of CPU limit.
limit_memory	Required	limit_memory	Specify the value of memory limit.
canary_rate	Optional	number	Specify the percentage of traffic to the model version. It only takes effect in the Canary Deployment.
bg_rate	Optional	number	Specify the percentage of weighted combination optimization to the model version. It only takes effect in the Blue/Green Deployment.

Output Parameters Description¶

Name	Type	Description
version_deploy_info	version_deploy_info	The information of the model version to be deployed.

Model Deployment Operator¶

The Model Deployment operator is used to deploy multiple model versions.

Note

To run this operator, you need to add at least 2 Model Version Info operators to the sub-canvas of the Model Deployment operator.

Input Parameters Description¶

br />

Name	Required/Optional	Type	Description
model_name	Required	model_name	Select the model to deploy.
instance_name	Required	instance_name	Select the model version instance to deploy.
request_timeout	Required	request_timeout	Enter the timeout threshold for requesting.
deploy_timeout	Optional	deploy_timeout	Enter the timeout threshold for deploying.
using_token	Required	boolean	Choose whether using token.
upgrade_method	Required	upgrade_method	The method to update model versions, supporting Blue/Green, Canary, and Multi-Armed.
combiner_strategy	Required	combiner_strategy	The strategy for model combination optimization, supporting Weighted Average and Specify Only. It only takes effect in the Blue/Green Deployment.
specify_version	Required	string	Specify the model version for the Specify Only optimization strategy. It only takes effect for the Specify Only optimization strategy.

Output Parameters Description¶

Name	Type	Description
create_model_deployment	string	The name of the deployment instance.

Service Prediction Operator¶

Achieve the prediction function based on model deployment service.

Input Parameters Description¶

Name	Required/Optional	Type	Description
model	Required	String	Model name.
instance	Required	String	Deployment instance name.
namespace	Required	String	Name of the resource pool, which comes from the resource name requested through Resource Management.
data_type	Required	String	Data type. Options: csv or json. The first row of a csv file should be headers. The data in the json file must be complete in the following format: { "data": { "names": [ "AGE", "RACE", "DCAPS", "VOL" ], "ndarray": [ [ 0, 0, 0, 0 ] ] } }
data	Required	File	Input data.

Output Parameters Description¶

Name	Type	Description
predictions	File	Output prediction results.

External Mlflow Prediction Operator¶

The external Mlflow prediction operator is used to perform the prediction based on the external environment through the model file recorded by Mlflow. The external environment refers to the Jupyter Lab environment in the third-party Lab.

Input Parameters Description¶

Name	Required/Optional	Type	Description
model_path	Required	Directory	Model file path, which can be derived from the output of the Git Directory operator.
data_type	Required	String	Select the data type, options are `csv` and `json`.
data	Required	File	Input data; options: json and csv.
requirements	Optional	List	Content of the requirements.txt file, which is outputted in List format, such as: pandas==1.0.0.

Output Parameters Description¶

Name	Type	Description
predictions	File	Output prediction results.

Internal Mlflow Prediction Operator¶

The internal Mlflow prediction operator is used to perform the prediction based on the internal environment through the model file recorded by Mlflow. The internal environment refers to the Jupyter Lab environment in the AI Lab.

Input Parameters Description¶

Name	Required/Optional	Type	Description
minio_paths	Required	List	Model file path, which is the path of the model file on minio.
data_type	Required	String	Select the data type, options are `csv` and `json`.
data	Required	File	Input data; options: json and csv.
requirements	Optional	List	Content of the requirements.txt file, which is outputted in List format, such as: pandas==1.0.0.

Output Parameters Description¶

Name	Type	Description
predictions	File	Output prediction results.

Model Monitor Operator¶

The model monitor operator is used to monitor the performance of a specified model in specified period of time. Use the returned metrics result as input of business decisions, such as whether to train the model again or use other algorithms to train new models for replacing the current model version. A model deployment instance can have multiple monitoring indicator, and each model monitor operator can monitor one indicator only.

Input Parameters Description¶

Name	Required/Optional	Type	Description
model_name	Required	String	Model name.
deployment_instance	Required	String	Model deployment instance name.
start_time	Required	Timestamp	Start time for the model monitoring (selected using the data/time widget, in second unit).
end_time	Required	Timestamp	End time for the model monitoring (selected using the data/time widget, in second unit).
metric_name	Required	String	Monitoring metric name (customized indicator added on the Indicator Monitoring page of AI Hub > Deployment Instance).

Output Parameters Description¶

Name	Type	Description
model_indicator_data	List	Output model metric monitoring results.

Model Log Operator¶

The Model Log operator can be used to get up to 5000 logs of a model instance in the last 7 days.

Input Parameters Description¶

Name	Required/Optional	Type	Description
model_name	Required	String	Specify the Model.
instance_name	Required	model_instance	Specify the model instance.
keyword	Optional	String	Specify the key words to filter model logs.
time_duration	Required	time_duration	Specify the time duration to get model logs.

Output Parameters Description¶

Name	Type	Description
matched_times	number	The time information of model logs.
log	file	Model log file.