Custom Operators¶

MI Pipelines allows you to customize new operators to suit your requirements. You can write business logic and define the input and output parameters of the operator according to actual business requirements to implement the operator logic.

Before adding a custom operator, you need to prepare the image used for implementing the operator function. The image can be uploaded to the corresponding image warehouse (harbor) of EnOS or the Docker Hub, and its path will be used as the input when creating a custom operator.

Design Operator Logic¶

Take a custom operator for calculating sum = a + b as an example:

Create a new operator development project, and create a new file named add_sum.py and save it in the C:UsersusernameDocumentsEAPadd_sumsrc directory.
Write the operator logic and save it in the add_sum.py file. See the sample below:

# The following packages are included. No need to download from DockerFile.
import argparse
import sys
from pathlib import Path

# Main function for controlling the operator I/O, getting the input parameters, and reading and writing data.
def main(args):
    parser = argparse.ArgumentParser(description='Returns sum of two arguments')
    parser.add_argument("--a", type=float, required=True) #Input parameter a, operator type: number
    parser.add_argument("--b", type=float, required=True) #Input parameter b, operator type: number
    parser.add_argument("--sum", type=str, required=True) #Output parameter sum, operator type: string
    args = parser.parse_args(args)

    Path(args.sum).parent.mkdir(parents=True, exist_ok=True) #Default output path
    with open(args.sum, 'w') as sum_path:
        sum_path.write('{}'.format(args.a + args.b))

# Entry point
if __name__ == '__main__':
    main(sys.argv[1:])

Containerize Operator Programs¶

Use the following methods to write a Docker file to containerize the operator program:

FROM: specify the base image
WORKDIR: specify the working directory
COPY: copy the directory from the context directory to the specific path of the container
RUN: load dependencies
ENTRYPOINT: set the command to start the program

For example:

FROM harbor.eniot.io/eap/base/python3-x86
COPY . /src
ENTRYPOINT python3 src/add_sum.py

Upload Images¶

You can upload the operator image to the corresponding image warehouse (harbor) of EnOS or the Docker Hub. The following example describes how to upload images to Docker Hub:

Log in to the Docker Hub official website (https://hub.docker.com) and register an account.
Use the docker login command to log in to Docker Hub.
```
docker login -u <account> -p <password>
```
In the directory where the local Docker File is located, use the docker build command to package the image (the “image name” can be specified arbitrarily, and the name will be latest by default if not specified). For example:
```
C:UsersusernameDocumentsEAPadd_sumsrc>docker build -t username/add_sum_cust:v0.1 .
```
In the directory where the local Docker File is located, use the docker push command to upload the image.
```
C:UsersusernameDocumentsEAPadd_sumsrc>docker push username/add_sum_cust:v0.1
```

Add Custom Operators¶

You can add a custom operator to the operator editing canvas of the MI Pipelines by following these steps:

Log in to the EnOS Management Console, and select Enterprise Analytics Platform > Machine Intelligence Studio > MI Pipelines from the left navigation bar to open the Experiment List homepage.
Click the Pipeline Designer for the target experiment in the experiment list to open the offline design editing canvas.
Click the + icon next to Custom in the operator list, select New Operator, and then complete the operator configuration information in the pop-up window:
- Name: enter the name of the custom operator
- Description: enter a brief description of the custom operator
- Image: enter the path of the operator image in Docker Hub
- Command: specify the command line to run (such as the entrypoint of the image)
In the Input Parameter and Output Parameter sections, define the input and output parameters of the operator (including the parameter name, type, default value, and mandatory properties). The names of input and output parameters need to be consistent with the parameters defined in the program file in the image.
Click OK to create a custom operator. Drag the custom operator to the editing canvas to orchestrate the operator into the pipeline for use.

Edit Custom Operators¶

After creating custom operators, you can edit and update the opertors based on your business needs.

In the list of custom operators, select Edit Operator from the menu beside the operator name.
In the Edit Custom Operator pop-up window, you can change the operator configuration except for the operator name.
Click OK to submit the changes.