Unit 5. Run the Pipeline and View Pipeline Running Results¶
Run the Pipeline¶
In this tutorial, you need run the pipeline twice to perform training tasks and prediction tasks. To run the pipeline:
- After saving the pipeline, click the Run icon in the functional operation area.
- Enter
training
as tasktype, and click OK to start training tasks. - After the model training completes, click the Run icon and change the tasktype value to
prediction
. - Enter
service
orfile
as predictiontype to specify the prediction type.- If you enter
service
as predictiontype, the pipeline will make predictions based on published model services. - If you enter
file
as predictiontype, the pipeline will make predictions based on Mlflow model files.
- If you enter
- Specify the value of HDFS_source and hive_source to save the prediction results to HDFS and EnOS Hive.
- Click OK to start prediction tasks.
View Pipeline Running Status and Prediction Results¶
During the pipeline running, you can view the pipeline running status by the following steps:
- Click Run Instances to view the instance list.
- Click the instance name of prediction tasks.
- View the running results in DAG Graph, Detail, and Gantt tabs.
- Under the DAG Graph tab, you can view the running results of running instance, including the running progress, input and output parameters, current running logs, and Pod information of operators.
- Under the Detail tab, you can view the details, running parameters, and advanced configuration of the running instance.
- Under the Gantt tab, you can view the running status distribution diagram of each operator and the running progress.
- After the pipeline is run successfully, you can view the operator outputs under the DAG Graph tab:
- If the predictiontype value is
service
, click the Write results operator to view the input/output parameter information. - If the predictiontype value is
file
, click the Write results 2 operator to view the input/output parameter information.
- If the predictiontype value is
- To view the prediction results:
- Click Create DataSet icon to create a dataset based on the output and analyze the data. For more information, see Dataset Management.
- Click Copy icon , and then open a Notebook instance in MI Lab, open a new document to run the pasted code.
- If the prediction results are saved to EnOS Hive, you can visualize the prediction results using EnOS Digital Twin Visualization (DTV).
(Optional) Visualize the Prediction Results in DTV¶
The EnOS DTV service provides enterprises data visualization solutions. You can use DTV to visualize the prediction results if:
- You are able to access the Data Federation module in the EnOS Management Console and the DTV in the EnOS Application Portal.
- Your OU has requested for EnOS Hive and Data Federation resources.
For more information, see Digital Twin Visualization.
Create a Data Federation Channel¶
Create a Data Federation channel to send the pipeline data to DTV:
Log in to the EnOS Management Console and select Enterprise Data Platform - Data Governance > Data Federation > Federation Channels.
Click New Channel, enter
wind_demo
as Name, then provide the following information in the popup window:Attribute Value Data Source > Add Data Source > Type HIVE (EnOS) Data Source > Add Data Source > Data Source HIVE (EnOS) Data Source > Add Data Source > Alias hive_enos Data Source > Add Data Source > Execution Queue root.eaptest01 Click Save and Next.
In the application list, click the Authorize/Revoke toggle button on the right side of the DTViz application to authorize permission for this application.
Click Finish and note the channel ID.
Connect the Data with DTV¶
Create a DTV Data Source to receive the pipeline data:
Log in to DTV in the EnOS Application Portal, click Dashboards > Data Source on the left navigation pane.
On the External Data Sources tab, click New Data Source and provide the following information in the popup window:
Attribute Value Type Data Federation Name wind_demo ChannelId ID of the data federation channel just created Click OK.
Create a Project and a Page in DTV¶
Create a new project and a new page in DTV:
- Click Dashboard on the left navigation pane, and the project page opens.
- click New Project and enter
Power Forecasting
as Name in the popup window. - Click OK, and the project creates.
- Click New Page and provide the following information.
- Name: Wind
- Project: Power Forecasting
- Click OK, and the page editor opens.
Create a Widget in DTV¶
Create a DTV widget to visualize the prediction results:
On the page editor, click Add icon on the toolbar to open the dropdown menu and click Widget (New).
Click the Advanced Mode toggle button on the right side of the top toolbar to enable more widget editing functions.
From the Chart Type section, select Mix Chart .
Enter
Wind Power Forecast
as Title in the Basic Information section.In the Data Fields section, provide the following information:
Attribute Value Data Source EAP Category Power Prediction Aggregation Raw Data Dimension x_basic_time Measurement r_speed, f_power, r_power, f_speed In the Measurement Style section, provide the following information:
Attribute Dual Y-axis Chart Type r_speed Enable Line f_speed Enable Line r_power Disable Area f_power Disable Area Click Save. You can now view the predicted values and actual values in the chart.
An example of the Wind Power Forecast widget: