Export Flow¶
Export a workflow configuration.
Prerequisite¶
The user must belong to the organization (OU) which the target workflow belongs to.
Request Format¶
GET https://{apigw-address}/dataflow-batch-service/v2.0/flows?action=export
Request Parameters (URI)¶
Name | Location (Path/Query) | Mandatory/Optional | Data Type | Description |
---|---|---|---|---|
flowId | Query | Mandatory | Integer | The workflow ID. |
userId | Query | Mandatory | String | The user ID. How to get userId>> |
orgId | Query | Mandatory | String | The organization ID which the user belongs to. How to get orgId>> |
Response Parameters¶
Name | Data Type | Description |
---|---|---|
data | Flow Struct | The detailed information of the workflow. For more information, see Flow Struct |
Flow Struct¶
Sample¶
{
"name": "workflow1",
"cycle": "D",
"cron": "0 0 0 * * ? *",
"parameters": "[]",
"alertMode": 3,
"alertTo": "",
"appId": "",
"submitter": "yourSubmitter",
"owners": "yourOwners",
"visitors": "yourVisitors;",
"type": 1,
"syncType": 1,
"desc": "",
"startTime": "2019-07-25",
"tasks": [
{
"name": "tass",
"resource": "default",
"type": "COMMAND",
"runMode": "{\"taskMode\":1,\"cpu\":0.5,\"memory\":1,\"maxParallel\":0,\"keyType\":0,\"datasourceId\":0,\"path\":\"\",\"content\":\"\"}",
"syncType": 1,
"cmd": "echo "hello"",
"submitter": "",
"filePackage": "",
"cron": "",
"priorityLevel": 0,
"timeout": 300,
"retryLimit": 3,
"retryInterval": 0,
"successCode": "0",
"waitCode": "",
"asLink": false
}
],
"flowLinks": [],
"taskLinks": [],
"relations": [],
"linkRelations": []
}
Parameters¶
Name | Data Type | Description |
---|---|---|
name | String | The name of the workflow. |
cycle | String | The scheduling cycle.
|
cron | String | If CronTab syntax is used when specifying the time to start the workflow in a scheduling cycle, a 7-character CronTab expression is used. For more information about CronTab, see the Cron Expression Explainer |
parameters | Map | The global parameters that are configured for the workflow to dynamically adapt to the environment changes (in the format of key=value). |
alertMode | Integer | The alert mode.
|
alertTo | String | The receiver of the alerts. |
appId | String | Null string |
submitter | String | The workflow submission account (the big data account corresponding to the organization). |
owners | String | The username of the owners of the workflow. The semicolon (;) is used to separate each owner, for example, owners=”userNameA;userNameB”. |
visitors | String | The username of the vistors who can access the workflow. The semicolon (;) is used to separate each visitor, for example, visitors=”userNameA;userNameB”. |
type | Integer | The scheduling type of the workflow.
|
syncType | Integer | The sync type of the workflow.
|
desc | String | The workflow description. |
startTime | String | The effective date of the workflow. |
tasks | Array | The collection of task nodes in the workflow, with each element representing a task in the workflow. For more information, see Task Struct |
flowLinks | Array of FlowLink Structs | The collection of workflow references, with each element representing a source workflow that the current workflow depends on. (Using the linkId in the FlowLink Struct, you can find the related link relation in the linkRelations collection. The link relation represents the relationship between a specific task in the current workflow with the source workflow). |
taskLinks | Array of TaskLink Structs | The collection of task references, with each element representing a task in a source workflow that a task in the current workflow depends on. (Using the linkId in the TaskLink Struct, you can find the related link relation in the linkRelations collection. The link relation represents the relationship between a specific task in the current workflow with the source workflow). |
relations | Array of Relation Structs | The collection of relations between tasks, with each element representing the upstream and downstream dependency. For more information, see Relation Struct |
linkRelations | Array of LinkRelation Structs | The collection of link relations. For more information, see LinkRelation Struct |
Task Struct¶
Sample¶
{
"name": "tass",
"resource": "default",
"type": "COMMAND",
"runMode": "{\"taskMode\":1,\"cpu\":0.5,\"memory\":1,\"maxParallel\":0,\"keyType\":0,\"datasourceId\":0,\"path\":\"\",\"content\":\"\"}",
"syncType": 1,
"cmd": "echo 'hello'",
"submitter": "",
"filePackage": "",
"cron": "",
"priorityLevel": 0,
"timeout": 300,
"retryLimit": 3,
"retryInterval": 0,
"successCode": "0",
"waitCode": "",
"asLink": false
}
Parameters¶
Name | Data Type | Description |
---|---|---|
name | String | The task name. |
resource | String | The task resource. |
type | Integer | The task type.
|
syncType | Integer | The sync type of the task.
|
cmd | String | The command line. |
submitter | String | The workflow submission account (the big data account corresponding to the organization). |
filePackage | String | The file location. |
cron | String | The time for scheduling the task. |
priorityLevel | Integer | The priority level of the task. |
timeout | Integer | The timeout value. |
retryLimit | Integer | The number of retries. |
retryInterval | Integer | The interval for retry. |
successCode | String | The success returned code. |
waitCode | String | Null string |
asLink | Boolean |
|
runMode | String | Task running mode. For more information, see Run Mode Struct |
Run Mode Struct¶
Sample¶
{
"taskMode": 1,
"cpu": 0.5,
"memory": 1,
"maxParallel": 0,
"keyType": 0,
"datasourceId": 0,
"path": "",
"content": ""
}
Parameters¶
Name | Data Type | Description |
---|---|---|
taskMode | Integer | Task running mode (1: Single Task; 2: Multiple Tasks). |
cpu | Float | CPU (unit: core; minimun: 0.1; maximum 2) that is required by running each task (single task or sub-task of multiple tasks). |
memory | Float | Memory (unit: G; minimun: 0.3; maximum 4) that is required by running each task (single task or sub-task of multiple tasks). |
maxParallel | Integer | In multiple tasks mode, the maximum concurrency of sub-tasks. |
keyType | Integer | In multiple tasks mode, the source of the distribution key (1: external files; 2: custom key, specified by the content parameter). |
datasourceId | Integer | When the source of the distribution key is from external files, the data source ID (generated when registering the data source through the Data Source Registration service). |
path | String | When the source of the distribution key is from external files, the path of external data source that stores the distribution key file. |
content | String | When the source of the distribution key is custom key, the content of the distribution key. |
FlowLink Struct¶
Sample¶
{
"linkId": "0",
"sourceFlowName": "jin"
}
Parameters¶
Name | Data Type | Description |
---|---|---|
linkId | String | The link ID. |
sourceFlowName | String | The name of the upstream workflow. |
TaskLink Struct¶
Sample¶
{
"linkId": "0",
"sourceFlowName": "dw",
"sourceTaskName": "tass"
}
Parameters¶
Name | Data Type | Description |
---|---|---|
linkId | String | The link ID. |
sourceFlowName | String | The name of the upstream workflow. |
sourceTaskName | String | The name of the upstream task (the exact upstream task can be found when used with sourceFlowName ). |
Relation Struct¶
Sample¶
{
"sourceTaskName": "tass",
"targetTaskName": "rf",
"rerun": true
}
Parameters¶
Name | Data Type | Description |
---|---|---|
sourceTaskName | String | The name of the upstream task. |
targetTaskName | String | The name of the downstream task. |
rerun | Boolean | Effective only for task-level re-running.
|
LinkRelation Struct¶
Sample¶
{
"linkId": "0",
"targetTaskName": "tass",
"rerun": false
}
Parameters¶
Name | Data Type | Description |
---|---|---|
linkId | String | The link ID. |
targetTaskName | String | The name of the downstream task. |
rerun | Boolean |
|
Error Code¶
Code | Message | Description |
---|---|---|
62102 | Flow validation exception | The format of the request body is invalid. |
62109 | Server internal exception | Internal server exception. |
Samples¶
Request Sample¶
url: https://{apigw-address}/dataflow-batch-service/v2.0/flows?action=export&flowId=2924&userId=yourUserId&orgId=yourUserId
method: GET
Return Sample¶
{
"status": 0,
"msg": "Success",
"data": {
"name": "nFlow",
"cycle": "D",
"cron": "0 0 0 * * ? *",
"parameters": "[]",
"alertMode": 3,
"alertTo": "",
"appId": "",
"submitter": "yourSubmitter",
"owners": "yourOwners",
"visitors": ";yourVisitors;",
"type": 1,
"syncType": 1,
"desc": "ga",
"startTime": "2019-07-24",
"tasks": [
{
"name": "tass",
"resource": "default",
"type": "COMMAND",
"syncType": 1,
"cmd": "echo 'hello'",
"submitter": "",
"filePackage": "",
"cron": "",
"priorityLevel": 0,
"timeout": 300,
"retryLimit": 3,
"retryInterval": 0,
"successCode": "0",
"waitCode": "",
"asLink": false,
"runMode": "{\"taskMode\":1,\"cpu\":0.5,\"memory\":1,\"maxParallel\":0,\"keyType\":0,\"datasourceId\":0,\"path\":\"\",\"content\":\"\"}",
}
],
"flowLinks": [],
"taskLinks": [],
"relations": [],
"linkRelations": []
}
}
Java SDK Sample¶
import com.alibaba.fastjson.JSONObject;
import com.envision.apim.poseidon.config.PConfig;
import com.envision.apim.poseidon.core.Poseidon;
public class Request extends PoseidonRequest {
public void setQueryParam(String key, Object value){
QueryParams().put(key, value);
}
public void setHeaderParam(String key, String value){
headerParams().put(key, value);
}
public void setBodyParam(Map<String, Object> bodyPara){
bodyParams().putAll(bodyPara);
}
public void setMethod(String method) {
this.method = method;
}
private String method;
public String baseUri() {
return "";
}
public String method() {
return method;
}
}
public void exportFlowTest(){
//1. Select Application Registration from the left navigation bar of EnOS Console.
//2. Open the App Detail page to get the AccessKey and SecretKey of the application.
String accessKey = "yourAppAccessKey";
String secretKey = "yourAppSecretKey";
//Create a request and save the required parameters in the map of the Query.
Request request = new Request();
request.setQueryParam("flowId","2924");
request.setQueryParam("userId","yourUserId");
request.setQueryParam("orgId","yourOrgId");
request.setMethod("GET");
try {
JSONObject response = Poseidon.config(PConfig.init().appKey(accessKey).appSecret(secretKey).debug())
.url("https://{apigw-address}/dataflow-batch-service/v2.0/flows?action=export")
.getResponse(request, JSONObject.class);
System.out.println(response);
} catch (Exception e) {
e.printStackTrace();
}
}