Internal HTTP Client*¶
- This stage supports HTTP requests through GET, POST, or PUT, and supports caching (generating keys with URL and request parameters) for further queries.
- Incompatible with old data formats, i.e. cannot filter data by
ModelId::PointId
.
Configuration¶
The configuration tabs for this stage are General, Basic, Input/Output, Request, and QueryConfig.
General¶
Name | Required? | Description |
---|---|---|
Name | Yes | The name of the stage. |
Description | No | The description of the stage. |
Stage Library | Yes | The streaming operator library to which the stage belongs. |
Required Fields | No | The fields that the data records must contain. If the specified fields are not included, the record will be filtered out. |
Preconditions | No | The conditions that must be satisfied by the data records. Records that do not meet the conditions will be filtered out. For example, ${record:value('/value') > 0} . For the syntax of EL expressions, see Expression Language. |
On Record Error | Yes | The processing method for error data.
|
Basic¶
Name | Required? | Description |
---|---|---|
Enable Cache Fields | No | Specify whether to enable caching. Local cache enables fast data processing. When the pipeline is paused, restarted, or retried, the cached data will be lost. |
ExpireRule | No | Select the expiring time of the cache. |
Quality Filter | No | Filter the data according to the data quality. Only records that meet the quality conditions will be processed by this stage. |
Input/Output¶
Name | Required? | Description |
---|---|---|
Input Measurement | Yes | Data input point |
Output Measurement | Yes | Data output point |
Request¶
Name | Required? | Description |
---|---|---|
Request Method | Yes | Select the API request method. + Get + Post + Put |
Request URL | Yes | Specify the API request URL. |
Param Generate: Position | Yes | Specify the position of the parameters and values that are required by the API request. + URL Path + URL Query + Request Body + Header |
Param Generate: Param Key | Yes | Specify the parameter name. |
Param Generate: URLParam Type | Yes | Select the parameter type. + ByField. If you choose byField, complex formats such as List/Map are not supported, and the value type of the passed parameter will be forced to String. + ByInput |
Param Generate: Param Value | Yes | Specify or select the value of the parameter. |
Access Key | Yes | Enter the access key of the application, which is used for API request authentication (the access key can be generated by registering the application on EnOS Management Console). |
Secret Key | Yes | Enter the secret key of the application. |
QueryConfig¶
Name | Required? | Description |
---|---|---|
Parallel Number | Yes | Select the method of parallel requests. Based on the limit of API service, options are System Default and User Defined.
|
Enable Retry | No | Specify whether to retry upon request failure, and enter a specific number of retry attempts in the Retry Attempts field. |
Cache Filter | No | When cache is enabled, specify conditions for filtering the cache. The default value is true. Aviator expressions are supported. For example:
|
Output Results¶
The output results of this stage are included in the lookupTag
field of the attr
struct. The description of the fields are as follows:
Name | Data Type | Description |
---|---|---|
hasData | Boolean | Indicates whether the API request is successful. |
checkSuccess | Boolean | Indicates whether the response matches the expression specified in Cache Filter. |
response | String | If the request is successful, displays the response data. The returnBy field indicates if the response is returned by Cache or HTTP. |
cacheKey | String | The request URL composed of Request URL + Param Generate configuration . |