Stream Processing Resource (Old Version)¶
Note
This section refers to Enterprise Data Platform > Stream Processing (Old Version). If you are using the new version of this service, see Stream Processing Resource
Based on Apache Spark™ Streaming, customized and optimized by Envision, the EnOS stream processing service has high scalability, high throughput, and high fault tolerance.
Broadly speaking, the generation of data can be considered as a series of discrete events. When drawing these discrete events on a time axis, an event stream or data stream is formed. Stream data consist of these endless event streams. Stream data is generated continuously from many data sources. However, the size of the stream data is normally smaller than that of the offline data.
The devices connected to a data center, the telemetry data of devices, and the log files generated by mobile or web applications are some common sources of stream data.
For more information, see Stream Processing (Old Version).
Resource Application Scenario¶
Before installing the stream processing templates and StreamSets calculator libraries, or before configuring stream processing jobs, you need to apply for the Stream Processing Resource.
The computing capability of different specifications correspond to different data processing capabilities and efficiencies. The higher the specification, the higher the processing efficiency, and the larger the amount of data processed per unit time.
The amount of data that can be processed by running stream processing tasks per unit time is referred to as data points. The method for counting data points is: Number of measurement points x Number of devices.
Stream Processing Resource Specifications (Old Version)¶
Different resource specifications are available to meet the requirements of different data and processing efficiency. The resource specifications and corresponding data processing capabilities are as per the below.
Specification |
Computing Capability |
Reference |
|---|---|---|
Standard |
1,500 points / second |
With 10 measurement points / second, supporting about 150 devices. |
Standard X 2 |
3,000 points / second |
With 10 measurement points / second, supporting about 300 devices. |
Note
The computing capability refers to the total amount of data that can be processed by running stream processing jobs per unit time. For a stream processing job, the number of data points is equal to the number of measurement points multiplied by the number of devices.