Unit 3: Synchronizing Data from External Data Sources¶
After the master data modeling is done, data sync is required to synchronize the data from external datasource to Data Catalog.
Step 1: Register Open APIs in the Data Source Registration Service¶
The following Open APIs need to be registered:
- Token API
- Call Service API
Log in to the EnOS Management Console and select Data Source Registration from the left navigation bar.
Click Add Data Source.
On the Add Data Source page, complete the relevant configuration.
Click Done to complete Open API registration.
For more about Open API registration configuration, see Registering an Open API Data Source.
Step 2: Configuring Data Sync Tasks¶
After registering the data source module, you need to go back to the Data Catalog to configure the data sync task.
Creating a Wind Farm Master Data Sync Task¶
- In the left navigation bar of the Data Catalog, click Synchronization Task.
- Click New Task > User Synchronization Task.
- On the User Synchronization Task page, enter the following information:
- Basic Information:
- Synchronization Task Name: enter
sync_site
- Data Source Type: select
Open API
- Data Source: select data source configuration in Step 1
- Data Sync Method: tick
Full-load Data
- Synchronization Task Name: enter
- Request Parameters: If the request parameters have been configured for the API in Step 1, fill in the corresponding request parameters in this item. If not configured, this item is displayed as empty.
- Synchronization Configuration:
- Content to Sync: select
entity synchronization
, then continue to select the main data object typeUD_site
- Info Mapping:
- Returned Field for field name: enter
farm_alias
- Returned Field for field siteId: enter
farm_id
- Returned Field for field name: enter
- Data Check: tick
enable real-time data check
- Content to Sync: select
- Basic Information:
- Click OK to complete the creation of the wind farm master data sync task.
- In the data sync task list on the Synchronization Task feature homepage, click Start at the end of the sync task row to start the data sync task.
Creating a Wind Turbine Master Data Sync Task¶
- In the left navigation bar of the Data Catalog, click Sync Task.
- Click New Task > User Synchronization Task.
- On the Create User Synchronization Task page, enter the following information:
- Basic Information:
- Synchronization Task Name: enter
sync_eam_turbine
- Data Source Type: select
Open API
- Data Source: select data source configuration in Step 1
- Data Sync Method: tick
Full-load Data
- Synchronization Task Name: enter
- Request Parameters: If the request parameters have been configured for the API in Step 1, fill in the corresponding request parameters in this item. If not configured, this item is displayed as empty.
- Synchronization Configuration:
- Content to Sync: select
entity synchronization, then continue to select the main data object type
UD_EAM_Turbine` - Info Mapping:
- Returned Field for field name: enter
description
- Returned Field for field turbineId: enter
location
- Returned Field for field turbineHeight: enter
towheight
- Returned Field for field latitude: enter
actual_lat
- Returned Field for field altitude: enter
actual_lon
- Returned Field for field name: enter
- Data Check: tick enable
- Content to Sync: select
- Basic Information:
- Click OK to complete the creation of the wind turbine master data sync task.
- In the data sync task list on the Sync Task function homepage, click Start at the end of the wind turbine’s master data sync task row to start the data sync task.
Creating a Monitoring Application Master Data Sync Task¶
- In the left navigation bar of the Data Catalog, click Sync Task.
- Click New Task > User Synchronization Task.
- On the Create User Synchronization Task page, enter the following information:
- Basic Information:
- Synchronization Task Name: enter
sync_app
- Data Source Type: select
Open API
- Data Source: select data source configuration in Step 1
- Data Sync Method: tick
Full-load Data
- Synchronization Task Name: enter
- Request Parameters: If the request parameters have been configured for the API in Step 1, fill in the corresponding request parameters in this item. If not configured, this item is displayed as empty.
- Synchronization Configuration:
- Content to Sync: select
entity synchronization
, then continue to select the main data object typeUD_APP
- Info Mapping:
- Returned Field for field name: enter
app_name
- Returned Field for field appVersion: enter
app_version
- Returned Field for field name: enter
- Data Check: tick enable
- Content to Sync: select
- Basic Information:
- Click OK to complete the creation of the monitoring application master data sync task.
- In the data sync task list on the Sync Task function homepage, click Start at the end of the monitoring application master data sync task row to start the data sync task.
Creating a Relationship Sync Task Between the Wind Turbine and the Wind Turbine Application¶
- In the left navigation bar of the Data Catalog, click Sync Task.
- Click New Task > User Synchronization Task.
- On the Create User Synchronization Task page, enter the following information:
- Basic Information:
- Synchronization Task Name: enter
sync_turbine_app_relation
- Data Source Type: select
Open API
- Data Source: select data source configuration in Step 1
- Data Sync Method: tick
Full-load Data
- Synchronization Task Name: enter
- Request Parameters: If the request parameters have been configured for the API in Step 1, fill in the corresponding request parameters in this item. If not configured, this item is displayed as empty.
- Synchronization Configuration:
- Content to Sync: select
entity relationship synchronization
, then continue to select the object relationshipUD_Turbine_APP_Relation
- Info Mapping:
- Returned Field of model UD_EAM_Turbine and primary key turbineId: enter
wtg_id
- Returned Field of model UD_APP and primary key name: enter
app_name
- Returned Field of model UD_APP and primary key appVersion: enter
app_version
- Returned Field of model UD_EAM_Turbine and primary key turbineId: enter
- Content to Sync: select
- Basic Information:
- Click OK to complete the creation of the wind turbine and wind turbine application relationship sync task.
- In the data sync task list on the Sync Task function homepage, click Start at the end of the wind turbine and wind turbine application relationship sync task row to start the sync task.
Creating a Relationship Sync Task Between the Wind Turbine and the Wind Farm¶
- In the left navigation bar of the Data Catalog, click Sync Task.
- Click New Task > User Synchronization Task.
- On the Create User Synchronization Task page, enter the following information:
- Basic Information:
- Synchronization Task Name: enter
sync_site_turbine_relation
- Data Source Type: select
Open API
- Data Source: select data source configuration in Step 1
- Data Sync Method: tick
Full-load Data
- Synchronization Task Name: enter
- Request Parameters: If the request parameters have been configured for the API in Step 1, fill in the corresponding request parameters in this item. If not configured, this item is displayed as empty.
- Synchronization Configuration:
- Content to Sync: select
entity relationship synchronization
, then continue to select the object relationshipUD_Site_Turbine_Relation
- Info Mapping:
- Returned Field of model UD_site and primary key siteId: enter
site_id
- Returned Field of model UD_EAM_Turbine and primary key turbineId: enter
location
- Returned Field of model UD_site and primary key siteId: enter
- Content to Sync: select
- Basic Information:
- Click OK to complete the creation of the relationship sync task between the wind turbine and the wind farm.
- In the data sync task list on the Synchronization Task function homepage, click Start at the end of the wind turbine and wind farm relationship sync task row to start the sync task.