Creating a Channel


Before using the Data Federation service to read data from and write data to data storage systems, you need to create a read channel or write channel.

Prerequisites

  • You are able to access the Data Federation module.

  • Your OU has requested for the Data Federation resource.

  • You have registered the data source types that you need through Data Source Registration.

  • Your OU has service accounts by creating or purchasing applications through the Application Registration page.

Creating a Channel

  1. Log in to the EnOS Management Console and click Data Federation > Federation Channels. If your OU has not requested for the Data Federation resource, go to the Resource Management page. If your OU has not registered the required data sources, go to the Data Source Registration page.

  2. Click New Channel to enter the basic information of the channel and bind data sources.

  3. In the Basic Information section, enter the channel name, select the channel type (READ, DOWNLOAD, or WRITE), select the usage of the channel (Testing or Production), and enter a short description about the channel.

  4. In the Data Source section, click + Add to configure the data source information in the pop-up window.

    • Type: Select the data source type from the drop-down list.

    • Data Source: Select the registered data source from the drop-down list.

    • Alias: Set the alias for the selected data source. Once submitted, the data source alias cannot be changed.

    • Execution Queue: If the data source type is HIVE(EnOS), select the queue resource for running batch data processing jobs (the Batch Processing - Queue resource requested through Resource Management)

    _images/data_federation_source.png


  5. Repeat step 4 to add more data sources (a single data source can be added only once, and the data source alias must be unique in the OU).

    _images/data_federation_submit.png


  6. When the above configuration is completed, click Save and Next.

Authorizing the Channel

Open the Channel Authorization page to authorize the created channel to the specified application, and authorize the application to read, download, or write data through the channel.

  1. In the application list, click the Authorize/Revoke button on the right side of the specified application to authorize and revoke the authorization.

    _images/data_federation_auth_new.png

  2. After completing the authorization, click Finish to submit the channel configuration. After submission, the default status of the channel is Stopped and you need to manually start the channel and specify the resources needed to run the channel.

    Note

    Authorized Data Federation channels can be authorized at the time of new creation or by locating the target channel in the channel list in Data Federation > Federation Channels and clicking on the Authorize icon.

Starting the Channel

When the channel is created, you can start the channel with the steps below.

  1. From the list of channels, find the created channel, and click the Start icon.

    _images/starting_federation_channel.png

  2. In the Startup Configuration window, complete the following settings:

    • Select the requested data federation resource for running the channel. If no resource is available, request for the needed resource through the Resource Management page.

    • For read channels, select the query type of the channel (Multiple Sources: When querying data, you can use unified SQL statement for querying data from multiple data sources. Single Source: When querying data, you need to select the target data source, and the SQL statement can be applied to the selected data source only).

    • Enter the amount of resource that is required for running the channel. If the requested resource does not meet the requirement, request for more quota through the Resource Management page.

    _images/starting_federation_channel_1.png

  3. Click OK to start the data federation channel.

    Note

    If the channel is to be used for downloading files, the cross-source analysis feature must be disabled. When the data source Hive uses the JDBC method, complex query statements can cause data return timeouts. For a detailed description of query types, see SQL Statement Samples.

Retrieving the Channel URL

When the channel is running normally, you can get the URL of the channel with the steps below.

  1. From the list of channels, find the running channel, and select … > Get URL.

    _images/data_federation_url.png

  2. In the pop-up window, copy the generic URL, JDBC URL, or OData URL, which can be used for accessing the data separately. For more information about accessing data through each protocol, see Data Read and Write.

    Note

    The download channel only generates generic URLs.

Viewing Channel Log

When the channel status is active, running, or abnormal, the channel log can be viewed by following these steps:

  1. From the list of channels, find the running channel, and select … > View Logs.

    _images/data_federation_log.png

  2. In the View Log page, view the containerName、k8sLog、podName and podLog information.

  3. Click Download Log to download the full size log CSV file.

    Note

    When the channel is restarted, the podID will change. Therefore, you cannot view the historical data, only the current latest logs.