- Enterprise Data Catalog for Cloud Data Integration
This solution accelerator helps configure the Enterprise Data Catalog for Cloud Data Integration. Powered by EDC and IICS, Informatica provides you with an option to search against the Catalog directly from IICS UI.
With this option, you can discover the Catalog assets directly from IICS UI instead of logging in to Catalog UI. The EDC assets can achieve IICS mappings, synchronization tasks, and Mass ingestion tasks.
This solution accelerator helped you configure the Enterprise Data Catalog for Cloud Data Integration.
With this option, you have discovered the Catalog assets directly from IICS UI instead of logging in to Catalog UI. The EDC assets had you achieve IICS mappings, synchronization tasks, and Mass ingestion tasks.
If the customer uses EDC and IICS, Informatica provides the customer with an option to search against the catalog directly from IICS UI.
With this option, customers can discover the catalog assets directly from IICS UI instead of logging in to Catalog UI and use the discovered EDC assets in IICS mappings, synchronization tasks, and Mass ingestion tasks.
- Customers should have EDC services configured, up, and running.
- Customers should have the following custom licenses with IICS:
- EDC for IICS
We need to configure EDC integration properties for each cloud organization.
1. Log in to IICS with a user account that has the Administrator role.
2. Select Administrator services.
3. Click on Organization from the Left Pane.
4. Scroll down to Enterprise Data Catalog properties.
The following describes the Enterprise Data Catalog integration properties:
Catalog URL: URL of the Enterprise Data Catalog Service. Use the following format:http://<fully qualified host name>:<port>
Runtime environment: Name of the Secure Agent group that is used to read data from the Enterprise Data Catalog.The agents in the group that you select must be able to communicate with the Enterprise Data Catalog. Therefore, the Enterprise Data Catalog host must be in the same network as the agent machines, or it must have the appropriate ports open for communication.
User name: Enterprise Data Catalog user account that the Secure Agent uses to access Enterprise Data Catalog. This user account must have privileges to view and search for objects in the Enterprise Data Catalog and perform functions using the Enterprise Data Catalog REST API.
Password: Password for the Enterprise Data Catalog user account.
Show the Data Catalog: Shows and hides the Data Catalog page in Data Integration.
If your organization includes sub-organizations, you can configure different Enterprise Data Catalog integration properties for the parent organization and each sub-organization.The Enterprise Data Catalog integration properties that we configure for the organization apply to all organization users.
Upon successful configuration, we can see the ‘Data Catalog’ option in the Cloud Data Integration service.
Clicking on that option will take us to the following page.
We can use this search field to search for an asset, description, business glossary term, or a data domain. We can perform all EDC search options like sort, filter, etc.
Customers can use the discovered assets to create a synchronization task, Mass ingestion task, or an IICS mapping by selecting the + icon.
You can add an object if the object is a valid source, target, or lookup type for a mapping or a valid source type for the task. For example, you can select an Oracle table to use as the source in a new synchronization task, but you cannot select a Hive table.
When you select the object, Data Integration prompts you to choose the task where you want to use the object and imports the connection if it does not exist.
We can either create a new asset with this object or add it to an existing asset.
Upon clicking one of those options, the Cloud Data Integration service creates/imports the connection to your Organization.
Connection properties vary based on the object type. Data Integration imports most connection properties from the resource configuration in Enterprise Data Catalog, but you must enter other required properties, such as the connection name and password.
After you configure the connection or if the connection already exists, Data Integration adds the object to a new synchronization task, mass ingestion task, or a new or open mapping inventory.
- Search for an asset from the ‘Data Catalog’ option of IICS and add it to mapping, as mentioned in section 3.
- Once you add the source to mapping, a pop will provide the source connection details. We need to add the Netezza and Teradata Connectors from IICS Administrator->Add-On Connectors for using Teradata and Netezza assets as mapping sources.
- Once the source is added to the mapping, assign the above-created source connection in the connection assignment tab.
4. Create a Target connection and assign it to the target.
5. Save and Run the mapping. Customers can track the status of the mapping from the ‘My Jobs’ tab.
Add the “Microsoft Azure SQL Data Warehouse V3” connector from IICS Administrator->Add-on Connectors. Free trail connector for 30 days is available, after which we need to get the license for extending the connector support.
From the Connections tab in Administrator service, customers can create a new Azure SQL.
- We display only the tables, views, and flat-file assets in the search results as customers can only use these with IICS mappings. We will not be able to see non-relational and EDC system resources in the search results.
- Do not append “/ldmcatalog” at the end of the URL in EDC configuration properties as it can lead to access errors.
- We cannot add an asset as a source to an existing IICS mapping task.
- We can’t transfer multiple tables using the single mapping.
- When the source data type is not supported in the Azure SQL Warehouse, the transfer will fail.
- IICS CDI does not show tables/views from Snowflake resources (EIC-47873).