Overview
The Advanced Cluster is a Kubernetes-based distributed processing environment provided by Informatica to handle complex data integration workloads. It offers scalability and flexibility by enabling mapping logic to run across multiple cloud platforms. There are three types of clusters:
- Fully-managed Clusters: Serverless clusters designed to automatically scale based on workload while minimizing total cost of ownership.
- Self-service Clusters: Kubernetes clusters deployed by organizations, compatible with AWS or Azure cloud platforms.
- Local Clusters: Simple, single-node clusters configured on Secure Agent machines for quick, localized project setups.
These cluster types enable efficient resource utilization, tailored cluster configurations, and enhanced job execution in larger datasets and advanced mapping scenarios.
Pre-Requisites
Technical Prerequisites
- Secure Agent Setup:
- Install and configure the Secure Agent on a Linux-based system (Windows is not supported for Advanced Clusters).
- Ensure the Secure Agent group includes only a single Secure Agent.
- Advanced Configuration:
- Define the cluster type and specify the required resources in the configuration settings.
- Verify that advanced services such as the Elastic Server are up and running.
- Network and Connectivity:
- Allow network traffic to essential domains such as artifacthub.informaticacloud.com. For EMEA regions, also permit access to artifacthubemea.informaticacloud.com.
- Disable or properly configure firewall settings, particularly for local cluster setups.
- Cluster-Specific Requirements:
- Fully-Managed Clusters: Grant minimal permissions to the Secure Agent for dependency management and runtime operations.
- Self-Service Clusters: Verify access to Kubernetes clusters using a valid kubeconfig file.
- Local Clusters: Ensure the Secure Agent machine has at least 8 CPUs, 32GB RAM, and 20GB of free disk space on directories like /tmp.
- Role-Based Permissions:
- Define appropriate IAM roles and policies for Secure Agent, master, and worker nodes to enable resource allocation and processing.
Environment-Specific Prerequisites
- Microsoft Azure:
- Required components include Azure Data Lake Storage Gen2, Key Vault, and a Virtual Network.
- Use Azure-managed identity for secure access to data sources and targets.
- Amazon Web Services (AWS):
- Mandatory services include Amazon S3, EC2 instances, Elastic Load Balancer, and IAM roles.
- For private clusters, specify Virtual Private Cloud (VPC) and subnet configurations.
- Google Cloud Platform (GCP):
- Ensure necessary permissions for GCP services such as Compute Engine and Cloud Storage are granted.
Self-Service Resources
Goals
- Facilitate scalable and efficient execution of data integration tasks in cloud environments.
- Minimize infrastructure costs by using serverless or optimized resource configurations.
- Leverage distributed processing capabilities for handling complex data volumes.
Outcome
- Streamlined onboarding of advanced mappings.
- Access to advanced processing capabilities under minimal resource requirements.
- Improved operational efficiency for data integration tasks.
- Flexibility in choosing and configuring suitable cluster types for specific organizational needs.
- Integration Developers: For mapping implementation and configuration.
- Cloud Architects: For designing runtime environments and advanced configurations.
- Administrators: For monitoring and maintaining cluster operations.
- Business Analysts: For evaluating data outcomes.
Actions
Add to Favorites
Engagement Details
Catalog Type
Ask An Expert
Engagement Category
Feature Clarity
Products
Cloud Data Integration - Elastic (CDI-E)
Engagement Type
Ask An Expert
Adoption Stage
Configure
Implement
Focus Area
Adoption - Technical
Functional
Engagement ID
AAE-CDIE-001
Disclaimer
- All the topics covered in the Success Accelerators/Ask An Expert sessions are intended for guidance and advisory only. This is implicit and it will not be called out under the scope of each engagement.
- Customers need to include their relevant technical/business team members highlighted in each engagement topic to derive the best out of each engagement.
- Customers need to perform any hands-on work by themselves leveraging the guidance from these engagements.
- Customers need to work with Informatica Global Customer Support for any product bugs/issues and troubleshooting.