Upgrading PowerCenter

Challenge

Upgrading an existing installation of PowerCenter to a newer version encompasses upgrading the repositories, application services, implementing any necessary configuration changes, testing and configuring new features and implementing new options purchased or available as part of upgrade to newer version. The challenge is for data integration administrators to approach the upgrade process in a structured fashion to minimize risk to the environment and on-going project work. The goal is to seamlessly transition to the newer PowerCenter version with minimum downtime for the business and maximum benefits.

Some of the challenges typically encountered during an upgrade include:

• Limiting downtime of any work in progress development.
• Ensuring that development work performed during the upgrade is accurately migrated to the upgraded environment.
• Testing the upgraded environment to ensure that data integration results are identical to the previous version.
• Ensuring that all elements of the various environments (e.g., Development, Test, and Production) are upgraded successfully.

Description

Typical reasons for initiating a PowerCenter upgrade include:

• To take advantage of additional features and capabilities in the new version of PowerCenter which enhances development productivity and administration.
• To keep pace with higher demands for data integration.
• To achieve performance gains.
• To maintain an environment of fully supported software as older PowerCenter versions end support status.
• To support the Infrastructure as old versions of databases and server components are retired and upgraded to new versions.
• To secure your data integration environment with centralized user and role administration, enhanced LDAP integration, and SSL remote access

Upgrade Planning

When planning the upgrade consider the following:

• Developing a detailed project plan as the upgrade should be treated as a project
• Training on new features for Developers and/or Administrators
• Testing that includes a baseline run and regression testing to fully certify the upgrade
• Use of Informatica Data Validation Option (DVO) is highly recommended for testing the Informatica upgrade process
• Communicating with teams that support external touch points (i.e. Scheduling group, Salesforce, FTP groups)
• Ensure support from DBAs and System Administrators group for the upgrade project
• Review the Product Availability Matrix (PAM) to ensure compatibility of all PowerCenter tools with the infrastructure components. This is especially important for operating systems and database versions.
• Complete the list of prerequisites to install the intermediate (if applicable) and final Informatica versions for the upgrade activity
• Analyze impacts of associated OS migration and other infrastructure changes needed to support the new version
• Maintaining a list of command scripts executed as part of the ETL process. Scripts using Informatica command line utility need to be analyzed for any binary path changes as a result of upgrade process
• Review the release notes and new features list

Upgrade Team

Assembling a team of knowledgeable individuals to carry out the PowerCenter upgrade is key to completing the process within schedule and budgetary guidelines. Typically, the upgrade team requires individuals or teams fulfilling the following roles:

• Project Sponsor
• PowerCenter Administrator
• Database Administrator
• System Administrator
• Informatica team - the business and technical users that "own" the various areas in the Informatica environment. These resources are required for knowledge transfer and testing during the upgrade process and after the upgrade is complete.

Upgrade Paths

The upgrade process details depend on which PowerCenter version is being upgraded (and to which newer version). The following summarizes the upgrade paths for the various PowerCenter versions:

• Typically, versions of PowerCenter that are within 1-2 prior versions can upgrade directly to the latest version.
• Older versions of PowerCenter might not be able to be upgraded directly to the newest version. In this case, the upgrade might require several steps to hop to intermediate versions before the final upgrade to the most recent version.
• Some of the included components, such as Metadata Manager, may require some additional upgrade steps beyond upgrading the server binaries and upgrading the repository metadata. If so, these requirements will be detailed in the Upgrade Guide for the new version.
• Always review the Installation and Configuration Guide for the new version to see if new repository database schemas/databases are required and if there are steps, as mentioned above, for specific application service upgrades.
• Always review the Upgrade Guide that indicates special considerations when upgrading from a specific version to the new version. These will include the details of any special steps required, including whether intermediate hops are required.

Configuration Support Manager

The Configuration Support Manager Tool (CSM) is a web-based application to provide a proactive health check of your Informatica environment. This tool provides the following benefits:

• Simple configuration management.
• Automated environment information collection via CSM Client
• Advanced diagnostics including health checks and environment comparison capability
• Proactive alerts (EBFs/KB) by email or RSS subscription)
• Informatica recommends that the CSM tool be installed in all environments prior to the upgrade process to ensure that all environments are similar. If the tool identifies any issues, they can be resolved prior to the actual upgrade to mitigate any issues with the new PowerCenter version.

Types of Upgrades

When planning an upgrade, there are three main types of upgrades:

1. In-Place upgrades where the same server and the same repository databases are being used
2. Parallel upgrades which are performed on new hardware and new or existing database repositories
3. Cloning of an existing set of repository schemas into new database repositories on new hardware

In-Place upgrades are the simplest, since there are no changes to hardware or databases, and there will be a development outage during the upgrade and testing period, and thus no changes happening concurrently with upgrade steps.

While Parallel upgrade tasks are underway, development and support of the older environment might be occurring. To avoid missing those changes in the upgraded instance, the upgrade may need to be repeated after the initial upgrade and full testing has completed. Any changes made for upgrade purposes, such as adding memory to sessions, or modifying the SalesForce.com version in SFDC connections, must also be made again after the repeat upgrade.

When cloning an existing environment, you must edit data in the copied Domain and PowerCenter Repository metadata because items such as the hostname are included in the metadata. After editing, you then perform the upgrade. Cloning requires extra attention and involves some risk. In-Place or Parallel upgrades are performed most frequently.

Upgrade Tips 

Some of the following items may seem obvious but adhering to these tips should help to ensure that the upgrade process goes smoothly.

1. Check the Installation and Configuration Guide for the new version to be sure that required operating system patches, server configuration, operating system user configuration, port numbers that must be opened, database requirements and permissions and sizes are understood and prepared ahead of the upgrade.
2. Work with the DBA to create a location for the any new repository databases required, and perhaps for interim database schemas if intermediate hops are required.
3. Be sure to have sufficient memory and disk space (server and database) for the installed software.
4. As new features are added into PowerCenter, the repository grows in size anywhere from 5 to 25 percent per release to accommodate the metadata for new features. Plan for this increase in all PowerCenter repositories.
5. Always read and save the upgrade log file.
6. Backup the Repository Server and the PowerCenter Server configuration files prior to beginning the upgrade process
7. Backup domains.infa, nodemeta.xml and the domain configuration repository. Use the infasetup backup domain command to backup the domain configuration.
8. If the domain uses native users and groups, export users/groups from the old Informatica version using the infacmd command line utility.
9. Test any Custom Transformation prior to beginning the upgrade; recompiling may be necessary.
10. Ensure that all repositories for upgrade are backed up and that they can be restored successfully. Repositories can be restored to the same database in a different schema to allow an upgrade to be carried out in parallel. Note that the restoration of the repository must be done in the same version as the backup. This is especially useful if the PowerCenter test and development environments reside in a single repository. Backup both from a database backup perspective and from the Informatica command line or Domain backup capabilities. It is usually simpler to restore from the Informatica formatted backup files.
11. When naming nodes and domains in PowerCenter think carefully about the naming convention before the upgrade. While changing the name of a node or the domain later is possible, it is not an easy task since it is embedded in much of the general operation of the product. Avoid using IP addresses and machine names for the domain and node names since over time machine IP addresses and server names may change.
12. If using the Grid option or High Availability option, it is important that the location of Domain logs and files used or created during execution are on a high-performance file system and viewable by all nodes in the domain.
13. If High Availability is configured, the file system should also be highly available. Before choosing a highly available filesystem, read the Informatica support statement for supported file systems. It is also possible to support a highly available environment with purely a highly available database. Request and review the latest high availability pre-installation document to see the requirements to enable high availability for your servers, file systems, and databases.
14. To prevent any folder level permission issues when upgrading to a sandbox environment, export the users and groups and import them into the new domain before upgrading the repository. This will ensure that all user and groups are present when the folders are upgraded.
15. The production cutover plan post upgrade should be well planned and executed at least once in a production like environment without any issues before executing in production.
16. Include backout plans if the upgrade does not go as planned for any reason.
17. If the infa_shared directory is kept in the default location, copy it from the old version to new version and make it accessible to the integration service process. Note that it is recommended that the infa_shared directory is moved from the default location.
18. Adhere to the minimum system requirement pre-requisites for installing/upgrading to Informatica new version
19. If your enterprise uses classic PowerExchange like PWX for z/OS or VSAM, make sure to upgrade the PWX version as well.
20. Some PowerExchange products are installed with the server installation and authorized by the license key. Some of these require re-install of the plugins to the Repository Service that utilize these PowerExchanges. See the Upgrade Guide and the PowerExchange guide for specific guidelines on any required steps to re-register upgraded plugins.
21. Native DB client software installed in new PowerCenter server should also be compatible with the databases supported by the new Informatica version. For example, Informatica supports the Oracle database for the domain and PowerCenter metadata repository. Make sure that the Oracle native DB client installed on the PowerCenter server is compatible with the Oracle database version.
22. Update the auto memory attribute at folder level to ensure good performance for sessions. The Default_Session_Config file should be updated to at least 5% for auto memory attributes. This is a common tuning attribute that is often overlooked for objects.
23. Choose an installation encryption key and secure it. The use of this key is a feature added beginning in 9.6.x and enhances security. The key is required when restoring and upgrading repositories.
24. Re-register any custom transformation (e.g., Union Tx) plugins if there are any mappings which are invalid post upgrade due to plugin validation errors.
25. Scan the server session logs and the metadata exchange tables holding the session log information for the PowerCenter Repository after executing samples of workflows with sessions using various connection types. Specifically look for error messages. In some cases, certain connection types require more memory after upgrading, or might have database connection issues. Find these during the upgrade testing and fix them as needed in each environment being upgraded.

Upgrading Multiple Projects

Be sure to consider the following items if the upgrade involves multiple projects:

• All projects sharing a repository must upgrade at same time. It is not possible to upgrade only certain folders and not others.
• Projects using multiple repositories must all upgrade at same time. If the projects are spread across different Informatica versions, they all must upgrade to the same Informatica version.
• After the upgrade, each project should undergo full regression testing.
• If you are not able to upgrade and test each project at one point in time, consider doing a parallel upgrade and having an interim repository that can be emptied and re-upgraded for each project’s folders in order to stagger the upgrade timeline. This process has some complexity and requires careful planning and execution. Once upgraded, the folders pertinent to the project are migrated from the interim repository to the new official repository once they are the same version and then removed from the old version after testing. This process is repeated as each project is ready to test and all folders from the old version have been upgraded to the new version in the official repository.

Upgrade Project Plan

The full upgrade process from version to version can be time-consuming, particularly around the testing and verification stages. Informatica strongly recommends developing a project plan to track progress and inform managers and team members of the tasks that need to be completed, dependencies, uncertainties or missed steps.

In addition, defect tracking and resolution is key to the success of the actual upgrade process. Capture and communicate any defects encountered during the upgrade.

Scheduling the Upgrade

When an upgrade is scheduled in conjunction with other development work, it is prudent to have it occur within a separate test environment that mimics (or at least closely resembles) production. This reduces the risk of unexpected errors and can decrease the effort spent on the upgrade. It may also allow the development work to continue in parallel with the upgrade effort, depending on the specific site setup.

Environment Impact 

With each new PowerCenter release, there is the potential for the upgrade to affect the data integration environment based on new components and features. Time should be spent in planning the upgrade strategy concerning domains, nodes, domain metadata and the other architectural components. Depending on the complexity of the data integration environment, this may be a minor or major impact.

Single integration server/single repository installations are not likely to notice much of a difference to the architecture, but customers striving for highly-available systems with enterprise scalability may need to spend time understanding how to alter the physical architecture to take advantage of these new features coupled with PowerCenter for their enterprise Data Integration need. With enhanced security structure options, the entire Informatica platform can be secured using SSL protocol and the user authentication can also be integrated with Kerberos authentication to facilitate SSO. Informatica also supports two factor authentication using Kerberos. Planning should be done if you want to leverage single sign on authentication using Kerberos and adhere to your enterprise security demand.

For more information on these architecture changes, reference the PowerCenter documentation and the Best Practice on Domain Configuration.

Upgrade Process

Informatica recommends using the following approach to handle the challenges inherent in an upgrade effort.

Choosing an Appropriate Environment

It is always advisable to have at least three separate environments (one each) for Development, Test, and Production.

The QA (Test) environment is generally the best place to start the upgrade process since it is likely to be the most similar to Production. If possible, select a test sandbox that parallels production as closely as possible. This enables performing data comparisons between PowerCenter versions. An added benefit of starting the upgrade process in a test environment is that development can continue without interruption. The corporate policies on development, test, and sandbox environments and the work that can or cannot be done in them will determine the precise order for the upgrade and any associated development changes. Note that if changes are required as a result of the upgrade, they need to be migrated to Production. Use the existing version to back up the PowerCenter repository, then ensure that the backup works by restoring it to a new schema in the repository database.

Alternatively, begin the upgrade process in the Development environment or create a parallel environment in which to start the effort. The decision to use or copy an existing platform depends on the state of project work across all environments. If it is not possible to set up a parallel environment, the upgrade may start in Development, then progress to the Test and Production systems. Using a parallel environment will minimize development downtime. All changes made in development will need to be incorporated into the parallel environment if the development environment is not upgraded. The important thing is to understand the upgrade process and the business and technical requirements, then adapt the approaches described in this document to one that suits the particular situation. Make sure to replicate your existing data warehouse targets to point to a test schema with identical table and object structures to facilitate a sandbox area for upgrade regression testing. Utilize Informatica Data Validation Option (a separately licensed product) to speed up the upgrade regression testing process and improve testing quality.

Organizing the Upgrade Effort

Begin by evaluating the entire upgrade effort in terms of resources, time and environments. This includes training, availability of database, operating system and PowerCenter administrator resources as well as time to perform the upgrade and carry out the necessary testing in all environments. Refer to the release notes to help identify mappings and other repository objects that may need changes as a result of the upgrade. Also, evaluate the resources and infrastructure required to accommodate new features like the SSO authentication using Kerberos. Make sure that the minimum system requirements for new Informatica version are met. If you are utilizing Informatica Data Validation Option (highly recommended) for upgrade regression testing, make sure that you have procured appropriate product license and the prerequisites are met before starting to regression test the upgrade process.

Also, Check for any new security protocol applied on the database infrastructure and scope the upgrade effort accordingly as this can complicate the installation process, (e.g. If Advance security option is introduced in Oracle database for the new environment.)

Provide detailed training for the Upgrade team to ensure that everyone directly involved in the upgrade process understands the new version and is capable of using it for their development work and to assist others with the upgrade process.

Run regression tests for all components on the old version. If possible, store the results so that they can be used for comparison purposes after the upgrade is complete.

Before beginning the upgrade, be sure to back up the repository and server caches, scripts, logs, bad files, parameter files, source and target files and custom transformations. Also be sure to copy backed-up server files to the new directories as the upgrade progresses.

If you must install the new version on a UNIX environment which also will include the older version, be sure to use separate users and directories for the new install. This allows the user profiles setting environment variables required for execution and installation to have separate values. Be careful to ensure that profile path statements do not overlap between the new and old versions of PowerCenter. For additional information, refer to the installation guide for path statements and environment variables for specific platforms and operating systems.

Prepare for the Upgrade

• Check that all required operating system components, patches, configuration settings and ports are configured on the server
• Check that all required repository metadata databases are configured and have appropriate rights
• Make sure that the operating system level user for the server has been created and has all required rights
• Validate the components against the Product Availability Matrix to be sure there are no incompatible components
• On the server, install any database clients required for the repository and source and target databases
• Edit the environment variables to point to the database clients and libraries in the PATH and LIBPATH and related variables
• Unset the INFA_DOMAINS_FILE and INFA_HOME environment variables on the server before starting the upgrade. (Be sure to reset these after the upgrade.)
• Know the security key used for the initial installation
• Download and unzip/untar the installation binaries on the server
• Download the license key and place it on the server
• Run the pre-install test (included with the installation binaries) before running the full installation/upgrade. This will alert you to many potential issues that will halt the installation.
• Cleanup unnecessary repository object versions in a versioned repository and unnecessary session and workflow logs from the repository metadata before the upgrade. This makes the repository smaller and faster to upgrade. Use the pmcmd TruncateLog or PurgeVersion command tasks or the Repository Manager tool to perform the cleanup. This is especially important in repositories that have been used for a long time, and in development repositories that have a lot of changes and execution runs made, and where this maintenance has not been done in some time.

Installing and Configuring the Software

• Install the new version of the PowerCenter components on the server.
• Ensure that the PowerCenter client is installed on at least one workstation to be used for upgrade testing and that connections to repositories are updated if parallel repositories are being used.
• Re-compile any Custom Transformations if necessary and test them.
• The PowerCenter license key is in the form of a file. During the installation of PowerCenter, you will be prompted for the location of this key file. The key should be saved on the server prior to beginning the installation process.
• Configure the domain, node, repository service and the integration service at the same time. Ensure that all necessary database connections are ready before beginning the installation process.
• If upgrading to PowerCenter from PowerCenter 7.x (or earlier), gather all the configuration files that are going to be used in the automated process to upgrade the Integration Services and Repositories. See the PowerCenter Upgrade Manual for more information on how to gather them and where to locate them for the upgrade process.
• Once the installation has been completed, use the Administrator Tool to perform the upgrade. The Administration Tool URL is http://hostname:portnumber where hostname is the name of their server where the PowerCenter services are installed and port number is the port identified during the installation process. The default port number is 6008.
• Re-register any plug-ins (such as certain PowerExchange products), if applicable, to the newly upgraded environment.
• Install/Upgrade PWX classic client and listener. Configure DBmover.cfg file. (as applicable)
• Both the repository and integration services can be started from the Administrator Tool.
• Analyze upgrade activity logs to identify areas where changes may be required, rerun full regression tests on the upgraded repository.
• Execute test plans. Ensure that there are no failures and all the loads run successfully in the upgraded environment.
• Verify the data to ensure that there are no changes and no additional or missing records.

Testing After the Upgrade

Testing is a key milestone during an upgrade project because it verifies that the upgrade has not changed the behavior of the application. Prior to the upgrade, it is suggested that a baseline set of data be saved to be used during the testing phase to compare the before and after results.

The following describes the approaches that can be used for testing after the upgrade is completed. 

• Brute force execution of each mapping in the environment to compare the baseline with the post-upgrade results. This type of testing is very time-consuming due to the fact that every mapping is being touched; however, this does provide the best coverage.
• Targeted execution of only the mission critical mappings and mappings which use specific data connections not used in the mission critical mappings. This type of testing reduces the testing time and ensures that the most important mappings have not been affected by the upgrade; however, not all the mappings will be touched.

Utilizing Informatica Data Validation Option (DVO, a separately licensed product) can reduce your testing time when compared to writing manual testing scripts and running test cases for regression test. DVO will test that the results of running mappings in the prior and new version of the software are unchanged by the upgrade. DVO executes tests and stores test results which can be retrieved in PDF or HTML format for audit purposes. Custom reports can be generated on DVO test results. DVO provides a UI for data validation testing and is extremely useful in saving time and cost for upgrade regression testing by providing capabilities like auto test generation when the stakes are high in running regression test for large set of Informatica mappings.

When using DVO, run the desired workflows for testing, as indicated previously, and then allow DVO to test all of the target tables modified during the execution of the mappings, comparing them against the results of running the mappings in the old environment on the same data. DVO pinpoints differences in data values for every column and every row selected and displays this in the UI and in reports that can be used to certify the upgrade success or point out issues to investigate.

Implementing Changes

If changes are needed, decide where those changes are going to be made. It is generally advisable to migrate work back from test to an upgraded development environment. Complete the necessary changes and then migrate forward through test to production. Assess the changes when the results from the test runs are available. If changes are made in test and migrated forward to production (which is a significant deviation from Best Practices) remember that these changes still need to be implemented in development. Otherwise, these changes will be lost the next time work is migrated from development to the test environment.

After the Upgrade

• If multiple nodes were configured and the business owns the PowerCenter Grid option, a server grid can be created to test performance gains.
• If the business owns the high-availability option, the environment should be configured for high availability including setting up failover gateway node(s) and designating primary and backup nodes for the various PowerCenter services. In addition, the shared file location for the domain should be located on a highly available, high-performance file server. Lastly, ensure that rep caching is turned on in the repository advanced settings to prevent session initialization issues.
• Start measuring data quality by creating a sample data profile.
• If LDAP is in use, associate LDAP users with PowerCenter users.
• If the older version of PowerCenter had command tasks running batch scripts, make sure that scripts are using appropriate parameterized paths for the infa_shared directory or the Informatica command line utilities. If the paths were hard coded in the older version, the paths would need to be modified in the batch script to use the new Informatica installation directory post upgrade.

Repository Versioning

After upgrading, the repository can be set to versioned if your business purchased the Team-Based Management option and enabled it via the license key.

Keep in mind that once the repository is set to versioned, it cannot be set back to non-versioned. The team-based development option can be invoked in the Administration Tool.

Upgrading pmrep, pmcmd, and infacmd Scripts

• Check to see if any of the pmcmd, pmrep, or infacmd commands have syntax changes or have been replaced by infacmd syntax. If so, plan to modify the scripts appropriately.
• Ensure that the workflow/session folder names match the upgraded names.
• Users that administer the domain must have Manage Service permissions. This is due to the fact that security has been moved to the domain level.

Upgrading XML Definitions

• The upgrade removes namespaces and prefixes for multiple namespaces.
• Circular reference definitions are read-only after the upgrade.
• Some datatypes are changed in XML definitions by the upgrade.

Sequence Generators

• The data type for NEXTVAL and CURRVAL is BIGINT. This is a change from older versions.
• The upgrade process updates the data type in all mappings.
• Downstream transformations must be manually updated with new data type. This must be done when mappings are imported into the 8.6 or newer environments or when they are upgraded in place.
• The End value must be updated to 9223372036854775807.

Informatica Security

• The entire Informatica platform can be enabled for SSL protocol security. The domain can be configured for TLS for domain communication. Make sure to install the SSL certificate and truststore files before configuring TLS for domain configuration.
• You can also enable Informatica Administrator to use HTTPS for secure communication between admin console and domain.
• The Administrator security group is created by default in the Informatica domain. If you are upgrading from an older version which has a manually created group named Administrator, make sure to rename this group to a different name before importing the user/groups to the new domain.

After upgrading, if you want to leverage SSO using Kerberos authentication, make sure that you meet the prerequisites. Refer to the Informatica installation guide for more details and the Informatica Velocity best practice Configuring Informatica Domain Security with Kerberos and SSL.