Migrating Axon/DQ/EDC/DPM from NON-PROD to PROD

Challenge

During architecture planning, questions often arise on how to set up the Axon/DQ/EDC/DPM environments for Data Governance (DG) (not only for the Production environment (PROD), but also for the Non-Production environments (NON-PROD).

Concerns also arise on where data steward and business collaboration work occurs (e.g., NON-PROD or PROD) and how to migrate from one environment to the other.

Description

This best practice provides recommendations for establishing an optimized architecture across PROD and NON-PROD environments that efficiently uses server and environment resources and minimizes the number of licenses required for the DG platform. It will also help in establishing a process to guide day-to-day usage and management of the Informatica Data Governance Platform (DG) products in support of the above recommendations (primarily the Axon, Data Quality, Enterprise Data Catalog and Data Privacy Manager products, but it also can be considered for other products that support the DG Platform such as Informatica Intelligent Cloud Services, persistent Data Masking/Dynamic Data Masking, Test Data Management, Data Archive, etc.).

This best practice will be beneficial for members of Data Governance Councils and Working Groups, Data Stewards, IT personnel or anyone who is responsible for using, managing or maintaining the Axon/DQ/EDC/DPM products or relies on them for decision-making and planning.

General Approach

NON-PROD Environment 

Use a NON-PROD environment to get things set up/configured and to work out the processes for a small subset of systems/processes, including sample EDC scans, DQ profiling against NON_PROD systems, and integration of sample EDC metadata and sample DQ profiles into Axon.

• Security likely won’t allow NON-PROD tools to touch PROD data.
• Using bulk upload spreadsheets is acceptable in the NON-PROD environment where full stakeholder approval for the data load may not be needed while the kinks are being worked out.

Prod Environment 

All other work is typically done in the PROD environment.

• Community collaboration is done in PROD via the Axon user interface, so that what’s been accepted and what’s been rejected (and associated discussions) can be tracked (via the workflow process).
• Using bulk upload templates in the PROD environment after the cutover (using a copy of NON-PROD data) is discouraged because standalone spreadsheets circumvent the community collaboration process established in the Axon workflows. Bulk uploading of data is an all or nothing approval process that may leave changes out of the otherwise established workflow processes. If bulk uploads are necessary in the PROD environment, a process needs to be established for how all the stakeholders will be notified that changes are taking place so that they can approve them.
• Bringing on additional Axon Glossary items, Processes, Systems, Policies, etc. is done in the PROD environment and follows the patterns/processes established in NON_PROD.
• However, if Axon is being loaded in phases (for example by Business Units), it is encouraged to bulk upload the new business unit data into the NON_PROD environment first to ensure its correctness and then re-use the bulk upload spreadsheets to load the PROD environment.

Note:  If workflows are set to automatically invoke upon the creation of a facet, it may be best to modify the notification from alerting on every instance and instead add it to a daily summary notification.

• Full EDC Scans and full DQ Profiling is done in PROD against PROD systems. PROD environments have the horsepower to do the full scans/profiles. As a cost savings measure, Non-Prod environments typically don’t have the same memory/cores/CPU processing that PROD has.

Development 

Data Governance using Axon, DQ, EDC, DPM doesn’t follow the typical Systems Development Lifecycle (SDLC) process that code development does with migrating things from DEV à TEST à Pre-PROD à PROD.

Most of the work is done directly in PROD, with a NON-PROD environment only used to establish a sample of data with a sample set of processes/systems enough to set the configurations, workflows and processes that will be followed by the rest of the Systems/Processes in the PROD environment, and to test upgrades/fixes before applying to PROD (that’s why Axon has different states for work going from DRAFT à In Review à Accepted or Rejected

Other Recommendations 

For Axon, a onetime backup/restore can be done from Non-Prod to Prod after the tool is installed and configured and some pilot use cases are developed in Axon’s Non-Prod environment.

• All DG activities in Axon should happen in the PROD environment after this point.
• For Workflow designs, the bpmn file can be downloaded by navigating to each workflow and then these files can be uploaded into the other environment. While uploading the bpmn file, the workflow template in the other environment must be blank.
  • There is a big caveat in this approach from a workflow perspective: Role ID’s must match exactly between the two environments since this is what Axon uses to identify the swim lanes. You may need to do a search and replace in the source file to make it match the role identifiers in the target.
• As of Axon 7.2 there is a Data Migration feature to move data from the Non-Prod to PROD environment. However there are some limitations to this approach. It’s similar to the bulk upload approach, but instead of migrating each facet and it’s data individually it does it all at the same time. These are the items that are not migrated with the Data Migration feature:
  • Customized changes to drop down choice lists
  • Custom field definitions (though the data for the custom field is migrated, therefore the custom field has to be configured in the PROD environment before the Data Migration utility is used).
  • Custom Workflows
  • Custom Roles
  • Essentially anything the customer configures in Axon different from the Out of the Box (OOTB) configuration is not migrated. These customizations need to be configured directly in the PROD Axon environment.
• For EDC and DPM new data domain development should always happen in the Non-Prod environment. Then it should be migrated to Prod using the EDC/DPM export /import process after testing and validation is done in the NON-PROD EDC/DPM environment against sample data.
• The Non-Prod environment should always receive the product upgrades and patching first with Prod following after. Nothing should be migrated from one environment to another.