Challenge
This document gives an approximate structure of the development architecture document that should be created at the start of every DX project. This document serves several purposes:
- Represents the customer requirements to be implemented using Informatica tools.
- Provides a detailed design of the system, which will serve as a basis for development (either by the customer or Informatica consultants) and, later on, as a basis for the test plan and user guides.
- Serves as a convenient way to gather all the necessary information for development together, which should be shared with the customer in order to spot and eliminate any mistakes or misunderstandings.
Helps estimate the effort required for development and testing. Once all the processes are documented and the implementation steps are clear, it will be much easier to estimate the work and the number of resources needed.
Description
The following describes the sections of a DX project design document. Generally, all the sections are optional, but adding more sections to the final document creates a more valuable project design.
Overview
This section provides a high-level overview of the business and the business requirements that the system is planned to meet.
Components
This section describes the high-level components and products for the project. This gives the reader an insight into which technologies are associated with the system.
Conceptual Design
It is helpful to provide an overview of the project (include a diagram for a visual representation) and specifically indicate the place where Informatica connects to the whole business process. The description should be general without going into too many technical details.
System Design
This section is separated into several aspects of system design.
External Entities and Protocols
This section provides details about third-party (external) software systems that will cooperate with the application. Information about protocols and interfaces will be helpful in building the next sections.
Database Schema
In most projects, the business processes assume that the data is loaded into relational tables. If so, create a database schema of the tables that will be used, since it will help clarify the requirements and plan the development.
Implementation Flowcharts
Normally, DX is integrated with PowerCenter, and together they form a set of business processes that implement the customer-business logic. Frequently, there are several PowerCenter procedures involved in processing, either because there are different stages of the same process or a few independent processes. In such cases, it is beneficial for the customer and the Informatica consultants to develop a set of flowcharts that visually represent the logic of all the business processes to be implemented using Informatica tools (PowerCenter, DX, DT, etc.).
The format of a flowchart is arbitrary but may be dictated by customer preferences or standards. Here is a basic example made in Microsoft Visio:
The Velocity database has a sample deliverable which provides an example of a detailed description for a specific workflow: [Include link here]. If the workflow is complex and requires a detailed design and analysis, it is advised to create such a document and reference it from the system design.
DX Entities Catalog
This section should include information about which DX objects need to be created. They include:
- Applications: generally, there is one application per business process. Note that applications are used together with accounts to resolve profiles. If there is more than one business process associated with a partner or account, then additional applications would need to be created.
- Templates: there should be one DX template for a PowerCenter workflow. However, the number may decrease if not all the PowerCenter workflows will be creating DX events of their own.
- Partners/Profiles: there is no need to list all partners at this stage; rather, only those necessary for development. For profiles, it is essential to specify delayed processing rules. The necessity of having delay rules remains with the implementation logic. Decide what—if any—delay rules are needed after the flowcharts for the workflows are ready.
- Monitors: monitors are used for alerting (and possibly workflow chaining).
- Custom Event Statuses and Types: If there is a need to classify events differently and if built-in statuses do not specify all the stages of business processing, utilize the custom event statuses and types.
- Profile Parameters: these parameters are used to customize workflow execution. Provide a proper description for each parameter, since this information will form a basis for the future operator’s guide.
Supported File Formats and Format Conversion
Frequently, DX is used to retrieve files that need to be parsed/converted/validated. If so, this section should list all file formats that the solution will need to support.
DT Services
If Data Transformation is used in the project, then include the DT services that are to be developed as part of the project. This section is usually related to the previous one, since DT is normally applied to handle file format conversions.
DX Endpoints
If DX is the point of file pick-up, then this section should describe the locations where DX will be configured to listen for incoming files and write the outgoing ones. The situation may become complicated if DX MFT needs to be used to retrieve the files using the standard protocols, like FTP(s), AS2 or etcetera.
This section should list all the folders (or connection details for non-file endpoints) and describe how files from a particular folder will be associated with a DX profile. This can be achieved by having account numbers in a file name, but in more complex scenarios, this may require opening the file).
File Name Convention
This section is partially linked to DX Endpoints and should set the name convention rules for the files that DX reads and writes, which will help avoid ambiguity in the future. Also, this will determine how files should be named when they are sent to DX or received from DX.
Event Status History
Every DX process deals with DX events which represent a piece of work and are used to store important information, like incoming data, results and other pieces of interest for the business.
Status is an essential piece of event information. The status represents the stage of the process, from Pending—when started—to some file status—Complete by default. However, there are several extra statuses that represent intermediary steps or different error situations. This section of the design document should list the pre-built and custom statuses that will be used by the DX processes.
It is recommended to specify briefly what actions should be taken by the customer to resolve error statuses. For example, if the incoming file failed validation, then a respective DX event can be marked with a specific status, and this guide can advise the DX operator how to resolve the issue. Additional recovery information should be placed into a separate document, which is normally created towards the end of development.
Event Model
The event model gathers information regarding what data the events should store, for example, source or target documents. It also defines any relationship between events, for example, reconciliation or hierarchies.
DX Support Alerts and Recovery
Alerting is an important feature of DX. This section should describe when alerting is required, how it will be implemented (e.g., using DX monitors to send warning emails to the customer support) and provide an overview of recovery steps, if required.
Scheduling of Business Processes
There are three ways to run business processes that are implemented in DX/PowerCenter:
- Real-time processing - A PowerCenter workflow listens for incoming JMS messages from DX, which DX generates once it has retrieved files from an endpoint.
- Batch processing - DX starts a PowerCenter workflow once it has picked up a file.
- PowerCenter Workflows - PowerCenter workflows are scheduled using PowerCenter means and either pick up the files without the help of DX or process the messages previously generated by DX.
In this section, it is essential to list the PowerCenter workflows that will be implemented and specify when and how they are started or scheduled.
This information may go to the PowerCenter administrators, and it will be their job to schedule the workflows.
Archiving Policy
Since the number of DX events stored in the DX repository will increase, it is essential to consider the events archiving policy and outline it in this section. The requirements should come from the customer.
Note that fully-featured arching only exists in DX 9.5. In the previous versions, DX could export the events to a file but could not restore them back to the database.
The basic approach to archiving is to delete events that are “N” days old; “N” representing an agreed upon number. This can be accomplished in all versions of DX. In a more complicated scenario events may be archived (and restored) based on their type or status, or partner and account combination.