Designing a Development Architecture

Challenge

For an efficient development and production implementation process, it is important to have the development architecture planned prior to the start of development work. Without a solid plan in place there is a risk of confusion and delays in moving objects from a development state into production. Spending time early in the project to ensure this has been thought out and planned will save time and effort later when it is needed most as the production go-live nears.

Description

The Development Architecture is the collection of technology standards, tools, techniques, and services required to develop a solution. The task of designing a development architecture involves developing a testing approach, defining the development environments, and determining the metadata strategy. The benefits of defining the development architecture are achieved later in the project and include good communication and change controls as well as controllable migration procedures. Ignoring proper controls is likely to lead to issues later in the project.

Although the various subtasks that compose this task are described here in linear fashion, all these subtasks relate to the others, so it is important to approach the overall body of work in this task as a whole and consider the development architecture as a whole.

Considerations

The Development Architecture should be designed prior to the actual start of development because many of the decisions made at the beginning of the project can have unforeseen implications once the development team has reached its full size. The design of the Development Architecture must consider numerous factors including the development environment(s), naming standards, developer security, change control procedures, and more.

The scope of a typical implementation, possibly covering more than one project, is much broader than a departmentally scoped solution. It is important to consider this statement fully, because it has implications for the planned deployment of a solution as well as the requisite planning associated with the development environment. The main difference is that a departmental data mart type project can be created with only two or three developers in a short time period. By contrast, a full integration solution involving the creation of an ICC (Integration Competency Center) or an analytic solution that approaches enterprise scale requires more of a "big team" approach. This is because many more organizational groups are involved, adherence to standards is much more important, and testing must be more rigorous since the results will be visible to a larger audience.

The following paragraphs outline some of the key differences between a departmental development effort and an enterprise effort:

With a small development team, the environment may be simplistic:

• Communication between developers is easy; it may literally consist of shouting over a cubicle partition.
• Only one or two repository folders may be necessary since there is negligible risk of the developers "stepping on" each other's work.
• Naming standards are not rigidly enforced.
• Migration procedures are loose; development objects are moved into production without undue emphasis on impact analysis and change control procedures.
• Developer security is ignored; typically, all developers use similar often highly privileged user ids.

However, as the development team grows and the project becomes more complex, this simplified environment leads to serious development issues:

• Developers accustomed to informal communication may not thoroughly inform the entire development team of important changes to shared objects.
• Repository folders originally named to correspond to individual developers will not adequately support subject area- or release-based development groups.
• Developers maintaining others' mappings are likely to spend unnecessary time and effort trying to decipher unfamiliar names.
• Failure to understand the dependencies of shared objects leads to unknown impacts on the dependent objects. The lack of rigor in testing and migrating objects into production leads to runtime bugs and errors in the warehouse loading process.
• Sharing a single developer ID among multiple developers makes it impossible to determine which developer locked a development object, or who made the last change to an object. More importantly, failure to define secured development groups allows all developers to access all folders, leading to the possibility of untested changes being made in test environments.

These factors represent only a subset of the issues that may occur when the development architecture is haphazardly constructed, or "organically" grown. As is the case with the execution environment, a departmental data mart development effort can "get away with" minimal architectural planning. But any serious effort to develop an enterprise-scale analytic solution must be based on well-planned architecture, including both the development and execution environments.

In Data Migration projects it is common to build out a set of reference data tables to support the effort. These often include tables to hold configuration details (valid values), cross-reference specifics, default values, data control structures, and table-driven parameter tables. These structures will be key components in the development of re-usable objects.