Data Access and Provisioning

Business Goal

Data is more critical than ever in delivering the right insights across industries and across use cases. However, managing data is only getting harder as we have exponential growth and constantly evolving types of data to keep up with. We are also catering to more user groups than ever before, looking to deliver data to the masses, not just a select few data scientists.

So how can we achieve this? The Data Marketplace plays a key role in this process by providing data owners with the means to curate and publish representations of their best data collections for all to see. This enables business consumers like report writers and dashboard developers as well as technical consumers, like integrators, to shop for approved, high quality data without having to navigate the complex world of the Data Management organization and vast on-premise and cloud data stores.

The foundation for this Data Marketplace lies in good governance, it is critical that organizations are able to:

• Catalog their available data at scale (using AI-powered automation wherever possible) so that data is easier to find and retrieve and its provenance is recognized
• Understand the quality of data across the enterprise (using AI-powered automation wherever possible) to ensure that the best possible data is being shared with consumers
• Comprehend the sensitivity and protection status of the data in question to ensure the correct policies are being enforced in relation to the usage of that data

Only then will data citizens be able to easily browse data collections, shopping for the best possible data for their purposes. Think of Data Marketplace as shopping for the data you need, in the data landscape that your company already owns, using governed provisioning capabilities. This knowledge base will help you get the best value out of your data by using the Marketplace in a structured, consistent manner.

Getting Started with a Data Marketplace

Let’s explore how to get started with Axon Data Marketplace, beginning with identifying core organization and data owners, then moving to Definition, Curation, and finally Publishing and Provisioning information for Data Consumers.

Note: This article is intended to provide an introduction; there are extensive Job Aids and demos available to provide more detail into each step shown here.

The high-level flow for managing and using the Marketplace looks like this:

0. Establish and Configure

To enable the Marketplace and make it a productive and seamless experience for Owners and Consumers there are some initial mandatory setup activities to stage for success:

• Establish a Hierarchy of Data Categories (broad groupings of data) that will make sense to users when shopping. Some examples of this include a top-level Finance category that contains subcategories of Accounting, General Ledger, Accounts Payable, etc. In this way, in addition to keyword searching, users new to the Marketplace can navigate through intuitive hierarchies to locate data that meets their needs.
• Establish Policies for data usage, these may include Acceptable Use, Retention, Privacy (PII, PHI, PCI) and others. These are defined using the Policy facet in Axon in preparation for publishing a Data Collection in the steps that follow.
• Establish potential Delivery Methods for the data. These are the specific means by which access could be provisioned. For example, it may be by pointing a user to an extracted flat file or by providing a link to the AWS S3 Bucket in which the raw data is stored.
• Establish Marketplace Administrators and Technical Owners who will serve as default Data and Technical Owners for all Data Collections where specific people are not defined. In short:
  • The Marketplace Administrator is generally a data owner and will Publish data sets to the Data Marketplace as data collections, assign ownership where they are not the correct person to handle requests and Approve or Reject access requests to data collections where they are.
  • The Technical Owner is a System Administrator for specific data sources, and will organize the provisioning of data access to data consumers, where they are the correct person to do so, or assign ownership where they are not.
  • The Data Consumer is the end user of the curated data and will use Marketplace to Find and request access to data collections or even request that new ones are created where they cannot quite find what they need.

Establishing these roles provides ongoing governance for access and distribution of your most valuable data

With these setup activities in place population of the Marketplace can begin.

1. Create Data Collections

The initial creation of the data collections begins with defining the Data Set in Axon; Informatica recommends reviewing Axon methodologies for onboarding technical data to understand all the nuances, buttypically integrated Axon, IDQ and Enterprise Data Catalog are used to Discover and Curate valuable data in the enterprise, tying the System, Data Set and Attribute technical components to Business Glossaries and other facets in Axon to give context.

Once satisfied with the quality and documentation of this data, the Data Owner will link the data set to the appropriate Policies and notify the Marketplace administrator that it is ready for publication.

The Marketplace Administrator then publishes the data, which involves finding an appropriate place in the Data Category hierarchy for the Data Collection and confirming appropriate provisioning possibilities with the Technical Owner. The Data Collection is now available for shopping.

2. Shop for Data

Now that the Data Collection is in the Marketplace, Data Consumers can search for relevant data to fulfill their use cases. As an example, Joe Smith, a Tableau report writer, is looking to develop a new dashboard summarizing sales operating margin in the Retail arm of his company.

He accesses the Data Marketplace and initially navigates to the Sales area of the hierarchy looking for data sets that meet his needs. He may also use a full text search to simply search on “Margin” or “Sales” or other keywords to uncover potential sources. Alternatively, he could use the glossary filter to ensure the data collection that he is seeking contains the right concepts for his analysis.

3. Request Access

Having now discovered the Data Collection that has the potential to meet his sales dashboarding needs, the Data Consumer will add the item to his shopping basket in Marketplace. Joe Smith, in our example, must describe his business justification for gaining access to the data, so that the Data and Technical Owners can make an educated decision about granting access. The Data Consumer will always have a list of active and fulfilled orders for reference, while the Data and Technical Owners will see those requests pertinent to them, based on their role against a Data Collection.

It should be noted that advanced configurations of Data Marketplace may involve configuring ServiceNow as the workflow tool instead of native Axon (if that is the organization’s fulfillment tool of choice).

4. Approve Access

The approval process has two elements to it: The Data Marketplace element, and the technical system element. Approving the request in Data Marketplace is a straightforward gesture for the Data and Technical Owners using the native workflow to approve or deny access based on the business justification provided by the Data Consumer.

The second part of this is actually granting access to the data itself. If the Delivery Method is, for instance, access to an AWS S3 Bucket, the Technical Owner is responsible for the provisioning of these rights, potentially creating user IDs, updating ACLs etc., for the requested data source. This will vary greatly depending upon the Delivery Method. Data Marketplace is the means by which the access is documented for the user but does not play a role in granting access to the technical source.

Once access is granted, the Data and Technical Owners execute final approval, at which time the Data Consumer will receive notification of the provisioning in their Axon notification center and via email.

5. Deliver Data

This aspect of the process is primarily outside of Data Marketplace. Once granted access, the Data Consumer will attempt to open the data by (in our example) querying the S3 Bucket to validate that data can be viewed and searched and fed into the Operating Margin dashboard. Technical connection and integration are the responsibility of the Data Consumer and their IT team.

The Data Consumer, now connected to the Data and Technical Owners, can provide feedback as to the quality and usefulness of the data, giving a rating and commentary to not only the Owners, but other Data Consumers that may be looking to leverage this information.

6. Track and Manage

Data Marketplace tracks and retains all requests, orders, and feedback in activity logs. An important activity for Data and Technical Owners as well as the overall System Administrators will be establishing and monitoring success metrics for the Marketplace. Specific metrics may include: Fulfillment Time, Fulfillment Accuracy, Quality and Ratings, and overall stakeholder involvement.

These may vary across an organization’s implementation needs and should be combined with other general Data Governance metrics to provide insight into the health of an organizations data governance program from creation through to consumption.