Data is more critical than ever in delivering the right insights across industries and across use cases. However, managing data is only getting harder as we have exponential growth and constantly evolving types of data to keep up with. We are also catering to more user groups than ever before, looking to deliver data to the masses, not just a select few data scientists.
So how can we achieve this? The Data Marketplace plays a key role in this process by providing data owners with the means to curate and publish representations of their best data collections for all to see. This enables business consumers like report writers and dashboard developers as well as technical consumers, like integrators, to shop for approved, high quality data without having to navigate the complex world of the Data Management organization and vast on-premise and cloud data stores.
The foundation for this Data Marketplace lies in good governance, it is critical that organizations are able to:
Only then will data citizens be able to easily browse data collections, shopping for the best possible data for their purposes. Think of Data Marketplace as shopping for the data you need, in the data landscape that your company already owns, using governed provisioning capabilities. This knowledge base will help you get the best value out of your data by using the Marketplace in a structured, consistent manner.
Let’s explore how to get started with Axon Data Marketplace, beginning with identifying core organization and data owners, then moving to Definition, Curation, and finally Publishing and Provisioning information for Data Consumers.
Note: This article is intended to provide an introduction; there are extensive Job Aids and demos available to provide more detail into each step shown here.
The high-level flow for managing and using the Marketplace looks like this:
To enable the Marketplace and make it a productive and seamless experience for Owners and Consumers there are some initial mandatory setup activities to stage for success:
Establishing these roles provides ongoing governance for access and distribution of your most valuable data
With these setup activities in place population of the Marketplace can begin.
The initial creation of the data collections begins with defining the Data Set in Axon; Informatica recommends reviewing Axon methodologies for onboarding technical data to understand all the nuances, buttypically integrated Axon, IDQ and Enterprise Data Catalog are used to Discover and Curate valuable data in the enterprise, tying the System, Data Set and Attribute technical components to Business Glossaries and other facets in Axon to give context.
Once satisfied with the quality and documentation of this data, the Data Owner will link the data set to the appropriate Policies and notify the Marketplace administrator that it is ready for publication.
The Marketplace Administrator then publishes the data, which involves finding an appropriate place in the Data Category hierarchy for the Data Collection and confirming appropriate provisioning possibilities with the Technical Owner. The Data Collection is now available for shopping.
Now that the Data Collection is in the Marketplace, Data Consumers can search for relevant data to fulfill their use cases. As an example, Joe Smith, a Tableau report writer, is looking to develop a new dashboard summarizing sales operating margin in the Retail arm of his company.
He accesses the Data Marketplace and initially navigates to the Sales area of the hierarchy looking for data sets that meet his needs. He may also use a full text search to simply search on “Margin” or “Sales” or other keywords to uncover potential sources. Alternatively, he could use the glossary filter to ensure the data collection that he is seeking contains the right concepts for his analysis.
Having now discovered the Data Collection that has the potential to meet his sales dashboarding needs, the Data Consumer will add the item to his shopping basket in Marketplace. Joe Smith, in our example, must describe his business justification for gaining access to the data, so that the Data and Technical Owners can make an educated decision about granting access. The Data Consumer will always have a list of active and fulfilled orders for reference, while the Data and Technical Owners will see those requests pertinent to them, based on their role against a Data Collection.
It should be noted that advanced configurations of Data Marketplace may involve configuring ServiceNow as the workflow tool instead of native Axon (if that is the organization’s fulfillment tool of choice).
The approval process has two elements to it: The Data Marketplace element, and the technical system element. Approving the request in Data Marketplace is a straightforward gesture for the Data and Technical Owners using the native workflow to approve or deny access based on the business justification provided by the Data Consumer.
The second part of this is actually granting access to the data itself. If the Delivery Method is, for instance, access to an AWS S3 Bucket, the Technical Owner is responsible for the provisioning of these rights, potentially creating user IDs, updating ACLs etc., for the requested data source. This will vary greatly depending upon the Delivery Method. Data Marketplace is the means by which the access is documented for the user but does not play a role in granting access to the technical source.
Once access is granted, the Data and Technical Owners execute final approval, at which time the Data Consumer will receive notification of the provisioning in their Axon notification center and via email.
This aspect of the process is primarily outside of Data Marketplace. Once granted access, the Data Consumer will attempt to open the data by (in our example) querying the S3 Bucket to validate that data can be viewed and searched and fed into the Operating Margin dashboard. Technical connection and integration are the responsibility of the Data Consumer and their IT team.
The Data Consumer, now connected to the Data and Technical Owners, can provide feedback as to the quality and usefulness of the data, giving a rating and commentary to not only the Owners, but other Data Consumers that may be looking to leverage this information.
Data Marketplace tracks and retains all requests, orders, and feedback in activity logs. An important activity for Data and Technical Owners as well as the overall System Administrators will be establishing and monitoring success metrics for the Marketplace. Specific metrics may include: Fulfillment Time, Fulfillment Accuracy, Quality and Ratings, and overall stakeholder involvement.
These may vary across an organization’s implementation needs and should be combined with other general Data Governance metrics to provide insight into the health of an organizations data governance program from creation through to consumption.
RESOURCES
Data Governance & Privacy
PLAN
IMPLEMENT
MONITOR
OPTIMIZE