MSO - Sourcing

Intended audience: ANALYSTS DEVELOPERS ADMINISTRATORS

AO Platform: 4.3

Overview

This section contains the configuration of the MSOs data source relative to availability of data in a tiered storage infrastructure. From the 4.1 release of the AO Platform, the Ontology now “owns” the data source(s) compared to previous releases where data sources were associated with the individual MSOs in an Ontology. This ultimately allows a more flexible way to switch data sources when multiple data sources are created and used to develop a solution and for a production environment.

Each MSO uses a data source system created at the Ontology level, hence a number of the properties at the MSO level are rendered read-only. Any new MSO created will be created based on the default data source selected from the data source systems at the Ontology level.

Tiers

Hot - data is accessed frequently - often based on an in-memory data source
Warm - data is accessed less frequently - typically retrieved from mid-tier database storage
Cold - data is only accessed sporadically - retrieval may be delayed as data may be in archived storage or cloud-based data lake

User Actions

Supports Federated - enable the toggle if the data source has been added to the Federated Data page in the Ontology.
Primary Data Source - select from the dropdown the primary data source to be used by the MSOs created for the Ontology.
+ (Add) - select to add a Data Source system created at the Ontology level.
Expand – use this option to expand/collapse the properties section when the property details are shown on the right-side of screen

Properties

The Property Details panel is used to configure the details for each property of the MSOs data source relative to availability of data in a tiered storage infrastructure.

Label	Description
Name	The Name field is the display name of the data source - read-only.
Description	The Description field displays a brief description of the data source - read-only.
Tier	The Tier dropdown field displays the list of storage classification options - read-only: Hot - data is accessed frequently - often based on an in-memory data source. Warm - data is accessed less frequently - typically retrieved from mid-tier database storage. Cold - data is only accessed sporadically - retrieval may be delayed as data may be in archived storage or cloud-based data lake.
Type	The Type dropdown field displays the list of data source types from which the data is retrieved, ie Database, Generic, Pipeline - read-only.
Sink/Source	The Sink/Source dropdown allows the user to select whether to use data source as either a data Source, Sink, or both (Source and Sink) - on read-only.
If Type = Database
Data Source Name	The Data Source Name dropdown allows the user to select the database name from where the data is retrieved - read-only.
Table Radio Button	When selected, enables the Table property.
Table	Click the Select Table button to select the Table name from available Tables in the Metadata Cache. See Adding Schemas and Tables to Metastore Cache for details.
SQL Radio Button	When selected, enables a SQL Query field.
Statement	Enter SQL Query to retrieve data from the selected database/table.
MAPPING
Update	Click the button to Update the mapping of fields between the data source and the MSO.
Refresh Table Columns	Click the button to Refresh Table Columns in case any fields in the data source tables have changed.
MSO Property	The MSO property section displays all the fields configured for the MSO.
DB Column	The DB Column section displays the list of Database table fields which maps to the MSO fields. Use the pencil icon after the DB Column field to enter an expression instead of a DB Column in case the MSO field should be populated based on an expression. Example of assigning Names to Age categories:
Additional Source/Sink Properties	A repeater section of Key / Value pairs to show in the table.
If Type = Generic
Target Type	Select between CSV, Excel, Shapefile, GeoJSON, XML, Large Excel, Solr, Google BigQuery or Neo4j for the Source/Sink.
SOURCE PROPERTIES
Key	A predefined list of keys that helps configure the Generic data source. Select and configure only the keys that are required to retrieve the data.
Value	The value content relating to the Key field selected.
MAPPING
MSO Property	The MSO property section displays all the fields which are configured for the MSO.
Column	The Column section displays the list of fields from Generic data source which maps to the MSO fields.
If Type = Pipeline
Source Pipeline	Click the Search icon to select from existing Pipelines to use as Source/Sink.

Contact App Orchid | Disclaimer