Ontology - Content Repositories
Intended audience: ANALYSTS DEVELOPERS ADMINISTRATORS
AO Platform: 4.3
Overview
Easy Answers supports a special type of Search using the Chatbot to find information across a repository of unstructured documents. Documents are organized in one or more Content Repositories, such as Contracts, HR Policies, or Service Manuals. The Content Repositories are configured in the Ontology Composer; however, users with adequate permissions can also upload documents into a Content Repository when selected directly from the Search option in Easy Answers.
Asking a question in plain English in Easy Answers will attempt to find the specific content across all documents in the selected Content Repository and create a summary of the content, which can be either downloaded or copied for use in other systems.
Creating a Repository
The Content Repositories page is available from the left-side navigational panel. The user will have the option to click on + Add Content Repository to create a Content Repository. A pop-up will appear for creating a Content Repository with various properties.
Properties
Label | Defaults | Description |
|---|---|---|
Name | The Name field is the internal name of the Ontology. | |
Description | The Display Name is the name that will appear in the UI at runtime. | |
LLM Model Configuration | Use the dropdown to select from available Large Language Model (LLM) configurations.
| |
Embedding Model Configuration | An LLM Embedding Model transforms words into numerical vectors, capturing the semantic meaning and relationships, enabling LLMs to understand and process information more effectively. Use the dropdown to select an Embedding Model from available configurations. This selection will be used as the default for interpreting questions in Easy Answers and Chatbot if LLM is enabled on the Text2SQL page.
| |
Cache Response | Enabled | If enabled, performance will be improved by reducing the load on the data source as data will be loaded from the cache. |
Open AI Instructions | ||
Repository Data Source | Repeater section - Use the dropdown to select below Repository Data Source:
| |
Source Config | Use the dropdown to select Repository Data Source (configured in Admin solution) which includes pre-configured repositories/folders. | |
Save a copy of repository files | Unchecked |
Editing a Repository
Upon creation of a Content Repository, it will be included in the left-side panel of the Content Repository page. Multiple repositories can be added as needed.
Select a Content Repository to access its properties divided over two tabs:
General
Files
Delete - On hover, use the trashcan icon to delete the Content Repository.
General Tab
Content Repository for General Tab Properties
Label | Defaults | Description | |
|---|---|---|---|
GENERAL | |||
| Text Field | The Name field is the internal name of the Ontology. | |
| Multi-Line Text Field | The Display Name is the name that will appear in the UI at runtime. | |
| Dropdown | Use Default | Use the dropdown to select from available Large Language Model (LLM) configurations.
|
| Dropdown | Use Default | An LLM Embedding Model transforms words into numerical vectors, capturing the semantic meaning and relationships, enabling LLMs to understand and process information more effectively. Use the dropdown to select an Embedding Model from available configurations. This selection will be used as the default for interpreting questions in Easy Answers and Chatbot if LLM is enabled on the Text2SQL page.
|
| Checkbox | Enabled | If enabled, performance will be improved by reducing the load on the data source as data will be loaded from the cache. |
| Multi-Line Text Field | Includes the default prompt instructions for Entity Extraction when the documents in the Content Repository are indexed. | |
| Multi-Line Text Field | Includes the default prompt instructions when using the Chatbot in Easy Answers to find answers to the questions asked. | |
| Tags | One or more OpenAI-specific tools can be selected to instruct the OpenAI LLM how and where to search for answers to questions in Easy Answers Chatbot. Tool options include: file_search, code_interpreter, bing_search, and google_search. | |
| Number Field | 1 | The number instructs the OpenAI LLM on how long to keep the documents that are being uploaded to the LLM. |
| Number Field | 60 | The number instructs the OpenAI LLM on how long the Entity Extraction may take before timing out. |
DATA SOURCE | In this section, you can identify the configured Storage locations for the chosen Content Repository. This is a repeating section that enables multiple locations to store documents for the selected Content Repository. | ||
| Dropdown | Use the dropdown to select below Repository Data Source, including: SharePoint. | |
| Dropdown | Use the dropdown to select Repository Data Source (configured in Admin solution) which includes pre-configured repositories/folders. | |
| Checkbox | Unchecked | If the checkbox is enabled, the AO platform server will automatically store copies of any documents uploaded to cloud storage. The checkbox is only shown when a Repository Data Source has been configured. |
TABLE SETTINGS | |||
| Dropdown | None | The Table Detection Strategy dropdown pertains to managing structured data in a semantic or metadata-driven environment, where strategies focus on recognizing, parsing, and extracting tabular structures from diverse data sources by leveraging their semantic and structural representations. Options include: None, LLM Vision, and Heuristic. |
| Multi-Line Text Field | The Table Regex refers as a pattern-matching approach to identify, parse, and extract tabular data from structured or semi-structured data sources. Regular expressions can help locate table-like structures within text, documents, or metadata by identifying patterns that signify rows, columns, and other table elements. | |
| ON/OFF Toggle | OFF | The ON/OFF toggle Extracting Lineless Tables in an MSO environment involves identifying and parsing data that is presented in a tabular format without relying on explicit visual cues. |
| ON/OFF Toggle | OFF | The ON/OFF toggle refers to tables where color is used to differentiate or highlight certain elements such as rows, columns, cells, or headers. This could be for visual purposes, like distinguishing categories, data significance, or status indicators. |
INDEX SETTINGS | |||
| Checkbox | Unchecked | Use the checkbox to enable the Search Store. |
| Checkbox | Checked | Use the checkbox to enable the Vector Store - used to suggest alternative questions in Easy Answers. |
| Checkbox | Unchecked | Use the checkbox to allow OpenAI to search files/documents being uploaded as part of the indexing process. Note: If this option is enabled (checked), then it’s very important to select a specific LLM Model Configuration. In the GENERAL section, select any of the OpenAI “Omni” models, - OpenAI ChatGPT 4.0 Omni-2024-08-08 is recommended. |
ENTITY EXTRACTION | |||
| Checkbox | Unchecked | Toggle to enable/disable Entity Extraction. |
| Multi-Line Text Field | The Persona refers to identifying, extracting, and organizing entities that define personas. | |
| Dropdown | Vector Store | Dropdown for selecting either Vector Store or Pipeline. This property is only available if Enable Entity Extraction is enabled. |
| Text Field w/Search | The Strategy Rule MSO can be used to define the rules for which the strategies will be executed. | |
| Text Field w/Search | Repeater section for selecting one or more MSOs from the Search dialog for use by Entity Extraction. This property is only available if Enable Entity Extraction is enabled. | |
| Text Field w/Query Builder | ||
| Text Field w/Search | The Main MSO is the primary MSO that has a relationship with other MSOs. | |
| Dropdown | The Key Properties serve as relationships between other MSOs. | |
| Text Field | The Source Key Fields refer to the custom fields used to create specific implementations. Currently, these are used to support page numbers and clause IDs to be fetched from vectorDB in contract AI. | |
| Dropdown | The Named Entities like ORGANIZATION, ADDRESS, and PERSON can be identified in the chunks. | |
DEFAULT QUESTION | |||
| Text Field | Repeater section for one or more default questions to be added. These questions will be shown by default in the Chatbot when a Content Repository is selected in Easy Answers. | |
SECURITY | These security measures go beyond the general permissions granted for access to the selected Ontology. A Role can be given permissions to an Ontology but may have access permissions revoked for one or more Content Repositories. | ||
| Dropdown | The user is allowed to select the roles from the list of roles configured on the system. | |
| Dropdown | Use the dropdown to select from available options: Allow or Deny. | |
ADDITIONAL PROPERTIES | |||
| Icon | Select an icon to be used when the Easy Answers Chatbot Welcome Message is shown. | |
| Text Field | Provide a short Welcome Message to the user when opening the Content Repository Search in Easy Answers | |
DOCUMENT HIERARCHY | |||
| ON/OFF Toggle | OFF | Enable the toggle to configure how the documents in the Content Repository shall be organized. |
| Text Fields | ||
| Multi-Line Text Field | ||
| Dropdown | ||
| ON/OFF Toggle | OFF |
Files Tab
Content Repository for Files Tab Properties
Label | Defaults | Description |
|---|---|---|
Files | Displays a File Manager with the following columns. | |
| An icon representing the document type of the document. | |
| Document Name representing the document type and the Filename of the document. | |
| File size of the uploaded document. | |
| Timestamp for when the document was uploaded. | |
| Name of the user who uploaded the document. | |
| Options include: Indexed, Pending Upload, Uploaded |
User Actions
In Header
Filter - click to filter based on an Ontology.
Upload - opens the File Manager dialog for selection of documents to upload. The following document formats are supported for upload to a Content Repository:
PDF (txt and scanned)
Word (.doc, .dot, and .docx)
PowerPoint (.pptx)
Navigate folders - accessing and exploring the files and subfolders within a specific folder to locate or manage data.
Show view as List or Grid - Click to view the files in list or grid format.
Search - enter a Search term to find relevant objects on the selected page.
Indexing Task - Task with optional schedule - enabled for documents added to the Indexing Queue.
Re-index - only enabled when documents are selected.
View - only enabled when documents are selected.
Filter - ability to filter documents based on Status: All, Pending Upload, Upload Failed, Uploaded, In Queue, Indexed, Indexing in Process, Indexing Failed, and Indexing Cancelled.
Delete - only enabled when documents are selected.
Refresh - refresh the page.
On Documents
Checkbox - to select one or more documents.
Icon - Add to Indexing Queue
Options menu:
Index Document - only available if not indexed.
Add to Queue - only available if not indexed.
View File - to view the document.
Delete File - delete the document.
Assign Permissions - opens a dialog to set Role-level permissions. Select one or more Roles and apply either Allow or Deny level permissions.