Manage metadata

hila metadata is the object that defines the elements required to run the hila system. These elements include database connections, LLM and embedding models, and database table schemas, among others.

The metadata appears in the hila UI as a datasource.

You can edit metadata, clone existing metadata, or create new metadata.

Usecase

See Ingest data from CSV files for an example of how to create metadata and ingest data from a CSV file into a table in the metadata.

Accessing the metadata

1. Open the hila monitoring application by changing the ui subdomain in your hila URL to monitoring. For example, if your hila URL is https://ui.example.site, change it to https://monitoring.example.site.

2. Log in to the hila monitoring application using your hila credentials.

3. In the left navigation pane:

   1. Choose All Projects from the Select Projects Type dropdown list.
   2. Choose LLM Project from the projects list.
   3. Choose the metadata you want to manage from the Select Metadata dropdown list.

   

4. Click Edit Metadata to edit the existing data source, click Clone Existing to make a copy of the existing metadata, or click New Metadata to create new metadata from scratch.

Edit metadata

Verify and Export are top-level controls for the Edit Metadata page, while Import is grayed out.

• Verify — opens the Metadata Verification dialog box from which you can choose a question to ask hila to verify that the metadata is a valid data source.
• Import — (not available for edit) opens a file browser to select a JSON file with the metadata to import. Only available when creating a new metadata.
• Export — opens a file browser to select a location to save the metadata as a JSON file.

The tab controls are Overview, Add Table, and then a tab for each table defined in the metadata. The Overview tab is the default tab when you open the Edit Metadata page.

• Overview — shows all the metadata parameters except for the table schemas.
  • Connection — the database connection name. Not available for editing a metadata or a clone. Only available when creating a new metadata.
  • Name — the name of the metadata. This text appears in the hila UI as the name of the datasource.
  • Description — a description of the metadata. This text appears in the hila UI when hovering over the name of the datasource.
  • LLM Models — shows the LLM models defined in the metadata. You can add models by choosing from the Add Model dropdown list and remove models by clicking Remove.
  • Embedding Models — shows the embedding models defined in the metadata. You can add models by choosing from the Add Model dropdown list and remove models by clicking Remove. The Preferred checkbox controls which embedding model hila uses by default when there are multiple embedding models defined in the metadata.
  • Preferred LLM Model — expand the list to set a preferred LLM model for each feature that uses a model.
  • Entity Type — shows whether the database contains single or multiple entities. This information is implied from the table schemas. It is not editable.
  • Default Currency — the default currency for the metadata.
  • Status — shows the status of the metadata. Currently, the only status is Active. This information is not editable.
  • Version — the version of hila Conversational Analytics schemas associated with this metadata. This information is not editable.
  • Views JSON — opens the Views JSON dialog box where you can create different views for your data.
  • Configuration —
• Add Table — Creates a new table tab.
• Table tab — shows the table schema for the table. The table tab is named after the table name. The table tab contains the following controls:
  • Table Name — the name of the table in the database.
  • Upload File — opens a file browser to select a CSV or PARQUET file of that table type to upload to the table.
  • Overwrite Method — the method to use to overwrite the table data. The options are:
    • replace — replace the existing data in the table with the new data.
    • append — append the new data to the existing data in the table.
    • upsert — update the existing data in the table with the new data, and insert any new data that does not already exist in the table.
  • Engine — the database engine to use for the table. Fill in the columns indicated by <specify_columns_selected_separated_by_commas> to connect to the database.
  • Add Transformers — select a transformer to the pipeline to apply to the table. You can edit the configuration of any transformer in the pipeline by clicking the edit button next to that transformer.
  • Columns — shows the columns in the table and provides controls for you to change the name, type, set as primary, remove, give a default value, give an initial value, give any aliases, set foreign keys, add and remove columns, and download the schema for local storage.
  • Preview Table Data — shows a preview of data in the table.
  • Code Generation — shows the code in Python or JavaScript that you need to run if you want to load the table for that database manually.
  • Remove Table — removes the table from the metadata.
  • Save Schema — saves the schema for the table. This button is grayed out until you make a change to the schema.
  • Save data into table — saves the data in the table. This button is grayed out until you make a change to the data.

Clone existing metadata

When you select Clone Existing, the Clone Metadata page opens.

1. In the Project dropdown list, set the project you want the new metadata to be associated with.
2. Give a name to the new metadata. This name appears in the hila UI as the name of the datasource.
3. Click Submit Clone to create the new metadata. After the clone is created, the Edit Metadata page opens with the new metadata appears in the hila UI under Datasources.

New Metadata

The items for new metadata are a subset of the editing metadata. However, the Connection field is available for editing. The Import button is available to import a JSON file with the metadata to import.

To create a new connection:

1. Click + Create new database and enter a database name in the Create New Database dialog box.

2. Click + Add Connection to open the Add New Data Source Connection window.

   1. Enter a unique name for the connection.
   2. Enter a description for the connection.
   3. Enter the connection URL.
   4. Choose the type of connection from the dropdown list.
   5. Click Test Connection to test the connection.
   6. Click Save.