Configure Conversational Finance
Configuring Conversational Finance involves setting up the models you want to use with Conversational Finance, configuring data ingestion, and then running the CF notebook to install Conversational Finance on hila.
Configure LLMs
Before you can use an external LLM with hila, you must configure hila to use the LLM; specifically, you must configure the LLMs with the API token from your LLM account.
Prerequisites
- You must have an API token from your LLM account so you can add it to the hila LLMs.
Procedure
- Go to the hila monitoring app by changing the subdomain of your hila instance to monitoring. For example, if your instance URL is qnaui.customername.site, then change it to monitoring.customername.site.
-
In the projects pulldown, select LLM Project.
The LLM Project folder contains the LLMs available for you to use with hila. Select the LLM in the Select Model pulldown.
-
In the Select Model pulldown, select the model you want to use.
The Select Model list may contain unsupported models. The supported models for this release are:
- gpt-4o
- gpt-4o mini
- OpenAI text embedding 3 small
- The Azure versions of each of these above models (see Azure OpenAI for Conversational Finance) at the end of this section.
- Select Metadata in the left pane.
-
Find the row with the name predict_metadata, click the three dots at the far right of that row, and click Edit.
The predict_metadata object contains the parameters needed for hila to interface with an LLM. The main field you need to change is the api_token field. The fields under init_config_params are the parameters that get passed directly to the selected LLM.
-
In the code window, find the field api_token. Replace the value in the field with your API token.
For Azure OpenAI models, you can update the api_version and azure_endpoint fields if you want to use a different version or different model.
- Click Save tag. The LLM is now configured for use with hila.
You can find available models in the hila monitoring app, LLM Project pulldown, for you to try for better results. You can also integrate your own custom models; see Integrate your own LLMs.
Azure OpenAI for Conversational Finance
With Conversational Finance, you can use the following Azure OpenAI models:
- Azure OpenAI gpt-4o
- Azure OpenAI gpt-4o mini
- Azure OpenAI text embedding 3 small
To create the Azure OpenAI service, see Create and deploy an Azure OpenAI Service resource.
You must send your configuration information for Azure OpenAI models to Professional Services. Contact your PS representative for what information you need to provide.
The currently supported region for these models is eastus. For the most up-to-date information from Microsoft, see: