Large Language Models (LLM Parameters)

AI-Horizon RAG Powered Agents (Chat Agents and Search Agents) integrate with various Large Language Model providers. By passing configuration details inside the llm_params dictionary during agent initialization, developers can customize model selection, endpoints, api versions, and generation limits.

1. Supported Providers

OpenAI: Default provider. Supports standard models like gpt-4o, gpt-4-turbo, and gpt-3.5-turbo.
Azure OpenAI: Recommended for enterprise deployments requiring dedicated cloud safety, compliance boundaries, and low latency.
Local/Self-Hosted LLMs: Integrates with local inference servers (e.g. Ollama, vLLM, or custom endpoints) using OpenAI-compatible APIs.

2. Configuration Parameters

The llm_params object accepts the following fields:

Parameter	Type	Default	Description
`provider`	`string`	`"openai"`	LLM provider: `"openai"`, `"azure"`, or `"custom"`.
`params`	`object`	`{}`	Key-value settings mapping provider authentication and endpoints.
`params.api_key`	`string`	`None`	Authentication API key.
`params.api_version`	`string`	`None`	API version string (required for Azure OpenAI, e.g., `"2024-02-15-preview"`).
`params.azure_endpoint`	`string`	`None`	Endpoint address (required for Azure, e.g., `"https://your-resource.openai.azure.com/"`).
`params.azure_deployment`	`string`	`None`	Name of your Azure deployment model (e.g., `"gpt-4o-glo-std"`).
`params.max_tokens`	`number`	`None`	Maximum limit of generated completion tokens.
`params.temperature`	`number`	`0.0`	Controls language generation randomness.

3. Configuration Examples

Example A: Azure OpenAI Integration

from aih import ChatBot
 
azure_llm_config = {
    "provider": "azure",
    "params": {
        "api_key": "your_azure_api_key",
        "api_version": "2024-02-15-preview",
        "azure_endpoint": "https://nexa-enterprise.openai.azure.com/",
        "azure_deployment": "gpt-4o-glo-std",
        "engine": "gpt-4o-glo-std",
        "temperature": 0.2
    }
}
 
chatbot = ChatBot.pdf_chat(
    input_files=["./reports.pdf"],
    llm_params=azure_llm_config
)

Example B: OpenAI Custom Model Selection

from aih import ChatBot
 
openai_llm_config = {
    "provider": "openai",
    "params": {
        "api_key": "your_openai_key",
        "model": "gpt-4-turbo",
        "temperature": 0.0,
        "max_tokens": 500
    }
}
 
chatbot = ChatBot.docx_chat(
    input_files=["./policy.docx"],
    llm_params=openai_llm_config
)

LanceDB Introduction