Skip to main content

AI Models

Configure the AI models used for chat, embeddings, and other features in ZenSearch.

Overview​

The AI Models settings allow you to:

  • View available models
  • Add new model configurations
  • Set default models
  • Monitor model usage

Accessing Model Settings​

  1. Click Settings in the sidebar
  2. Select the AI Models tab

Available Models​

Model Types​

TypePurpose
ChatConversational AI responses
EmbeddingDocument vectorization
RerankerResult reranking

Supported Providers​

ProviderModels
OpenAIGPT-4o, GPT-4 Turbo, GPT-3.5 Turbo
AnthropicClaude 3.5 Sonnet, Claude 3 Opus
CohereCommand, Command-R
CustomOpenAI-compatible endpoints

Adding Models​

Add a New Model​

  1. Click Add Model
  2. Select the provider
  3. Choose the model
  4. Enter API key (if required)
  5. Click Add

Configuration Fields​

FieldDescription
ProviderModel provider (OpenAI, Anthropic, etc.)
ModelSpecific model name
API KeyProvider API key
EndpointCustom endpoint URL (if applicable)

Default Models​

Setting Defaults​

Set default models for each use case:

  1. Find the model in the list
  2. Click Set as Default
  3. Select the use case (Chat, Embedding)

Default Assignment​

Use CaseRecommendation
ChatGPT-4o or Claude 3.5 Sonnet
Embeddingtext-embedding-3-small
RerankerCohere rerank-v3

Model Usage​

Viewing Usage​

Navigate to the Model Usage tab to see:

  • Tokens consumed per model
  • Cost breakdown
  • Usage over time
  • Per-team breakdown

Usage Metrics​

MetricDescription
Input TokensTokens sent to model
Output TokensTokens received from model
Total CostEstimated cost
Request CountNumber of API calls

Testing Models​

Test Connection​

Before saving, test the model:

  1. Click Test Connection
  2. Wait for verification
  3. Check for errors

Test Results​

ResultMeaning
SuccessModel is accessible
Auth ErrorAPI key is invalid
Network ErrorCannot reach endpoint
Model ErrorModel not available

Custom Endpoints​

OpenAI-Compatible APIs​

For local or self-hosted models:

Provider: Custom
Endpoint: http://localhost:8000/v1
Model: local-llama
API Key: (optional)

Supported Endpoints​

  • Ollama
  • LM Studio
  • vLLM
  • Text Generation Inference

Best Practices​

Model Selection​

  1. Use GPT-4o or Claude for complex queries
  2. Use faster models for simple tasks
  3. Consider cost vs. quality tradeoffs
  4. Test models before production use

API Key Security​

  1. Never share API keys
  2. Rotate keys periodically
  3. Use separate keys per environment
  4. Monitor for unauthorized usage

Troubleshooting​

Model Not Responding​

  1. Verify API key is valid
  2. Check provider status page
  3. Test connection in settings
  4. Review rate limits

High Costs​

  1. Review model usage dashboard
  2. Consider using smaller models
  3. Optimize query complexity
  4. Set usage limits

Next Steps​