Local and self-hosted LLMs with OpenCode

This guide explains how to configure and use local or self-hosted Large Language Models (LLMs) within your development environment, leveraging an opencode.json file for unified configuration. This approach allows you to integrate LLMs, such as those served by Ollama, directly into your workflows without relying on external cloud services for inference, enhancing privacy and control.

Understanding opencode.json

The opencode.json file serves as a central point to define various LLM providers and their available models, including those running locally. Below is a typical example of an opencode.json configured for a local Ollama server:

{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "ollama-direct": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "Ollama Server",
      "options": {
        "baseURL": "http://ollama.example.internal:11434/v1",
        "apiKey": "ollama" 
      },
      "models": {
        "mistral:latest": {
          "name": "Mistral"
        }
      }
    }
  }
}

Key configuration details

• $schema — validates the structure of the opencode.json file
• provider — defines different LLM providers you wish to configure
  • ollama-direct — custom identifier for your Ollama provider
    • npm — client library to interact with this provider (@ai-sdk/openai-compatible for Ollama)
    • name — human-readable display name, e.g. “Ollama Server”
    • options.baseURL — network endpoint where your local Ollama server is accessible
    • options.apiKey — placeholder value ("ollama") since local instances don’t require a real key
    • models — specific models this provider makes available, keyed by their Ollama identifier

Setup

1. Ensure your local LLM server (e.g. Ollama) is running and reachable at the baseURL
2. Place opencode.json in your project root or tool config directory
3. Set baseURL and models to match your local setup
4. Any tool respecting the OpenCode.ai schema will now use your local LLMs