The ai capability

Your code asks a model through ai, never through a provider's own operations. So moving to another provider changes app.jadpo and nothing else.

ai is a capability: a job Jadpo does itself and covers whole, with a provider package filling in the part that talks to the provider (Capabilities). Jadpo renders the prompt from a checked file, asks the provider, enforces the budget, decodes the answer into a type and records the call. The provider only turns text into text or numbers.

Operations

• Options. Each operation takes the named options the table lists: model:, max_output:, timeout: and attempts:. model: is a name from ai: { models: }. Leave it out and generation uses the model called default, and embeddings the one called embedding. There are no other options: temperature and other sampling settings stay at the provider's defaults.
• Only actions call them, including a subscriber's and a schedule's, as with every outside call (Calling). A function or a query can't. A route's deadline must leave room for the call's timeout: times its attempts. ai.generate's defaults come to 50 seconds, so a route that asks a model sets its deadline: to up to 60s. Longer work runs in a subscriber, which the route either waits for with settle: (Waiting for follow-on work) or streams.
• Generating changes nothing at the provider, so it counts as a call that changes nothing. It is retried on a timeout, can go anywhere in an action, and holds no transaction open (Outside calls hold no transaction). When an action runs again after a concurrent change, it gets the answers from its first run, kept in the same step records that keep a provider's answers (Several changing calls in one action). So a rerun neither spends twice nor answers differently.
• A problem you don't map is a fault, a failure the runtime raises itself (Failures): misconfigured for a rejected key or an unknown model, and unavailable for rate limits and server errors once the connection's retries are spent.

These are the ways a call can fail. Your action maps each one a call can raise to a failure of its own. ai.generate can raise all five; the table above lists which the others raise.

Connecting a provider

You connect a provider in app.jadpo: a connection to its package, and an ai: block naming that connection, the models your app uses, its limits and the personal data it may send. Here an app connects to OpenAI, with each model, the key and the daily spend as settings in config.jadpo:

// config.jadpoconfig {    ai_daily_spend: Money {        default: "GBP 20.00"    }    drafting_model: ModelId {        default: "openai/gpt-6-luna"    }    openai_api_key: ApiKey {        secret: true    }    openai_host: Url {        default: "https://api.openai.com/v1"    }    search_model: ModelId {        default: "openai/text-embedding-3-small"    }}// app.jadpoapplication {    connections: {        openai: {            package: OpenAi.v1            domains: [support]            credential: config.openai_api_key            host: config.openai_host        }    }    ai: {        connection: openai        models: {            default: config.drafting_model            embedding: config.search_model        }        embedding_dimensions: 1536        limits: {            per_request: 20000            per_day: {                tokens: 2000000                cost: config.ai_daily_spend            }            per_caller_per_day: 60000        }        personal_data: {            allowed: [PersonalData.identifying, PersonalData.health]        }    }}

• Models are settings. Each entry under models: is a ModelId setting (Settings), so moving to a newer model is a configuration change, reviewed like any other. A ModelId written anywhere but a setting's default: is semantic.ai_model_literal. default is required, and so is embedding when your app makes embeddings. Any other names, such as drafting or cheap, are your app's own.
• embedding_dimensions: is how many numbers each embedding has. Jadpo asks the provider for that many, where it can shorten its vectors. Every vector type you pass as ai.embed's as: must have the same dimensions: (semantic.ai_dimensions_mismatch), so a model change that alters the length is caught before a column holds vectors of two lengths.
• The credential is an ordinary secret: true setting. Code never reads it (Secrets).
• One ai connection per app. All of an app's models, such as a cheap one and a strong one, are reached through the same provider.

How a provider fills it

You only need this if you write a provider package. A package fills ai with a file, provides/ai.jadpo, that maps each of ai's operations onto the provider's own operations. The mapping is data, not code, as a package's mapping for email is (Capabilities).

Jadpo hands the mapping a request it has already rendered: request.instructions, request.messages (each a ChatMessage), request.model, request.max_output and, for generate, request.schema, the JSON Schema of T. The field names below follow OpenAI's Chat Completions API. Each package's names are checked against its provider's documentation.

// packages/openai/v1/provides/ai.jadpoprovide ai {    generate: {        call: chat.completions.create        input: {            model: request.model            messages: {                each: request.messages                as: {                    role: item.role                    content: item.content                }            }            max_completion_tokens: request.max_output        }        answer: response.choices.first.message.content        refused_when: response.choices.first.message.refusal        cut_off_when: {            path: "choices.0.finish_reason"            equals: "length"        }        usage: {            input: response.usage.prompt_tokens            output: response.usage.completion_tokens            cached: response.usage.prompt_tokens_details.cached_tokens        }        schema: {            mode: AiSchema.strict_subset            at: "response_format"        }    }    stream: {        call: chat.completions.stream        pieces: event.choices.first.delta.content        usage: event.usage    }    embed: {        call: embeddings.create        input: {            model: request.model            input: request.texts            dimensions: settings.embedding_dimensions        }        vectors: response.data        vector: item.embedding        batch: 100        usage: {            input: response.usage.prompt_tokens            output: 0        }    }    models: {        context: models.context_windows        prices: models.prices    }}

The example is a sketch. OpenAI's package as shipped names its operations chat_completions.create, chat_completions.stream and embeddings.create, and marks a refusal and a stream as Streams, embeddings and the OpenAI and Anthropic packages describes.

• schema: says how much of a JSON Schema the provider's structured mode enforces, and where it goes. Its mode: is one of:

  • AiSchema.strict_subset: the provider enforces part of a schema. Jadpo sends that part and checks the rest by decoding.
  • AiSchema.json_only: the provider promises JSON, but not its shape.
  • AiSchema.none: the provider promises nothing.

  at: is where in the request the provider takes the schema. Jadpo writes there the wrapper that mode needs, such as OpenAI's {"type": "json_schema", ...}. See Typed output.

• models: points at two tables in the package's libraries/: each model's context window, and its dated price per million input, cached and output tokens. The context window decides too_long, and the prices feed the cost estimate. Your app can override a price with a setting.

• The built-in driver, Driver.openai_compatible { url: }, fills ai for any server that speaks OpenAI's API, such as Ollama, llama.cpp's server, vLLM or LM Studio. It is the local driver every capability has, so in development you can generate for real at no cost. It accepts http:// only for an address on this machine.

• Conformance. jadpo package conform <package> ai runs the capability's tests against the provider: a typed answer, a refusal, a cut-off, a stream, an embedding of the declared length and the usage reported. The registry lists a package as filling a capability only with a passing run. By default the suite runs against the package's recordings and a local fake. With --live it runs against the provider's API through your app's connection, and is refused if no credential is set for it (The conformance suite).

A model on this machine

The quickest way to generate for real is a model running on your own machine, with no key and no cost. The built-in driver speaks OpenAI's Chat Completions API to a server that runs a model, here vLLM. Its url: is the address the API is under, as a Url setting or written out. Plain HTTP reaches only this machine:

application {    connections: {        local_model: {            driver: Driver.openai_compatible            url: config.model_url        }    }    ai: {        connection: local_model        models: {            default: config.drafting_model        }        limits: {            per_day: {                tokens: 2000000            }        }    }}

config {    drafting_model: ModelId {        default: "meta-llama/Llama-3.1-8B-Instruct"    }    model_url: Url {        default: "http://localhost:8000/v1"    }}

An action asks the model with a prompt, and names with as: the type the answer is decoded into. It maps each of the five failures to one of its own, each of kind unavailable. The failure that malformed maps to receives the last DecodeProblem, which says what didn't decode:

use support/types { CustomerQuestion, SupportAnswer }use support/failures { AnswerUnreadable, AssistantUnavailable }use support/prompts { AnswerQuestion }/// Answers a customer's question, in English for now.action answer_customer(    customer_question: CustomerQuestion) fails AnswerUnreadable, AssistantUnavailable -> SupportAnswer {    return attempt ai.generate(        AnswerQuestion {            question: customer_question            reader_locale: Locale.en_gb        },        as: SupportAnswer,        max_output: 600    ) {        refused: AssistantUnavailable        malformed: AnswerUnreadable        timed_out: AssistantUnavailable        over_budget: AssistantUnavailable        too_long: AssistantUnavailable    }}

This is what the server is sent for "Does your sourdough contain sesame?". The schema is cut to what the server's structured mode enforces, as Typed output describes:

POST http://localhost:8000/v1/chat/completions{"model": "meta-llama/Llama-3.1-8B-Instruct", "messages": [{"role": "system", "content": "You answer questions from customers of an online bakery. ... Write in\nthe language of en-GB. ..."},              {"role": "user", "content": "<question>\nDoes your sourdough contain sesame?\n</question>"}], "max_tokens": 600, "response_format": {"type": "json_schema",                     "json_schema": {"name": "answer_question", "strict": true, "schema": {...}}}}

In jadpo dev each call goes to the server, and jadpo calls shows each attempt with what was sent and what came back. In a test, the ai fake answers instead (Testing AI calls).