Generating, its fake and the local driver

The full rules for ai:, ai.generate and the built-in Driver.openai_compatible, and for how the fake stands in for them in tests. The common case is under The ai capability and Typed output.

• ai: takes connection:, models:, embedding_dimensions:, limits:, personal_data: and prices:. Any other key, a connection that answers no model, a model that isn't a ModelId setting, and models: without default are ai.settings_invalid. An action that asks a model when app.jadpo has no ai: is ai.connection_required, as sending email with no email connection is email.connection_required. A package's connection fills ai too (A package filling ai). A ModelId setting is read from its text, so a setting may hold one (What a setting can hold).

• Driver.openai_compatible takes url: alone, as a Url setting or an address written out, with plain HTTP only to this machine (connection.key_missing, connection.key_invalid, connection.value_invalid). It sends no credential. Each call is OpenAI's Chat Completions request to <url>/chat/completions, with:

  • the model's ModelId as written, such as meta-llama/Llama-3.1-8B-Instruct, since vLLM and LM Studio name a model by its repository;
  • the messages prompts.render gives;
  • max_output: as max_tokens, which every server named on this page reads;
  • its structured mode, response_format: { type: "json_schema", json_schema: { name: <prompt>, strict: true, schema } }, so its mode is strict_subset.

  For a package whose mode is json_only or none, the schema goes in the instructions instead, under # Answer format, asking for one JSON document alone.

• Reading an answer. The text is choices.0.message.content. A refusal or a content_filter stop is refused, and a length stop is over_budget. The built-in driver declares no context window to check before sending, so too_long is a 400 whose error.code is context_length_exceeded, as OpenAI answers one. Any other 4xx, a redirect, a 401 or a 403 is misconfigured. A server that can't be reached, a 429 (after its Retry-After) or a 5xx is sent again, at most 3 times in all within timeout:, then is unavailable. No whole answer within timeout: is timed_out at once, since that attempt had all the time left. An answer that isn't a chat completion is a fault naming the connection and the prompt, never its text. A JSON answer in a Markdown code fence is read without the fence.

• The schema is OpenAPI's whole schema for T (OpenAPI): its objects and enums as $defs, and each field's /// description on its property. The strict mode is sent it cut to types, properties, required fields, enums, const, anyOf (for oneOf) and nesting. Every property is required, one that may be left out becomes one that may be null, and additionalProperties is false. A T whose JSON isn't an object (an enum, a list, a number) is asked for as the member value of an object, since the mode wants an object at its root, and read from it. A T built on Text asks for plain text, held to T's rules as it comes.

• A repair adds the answer as the model's turn and a fixed correction as the user's: "In your answer, <path> breaks the rule <rule>. Answer again with one JSON document alone that matches the schema." For text that isn't JSON, the correction is "Your answer wasn't one JSON document.", and its DecodeProblem is its root's, malformed.

• The options are written out:

  • model: is a bare name ai: { models: } gives (semantic.unknown_name otherwise);
  • max_output: is 1 to 1,000,000, timeout: is 1s to 10m, and attempts: is 1 to 3 (type.invalid_literal otherwise);
  • there are no others (type.named_argument_unknown).

• Mapping the failures. Each of the five conditions needs an arm naming a failure of kind unavailable, as a store's storage_full does (failure.rule_unmapped, failure.mapping_kind), and invalid: covers none of them. The failure malformed: names receives the last DecodeProblem in a problem: DecodeProblem field, as json.read's mismatch does.

• The route's deadline must leave room for the call. The call's timeout: times its attempts:, added to the route's other outside calls on the longest way through its action, must fit the route's deadline:. Otherwise it is route.calls_past_deadline (Calling).

• A ModelId written in code, in a function, an action, a constant or a test's statements, is semantic.ai_model_literal. A setting's default: and test:, and a test's config:, give a setting its value, so they may write one. A test that reaches a call reads its model's setting, so a setting with neither test: nor default: is configuration.test_value_missing there.

• as: refuses a reference, a File, Bytes and PresentationText anywhere in T, naming the field (semantic.ai_output_type). It refuses a secret value (configuration.secret_flow) and anything else json.read can't read (type.mismatch).

• Answers reused on a rerun. Each call takes the invocation's next outside-call step. Its answer, or the condition it ended in, is kept with a digest of the request it sent. When an action runs again after a concurrent change (All-or-nothing actions and concurrent changes), it takes the same steps, and is given the answer for the same request without asking again, logged as replayed. A request that changed is asked anew. After an action's first changing call, a call is a step of its record, its answer sealed as a provider's is (Several changing calls in one action), so a redelivery replays it too. Before the first, nothing is recorded, as for a provider's read.

• The fake. Every ai call in jadpo test and jadpo call answers through one fake in the ai: connection's place, whatever its driver. It answers in the built-in driver's own form, so the real reading, decoding and repair run. Each scripted answer is one attempt, and is one of:

  • a value of the call's as: type (a prompt asked for as two types takes only raw(...) and failures);
  • raw("...") text;
  • a condition. malformed is an answer with no text, which no type decodes.

  An attempt past the end of the script gets a generated value of T and fails the test as test.fake_exhausted. Answers left over fail it as test.fake_unused. Each names ai.<prompt>. ai.<prompt>.requests lists each call's prompt value once, however many attempts it took, and only a test may read it (semantic.not_callable). recorded(...) is under Embeddings and recordings.

• jadpo dev sends every call of the model connection to its server. The call log keeps each attempt with its request and answer in full (source live); production keeps neither. How a package's ai is faked in jadpo dev, development: { sandbox: [<prompt>] } and jadpo calls script for a prompt are under A package filling ai.

• The network report lists the model connection, for ai, by the domains whose actions ask it (The network report).

keep_content: on ai: isn't built yet: it is toolchain.not_built_yet.