# Safety and privacy

## Personal data sent to a model

By default, your app may send a model no personal data at all. Each
kind it sends, such as health records or names, must be listed on the
connection, and the compiler follows every value to check. A field's
kind, its category, comes from its `personal:` mark
([Marking personal fields](marking-personal-fields.md))
or from a standard type that is always personal data, such as an
`NhsNumber`.

| Rule | What it means |
|---|---|
| Every category listed | Each `PersonalData` category that reaches a prompt must be in the connection's `personal_data: { allowed: }`; one that isn't is `semantic.ai_personal_data_not_allowed`. A sensitive category is flagged in the approval report |
| Followed to its fields | The compiler follows each placeholder back to the fields it was read or built from, so a name copied into a `Text` keeps its category |
| A conversation is health | `ChatMessage` is `health` wherever it is ([AI types](ai-types.md)), so a chat sends `health` and must list it |
| Per-field opt-out | `to_models: false` on a field keeps it from every prompt, whatever its category allows (`semantic.ai_field_refused`) |
| What decided it | A value an `if`, a `match` or a `list.filter` on a marked field decided carries that field; only the records a loop goes through are left out |
| Kept when stored | A field that stores a marked field's value has its marks (`semantic.ai_field_unmarked`) |
| An answer carries its prompt | A model's answer, and each field read off it, carries everything its prompt was built from, checked like those fields when stored or sent again |
| Reported | [The personal-data report](the-personal-data-report.md) lists each prompt, the fields and categories it sends and the connection |
| Erasure | A provider can't erase one person's prompts, so the `ai` connection needs an `ErasureNotPossible` [acceptance](acceptances.md) stating the provider's retention, as any connection that receives personal data and can't erase it does ([Erasure at providers](erasure.md#erasure-at-providers)) |

The full rules are under [Following personal data](following-personal-data.md).

A clinic whose handover summaries send each patient's notes to a model
lists `health`. The NHS number never leaves, whatever is listed, because
its field is marked `to_models: false`:

```jadpo
application {
    connections: {
        local_model: {
            driver: Driver.openai_compatible
            url: config.model_url
        }
    }
    ai: {
        connection: local_model
        models: {
            default: config.drafting_model
        }
        limits: {
            per_day: {
                tokens: 2000000
            }
        }
        personal_data: {
            allowed: [PersonalData.health]
        }
    }
}
```

```jadpo
use clinic/types { ClinicalNote }

/// A patient of the clinic.
entity Patient {
    id: Uuid
    patient_nhs_number: NhsNumber {
        personal: PersonalData.government_id
        to_models: false
    }
    notes: ClinicalNote {
        personal: PersonalData.health
    }
}
```

```jadpo
use clinic/entities { Patient, PatientNotFound }
use clinic/types { HandoverSummary }
use clinic/failures { SummaryUnavailable }
use clinic/prompts { SummariseNotes }

/// Summarises a patient's notes for the clinician taking over.
action hand_over(
    patient_id: Patient.id
) fails PatientNotFound, SummaryUnavailable -> HandoverSummary {
    var patient: Patient = attempt Patient.get(patient_id)
    return attempt ai.generate(
        SummariseNotes {
            notes: patient.notes
        },
        as: HandoverSummary
    ) {
        refused: SummaryUnavailable
        malformed: SummaryUnavailable
        timed_out: SummaryUnavailable
        over_budget: SummaryUnavailable
        too_long: SummaryUnavailable
    }
}
```

- **What reaches a model** is each field of a `Prompt` an action or
  function builds, whichever of `ai.generate`, `ai.stream` or
  `prompts.render` it is given to. It is also the text `ai.embed` and
  `ai.embed_all` send, since an embedding model is sent the text too.
- **Each is followed back** through variables, assignments, your app's
  functions and actions, standard operations (text built from it,
  `text.join`, `list.map`), lists, maps and objects. A query's `Output` is
  followed to the fields it was read from, and an event to what every
  `emit` of it put in it. A parameter that reaches a model is checked
  where it is passed. Where the compiler can't follow a value to a
  record, a marked field read from it counts as that field.
- **A category comes from** a field's `personal:`, or from a standard
  type that is personal data whatever field holds it
  ([Standard library](standard-library.md)). An `NhsNumber` and a `ChatMessage` are
  `health`, a `PaymentCardNumber` and an `Iban` are `financial`, and a
  `UkNationalInsuranceNumber` is `government_id`. A placeholder's own
  type counts too, so a prompt with a `List<ChatMessage>` sends `health`.
- **What decided a value is sent with it.** A placeholder carries a
  marked field when it was made under an `if` or a `match` on that field,
  or from what a `list.filter` testing it kept, or after a `return` or
  `reject` such a condition may have taken. This is the same rule policy
  uses ([Field restrictions](field-restrictions.md)). So
  `match patient.diagnosis { Diagnosis.hiv -> "hiv" ... }` sends the
  diagnosis. Only what a loop goes through is left out: the records and
  rows of a `for`'s list, and of `list.map`'s and `list.count`'s, carry
  nothing into what is made from them but what decided which ones they
  are. Otherwise a loop over records would send each one's every field.
- **A field keeps the marks of what it stores.** A create, an update or a
  transition may store a marked field's value, or a value one decided,
  only in a field with the same marks: each category, from `personal:`
  or the field's type, and `to_models: false`. Otherwise it is
  `semantic.ai_field_unmarked`, since whatever reads that field later, a
  prompt among them, would send it unchecked. A value from a parameter
  is checked where it is passed. One from an event is checked where a
  subscriber passes it, against what every `emit` put there and the
  event's own marks.
- **A model's answer carries what its prompt was built from.**
  `ai.generate`'s answer and `ai.stream`'s text, and each field read off
  them, hold everything the prompt's placeholders were built from, since
  the model may put any of it anywhere in what it writes. So the
  `.summary` of a patient's notes is `health` when it is stored
  (`semantic.ai_field_unmarked`) or sent to a model again, and
  [policy](policy.md) holds it to who may read the notes. An embedding
  already carries the text it was made from.
- **Errors name the field and its category,** never a value.

**Jadpo doesn't guess what to redact.** It doesn't scan free text for
names or numbers to blank out, because detectors miss things and give
false assurance. What reaches a model is what the prompt type's fields
hold. So you control it with the type (send a `pick` of only the fields
needed), the category list and `to_models: false`. Jadpo doesn't swap a
person's details for stand-ins and restore them in the answer.

## Prompts and answers in logs

- **Never by default.** In production a call's trace span
  ([Telemetry](telemetry.md)) holds the prompt's
  name and digest, the model, the attempts, the `TokenUsage`, the
  estimated cost, the duration and the outcome. It never holds the
  rendered prompt or the answer.
- **In development, in full.** `jadpo calls` shows them with every other
  provider call
  ([Fakes and the development call log](fakes-and-the-development-call-log.md)).
  They are kept with the last 500 timelines and never committed.
- **Keeping them in production is a choice per prompt:**
  `keep_content: { prompts: [answer_question]  store: main  keep: 30d }`
  on `ai:`. They are stored encrypted as `health` data: every read
  recorded, exported and erased with the person the call was made for,
  and deleted at `keep:`.

## Prompt injection

Text from users can contain instructions, and no wording or format stops
a model from sometimes following them. So Jadpo makes following them
harmless, in four ways:

1. **Instructions and data are kept apart.** The instructions are fixed
   text in a reviewed file. Free text is only ever sent inside a tagged
   data block under the request, never pasted into the instructions.
2. **The answer is data, not action.** It is a typed value, checked like
   a request body, and your code decides what happens next. A model can
   call nothing: tool calling is left to packages.
3. **The answer proves nothing.** Policy never grants access because of
   it, an answer can't be a reference to a record, and its text is
   escaped wherever it is shown.
4. **The budget caps the damage** of an injected "write ten thousand
   words".

## Budgets

Every app that asks a model sets a daily limit, so a loop or an injected
prompt can't spend without end. You can add limits per call, per request
and per caller:

| Limit | Written | Counted | Over it |
|---|---|---|---|
| Per call | `max_output:` on the call (default 1,000) | Sent as the provider's maximum output tokens; the input checked against the model's context before sending | `over_budget` (cut off) or `too_long` |
| Per request | `limits: { per_request: }` in tokens | Every `ai` call in one route request, subscriber delivery or schedule run, attempts included | `over_budget` before the call that would pass it |
| Per day | `limits: { per_day: { tokens: cost: } }` | The app's calls since midnight UTC; cost estimated from the package's prices | `over_budget` |
| Per caller per day | `limits: { per_caller_per_day: }` in tokens | As per day, for each signed-in principal (or address) | `over_budget` |

- **Checked before sending.** A call is refused when the remaining
  budget is below the most it could use: the input estimate plus
  `max_output:`, times `attempts:`. Each repair is counted with the
  conversation it resends, and each send again after a provider's error
  is taken before it goes ("Budgets count every send" in the
  [design notes](https://github.com/jadpo-dev/jadpo/blob/main/docs/language/notes/21-ai.md)). After the call, it is charged its real
  `TokenUsage`, every send included.
- **Counts live in the main store,** in a runtime table updated with a
  one-step counter
  ([Counters and other one-step updates](counters-and-other-one-step-updates.md)).
  So every instance shares them and a count is never lost. A store that
  can't be reached refuses the call `unavailable` rather than spend
  without counting.
- **Nothing is unlimited.** `per_day` is required on `ai:`
  (`semantic.ai_budget_missing`).
- `jadpo ai usage` shows the day's and month's tokens and estimated cost
  by prompt and model, from the counts in the main store.

The details are under [How budgets are counted](how-budgets-are-counted.md).

Not built yet: the personal-data report's entry for prompts, the
`ErasureNotPossible` acceptance for the `ai` connection, and
`keep_content:`. [What is built](ai-built.md) lists each.