# Embeddings and search An embedding is a vector, a list of numbers, that stands for a text's meaning: texts that mean much the same have vectors close together. You store a record's embedding in a field and find the records nearest a question with a query. PostgreSQL holds and searches them through its pgvector extension. ## Storing a vector A field of a vector type with `dimensions:` is stored in a pgvector `vector(n)` column. The type also says how its vectors are compared, since that depends on the model that made them: ```jadpo /// A help article's meaning, as the configured embedding model makes it. type ArticleMeaning = Vector { dimensions: 1536 distance: VectorDistance.cosine } ``` ```jadpo use support/types { ArticleMeaning, ArticleSlug, ArticleTitle } /// An article on the help site, with its meaning once it is embedded. entity HelpArticle { id: Uuid slug: ArticleSlug { unique: true } title: ArticleTitle body: Markdown shown_publicly: Bool body_meaning: ArticleMeaning? } ``` Its migration creates the extension first, since this is the store's first vector column, and the column takes the type's dimensions: ```sql -- store: main CREATE EXTENSION IF NOT EXISTS "vector"; CREATE TABLE "help_article" ( "id" uuid NOT NULL, "slug" text NOT NULL, "title" text NOT NULL, "body" text NOT NULL, "shown_publicly" boolean NOT NULL, "body_meaning" vector(1536), CONSTRAINT "help_article_pkey" PRIMARY KEY ("id") ); ``` | `VectorDistance.` | pgvector operator | Operator class | Use | |---|---|---|---| | `cosine` (the default) | `<=>` | `vector_cosine_ops` | Most text embeddings | | `euclidean` | `<->` | `vector_l2_ops` | Models that say so | | `inner_product` | `<#>` (the negative inner product) | `vector_ip_ops` | Normalised vectors, slightly faster than cosine | Use cosine unless your model's documentation says otherwise. Smaller is always nearer, so `inner_product` gives pgvector's negative inner product as it is. `distance:` fits any type built on `Vector`, with or without `dimensions:`. A type built on another keeps its metric, and giving it a different one is `type.constraint_contradiction`. With no `distance:` anywhere in the chain of types, it is cosine. A plain `Vector`, with no `dimensions:`, is stored as `vector` with no length and can't be searched (`query.nearest_unindexed`). An `Embedding` is stored in three columns: its model's provider and name, as a `ModelId` is, and its vector in a `vector` without a length, since one column would lose which model made it. A vector column checks no length beyond what its type says: `vector(n)` keeps the dimensions, and pgvector keeps its limit of 1 to 16,000 items. A vector goes to PostgreSQL in pgvector's binary format, sent with no type number, so the server infers `vector` from the column it meets. That is because the extension's type number differs in each database. A column of it is known by its type's name. `ai.distance` works out the same distance in code. It follows pgvector's C source: each item a single-precision float, the sums kept in single precision item by item, then the square root or the division in double precision. The result is the shortest decimal that is that double, as PostgreSQL writes a `float8` (rounded to a decimal's 38 places below `1e-21`). Where the items' products and running sums are exact floats, the two agree to the bit. Elsewhere they can differ in a float's last place, since pgvector's compiler vectorises and may add up in an order that depends on the server (within 1e-4, relative, on model-like vectors). A negative zero, the inner product of vectors at right angles, is `0`. A sum past a float's range, which pgvector answers with infinity, is a decimal overflow. `ai` is a standard namespace, so a domain, library or variable called `ai` is `semantic.standard_namespace_reserved`, as with every standard namespace ([Words that are names everywhere else](naming.md#words-that-are-names-everywhere-else)). Two vectors of different types can't be compared (`type.mismatch`), even with the same dimensions and metric, since they may come from different models. Nor can a vector that may be `none`: ```jadpo /// A help article's meaning, as the configured embedding model makes it. type ArticleMeaning = Vector { dimensions: 1536 } /// Whether two articles say much the same, by the angle between them. type MuchTheSame = Bool {} function much_the_same(a: ArticleMeaning, b: ArticleMeaning) -> MuchTheSame { return MuchTheSame(ai.distance(a, b) <= 0.2) } ``` A vector compared by cosine needs a direction, so a vector of zeros breaks `distance`, as a vector of the wrong length breaks `dimensions`. pgvector answers a cosine from it with `NaN`, and its index leaves it out, so a search would treat it differently from a scan. It is refused wherever it is made, so no row Jadpo writes holds one: - Read from JSON, it is a `mismatch`. - Built from a list only the running program knows, the construction's failure for `distance:` must be mapped. - Written out in code, it is `type.invalid_literal`. A list written out passes when one item is clear of zero (above `2e-30` either way, since an item below `1e-30` is held as 0). A vector compared by straight-line distance or inner product may be all zeros. Here a model's reply is read as a vector compared by cosine: ```jadpo type ModelReply = Text {} type MeaningUnreadable = Failure { kind: FailureKind.invalid_value } /// A help article's meaning, compared by cosine, the default. type ArticleMeaning = Vector { dimensions: 3 } /// A help article's meaning from the model's reply: JSON that isn't /// one, a vector of another length, or one with no direction. function article_meaning(model_reply: ModelReply) fails MeaningUnreadable -> ArticleMeaning { return attempt json.read(model_reply, as: ArticleMeaning) { malformed: MeaningUnreadable mismatch: MeaningUnreadable } } ``` ```jadpo test "a meaning has a direction" { var read_back: ArticleMeaning = attempt article_meaning("[0, 0, 0.5]") assert read_back == ArticleMeaning([0.0, 0.0, 0.5]) assert article_meaning("[0, 0, 0]") fails MeaningUnreadable assert article_meaning("[0.5, 0, 0.5, 0]") fails MeaningUnreadable } ``` ```jadpo /// A help article's meaning, compared by cosine, the default. type ArticleMeaning = Vector { dimensions: 3 } /// A meaning to start a search from before anything is known. function blank_meaning() -> ArticleMeaning { return ArticleMeaning([0.0, 0.0, 0.0]) } ``` ```text error: `[0.0,0.0,0.0]` isn't a valid `ArticleMeaning` --> main.jadpo:8:27 | 8 | return ArticleMeaning([0.0, 0.0, 0.0]) | ^^^^^^^^^^^^^^^ breaks `distance: VectorDistance.cosine` reason: `Vector` declares `distance: VectorDistance.cosine`, and the value is a vector of zeros, which has no direction to compare by cosine. A literal is checked against every rule while compiling. next: Use a value that `ArticleMeaning` allows (agent) rule: type.invalid_literal help: https://jadpo.dev/docs/errors/type.invalid_literal check failed: 1 error, 1 file ``` ## The nearest-neighbour query To find the records whose vectors are nearest one you have, a query orders by nearness with `nearest`. Its result is a list of `Nearest`, a standard type holding `item: T` and `distance: Decimal`, as `Page` holds a page. Here the query finds the five public help articles nearest a question: ```jadpo use support/entities { HelpArticle } /// What a reader sees of a help article in a list of answers. type ArticleExcerpt = Output { pick: [HelpArticle.id, HelpArticle.slug, HelpArticle.title] } ``` ```jadpo use support/entities { HelpArticle } use support/types { ArticleExcerpt, ArticleMeaning } /// The five public articles nearest a question, none further than 0.6. query closest_articles(question_meaning: ArticleMeaning) -> List> { from: HelpArticle where: HelpArticle.shown_publicly == true where: ai.distance(HelpArticle.body_meaning, question_meaning) <= 0.6 order: [HelpArticle.body_meaning nearest question_meaning] limit: 5 } ``` It runs as three settings and one statement, which `jadpo sql closest_articles` prints: ```sql -- support.closest_articles SET LOCAL hnsw.ef_search = 40; SET LOCAL hnsw.iterative_scan = strict_order; SET LOCAL hnsw.max_scan_tuples = 2147483647; SELECT * FROM ( SELECT "id", "slug", "title", "body_meaning" <=> $1 AS "jadpo_distance" FROM "help_article" WHERE "shown_publicly" = $2 AND "body_meaning" <=> $1 <= $3 AND "body_meaning" IS NOT NULL ORDER BY "body_meaning" <=> $1 LIMIT 5 ) AS "nearest" ORDER BY "jadpo_distance", "id"; ``` - **`nearest`** is an `order:` entry, and the only one in its query. It goes on a vector field with `dimensions:`; on any other vector field, an `Embedding` field included since its vector has no fixed length, it is `query.nearest_unindexed`. It measures from a parameter of the field's own type, or a field of one, never a vector written out (`query.nearest_type` otherwise). A value to measure from that may be `none` is `query.optional_value`. - **The result** is `List>`, and the query needs a `limit:` of at most 100. A `limit:` above 100, or a value whose type allows more, is `query.limit_range`. A field that isn't a vector, a result that isn't `List>`, and a `Nearest` result without `nearest` are `query.nearest_type`. A `Page` of nearest records is refused, since a cursor over distances isn't stable as rows change (`query.nearest_page`). Records whose vector is `none` are left out. - **The order.** The inner statement orders by distance alone, which is what the index serves. PostgreSQL won't read an HNSW index for an order with a second key, so the tie-break on `id` is applied outside, to the rows the index found. (With `, id` in the same `ORDER BY`, the plan is a scan of the whole table and a sort.) - **`distance`** is the distance pgvector worked out: its `float8` read by its bits and written as `ai.distance` writes the same double. Where the sums are exact, it is the same to the bit as `ai.distance` in code. A cosine that pgvector answers with `NaN`, from a vector of zeros written outside Jadpo, is a fault, never a made-up distance. - **`ai.distance(a, b)` in a query.** `ai.distance` is a standard operation on two vectors of one type, giving that type's distance. In code it is worked out by the program. In a query's `where:`, whether the query orders by `nearest` or not, it becomes pgvector's operator. There you compare it with `<`, `<=`, `>` or `>=`, the field first and a parameter of its type second (`type.mismatch` otherwise, as in code), against a number written out or a decimal or whole-number parameter. `==` and `!=` are `query.condition_unsupported`, since two computations of a float may differ in its last place. So is a distance in `where_present:`. - **What never matches.** A record whose vector is `none` meets no distance condition, `not` included. Nor does one whose distance is `NaN` or an infinity, which code never gives, so a row whose vector was written outside Jadpo meets nothing. PostgreSQL sorts `NaN` above infinity and infinity above every number. So `>`, `>=` and the `not` of `<` and `<=` each add `distance < 'Infinity'`. An inner product can be minus infinity, so it adds `distance > '-Infinity'` to `<` and `<=`. - **Code and the database agree on the bound.** Code compares `ai.distance`'s decimal, the shortest that is the double, with the bound. PostgreSQL would compare the double with its own rounding of the bound, which differs for a bound of 17 or more places. So the bound is sent as a `float8`: the largest double whose decimal is at most the bound (or below it, for `<` and `>=`). Every comparison becomes "at most" or "above" that double, and the two agree on every double. A bound the caller gives is worked out as the read runs. - **Filters still fill the limit.** An approximate index filters after it scans, so with a `where:` it can return fewer rows than `limit:`. To prevent that, the runtime: - sets `hnsw.ef_search` to the larger of pgvector's default, 40, and the most records the query returns (its `limit:`, or the `max:` of its value's type); - turns on pgvector's iterative scans (0.8.0 and later) in strict order; - sets `hnsw.max_scan_tuples` to pgvector's most, 2,147,483,647. pgvector's default, 20,000, would end a filtered search early, silently. A scan then ends only when it has filled the limit or visited every record, so a filtered query fills its limit when enough rows match. The memory a scan may use stays pgvector's bound, `work_mem` times `hnsw.scan_mem_multiplier`, at the server's settings. Raising it is tuning for the database, and a scan that reaches it still ends early. - **`Nearest` stays out of routes.** A route returns only an `Output` type, or a list or page of one ([Output and status](output-and-status.md)), so code maps the items it needs.