QueryForge — Configuration Reference

A QueryForge configuration is a single JSON file. It does four jobs at once: it is the prompt context the model sees, the validation rulebook, the logical→physical field mapping per backend, and the model selector. This page documents every option so you can write your own.

What is the Query AST?

AST stands for Abstract Syntax Tree. In QueryForge it is a plain JSON object describing what the user asked for, sitting between the English sentence and the finished database query. You will see the term throughout this page and in the API responses, so it is worth two minutes.

The name decodes literally:

• Tree — filters nest. A AND (B OR C) is a branching structure, not a flat list, so a logical node holds children, and each child may itself be another logical node.
• Abstract — it knows nothing about SQL or Mongo. It says "operator": "gt", never > and never $gt. It carries the meaning, stripped of any one database's syntax.
• Syntax — it is a fixed grammar with a small set of node types, not free-form text.

One AST, every backend

Ask: "orders over 500 dollars in the last 30 days that were not cancelled, newest first, top 20". The model produces this — and nothing else:

{
  "entity": "Order",
  "filter": {
    "type": "logical", "op": "AND",
    "children": [
      { "type": "comparison", "field": "amount",    "operator": "gt",        "value": {"kind":"number","v":500} },
      { "type": "comparison", "field": "createdAt", "operator": "after",     "value": {"kind":"relative_date","unit":"day","amount":-30} },
      { "type": "comparison", "field": "status",    "operator": "notEquals", "value": {"kind":"enum","v":"CANCELLED"} }
    ]
  },
  "sort":  [{ "field": "createdAt", "dir": "DESC" }],
  "limit": 20
}

That single object then compiles deterministically into both of these:

SELECT … FROM orders
WHERE (status <> $1 AND created_at >= $2 AND amount > $3)
ORDER BY created_at DESC LIMIT 20

{ amount: {$gt: 500}, createdAt: {$gte: "…"}, status: {$ne: "CANCELLED"} }

Same AST, two databases, no second prompt. Adding a backend means writing one generator — it never means rewriting the prompt.

Why not just have the model write SQL?

This is the central design decision, so the alternative deserves a straight answer. A model asked for SQL directly can invent a column, invent a table, or emit a statement you cannot check before running it. There is no good place to ask "is this user even allowed to filter on that field?", and every new dialect means a new prompt.

With an AST in the middle, the model's job shrinks to filling in a form. Everything after that is ordinary, testable Go:

1. Validation — every field, operator, and value is checked against your config. An invented field is rejected here, before anything is compiled.
2. Generation — the approved AST becomes a real query, with values bound as parameters ($1, $2) rather than pasted into the statement, so a string like Robert'); DROP TABLE orders;-- arrives as data and can never execute.

The node types

The root object accepts these keys. Everything is optional except entity; omit what you do not need.

Inside filter, every node is one of exactly two shapes:

Seeing it yourself

Every /api/translate response includes the AST it produced, in the "ast" field. The companion /api/generate endpoint accepts one directly and makes no model call at all — so it needs no API key and always returns the same query for the same input. Take the ast from a translate response, post it to generate with a different backend, and watch one object become the other dialect.

Top-level structure

A config has six top-level keys. Only entity and fields are strictly required, but a real config also sets model (to translate natural language) and backends (to map to physical tables/collections).

The loader is strict: an unknown key anywhere in the file is rejected, so typos surface immediately rather than being silently ignored. JSON only (no comments).

model

Selects the AI planner target. Switching provider or model is a config change, never a code change — any OpenAI-compatible /chat/completions endpoint works (hosted or self-hosted).

models — fallback chain (optional)

Beyond the single model block, an optional models array lets you list several models in priority order. QueryForge tries them in order and uses the first that answers, so a rate-limit, quota, or billing block on one provider transparently falls through to the next. model is the primary; models are the ordered fallbacks, and a chain can freely mix providers (e.g. Gemini → Groq → Anthropic → local Ollama). Each entry is a full model block with its own apiKeyEnv.

"model":  { "provider": "gemini", "baseURL": "…/v1beta/openai", "model": "gemini-3.1-flash-lite", "apiKeyEnv": "QF_API_KEY" },
"models": [
  { "provider": "groq",      "baseURL": "https://api.groq.com/openai/v1", "model": "llama-3.3-70b-versatile", "apiKeyEnv": "GROQ_API_KEY" },
  { "provider": "anthropic", "baseURL": "https://api.anthropic.com",      "model": "claude-opus-4-8",          "apiKeyEnv": "ANTHROPIC_API_KEY" },
  { "provider": "ollama",    "baseURL": "http://localhost:11434/v1",      "model": "qwen2.5" }
]

Omit models entirely for a single model. The provider: "anthropic" label (or an api.anthropic.com base URL) selects the native Anthropic Messages API; everything else uses the OpenAI-compatible dialect.

apiKeyEnv holds a NAME, not a key

This is the single easiest thing to get wrong, so it is worth stating twice. apiKeyEnv names the environment variable that holds your key; the key lives in the environment, never in the file.

// Correct — the field names a variable, and the key stays outside the file
"apiKeyEnv": "QF_API_KEY"        →  export QF_API_KEY=AIza…

// Wrong — the key pasted into the field
"apiKeyEnv": "AIzaSy…"           →  rejected at load

Getting this wrong used to fail in a way that pointed nowhere near the cause: the library would look up an environment variable literally named AIzaSy…, find nothing, send no Authorization header, and the provider would answer "Missing or invalid Authorization header" — an error about a header you never wrote, describing a key you were certain you had configured. The pasted secret would also travel wherever the config travelled, including into version control and startup logs.

QueryForge now rejects it when the config loads, in the primary model block and in every models fallback entry. A value is refused if it opens with a known key prefix (AIza, sk-, sk-ant-, gsk_, ghp_, xox) or is not shaped like a variable name (letters, digits and underscores, not starting with a digit). The error explains the fix and never repeats the value back — echoing it would put the secret straight into your logs.

backends

Maps the logical entity to a physical source per backend. Each backend uses its idiomatic key. If omitted, the entity name is used as the source.

fields

The heart of the config: each entry is one queryable attribute. This is the complete list of per-field keys.

Field types

The type bounds which value kinds and default operators are legal.

Nested fields (MongoDB)

Mongo documents are rarely flat. QueryForge handles nesting entirely in the mapping layer, so the logical vocabulary stays flat: the model still asks about itemSku and never learns that the physical path is items.sku. Two shapes need different treatment.

Embedded documents — a dot path is enough

When the parent is a single sub-document, a dot path in mapping.mongo is exact. Nothing else is needed.

{ "name": "city", "type": "string", "mapping": { "mongo": "shippingAddress.city" } }

"orders shipping to Pune"
→ { "shippingAddress.city": "Pune" }

Arrays of sub-documents — declare the array with elemMatch

When the parent is an array, dot paths alone are silently wrong. Mongo satisfies each dot-path predicate independently, so two conditions can be met by two different elements. Given this order:

{ "_id": 1, "items": [ { "sku": "ABC", "price": 20 },
                       { "sku": "XYZ", "price": 900 } ] }

the question "orders containing item ABC costing over 100" compiles two ways:

Declaring the array on each field is what selects the second form:

{ "name": "itemSku",   "type": "string", "mapping": { "mongo": "items.sku" },   "elemMatch": "items" },
{ "name": "itemPrice", "type": "number", "mapping": { "mongo": "items.price" }, "elemMatch": "items" }

Rules

• elemMatch must be a leading dot-path segment of the field's Mongo path, and cannot be the whole path. elemMatch: "items" with mongo: "items.sku" is valid; with mongo: "lines.sku" the config is rejected at load.
• Paths deeper than one level work: mongo: "items.dims.w" with elemMatch: "items" produces {"items":{"$elemMatch":{"dims.w":…}}}.
• Grouping is per array. Predicates on items and on payments produce two independent $elemMatch documents.
• Grouping applies to AND only. Under OR each branch is satisfied independently already, so the branches stay separate — folding them would change the question.
• Two operators on one relative key merge into a range: price >= 100 AND price <= 500 becomes {"price":{"$gte":100,"$lte":500}}. Predicates that cannot merge without dropping one are kept in an $and inside the $elemMatch, so the same-element guarantee holds either way.
• Projection and sort keep the full path (items.sku). $elemMatch is a filter construct; a projection keyed sku would name a field that does not exist at the document root.
• A malformed path — an empty segment (items..sku), a leading or trailing dot, or a segment starting with $ — is rejected at load. Left in, it would build a filter key that simply matches nothing, with no error to explain the empty result.
• Explain says which reading it picked: "Conditions on items apply to the same array element."

Capability flags

Five boolean flags gate what a field can do. They sit alongside operators: operators bounds which comparison operators are legal; the flags bound where the field may appear.

Use queryable: false + returnable: false together to fully hide a sensitive column (e.g. an SSN) from both querying and results.

Value case — matching a column that stores UPPERCASE v0.0.9

Databases and people rarely agree on case. A status column holding SHIPPED will never match the shipped someone typed, and asking the model to shout is both unreliable and beside the point: the case belongs to the storage, not to the question. valueCase states it once, per field:

{ "name": "status", "type": "enum", "values": ["shipped", "cancelled"], "valueCase": "upper" }

The conversion happens only when the query is built. Everything above the generator keeps the spelling you wrote:

• The model never learns the key exists, and still emits the values exactly as listed.
• Validation still checks that spelling verbatim — with the config above, an AST carrying "SHIPPED" is rejected as out of domain. Your enum stays one readable vocabulary rather than two.
• The AST stays round-trippable, so the same AST compiles correctly for a second backend that cases things differently.
• Application-supplied scope filters are cased too — they reach the generator like any other predicate, and a tenant column stored upper-case needs it just as much.

It applies to every string a predicate carries — equals, notEquals, in, notIn, between, contains, startsWith, endsWith, containsAny, containsAll — on both backends, including fields inside an elemMatch array. Values that are not text are never touched: numbers, booleans and dates pass through unchanged.

Only fields whose values are text may set it: string, enum, or an array of either. On a number, date or boolean the key would do nothing, so the config is rejected at load rather than loading with a setting that quietly never fires. A misspelt setting ("uppercase", "UPPER") is rejected for the same reason — the only accepted values are "upper" and "lower".

indexed & priority

These two are hints, not gates — they never reject a query. They steer deterministic behavior:

• Predicate ordering: within an AND/OR, predicates on indexed fields are emitted first, then higher priority first. Because AND/OR are commutative, this is safe and improves how the database plans the query.
• Soft warnings: filtering on a non-indexed field produces a non-fatal warning in the result ("filtering on non-indexed field … may be slow") that your app may surface or ignore.
• Prompt ordering: higher-priority fields are presented earlier to the model.

Operator catalogue

The fixed set of operators. Which ones a field may use is bounded by its operators list (or the type default).

Logical connectives AND, OR, NOT combine predicates in the AST.

Value kinds

Each comparison value in the AST is tagged with a kind so the validator can enforce type legality before any query is built.

defaults & policy

defaults

policy

The remaining keys bound the work one request can cause, which depth alone does not: a filter can be flat, enormous, and entirely legal. Each has a built-in default, so leaving it unset is already bounded; set a negative value to switch a bound off.

One regex rule has no key and cannot be switched off. A pattern that nests an unbounded quantifier inside a quantified group — (a+)+, (x+x+)+y, ^(\w+\s?)*$ — is rejected with regex_unsafe, whatever allowRegexOn says. Those take exponential time in the database process, once per row, and are reachable from an ordinary English sentence; unlike the two gates above, there is no legitimate use to weigh against. Rewrite with a bounded repetition such as {1,20}. The check is structural and narrow on purpose — it does not catch alternation overlap like (a|ab)*, so set a statement timeout at the database as well.

NULL and missing fields across backends

SQL uses three-valued logic: status <> 'CANCELLED' is NULL — not TRUE — for a row whose status is NULL, so that row is excluded. MongoDB has no such rule: $ne, $nin and $nor all match a document that lacks the field.

QueryForge follows the SQL reading on both, because it is the one that never over-returns. The Mongo generator adds "$exists": true to notEquals, to notIn, and to a NOT whose single child is a value comparison, so the same AST selects the same logical rows on either backend.

Two deliberate exceptions. isNull/isNotNull are about absence and already agree, so they are left alone. And a NOT over a whole logical subtree keeps plain $nor: there is no single field to require, and requiring all of them would be wrong — SQL's NOT(A AND B) is TRUE when A is NULL and B is FALSE, so the row comes back even though A's column is empty. Negate individual predicates rather than groups if you need the two backends to agree exactly there.

Scope filters — extra filters you supply per call v0.0.2

Some predicates are not the user's to choose. Your application already knows the caller's subscription, user, and enterprise before any question is asked, and every query must be confined to them whatever the question says. Those values are not query vocabulary — nobody should be able to phrase, widen, or omit them — so they do not belong in fields and must never be shown to the model.

Pass them as an extra map argument instead. Every entry is AND-ed onto the query:

res, err := engine.Translate(ctx, "delivered orders over 500 dollars", "sql", qf.Scope{
    "subscriptionId": session.SubscriptionID,   // "SUB-42"
    "userId":         session.UserID,           // 9
    "enterpriseId":   session.EnterpriseIDs,    // []string{"E-1", "E-2"}
})

SELECT ... FROM orders
WHERE (subscriptionId = $1 AND userId = $2 AND enterpriseId IN ($3, $4)
       AND status = $5 AND amount > $6)
ORDER BY created_at DESC LIMIT 50
-- args: ["SUB-42", 9, "E-1", "E-2", "DELIVERED", 500]

The same map works on the deterministic path, which takes the argument too — otherwise a caller could sidestep tenancy just by building the AST themselves:

q, err := engine.GenerateFrom(ast, "sql", scope)

Pass nil for no scope; that is exactly the pre-v0.0.2 behaviour.

Why this is safe

Accepted values

Keys are applied in alphabetical order, so the generated query and its argument order are identical run to run.

Two ways to name a scope field

The second form matters as soon as the physical names differ. Declare the column once and one scope map compiles correctly everywhere:

{
  "name": "tenantId",
  "type": "string",
  "queryable": false,                                   // hidden from the model, still scopable
  "mapping": { "sql": "tenant_id", "mongo": "tenantId" }
}

engine.GenerateFrom(ast, "sql",   qf.Scope{"tenantId": "T-1"})  // ... WHERE tenant_id = $1
engine.GenerateFrom(ast, "mongo", qf.Scope{"tenantId": "T-1"})  // { tenantId: "T-1", ... }

What you get back

TranslateResult.Scope always lists the filters that were applied, normalized — the record an audit log should keep. The explanation states them separately, so a readback never presents a forced predicate as something the user asked for:

Return all fields from Order where status equals "DELIVERED".
Always scoped to subscriptionId equals "SUB-42", userId equals 9.

The AST itself is controlled by engine.ScopeInAST. The compiled query and the explanation always include the scope; only this one field changes:

Requests the config cannot express

Your config defines the entire vocabulary. When a user asks for something outside it — a field you never configured — the honest answer is "I can't", and QueryForge is built to give that answer rather than to improvise.

Left to itself a model will happily map an unknown concept onto whichever configured field looks closest. Asked for "orders where the shipping warehouse is in Berlin and the courier is DHL" against a config with neither field, one model produced tags contains "Berlin" AND tags contains "DHL". That AST is structurally legal, so it passes validation, and the caller receives confident, silently wrong rows. For a correctness-critical library that is worse than an error.

So the planner gives the model an explicit way to decline. If a request needs a field, operator, or enum value that the config does not define, the model returns a refusal marker instead of an AST, and QueryForge surfaces it as a typed *UnsupportedRequestError carrying the reason. No query is generated and no retry is attempted — a refusal is a definitive answer, not a failure. Callers should branch on it with errors.As:

res, err := engine.Translate(ctx, text, "sql", nil)
var unsupported *qf.UnsupportedRequestError
if errors.As(err, &unsupported) {
    // Not a bug: this config simply cannot express the request.
    fmt.Println("Cannot answer:", unsupported.Reason)
    return
}

Declining is reserved for genuinely missing vocabulary. A phrasing that matches a field's synonyms still resolves normally — "buyer" and "order value" map to customerName and amount without complaint. If you find legitimate questions being refused, the fix is usually to add synonyms to the field rather than to change the query.

Full examples

Four complete configs ship in examples/. Study them to see the patterns for each database family.

• examples/orders.config.json — an e-commerce Order, mapped to both SQL and Mongo from one config.
• examples/sql_employees.config.json — relational patterns: indexed + prioritized columns, snake_case mappings, RBAC on salary, an excluded ssn, numeric validators.
• examples/nosql_products.config.json — document patterns: array fields (categories, tags) with contains operators, text search on title/description, rating bounds.
• examples/mongo_nested.config.json — nested document patterns: an embedded address on a dot path, plus two arrays of sub-documents (items, payments) declared with elemMatch.

Annotated field

{
  "name": "salary",              // logical name used in NL and the AST
  "type": "number",              // bounds operators + value kinds
  "operators": ["gt","lt","gte","lte","between"],
  "synonyms": ["pay","compensation","wage"],
  "validators": { "min": 0, "max": 10000000 },  // rejected outside this range
  "sortable": true,              // may appear in ORDER BY
  "searchable": false,           // no text-search operators
  "mapping": { "sql": "salary" } // physical column name
}

Author your own config

The config builder walks these steps as a form and flags each mistake as you make it. If you would rather write the file by hand, this is the order that works:

1. Pick an entity name and set the model block (start with a free Gemini/Groq tier; put your key in the env var you name in apiKeyEnv).
2. Add a backends entry for each database you target, mapping to the physical table/collection.
3. For every attribute users should be able to filter/sort/search on, add a fields entry. Choose the right type, list synonyms people actually say, and set mapping if the physical name differs.
4. Mark indexed: true and a priority on the columns your database indexes.
5. Set queryable: false / returnable: false on anything sensitive.
6. For tenancy columns (subscription, user, enterprise), do not make them queryable. Either leave them out of the config entirely or declare them queryable: false with a mapping, then pass them per call as a scope.
7. Add defaults (limit/maxLimit) and policy (maxNestingDepth, denyRegexOn) to bound the blast radius.
8. Load it with queryforge.LoadConfig("your.config.json") — the loader validates structure and rejects typos immediately.

QueryForge · configuration reference · this document is generated as part of the library and lives at docs/config.html.