Smart generalization

Placeholders are safe but opaque: an LLM that receives DATE_01 or ADDRESS_01 cannot reason about a patient's age or a claim's region. Smart generalization replaces a detected DATE_TIME or ADDRESS span with an abstracted value instead of a placeholder, so the prompt keeps the shape of the fact while losing the identifying detail.

Generalization is opt-in per request, and only on the irreversible surfaces:

POST /api/v1/redact, via the transforms body field.
The LLM proxy, via the X-Kindi-Transforms header.

/api/v1/mask does not support it: generalized text has no {token: original} mapping, so it would break the envelope round-trip. That is v2 territory.

What it does

Input span	Context	Output
`01/15/1985`	near a DOB cue (`DOB`, `date of birth`, `born`, `تاريخ الميلاد`, `مواليد`)	`age 41`
`١٤ أبريل ١٩٩٠`	near `مواليد`	`العمر: 36 عاماً`
`14 April 2024`	no DOB cue	`April 2024`
`14/04/2024`	no DOB cue	`2024` (a numeric month would leak original digits)
`١٤/٠٤/٢٠٢٤`	no DOB cue	`٢٠٢٤` (Eastern-Arabic digits kept as written)
`King Fahd Road, Al-Olaya District, Riyadh 12345`	—	`Riyadh`
`شارع الملك فهد، حي العليا، الرياض 12211`	—	`الرياض`
`حي الشاطئ، الدمام، المنطقة الشرقية`	—	`المنطقة الشرقية` (region outranks city)

Ages are computed against the server date at call time and must land in a sane range (over 0, under 120). The address gazetteer covers the 13 Saudi administrative regions and about 30 major cities, Arabic and English forms.

Fail-closed

Anything the generalizer cannot handle safely gets the normal placeholder, never a partial reveal:

Hijri dates, numeric (١٤ / ٠٣ / ١٤٤٦ هـ) or verbal; v1 has no Hijri calendar support.
Clock times (10:30 صباحاً, 9:15 PM).
Date fragments with no Gregorian year (17 March).
A DOB-context date that does not parse to a full calendar date, or whose computed age is out of range.
An address with no recognizable Saudi city or region.

A generalized value never carries digits from the original beyond the year: day numbers, numeric months, street, building, and postal numbers are always dropped. The response's generalized_count reports how many spans actually generalized; everything else fell back to placeholders. pii_count and entity_counts are unchanged either way, and detection itself is untouched; generalization runs strictly after it.

On /redact

1234{
  "text": "Patient DOB: 01/15/1985, seen at Al-Olaya District, Riyadh",
  "transforms": {"DATE_TIME": "generalize", "ADDRESS": "generalize"}
}

123456{
  "redacted_text": "Patient DOB: age 41, seen at Riyadh",
  "pii_count": 2,
  "entity_counts": {"DATE": 1, "ADDRESS": 1},
  "generalized_count": 2
}

Only DATE_TIME and ADDRESS keys are accepted, only "generalize" as the value; unknown keys or values are rejected with 422.

On the LLM proxy

Send the same opt-in as a comma-separated header:

1234POST /api/v1/proxy/openai/v1/chat/completions
Authorization: Bearer mk_live_...
X-Provider-Key: sk-...
X-Kindi-Transforms: DATE_TIME,ADDRESS

The provider receives the abstracted value (April 2024, Riyadh) instead of a <MASKED_…> token. A generalized span gets no mapping entry, so the reply is not rewritten back; the abstraction is permanent for that conversation. Spans that fail closed still get a mask token and round-trip normally. An unknown type in the header is a 400 invalid_transforms.

The privacy trade-off, plainly

Generalization keeps more information than a placeholder. An age narrows a person to a birth-year cohort; a city narrows them to a population; together they can narrow further, and in a small or unusual population they can identify. The abstracted values also remain in the output permanently; there is no envelope and no way back.

Use generalization only where that residual information is acceptable for your data and your recipients. When in doubt, leave transforms off; the default placeholder behavior discloses nothing.

Not in v1

/mask support (breaks the envelope round-trip).
Hijri date parsing; Hijri spans fail closed.
File redaction and per-user default transforms.
Other entity types (a generalized PERSON or ORGANIZATION has no obviously safe abstraction).