People and faces
Beta
The People endpoints are marked beta: they may change on 30 days notice rather
than the usual 90. See versioning.
Face recognition here is deliberately not automatic end to end. Detection is automatic, but identity is a decision a person makes, and the API is shaped around that:
Code
The middle step is the important one. A suggestion is the matcher proposing a probable match. It is not an assignment, and nothing is attributed to anyone until it is confirmed.
Detections
Faces are found during analysis. GET /files/{id}/faces lists
what was found in one photo, most confident first, each with the person it was
matched to if any:
Code
fd_low_quality marks a face too blurry to match reliably. These detections still exist
and still display, but they are excluded from matching and from face counts, which is why
a photo with six blurry people in the background does not report six faces.
The photo's file_metadata.faceCount is the countable total, and has_faces=true on the
media library filters on it.
People
A person is a cluster of faces, with or without a name attached. A face detected in twelve photos that nobody has named is still a person, just an unnamed one.
Code
Most-photographed first, each with counts:
Code
face_count | Faces assigned to this person. |
manual_count | How many of those a human confirmed. These are the references the matcher trusts most. |
suggestion_count | Faces waiting for review. |
Two filters exist for the two queues worth working through: unnamed=true for clusters
nobody has named, and needs_review=true for people with suggestions pending.
GET /people/{id}/photos lists the photos someone appears in and applies each file's own
access rules, so two callers asking about the same person can legitimately get different
photos back.
Assign a face to a person
POST /people/assign assigns one detection. Omit fp_id and a new person is created for
it, which is how someone is named for the first time:
POST /people/unassign reverses it. PUT /people/{id} renames,
POST /people/{id}/set-cover chooses the face shown as their thumbnail, and
DELETE /people/{id} removes the person while leaving the underlying detections
alone.
Each confirmed assignment improves the next match. Manual assignments are the references the matcher ranks first, so naming a handful of clear photos of someone early is worth more than correcting fifty later.
The review queue
Code
Faces the matcher believes belong to someone already named, closest first. Filter to
one person with fp_id. Resolve them one at a time with
POST /people/suggestions/{id}/confirm or /reject, or in bulk:
Code
Code
skipped counts suggestions already assigned or already resolved by someone else. The
batch does not fail over them, which matters when two reviewers work the same queue. Read
the counts to see what your call actually did.
What the queue does not show you
The suggestion list is filtered before you see it rather than after. Three checks withhold a suggestion rather than making you reject it:
- Gender. A candidate with a confidently estimated gender that never appears among the person's confidently-gendered references is never suggested.
- Age. A candidate more than 20 years from the person's median age is withheld.
- A
no_matchverdict from the vision check below.
This is why the queue is usually shorter than the raw match count, and why an obviously wrong suggestion is worth reporting: it means one of these checks is not working.
Verify suggestions with vision
Code
Runs one vision comparison of the person's reference faces against each unverified
candidate and records match, no_match or unsure. It catches what face embeddings
cannot: twins, siblings, and the same person at very different ages.
It decides nothing on its own. Verdicts come back through the suggestion list, and a
no_match withholds that suggestion from the queue. It is idempotent and skips
already-verified candidates, so calling it twice costs nothing. Run it before a review
session so the queue you hand to a person is already filtered.
Merge and reindex
Two clusters representing the same person is the normal failure mode: someone photographed across a decade, or with and without glasses.
Code
Every face moves from source_id onto target_id, and the source is retired. There is no
unmerge, so confirm the direction before calling it.
POST /people/{id}/reindex rebuilds a person's face index from their current assignments.
Run it after a large merge or a batch of corrections, so subsequent matching uses the
improved reference set.
Search by face
Two options, answering different questions:
face_person=<fp_id> on GET /media-files | Photos of someone you have named. Exact, cheap, no model call. |
face_embedding=[…] | Photos resembling a face you supply, named or not. See AI enrichment. |
Use the first whenever the person already exists. It is a lookup rather than a search.
How matching works
Every face is reduced to a numeric descriptor, and matching compares the Euclidean distance between two of them. Three outcomes follow. Close enough, and the face is assigned automatically. In a middle band, it becomes a suggestion for someone to confirm or reject. Beyond that, it is not treated as a match at all.
That middle band is the entire review queue. Tightening the point where a match becomes automatic sends more faces to review and produces fewer mistakes; loosening it does the reverse. Faces below a minimum size are never matched automatically whatever the distance, because a small crop's descriptor is not reliable.
The thresholds are workspace-level tuning rather than per-request parameters, and the defaults are calibrated against the canonical cutoff for the descriptor in use.
Next steps
- AI enrichment covers what detection runs alongside.
- Organizing and finding assets shows the face filters in context.
- Sharing and access explains why two people see different photos of the same person.