- Before
- Answers for the 23 systems that need a decision under the Act, and will not sign a declaration she cannot trace to a document.
- Now
- Opens a file where every sentence shows its source, sees what stands between it and sign-off, and signs only when every question is closed.
One high-risk system, from classification to a signed declaration
Kestrel & Vane Group builds its own candidate shortlisting model, AIS-0417 v3.2, used by recruiters in 30 countries — about 212,400 applications a year. As the provider, it needs a complete technical file before 2 December 2027. Here is what happens, screen by screen, in the working solution.
- 01Wednesday, 09:40
Every AI system under the Act, and what needs her today
Helena Marsh · AI Compliance LeadOf the 186 systems in the register, 23 need a decision or carry duties. Eleven are high-risk — 6 the group provides, 5 it deploys — with 414 days to 2 Dec 2027, files 44% filled on average and 14 assessor questions open. One prohibited system, emotion analytics at work, is blocked. Six items need her today, ranked by risk to the date.
Needs you today: “AIS-0417: 3 assessor questions stand between the file and sign-off.”
- 02Classified
Why the shortlisting model is high-risk, test by test
Classification agentThe agent runs seven tests, each with the rule it applies: an AI system under Art. 3(1); no prohibited practice under Art. 5; not a safety component under Annex I; recruitment under Annex III point 4(a). The Art. 6(3) derogation cannot apply because the model profiles applicants. Kestrel is provider and deployer, the duties apply from 2 Dec 2027, and the route is internal control under Annex VI. Confidence 0.97.
What would change this: “If it only turned CVs into application fields, with no ranking — not high-risk: Art. 6(3)(a), a narrow procedural task.”
- 0341 minutes
Twenty-three documents found, seven sections drafted
Evidence finder · Section drafters · AssessorThe evidence finder searches six sources — the design wiki, document library, test library, model registry, code repository and data catalogue — and matches 23 documents to the nine Annex IV points. It flags one not reviewed in 12 months. The drafters write sections 1 to 7: 318 sentences, each cited to a passage, none without a source. Then the assessor reads the file and raises 9 questions, closing 6 on evidence the agents already hold.
“Nothing invented. Each sentence carries its document, version and passage; a person can check any of them in one click.”
- 04One click
Every sentence shows the passage it came from
Helena Marsh · AI Compliance LeadThe file opens as nine Annex IV tabs, each with its fill, its documents and anything missing in red. Helena clicks the first sentence of section 1 and the source lights up: the intended purpose statement IPS-0417-3, §1.1, from the design wiki, version 3.2, signed. Beside it sit the Article 9 risk file — 14 risks, 3 open — and the 17-point conformity checklist, 8 met.
“The system ranks each application against the vacancy's published requirements and presents a shortlist of no more than 25 applicants to the recruiter.”
- 05Section 3 · 74%
A gap in red, with the article and the person asked
AssessorSection 3 states accuracy of 0.91 overall but no figure for the groups of persons the system is used on. The drafters will not write it without a source, so it shows as missing. The assessor, reading as a market surveillance authority would, raises Q-0417-03 under Art. 15(1) and Annex IV 3 and sends it to Lucía Ferreira-Hahn, due Wed 21 Oct.
“Missing: Accuracy for the specific groups of persons the system is used on (age bands) … the drafters will not write this without a source.”
- 06She has the test
She answers with a signed test report
Lucía Ferreira-Hahn · Senior Data ScientistLucía uploads TR-0417-19, the subgroup accuracy test report, 14 pages, signed on 13 Oct 2026. The agents read the accuracy tables for 4 age bands, 2 sexes and 30 countries, check the date and signature, and compare the bands with the 0.05 tolerance: 55 and over at 0.88 against 0.92 for 18–29, a largest gap of 0.04.
The question closes only on a document or a signed answer.
- 07Seconds later
Section 3 turns green, and a risk is mitigated
Section drafters · Risk file keeperThe drafters add six cited sentences: 0.92, 0.92, 0.90 and 0.88 by age band, 0.91 for women and men, and 0.89 in Poland to 0.93 in Ireland by country. Section 3 reaches 100%, the file 76%, and risk R-07, lower accuracy for applicants aged 55 and over, moves to mitigated. Sign-off stays locked: two questions are still open.
“Section 3 complete · Q-0417-03 closed · R-07 mitigated”
- 08The owner decides
He closes the last questions and accepts residual risk
Tomasz Weller · AI System OwnerTomasz answers Q-0417-08 with the override logging specification OLS-0417-1: every override records the recruiter, one of 6 reason codes and the time. For Q-0417-07 the drafter builds a post-market monitoring plan only from processes Kestrel already runs — a monthly sample of 2,000 shortlists, weekly drift, complaints — and he approves it. Then he accepts residual risk for R-07, R-09 and R-12 with reason code A1.
“Signing as Tomasz Weller … prepared by the risk file keeper, not by you.”
- 09Signed
She signs the declaration; a person registers it
Helena Marsh · AI Compliance LeadWith every question closed, the declaration DoC-0417-01 unlocks. Helena signs with reason code D1, “Conforms: file complete, every question closed”, and the declaration preparer writes section 8 from the signed document. The EU database record — Annex VIII Section A, 11 fields — is prepared for her; she submits it herself and enters the number, EU-HR-2026-004127.
The declaration preparer never signs and never submits to the EU database.
- 10Kept current
A retrain is checked against the change plan
Change monitorOn 6 Oct the model was retrained as v3.3. The change monitor checks it against the pre-determined change plan PCP-0417-02: architecture, features and data window unchanged, overall accuracy 0.91, but applicants aged 55 and over at 0.85, under the 0.87 floor. Promotion is held and v3.2 stays in service. If promoted, sections 2(d), 3 and 6 would re-open.
“Whether this is a substantial modification (Art. 43(4)) is for the system owner to decide.”