The workflow goes: extract the real job criteria from the job description, redact identity signals from each resume, score every candidate against those criteria, and keep an audit trail recruiters and legal can review later. Each step exists for a reason. Naive ranking — paste resumes and the job description into an LLM (the technology behind ChatGPT, Claude, and Gemini), ask for a list — produces results that correlate uncomfortably well with proxies for race, gender, school prestige, and age.
Regulators have noticed. NYC’s Local Law 144, the EEOC’s automated-hiring-tools guidance, and similar rules in other places now require bias audits, disclosure, and documented criteria for automated screening tools.
This piece walks through the pipeline end to end: how to extract and weight criteria, how to redact protected-characteristic proxies, how the scoring works, what the audit trail captures, and how the periodic bias audit catches drift. The goal is a screen that holds up to scrutiny — not just one that’s faster.
Where this fits — and where it doesn't
Use this if you screen 100+ resumes per hiring cycle, your recruiting team is currently spending substantial time on the first-pass screen, and you have a legal and HR team willing to engage with the bias-audit requirements. Common fits: companies hiring at scale, talent acquisition operations supporting 50+ hires per year, recruiting agencies running pipelines for clients.
Don’t use this if your hiring volume doesn’t justify the system (under ~50 hires per year — manual screen by trained recruiters is more reliable), your legal team hasn’t signed off on automated screening for your jurisdictions (NYC, Illinois, several states have specific requirements; check before building), or your roles are so specialised that the criteria-based approach can’t capture what matters (some senior technical roles, executive searches). For roles where the screening signal is largely portfolio review or technical interview, the resume screen has limited leverage.
What you'll need before starting
- ATS access — Greenhouse, Lever, Workday, Ashby, or whatever your team uses. The pipeline integrates with the ATS rather than replacing it.
- A bias-audit framework — either internal (HR + legal) or third-party (compliance vendors offer this). Required in some jurisdictions; good practice everywhere.
- Documented job criteria for each role. Most JDs aren’t this — they’re aspirational lists. You’ll need to convert them to a structured form: required vs preferred skills, years-of-experience ranges, location requirements, certifications.
- A model API key. Cheap-tier models suffice; this is structured extraction, not heavy reasoning.
- Buy-in from recruiting leadership and legal. The pipeline is a decision-support tool, not a decision-maker; the cultural framing matters.
Six steps to a screen that holds up to audit
- Convert the JD to structured criteria before any candidate is seen
For each role, extract a structured criteria object from the JD: required skills (with proficiency level), preferred skills, years-of-experience range, location/remote requirements, required certifications, education requirements (if legally and operationally relevant). Lock the criteria before the candidate flow begins. The criteria are the screen’s spec; changing them mid-cycle creates inconsistency and undermines the audit trail. Hiring managers approve the criteria; recruiting executes against them.
- Redact protected-characteristic proxies from resume input
Before scoring, redact: candidate name (initials only); college / university name (replace with degree-and-major-only); graduation year; address (replace with city / region only); photo (remove entirely); employer logos / icons; pronouns. The redaction reduces the risk that the model is keying on identity-correlated signals rather than job-relevant content. Note that this isn’t bulletproof — names persist in some fields, school names can leak through extracurriculars — but it removes the most direct correlations.
- Score against criteria with structured output, citing source spans
For each redacted resume, ask the model to score against each criterion separately, with the verbatim resume quote supporting each score. Structured output: criterion name, score (0–1 or 1–5), supporting quote, and a one-sentence rationale. Per-criterion scoring beats overall scoring because it produces a defensible decomposition — you can show why someone scored high on one dimension and low on another, rather than producing a black-box ranking. The supporting quote is the audit anchor; without it, the screen’s reasoning is opaque.
- Combine scores deterministically — not via another LLM call
Apply the documented weights from the criteria (required skills weighted higher than preferred, must-have certifications as hard filters) to produce an overall score. The combination is arithmetic, not judgement: weighted sum, optionally with hard-filters that drop candidates missing must-haves. Deterministic combination produces consistent, auditable rankings; LLM-based combination introduces drift and makes the screen harder to explain to a compliance reviewer.
- Route by score band — review, interview, screen out
Three bands: (a) top scorers → fast-track to recruiter screen and interview pipeline; (b) middle band → recruiter review of resume + AI rationale, with human override; (c) bottom band → screened out, with the structured rationale stored. The middle band is the most important — the AI is most likely to be wrong on borderline cases, and recruiter review catches what the AI missed. Don’t auto-reject without recruiter eyes on the bottom band initially; once you’ve tuned the system, you can lower the recruiter-review threshold for screen-outs.
- Run a bias audit on outcomes — quarterly, with legal review
Quarterly, audit the screen’s outcomes for adverse impact: does the candidate pool that advanced to interview look representative of the candidate pool that applied, across protected categories (where you have voluntary self-ID data)? If the screen is producing adverse impact, investigate which criteria are driving it — sometimes a “required skill” is a proxy for something protected (e.g., requiring 10+ years of experience for a role that doesn’t need it can disproportionately filter younger candidates). NYC LL144 requires this audit; doing it everywhere is good practice.
What it costs and what to expect
The hours-saved-per-recruiter is the operational ROI. The audit-pass result is the compliance ROI — and the avoided legal exposure is the strategic one for any company operating in a regulated jurisdiction.
Other ways to solve this
ATS-bundled AI screening (Greenhouse AI, Workday Skills Cloud, Ashby AI). Increasingly bundled with major ATS platforms. Right answer if your ATS provides it and you’re comfortable with their bias-audit posture. Trade-off: less control over criteria and scoring; varying levels of transparency on bias mitigation; dependency on the platform’s compliance work.
Specialised AI screening vendors (HireVue, Phenom, Eightfold). Dedicated platforms with structured screening workflows and bias-audit support. Higher cost than ATS-bundled; more rigorous bias-audit features. Strong fit for high-volume recruiting operations.
Skill-based assessment instead of resume screening. Some teams replace the resume-screen entirely with a skill-based pre-assessment (TestGorilla, Karat, structured technical-skills test). Demonstrably less biased than resume screening for many roles; doesn’t scale to senior or specialised roles where the assessment design is hard.
Recruiter-only screening. The traditional approach — trained recruiters review every resume. Highest fidelity per resume; doesn’t scale past a certain volume. The AI screen is the second-pass-becomes-first-pass evolution; the recruiter’s time shifts to the borderline cases and the interview pipeline.
Related work
For the document-extraction pattern that powers the resume parsing, see Extract structured data from PDFs. For the broader risk-and-compliance framework when AI handles hiring decisions, see AI risk assessment for legal and compliance teams. For the onboarding workflow that follows once a hire is made, see Onboarding documentation generation for new hires. For the broader pattern of AI in high-stakes decision support, see AI hallucinations explained.
FAQ
Is automated resume screening legal?
Yes, broadly — with jurisdiction-specific requirements. NYC's Local Law 144 requires a bias audit and candidate notification for automated employment decision tools used on residents. Illinois requires consent for AI-analysed video interviews. The EEOC has issued guidance treating automated tools the same as any other selection procedure under Title VII. The base rule everywhere: automated tools can't produce adverse impact without business necessity. Talk to employment counsel before deploying in any jurisdiction with specific automated-hiring regulations.
Should we tell candidates that AI is being used?
In some jurisdictions, yes — required by law. NYC LL144 mandates notice. Several other jurisdictions are following. Even where not required, disclosure is good practice and supports trust. Most major ATS platforms now ship a standard notice template; use it. Disclosure framing matters — "we use AI as a first-pass review, recruiters make all final decisions" lands differently than "AI screens your resume."
What about candidates who try to game the screen with keyword stuffing?
Common, and the screen should be robust to it. Two patterns. (1) Score on context, not keyword count — "Python" mentioned in a real project description scores higher than "Python" listed in a skills section. (2) Validate against career narrative — does the experience timeline support the claimed skill level? Keyword-stuffing typically produces internal inconsistencies the structured-extraction approach can catch. The fully gameable screen has been the criticism of ATS-keyword-match systems for decades; the criteria-based approach is more resistant but not immune.
How do we handle candidates with non-traditional backgrounds?
The bias audit is where this surfaces. Non-traditional backgrounds (career switchers, bootcamp grads, international degrees, employment gaps) often correlate with protected categories or with high-potential candidates the team would want to consider. If the screen is consistently filtering these out, the criteria themselves may need adjustment — "X years of experience in role Y" can be too narrow; "demonstrated experience in skill Z" is broader and admits non-traditional paths. The structured criteria are the lever; tune them based on what the team actually wants.
What about senior or executive roles where resume signal is weaker?
Less leverage. Senior roles are typically screened on portfolio review, references, and structured interviews — the resume is necessary but not dispositive. Use the screen for the lightweight pre-filter (location, basic qualifications), not for the meaningful evaluation. For senior roles, the recruiter-led process remains the primary path; the AI screen is a small accelerator at the top of the funnel.
How often should we re-run the bias audit?
Quarterly minimum; immediate after any material change to the criteria, the model, or the candidate pool composition. NYC LL144 requires annual audit; quarterly is the operational best practice because issues caught at three months are easier to fix than issues caught at twelve. Build the audit cadence into the system from launch; retrofitting audit infrastructure is harder than building it in.