HackerRank open-sourced an AI hiring agent: what to know

In October 2025, HackerRank open-sourced an AI resume screener on GitHub. It is the MIT-licensed `interviewstreet/hiring-agent` repo. For the first time, anyone can read how a major ATS vendor scores candidates. They can also see its flaws. This article explains what the tool is, why its results are unstable, and what it means for both employers and job-seekers.

What HackerRank open-sourced

The Hiring Agent is a Python pipeline. It turns a resume PDF into a scored, explained evaluation. It runs in five stages:

1. Extract the PDF to text (using PyMuPDF).
2. Use a large language model (LLM) to parse each section into structured JSON data.
3. Enrich from GitHub. It fetches the candidate's repositories and picks the top ~7 projects.
4. Score with an LLM across four categories: open-source, self-projects, production experience, and technical skills. Each score cites evidence.
5. Export results to CSV for tracking.

One detail stands out. The tool can run entirely locally with Ollama (for example Gemma3). It can also run via Google Gemini in the cloud. The full scoring prompts live in a `prompts/` directory. So the evaluation logic is open and editable. That is rare for an ATS vendor.

The catch: the same resume scores 66 to 99

An independent analysis at danunparsed.com ran the same resume through about 100 evaluations. Scores ranged from 66 to 99. At an 85-point cutoff, that same candidate is rejected about 65% of the time. The cause is run-to-run randomness, not merit.

A screener that cannot differentiate is not filtering for quality. It is just filtering.— danunparsed.com

This is not unique to HackerRank. Non-determinism is a property of how LLMs generate text. The analysis found one more flaw. The 25-point "experience" component barely varied between runs. It is consistent, but its guidance is too thin to tell candidates apart.

Why GitHub weighting skews results

Roughly 65% of the score rides on open-source and GitHub project work. That structurally undervalues many strong engineers. These are people without much public code. The situation is common at large companies, in security, or in non-coding roles. If the model cannot "see" your work, it cannot score it.

What the data says about AI in hiring

AI screening is now widespread. But candidate trust has not caught up. A few notable figures:

• 69% of HR professionals used AI to support recruiting in 2025, and 44% used it to screen resumes, per SHRM 2025 Talent Trends.
• 71% of Americans oppose letting AI make final hiring decisions, and only 7% favor it, per Pew Research.
• About two-thirds of adults say they would not apply to an employer that uses AI to help make hiring decisions, also per Pew Research.
• Academic testing (the FAIRE paper, arXiv 2025) documents measurable racial and gender bias in LLM resume screeners.

The legal backdrop: bias audits and the EU AI Act

Regulation is tightening. From 2 August 2026, AI tools used for recruiting, screening, or performance evaluation are classified high-risk under the EU AI Act. Such tools must run risk assessments, test for bias, keep human oversight, and disclose their use. In the U.S., jurisdictions like New York (Local Law 144) already require bias audits and candidate notice. For Vietnamese employers, CV data is personal data under the PDPL (Law 91/2025). That is another reason to favor locally-run models and keep humans in the loop.

Bottom line

HackerRank's open-sourcing is a transparency win. For the first time anyone can read and critique how a real hiring tool scores. But "open" also exposes that the tool is unstable and skewed in its weighting. For employers, treat it as a starting point to *audit*, not a product to ship as-is. For job-seekers, remember that randomness sits behind many rejections. So keep going.

*Written by the Tuyển Dụng team, based on the public source code and the independent analysis cited below.*