Who grades the AI?

Job boards are full of duplicates, stale listings and roles that look like a perfect skills match until you reach the location or work-authorisation requirement. Reading and applying takes real time. AI can make the problem worse when it confidently ranks a role it has no business recommending.

I wanted ApplyOS to make the first decisions properly: is the vacancy current, is it actually available to this candidate, and what does the evidence say about fit?

I separated eligibility from fit. Hard requirements are checked first. If the listing or candidate profile leaves something unresolved, ApplyOS shows the missing information instead of inventing an answer.

After that, structured job data feeds a repeatable scoring model. A candidate can inspect which requirements matched and why a role appeared. The same evidence discipline carries into application answers: drafts draw from recorded career facts and stay open to human review.

The candidate moves from a role-discovery feed to a more detailed eligibility explanation, then to application preparation and a final review. The product keeps facts, preferences and unanswered questions separate so each decision has somewhere to live in the interface.

The current product screens show role discovery, unresolved eligibility and the evidence behind an individual recommendation.

The early scoring approach gave the language model too much freedom to rank and explain a role. I replaced it with rules that can be inspected and calibrated against a labelled set.

I also rejected automatic submission based on a confidence score. An application goes out in someone's name. I want the candidate to see and approve what is being submitted.

I hand-labelled 171 jobs, including why I would pursue or reject each one. Against those labels, the first feed had 59% precision. A market-aware eligibility pass helped move it to 89%.

That exercise exposed a useful distinction: a job can match someone's skills perfectly and still be unavailable to them. The product has to make that visible early enough to save the candidate time.

The next challenge is handling missing information with less friction. The October operator snapshot showed 2,412 discovered roles across searches for 21 candidate profiles, while separate eligibility results showed how often additional information was needed before making a decision.

I've kept building ApplyOS around that lesson: the best automation is the part you can interrogate when it looks wrong.

Explore ApplyOS: apply-os-bice.vercel.app