Hire or Fire · video notes
The AI agent interview card
Don't hire AI agents yet. Interview them. Here's the free card from the video: three questions, then one rule for what to do with the answers.
Interview card
- How long before it's a coin flip? Can it do this job reliably? Look for what the vendor publishes about reliability on tasks like yours.
- What do you do without asking? Does it ask before it sends, buys, deletes or submits? Is that the default, or a setting?
- What's your cap? Find the limits page. Know how much your plan allows and what happens at the cap.
- Hire
- for short, checkable, undoable tasks.
- Probation
- for anything that sends, buys or deletes. Approval stays on.
- Fire
- if it can't tell you its limits.
Our verdict: hire it like a temp, not a manager.
Our rule. The HIRE / PROBATION / FIRE rule is our opinion, built on the sources below. It is not a vendor or industry standard.
Download the interview card (.md)
Why these three questions
1. The coin flip
METR, an AI evaluation group, scores agents by “time horizon”: the length of task (in human expert time) an agent completes with 50% success. METR itself warns that a 50% horizon of X hours “does not mean we can delegate tasks under X hours to AIs”, and that reliability-critical work needs “98%+ success probabilities”.
2. What it does without asking
Defaults differ. Google says Gemini Spark is “designed to ask you first before performing high-stakes actions like spending money or sending emails.” Claude in Chrome and OpenAI's Dots let an automatic check decide which actions need you. And no vendor says the main attack on agents is solved: OpenAI says prompt injection is “unlikely to ever be fully ‘solved’”, and Anthropic says “No browser agent is immune to prompt injection.”
3. The cap
Agent plans have usage limits, and they change often. Google's limits page, for example, describes compute-based limits and shows yours in Gemini under Settings → Usage Limits. Check your own plan's limits page rather than a blog post.
Our doc check
Our doc check, 10 Oct 2026. We read each vendor's public help and announcement pages again on 10 Oct 2026; this replaces our 8 Oct check. This is what the docs say, not how the products behave: no agent was run, three products, one reviewer.
| Question | Gemini Spark (Google) | Claude in Chrome (Anthropic) | Dots (OpenAI) |
|---|---|---|---|
| Publishes how reliable it is on tasks like yours? | Not published; warns “Gemini can make mistakes and do unexpected things” | Partly: publishes prompt-injection test results, but no task-success rate on the pages we checked | Not published on the pages we checked |
| Asks before consequential actions by default? | Designed to ask first: “Gemini is designed to ask for your review and confirmation” before sending communications, modifying your data, making purchases and submitting web forms (exception: bulk actions on private Google Tasks) | An automatic check decides: a safety classifier approves actions it judges safe; you can switch to “Manually approve”. Purchases and permanent deletions are prohibited in every mode | An automatic check decides: a review settles whether an action “can proceed, needs your approval, or includes a step you must do yourself”. Asking it to draft replies “doesn't give it permission to send them”; you can add an “Ask before taking action” rule |
| Says prompt injection is solved? | Not stated on the page we checked | No: “No browser agent is immune” | No: OpenAI (Dec 2025): “unlikely to ever be fully ‘solved’” |
Sources
Every fact on this page comes from these pages, opened on 8 October 2026. Things change: check the live page before you rely on it.
- METR, time horizons Last updated 8 Sep 2026 and no longer actively updated; read 10 Oct 2026.
- METR, “Measuring AI ability to complete long tasks” (Mar 2025)
- METR, time horizon limitations (22 Jan 2026)
- Google Gemini Help, Gemini Spark Checked 10 Oct 2026.
- Google Keyword blog, on Gemini Spark (19 May 2026) Checked 10 Oct 2026.
- Google Gemini Help, Gemini Apps limits & upgrades Checked 10 Oct 2026.
- Claude blog, “Claude in Chrome generally available” (26 Aug 2026) Checked 10 Oct 2026.
- Claude Help, Claude in Chrome permissions guide Checked 10 Oct 2026.
- ChatGPT docs, Dots Checked 10 Oct 2026.
- ChatGPT docs, Dots controls (how action review works) Checked 10 Oct 2026.
- OpenAI, “Hardening Atlas against prompt injection” (Dec 2025)
- Anthropic, prompt injection defenses (Nov 2025)
No sponsor and no affiliate links on this page. Vendors are named so you can check them yourself, not as recommendations.
Follow Hourwiz
- YouTube@hourwiz (opens in a new tab)
- Instagram@hourwiz.ai (opens in a new tab)
- Facebookhourwizhq (opens in a new tab)