Nine questions separate an AI search agent that will actually move your clients’ visibility from a chatbot wearing the label. Ask every vendor all nine.
The pattern in the answers matters more than any single feature: agents that do real work can show you evidence, limits, and an undo button. Agents that cannot will answer every question with a demo.
An AI search agent (also called an AI SEO agent) is software that does search work rather than reporting on it. That job description is exactly why evaluation matters more here than for any tool you have bought before: you are not choosing a dashboard, you are choosing something that will touch client sites, client data, and your agency’s reputation.
Full disclosure before the list: we build Serena, an AI search agent for SEO and AEO. These nine questions are the standards we built her against, which means two things: yes, she is designed to pass them, and no vendor including us should be graded on their own claims.
Ask the questions. Watch the demos squirm or not.
1. Is it grounded in a real methodology, or is it generic model output?
The rule: an agent is only as good as the expertise it was trained to apply.
Every AI agent sits on a large language model, and every large language model can produce fluent SEO advice. Fluent is not the bar.
The bar is whether the advice comes from a documented, tested body of practice with a point of view, or from the averaged consensus of the internet, which is exactly the content that AI platforms are learning to ignore.
Ask the vendor: what is your agent’s advice actually grounded in? Whose methodology?
How is it kept current when search changes?
Good answer sounds like: a named methodology, real practitioners behind it, a process for updating positions as platforms change.
Red flag: “it uses GPT-5” presented as the answer. That is a model, not a methodology.
2. Does it work from your data or from averages?
The rule: advice that does not cite your numbers is content, not consulting.
An agent recommending what “sites like yours” should do is a blog post with a chat interface. The whole point of hiring an agent is that it knows this client: this Search Console profile, this analytics data, this Business Profile, these rankings, these AI citations.
Ask: which of my data sources does the agent actually read, and can it quote my numbers back to me in its recommendations?
Red flag: recommendations that would read identically for any business in the niche.
3. Does it execute, or does it only recommend?
The rule: a tool shows you the work. An agent does the work.
Recommendations are cheap and agencies drown in them. The question that splits the market: after the diagnosis, does anything happen without a human copying tasks into a project manager?
If the answer is no, you are buying a smarter report, and it should be priced like one.
Ask: show me a change your agent made on a real site, end to end.
Red flag: “execution” that turns out to mean exporting a to-do list.
4. How do changes actually happen, and who owns them?
The rule: changes written into your site are yours. Changes injected by a script are rented.
Some agents apply changes through a JavaScript pixel or snippet layered over the site. It is fast and works on any CMS, but the changes live in the overlay, not the source.
Some AI platforms read JavaScript poorly or not at all, and if the snippet comes off, every change disappears with it. Other agents write changes into the site itself, where they persist, get crawled as HTML by everything, and belong to the client.
Ask: if we cancel this subscription tomorrow, do the changes survive?
Red flag: a pause before the answer.
5. Is every change verified, with evidence?
The rule: unverified execution is a liability with good marketing.
An agent that changes client sites at scale can break client sites at scale. The difference between those two outcomes is verification: after every change, does the agent confirm the change landed, confirm nothing else moved, and show you the before and after?
Ask: what does the agent do when a change fails verification?
Good answer: automatic revert plus a report, without being asked.
Red flag: “our changes do not fail.”
6. Can you roll anything back?
The rule: never let anything touch a client site that does not come with an undo button.
Every change, individually reversible, one click, and a record of who approved what and when. This is table stakes for letting software touch production websites, and it is astonishing how few vendors can answer it plainly.
Ask: show me the rollback. Not the slide about the rollback.
The rollback.
7. Can it exceed a budget you set?
The rule: an agent that can surprise you on a client site should never be able to surprise you on an invoice.
Usage-based pricing is fine and honest when it meters real work. The question is control: hard caps per client, spending visible at all times, and a guarantee that the agent stops before the cap, every time.
Agencies live on margins; a billing surprise is a client-relationship surprise.
Ask: what is the maximum this could cost me next month, and what enforces that number?
8. Does it use source-of-truth data or estimates?
The rule: never make decisions from a tool that estimates traffic from rankings.
Third-party traffic estimates are rankings multiplied by assumed click-through rates. They are useful for competitive intelligence and dangerous as a decision basis.
An agent making real changes should be grounded in Search Console, analytics, and actual AI citation scans, not modeled guesses.
Ask: when your agent says a page is declining, where does that number come from?
9. Are its capability claims honest?
The rule: the fastest way to evaluate an AI agent is to ask it what it cannot do.
Every agent in this market is early. Every honest vendor knows exactly where their scope ends: which platforms, which change types, which tasks still route to humans.
Ask the agent itself, in the demo: “what can you not do?” A grounded product answers with a list.
An overhyped one answers with a pitch.
This one question predicts everything else, because a vendor that overclaims capability will also overclaim verification, safety, and results.
The scorecard
| # | Question | Pass looks like |
|---|---|---|
| 1 | Grounded in real methodology? | Named practice, updated positions |
| 2 | Works from my data? | Quotes your numbers back |
| 3 | Executes or recommends? | Real change, end to end |
| 4 | Who owns the changes? | They survive cancellation |
| 5 | Verified with evidence? | Auto-revert on failure |
| 6 | Rollback? | One click, shown live |
| 7 | Budget control? | Hard caps, agent stops first |
| 8 | Source-of-truth data? | GSC/analytics, not estimates |
| 9 | Honest about limits? | Answers “what can’t you do” with a list |
No agent on the market today passes all nine at full scope, including ours. The vendors worth your money are the ones who tell you which ones they miss and when that changes.
The vendors to avoid are the ones who claim a clean sweep. For a side-by-side view of how current agents stack up, see our comparison of the best AI SEO agents.
Ask Serena all nine in a live demo
Serena is an AI search agent for SEO and AEO, built by Rankability on 15 years of agency-tested methodology. She was built to answer every question on this list — including question nine. Sign up and ask her yourself.
Included with every Rankability plan · No separate agent subscription
FAQ
How do I choose an AI SEO agent for my agency?
Decide which job you are hiring for first: monitoring, content, or execution. Then run the nine questions above, in a live demo, on your own client data.
The pattern in the answers tells you more than the feature grid. For a breakdown of the current options by category, see our roundup of the best AI SEO agents in 2026.
For a job-by-job map of what those agents can actually do — and cannot — see What can an AI SEO agent actually do?.
Are AI SEO agents safe to let loose on client sites?
Only with four things in place: approval before changes, verification after them, one-click rollback, and hard budget caps. If any of the four is missing, keep the agent in advisory mode and implement changes yourself.
What should an AI SEO agent cost?
The honest pricing pattern is a platform subscription plus metered charges for completed work, where checking and diagnostics are free and failed work is never billed. Be suspicious of per-seat pricing for an agent (agents are not seats) and of unlimited-execution claims (unlimited usually means unverified).
What is the difference between an AI SEO agent and ChatGPT?
ChatGPT gives you general answers from general knowledge. An AI search agent works from your client’s actual data, applies a specific methodology, and can execute and verify changes.
If you cancel ChatGPT, nothing about your client work changes. That difference is the product.
For a full task-by-task view of how agents compare to traditional SEO tools, see AI SEO agent vs AI SEO tools.
Hire the agent that passes the checklist
Serena is built on a documented methodology, works from your clients' own data, executes at the source level, verifies every change, and ships with an undo button. She is included in every Rankability plan.
Included with every Rankability plan · No separate agent subscription