Skip to main content

Field note · AI vendors

Before You Sign the AI Proposal, Read It for What’s Missing

A sound AI proposal describes your work: the problem, the exceptions, the data, the review step, what done means, and who owns it after the builder leaves.

tl;dr

A sound AI proposal describes your work rather than the tool. It states the problem and the success measure, the workflow and its exceptions, what the system reads and writes, who reviews output and what they can do, what done means, and who owns everything at handover. Where it’s silent, ask. The gaps tell you what you’ll be left owning.

Allison MarxPublished 8 min read

Allison Marx at work.

There’s a proposal in your inbox. An agency, a freelancer, or a software vendor has offered to build or set up something with AI. It has a price, a timeline, a list of features, and a few tool names you half recognize.

It reads well. Proposals are written to. And you’re the one deciding this week whether it’s sound, without a technical team to hand it to.

Much of what decides whether an AI build works after launch is something a non-technical owner can check on the page: the problem, the workflow, the data, the review step, the definition of done, and who owns the result when the builder leaves.

A sound proposal describes your work. A weak one describes the tool. The standard I’d hold it to: if a vendor can’t explain how the system will be evaluated, maintained, corrected, and retired when conditions change, the proposal is incomplete however impressive the demo was.

The Demo Got You This Far

A demo shows what a tool can do under prepared conditions. It doesn’t show what the workflow will do with your definitions, exceptions, data quality, permissions, and users.

The proposal is where that gap should close. Look for an evaluation plan: realistic test cases drawn from your own work, known failure conditions, and the conditions that require human review or a pause. If the only evidence on offer is the demo, you’re being asked to rely on a confident output rather than proof.

Perfect prediction was never on offer. You’re looking for a process that shows where the system fails and whether your team can catch it.

Scale the Scrutiny to What the Tool Touches

Price is a poor guide to how hard to read. What matters is what the tool touches. Raise the level of scrutiny when the data is sensitive, the output carries consequence for a person, a vendor receives your information, an action is hard to reverse, or an error would be hard to catch.

Still, a subscription and a custom build ask for different amounts of paper.

For a $50-a-Seat Tool

  • Problem and success measure. One written sentence on what it’s for and how you’ll know.
  • Data. Whether your data trains a shared model, where it’s stored, and whether you can keep specific data out.
  • Review. A bounded, low-risk drafting aid can run on sampling and feedback.
  • Ownership. Who administers the seats, and who’d notice if people stopped using it.
  • Running cost. Seats, training, and the hours people spend checking output.
  • Exit. Whether you can export what people put into it.

For a $20,000 Build

  • Problem and success measure. A decision statement, acceptance criteria, and a named person who signs off.
  • Data. Everything a seat-based tool needs, plus written access rules for every system it reads from or writes to.
  • Review. A named reviewer, an exception route, and a stated answer for what happens if review doesn’t occur.
  • Ownership. Whose accounts it runs in, who holds the credentials, who has the documentation.
  • Running cost. Model and platform usage, monitoring, maintenance, and changes after launch.
  • Exit. What leaves with the builder or the vendor if you stop in eighteen months.

The data row doesn’t shrink with the price. A cheap tool that receives customer or employee information deserves the same questions as an expensive one. The rest of this note is written for the right-hand column.

It Should Describe Your Work, Not the Tool

Does the proposal name a specific task or decision that’s slow, repetitive, or inconsistent today, the people who do it, and what should improve? Does it say how that improvement will be observed? A proposal that lists capabilities without a problem is selling a capability, and the pilot it produces will be a demo with no viable success condition.

Then look for your workflow as it runs today: the inputs, the handoffs, the downstream actions, and above all the exceptions. Exceptions need a route, not a dropped record. A sound proposal says where unmatched, uncertain, or failed items go, and to whom, with the original attached.

If it’s missing, the builder is quoting against a workflow nobody has drawn. That’s a design step, and it belongs before the build quote. On my own builds an approved workflow design is the first precondition, whether it came from me, your team, or another builder.

It Should Say What the System Reads, Writes, and Sends Elsewhere

A sound proposal names each system the build connects to, confirms each one has a supported, documented way in, and states what the automation may read and what it may write. Access rules belong in the specification before anything is connected, with only the access the work requires. A custom connector for a system with no supported interface is separate work. Ask whether it’s priced.

Look for data quality too. A proposal that’s silent on the state of your data is assuming it’s fine. Ask whether cleanup is in scope, and whether anything checks the data before go-live.

And when information leaves your systems, the data decision stays yours. A contract allocates obligations. It can’t replace your understanding of where the data goes. Ask:

  • Will our data train a shared model or improve the service for other customers?
  • Where is it processed and stored, and which subprocessors receive it?
  • What controls exist for access, retention, deletion, and export?
  • Can we prevent specific data classes from entering the workflow at all?

If nobody on your side can explain what leaves your systems and where it goes next, the proposal isn’t ready to sign.

It Should Give the Reviewer a Job

Almost every proposal says there’s a human in the loop. The phrase means something only when that person can affect the result.

A sound proposal describes the review step in working detail: the reviewer’s role and backup, what they see, the criteria they’re checking, the expected volume and time per item, and the actions available to them (approve, edit, reject, pause). It names the escalation route. And it states what happens if review doesn’t occur, rather than leaving that to a product default.

It also asks something of you: a named owner with time to check output during testing, sign off at acceptance, and run the workflow afterward. On my builds that’s about an hour a week while the work is under way. If the proposal asks nothing of your team, ask who’ll be checking output while it’s being built.

Be wary of a promise that the model won’t make things up. It can, and I won’t promise otherwise on my own builds. What a builder can commit to is a design where unsupported output is detected, constrained, and routed to a person, with acceptance tests that check the routing works.

If it’s missing, you’re looking at one of the clearest reasons to pause: a vendor promising autonomous outcomes without showing how people will review, correct, or stop the system. You Put a Human in the Loop. Can They Say No? lists what the review step needs in writing.

It Should Define Done and Say What’s Left Out

“Done” needs acceptance criteria a builder can test against, an agreed test plan, and a named person on your side who runs the checklist on realistic inputs and either signs off or sends it back with specifics. Look at what the payments are tied to. On my builds, the final portion is billed at accepted handoff.

Then read the exclusions. A sound proposal has a list of what’s left out. When there’s no list, the disagreement about what’s included happens later. Check:

  1. What the base price covers: how many workflows, how many systems, how many environments
  2. Whether testing, documentation, and training sit inside the price
  3. What counts as a defect, for how long it’s fixed at no charge, and what counts as new scope (a new feature, a changed requirement, a system connected afterward)
  4. When the clock starts. Procurement, security review, and account provisioning can all sit before it, and delays there move the schedule.

It Should Leave You Owning It

Ask where the thing will live. The answer you want is your accounts, on your licenses, documented for your owner, with nothing depending on the builder staying involved. Look for an operating guide (what to check, how often, what “wrong” looks like, who to call) and training for the people who’ll run it.

Then the money after launch. License cost is the visible part. Ask who pays for model and platform usage. On my builds it’s billed to your accounts, and I estimate it from testing volume before handoff. Add monitoring, maintenance, and the work that appears when a vendor changes an API. Code you didn’t write and don’t understand is technical debt regardless of what produced it, so ask who the named technical owner will be.

And exit. If you removed this in eighteen months, what would leave with it? If the definitions, history, or business rules live only inside the vendor’s product, set export and retirement requirements before dependence deepens. The Integration Tax Just Got Cheaper has the full set of questions.

What the Proposal Says, and What to Ask

Each line starts with a phrase you’ll find in the proposal, then the questions to put back to whoever wrote it.

  • “Fully automated” or “no manual work.” How do people review, correct, or stop it? What happens to the items it can’t handle?
  • “Human in the loop.” Who, by role? What are they looking for, and what can they do when they find a problem?
  • “Integrates with your systems.” Which systems, through which supported interface, with permission to read or write what? Is any custom connector priced?
  • “Learns from your data.” Which data, collected under what terms? Does it train a shared model? Is there a data-readiness check before go-live?
  • Accuracy figures from the demo. Measured on whose data? Can we test it on our own realistic cases, including the exceptions?
  • “Live in a few weeks.” Does the clock start before or after access, security review, and account provisioning?
  • “Support included.” What’s a defect and what’s new scope? For how long? Who monitors it after that?
  • A license fee and a build fee. What will usage cost, who pays it, and what’s the estimate from testing? Who maintains it?
  • “Hosted on our platform.” If we stopped in eighteen months, what leaves with you? What can we export?

Get a Second Read Before You Sign

Gaps in a proposal aren’t proof of bad faith. Treat them as questions meant to strengthen the work: is this coming in a later phase, or has nobody decided yet? Send them to the builder. Some answers will be “not yet.” “Not yet” saves teams from expensive detours disguised as roadmaps.

If you’d like someone independent to look at it with you, bring it to a Working Session. We read it together on a 90-minute call, and you leave with a named constraint and a first step with an owner. There’s no document review before or after the call, so bring the person who’ll run the workflow. If proposals and vendor claims keep arriving faster than your team can evaluate them, Ongoing AI Advisory is a standing second read: written comments on the material and a decision log that stays with you. I’m independent of vendor incentives, so the read doesn’t depend on which tool you choose.

The proposal tells you what you’re buying. The gaps tell you what you’ll own.

Inspired? Take the next step that’s right for you.

Occasional mail

New notes, when there's one worth sending.

No schedule and no roundup filler. One email when a note is published. Unsubscribe from any message.

Name and email

Used only to send new notes. See the privacy page for how it's stored.