AI Integration
Decision Matrix
Stop shipping AI features on gut feel. Score every decision before you build.
Built for engineering leaders managing teams in the age of AI acceleration. 16 years of build decisions distilled into one framework. Four options. Five scoring criteria. Hard gates for when the answer is obvious. And kill criteria for when you need to stop mid-build and pivot.
The Four Options
You write the AI capability from scratch or fine-tune a base model.
// When it's right
- Proprietary data advantage that competitors cannot replicate
- The AI capability IS the product — not a feature of it
- Regulatory or compliance requirements mandate on-prem model control
- Your team already has ML expertise and capacity
Hidden cost
Maintenance burden is permanent, not one-time. Models drift, data pipelines rot, and your on-call rotation now owns it forever.
You license a third-party AI product or SaaS solution.
// When it's right
- A point solution exists with >80% of what you need today
- This feature is table stakes, not a differentiator
- Speed to ship is the binding constraint
- Pre-product/market-fit — you need to learn before you build
Hidden cost
Vendor lock-in, pricing risk at scale, and a hard ceiling on customization. You own the configuration, not the capability.
You deliberately delay the decision with a defined review date.
// When it's right
- The space is moving so fast that today's winner may not exist in 6 months
- No clear leader has emerged and the capability gap is tolerable
- Your team has zero AI capability for a bet-the-company decision
- You know a major release is 6–12 months out from a credible source
Hidden cost
Competitive lag compounds fast. The discipline is knowing exactly when to stop waiting — set a hard date, not an open-ended pause.
You call a model API (OpenAI, Anthropic, Gemini) with custom prompting or orchestration.
// When it's right
- The LLM is the commodity layer — your differentiation is product, UX, or domain logic
- You need to ship something now and want future model portability
- None of the Build/Buy/Wait hard gates apply
- Orchestration and context management is your actual engineering problem
Hidden cost
Latency, variable cost at scale, and dependency on model provider stability. A pricing change or model deprecation is now a production incident.
The Scoring Matrix
Strategic Differentiation
Time to Ship
Data / IP Sensitivity
Team AI Capability Required
Cost at Scale
Decision Rules — Hard Gates
These are not guidelines. If any of these conditions are true, the decision is already made.
ALWAYS BUILD IF
- 1Your training data is proprietary and irreplicable
- 2The AI capability IS the product — not a feature of the product
- 3Regulatory or compliance requirements demand on-prem model control
ALWAYS BUY IF
- 1A point solution exists with >80% of what you need
- 2This feature is table stakes — not a differentiator
- 3You're pre-product/market-fit and speed is the variable that matters most
ALWAYS WAIT IF
- 1The best-in-class solution is 6–12 months from release and you have a credible source
- 2Your team has zero AI capability and this is a bet-the-company decision
- 3The space is consolidating fast — early mover here means early re-builder
DEFAULT TO WRAP IF
- 1None of the above hard gates apply
- 2You need to ship now and want future model flexibility
- 3Your differentiation is orchestration, UX, or domain logic — not the model itself
Kill Criteria
When to stop mid-build and pivot. The sunk cost fallacy kills more AI projects than bad engineering.
3 months in, the open-source model you chose has been superseded by a commodity API
The Buy tool's pricing at scale now exceeds what Build would cost
Data sensitivity concern emerges post-integration
Team attrition removes the ML expertise your Build depends on
The Honest Scorecard
5 questions. Answer honestly. Let the answers do the work.
Is this AI capability core to our competitive moat?
Does our data advantage survive if a competitor licenses the same Buy tool?
Can we ship real value to users in <30 days with a Wrap approach?
Has this space had a major new entrant or capability jump in the last 6 months?
Do we have engineers who can fully own this without burning out?
If you answer 3 or more questions with the same direction, that's your answer. If the scorecard produces a tie, default to Wrap — it preserves the most optionality.
jeremyknox.ai
Knox writes weekly on AI strategy, engineering leadership, and the intersection of military doctrine and technology. Signal over noise — every time.