The pilot worked, the model shipped, and the project was declared a success. Then everyone went back to their day jobs, and the AI kept running with no one watching it. This is where lean operators get hurt. Not in building AI, which they can do, but in the quiet stretch after launch, when a model that was right in March is slowly going wrong in September and nobody owns the job of noticing.
You don't need a data-science team to handle this. You need a small amount of the right discipline.
The part nobody budgets for
A demo is a moment. Production is a commitment that runs every day after. The work that decides whether AI keeps paying back isn't the build; it's the operating. Large companies staff an MLOps function for exactly this. A lean operator can't, and the instinct is to skip it. Skipping it is how a model that worked becomes a model nobody trusts, which is worse than no model at all, because someone made a decision on it.
The three things that actually need watching
You can cover the essentials without a dedicated team if you watch three things:
- Drift. Conditions change, and a model trained on last year's reality slowly stops matching this year's. You don't need to watch it daily. You need a scheduled check and a threshold that triggers a second look.
- Cost. Token and compute costs scale with usage in ways that surprise people. A simple monthly view of what the system costs against what it returns keeps it honest.
- An owner. Someone has to be the named person who answers when the AI is wrong. Not a committee. One person who knows the system and has the authority to pause it. A model with no owner is an incident waiting to happen.
That's the whole minimum: drift, cost, owner. Everything else is refinement.
A lightweight operating model
The realistic version for a small team is a recurring half-day, not a standing function. Once a month, the owner checks the drift indicator, reviews the cost-versus-value view, and confirms nothing in the operating environment has shifted enough to retrain. Most months it's quiet. The discipline is in doing it before there's a problem, not after.
When to keep it in-house, and when not to
Keep it in-house when the system is stable, the owner has the time, and the stakes of being wrong are modest. Bring in an outside hand when the system touches something consequential, when the one person who understands it is also the person you can't afford to lose to it, or when the honest answer is that nobody actually has the monthly half-day. Often an outside operating arrangement is simply cheaper than the hire it would otherwise take, and it doesn't tie up a person you need on the asset.
The operators who get lasting value from AI aren't the ones with the biggest teams. They're the ones who decided, before launch, who would watch the thing once everyone else moved on.