Build an AI stack that can keep up with your business

· Rush Deshpande, ex-Head of IT at Vanta and Robinhood

Sponsored

Even the best AI model doesn't know how your business actually works, at least not yet. It doesn't know which exceptions matter, who can approve what, or what your company has committed to a customer. That knowledge still lives with your people, in your systems, and in the processes you've built over the years.

AI agents can bring tremendous scale and efficiency. But without that business context, even a great model can produce a very convincing wrong answer.

For example, a security review holding up a major enterprise deal.

The customer's security team asks whether multifactor authentication is enforced for every administrative access path into production. Your new AI security-review assistant checks the access policy and identity-provider configuration and drafts a confident "yes." Except it missed something important: a documented break-glass administrative account sitting outside the standard MFA enforcement policy.

Now security is investigating the exception, sales wants an answer, and the tool that was supposed to speed things up has almost sent a customer an assurance the company can't support.

I've seen versions of this in enterprise IT. Security reviews claiming "SSO everywhere" when several applications still had local logins. Access reviews described as quarterly when in actuality, they'd quietly slipped to twice a year. The answers weren't necessarily fabricated. They simply went further than the evidence supported.

Some models might catch these issues. But even a frontier model won't automatically know which evidence is authoritative, which exceptions are acceptable, or what the company is comfortable asserting.

Those decisions belong to the business and need to be reflected in the workflow.

I've worked in and led IT teams at Facebook, Square, Netflix, Uber, Robinhood, and Vanta, and now advise growing companies on building their own IT teams. Across those environments, the difficult work has rarely been just picking the technology. It's how that technology fits into existing systems, who owns it, who can access what, and what happens when the requirements change.

AI is no different. Build the AI stack around the business, not the business around the AI stack.

A better model won't fix a disconnected system

According to Dataiku's Global AI Confessions Report: CIO Edition 2026, 75% of global CIOs plan to use more or different AI models to ensure continuity, all the while 60% say they lack a central AI governance layer.

The research, conducted by The Harris Poll on behalf of Dataiku, surveyed 685 CIOs at companies with annual revenue exceeding US$500 million or the regional equivalent, across the US, UK, France, Germany, UAE, Japan, South Korea, and Singapore, from July 9-29, 2026.

Adding more models doesn't necessarily make an organization more capable.

Consider separate teams building a security-review assistant, an internal IT support workflow, and a document processing service. Each team selects a capable model, then builds its own integrations, permissions, evaluation process, and monitoring. Individually, those tools might work well. Across the company, you now have several disconnected systems with different dependencies, owners, and operating models. Sound familiar?

The same thing can happen inside a single workflow. Different models might handle classification, evidence analysis, and drafting, but without consistent handoffs and controls, changing any one component becomes harder than it should be.

And the next change may have nothing to do with AI. A new product launches. A customer requires a different deployment model. An acquisition brings additional systems. A compliance requirement changes how evidence is collected.

The business will continue to evolve, regardless of which models you've selected.

That's why the goal can't simply be easier model switching. The architecture needs to be able to accommodate changing business requirements just as readily as changing technology. Optionality matters, but it has to start with what the business needs.

Start with the business workflow

Go back to the security-review example. The customer needs an accurate response, and the security team needs evidence to support it. The relevant control owner needs to validate exceptions. And the business needs to get through the review without unnecessarily delaying the deal.

Those requirements should drive the architecture.

The workflow itself is straightforward to describe: understand the customer's question, identify the relevant product and environment, retrieve evidence, evaluate it, draft a response, and route it for approval.

Not every step needs an LLM. Access controls, document versions, approval status, and workflow state should stay in the systems responsible for them. Other steps may benefit from different models. A smaller model might be perfectly adequate for categorizing questions. Evaluating conflicting evidence may require more sophisticated reasoning. Drafting a customer-facing response has its own requirements.

Use one model where it works. Use different ones where there's a measurable benefit. There's no benefit for adding more models.

But the most important decisions aren't model decisions.

Which evidence is authoritative? Can this customer see that document? Who approves a statement about prod security? What happens when two sources contradict each other? The people who own the process need to answer those questions.

The infrastructure team understands how administrative access is configured, security understands the controls, and the assurance team knows what can be shared with the customers. Their knowledge needs to be built into the workflow, not handed to an AI team as a few documents and an instruction to follow company policy.

For example, the AI could identify a suggested answer, the supporting documents, and any missing or conflicting information. The next step uses that information to draft a response. If something doesn't add up, the question goes to the appropriate person for review.

In our MFA example, catching the exception and sending it for human review is exactly what the workflow should do.

The model interprets the information and the business determines what happens next.

The harness is where business knowledge becomes operational

A lot of attention goes to the models themselves, but the harness around them is just as important.

Think of the harness as the application-level setup that gives a model the context, tools, integrations, and instructions needed to accomplish a task. It determines what information the model receives, what it can do with that information, and how its output moves through the workflow.

For our security-review assistant, that means knowing where to retrieve evidence, what the requesting user is permitted to access, how to handle conflicting information, and when to involve a control owner.

The model might identify a problem or recommend an action. The harness and connected services determine how that recommendation is validated and what happens next. This becomes especially important with agentic AI, where models can use tools and initiate actions rather than just generate text.

An assistant that drafts a response is very different from one authorized to send it. That distinction should exist in the workflow's permissions, not just in a prompt asking the model to be careful.

This is where the business gets encoded into the system.

Security defines which exceptions require review. The assurance team establishes what evidence can be shared externally. IT controls the systems and service identities the assistant can use. The harness brings those requirements into the execution flow.

It also shouldn't become a collection of assumptions tied to one particular model. Prompts, tool calling, and context handling may differ between providers, but the business rules and expected outcomes should remain understandable.

That separation makes it easier to improve individual components as better models become available.

And importantly, the people who understand the business need a meaningful role in defining and improving the workflow. Otherwise, you're just automating someone else's interpretation of how the business works.

Own the business logic, rent the plumbing

One rule of thumb from hypergrowth IT: own what's strategic and differentiating, and use shared services for what's operational and repeatable.

For enterprise AI, what's strategic is your business knowledge and how you put it to work: the data, decisions, workflows, exceptions, and people responsible for them.

The model is important, but it's still part of the plumbing.

That doesn't mean all models are interchangeable. Different models have different strengths, costs, and limitations. The point is that the business shouldn't have to rebuild its operating process whenever a better option comes along.

The same thinking applies to the infrastructure around those models.

Every team building an AI assistant shouldn't need to independently solve model connectivity, credential management, usage tracking, and governance. That's how you end up with multiple tools doing the same job and nobody quite sure who owns them. The better approach is to share the operational capabilities while allowing teams to build workflows around their actual business requirements.

This is where Dataiku, The Platform for AI Success, can help. Its approach brings together three things that need to work together: people, orchestration, and governance.

People: Business experts should be directly involved in building and improving the workflows that reflect their knowledge. Dataiku provides visual workflows alongside code-based tools, allowing analysts, business users, and engineers to contribute in ways that fit their skills. IT still establishes the technical and security boundaries.

Orchestration: Models, data, tools, business rules, and human decisions need to work together as one process. In our example, the security-review workflow should continue to function whether the evidence-analysis model changes or a new source of evidence is introduced.

Governance: The organization needs to know what's running, who owns it, what it can access, how changes are approved, and whether the workflow is delivering the intended outcome.

It's the combination that matters.

Dataiku's LLM Mesh supports a part of that architecture, providing shared connectivity to different models with controlled access, query tracing, and cost monitoring. But model connectivity alone doesn't solve the business problem. The harness manages the application-specific behavior and handoffs. The coordination layer provides shared infrastructure, orchestration, and governance. The business owners determine the rules the process needs to follow.

A shared model connection doesn't decide who can read an internal incident report or whether an assistant can send a security response to a customer. The systems responsible for those permissions still need to enforce them.

Similarly, a new model becoming available doesn't automatically make it an approved destination for sensitive company information.

For compatible, approved models, changing the connection can become a configuration decision within an established release process. The surrounding workflow still needs testing, but it shouldn't require redesigning the business process.

That's the practical benefit of putting the model in its proper place.

Bring business experts closer to the technology

One of the more interesting opportunities with AI is giving people who understand a process more direct control over improving it.

A security leader knows what constitutes acceptable evidence. Finance knows the rules for approving a payment. IT operations know which employee requests can be automated and which require someone to investigate.

These are business requirements, not primarily modeling problems.

Traditionally, translating those requirements into software involved several handoffs between business owners, engineering, and IT. Things inevitably got lost or simplified along the way.

Visual workflows and AI-assisted development can reduce some of that translation work. Dataiku's approach is designed to let business experts contribute more directly to working with data, workflows, and AI, while technical teams retain the controls needed to deploy and operate them.

That doesn't mean everyone should be able to push an agent into production.

It means the person who understands the process shouldn't be limited to reviewing the finished product.

In our security-review example, a control owner should be able to identify missing evidence, define an exception that requires approval, and help test whether the workflow handles it correctly. IT and security still own the access controls, integrations, and production requirements.

As the business changes, those same people can help update the process without waiting for the underlying model to somehow understand the new requirement.

Governance should make the approved path easier

The CIO shouldn't be approving every prompt change or choosing the model for every application. The people closest to the problem need room to improve their workflows.

Leadership establishes the boundaries: which data can be accessed, where it can go, what actions require approval, who owns the process, and what evidence is needed before a change goes live.

Platform teams provide shared capabilities. Business and application owners build within those boundaries.

This is where governance becomes an accelerator for agentic AI.

Our security-review assistant might draft an answer, open a follow-up task, or route missing evidence to a control owner. Publishing an answer directly to a customer or changing a control's implementation status is a different level of authority.

Those permissions should be explicitly defined, regardless of which model is running.

Once these boundaries are established, teams can improve the workflow without reopening every foundational decision. Changes within approved data, tool, and action boundaries can follow a known process. New destinations or expanded permissions get the additional review they need.

Dataiku's governance capabilities support centralized visibility, ownership, approval workflows, lineage, and documentation.

But someone still needs to own the decisions. The control owner validates the security claim. The assurance owner approves the customer response. The platform team manages shared connections and infrastructure. A service owner remains accountable for the workflow.

The objective isn't to add more processes. It's to make the approved way of doing something the easiest way to do it correctly.

Evaluate the business outcome, not just the model

A better-looking answer doesn't necessarily mean a better workflow.

For security reviews, success means accurate answers you can stand behind, less reviewer effort, and reasonable turnaround time.

Build a test set with the people who actually approve the responses. Include straightforward questions and the difficult ones: conflicting evidence, expired documents, open exceptions, and questions that only apply to a particular product.

The MFA question from the opening goes in that set. A passing answer flags the break-glass exception. A failing one confidently says "yes."

Test the individual steps and the complete workflow.

Did classification identify the right control? Did retrieval find the exception? Did analysis recognize the conflict? Did the final draft preserve the qualification?

A better evidence-analysis model doesn't help if the drafting step removes the caveat. And a retrieval problem won't necessarily be fixed by replacing the model.

Compare candidates using the same questions and evidence snapshot, with configurations versioned. Dataiku's Evaluate LLM, part of the Advanced LLM Mesh add-on, supports evaluating and comparing pipeline outputs. Combine those results with expected answers reviewed by the business experts, plus tests of permissions and approval routing.

Track what matters: reviewer correction effort, unsupported claims, missed exceptions, processing time, and cost per approved response.

Separate processing time from time waiting for a control owner. If the bottleneck is someone's inbox, a faster model won't solve it.

Some failures should block a release, period. Disclosing restricted information or missing a known exception isn't an acceptable tradeoff for faster responses.

And keep the evaluation set current. New products, policies, and customer requirements should update it. Changes to models, prompts, or integrations should trigger relevant tests.

The question is whether the workflow still meets the business requirement, not whether the model performs well on a benchmark.

Make change part of normal operations

Once a candidate passes evaluation, treat the switch like any other production change.

Record the model connections, prompts, workflow configuration, evaluation results, and approvals. Start with a limited rollout, retain human review where required, and define the conditions for stopping or reversing the change.

Already approved customer responses should stay tied to the wording and evidence that were reviewed. A model update shouldn't silently regenerate them. Recovery also needs to cover what the workflow has already produced.

Pausing the process or returning to a previous approved configuration is one part. Identifying affected drafts and responses already sent, so the appropriate owners can determine whether corrections are needed, is another.

Rolling back a model doesn't retract a customer statement.

And none of this is free. Connections need maintenance, permissions need review, evaluation sets go stale, and someone has to investigate failures.

For a pilot, start small: approved evidence sources, defined inputs and outputs, versioned configuration, and human approval. For an existing implementation, map the dependencies before redesigning it. Find business rules buried in prompts, provider-specific integrations, evidence that can't be traced, and decisions that only exist in conversation history.

Separate the highest-risk dependencies first, then centralize what's genuinely shared.

Hold the coordination platform to the same standard. Understand what can be exported, which integrations remain reusable, and where platform-specific dependencies exist.

You won't eliminate every dependency. Make them visible and manageable.

Build for the next business requirement

Before calling an AI stack adaptable, ask the team to demonstrate a change in one meaningful workflow.

The exercise should answer five questions:

QuestionWhat to look for
Does the workflow reflect the business?Business owners have defined the requirements, exceptions, approvals, and acceptable outcomes.
Can models be selected by task?Clear responsibilities, defined interfaces, and evidence supporting the choices.
What stays consistent?Access controls, source tracking, business rules, and approval requirements.
Does the change help?Repeatable evaluations tied to accuracy, human effort, time, and cost.
Who owns it, and can we recover?Clear accountability, an approved release process, and tested recovery procedures.

The answer may be to keep the current model, replace one component, or introduce something new. Each can be the right decision.

Back to the enterprise deal. The business needs a security response it can stand behind, without unnecessary delays. As products, customers, and technology change, the workflow needs to keep delivering that result.

Nobody can predict which model will be the best choice a year from now, and a CIO doesn't need to. The job is to create an environment where people who understand the business can use the best available technology, within an architecture the organization can operate and govern.

Build the AI stack around the business, with enough structure to run it and enough flexibility to keep improving it.

Read Dataiku's Global AI Confessions Report: CIO Edition 2026 for more on how CIOs are approaching model choice, governance, and business outcomes.