This past week I got to have a lot of conversations with customers, industry partners, organizational leaders, and the like. And a main theme kept coming up, specifically that the landscape for these kinds of solutions have become increasingly crowded and competitive. It doesn’t matter where you turn, you see a new recommended architecture for implementing a RAG-based generative chat solution. And that’s just the custom solution, but the market for SaaS offerings has become equally crowded.
Which then leads to the question of “How do you differentiate?” or if you are building one of these platforms, “How do you stay competitive?”
These questions are both feel incredibly basic, but are also surprisingly hard to answer. If you asked 20 different decision makers or technologist, you are likely to get 20 different answers. Ranging from things like measuring model efficiency, to a whole host of quality metrics on the models. And I’m not putting those down, they are important, but I would argue that given the availability of these models, through services like AI Foundry, that the key strategic differentiator for any Generative AI solution being developed today, comes down to three strategic goals:
Access / Integration with your Data Estate.
Making Access and Use as Close to the Work as Possible.
Bi-directional Interaction with that Data Estate.
Now again, I’m not saying that the other metrics aren’t important. They really are. But if you architecture your solution correctly, the models should become interchangeable to support innovation in the future. If we aren’t solely looking at model performance, what should we focus on.
Access / Integration with the Data Estate is absolutely imperative to the success of any AI platform. The only reason anyone won’t just go to chat-gpt’s public website, is for it to be productive, the AI tooling whether a generative chat, or agentic solution needs to be able to ground itself in the data it needs to do the job. And some people would read this and say, “Kevin I know what RAG is, old news.” But really, I’m talking about broad cross-domain data access from a single-entry point. I want to go and ask CoPilot questions and know that it’s going to pull and aggregate across data silos to build it’s responses. People want to go to one-place and get a complete picture.
The next element that I would argue is table-stakes, is putting the AI as close to the work as possible. Github CoPilot is a great example of this, the AI tools are embedded directly into the IDE, making it part of the user’s existing workflow. Users have become extremely jaded of having to go to different sites and places for things. So if you have an application, embed the capabilities directly with the current application that user’s already know.
Finally, bi-directional functionality with the Data Estate is absolutely critical. A lot of these RAG solutions allow me to do things like “Write an email,” or “Create a presentation,” but if you are building a solution to really deliver on the promises of AI, it’s absolutely critical, that we enable and empower users to be able to ask the AI to get data from across the different silos in the organization, but the real interesting power comes from being able to take action from the same exact interface. Take the simple idea of a workflow, but how great would it be for a user to be able to say “What items are awaiting my approval”, then ask clarifying questions, and then respond with “Please mark #7 approved.” And having the agent take action based upon their ask.
Now obviously that last one requires a lot of control, and governance, but that bi-direction interaction with data is absolutely critical to any AI solution being adopted and truly delivering on the promises of AI.