Why Enterprise AI Fails: Fixing Dirty Data Before the Black Box | Joe Onisick & Keith Townsend
Transcript
However, that doesn't solve the problem of the black box. What is the black box doing to my data? And secondly, what happens when the black box goes away? >> That is those are very very good questions. So, right now, I'm basically describing the the the black box as magic beans. And you're not trying to buy my magic beans? >> No, I'm not. Okay. I've I got a whole case of them over there. >> [music] >> All right, from I'm going to steal this from my friends over at the podcast, the Massive Studios, CTO Advisor HQ, Joe Onc, CTO of Unicorn IQ.
This is a big deal. Launch launch day. First of all, it's a big deal that I get to I get to be here in your home now. We've now we've both seen each other's homes. >> We've both seen each other's homes. >> Um but yeah. No, no, this is a big deal. So So we're recording what, five days away from my launch. >> And uh you've gotten some time to sit in kind of convince me of why Unicorn IQ is different than a lot of the data solutions that come before it.
And we're going to talk about the problem that you solve and how you're solving it in ways that you know what that I'm gonna frankly challenge you on because Joe that's why you're on that's that's why you're on the CTO advisor lightboard. >> That's that's why I came to you. You promised me that if I'm trying to sell garbage you will tell the customers that it's garbage. >> This video won't see the light of day. >> I appreciate it. >> So Joe, let's talk about the problem.
>> Yep. I I don't believe the stat that 95% of the AI projects fail. >> But what I do believe at uh Davos a couple of weeks ago, PWC interviewed >> uh something like 4,000 plus CEOs and only 12% have seen a return on investment on AI. >> Yep. >> It's all about scale and not trusting the data that goes in. The models are good enough. I think I've said this over and over again, the models are good enough, but you've come up with this term that I think you're going to coin, zero trust data.
>> Yeah. I I mean, so zero trust of of the data itself is something that we're talking about quite a bit. Um, but yeah, no, no, I I don't like the scare tactics numbers. I've never been a scare taxi guy. So, you hear 95% of AI failed, 85%, you hear all these different numbers, but at the end of the day, I don't need to use those numbers. I don't think anyone is happy with the results they're getting with the money they've spent.
And the I think the thing that people aren't realizing is what you just said. The models are good. The other day I used, it didn't matter which model, just one open model on the public. And I had it vet my infrastructure theory, my engineering, everything we're about to talk about through John uh through Edward Dimming, the god of process, through um several other engineering experts throughout history and through their perspective, Henry Ford and several others. And it nailed it across the board.
It picked apart things. It showed me things I didn't think. It showed me things I should be thinking. models are really good, but when we try and apply them to enterprise problems, things fall apart. Same model. What's the problem? And I think you just hit on it. >> Yeah, we've had this problem in enterprise IT ever since we've talked about data lakes and data warehouses. It really doesn't matter. When we collect data over a long period of time, truth changes.
And even within the time period, like if you've ever looked in your sales force, the same thing for the same customer, two different salespeople input completely different data sets. And data scientists, this stat I've gotten over and over again, uh, preAI spend 80% of their time cleaning up data so that they can get useful output. That won't work in the day of days of AI. We expect to connect these models into our data lakes, have them provide rag via vector uh search to get answers, and we're constantly getting garbage out.
>> Agreed. And and and garbage out. I think that that gets me back to one of one of our problems. Garbage in, garbage out. We're technologists. We've always known this phrase. I was a storage engineer. I was a server engineer. I was a network engineer. I've been a data engineer. I've been an SDN engineer. Every one of those fields we talk about garbage in, garbage out. I think we use that so much that we forget how important it is because we hear it too much.
AI isn't exposing a new problem. AI is showing us the problem we've been pushing and pushing out over the years. We talked about garbage in garbage out for data with big data and cloud migrations. Most ignored it and just moved. Now AI is using that garbage data faster than humans could ever imagine and it's exposing it. >> So let's draw this out. Traditional rag system very simple. We have our database. We have our rag processing and then we have our prompt that has been a motion that has proved well from everything from chatgp to Gemini to every chatbot in the world.
The problem that we have is that that vector database that we're feeding in the enterprise, we're that's where we're feeding the garbage into. Yeah. So your theory is that if we clean up that data beforehand, the better data that we have going into our rag pipeline, our inference pipeline, the better the results. >> Yep. that that's that's the theory and it's proving out in our shipping GA code right now. So right now, rag is designed to augment the learning of the existing model, right?
So it's pulling in your corporate data from the left and we all know we have low confidence in our data. Unstructured data is a great example and that's where I'm focused up front. It's your PDFs, your documents, your PowerPoints, and the rest. And I know everyone goes, well, what about my databases? Those are critical and they are, but 80% of your nastiness and rag came out of unstructured data. This is a language processing system. It's looking at PDFs and word documents.
That's the corpus of knowledge it's based on. All of this stuff is continuously getting ingested into your rag vacuum. Dirt, good, bad, stored in your vector database. Your results right now are hallucinating at the inference layer, churning while costing you a fortune at the rag layer. And meanwhile, your vector database is continuing to grow with dirty data that you'll be processing over time, trying to optimize, trying to store, trying to replicate, trying to back up and then dealing with governance and compliance on and you're paying to secure all that dirt.
" This is the argument, let's just make better models. >> I would say it's the let's throw more money at the problem. That is the end result is that I end up with bigger GPUs, more CPUs, more latency in the process. And we're we've kind of designed for the exception versus cleaning up the data. And this is where I'm going to challenge you in the first layer is >> cleaning up the data is not possible. >> It's not possible. >> The the we we've tried this in the past and uh I can throw people at the problem.
I can create automations cleaning up the data. We used the the Ford example, 1930s, you could buy any color Ford truck you wanted, just as long as it was black. >> That's correct. >> You fast forward to the 50s, 60s, you get different colors. You uh uh you forward to now, now you get, you know, hey, you can get a truck in red. >> However, all of that data is in my data set. And how do I infer the truth from that?
So that's a great example because in theory, every time somebody's asking AI at Ford a question, that vector database has some file about we only sell black trucks and then over here there's a red truck data sheet and inference has to figure out, oh, way back when. Now, can inference do that? Sure, it can do that, but it costs you a lot to make a decision somebody already knew that shouldn't be in your data. And oh, by the way, inference can choose differently every time it hits.
You're completely non-deterministic from this point forward. >> So, how do you fix it? So, how I fix it? Now, first of all, you said it. It's impossible to clean up the data. So, let's just start there. I don't want anyone to believe that I can fix it until I put this in and they can see it work for themselves. But, we looked at this line as what I would call the point of no return. If I go over here to the right, I'm in a non-deterministic, extremely expensive environment.
I couldn't find more expensive compute unless I called IBM for big iron. In that stack, can you fix this problem? Sure. I'm not going to say it's impossible, but it's the most costly and complex possible place to fix it. Whereas over here, it's easier. So, the way we looked at this is we're going to insert over here. So, if you don't mind, I'm going to I'm going to segment this board a bit and we're going to assume that's all over here on the left.
What we're doing is first of all we just connect not inline connect to your data sources first data sources we support today we're starting with unstructured data which is the data your sales your marketing whoever all of those people use everyone uses PDF PowerPoint whatever so this is G Suite for us I'll just do a GS we have a data plugin for G Suite we can go into G drive and pull it out so on and so forth other systems within the G Suite from there we pull that out into our engine.
So again, this is not in line. You can have other things hooked up to it. Don't care. The UIQ engine sits here. And then before I get too far into what we do, the UIQ engine then exports to whatever you want it to via API or MCP. So MCP for kind of natural language AI, API if you prefer. They both have their use cases depending on implementation details. So up here is whatever AI/BI stack you want. >> You're just calling me.
Now if this wants to also call straight back to a rag layer, I don't care. That's not my problem. Do whatever you want. We're designed to fit in and build truth. So we sit here within the stack before it gets here. And I'll pause there and kind of does this track so far. So I it tracks from the idea that I'm extracting the process or the problem of cleaning up the data, >> right? I'm I'm I'm I'm shifting left, even though we're I'm shifting left.
Okay, great. However, that doesn't solve the problem of the black box. What is the black box doing to my data? And secondly, what happens when the black box goes away? >> That is those are very very good questions. So right now I'm basically describing the the the black box as magic beans and you're not trying to buy my magic beans. >> No, I'm not. Okay, that's I've I got a whole case of them over there. >> Tried playing and it didn't work.
I know. All right. So So basically, first of all, if there's if there's a failure for any reason, obviously we're building in redundancy and availability and everything else, but if there's a failure for any reason here, your AI and BI stack can still do what they want. Um meaning they can still be calling standard inference and rag. They can still be using the rag vector database. You may have settings to say, "Hey, don't launch nuclear codes until we can get back to the truth, right?
But we're not in line of how the system works. That's a big piece of what we're doing for two reasons. One, you want to be able to have optionality and how you install us and where you work. " So, I've designed that way. There's no artificial lock in built into anything I ever touch. >> Okay? So what we do is where it gets interesting. Coming down, we can start to assume some data pipeline stuff here. A lot of what I need to do inside the UIQ black box is pretty basic stuff.
Anyone with a, you know, data science background is thinking, well, we first need to handle duplication at the file level and then duplicate. Then we need to be able to probably distill the information out of the the files. Right? If you send me a PowerPoint, I don't need a PowerPoint, right? customer that you're presenting a PowerPoint to doesn't need a PowerPoint. They need the information that the PowerPoint is intended to convey. >> So the first thing we do is pull through a distillation process.
Just think of it as information distillation. We dduplicate at the file level and then we distill. And what I mean by distill is we use generative AI and natural language to go through your PowerPoints, your PDFs, all of that that those files and find out what are they trying to say. Right? A 20page white paper isn't 20 pages of facts. It's some number of facts in a bunch of context, >> right? >> So we we're pulling out what is the actual facts from that.
Now the interesting thing is we are using AI processes, non-deterministic processes here. But I can create a deterministic output even though I'm using a non-deterministic process. Here is your original source. I don't touch it. So, I'm going to call that O source for whatever reason. Um, and then over here is what I'm going to expose to you, right, to your upstream system. So, whatever I use here doesn't matter as long as I can verify that anything I would expose has a direct representation in your source data.
>> So, this is the thing, right? This is where we have confidence in the data. >> I'm not making new information. >> You're not making new information. You're not hallucinating. We have this you can see this problem in today's AI chat box where it it will hallucinate a source that does not exist and I can't link back to that source and then build from there. You're solving that problem. The question that I have now though is okay I don't trust I I I trust AI the models are good enough.
I uh trust AI to come up with conflicts. >> Yeah. to identify the conflicts. That's as deterministic as I would like it to be. >> It's a little It's a little harder than I thought it was going to be, but it's doable. >> It's doable. But someone has to distill this. If people have followed the CTO advisor, I've created the decision authority placement model and saying you cannot leave decisions up to automated uh uh solutions. Not these decisions.
You need a human in the loop. Does your system allow for a human in the loop? >> Yeah, so our system allows for human in the loop, but we looked at human in the loop a little different. So I I'm I'm going to belabor this point a bit. When I say human in the loop in AI crowds, people get all up in arms and CEOs start touching their wallet all funny because human in the loop translates in a CEO spreadsheet mind to recurring cost.
>> Yes. >> Right. And so they immediately think, why am I throwing away money to have humans answer questions? That's what I paid AI to do. Great. That's a possibly valid question. Not the right question, though. The right question is at what point is the value of human in the loop worth more to me than the cost? And the second question is what is the risk to my business to not have a human in the loop on bad data decisions from undocumented processes that are the things that run my business.
>> And then the we talked about this in our AI factory model because what we've basically built is a AI factory and then this AI factory the question is how much waste is in the system? How do we reduce that waste and how do we get our factory to the point where it's running without human engagement? We're doing all the pre-work for that to happen, right? So, I understand from an investment perspective, if I can get the human in the loop to help >> fine-tune the system, that human in the loop that my reliance on that reduces and that capital or opex investment reduces over a period of time as I optimize.
How do I get what what's the end result like? What what what do what system do I have that fixes all of this? >> All right, so let I'm going to walk you through the rest of what we do here and then how human loop becomes a feature instead of a cost center. So UIQ is going to come down. It's distilling. So I'm pulling, think of it, I'm pulling text out of whatever the file is. I just need text.
I'm also creating a vector database. That's my source of truth. So I'm taking text. I'm adding an index hash for vector. I'm storing in a vector database. All nice and optimized for upstream. I distill I then um find conflicts. So conflict resolution there are some conflicts I can automatically resolve depending on the data you have depending on how long we've been running depending on hitl or human in the loop that had already been done. So we do some automated conflict resolution the the real magic of what we do is I don't tell you this is true or false because I don't know and you don't know and the system doesn't know.
What I do is I tell you how confident I am in the data based on your data, not based on general knowledge, not based on the world. This is how confident I am that your data is correct about itself. So we assign confidence and then we maintain lineage or you could call it pro data providence. So these two work together in a traditional onthology of an enterprise because what's true in a manufacturing firm is not necessarily true in a consulting firm. >> Right.
Absolutely. So the example I use to be funny is if you were the CEO of the flat earth society, >> your data would contradict everyone else in the world, but it's yours and you want it to be correct for your view of the world. That's what we do. >> It's your data's truth about your data. If it conflicts Webster's dictionary, that's not my problem. >> So, this fixes the problem that if I love the flat earth example that if I'm using one of the big public models that have truth in it and then I augment it with my data, it conflicts with what's truth in the model and what's truth in my data and the resulting artifacts that I want coming out of my AI system.
>> Exactly. Now, here's the interesting question to me, right? Like in my mind, let's say most CEOs think that they work that they're innovative, that their companies are innovating, that they're doing innovative things. Well, generate generative AI is trained on the largest corpus of knowledge ever. Your corporate data is 1 1 billionth of the amount of information that's sitting in that learning model. So if you're doing an innovative thing, the majority of the data your AI is looking at is telling you you're wrong because it's not what everyone else is doing.
you you need to get that truth correct to have any chance of being of being good on the outside. So we come through here. We've got confidence and providence. So what I'm going to do is all of this gets wrapped up and exposed depending on the ask or request from the API or MCP. But generally what you're going to get is a fact or set of facts depending on how many you ask asked for, right? you're going to get the confidence score that for that and then you're going to get a link to the original source it has.
Right? So if we take this AIB section here and we call it something like um chatbot, right? That's a pretty common use case for AI today. Maybe a chatbot for sales, marketing, HR, don't care, it doesn't matter to me. Um so now let's talk about how we get human a loop. So, you're sitting in this prompt and you ask a question. >> We're out of We're out of the range there. >> Okay. Um, you're sitting in the prompt that I just drew off the screen and instead, let's go here.
You're sitting in this prompt where you ask a question. When you ask a question, I call UIQ. UIQ provides back a number of facts if they exist in the debate database pertaining to whatever that query that was sent to me from. Now this inference and rag layer here gets those facts to work with. If it wants to use retrieval augmentation to get more context, it's more than welcome to. It doesn't bother me, but it has facts with confidence waiting in an original link.
Great. So that at this point inference does a really good job with it on its own because it can say okay well this is an 80% confidence fact and this is a 70%. I'll use the better confidence. Right? >> Now where it gets interesting is in different implementations how you use it. So, let's say in my organization, we don't want our sellers ever saying anything that is incorrect to a customer, right? I don't know if it's just me, but I hate lying to my customers.
Everyone does it. I don't. Um, so when we look at that, if they ask a question where there's no facts, I can tell the LLM, don't give them an answer. >> If they ask a question, there is facts. They get the answer plus source links. Here's your bibliography. Check the work or go do more research if you need. But if it doesn't give them an answer, it gives them an ask the theme button in the LLM prompt exactly where they are.
That's my hit loop. So here's Keith in his natural workflow working with AI. " And AI goes, "I don't know. " So he hits askme. When you hit this button, it triggers it out to a pool that has been preassigned ofmemes, right? You can keep doing your work, whatever, but now you got a chat window open with the theme just like you might call a human in the loop. Our business is run on human in the loop. We just don't do it in a digitized fashion.
So now this person responds and as soon as you say that response was good for Keith, this was Keith's question. That was an answer to Keith's question. I create a new source file data artifact and I send it back to the beginning of my pipeline. It's an untrusted fact just like the rest of the facts. Once it goes through the system, the next user or the next time Keith asks the same question, the system answers it. We reduce the friction of your knowledge workers while extracting tribal knowledge as human in the loop to improve your source of truth at the core.
>> That might be one of the most brilliant things I've seen in the in the wild. Simple because we're taking >> I'm a marine. I can only do simple and stupid. So Joe, this is really interesting. I think we've we've consumed a lot. We've uh talked a lot. I have a lot to go back and read. There's a launch paper that I need to dig back into. What if anything have we missed? >> So I think what I'd like to to get across if we got just one minute, what we're not building here.
If you've worked in enterprise tech across the last 10 years, you've seen every startup that exit, focus on building a company that could exit with product as an afterthought. You get those things into your environment and they fall apart or they get acquired by one of the big names and then the core features get gutted in the rest. We're taking a different approach. We're building a very, very modular product designed to fit into any AIBI stack you want. We're not going to lock you into us.
So you don't have to buy an entire platform of 90% crappy features with one thing we do well. We do one thing well. That's all we charge you for. That's all we sell. We flat rate price so you're never going to be surprised. We're building a product that solves problems for people and that's all we care about. >> Joe, well I appreciate you coming all the way from the west of the US, from your cattle ranch to the Midwest to talk about this.
This has been worth my time. I hope it's been worth yours. >> Absolutely. If you want to learn more about Unicorn IQ, follow the link below. com, which is a big deal for me. Talk to you next. CTO Advisor lightboard.
