Nvidia Spark Doing Real Work - What is the top CTO concern across 15 years of Tech Field Day?
Transcript
went out and purchased this Nvidia Spark and not blindly, I wanted to do something specific. We're in a marketplace where firms like SM like mines, a twoperson firm can compete with much larger analyst firms, consulting firms, system integrators for a world where AI is generating slop. I wanted to set out to one enable my firm, the CTO advisor by tab to not just compete but show other enterprises how to enable their workforce to get insights using these tools. That's one of the few tools I went out and purchased this Nvidia Spark and not blindly I wanted to do something specific.
I participated in tech field day sessions for the past 12 years. This is where 12 delegates are presented by to by some of the industry leading vendors, startups, their vision for the future of their respective disciplines, whether you're talking about wireless networking, storage industry, AI specific capabilities, servers, cloud, we've got some of the most interesting conversations within our community over on the Tech Field Day YouTube channel. Over 15 years, we have over 2,000 videos with 7 to 8,000 hours of not pre just presentations, but conversations.
These are people like me challenging these vendors on their presentations. I wanted to get insights beyond the technical conversations that we were having. I wanted to know what were top three to five CTO level concerns and the insights that I got out of analyzing this content is amazing. However, we had to first tackle the problem. And this is where I think the learnings from what AI is capable of doing and versus what you need to arm your teams with from a capability perspective.
This is a basic data science problem. You need to first chunk your data. So cut this data up into smaller chunks to to be classified. Then after we're we've classified the data, we need to then analyze the data. So nothing really amazing about the technology, the the friction was the fact of the scale of the data. 2,000 videos over 7 to eight hours of content translated into 230,000 chunks. 230,000 segments that have to be classified. We ran into the classic demo versus production.
In demo, I can run a 30 billion parameter, 70 billion parameter model on this hardware with no problem. I'm sending it a single question. I get back answers. Much of the content we see on YouTube is about how fast uh something like a Spark or M2 M3 Mac Pro can get through this uh this model. But what it doesn't do is, you know, this real problem, we need to chump and classify all of this data. When I use these demo techniques that we see on uh YouTube, it was going to take two and a half days.
That's unacceptable from an iteration perspective. What I had to do was load up VLLM on the Nvidia Spark and it flew through it in about an hour and a half. This taught me two things. One that demos are goss and that two platforms and the applications we build platforms on matter in this echo ecosystem. We have VLOOM. We have Rockm which is the AMD alternative. And then we have MLX which is the Matt alternative. Don't want to get into too many details.
The main thing I want you to get out of this ecosystem is that VLM is the most widely supported platform for models. When you get down to the accelerator part of it, AMD versus CUDA, there is much much wider support for CUDA than there is AMD, which translates to your ability to just use models. Which models are available to you when you're doing your analysis. So, what was the insight? What did I discover with all of this? What's the top CTO concern over 200,000 segments or 8,000 hours of conversation?
It's risk. Most CTO's are concerned with risk over cost or technical capability. They are they want to know what are the risks associated to implementing a technology to maintain that technology and for that technology basically getting them fired. Fascinating uh insight. I anticipate creating different mixes of this analysis that I can then feed up to Gen AI and make, you know, the the the next level of insight to be able to have conversations with this data. But this is what it took.
com. You can follow me on social media at CTO Advisor. Talk to you next CTO Advisor Lightboard.
