Right-Sizing AI, Rationalizing Virtualization, and Becoming an Infrastructure Arbiter — with Melissa Palmer

Machine-generated from the episode audio and not hand-corrected, so names and technical terms may be imperfect. The audio is authoritative.

All right, you're listening to an episode of the CTO Advisor podcast that we should have done a long time ago, but I'm glad that we held off. Longtime industry peer and friend, Melissa Palmer. Melissa, what's your title these days? My title is whatever I decide it is that morning when I wake up, because it's kind of the glory of working for yourself, right? Yeah. I would say principal architect, analyst, enterprise architect, somewhere in those lanes on most days. I love it.

I love it. I love it. The great thing about entrepreneur, you can change your LinkedIn status and job title every week if you want. That is absolutely the advantage. I can't give a snapshot of your career enough justice. I know you as Melissa Palmer, the IT infrastructure technical expert. I think people view me sometimes as a technical expert, and this is a rare opportunity for me to have on someone who lapsed me from a technical capability perspective. You have your VMware BCDX.

You've been posting recently about AI, fine-tuning, not just from the abstract, but details about how to implement AI at an enterprise infrastructure perspective. This is why I wanted to have you on. Is there anything big in your portfolio of skill that we should highlight before we dive deeper into this conversation? Thank you. Got it, Keith. Long-time, full-stack infrastructure person, right? Not much of a coder until the last couple months. Maybe we'll talk about that a little bit, but coding was always a thing for me where I coded like a maniac probably from the ages of like 12 to 17.

And then when I was looking at colleges, I was like, but coding is this thing I kind of know how to do. I don't want to just go learn 75 different types of more syntax thing, which is actually why I went to study electrical engineering. And I'm like in my glory with all this AI stuff coming out because I'm like, oh, I get to actually think about circuits and all these things I haven't used in 20 years or something ridiculous like that. But no, I think you summarized me pretty good.

So the reason why I've been wanting to have you on, I think most CTOs, CIOs, VP of infrastructure are looking at their existing staff. They're looking at their smart people. On the CTO advisor week, we have this shorthand for, we have so, you know, all of our team is smart, but we only have so many talking to the Phoenix project. We only have so many breads, right? So my shorthand for that is our smart people. We can only put our smart people on so many projects.

And what I've noticed in the days of AI is that I'm torn between, do I put Melissa Palmer on my AI project that is burgeoning or do I put her on my real problem, which is VMware validation and assessment? Should I continue my relationship with Broadcom? Two equally difficult problems with two very different outcomes. What have you seen from, you know, the ground? I have seen a lot of what I like to call the virtualization rationalization or hypervisor hunger games, right?

Everybody, it's kind of like a really good time for everybody to take a look at their virtualization environment, dial back down to their business requirements at the end of the day. Like, why does this exist? Why am I doing the things I do? And figuring out if you're still making the right choices, right? And that's a good place to be because you can take that information of why are we here? What do we do as a business? What are our goals?

And start then thinking about the terms in AI, right? So we're talking about CTOs, right? CTO Keith is probably going to come up to enterprise architect Melissa with CTO magazine one day and be like, Hey, Melissa, AI is on the cover. What do we do now? Right? So we got to have our house in order before we even start an AI project and know our core businesses are problems that we have to solve, right? What makes us money? How can we make more money?

What makes us lose money? Understand all that. And then what is the right application of AI to make that happen better? Either save more money, make more money, something like that, right? We can't just go AI for the sake of going AI. And believe me, I want to because I was watching all the announcements at GTC this week and the beautiful Rubin racks that are coming out and like diving into like the six new chips and here's all the flops and specs and tokens and I love that stuff.

It's so cool. Tokens for a second. It's all that token. No more IOPS. I'm so glad. I hate IOPS. I'm so glad they're gone. And as cool as that all is, that's not really representative of the enterprise and how we're going to use AI. Like we're not going to roll in a bunch of data center full of Rubin racks for our enterprise, right? There's very few organizations that will do that. Other side of things. I was talking to my friends at Kamiwaza and they were telling me that there are fortune 100s that a single B200 will accommodate all of their inferencing needs.

So the kind of right sizing AI for my infrastructure. I mean, I want one of those racks. They're pretty. I mean, they are. They're beautiful. In L72, I would love to have one and I can't have one in my basement because it's not going to work. I can power one of the nodes in the rack. But you're highlighting kind of one of the core problems I'm seeing, which is sizing. And it brings us to kind of what one of your pet projects have been.

I'm not going to put an NV72, NVIDIA's rack level AI. I'm not going to put that in my existing data center. It's not. I can't be able to- I can't power it. Cool it. Let's like take the facility stuff off the table. You're not putting that in there. And second of all, chances are you do not need it. Chances are very high you do not need it. And if you do need that capability, there are other ways to get it.

You can go to a hyperscaler or a NeoCloud and rent that capacity from an NBL72, right? Without having to have it in your data center as well. But you probably don't even need it, honestly. And that is the tension, right? Even when I get past, let's talk realistic, you know, I can probably run most businesses on a couple, if not a handful of RTX 6000s, RTX 5000s, which are the entry level Blackwell professional level graphics cards. Even those are a bit power hungry, but I can work them out.

When I start to get to the requirement where I need an H100 or a couple of H100s, I may want to co-locate that. Yes. So I may want to put it in the data center where- If your co-location partner can support it, right? No, this gets me kind of to your- You'll find a new one. This gets me to kind of your passion project that Eve is working on, which is data center analysis on the local level, right? Because you're not going, you know, I can read about the Switch Data Center in Vegas, but I'm here in the Midwest and I may want something in Springfield, Illinois or wherever I want the data center.

And you've been collecting data. First off, tell me what challenge you're trying to solve. And we're going to dig a little bit into the how you've finally been able to crack this nut. Yeah. So it's really interesting because I kind of had one intention for this passion project and it went in a completely different direction once I started working on it. So I have been on my own independently a little over three years at this point. And when I left my corporate job, I really wanted to start like an infrastructure news site.

Like that was my thing. I've wanted to do it for years, but I worked for vendors, so I couldn't be writing about all these different vendors and I really wanted to do that. I started trying to put it together and at least to like be able to scan the news for what's going on. And it lasted about two weeks. Like I just could not, it was the only thing I could do. I could not keep up with it and it just didn't work for me.

So that was kind of like really disappointing. And then I kind of tabled it for a while. And then when I started getting really into some of the Nvidia equipment, actually like reading about the DGXs and how you work, how you cluster them, super pods, base pods, all that stuff. I'm like, I have this infrastructure bug again. I'm going to do it. I know what I need to do. I kind of know how to do it, but I am going to, since I'm talking about AI infrastructure, I'm going to use AI to help me.

So the brand and the presence is the Infrastructure Constellation. com. I have a sub stack. I'm on LinkedIn with it, all those good things. And what the intention was, it was supposed to be the Infrastructure Constellation. I was supposed to be mostly talking about infrastructure components, like deep dive on all the AI storage vendors and server vendors and DGX, HGX, all that stuff. And I started it until I started kind of refining my news viewing, right, and looking at that kind of stuff.

And I was using OpenAI Codex to write me scripts to help me scour the internet, right, get all the data in one place so I as a human can make a decision about what is hot that day, basically. And then I started seeing all of these local news articles about data centers being built in all different places. And I started diving in on them and seeing trends and the same problems over and over and more and more stories that aren't reflected in like the mainstream news, right?

And if you go put on, I don't know, whatever people watch these days, like the, what is it? MSNBC is kind of like the money channel, maybe with all the stocks and all the analysts going as they had. So what is called now, SWOT network? These things are called, right? And we're talking about, oh, OpenAI and this and $40 billion in this data center, this data center, they're going to build data centers, we're going to build data, everybody's building data centers.

And stocks are fluctuating based on everybody building these data centers. But I'm sitting down looking at the local level going, but that County in Michigan or wherever just put a six month moratorium on data centers. So you're not, you're not building that data center when you think you're building it. So I've kind of started to fall following the news at a local level and trying to analyze what are the same issues that we're seeing over and over again, what particular data center companies are doing really well and actually not having issues versus some companies having issues over and over again.

So it's been really enlightening. And it does kind of tie back to AI infrastructure in the sense of we have to put this infrastructure someplace, right? It goes in a data center. The data centers of today do not work for these AI workloads. So we need to build new ones. But if we can't build new ones, what exactly is going to happen? So it sounds like you've built a arbiter of what's real and what's not in the data space.

What's the other way to put it? I like that. I've been using this term, I've learned this term arbiter the other day and I've been using it quite a bit. I like it. Well, yeah, you're this arbiter of what's happening, what's real, what I see on SWAP network. And then what I'm doing on a local level, that's a tremendous amount of information. I'll make sure to link your substack there so people can start following that information because it's valuable information.

But what struck me was that you've been thinking about working on this for three years. Three years. And I couldn't figure out how to do it in the sense of I'm not a coder, right? I'm pretty dangerous with PowerShell because of VMware and stuff like that. I will learn enough coding to be dangerous, but I'm not like a sit down and code person. I was as a teenager, which sounds silly, but my teenage years were spent coding to the point where it was time to go to college.

I'm like, I don't want to do computer science because I've just been coding for like six years, right? And that's all I do. I want to learn something new. So I actually studied electrical engineering, which is also great now because when we get into all the crazy stuff about the ASICs and chip production and stuff like that, I feel like it's coming home, right? I'm coming home. I can relearn all this stuff. It's so fascinating to me. And I've been not a big coder type person, but when you use something like ChatGPT or OpenAI's codecs, that's different because I have enough product development experience that I can frame out my requirements and the gist of how I want to meet them.

But I don't want to spend every night and every morning and just all of my time coding it. So when I come up with my application and my plan and what I want in this version and this version, this version and the gist of how I want it implemented, and that's the big thing. I knew how I wanted to do it. But the coding skills, I don't want to say like I couldn't upskill to do it, but I also didn't see the value in doing it.

Right. There's the there's the time to value. There's. And I spent all of my free time coding like it didn't really fit me per se. So I started spinning up codecs and having it all me. So you are inadvertently now becoming the poster child for what folks like Martine Casado from A15Z is talking talking about. He and his friends are now coding again because they haven't coded in 10, 15, 20 years. It's been coded in like, well, like I said, PowerShell.

I need to for VMware, but like sitting down and coding like I haven't done that in years. So I think what. Is not as clear to people is what skill. Do they need to be able to leverage something like a codex, a clod, a cursory eye? Well, here's the trick. There's two schools of thought. You don't need any skills. You can just tell it what you want and it will go do it. And it's going to decide how it's going to do it.

Right. Or if you're more of a architecture systems level type thinker like me, you know the how already. Right. So for me going in and saying, I know exactly how I want to do this. Here's my requirements. Here's how we're going to achieve each one. Now go code it for me, like telling them what web framework I want to use, what scripting language I want to use, all that kind of stuff. And then you have kind of like the vibe coding crowd who's just like make me an iPhone app about maybe an iPhone app where I can put hats on cats or something like that.

I don't know. That's actually not a bad idea. But something like that. Right. It's become very accessible to everybody. And I hope that people do that. I hope people go make the cat hat app, which I think I might have to do now. And they get exposure to it and interest to it. And maybe that prompts them to go a little deeper. Right. Because if you give codecs or cursor or claud to anyone who doesn't know how to code, they'll make an app.

But they don't know how it works. They don't know what they did. And if they want to change something, probably stuff is going to break. So my hope that is that it inspires people to go learn the theory behind how it all works. Right. And then we take off kind of the need to spend hours learning syntax and forgetting semicolons. Go now. You're interested. Go learn the theory and learn how to properly architect these pieces of software.

So if I hear correctly, you know, I'm a big cell phone game nerd. Flappy Bird is no more. And I'm like, you know what? I really like Flappy Bird. I'm going to go to codecs and I'm going to have it make me a Flappy Bird clone. And then I'm going to, you know, play with it for a couple of days. It works fine, but it's too difficult. And I want to change the difficulty level. And I go to change the difficulty level.

And that's when the app breaks. Right. And I need to now figure out the logic of how to work with it, which is a fascinating role to think about, because I remember when I learned how to code and when I needed to learn how to troubleshoot my code and I need to learn logic. It is similar, but not the same. So I'm fascinated for those folks who are learning code for the first time doing this path. But if I hear you correctly, there's two types of capabilities.

Both get accelerated. The engineer like you who already knows distributed systems, you know where code breaks. You may not know the specifics of how to write something in Go, Python or pick your language, but you understand coding logic. And you just need to be able to give the required you know how to build the requirements, give the requirements to codecs and then maintain the code. Then there's the business user who just knows the requirements. They know how to give the requirements, but they need to have a path.

You need to support to both. Yeah, that's why I don't see software engineers going away or anything like that. Right. We're just not me. I don't want to call myself a software engineer. That's not my area of expertise. I'm pretty sorry. You built a pretty complex application here. That's not what I studied. Right. Like it's not what my career has been. Those people are still going to be valuable because like who's going to OK, let's say I'm let's say I am an infrastructure person in XYZ company.

And I go write an app that goes and looks at my infrastructure and pulls all the different utilizations, puts it in a dashboard, whatever. Right. And then I do go break it. And it was really helpful. It was helping my team. But now I broke it and I don't know how to fix it because I'm not a software engineer. Right. We still need those people in our organization, like the SWAT team almost right to call in and help the vibe coders that come in and sit down with you.

Like, OK, well, here's how you this went wrong and this went wrong. Here's how we fix it type of thing. So I don't think I think there's value in understanding the process. I don't know that every cloud code chat GPT codecs cursor user is going to go that far, but you definitely still need to have that skill set in house someplace. So, Melissa, I really appreciate you taking time out of your schedule. Thank you for having me. If people want to contact you to engage you, to help them figure out their VMware landscape, to figuring out their AI infrastructure strategy, what's the best way to get ahold of you?

Yeah, you can find me on LinkedIn. My name is Melissa Palmer, also known as Vimas. I'm on Twitter at Vimas 33. And you can find kind of my passion project that we've been talking about at the infrastructure constellation dot com. I will make sure to link all of that in the show notes. If you want to learn more about the CTO by tab, you can find me on the web. I'm still on the Twitters. I'll continue to call it the Twitters.

I know. I just realized it's X now, isn't it? And I just. Yeah, it is. As I gave up the domain for the for years, I owned the domain name, the Twitters. Because my daughter hates it, hates it when I called it that. But I'm still at CTO advisor on there and Keith Townsend on LinkedIn. DMS are open on X until then. Talk to you next. CTO advisor. Thanks, Melissa. Thank you for having me.