Should Your Business Host Its Own AI?

Speaker Hey everybody, welcome back to Make Others Successful podcast where we share insights, stories and strategies to help you build a better workplace. Surprise, surprise, we are continuing our conversation a little bit from last time. Towards the end of the episode, we started to talk about the idea of owning your own AI. And as something gets more mainstream, someone commoditizes it and allows people to get their own version of it. And so, um, we wanted to take that a little bit further today and share some more thoughts. This is far from a buttoned up opinion that you should take, uh, as, as truth because it's moving a million miles an hour and it's just something we're considering and we think the thoughts, uh, that, that we have might be worth pondering and talking about. So we want to talk about the concept of on premise, on premises AI. Isn't that supposed to be on prem for short? I think it's on premises. We don't have to debate it if we just say, I'm going to look this up when you guys start talking. Okay. But if you're new and this is all your if this is your first episode, my name is Mitch. If we haven't had a chance to meet yet, this is Mike. He is the one who's super excited about this idea. And it's going to really carry the torch for us. And we have. Matt, how did this conversation get started? Why? Obviously, we talked about it a little bit last time, but then it trickled into some in-office conversations and something that you were a little bit, uh, excited about. What, what's the story there? Yeah. So, uh, I had the opportunity to go sit in on a kind of an AI in your business. Um, I don't know, fireside chat, webinar, seminar type, uh, thing here in West Michigan a few weeks ago where a business leader talked about how he's using AI in his organization to facilitate annual planning. So the story goes, this typical strategic planning that you do every year, the leaders go off and go to a retreat, and they come up with some ideas, and then they come back and they disseminate the Ten Commandments to the team, and then you go off and do the thing. Um, and he's figured out over the years that the more people he gets involved in that process, the better. And so as that process has evolved for him over the years, he's included like department heads and other leaders in the organization to participate in that process. And then when AI came into being, he realized that he had an opportunity to expand that to everyone in the organization. And so he went about the process of building, um, all of the collection mechanisms, recording mechanisms, um, tools and so on and so forth to be able to solicit like survey responses from his entire team. Get the entire team together in rooms where they're recording their thoughts and ideas about all of these things that are going on. And then he feeds that all into an AI engine, and out comes a strategic plan for the next three years. And so that's kind of the the story that he built and all of that stuff. He essentially hired somebody to build it all in-house for him. So he's all running on a, like a Mac studio machine. And he's got, uh, at least one developer consultant who is assembling all of the tools and the parts and the pieces and building the data storage and the using a particular AI model and engine to process all of that data. And now he's going to write a book about the whole thing too, which is kind of an interesting idea, but it made me think about what we do and how we coach some of our clients and how, you know, we're very m365 focused and copilot is the thing, and it has access to all of your data. And so the whole time I was listening to him, I was like, man, why isn't he just using copilot and M365 and SharePoint and teams and all of that stuff? And by the end of it, I think the what he basically came down to was, I want my stuff secure. I want to be able to trust it's inside my bricks and mortar was was one of the things because I think he's probably heard all of the, the stories about data leaks and all that kind of stuff. And then the other thing that I think was important to him was cost. And he figured out that he didn't want to have to like, have an unknown consumption model based cost where he didn't recognize or know what it was going to cost him. He wanted it to be more predictable. And so he's like, we're going to use open source and I'll pay a consultant and I'll own it all. And that's kind of where this comes from. Long, long story. Yeah. Long story. Long. Uh, yeah. Yeah. Okay. So he is running things on a Mac studio that I feel like there was a big push recently for both Mac Minis and Mac Studios because people said, oh, these things are capable. I can install these models on my own systems and run with them, and effectively have no subscription costs. Just have your own little world. Uh, Matt, when you hear about this reality, what what triggers get triggered, get pulled in your mind that you think either they've got something figured out or they haven't even thought of this, and they really need to think about that. I mean, it's, it all comes down to your methodology and your way of doing work. Uh, taking something on premise is not it is on premises, by the way. I looked it up. Oh my goodness. That's the grammatically correct. Okay, well, you can keep your grammatically correct things over there on premise. I'm going to continue to say it. You heard it here. Matt is declaring a new term. Uh, so on premise installations is nothing new. Uh, there are have been for a long time valid reasons for that. Uh, those reasons are still valid and they're largely all the same situation. Uh, the big differentiation between the AI world and what has happened in the past is mostly related to uncertainty, right? Uh, both in what changes. So an AI model changes and now I have to adapt to that change. And now I have to do something different to make this thing continue to work, etc.. But then also, uh, adapting to how much am I going to really use? How much is this thing going to cost? Who is going to, who owns that cost? Um, and if anything has been proven in the AI revolution is those things are real and they change and they can be very significant. Um, you know, we've talked about it in previous podcasts about the move from predictable pricing models to the consumption based model. Uh, the shift between, uh, worrying about individual tokens and, and context windows and, you know, all the level of detail to, oh, I just how many credits am I using? What, you know, more aggregate that up, go across models. I want to be able to use five different models for the same thing and, and use what's best for the purpose that I'm doing. Uh, so it's a real concern, but it is not as simple as saying that's the right way or the easy way to do it. Okay, well, maybe to give some background, like self-hosting AI is like bleeding edge. Like what are some times that either we or customers we've seen have been trying to make the decision? Should we have something physical or in the cloud? And how did we talk through? Well, it's interesting because you call it bleeding edge. I would say it's not. I would say it's actually old school, uh, AI before AI was in the current world, it was only self-hosted. You built your own thing and you built, you know, if you needed some sort of supercomputer, well, no, if if you needed to do object detection for, uh, understanding, um, uh, whether or not Apple is good, right? I'm looking for spots and blemishes or eggs. They shine a light and they're looking for eggs that have been, that's been around for a very long time. That's not new. And that was all self-hosted. That was all on hardware that's in the facility doing its thing, right. It was not a cloud thing that I went out and used. They spent millions of dollars coming up with a very special detection system that's targeted for a particular purpose and built into a specific thing, right? That level of thing has been around forever, right? Almost as long as AI has been around. Uh, the difference is that those things are very expensive and very few people had them. Um, they were for a particular purpose. You either bought a third party tool that was already built to do this type of thing, or you spent a lot of money building your own thing. Right. The new AI wave is all about more generic models and generic, uh, access to information and building models yourself and, you know, adding layers upon layers upon layers to this technology so that it's more, uh, user friendly for users to just to ask it to do what it really wants and customize it to do what it wants. That has been done more in the cloud because it is trying to get to a mass audience. It's trying to enable this to lots of different people. You're not trying to put it on a piece of hardware that's in a factory or in some other purpose. Uh, and so now you're really in this world where, great, that's what we're doing. But I'm not seeing the predictable payment. Right. And so instead of being think of it as bleeding edge, it's, it's really starting to say, hey, is this thing really to the point where I want to trust a cloud platform to do this stuff? Um, and that's, I think part of what that gentleman's whole deal is, is, you know, I've got all these, these cloud solutions. They do all of these crazy generic wild things. I have a specific use case, right? In his model. He's like, I have this application that I want to build AI into. That's a perfect model for on premise might be an okay choice, right? You want to control that model really well. You want to be able to manage that cost really well, but it's not that simple, right? You're also taking on all the costs associated with that maintenance outages, which in his case, it sounded like it's not like I'm running it every day of the week all the time. And it's got to be available, you know, all the time he's doing a once a year, we're going to do this process and make this thing happen. So he may be in a very unique situation where it may very well make sense. It's very interesting the progression that he was on. So the latest, as far as they have gotten, is to the point now that where they've decided we should build agents and make those agents available to everyone all the time so that they can so they can be connected to this thing that we're doing together, right? Yeah. And they straight up, I think it stalled out. I think they're struggling with that. That part of the implementation, which is really interesting to me. There's some parallels that I'm hearing. You know, we've talked about stuff before, like when do I build apps in the cloud versus when do I need to like, do custom like, and the like the egg processing thing sounds a lot like I need like sub millisecond transactions when I'm doing predictable stock trades, predictable. So there's some parallels there in terms of like, what are you actually trying to do with it versus how you have to do it, right? And then also the cost of ownership question, I think, is also a similar parallel in that way. The cost of ownership is an interesting one that I think is by some of these people who are talking about, hey, I want because I've heard this before, this isn't the only person we've heard this from. Uh, and they talk about it, but I think that predictability of, oh, I want my thing to be stable or the same for a period of time. That is one perspective on it. Another is I spent all this money, I built all this thing, and now anthropic has a brand new thing that only works on this hardware, and I don't have that hardware. What do I do? Oh, I have to if I want that, I have to throw my old hardware away and get new hardware. And that type of thing is happening so much. Uh, they're just like, they're the iteration and rate of change in these core technologies, which is a lot of where the open source things are really coming from. You just get access to the raw model, and then you have to build everything on top. Uh, it changes all the time, all the time. So is it safe to say that because it's formulating and changing all the time, that it would be considered bleeding edge? Yeah, not to put it on, not the on premise piece. No, I'm not saying the concept is that's what that's when you said that that's what I thought you were saying, is that that the concept of putting it on prem is bleeding edge. That's what I thought you meant. If I'm a business leader or if I'm one of our clients and I'm trying to like incorporate iOS, or I'm trying to do annual planning, or I'm trying to come up with a better process for my team. Like, how do I know if the Microsoft tools, if copilot, for example, is capable of being enough and being good or even great, or that I need to go down the other road and just roll my own and hire somebody. I think it all comes down to the same. This is a this is a tale as old as time. Uh, it all comes down to whether how true as it can be, how, uh, how open you are to living within constraints and having somebody own your destiny. And, and that a lot of times comes down to is how core is this to your business? How core is this to your philosophy? This gentleman, it was core to his business to change the way that they do yearly planning. Like that's very important to him. Right? Uh, and there was nothing on the market that can do what he wants to do, exactly how he wants to do it. Right. If you or I were looking at it, we might look at it and go, well, let's wrap some process around it. Yes. Use some technology. Yes. Use some AI pieces here or there. But let's try to keep it loose. Let's try to keep it flexible. Uh, let's, let's the people are what really matters. Let's go make this work the best we can and we might find a lot of value from something like copilot or one of the other more generic AI tools. But for somebody who says, this is really key to my business, and I cannot afford it to afford for it to not respond the way I expect and function the way I expect. You got it. The only way to really control that is to, to own it. Um, that's the thing that goes back to the that's the thing is I don't think he is thinking about that. I don't think he had an expectation. Yeah. I think he was trying some things out and well there's a whole. And he wasn't willing to take the cloud risk. Right. And he he met somebody that like hey, yeah, I can do. I can put that on a Mac studio. And but it's not just that. I mean, uh, there are a lot of people in the technology world and that use technology that are and were not immune to it. We're like, we're building our own CRM for our own reasons. Right. Uh, the, you know, you get somebody who has got a little technical knowledge and has a vision for what they really want, and realize that these tools can't do exactly what I want, the way I want it to be. They're gonna that's. They take ownership. It's not they're not the I've heard this again, multiple organizations, uh, like the reason that you're talking about it, which I would consider to be not a true altruistic, uh, like this is best for the company thing and more of a personal like, oh, I really want this thing and I'm tied to this thing. And so then I just want to do it. And if you got the money and you got the time, like, and, and it's going to like, I also think at the time that, that they started that endeavor, like I don't think a few more years. Yeah. I think it was at least two years ago, at least two years ago, which the world was cloud tools were different. They're two years ago. You definitely shouldn't have used copilot. Yeah. Are you trying to suggest we're building our CRM because of me? Uh, us and I'm not. Are you are you actually the company? I would tell you, it's us. Are you actually making this all about you again? I would say no. I'm trying to say this is for you. It. No, it's. How do you not see that? I go to work every day. It's us to provide for you. And we are the worst. How do we not see it? And we are the worst compounding force in that regard, because there's pieces of it that I want. And there's pieces that you want and there's that he wants. And then and it's good, not great, but it's good. Anyway. I like it, to be honest, I like it. Uh. It's pretty. That's great. That's all you like about it, Mike? Is that. It's pretty okay. I'm gonna. It got better in the last forty eight hours. I'll say that. I'm going to shift our direction a little bit. There's a few times where we've been deciding on premise versus in the cloud. One that I can think of is we have our own little server here running in my eyesight. Yeah. Uh, should we talk about how is that similar or different from this? Should we call it a decision? Yeah. At a core, it's the same decision. Right. Uh, the cost of doing, of the cost of doing, uh, cloud based storage. So what he's talking about is a file server effectively where all of our videos are stored, etc., etc.. If you know where we're located and you can break into the room, you can see all our videos. It will be amazing. Uh, come on in. Come on, come on, try it. Uh, the, the, the decision for us is a lot up to do with what's the cost and what's the, what's the value? We don't have a big enough internet connection to make it really reasonable to be dealing with all these files, all in the cloud, number one. So we'd have to get a bigger internet connection then we can we. And it's not like we have a small internet connection, but it's not big enough for those files. Uh, and in addition, uh, we have a lot of files like the video content that we're producing. There is a lot of it. Um, and it doesn't really warrant at this time, uh, better storage. Um, could it in the future? Yeah. Like part of our problem is going to be soon. It just keeps growing and it never stops growing. And you know, USB hard drives on the floor isn't exactly a solution to storage stuff. Uh, so we gotta, we gotta put it somewhere. But, uh, I don't see that as being a forever thing, but it definitely is the right choice for today. Um, and maybe even for the next year or two. Mhm. The interesting thing for me in this conversation really comes down to, uh. Thinking about your business process, uh, that you're trying to optimize and choosing technology that works for you and that business process. So when you talked about this, the first time when you came in, you focused a lot on the AI, uh, on premise situation. But in this latest one, you really talked about, well, this really came from this business purpose, right? Uh, and, and it, it strikes me as, you know, that is what somebody needs to keep in the forefront of their mind all the time. Um, and sometimes that may mean bring something in on prem. Uh, it could also mean two years later, it doesn't make sense to have it on prem anymore. And you really should do it. Put it somewhere else. Right? Sure. Um, and we talk about that a lot in a lot of the things that we're doing. We've also, there's also been a lot of news articles about organizations who are getting rid of, um, third party tools and just using AI to build their own custom ones, which is again, very similar conversation. It's not on premise. It's not like, hey, I'm going to store this server here physically located, but it's saying, I'm going to take this application that was owned by a third party, a cloud ownership. Now it's owned by me and I have the developers and I have the all of that stuff and self sourced, self managed. Mhm. And, and, uh, there is no doubt that today more than ever before, people are able to do that effectively. I'm going to use air quotes effectively, whatever that means to you, more better than they did. Could do before. Right. Uh, did you see someone ripped off Adobe? The Adobe suite. Uh, they recreated it as open source. I think it's running on rust or something. So it's, like, super fast. And, uh, and whenever you get like, one or two levels of comments deep, you realize it's a little bit half baked. Like it's, it's AI created software, but it's like, it looks the same. It seems to do the same thing. And it's literally free open source code and you're not tied to some, uh, sketchy license, subscription sign your life away thing. Yeah. And I mean, it's, there is a, there is a whole world about that that is really interesting. And, and I think this, this, if you think, if I can think about, um, the concept of the Microsoft three. Sixty five tools, not copilot, it's the same kind of things, right? Uh, I used to have to build a third, spend hundreds of thousands of dollars to build a a survey system that I could go send surveys out to. People. Well, now I just have miss. Forms, right. And I can go create my own forms and ask my own. It's different levels, but it's all the same thing, right? And I would look at these AI tools all like bringing AI in house the same way we, for us individually, like for our business and for most of our customers, we're never going to recommend it. There's very few business cases where that makes a lot of sense. Uh, but there are some that do, uh, and more in our case, more often than not, it's in the business apps and automation space, right? Where somebody is trying to do something. They're frustrated with one of either the Microsoft products or maybe a third party tool, or they don't have a tool at all, and they're all doing it manually. And it's like, no, no, no, build an app that does the thing like it's not that expensive anymore. You can like, you can do this and it can provide so much value. Can, can you see though, like on the horizon, some world where there's a utility, there's an appliance you can buy. AI growth has plateaued a little bit, and we've found some sort of. This is plenty good enough. We don't need to take it farther and let's put it back up. We're going to be back on the bleeding edge thing that's been there for twenty years. Like I said, there are utility applications where people package up AI, put it into a device and give it to you. Will there be more? Will there be more utility, though? Not as a you. Everybody should own a furnace and a water heater and a Wi-Fi router. But do you have your AI server? I don't think so, no. I mean, people buy NASA's today and stuff like that. Sure. But most, most, most models are not trained on your personal information. Like that's not why you use them today. You would need to have a use case that says, uh, okay, okay, uh, uh, a, uh, video or a photo or a video storage thing that has AI functionality. Maybe something, a language model. Yeah, that's I, I don't think so. That's just my take. I don't know, like the, the, uh, I, it doesn't make much sense to me. Okay. Uh, until, until the utility is a replacement for my brain. And your essence is your soul. Like. Yeah, but what about because what about when you think about what a language model is and the, the, the training behind it, it is effectively in the beginning was the word was God. Okay. Uh, what about libraries? What about him? Why can't I have an appliance in a library that we could access appliance in a library, or that had all of the shared as a shared library? Like, yeah. What what what do you think? Like, like a pseudo public resource, right? How is that very much different than what we already have? I can't. One is paid with taxes and the other one isn't. Okay, that sounds like not a technology problem. That That sounds like a political problem. I mean, but it's a good question of like, when does something become good enough that it should be a shared resource among people that the government says, like, this is important to our infrastructure? That's that's the internet. And the government's not really standing that up. No. So but like the the use case that just came into my mind was like, oh, man, I wish I could record everything I ever said and had metal glasses on my whole life and a copy of everything I ever wrote down. And then my kids could use it when I'm gone. That's what I just said. I was just talking about being, like, replacing you as a person. Uh, but now you get into all sorts of existential, like the use case you talked about, you're dead. Your kids want it. That could make sense. Like, maybe we were just talking about your worldview. How do you. Wait a minute, wait a minute. Maybe this brings us back to the ecosystem of death. Uh. Hang on. You guys are looking at it the negative way. Typically you usually do. Uh, what if what if that's what eternal life really means? That's my point is, that's what's the that's my point is that that brings up the whole existential. And you guys are gonna be able to watch this podcast forever. It's going to be around and, uh, for ever. That means you have us forever. Yeah. We could go down a lot of places. Right? Thank you. No, you're supposed to say you're welcome after that. No, no, I'm saying thank you, Mitch. You introduced us to this medium. We make videos and podcasts. And so now I will be eternal. Thank you. What will happen to the videos that have never been watched on YouTube? Anybody ever wondered that? Are there videos that have never been watched on YouTube? Like if it doesn't, I'm sure if a tree falls in the forest and nobody hears it. That's what I'm saying. Like I want to go watch them. Why is that? How can I? Right. Give me a list of videos that have never been watched. People on Reddit will find channels that have like twenty views and be like this. This guy's actually good at singing. And they'll just go bomb him and like, sub and comment and all of a sudden it blows up. And that's what I post a response. That's what I'm saying is like, if you never if, if like, will they, will they? If it's never been watched, will it stay forever? I don't know. Now we're into existential conversations. Well, your favorite Mike, even if it has been watched, will it stay forever? Somebody can always take it down. Once it's been watched, somebody has consumed it, and it will at least stay with them for some period of time. If nobody's ever watched it except for the person that created it, maybe. Right? Like, I don't know, this is an interesting sorry, we're way off the deep end right now, blowing your minds. Okay, so Matt is declaring there is not going to be an AI appliance that is sort of like a foregone conclusion. That's part of owning a house as you get your your AI appliance. And I think that's a fine, fine guess to make. Um, you also said we are not going to own one as a business, right? I don't know. I don't, I mean, our preference is not to do on like owned things like that, right. But then we bought a server. Yes, I know. Yeah. It was very limited. Very limited. I cannot it's you know, you never know what the future holds, but I cannot think of a scenario by which I'm like, yeah, you know, Matt Dressel is my large language model. Oh my God. Uh, this episode is, uh, is turning into a mess. It's Monday afternoon. We sort of second guessed recording today. So if, if you have that feeling listening to this where it's like, you guys, this feels like a Monday afternoon. Yeah. It's because it's a Monday afternoon. If you feel like this is the worst one we've done, just wait till the next one. Give us one more shot. No it's not. It's never the worst one we've done. It's just. Yeah, it's always the next one. Could be the one that doesn't get any views. Okay, let's keep all of this in the episode because this is really, really good. This is the most important part of the episode. Can someone wrap up just a really quick like you should consider this only if this is the case. Uh, yeah, so very unique purpose. Like think something physical, maybe that you, that you, that it touches or needs to process with, uh, very fast transactions like, like that type of stuff. Uh, security concern, like really hardcore security. Like I cannot let it out the door. Like no leaks. Uh, no other people that I don't know, having access to it, that kind of a thing. Um, and then the only other one would be like a cost thing. But I think that's probably the most difficult because you have to figure out how to do the math. And there's no simple equation for it for cost of ownership versus cost of credits. I don't know how many I'm going to spend this month. And that's tough because it's all dependent on what you're actually doing, the model that you're using, how much hardware it takes, how much storage you have to have to be able to do it in-house. Um, what would you and who do you have to hire to maintain it? Like it's very difficult. What would you consider the benefit of not being beholden to someone else's uptime and service level agreements? That means you're beholden to yourself. I know, so when snow hits and you're this person's in Michigan and you lose power, you're really, you can't even just go home and work from there. But that doesn't work at the office where the MacBooks Mac studio is. So the world we live in today, the. The cloud infrastructure is like the uptime is ninety nine point nine nine nine nine. Like it's pretty rare. And so as long as that continues and we don't start to live in a place where we have, like, rolling blackouts. Sure. Do it in the cloud. Okay. So those are all good. Uh, that's a good summary of who should use it and why. It's definitely a small piece of the pie chart. I think the other way we might summarize it is preppers. People who are preppers should have their own AI device. If you feel more comfortable and the only way you want to use AI is when you have control over it, that's just fine. Listen, I don't like, like, I'll be really clear. Like we, there are, uh, there are people who do research and other content that need to be able to train models and use models in a way that is non standard. And it can be, especially in the early days of doing that when I'm messing around, it can be extremely expensive to do that on cloud resources, because I'm going to try a billion times and I'm going to get it wrong, you know, three quarters of the time. And then the next time I'm going to get it the right way and all of a sudden it does what I want, right? So like, there are definitely legitimate reasons to do things in-house owned, but that's probably like half a percent of the scenarios if that. Yeah, like it's, it's just not that many. So set it up right next to your baked beans, your water and your Mrs.. I feel like you'll be connected to your generator. I feel like I need to apologize, uh, for Mitch, uh, for categorizing some large segment of the world in that prepper category. I did not I like, I like being generalized. Do you have a large can of baked beans in your basement? I don't have a large can of baked beans in my basement. Oh, then you're just fine. Even if you do have a large can of baked beans, you also are just fine. Yeah, just to be clear. Exactly. That's what I said. Anyway, that's it for today. Um, we're always interested in your feedback. This one went a little off the rails, but feel free to share any thoughts, comments, suggestions and we'll wrap this one up here. Thanks for talking about on premise AI models. And if you have, if you're using on premise AI models, we'd love to hear about it. Yeah. Let us comment. Let us know. I would, we are very interested or what people are using in the marketplace not to extend this thing, but if you're, if you want to have a discussion about, we'd love y one versus the other directly, we'd be very interested. Mike's phone number will be in the description. We'll see you next time. Bye, everybody.