If you’ve been following our AI series, we have covered the art of the possible, reviewed how to get started, defined a use case, and assessed AI preparedness. At this point, we arrive at the first inflection point in the AI journey: Do we build the infrastructure needed to support our use case or consume it from a service provider?
As with most IT projects, the answer depends on a variety of factors, and it can be different with every use case.
- Infrastructure Availability
Do you have the necessary infrastructure to bring your use case to life? If you have some infrastructure deployed today, and you can use it, that certainly might be the easiest and lowest cost option.
The reality, however, is most organizations don’t have the infrastructure they need. AI workloads frequently require GPUs and specialized infrastructure to support the processing demand, and most organizations don’t have these hanging around unused.
For many companies, consuming AI services is going to be the easiest way to deploy a use case proof of concept (POC). This enables you to leverage infrastructure someone else has already built and allows you to test out your use case with minimal upfront investment.
- Cost
Cost is obviously an important consideration. This includes upfront costs and a forecast of the total cost of ownership (TCO). If you have to spend millions of dollars on infrastructure just to get started, that's probably cost prohibitive. Your TCO over time might be attractive, but your upfront costs would be incredibly painful for a use case that hasn’t been proven out yet.
Choosing a SaaS or consumption model might cost more over time because you’re paying for the investment made by the service provider and the ability to leverage the model’s flexibility and convenience, but the upfront cost would be much smaller. Make sure to do a thorough analysis though, because the TCO might be better or worse, depending on your use case and future plans.
- Manageability and Scalability
What are your future plans for AI? You might only be bringing a very small use case to life at first, but if the vision is for many different AI use cases, are you ready to do what it takes to scale it?
For example, let's say you were able to deploy your initial use case on some existing infrastructure. But to get to phase two, you're going to have to invest significant money. If you decide that you never want to scale the application, it may be better for you to simply choose to consume the infrastructure from the beginning, instead of deploying it on existing infrastructure.
Make sure to consider future state manageability and scalability upfront. Are you ready to start managing things like large GPU clusters, water-cooling, and extreme density racks, or are you really just looking to consume those things from a provider who specializes in them?
- Platform
Do you have a preferred platform? How important is platform flexibility to you? Similar to cloud, if you are concerned with getting locked-in to one vendor, you can choose to prioritize flexibility over convenience and cost efficiency. Investing with a single vendor, however, is typically more scalable and cost-effective, but that is a choice for you to make.
Vendor lock-in often gets a bad rap. After all, if you built your AI solution using one cloud provider’s native services for example, the solution would cost less and be less complex than if you built it yourself with multi-platform tools.
Going with one vendor means you are bought into the platform. This also means you are leveraging that vendor’s platform tools to the fullest extent, which likely will result in your solution being hyper-efficient, very secure, and cost optimized.
With AI, you will need to build into some vendor’s ecosystem, whether you deploy on your own infrastructure, use IaaS, or opt for a SaaS solution. There's no right or wrong answer. The key is to go in eyes wide open and make the right decision for your organization for your particular workload.
- Data Security and Governance
Data security and governance are paramount for AI, and you will have strict requirements for both. Your use case will have a specific subset of data that is defined as “in-scope” for the initiative. Where is that data located today? What security protections do you have in place and how well is that data tagged and governed? There are a lot of questions to unpack here. But suffice it say, the answer to these questions will be a key factor in helping you decide between building or consuming.
Taking the Next Best Step
Choosing to build infrastructure for AI or to consume it is an iterative process. The answer depends on the use case—what works for one use case may not work for another.
As you move forward in your AI journey, concentrate on one use case at a time. Don’t get overwhelmed thinking you have to make a decision for every AI use case in the beginning. If you decide to consume infrastructure at first, it doesn’t mean you have to consume infrastructure forever. The same is true for build.
Take each use case one at a time, prove them out, and then your options will become clearer. In my next blog, I will share tips on how to conduct a successful AI proof of concept (POC). Keep an eye out for it.
For help with any stage of your AI journey, ePlus offers a comprehensive set of services. Check out ePlus AI Ignite for more information.
