- Organizations are shifting to hybrid cloud and on-prem IT infrastructure for more control over data.
- This trend is driven by data privacy, security concerns, and AI's growing influence.
There's a change in the winds of IT infrastructure. After a decadeslong rush to the cloud, many organizations are exploring ways to exercise more control over their data and IT infrastructure.
Rather than depending solely on third-party cloud services, they're looking to operate their IT data networks on-site. This strategy, often referred to as on-premises or on-prem, is gaining renewed interest thanks to factors such as data privacy and security concerns, regulations designed to hold organizations accountable for the data they collect, and the rise of artificial intelligence.
The interest in on-prem doesn't represent a wholesale rejection of the cloud; many organizations are looking for more control from the cloud services they choose and seek to blend them with on-prem hardware.
"We're seeing more demand for on-prem," Hang Tan, the hybrid cloud chief operating officer of Hewlett Packard Enterprise, told Business Insider. "I don't think customers are thinking about it like it's a zero-sum game where either something goes to the public cloud or entirely on-prem. There's a recognition that customers need to have a more hybrid environment."
A push for on-prem
This trend is a response to concerns about the reliability, security, and cost-effectiveness of cloud services. But it's not a binary choice between hosting IT networks on the cloud and hosting them on-prem. Often the most important decision isn't about the physical location of an organization's infrastructure but about the level of security and control the organization is given over it.
"It's more about the governance of infrastructure, including data, networking, and servers," said Peter Wang, the cofounder and chief innovation officer of the software developer Anaconda. "Who manages it? Who gets to say, 'No, absolutely, you can't do that.'" His company, founded in 2012 to help organizations use Python for business analytics, found itself at the leading edge of this discussion as Python became a popular programming language for AI workloads.
Anaconda works with organizations that pursue a spectrum of solutions. Those seeking the most extreme security may operate "a box in a building somewhere, oftentimes guarded by people with machine guns, where you go in with a flash drive," Wang said. But that's applicable to only the most security-conscious industries and some public-sector customers.
Rather than placing infrastructure under lock and key behind a barbed-wire fence, Wang said, many organizations prefer to see cloud services include more detailed data-governance controls and low-level access to system resources. This can give organizations a level of control over cloud infrastructure that's more like what they'd have if the hardware were on-prem.
Dennis Duckworth, the head of product and end-user experience at Kindo, an AI-driven workforce-software developer, offered a similar perspective. Kindo provides a secure management platform for AI systems that lets organizations control which users have access to internal AI resources.
Duckworth said that some Kindo customers used an air-gapped on-prem strategy but that most were using, or looking to use, a secure and controllable solution that could be deployed within a cloud service such as Amazon Web Services. "They're not hosting it themselves, but we can deploy within that infrastructure," he said.
Perspectives on the trend differ depending on an organization's size and industry. Chris Stegh, the chief technology officer at the IT consulting company eGroup Enabling Technologies, said some sectors, like education and healthcare, pursued on-prem solutions for cost reasons.
"The commonalities of those organizations are they've got brick and mortar," Stegh said. "They've got sunk costs in their data center because they own some real estate."
AI is igniting changes in IT strategy
Tan said AI was "pouring rocket fuel" into the on-prem trend by driving a desire for greater control and better governance of data and IT infrastructure. While many organizations hope AI will improve efficiency, they've discovered it can require a rethink of how their infrastructure operates.
The particulars of IT infrastructure are often guided by the relationship between the applications people want to use and the data those applications require to function, Tan told BI. "With AI, because the data is dominating the equation, you have to move the application to the data, not the other way around," he explained.
Tan said that an HPE customer that runs a large retail business originally put infrastructure on-prem to allow operations to continue if internet connectivity was interrupted. Over time, the retailer expanded its capabilities with theft detection and inventory control powered by AI video analytics handled on hardware in stores.
These edge servers were "becoming AI-enabled cloudlets," or small-scale data centers that use graphics processing units to operate "a whole private cloud stack," Tan added.
In other cases, a company may need to take multiple approaches. Tan used self-driving vehicles as an example: Autonomous vehicles make decisions on local hardware, but the data is also sent to a data center, where it may be used to train new AI models.
Because of that, many organizations interested in AI need a hybrid cloud infrastructure that includes cloud and on-prem elements — and that can connect them when needed.
The role of regulation
Regulatory concerns are further strengthening interest in on-prem and hybrid cloud strategies. The landscape of data protection and AI regulation is evolving rapidly as different nations and states or jurisdictions within nations implement varying rules and requirements. Duckworth said this would lead to a complex regulatory future.
"We're going to see a patchwork of different regulations across the globe," he said. "It might be that in Germany you can't use ChatGPT, but you can use Mistral," he said. He also referenced recent legislation in Switzerland mandating that government agencies use open-source software.
Even cities may implement specific regulations. In 2021, Portland, Oregon, passed restrictions on facial recognition that applied to both public and private services. Other cities, such as San Francisco, have experimented with bans that apply to specific public services, such as police departments.
Duckworth argued that organizations were struggling to keep up with existing regulations, never mind those that might be implemented in the future. As more regulations arrive and are enforced, organizations could find themselves scrambling to adapt. "It will, in my personal opinion, be a madhouse for three to five years," he added.
Moving ahead with caution
AI might be the catalyst for a broader reexamination of how organizations manage their IT infrastructure. Cloud services aren't going away but may be approached with more skepticism and a greater focus on the long-term implications of adopting a particular service. The winners will be those that can bridge cloud and on-prem to provide the level of control organizations are demanding.
"My view is that customers that made the cloud journey learned some lessons. They know, 'Oh, if I move my data to the cloud, I'm kind of stuck,'" Tan told BI. He said he expects that organizations burned by that experience will be more skeptical of going all in on the cloud moving forward, adding momentum to interest in on-prem and hybrid cloud alternatives.