For several years now vendors have been re-tooling their product stacks, if not only their talk tracks, to endear themselves to a value proposition including AI. You seemingly can’t sit through any presentation on any topic without encountering it. And while prior to our immersion in services like ChatGPT, AI might have felt extraneous to many in the commercial enterprise, few could argue today that there is not an AI gold rush happening.
Across customer demographics the most popular AI projects remain the development of Agentic, GenAI, and LLM products. What is interesting though is the shift in market for how and where these projects are being built. While only a short period ago for the mid-market, AI was almost exclusively built in the public cloud (in fact the idea that AI would be built on-prem was anathema to the AI community) today we are seeing a growing trend of hybrid cloud deployment with substantial investment in on-premises solutions. IDC has recently identified that as many as 65% of AI workloads will move from the public cloud to private, sovereign AI infrastructure by 2027. The driving forces here are simple; cost, security, data leakage risks, and data locality. However, the solution has been anything but simple. The friction to the ambition of building on-prem AI is laden with complexity, inflexibility, and a legacy mindset. In fact, some analysts have quantified the percentage of failure in AI pilots to exceed 90% with many of these reasons identified as the culprit. Traditionally, vendors have been keen to merely deliver a skid of components to customers, adorned with the shiniest GPUs and accompanied with best wishes for an outcome that defies the failure trend. But a new strategy has emerged to change the narrative, a simplified AI vending machine. The AI Factory.
In general, an AI Factory is a reference architecture for on-premises AI, focused on simplifying the planning, deployment, and on-going management of the solution. The features of the offerings across the industry as you may suspect, vary greatly. This post aims to highlight the journey to date with HPE’s Private Cloud AI, and how it differentiates itself in a market inundated with pedestrian alternatives.
HPE & AI
HPE has a storied history in AI, with significant contributions to the underlying super-computing that has pioneered the modern iterations of the science, as well as a software ecosystem to advance and democratize access to its capabilities. In fact, a quick review of Top500.org’s list of supercomputers, will reveal the top three systems on the planet are HPE Cray systems, which is impressive enough on its own without considering that those three systems are actually faster than the remainder of the top twenty systems combined.
These computers deliver simulations in climate, nuclear and material science research, the fastest of which, El Capitan located at Lawrence Livermore National Laboratory, delivers more than 1.8 exaflops of performance (1.8 quintillion calculations per second).
More practically for most of us though, HPE has taken the lessons learned in their success in the extreme and distilled that into an AI Factory aimed at those of us with less than the compulsory half a billion dollars a modern supercomputer demands.
What is HPE Private Cloud AI
While HPE’s supercomputer AI Factories scale to 1000s of GPUs and are designed for the scientific research community, PCAI is focused on the mid-market, with an eye to simplicity and automation for customers requiring less than 100 GPUs in a single system.
Its core tenet is its focus on a cloud-like delivery for a sovereign AI system. It exists in HPE’s GreenLake portfolio alongside Private Cloud Enterprise, and Private Cloud Business edition; HPE’s private cloud virtualization platforms. And in the same chord as these solutions, PCAI delivers an orchestration and automation experience that divorces an AI developer from infrastructure management and allows them to focus on outcomes in their data pipelines. A combination of HPE ProLiant Servers, Alletra MP Unstructured Storage, HPE AI Essentials, and NVIDIA GPUs, networking & NVAIE; PCAI differs from other vendors AI factories in that it was not simply designed to work together, each system is fully integrated in the factory and tailored to a customer’s specific outcome. No skid of unassembled components, no empty best-wishes for the unlikeliest of outcomes. A serious step forward in the search for favorable effects in a sovereign AI package.
HPE Private Cloud AI delivers a cloud-like experience primarily through these features:
- A workload sized appliance delivery model reducing infrastructure sizing fatigue
- A catalogue delivered software ecosystem and updates to remove the never-ending DevOps cycle
- End to end turn-key solution co-designed with NVIDIA
- Embedded delivery from HPE’s data scientists and analytics experts to accelerate outcomes and measurably improve the opportunity for success
Time to first token (TTFT) has become a troublesome measurement for the success of AI projects. If we are in fact in an AI gold rush, a token would be the elusive nugget waiting to be discovered. An AI artifact as small as a character, word, or pixel, a token is the outcome of an AI model processing data. As organizations attempt to build their own bespoke AI and data systems, overoptimistic estimations in the amount of time it will take to unearth these intelligence deposits has seen many projects funding reallocated or cancelled altogether. The time to appropriately size and deliver an effective pilot environment with a transitionary strategy to production can be substantial, and as many organizations have realized, a business’s patience with AI projects that are not operational is not unlimited. Combine that with potentially hundreds of pieces of software in an average machine learning pipeline, and we find ourselves in territory that most general IT departments would struggle. In comparison with PCAI, similar customers have seen their delivery times for on-prem AI infrastructure solutions shrink from 18 months to weeks through the adoption of the HPE turnkey appliances, while accelerating their critical TTFT by a factor of as much as 4x.
With PCAI new AI models can be deployed in minutes from a GreenLake software dashboard hosted in HPE’s cloud. More importantly, customers can download packages from the latest distributions, connect them with critically sensitive information in a secure sandbox, and test and refire with new components and parameters within minutes. The speed and agility of the automation environment is one of the primary cloud-like enablement features. This is reinforced with the included NVIDIA NVAIE ecosystem. Customers can securely download new models from NVIDIA NIM, Hugging Face, or if desired include code from the latest emerging open-source providers. The open design is a key aspect of the experience providing a look and feel comfortable with cloud developers, with the security of NVIDIAs NeMO AI guardrails.
Beyond compiling the successful recipe of hardware and software, a key quotient in the viability of any project is access to critical expertise. With many ambitious organizations just starting on their journey in AI, this remains one of the biggest hurdles. The final component in the PCAI success equation is the inclusion of HPE AI specialists as a functional part of every deployment. The understanding of how to extract important data features, combined with the experience of what the important data features are, make this a crucial deliverable. One of the biggest markers for success in this space is having done it before, when you are just starting out, this is a problematic and expensive risk to undertake on your own.
In short, PCAI de-risks AI projects through the delivery of a tailored solution environment and includes a comprehensive software ecosystem in an on-demand cloud delivery model that enables rapid deployment of new AI applications, and drastically reduces time to production.
HPE Private Cloud AI: The next chapter
Having just announced new hardware support for next generation compute and GPU platforms, as well as a disconnected variant for customers whose mandate requires containment of systems even for the management and workflow of environments, there is a lot of new capabilities to get excited about with PCAI for the coming year. Perhaps the most exciting evolution though is the addition of HPE’s Unleash AI portfolio. A collection of AI solutions with unique cross-industry capabilities, accessible and ready to deploy within a customer’s HPE Private Cloud AI system. I recently caught up with Robin Braun, HPE’s Worldwide VP of AI Business Development and champion of the Unleash AI community to hear more about her perspective on HPE’s position within the market and what she envisions is next for the program.
According to Robin, a primary challenge in the industry, especially within sovereign AI, is the ability to seamlessly integrate AI applications from multiple vendors quickly. Unleash AI aims to solve this. It introduces a curated ecosystem of AI use cases which have been verified within HPE’s AI platform. From Agentic AI to Vision AI to enhanced coding assistance, options abound for next generation creation including direct plugins to industry standard business applications for simplified integration with operational procedures. This is the magic of the Unleash AI portfolio; it extends the turn-key experience of PCAI to include our partner ecosystem, and just as importantly allows customers to transact the entire solution in a single process from HPE or a channel partner.
“Simply working with AI partners isn’t enough, we need to extend the experience to include the last mile, AI private cloud to outcome, in a single package.”
Robin shared that most customers continue to partner with readily available off-the-shelf AI software solutions, however the expected ease of deploying these AI packages can be met with unexpected friction when partners then need to validate a customers entire AI ecosystem before they can get started. This often requires changes, and incurs lost time. This is a big differentiator for HPE. Their partners have pre-qualified the solution, enabling customers to proceed with confidence and agility.
With nine months of racetrack now behind the launch of Unleash AI, Robin is excited that HPE has introduced a “practical, tactical, and accessible AI solution that accelerates customer outcomes.” This also exponentially increases the value of a private cloud AI suite as customers typically have a “mode 1” AI plan that is the anchor incubator, but Unleash AI allows customers to have rapid successive wins by leveraging their new PCAI environment for additional applications. Robin identified we are now in the “Acceleration Phase” having successfully launched 48 partners with plans to continue growth, and the corollary relevance of the solution.
Final Thoughts
With the very real challenges in the delivery of public cloud fractional GPU services in a cost-effective envelope, and scrutiny over the use of public GenAI and RAG features along with proprietary data, sovereign AI strategies will continue to be at the center of consideration for customers. The axiomatic truth though is that everyone pursuing AI outcomes has very real expectations that they will be delivered with cloud primitives intact. Automated provisioning, with menu driven functionality to test and evaluate the newest emerging models at the speed of innovation, and importantly, without being encumbered by the realities of traditional IT.
Leave a Reply