Red Hat, Inc., the world’s leading provider of open source solutions, announced the latest release of Red Hat Enterprise Linux AI (RHEL AI), Red Hat’s foundation model platform for more seamlessly developing, testing and running generative artificial intelligence (gen AI) models for enterprise applications. RHEL AI 1.3 brings support for the latest advancements in the Granite large language model (LLM) family and incorporates open source advancements for data preparation while still maintaining expanded choice for hybrid cloud deployments, including the underlying accelerated compute architecture.
According to IDC’s “Market Analysis Perspective: Open GenAI, LLMs, and the Evolving Open Source Ecosystem,” 61% of respondents plan to use open source foundation models for gen AI use cases, while more than 56% of deployed foundation models are already open source1. Red Hat sees this trend validating the company’s vision for enterprise gen AI, which calls for:
- Smaller, open source-licensed models that can run anywhere and everywhere needed across the hybrid cloud.
- Fine-tuning capabilities that enable organizations to more easily customize LLMs to private data and specific use cases.
- Optimized and more efficient AI models driven by inference performance engineering expertise.
- The backing of a strong partner and open source ecosystem for broader customer choice.
RHEL AI forms a key pillar for Red Hat’s AI vision, bringing together the open source-licensed Granite model family and InstructLab model alignment tools, based on the Large-scale Alignment for chatBots (LAB) methodology. These components are then packaged as an optimized, bootable Red Hat Enterprise Linux image for individual server deployments anywhere across the hybrid cloud.
Also Read: Adaptive Shield Named the Frost Radar™ Leader in SaaS Security Posture Management
Support for Granite 3.0 LLMs
RHEL AI 1.3 extends Red Hat’s commitment to Granite LLMs with support for Granite 3.0 8b English language use cases. Granite 3.0 8b is a converged model, supporting not only English but a dozen other natural languages, code generation and function calling. Non-English language use cases, as well as code and functions, are available as a developer preview within RHEL AI 1.3, with the expectation that these capabilities will be supported in future RHEL AI releases.
Simplifying data preparation with Docling
Recently open sourced by IBM Research, Docling is an upstream community project that helps parse common document formats and convert them into formats like Markdown and JSON, preparing this content for gen AI applications and training. RHEL AI 1.3 now incorporates this innovation as a supported feature, enabling users to convert PDFs into Markdown for simplified data ingestion for model tuning with InstructLab.
Through Docling, RHEL AI 1.3 now also includes context-aware chunking, which takes into account the structure and semantic elements of the documents used for gen AI training. This helps resulting gen AI applications maintain better levels of coherency and contextually-appropriate responses to questions and tasks, which otherwise would require further tuning and alignment.
Future RHEL AI releases will continue to support and refine Docling components, including additional document formats as well as integration for retrieval-augmented generation (RAG) pipelines in addition to InstructLab knowledge tuning.
Broadening the gen AI ecosystem
Choice is a fundamental component of the hybrid cloud and with gen AI serving as a signature workload for hybrid environments, this optionality needs to start with the underlying chip architectures. RHEL AI already supports leading accelerators from NVIDIA and AMD, and the 1.3 release now includes Intel Gaudi 3 as a technology preview.
Beyond chip architecture, RHEL AI is supported across major cloud providers, including AWS, Google Cloud and Microsoft Azure consoles as a “bring your own subscription” (BYOS) offering. The platform is also available soon as an optimized and validated solution option on Azure Marketplace and AWS Marketplace.
RHEL AI is available as a preferred foundation model platform on accelerated hardware offerings from Red Hat partners, including Dell PowerEdge R760xa servers and Lenovo ThinkSystem SR675 V3 servers.
Model serving improvements with Red Hat OpenShift AI
As users look to scale out the serving of LLMs, Red Hat OpenShift AI now supports parallelized serving across multiple nodes with vLLM runtimes, providing the ability to handle multiple requests in real-time. Red Hat OpenShift AI also allows users to dynamically alter an LLM’s parameters when being served, such as sharding the model across multiple GPUs or quantizing the model to a smaller footprint. These improvements are aimed at speeding up response time for users, increasing customer satisfaction and lowering churn.
Supporting Red Hat AI
RHEL AI, along with Red Hat OpenShift AI, underpins Red Hat AI, Red Hat’s portfolio of solutions that accelerate time to market and reduce the operational cost of delivering AI solutions across the hybrid cloud. RHEL AI supports individual Linux server environments, while Red Hat OpenShift AI powers distributed Kubernetes platform environments and provides integrated machine-learning operations (MLOps) capabilities. Both solutions are compatible with each other, with Red Hat OpenShift AI will incorporate all of RHEL AI’s capabilities to be delivered at scale.
SOURCE: Businesswire