SURF partners with Cerebras Systems to accelerate computing and AI research workflows
In early 2023, SURF joined forces with Cerebras Systems—a pioneer company in AI technology—to conduct a technology assessment of the unique CS-2 wafer-scale engine (WSE). The CS-2 exhibits outstanding capabilities; many of its specifications exceed that of other general-purpose processors in orders of magnitude, unleashing common bottlenecks.
The partnership focused on exploring and understanding the potential of the WSE for AI and scientific computing workloads with the ultimate goal of facilitating access to Cerebras systems for Dutch researchers in the future. Cerebras provided SURF with access to their simulator, documentation, and physical hardware through the Cirrascale cloud provider. This trial offered SURF a unique opportunity to delve deep into the exceptional capabilities of the Cerebras engine. The assessment was led by two SURF teams: high-performance machine learning (HPML) and computing and visualization (HPCV).
Collaboration background
Cerebras Systems Inc. is an artificial intelligence company that builds computer systems for complex artificial intelligence and deep learning applications. Their key product, the CS-2 wafer-scale engine (WSE), delivers the wall-clock computing performance of many tens to hundreds of graphics processing units (GPU) or more. In one system less than one rack in size, the CS-2 delivers answers in minutes or hours that would take days, weeks, or longer on large multi-rack clusters of general-purpose processors.
The Cerebras CS-2 stands out with its unique design and dataflow architecture. It comprises 850,000 independent processing elements (PEs) implemented as a two-dimensional rectangular mesh on a single silicon wafer, respectively. PEs consists of a processor, a router, and local tile memory, collectively forming the fabric. Data flows through the mesh of PEs in 32-bit packet wavelets triggering data transformations. With 48 KB of dedicated memory per PE, the CS-2 enables efficient read and write operations of up to 16 and 8 bytes per cycle. The interconnect fabric between PEs offers an injection bandwidth of 16 bytes per core per cycle.
Developing kernels for the CS-2 involves utilizing the Cerebras-specific CSL language. The code written in CSL uses tasks to define the operations performed by PEs on data. Data wavelets are bound to tasks based on their color. Tasks can be assigned to groups of PEs, named rectangles, for task execution. In the background, the task scheduler determines the order of execution of tasks during compilation.
Goal partnership
To comprehend such unconventional hardware, SURF partnered with Cerebras to conduct a technology assessment that focused on exploring the capabilities of the unique CS-2. The collaboration commenced with a conversation in late 2022 and culminated in the execution of a non-disclosure agreement. Cerebras shared their simulator and documentation and, from February 2023, provided trial access to the physical hardware in the Cirrascale cloud provider. The trial access granted SURF seven allocations of 24 hours for three months, providing an opportunity to explore the capabilities of the Cerebras CS-2. Two SURF teams, the high-performance machine learning (HPML) and computing and visualization (HPCV), engaged in the assessment driven by their interest in exploring state-of-the-art technologies.
This collaborative effort had a dual purpose. First, to understand the potential of the Cerebras system’s capabilities for both artificial intelligence and scientific computing workloads. On the other hand, to establish a cooperative relationship that facilitates access to Cerebras systems for the broader Dutch research community.
After the completion of the initial assessment, SURF and Cerebras plan to strengthen the partnership. Future endeavors include identifying new members and their respective use cases to onboard them into the CS-2 platform. Additionally, SURF aims to provide access to CS-2 through the SURF Research Cloud soon, ensuring that the benefits of this partnership continue to resonate and empower the broader research community.
Interested?
SURF's Experimental Technologies Platform has a dedicated team of expert system administrators and advisors. Are you interested in co-creating and experimenting with emerging technologies and methodologies? Then share your innovative challenges with us!