CROSS — Leveraging AI ASICs for Homomorphic Encryption

The talk presents CROSS and Morph, two compiler frameworks that enable existing AI accelerators, like Google's TPUs, to efficiently execute cryptographic workloads. CROSS focuses on Homomorphic Encryption (HE) and Morph on Zero-Knowledge Proofs (ZKP), demonstrating how to transform high-precision modular arithmetic into low-precision matrix operations that TPUs excel at, thereby achieving state-of-the-art performance and energy efficiency without any hardware modifications.

March 6, 2026 5 min read

CROSS — Leveraging AI ASICs for Homomorphic Encryption — source: https://www.youtube.com/watch?v=cN79ELoecNI

The talk explores how to leverage existing AI accelerators, specifically Google’s Tensor Processing Units (TPUs), to accelerate cryptographic workloads like Homomorphic Encryption (HE) and Zero-Knowledge Proofs (ZKP), bypassing the need for expensive, specialized hardware.

The Problem: Privacy vs. Performance in AI

Artificial Intelligence is driving an industrial revolution by tokenizing workflows, but this creates significant privacy risks, stalling its adoption. While Homomorphic Encryption (HE) offers a robust solution by enabling computation on encrypted data, it introduces prohibitive overhead, with performance degradation of up to 10,000x. Building dedicated HE hardware is costly and slow, creating a need for a more immediate and accessible solution.

The CROSS Framework: HE on TPUs

The core idea is to use existing, highly efficient AI ASICs like TPUs for HE. The project, named CROSS, is a compilation framework that transforms HE workloads into kernels that can be executed natively and efficiently on TPUs without any hardware modifications.

Why TPUs?

TPUs offer superior performance-per-watt compared to commodity hardware like GPUs and FPGAs. Their architecture is optimized for AI workloads and features:

Massive Compute: Including a Vector Processing Unit (VPU) for 32-bit operations and a high-throughput Matrix Multiplication Unit (MXU) for low-precision (e.g., 8-bit) matrix math. The MXU is roughly 100x faster than the VPU.
Massive Memory: Hundreds of GBs of HBM and hundreds of MBs of on-chip cache.
Flexible Data Reordering: A dedicated cross-lane unit (XLU) for efficient data shuffling.

Bridging the Compute and Memory Gaps

A fundamental mismatch exists between HE requirements and TPU capabilities. CROSS introduces two novel compilation techniques to bridge this gap:

Basis-Align Transformation (BAT) for Compute: This technique addresses the fact that TPUs lack native modular arithmetic and require lower precision than HE.
- It transforms high-precision (e.g., 28-59 bit) modular multiplication into a combination of low-precision (8-bit) dense matrix multiplications and 32-bit vector additions.
- The key insight is that high-order bits from intermediate multiplication results are temporary. Their contribution to the final modularly-reduced result can be pre-computed offline (at compile time) because the modulus is known.
- This “basis alignment” converts what would be an inefficient sparse matrix multiplication into a smaller, dense one, perfectly suited for the TPU’s high-performance MXU. Modular reduction is effectively replaced by a series of efficient vector additions, a form of “lazy reduction.”
Memory-Align Transformation (MAT) for Memory: This technique eliminates expensive, explicit data reorganization operations like transpose and shuffle.
- It embeds these data movements directly into the computation by pre-permutating the parameter matrices offline.
- For example, in a Number Theoretic Transform (NTT), the required data transpose and bit-reversal steps are factored into the twiddle factor matrices at compile time, making the runtime operation a series of efficient matrix multiplications without explicit data movement.

The evaluation of CROSS shows it achieves state-of-the-art throughput and energy efficiency, outperforming the best GPU implementations by up to 47% and FPGA solutions by two orders of magnitude.

The Morph Framework: Extending to Zero-Knowledge Proofs

The talk also introduces Morph, a framework that extends these concepts to accelerate ZKPs. ZKPs present an even greater challenge due to their extremely high precision requirements (256-700 bits) and their use of prime fields, which prevents the use of standard RNS decomposition.

Morph’s key innovation is to perform computations in a larger, non-prime “extended field” (F_m where m > q^2). Because this new modulus m is not prime, it can be factored, allowing the use of RNS-like techniques to decompose the high-precision arithmetic into linear-cost, low-precision operations.
The computation is then mapped to low-precision matrix multiplications that can be accelerated by the TPU’s MXU, with results later converted back to the original prime field.

Morph achieves up to a 100x speedup on high-precision cryptographic primitives like Multi-Scalar Multiplication (MSM).

Conclusion

Together, CROSS and Morph demonstrate that a new class of compilation techniques can successfully repurpose existing AI hardware for demanding cryptographic applications. They turn TPUs into state-of-the-art throughput machines for both HE and ZKPs, paving the way for scalable, low-cost, and immediately available privacy-preserving computation.