Exo is a low-level language (and exocompiler) designed to help performance engineers write, optimize, and target high-performance computing kernels onto new hardware accelerators.
Exo is a domain-specific programming language that helps low-level performance engineers transform very simple programs that specify what they want to compute into very complex programs that do the same thing as the specification, only much, much faster.
The highest performance hardware made today (such as Google’s TPU, Apple’s Neural Engine, or Nvidia’s Tensor Cores) power key scientific computing and machine learning kernels: the Basic Linear Algebra Subroutines (BLAS) library, for example. However, these new chips—which take hundreds of engineers to design—are only as good (i.e. high performance) for application developers as these kernels allow.
Unlike other programming languages and compilers, Exo is built around the concept of exocompilation. Traditionally, compilers are built to automatically optimize programs for running on some piece of hardware. This is great for most programmers, but for performance engineers the compiler gets in the way as often as it helps. Because the compiler’s optimizations are totally automatic, there’s no good way to fix it when it does the wrong thing and gives you 45% efficiency instead of 90%.
With exocompilation, we put the performance engineer back in the driver’s seat. Responsibility for choosing which optimizations to apply, when, and in what order is externalized from the compiler, back to the performance engineer. This way they don’t have to waste time fighting the compiler on the one hand, or doing everything totally manually on the other. At the same time, Exo takes responsibility for ensuring that all of these optimizations are correct. As a result, the performance engineer can spend their time improving performance, rather than debugging the complex, optimized code.
Another key part of exocompilation is that performance engineers can describe the new chips they want to optimize for, without having to modify the compiler. Traditionally, the definition of the hardware interface is maintained by the compiler developers. However, for most new accelerator chips, the hardware interface is proprietary. It also changes more frequently than for general purpose chips. Currently, companies have to maintain their own fork of a whole traditional compiler, modified to support their particular chip. This requires hiring teams of compiler developers in addition to the performance engineers.
We’ve shown that we can use Exo to quickly write code that’s as performant as Intel’s hand-optimized Math Kernel Library. We also have an ongoing collaboration with UC Berkeley to create code for GEMMINI, their open-source machine learning accelerator. We’re actively working with engineers and researchers at a number of companies, and are eager to work with more!
Here is the talk we gave at PLDI 2022.
Working with Exo is easy! We’ll follow a standard workflow for creating a new Python project. Start by creating a virtual environment for your project. Note that Exo requires Python 3.9
$ python3 -m venv venv
$ . venv/bin/activate
Now, install exo-lang:
$ python -m pip install -U setuptools wheel
$ python -m pip install exo-lang
Let’s write a naive matrix multiply function in Exo. Put the following
code in a file called example.py:
# example.py
from __future__ import annotations
from exo import *
@proc
def example_sgemm(
M: size,
N: size,
K: size,
C: f32[M, N] @ DRAM,
A: f32[M, K] @ DRAM,
B: f32[K, N] @ DRAM,
):
for i in seq(0, M):
for j in seq(0, N):
for k in seq(0, K):
C[i, j] += A[i, k] * B[k, j]
And now we can run the exo compiler:
$ exocc -o out --stem example example.py
$ ls out
example.c example.h
These can either be compiled into a library (static or shared) or compiled directly into your application. You will need to write a short runner program yourself to test this code. For example:
// main.c
#include <stdio.h>
#include <stdlib.h>
#include <time.h>
#include "example.h"
float* new_mat(int size, float value) {
float* mat = malloc(size * sizeof(*mat));
for (int i = 0; i < size; i++) {
mat[i] = value;
}
return mat;
}
int main(int argc, char* argv[]) {
if (argc != 4) {
printf("Usage: %s M N K\n", argv[0]);
return EXIT_FAILURE;
}
int M = atoi(argv[1]);
int N = atoi(argv[2]);
int K = atoi(argv[3]);
if (M < 1 || N < 1 || K < 1) {
printf("M, N, and K must all be positive!\n");
return EXIT_FAILURE;
}
float* A = new_mat(M * K, 2.0);
float* B = new_mat(K * N, 3.0);
float* C = new_mat(M * N, 0.0);
const int n_trials = 1000;
clock_t start = clock();
for (int i = 0; i < n_trials; i++) {
example_sgemm(NULL, M, N, K, C, A, B);
}
clock_t end = clock();
int msec = (end - start) * n_trials / CLOCKS_PER_SEC;
printf("Each iteration ran in %d milliseconds\n", msec);
}
Then this can be easily compiled and run:
$ gcc -I out/ -o runner main.c out/example.c
$ ./runner 128 128 128
Each iteration ran in 11590 milliseconds
Now it’s time to write a schedule to make this fast. See our scheduling example for a tutorial on using AVX2 to optimize the innermost kernel.
If you answered “yes!” to all of three questions, then Exo might be right for you!
In particular, if you just want to optimize image processing code for consumer CPUs and GPUs, then Halide might be a better fit.
Exo is a young research project and under active development. Here are some of the things we’d like to add to Exo (or perhaps know we cannot).
So far, we have published the following papers on Exo: