Thirty-two bytes at once

SIMD tutorials love to open with a tour of register names. Skip that part. Your first SIMD program should start with a loop so boring that no iteration has any reason to care what the others did.

Say you're comparing two grayscale image buffers with a sum of absolute differences. Each pixel pair produces one number, and nothing that happens at pixel 4,000 depends on pixel 3,999 — which is about as SIMD-shaped as work gets.

pub fn sad_scalar(a: &[u8], b: &[u8]) -> u64 {    assert_eq!(a.len(), b.len());    a.iter()        .zip(b)        .map(|(&x, &y)| x.abs_diff(y) as u64)        .sum()}

Keep this function even after you've written the fast one. It's the fallback for machines without vector support, and it's the oracle that tells you whether the clever version still computes the same answer.

Vectorize the loop, not the story#

SIMD means one instruction operating on several lanes at once. AVX2 gives this example a 256-bit register, so a single load carries 32 u8 values, and the _mm256_sad_epu8 intrinsic computes all 32 absolute byte differences and hands back four partial sums.

Before writing a single intrinsic, benchmark an optimized scalar build. Rustc already runs loop and SLP vectorization passes, and -C target-cpu=native lets a local benchmark target the exact processor underneath it. More than once I've written the kernel first and then discovered the autovectorizer had gotten there already. Sometimes the boring loop already won.

Fence the unsafe#

Don't compile the whole program for AVX2 unless every machine you deploy to actually has it. Dispatch once, and keep the architecture-specific code behind one private function.

pub fn sad(a: &[u8], b: &[u8]) -> u64 {    assert_eq!(a.len(), b.len());    #[cfg(target_arch = "x86_64")]    if std::is_x86_feature_detected!("avx2") {        // SAFETY: AVX2 was checked above.        return unsafe { sad_avx2(a, b) };    }    sad_scalar(a, b)}

The runtime feature check is what makes the unsafe call sound: the public function stays portable, and a machine without AVX2 quietly takes the scalar path instead.

#[cfg(target_arch = "x86_64")]#[target_feature(enable = "avx2")]unsafe fn sad_avx2(a: &[u8], b: &[u8]) -> u64 {    use std::arch::x86_64::*;    let mut sums = _mm256_setzero_si256();    let mut i = 0;    while i + 32 <= a.len() {        let x = unsafe { _mm256_loadu_si256(a.as_ptr().add(i).cast()) };        let y = unsafe { _mm256_loadu_si256(b.as_ptr().add(i).cast()) };        sums = _mm256_add_epi64(sums, _mm256_sad_epu8(x, y));        i += 32;    }    let mut lanes = [0_u64; 4];    unsafe { _mm256_storeu_si256(lanes.as_mut_ptr().cast(), sums) };    lanes.into_iter().sum::<u64>() + sad_scalar(&a[i..], &b[i..])}

The loop loads 32 bytes from each slice. The unaligned load takes ordinary slice addresses, so you don't have to redesign your allocation to make it happy. Four partial sums accumulate in the vector as the loop runs, and whatever bytes are left over after the last full block go through the scalar oracle.

Stable Rust exposes these target-specific intrinsics through std::arch. The nicer std::simd API is still nightly-only in Rust 1.97, which is a shame. But it also makes containment worth the trouble: keep the ugly code private now and you can swap the kernel out later without touching a single caller.

Prove the edges#

SIMD bugs hide at the widths: zero, one, 31, 32, 33. Every one of these bugs I've written lived in the tail. Test the optimized function against the scalar one across every remainder before you spend any time admiring the assembly.

#[cfg(target_arch = "x86_64")]#[test]fn avx2_handles_every_tail() {    if !std::is_x86_feature_detected!("avx2") {        return;    }    for len in 0_usize..97 {        let a = (0..len).map(|i| (i * 17) as u8).collect::<Vec<_>>();        let b = (0..len).map(|i| (i * 29 + 7) as u8).collect::<Vec<_>>();        let got = unsafe { sad_avx2(&a, &b) };        assert_eq!(got, sad_scalar(&a, &b), "len={len}");    }}

Then benchmark at more than one size. A buffer smaller than a single vector shows you what the dispatch costs; a buffer too big for cache tells you what the memory bus thinks of your kernel.

cargo test --releaseRUSTFLAGS="-C target-cpu=native" cargo bench

Use target-cpu=native only when the benchmark binary runs on that same machine — a binary you ship to other people still needs the runtime dispatch. And compare throughput, not register width. A wider instruction can still lose, to dispatch overhead or to a memory bus that was never going to feed it that fast.

From the outside, none of this should look interesting. Callers see one safe function, the unsafe kernel stays behind its fence, and the scalar loop keeps its job as the answer key. If the benchmark can't find a real speedup, delete the kernel and move on; the loop was always correct.