engineeringactive

GPU / HPC Performance Engineering

Custom CUDA and Rust kernels for inference workloads: measuring where latency and cost actually come from, then removing it.

Current objective
Reduce inference cost through kernel optimization.

68%

Current work

Rust + CUDA Inference Benchmark

42%

Benchmarking custom CUDA kernels against the current baseline.

Current task: Profiling kernel memory access and occupancy
Next task: Implement shared-memory optimization

baseline Latency Ms: 8.7
current Latency Ms: 7.1

gpucudarustbenchmark

Progress history

42%21 Augpublic
Memory bottleneck identified
Occupancy is fine; the kernel is stalling on uncoalesced global reads. Next pass moves the tile into shared memory.
latency Ms
7.1
baseline Ms
8.7
25%19 Augpublic
Baseline benchmark running
Reproducible harness in place: fixed batch shapes, warmup, 200 measured iterations, p50/p99 reported.
baseline Ms
8.7