EdgeKernelBench: Unified Benchmark and LEKA Agent for LLM-Driven Kernel Generation on Heterogeneous Edge Backends

Background and the Problem

While LLM-driven generation of high-performance tensor kernels (using Triton, CUDA, and TileLang) has advanced rapidly in cloud datacenters, deploying such methodologies to edge hardware (mobile SoCs, embedded NPUs, and robotics micro-controllers) presents severe roadblocks. Datacenter GPUs benefit from standardized micro-architectures and mature toolchains; edge environments, conversely, feature extreme hardware heterogeneity, heavily fragmented driver ecosystems (OpenCL, proprietary NPU toolchains, embedded CUDA), and punishing memory-bandwidth constraints. Existing benchmarks (such as KernelBench) focus almost exclusively on server-class accelerators. In arXiv:2610.24648, researchers addressed this vacuum by introducing EdgeKernelBench and the companion LEKA agent architecture.

Architecture and How It Works

1. Unified IR Hierarchical Benchmark Pipeline:

• Transcends fragmented platform-specific C/C++ variants by anchoring generation on unified Intermediate Representation (IR) abstraction using production compiler stacks (such as TVM);
• Contains 331 canonical PyTorch computation patterns spanning 650 rigorously verified task instances across multiple complexity tiers;

1. Comprehensive Heterogeneous Backend Evaluation:

• Executes across real-world edge hardware configurations: CPU/LLVM, GPU/CUDA, and cross-vendor GPU/OpenCL environments;

1. Layered Edge Kernel Agent (LEKA):

• Identifies the critical failure of naive single-shot generation (frontier models achieved 73.5% compilation but only 25.5% reached parity with expert baseline speed [email protected]);
• LEKA vertically decouples complex kernel scheduling decisions along the IR hierarchy, assigning specialized sub-agents to loop tiling, memory staging, and vectorized hardware alignment.

Benchmarks and Measured Results

Evaluated across five frontier coding LLMs on physical edge testbeds (Section 4, Table 2, Figure 4):

• Compilation & Functional Correctness:
• Frontier models achieved an average 73.5% Compile@1 and 64.5% Correct@1;
• However, the competitive speed threshold ([email protected]) lagged at a meager 25.5%, exposing LLMs' inherent blind spots regarding physical cache topologies;
• LEKA Hierarchical Acceleration:
• Deploying the multi-agent LEKA framework more than doubled high-performance kernel yield, delivering human-competitive speedups across diverse OpenCL and CUDA edge profiles.

Getting Started for Developers

• Paper & Benchmark Access: Documented on arXiv:2610.24648;
• Engineering Implications: Edge AI optimization teams should replace manual bespoke kernel tuning with the unified IR protocol defined in EdgeKernelBench. Coupling hierarchical multi-agent frameworks like LEKA with compiler IRs enables rapid, automated saturation of edge silicon bandwidth.