StarryOS K230 NNCase Runtime Demo

Chinese version: k230-kpu-nncase-runtime.zh.md.

This document records the K230 KPU/NPU runtime demo added for the StarryOS K230 QEMU path. The operator-facing app lives under apps/starry/k230-kpu-nncase; the test-suit/starryos case is a CI-oriented wrapper that reuses the same app sources. The demo is intentionally separated from the lower-level /dev/kpu smoke case: kpu-smoke proves the device interface, while kpu-nncase-runtime proves that a real .kmodel can be loaded by the official NNCase runtime inside the StarryOS guest.

Goal

The runtime path is:

yolov8n_320.kmodel
  -> K230 NNCase interpreter in StarryOS userspace
  -> KPU command stream generated by the runtime
  -> StarryOS /dev/kpu ioctl/mmap
  -> QEMU K230 KPU model
  -> done/IRQ and output tensor hashes

The important distinction from .krun replay is that command generation happens inside the StarryOS guest. The replay path still remains useful as a stable fallback for a captured 54-command workload, but it is not the primary runtime evidence in this demo.

Local Assets

The runtime demo intentionally does not commit large or third-party assets:

yolov8n_320.kmodel
bus.jpg
K230 SDK NNCase static libraries and headers
K230 SDK C++ toolchain
prebuilt StarryOS guest demo binaries

These assets come from the official Kendryte K230 SDK and are prepared under target/official-k230, which is outside the tracked source tree.

1. Prepare the official K230 SDK

From the tgoskits repository root:

mkdir -p target/official-k230
git clone https://github.com/kendryte/k230_sdk \
  target/official-k230/k230-sdk-src

The official SDK README also lists a release tarball mirror. If GitHub cloning is slow, unpack that release tarball into the same directory:

target/official-k230/k230-sdk-src/

Then download the SDK toolchain, NNCase package, utility package, and kmodel package with the SDK's own preparation target:

docker run --rm --platform linux/amd64 -u root \
  -v "$PWD/target/official-k230/k230-sdk-src":/k230_sdk \
  -v "$PWD/target/official-k230/k230-sdk-src/toolchain":/opt/toolchain \
  -w /k230_sdk \
  ghcr.io/kendryte/k230_sdk:latest \
  bash -lc 'make prepare_sourcecode'

The --platform linux/amd64 part is intentional: the SDK toolchain used by this demo is the SDK's x86_64-hosted RISC-V Linux musl toolchain.

After make prepare_sourcecode, these files must exist:

target/official-k230/k230-sdk-src/
  toolchain/riscv64-linux-musleabi_for_x86_64-pc-linux-gnu/bin/riscv64-unknown-linux-musl-g++
  src/big/nncase/riscv64/nncase/lib/libNncase.Runtime.Native.a
  src/big/nncase/riscv64/rvvlib/
  src/big/utils/lib/hhb-prebuilt-decode/
  src/big/kmodel/ai_poc/kmodel/yolov8n_320.kmodel
  src/big/kmodel/ai_poc/images/bus.jpg

The expected reproducible source for the model and image is the official SDK path above. The demo CMake file also accepts explicit K230_KMODEL and K230_BUS_JPG paths for local review or classroom environments.

2. Build the StarryOS guest demo binaries

This PR does not commit the real model, SDK libraries, or prebuilt binaries. Prepare them locally with:

bash apps/starry/k230-kpu-nncase/c/tools/build-nncase-runtime-binaries.sh

The script:

finds target/official-k230/k230-sdk-src from the current worktree;
copies the StarryOS dev image's /opt/riscv64-linux-musl-cross sysroot into a Docker volume named tgoskits-riscv64-linux-musl-cross;
enters ghcr.io/kendryte/k230_sdk:latest as an amd64 container;
links the demo programs against the K230 SDK NNCase runtime, RVV library, JPEG decode library, K230 SDK C++ runtime, and the Linux musl sysroot;
places ignored riscv64 demo binaries under:

apps/starry/k230-kpu-nncase/c/assets/bin/
  kpu-nncase-minimal
  k230-yolov8n-demo

If a reviewer already has equivalent binaries from another trusted build, they can place those two files in the same ignored directory. The test case will use them without rebuilding from source.

3. What `cargo xtask` installs into the guest

When the kpu-nncase-runtime case runs, CMake installs:

/usr/bin/kpu-nncase-minimal
/usr/bin/k230-yolov8n-demo
/usr/bin/k230-nncase-runtime-demo
/usr/share/k230-nncase-runtime/models/yolov8n_320.kmodel
/usr/share/k230-nncase-runtime/images/bus.jpg

The model and image are copied from the SDK paths listed in step 1. The two demo binaries are copied from apps/starry/k230-kpu-nncase/c/assets/bin/ unless the case is being built directly inside an amd64 K230 SDK environment with -DK230_CXX=....

4. K230 QEMU itself

The runtime case also requires the K230 QEMU fork described in apps/starry/k230-qemu/README.md. Prepare it with:

bash apps/starry/k230-qemu/prepare-k230-qemu.sh

Then put that QEMU build before the default QEMU path when running the case:

PATH="$PWD/target/qemu-k230-docker-build:$PATH" \
  cargo xtask starry app qemu -t k230-qemu/qemu-k230/kpu-nncase-runtime --arch riscv64

Validation

Run the QEMU case:

cargo xtask starry app qemu -t k230-qemu/qemu-k230/kpu-nncase-runtime --arch riscv64

The expected evidence includes:

NNCASE_MINIMAL: load_model ok
NNCASE_MINIMAL: model io inputs=1 outputs=4
K230_SDK_COMPAT: identity mmap l2 0x80000000..0x80200000
K230_SDK_COMPAT: mirrored runtime rdata 0x3c000020 -> 0x10000020
K230_SDK_COMPAT: gnne_enable run=54
NNCASE_MINIMAL: interp.run done
NNCASE_MINIMAL_PASS
YOLOV8N_DEMO_PASS
K230_NNCASE_RUNTIME_PASS
all starry k230-qemu qemu tests passed

For a teacher-facing terminal demo, use:

bash apps/starry/k230-kpu-nncase/demo-teacher.sh

The script streams the full QEMU/Cargo output to the terminal and saves the same output under target/k230-kpu-demo/teacher-nncase-runtime.log. It prints a short evidence summary after the case finishes.

The app can also be run through the Starry app runner:

PATH="$PWD/target/qemu-k230-docker-build:$PATH" \
  cargo xtask starry app qemu -t k230-kpu-nncase --arch riscv64

The migrated K230 app-QEMU teacher script path is:

bash apps/starry/k230-qemu/qemu-k230/demo-teacher.sh

Current Boundary

Completed:

K230 runtime demo binary runs inside the StarryOS guest.
The real yolov8n_320.kmodel is loaded by NNCase runtime.
The runtime generates KPU commands and submits them through /dev/kpu.
StarryOS observes KPU done/IRQ and nonzero output tensor hashes/stats.

Not completed in this PR:

Full YOLO detection-box semantic alignment with the official RT-Smart reference.
Camera/VICAP/VO or full K230 MPP integration.
Committing the real .kmodel, SDK static libraries, or prebuilt demo binaries.

If detections=0 appears in the demo output, it does not mean the KPU did not run. It means the remaining work is application-level YOLO output interpretation and postprocess alignment, while the runtime/device path has already completed.

Goal​

Local Assets​

1. Prepare the official K230 SDK​

2. Build the StarryOS guest demo binaries​

3. What cargo xtask installs into the guest​

4. K230 QEMU itself​

Validation​

Current Boundary​