Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
---
title: Run ExecuTorch Llama 3.2 1B Instruct on a Vivo X300 Pro with Vulkan

description: Learn how to export Llama 3.2 1B Instruct with ExecuTorch, build the Android Vulkan runtime, and run the model on a Vivo X300 Pro.

minutes_to_complete: 120

who_is_this_for: This Learning Path is for developers who want to export a Llama 3.2 model with ExecuTorch and run it on an Android phone through the Vulkan backend.

learning_objectives:
- Set up a Linux host for Android cross-compilation, ADB deployment, and Vulkan-backed ExecuTorch builds.
- Pin the host Python and PyTorch environment required by the ExecuTorch 1.4 release branch.
- Export Meta Llama 3.2 1B Instruct to a Vulkan-enabled `.pte` with the exact quantization settings used in the guide.
- Build, deploy, validate, and troubleshoot the Android `llama_main` runner on a Vivo X300 Pro.

prerequisites:
- A Linux host with enough disk space for the ExecuTorch source tree, Android SDK and NDK, Vulkan SDK, and the Llama checkpoint.
- A Vivo X300 Pro or a similar Android phone with USB debugging enabled.
- Access to the gated Hugging Face repository `meta-llama/Llama-3.2-1B-Instruct`.
- Working familiarity with the shell, Python virtual environments, and Android ADB workflows.

author: Ash Naik

skilllevels:
- Advanced

subjects:
- ML

operatingsystems:
- Linux
- Android

tools_software_languages:
- ExecuTorch
- PyTorch
- Python
- Android SDK
- Android NDK
- ADB
- Vulkan
- glslc
- Hugging Face
armips:
- Cortex-A
- Mali

further_reading:
- resource:
title: ExecuTorch repository
link: https://github.com/pytorch/executorch
type: documentation
- resource:
title: ExecuTorch Llama documentation
link: https://github.com/pytorch/executorch/blob/main/docs/source/llm/llama.md
type: documentation
- resource:
title: ExecuTorch Android Vulkan documentation
link: https://github.com/pytorch/executorch/blob/main/docs/source/android-vulkan.md
type: documentation

# ================================================================================
weight: 1 # _index.md always has weight of 1 to order correctly
layout: "learningpathall" # All files under learning paths have this same wrapper
learning_path_main_page: "yes" # This should be surfaced when looking for related content. Only set for _index.md of learning path content.
---

## About this Learning Path

This Learning Path turns the material from `ExecuTorch_Llama32_Vivo_X300Pro_End_to_End_Guide.docx` into an Arm-style walkthrough. It keeps the exact versions, commands, and measured results from the successful run, while reorganizing them into a sequence you can follow from environment setup through validation.

### Who is this for?

This topic is for developers targeting Android Arm64 devices who want to run an LLM locally with ExecuTorch and the Vulkan backend instead of CPU-only inference.

### What will you learn?

Upon completion of this Learning Path, you will be able to:

- Prepare the Android SDK, NDK, ADB, and host Vulkan toolchain required for cross-compiling ExecuTorch.
- Fix the PyTorch version mismatch that can break ExecuTorch 1.4 builds on the host.
- Export Llama 3.2 1B Instruct to a Vulkan-ready `.pte` using the same `8da4w` and KV-cache settings as the measured run.
- Build the Android runtime and `llama_main`, deploy them to the phone, and confirm Vulkan-backed execution.

### Prerequisites

Before starting, you will need the following:

- A Linux machine that can build native and Android targets.
- Android Studio or an equivalent Android SDK and NDK installation path.
- A phone connected over USB with developer mode and USB debugging enabled.
- Hugging Face access to download `meta-llama/Llama-3.2-1B-Instruct`.
- Sufficient storage for a multi-gigabyte checkpoint and exported `.pte`.
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
---
# ================================================================================
# FIXED, DO NOT MODIFY THIS FILE
# ================================================================================
weight: 9
title: "Next Steps"
layout: "learningpathall"
---

Use the metadata in `_index.md` to populate the standard Arm Learning Paths next-steps experience when this content is imported into a Hugo tree.

If you extend this guide, the most useful follow-on topics are:

- compare Vulkan against XNNPACK or other Android backends on the same phone
- rebuild with ETDump and capture delegated-region timings
- retune quantization, context length, and prompt format for memory or throughput
- adapt the same flow to a different Android device with a different GPU and driver stack
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -0,0 +1,139 @@
---
title: Prepare the host and device
description: Install the Android toolchain, configure ADB access, and verify Vulkan support on the target phone.
weight: 3

### FIXED, DO NOT MODIFY
layout: learningpathall
---

## Install Android SDK and NDK

Use Android Studio or the SDK Manager to install the following components:

- Android SDK Platform-Tools
- Android SDK Command-line Tools
- CMake
- NDK (Side by side)

This workflow used Android NDK `r28c`:

```text
28.2.13676358
```

Typical locations:

```text
$HOME/Android/Sdk
$HOME/Android/Sdk/ndk/28.2.13676358
```

Validate the NDK layout:

```bash
ls "$HOME/Android/Sdk/ndk/28.2.13676358"
test -f "$HOME/Android/Sdk/ndk/28.2.13676358/NOTICE" && echo "NDK OK"
test -f "$HOME/Android/Sdk/ndk/28.2.13676358/build/cmake/android.toolchain.cmake" && echo "Toolchain OK"
```

## Persist Android environment variables

Add the SDK and NDK paths to your shell startup file:

```bash
cat >> ~/.bashrc <<'EOF'
# Android SDK / NDK
export ANDROID_HOME="$HOME/Android/Sdk"
export ANDROID_NDK="$ANDROID_HOME/ndk/28.2.13676358"
export PATH="$ANDROID_HOME/platform-tools:$PATH"
export PATH="$ANDROID_HOME/cmdline-tools/latest/bin:$PATH"
EOF

source ~/.bashrc
```

Verify the configuration:

```bash
echo "$ANDROID_HOME"
echo "$ANDROID_NDK"
adb --version
test -f "$ANDROID_NDK/NOTICE" && echo "NDK OK"
test -f "$ANDROID_NDK/build/cmake/android.toolchain.cmake" && echo "Toolchain OK"
```

## Connect the Vivo over ADB

If `adb devices` shows `no permissions`, add the user to `plugdev`, install the generic Android udev helpers, and reload the rules:

```bash
sudo usermod -aG plugdev "$USER"
sudo apt install -y android-sdk-platform-tools-common
sudo udevadm control --reload-rules
sudo udevadm trigger
newgrp plugdev
```

The tested device reported this USB ID:

```text
Bus 002 Device 002: ID 2d95:6001 vivo vivo X300 Pro
```

Add a Vivo-specific rule if the default rules are not enough:

```bash
sudo tee /etc/udev/rules.d/51-vivo-android.rules >/dev/null <<'EOF'
SUBSYSTEM=="usb", ATTR{idVendor}=="2d95", MODE="0660", GROUP="plugdev", TAG+="uaccess"
EOF

sudo chmod 644 /etc/udev/rules.d/51-vivo-android.rules
sudo udevadm control --reload-rules
sudo udevadm trigger
```

Then restart ADB and accept the RSA prompt on the phone:

```bash
adb kill-server
adb start-server
adb devices
```

Expected final state:

```text
10AFB40J6Q0031C device
```

## Verify Vulkan support on the phone

Check the Vulkan implementation:

```bash
adb shell getprop ro.hardware.vulkan
```

The measured device returned:

```text
mali
```

Then verify the relevant Android features:

```bash
adb shell pm list features | grep -i vulkan
```

Expected features include:

```text
feature:android.hardware.vulkan.compute
feature:android.hardware.vulkan.level=1
feature:android.hardware.vulkan.version=4206592
feature:android.software.vulkan.deqp.level=132711169
```

`vulkan_renderengine: false` from SurfaceFlinger does not block application-side Vulkan compute. The more important signal is that the device advertises `android.hardware.vulkan.compute`.
Loading