farm_uf850_home_pi05

farm_uf850_home_pi05

A π0.5 vision-language-action policy fine-tuned for the UFactory UF850 arm on the farm_uf850_home tabletop manipulation task. It takes two camera views (base + wrist), the arm's joint state, and a language prompt, and predicts 50-step action chunks.

Training summary

One GPU, about five and a half hours, roughly twelve dollars.

What's in this repo

params/              policy weights — everything you need for inference
assets/              action/state normalization stats
train_state/         optimizer state — only needed to RESUME training
_CHECKPOINT_METADATA orbax checkpoint metadata
register_config.py   registers the openpi serve config (run once during setup)
patch_openpi_config.py  adds the FARM data config to openpi (run once during setup)

Running inference

⚠️ π0.5 is a ~3.3B JAX/CUDA model — it needs an NVIDIA GPU. Inference is light (forward pass only, ~8 GB), so a 24 GB card (RTX 4090 / L40S / A100) is plenty — you do not need a big training GPU. A Mac/CPU won't run it.

1. Set up openpi + register the config

# clone openpi at the pinned commit this policy was trained against
git clone https://github.com/Physical-Intelligence/openpi && cd openpi
git checkout c23745b5ad24e98f66967ea795a07b2588ed6c79
uv sync

# pull the two setup scripts from this repo and register the config
huggingface-cli download noahnowac/farm_uf850_home_pi05 \
    patch_openpi_config.py register_config.py --local-dir .
export OPENPI_CONFIG_PY=src/openpi/training/config.py
python patch_openpi_config.py      # adds the FARM data config
python register_config.py          # registers TrainConfig "pi05_farm_home"

2. Download the checkpoint

huggingface-cli download noahnowac/farm_uf850_home_pi05 \
    --include "params/*" "assets/*" "_CHECKPOINT_METADATA" \
    --local-dir ./ckpt

3a. Serve the policy for a robot (recommended)

Starts a websocket policy server on port 8000. Your robot client streams observations (base RGB + wrist RGB + joint state + prompt) and receives action chunks.

uv run python scripts/serve_policy.py --port 8000 \
    --default_prompt "pick up the object and place it in the box" \
    policy:checkpoint \
    --policy.config=pi05_farm_home \
    --policy.dir=./ckpt

Then point your arm controller at ws://<gpu-host>:8000 (e.g. SSH-tunnel the port to the machine running the UF850 + cameras). Cameras map as: base → base view, wrist → wrist view (the same mapping used in training).

3b. Quick offline test in Python (no robot)

import numpy as np
from openpi.training import config as _config
from openpi.policies import policy_config

cfg = _config.get_config("pi05_farm_home")
policy = policy_config.create_trained_policy(cfg, "./ckpt")

obs = {
    "observation.images.base":  base_rgb,    # np.uint8 [H, W, 3]
    "observation.images.wrist": wrist_rgb,   # np.uint8 [H, W, 3]
    "observation.state":        joint_state, # np.float32 [state_dim]
    "prompt": "pick up the object and place it in the box",
}
result = policy.infer(obs)
actions = result["actions"]   # [50, action_dim] — the predicted 50-step action chunk
print(actions.shape)

The policy applies the same image/state normalization and camera mapping used in training, so feed it the raw dataset-style observation keys shown above.

License

apache-2.0