IB_Robot_ACT_banana_pick_distill

Model Card for ACT/BananaPick (Distilled, Multi-Backend)

Action Chunking Transformer Policy (as per Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware) trained for banana pick-and-place on a 1-arm SO-101 robot. This checkpoint is a distilled model obtained via knowledge distillation.

This repo bundles multiple deployment backends so the same policy can run on GPU/CPU (PyTorch), Huawei Ascend (OM), and Rockchip RK3588 (RKNN) without per-backend glue code. All backends are described by a single inference_manifest.json routing table.

Repository Structure

├── config.json                                                 # torch 权重配置
├── model.safetensors                                           # torch 权重 (340 MB)
├── policy_preprocessor.json
├── policy_postprocessor.json
├── policy_preprocessor_step_3_normalizer_processor.safetensors
├── policy_postprocessor_step_0_unnormalizer_processor.safetensors
├── train_config.json
├── inference_manifest.json                                     # 部署路由表 (权威)
└── artifacts/
    ├── ascend/ascend_310b1/policy-1031c8ca5f1e.om              # Ascend OM (123 MB)
    └── rknn/rk3588/policy-cfcf62db3da4.rknn                    # Rockchip RKNN (146 MB)

The directory layout and every file path inside inference_manifest.json must stay in sync. Do not rename or relocate the weight/artifact files — the manifest's sha256 checks and path bindings depend on them.

Deployment Backends

Read inference_manifest.json → deployments[<target>] to route to the right backend. Each entry carries the artifact path, runtime, SoC, and full input/output tensor bindings (shape, dtype, semantic name).

Input tensors (all backends): observation.state [1,6] float32, observation.images.top [1,3,480,640] NCHW, observation.images.wrist [1,3,480,640] NCHW. Output tensor: action [1,100,6] float32 (100-step action chunk, 6-dim).

How to Get Started with the Model

See the IB-Robot project (particularly the inference_service) for instructions on how to load and deploy this model with ROS 2.

To load the PyTorch backend directly in Python (weights are at the repo root):

from lerobot.common.policies.act.modeling_act import ACTPolicy

policy = ACTPolicy.from_pretrained("openEuler/IB_Robot_ACT_banana_pick_distill")

For Ascend (OM) or RKNN deployment, consume inference_manifest.json and load the artifact under the matching deployments[<target>].artifacts.policy.path.

Training Details

This model was trained via knowledge distillation (kd: true) within the IB-Robot framework. The OM/RKNN artifacts were converted from these torch weights (verified same-origin by sha256).

• Policy: ACT (Action Chunking with Transformers)
• Training method: Knowledge distillation
• Robot: 1-arm SO-101
• Cameras: top, wrist (480x640)
• Action dim: 6
• Training steps: 500,000 (configured)
• Checkpoint: step_160000
• Chunk size: 100
• Batch size: 32
• Vision backbone: ResNet18
• Optimizer: AdamW (lr=1e-5, weight_decay=1e-4)
• Latent dim (VAE): 32
• Dim model: 1024

Citation

@software{ib_robot,
  title = {IB-Robot: Intelligence Boom Robot},
  url = {https://gitcode.com/openeuler/IB_Robot},
  license = {Apache-2.0}
}