lidar_frnet

FRNet for Autoware (lidar_frnet)

LiDAR semantic segmentation models used by the autoware_lidar_frnet node in Autoware.

The models follow the FRNet (Frustum-Range Network) [1] architecture and perform per-point 3D semantic segmentation on LiDAR data (x, y, z, intensity). They are exported as ONNX and run with TensorRT inside Autoware; the node builds the TensorRT engine from the ONNX file on first launch.

Model overview

Segmented classes (as defined in the ml_package_frnet_*.param.yaml files):

drivable_surface, other_flat_surface, sidewalk, manmade, vegetation, car, bus, emergency_vehicle, train, truck, tractor_unit, semi_trailer, construction_vehicle, forklift, kart, motorcycle, bicycle, pedestrian, personal_mobility, animal, pushable_pullable, traffic_cone, debris, stroller, other_stuff, noise+ghost_point, unknown

Pre-processing (range-image projection, frustum construction) and post-processing (per-point argmax, probability-based filtering, visualization coloring) run in the node, not in the ONNX graph.

Variants in this repository

FRNet uses range images, so each variant is tied to the field of view and resolution of a specific LiDAR sensor:

Both variants share the same 27-class label set and color palette. The point count ranges define the dynamic TensorRT optimization profile for the input point cloud.

Files

TensorRT engines are not distributed here. TensorRT engines are specific to the GPU architecture and TensorRT version they are built on and are not portable, so Autoware builds them locally from the ONNX files on first launch (or via build_only:=true).

Inputs and outputs (as used by the node)

Input: ~/input/pointcloud (sensor_msgs/msg/PointCloud2). The node operates on the x, y, z and intensity fields and auto-detects the input layout on the first received message; supported layouts are XYZIRCAEDT, XYZIRADRT, XYZIRC and XYZI.

Outputs:

The node also publishes processing time and latency debug topics and /diagnostics.

Usage in Autoware

Autoware's setup tooling (the Ansible artifacts role) downloads these artifacts to ~/autoware_data/ml_models/lidar_frnet/. The node then loads frnet_<sensor_model>.onnx together with the matching ml_package_frnet_<sensor_model>.param.yaml from model_path:

ros2 launch autoware_lidar_frnet lidar_frnet.launch.xml \
  sensor_model:=ot128 \
  model_path:=$HOME/autoware_data/ml_models/lidar_frnet

sensor_model accepts ot128 (HESAI OT128) or qt128 (HESAI QT128). Add build_only:=true to build the TensorRT engine from the ONNX as a one-off pre-task. See the package README for the full parameter reference.

Training

The models were trained with AWML (https://github.com/tier4/AWML) on the T4Dataset, using approximately 16,000 frames (4,000 frames x 4 surrounding sensors), as documented in the package README.

• FRNet project page: https://xiangxu-0103.github.io/FRNet
• FRNet paper: arXiv:2312.04484

Provenance and versioning

The model and parameter files (frnet_*.onnx, ml_package_frnet_*.param.yaml) byte-match the artifacts previously hosted at the source path above; deploy_metadata.yaml is generated for this repository and has no upstream counterpart. Consumers should pin a tag (--revision v2.0), never main.

Limitations

• FRNet uses range images, so the input point cloud must be referenced to the sensor's origin.
• Each variant is tuned to its sensor's FOV and resolution; differences in your sensor's specifications (FOV, horizontal/vertical resolution) may affect performance.
• Only the two sensor variants above are provided; other LiDAR models require retraining or fine-tuning.

Citation

@article{xu2025frnet,
  title   = {FRNet: Frustum-Range Networks for Scalable LiDAR Segmentation},
  author  = {Xu, Xiang and Kong, Lingdong and Shuai, Hui and Liu, Qingshan},
  journal = {IEEE Transactions on Image Processing},
  volume  = {34},
  pages   = {2173--2186},
  year    = {2025},
  doi     = {10.1109/TIP.2025.3550011}
}

References

• [1] X. Xu, L. Kong, H. Shuai and Q. Liu, "FRNet: Frustum-Range Networks for Scalable LiDAR Segmentation", IEEE Transactions on Image Processing, vol. 34, pp. 2173-2186, 2025. arXiv:2312.04484.
• [2] AWML: https://github.com/tier4/AWML
• [3] FRNet project page: https://xiangxu-0103.github.io/FRNet