Cosmos-Tokenizer-CI8x8-Lidar

Cosmos-Tokenizer-CI8x8-Lidar: A tokenizer for Lidar range map

Model overview

Description:

Cosmos-Tokenizer-CI8x8-Lidar tokenizes LiDAR point cloud data into continuous latent representations for efficient compression and generation tasks. This tokenizer encodes and decodes LiDAR range map data, providing high-quality reconstructions with significant improvements in reconstruction accuracy after fine-tuning on Lidar range map data. Lidar tokenizer was developed by NVIDIA as a part of the Cosmos Tokenizer family.<br>

This model is ready for commercial/non-commercial use.

License/Terms of Use

This model is released under the NVIDIA Open Model License. For a custom license, please contact [email protected].

Under the NVIDIA Open Model License, NVIDIA confirms:

• Models are commercially usable.
• You are free to create and distribute Derivative Models.
• NVIDIA does not claim ownership to any outputs generated using the Models or Derivative Models.

Important Note: If You bypass, disable, reduce the efficacy of, or circumvent any technical limitation, safety guardrail or associated safety guardrail hyperparameter, encryption, security, digital rights management, or authentication mechanism (collectively “Guardrail”) contained in the Model without a substantially similar Guardrail appropriate for your use case, your rights under this Agreement NVIDIA Open Model License Agreement will automatically terminate.

Deployment Geography:

Global

Use Case: <br>

Research related to autonomous driving; enabling efficient compression and reconstruction of Lidar data for generative AI models and LiDAR processing pipelines. <br>

Release Date: <br>

Hugging Face [11/03/2025] via [https://huggingface.co/nvidia/Cosmos-Tokenizer-CI8x8-Lidar] <br>

References(s):

Cosmos-Dream-Drive

Model Architecture:

Architecture Type: Auto-Encoder <br>

Number of model parameters: 7.7*10^7 <br>

Network Architecture: This model was developed based on Cosmos-0.1-Tokenizer-CI8x8. We designed Cosmos Tokenizer using a lightweight and computationally efficient architecture, featuring a temporally causal design. Specifically, we employ causal temporal convolution and causal temporal attention layers to preserve the natural temporal order of video frames, ensuring seamless tokenization of images and videos using a single unified network architecture. The encoder and decoder form a symmetrical pair, which are mirrors of each other. The encoder starts with a 2-level Haar wavelet transform layer, which down-samples inputs by a factor of 4 in both spatial and temporal dimensions. Likewise, the decoder ends with an inverse wavelet transform. We employ the vanilla autoencoder (AE) formulation to model the latent space for continuous tokenizers.

Input: <br>

Input Type(s): Lidar data <br> Input Format: Lidar range map, 1 x H x W <br> Input Parameters: Two-Dimensional (2D) <br> Other Properties Related to Input: LiDAR point cloud data with range coverage of 5-100 meters, the original resolution of the range map is 128 x 3600. We repeat rows by a factor of 4 and downsample the columns by a factor of 2, the final processed range map is of resolution 512 x 1800. The spatial compression rate is 8x8. This tokenizer is compatible with Nvidia internal 128-beam LiDAR sensor only. Users can post-train their own lidar tokenizer following the provided guidance. <br>

Output: <br>

Output Type(s): Lidar <br> Output Format: Lidar range map, 1 x H x W <br> Output Parameters: Two-Dimensional (2D) <br> Other Properties Related to Output: Continuous latent space <br>

Software Integration:

Runtime Engine(s):

• Cosmos-Drive-Dreams

Supported Hardware Microarchitecture Compatibility: <br>

• NVIDIA Ampere <br>
• NVIDIA Blackwell <br>
• NVIDIA Hopper <br>

Preferred/Supported Operating System(s):

• Linux <br>

The integration of foundation and fine-tuned models into AI systems requires additional testing using use-case-specific data to ensure safe and effective deployment. Following the V-model methodology, iterative testing and validation at both unit and system levels are essential to mitigate risks, meet technical and functional requirements, and ensure compliance with safety and ethical standards before deployment.

Usage

• See Cosmos-Drive-Dreams for inference details.

Note: We have only tested doing inference with fp32 precision. On a single NVIDIA A100-SXM4-80GB GPU, our tokenizer runs at 1.7 fps for a processed lidar range map of shape 512 x 1800.

Model Version(s):

Cosmos-Tokenizer-CI8x8-Lidar

Training, Testing, and Evaluation Datasets:

Data Collection Method:

• AV: Automatic/Sensors

Labeling Method:

• AV: Hybrid: Human, Automated

Inference:

• Acceleration Engine: None <br>
• Test Hardware: NVIDIA NVIDIA A100-SXM4-80GB

Ethical Considerations:

NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse. <br>

Please make sure you have proper rights and permissions for all input image and video content; if image or video includes people, personal health information, or intellectual property, the image or video generated will not blur or maintain proportions of image subjects included. <br>

For more detailed information on ethical considerations for this model, please see the subcards of Explainability, Bias, Safety & Security, and Privacy below. Please report model quality, risk, security vulnerabilities or NVIDIA AI Concerns here.

Bias

Explainability

Privacy

Safety