DeepSeek-R1-0528-q2ks-mixed-AutoRound-inc-v0

Model Details

This model is a mixed gguf:q2ks of deepseek-ai/DeepSeek-R1-0528 generated by intel/auto-round algorithm. Please refer to Section Generate the model for more details.

Theoretically, this model(https://huggingface.co/Intel/DeepSeek-R1-0528-q2ks-mixed-AutoRound-inc-v1) should perform better, but we don’t yet have accuracy metrics to confirm it.

How To Use

./llama-cli --model ./DeepSeek-R1-0528-hf-256x20B-Q2_K_S-00001-of-00005.gguf --prompt "<｜User｜>Create a Flappy Bird game in Python.<｜Assistant｜>"

LLamacpp Inference

"""
Create a Flappy Bird game in Python.<think>
We are going to create a simple Flappy Bird game using Pygame.
 The game will have:
   - A bird (represented by a circle or an image) that can flap (jump) when the space bar is pressed.
   - Pipes (obstacles) that move from the right to the left of the screen.
   - The bird must navigate through the gaps between the pipes without colliding.
   - The game ends if the bird collides with a pipe or the ground, or if it goes off the top of the screen.
   - The score increases each time the bird passes a pair of pipes.

 Steps:
 1. Initialize Pygame and set up the display.
 2. Define the game constants (screen size, colors, gravity, flap strength, pipe speed, etc.).
 3. Create the bird: position, velocity, and a method for jumping.
 4. Create the pipes: manage their positions and movement. We'll have a list of pipes.
 5. Implement collision detection between the bird and the pipes, and between the bird and the ground/top.
 6. Implement the main game loop: handle events, update positions, check collisions, draw everything.

 We'll use a simple representation for the bird (a circle) and the pipes (rectangles).

 Let's code accordingly.
</think>
# Flappy Bird Game in Python using Pygame

Here's a complete implementation of the Flappy Bird game using Pygame:

```python
import pygame
import random
import sys

# Initialize pygame
pygame.init()

# Game constants
WIDTH, HEIGHT = 400, 600
FPS = 60
GRAVITY = 0.5
FLAP_STRENGTH = -10
PIPE_SPEED = 5
PIPE_GAP = 200
PIPE_SPAWN_FREQUENCY = 1500  # milliseconds
GROUND_HEIGHT = 100

# Colors
SKY_BLUE = (113, 197, 207)
GREEN = (111, 196, 70)
DARK_GREEN = (76, 145, 65)
BIRD_YELLOW = (255, 221, 45)
BIRD_ORANGE = (255, 150, 0)
WHITE = (255, 255, 255)
BLACK = (0, 0, 0)
"""

too long to copy

## Generate the model

auto-round>0.5.1

700G RAM and 30G vram(optional) are required

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from auto_round import AutoRound

model_name = "/models/DeepSeek-R1-0528-hf"  ##must be BF16 model

model = AutoModelForCausalLM.from_pretrained(model_name,
                                             device_map="cpu", torch_dtype="auto")
tokenizer = AutoTokenizer.from_pretrained(model_name)
layer_config = {}
for n, m in model.named_modules():
    if n == "lm_head":
        layer_config[n] = {"bits": 8}
    elif isinstance(m, torch.nn.Linear) and (not "expert" in n or "shared_experts" in n) and n != "lm_head":
        layer_config[n] = {"bits": 4}

autoround = AutoRound(model, tokenizer, iters=0, layer_config=layer_config, batch_size=8, nsamples=512)
autoround.quantize_and_save("/models/DeepSeek-R1-0528-q2ks", format="gguf:q2_k_s")

Ethical Considerations and Limitations

The model can produce factually incorrect output, and should not be relied on to produce factually accurate information. Because of the limitations of the pretrained model and the finetuning datasets, it is possible that this model could generate lewd, biased or otherwise offensive outputs.

Therefore, before deploying any applications of the model, developers should perform safety testing.

Caveats and Recommendations

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model.

Here are a couple of useful links to learn more about Intel's AI software:

• Intel Neural Compressor link

Disclaimer

The license on this model does not constitute legal advice. We are not responsible for the actions of third parties who use this model. Please consult an attorney before using this model for commercial purposes.

Cite

@article{cheng2023optimize, title={Optimize weight rounding via signed gradient descent for the quantization of llms}, author={Cheng, Wenhua and Zhang, Weiwei and Shen, Haihao and Cai, Yiyang and He, Xin and Lv, Kaokao and Liu, Yi}, journal={arXiv preprint arXiv:2309.05516}, year={2023} }

arxiv github