VedaX-0.7B-Base

VedaX-0.7B-Base

VedaX-0.7B-Base is a lightweight open-weight language model developed by VedaX Labs. The model is designed for efficient conversational AI, reasoning tasks, and lightweight deployment on consumer hardware. The model is entirely trained on CPU and on cosmopedia dataset

Training Specs

• CPU:- Intel(R) Core(TM) i3-14100
• Memory:- 16.0 GB DDR5
• GPU:- None (Intel UHD Graphics 730)

Model Overview

• Model Name: VedaX-0.7B-Base
• Organization: VedaX Labs
• Parameters: 0.7 Billion
• Architecture: Transformer-based causal language model
• Type: Base/Foundation Model
• License: Apache-2.0

Capabilities

• Text generation
• Lightweight inference
• Fine-tuning friendly

Intended Use

VedaX-0.7B-Base is intended for:

• Research and experimentation
• Educational purposes
• Building lightweight AI assistants
• Fine-tuning for downstream tasks
• Local inference applications
• Suitable for all devices

Context Length

• Context Window: 4096 tokens

Training

The model was trained on a mixture of publicly available text datasets and curated instruction-style data.

Example Usage

from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "VedaX-Labs/VedaX-0.7B-Base"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)

prompt = "The magic forest was"

inputs = tokenizer(prompt, return_tensors="pt")

outputs = model.generate(
    **inputs,
    max_new_tokens=128
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Benchmarking(Higher number means higher score)

Limitations

• May generate inaccurate information
• Limited reasoning compared to larger models
• Responses may contain hallucinations
• Performance depends on prompting quality
• May generate repettetive words(Chances is very low)

Future Plans

• Instruction-tuned versions
• Reasoning-enhanced variants
• Multimodal support
• Larger parameter models

---

Developed by VedaX Labs.