Iker_-_Llama-3-Instruct-Neurona-8b-gguf

RichardErkhov/Iker_-_Llama-3-Instruct-Neurona-8b-gguf

Iker_-_Llama-3-Instruct-Neurona-8b-gguf is a machine learning model from RichardErkhov. Use it for the machine learning task on the model card, and read the license before you ship it in a product.

Downloads · 30 days

9% of all-time downloads

All-time downloads

797

Public

Repo size

92.9 GB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.gguf108 GB · 100%

At a glance

Access: Public
Created: Aug 23, 2024
Updated: Aug 23, 2024
SHA: 715db89d

Model card

From the Hugging Face model README

Quantization made by Richard Erkhov.

Github

Discord

Request more models

Llama-3-Instruct-Neurona-8b - GGUF

Model creator: https://huggingface.co/Iker/
Original model: https://huggingface.co/Iker/Llama-3-Instruct-Neurona-8b/

Name	Quant method	Size
Llama-3-Instruct-Neurona-8b.Q2_K.gguf	Q2_K	2.96GB
Llama-3-Instruct-Neurona-8b.IQ3_XS.gguf	IQ3_XS	3.28GB
Llama-3-Instruct-Neurona-8b.IQ3_S.gguf	IQ3_S	3.43GB
Llama-3-Instruct-Neurona-8b.Q3_K_S.gguf	Q3_K_S	3.41GB
Llama-3-Instruct-Neurona-8b.IQ3_M.gguf	IQ3_M	3.52GB
Llama-3-Instruct-Neurona-8b.Q3_K.gguf	Q3_K	3.74GB
Llama-3-Instruct-Neurona-8b.Q3_K_M.gguf	Q3_K_M	3.74GB
Llama-3-Instruct-Neurona-8b.Q3_K_L.gguf	Q3_K_L	4.03GB
Llama-3-Instruct-Neurona-8b.IQ4_XS.gguf	IQ4_XS	4.18GB
Llama-3-Instruct-Neurona-8b.Q4_0.gguf	Q4_0	4.34GB
Llama-3-Instruct-Neurona-8b.IQ4_NL.gguf	IQ4_NL	4.38GB
Llama-3-Instruct-Neurona-8b.Q4_K_S.gguf	Q4_K_S	4.37GB
Llama-3-Instruct-Neurona-8b.Q4_K.gguf	Q4_K	4.58GB
Llama-3-Instruct-Neurona-8b.Q4_K_M.gguf	Q4_K_M	4.58GB
Llama-3-Instruct-Neurona-8b.Q4_1.gguf	Q4_1	4.78GB
Llama-3-Instruct-Neurona-8b.Q5_0.gguf	Q5_0	5.21GB
Llama-3-Instruct-Neurona-8b.Q5_K_S.gguf	Q5_K_S	5.21GB
Llama-3-Instruct-Neurona-8b.Q5_K.gguf	Q5_K	5.34GB
Llama-3-Instruct-Neurona-8b.Q5_K_M.gguf	Q5_K_M	5.34GB
Llama-3-Instruct-Neurona-8b.Q5_1.gguf	Q5_1	5.65GB
Llama-3-Instruct-Neurona-8b.Q6_K.gguf	Q6_K	6.14GB
Llama-3-Instruct-Neurona-8b.Q8_0.gguf	Q8_0	7.95GB

Original model description:

library_name: transformers tags:

synthetic license: llama3 datasets:
pinzhenchen/alpaca-cleaned-es
Danielbrdz/Barcenas-Economia
HiTZ/casimedicos-exp
somosnlp/coser_resumenes
csebuetnlp/CrossSum
Iker/Document-Translation-en-es
somosnlp/es-inclusive-language-it
FreedomIntelligence/evol-instruct-spanish
glaiveai/glaive-code-assistant-v3
glaiveai/glaive-function-calling-v2
Iker/InstructTranslation-EN-ES
somosnlp/lenguaje-claro-dataset
somosnlp/LingComp_QA
bltlab/lr-sum
Iker/NoticIA
xaviviro/oasst2_es_gpt
teknium/OpenHermes-2.5
Iker/OpenHermes-2.5-Spanish
Helsinki-NLP/opus-100
projecte-aina/RAG_Multilingual
sem_eval_2018_task_1
davidstap/ted_talks
HiTZ/This-is-not-a-dataset
wikipedia language:
es
en pipeline_tag: text-generation base_model: meta-llama/Meta-Llama-3-8B-Instruct

image/jpeg

Neurona 8B Beta: Un Modelo de Lenguage en Español

Esta es una versión preliminar del dataset card. El modelo está en desarrollo y no es la versión final. Si quieres saber más sobre este modelo, escribe a [email protected]

Neurona 8B es un modelo de lenguaje en Español. Esta es la primera iteración y un experimento para poner a punto los scripts y la infraestructura.

Neurona 8B ha sido entrenado con los siguiente datasets. No en todos los casos se ha usado el dataset completo

Esta mezcla de datasets en Inglés y Español, permite al modelo adquirir diferentes capacidades, como RAG, function calling, code assistant, question answering, summarization... tanto en Inglés como en Español.

Entrenamiento

Este modelo se ha entrado usando 4xNvidia A100 80Gb y axolotl <img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>

Esta es la configuración usada

base_model: meta-llama/Meta-Llama-3-8B-Instruct
model_type: AutoModelForCausalLM
tokenizer_type: AutoTokenizer
is_falcon_derived_model:
is_llama_derived_model:
is_qwen_derived_model:
is_mistral_derived_model:

load_in_8bit: false
load_in_4bit: false
strict: false

device_map: null

datasets:
  - path: /ikerlariak/igarcia945/InstructDatasets/alpaca-cleaned-es.jsonl
    type: sharegpt
    conversation: llama3
    field: conversations
    roles:
      input:
        - system
        - gpt
      output:
        - human
  - path: /ikerlariak/igarcia945/InstructDatasets/Barcenas-Economia.jsonl
    type: sharegpt
    conversation: llama3
    field: conversations
    roles:
      input:
        - system
        - gpt
      output:
        - human
  - path: /ikerlariak/igarcia945/InstructDatasets/casimedicos.jsonl
    type: sharegpt
    conversation: llama3
    field: conversations
    roles:
      input:
        - system
        - gpt
      output:
        - human
  - path: /ikerlariak/igarcia945/InstructDatasets/coser_resumene.jsonl
    type: sharegpt
    conversation: llama3
    field: conversations
    roles:
      input:
        - system
        - gpt
      output:
        - human
  - path: /ikerlariak/igarcia945/InstructDatasets/CrossSum_en.jsonl
    type: sharegpt
    conversation: llama3
    field: conversations
    roles:
      input:
        - system
        - gpt
      output:
        - human
  - path: /ikerlariak/igarcia945/InstructDatasets/CrossSum_es.jsonl
    type: sharegpt
    conversation: llama3
    field: conversations
    roles:
      input:
        - system
        - gpt
      output:
        - human
  - path: /ikerlariak/igarcia945/InstructDatasets/Document-Translation-en-es.jsonl
    type: sharegpt
    conversation: llama3
    field: conversations
    roles:
      input:
        - system
        - gpt
      output:
        - human
  - path: /ikerlariak/igarcia945/InstructDatasets/es-inclusive-language.jsonl
    type: sharegpt
    conversation: llama3
    field: conversations
    roles:
      input:
        - system
        - gpt
      output:
        - human
  - path: /ikerlariak/igarcia945/InstructDatasets/evol-instruct-spanish.jsonl
    type: sharegpt
    conversation: llama3
    field: conversations
    roles:
      input:
        - system
        - gpt
      output:
        - human
  - path: /ikerlariak/igarcia945/InstructDatasets/glaive-code-assistant-v3-small.jsonl
    type: sharegpt
    conversation: llama3
    field: conversations
    roles:
      input:
        - system
        - gpt
      output:
        - human
  - path: /ikerlariak/igarcia945/InstructDatasets/glaive-function-calling-v2.jsonl
    type: sharegpt
    conversation: llama3
    field: conversations
    roles:
      input:
        - system
        - gpt
        - tool
      output:
        - human
  - path: /ikerlariak/igarcia945/InstructDatasets/InstructTranslation-EN-ES.jsonl
    type: sharegpt
    conversation: llama3
    field: conversations
    roles:
      input:
        - system
        - gpt
      output:
        - human
  - path: /ikerlariak/igarcia945/InstructDatasets/lenguaje-claro-dataset.jsonl
    type: sharegpt
    conversation: llama3
    field: conversations
    roles:
      input:
        - system
        - gpt
      output:
        - human
  - path: /ikerlariak/igarcia945/InstructDatasets/LingComp_QA.jsonl
    type: sharegpt
    conversation: llama3
    field: conversations
    roles:
      input:
        - system
        - gpt
      output:
        - human
  - path: /ikerlariak/igarcia945/InstructDatasets/lr-sum-es.jsonl
    type: sharegpt
    conversation: llama3
    field: conversations
    roles:
      input:
        - system
        - gpt
      output:
        - human
  - path: /ikerlariak/igarcia945/InstructDatasets/NoticIA.jsonl
    type: sharegpt
    conversation: llama3
    field: conversations
    roles:
      input:
        - system
        - gpt
      output:
        - human
  - path: /ikerlariak/igarcia945/InstructDatasets/NoticIA-large.jsonl
    type: sharegpt
    conversation: llama3
    field: conversations
    roles:
      input:
        - system
        - gpt
      output:
        - human
  - path: /ikerlariak/igarcia945/InstructDatasets/NoticIA-summary.jsonl
    type: sharegpt
    conversation: llama3
    field: conversations
    roles:
      input:
        - system
        - gpt
      output:
        - human
  - path: /ikerlariak/igarcia945/InstructDatasets/oasst2_es_gpt.jsonl
    type: sharegpt
    conversation: llama3
    field: conversations
    roles:
      input:
        - system
        - gpt
      output:
        - human
  - path: /ikerlariak/igarcia945/InstructDatasets/OpenHermes-2.5-English.jsonl
    type: sharegpt
    conversation: llama3
    field: conversations
    roles:
      input:
        - system
        - gpt
      output:
        - human
  - path: /ikerlariak/igarcia945/InstructDatasets/OpenHermes-2.5-Spanish.jsonl
    type: sharegpt
    conversation: llama3
    field: conversations
    roles:
      input:
        - system
        - gpt
      output:
        - human
  - path: /ikerlariak/igarcia945/InstructDatasets/opus-100-en-es.jsonl
    type: sharegpt
    conversation: llama3
    field: conversations
    roles:
      input:
        - system
        - gpt
      output:
        - human
  - path: /ikerlariak/igarcia945/InstructDatasets/RAG_Multilingual-es.jsonl
    type: sharegpt
    conversation: llama3
    field: conversations
    roles:
      input:
        - system
        - gpt
      output:
        - human
  - path: /ikerlariak/igarcia945/InstructDatasets/sem_eval_2018_task_1.jsonl
    type: sharegpt
    conversation: llama3
    field: conversations
    roles:
      input:
        - system
        - gpt
      output:
        - human
  - path: /ikerlariak/igarcia945/InstructDatasets/ted_talks-es_en.jsonl
    type: sharegpt
    conversation: llama3
    field: conversations
    roles:
      input:
        - system
        - gpt
      output:
        - human
  - path: /ikerlariak/igarcia945/InstructDatasets/This-is-not-a-dataset.jsonl
    type: sharegpt
    conversation: llama3
    field: conversations
    roles:
      input:
        - system
        - gpt
      output:
        - human
  - path: /ikerlariak/igarcia945/InstructDatasets/wikipedia-es.jsonl
    type: sharegpt
    conversation: llama3
    field: conversations
    roles:
      input:
        - system
        - gpt
      output:
        - human

chat_template: llama3

dataset_prepared_path: /ikerlariak/igarcia945/Mortadelo-Filemon/Meta-Llama-3-8B-Instruct-Spanish/dataset

shuffle_merged_datasets: true

val_set_size: 0.005

output_dir: /ikerlariak/igarcia945/Mortadelo-Filemon/Meta-Llama-3-8B-Instruct-Spanish

adapter:
lora_model_dir:

sequence_len: 8192
sample_packing: true
eval_sample_packing: false
pad_to_sequence_len: false

tokens:
  - "<tool_call>"
  - "<tool_response>"
  - "<tools>"
  - "</tool_call>"
  - "</tool_response>"
  - "</tools>"
  - "<reserved1>"
  - "<reserved2>"

special_tokens:
   pad_token: <|end_of_text|>

neftune_noise_alpha: 5

wandb_project: Mortadelo&Filemon
wandb_entity: igarciaf
wandb_watch:
wandb_name: Meta-Llama-3-8B-Instruct-Spanish
wandb_log_model: 

gradient_accumulation_steps: 32
micro_batch_size: 2
eval_batch_size: 2
num_epochs: 2
optimizer: adamw_torch_fused
lr_scheduler: cosine
learning_rate: 0.00007


train_on_inputs: false
group_by_length: false
bf16: true
fp16: false
tf32: false

gradient_checkpointing: true
early_stopping_patience:
resume_from_checkpoint:
local_rank:
logging_steps: 1
xformers_attention:
flash_attention: true

warmup_ratio: 0.03
evals_per_epoch: 4
eval_table_size:
save_strategy: "no"
debug:
deepspeed: /ikerlariak/igarcia945/Mortadelo-Filemon/train_configs/deepspeed_zero3.json
weight_decay: 0.0
fsdp:
fsdp_config:

seed: 33

Created: Aug 23, 2024
Updated: Aug 23, 2024