VerwaltungsAnthologie_talky_7B

BEWARE – this model is experimental ONLY!

This is a merge of pre-trained language models created using mergekit – it is built for experimental purposes only as I am too dumb how to fix all those different 'promptings' and BOS/EOS-token mixtures due the merges.

Main Idea behind this model

This model has been created by merging different german language specialized models to fulfill possible "downstream" tasks as an "instruct"-model for german based texts like "Write a Résumée", "Name the Top-10 keywords" and so on… It is not really a "Chatbot".

Merge Details

As many models emerge at the moment, I got also inspired to test several different combinations of mainly german based and/or finetuned models that have not yet been tested. While doing those first experiments, the first series of models like latest hiig-piai/simba-v01c by HIIG emerged with the purpose of writing "shorter" and "easier" written summaries.

Being an enthusiast and working for a company that supports only the public administration sector and legislative institutions in germany there always is the problem to 'translate' words and sentences between "bureaucratic" language vs. "regular" language everybody uses and/or even understands it. Especially even more if we start talking about the "juridical" (legal) language. I thought it might be worth a try to combine a specialised fine-tuned experimental model like hiig-piai/simba-v01c out of scope (as the Humboldt team mentioned) with several of the other finetuned models for the german language. That was also the reason to name this model like "BureaucraticAnthology" (VerwaltungsAnthology) instead of "MistralLaserKrautSimbaPhoenix1227Disco-7B" (or something like that …).

This experimental model is a bit "talkier" than my unreleased "clear"-version as DiscoResearch/DiscoLM_German_7b_v1 seems to push the 'character' into a short 'Logorrhea' – I am not capable of changing the EOS-tokens as all models used are very different for prompting and tokenisation.

It took me 10 tries of different mergings to get a combination that seems to work – this "talky" one was only the addition of a fifth model after I finished the (unreleased) "clear" model.

Merge Method

This model was merged using the DARE TIES merge method using mistralai/Mistral-7B-v0.1 as a base.

Models Merged

The following models were included in the merge:

• hiig-piai/simba-v01c
• OpenPipe/mistral-ft-optimized-1227
• DRXD1000/Phoenix
• VAGOsolutions/SauerkrautLM-7b-LaserChat
• DiscoResearch/DiscoLM_German_7b_v1

Configuration

The following YAML configuration was used to produce this model:

  - model: mistralai/Mistral-7B-v0.1
    # No parameters necessary for base model
  - model: VAGOsolutions/SauerkrautLM-7b-LaserChat
    parameters:
      density: 0.53
      weight: 0.1875
  - model: hiig-piai/simba-v01c
    parameters:
      density: 0.53
      weight: 0.25
  - model: DRXD1000/Phoenix
    parameters:
      density: 0.53
      weight: 0.1875
  - model: OpenPipe/mistral-ft-optimized-1227
    parameters:
      density: 0.53
      weight: 0.1875
  - model: DiscoResearch/DiscoLM_German_7b_v1
    parameters:
      density: 0.53
      weight: 0.1875
merge_method: dare_ties
base_model: mistralai/Mistral-7B-v0.1
parameters:
  int8_mask: true
dtype: bfloat16
name: VerwaltungsAnthologie_talky_7B

Examples

Based upon my hobbies I rechecked some of those possible sources where "readable" summaries could become a real help – as long as there are no free and curated summaries available.

Using a local llama.cpp-server-instanza with a Q6.GGUF (I can not afford a real GPU - just kidding – this is a sustainabilty debate ;) – and I do always test with a temperature of "0.0" to check the most realistic summary hoping that the system will not start to hallucinate. The full example including the prompts (btw. DiscoLM-based) is in the /examples/ folder.

Server

llama.cpp/build/bin/server -m ./VerwaltungsAnthologie_talky_7B/ggml-model-Q6_K.gguf -c 4096 --override-kv tokenizer.ggml.add_eos_token=bool:true

Client

curl -s /dev/null 127.0.0.1:8080/completion -H "Content-Type: application/json" -d '{"temperature": 0.0, "n_predict": 2048, "n_keep": 4096, "cache_prompt": true, "prompt": '"$( cat ./examples/example_34c_gewo.txt | jq --raw-input --slurp . )"'}' | jq '.content' ; echo;

Example #1 - the short stuff

I prepared a short example taken from the pages of official german legislation: § 1 Podologiegesetz (PodG)

Example #2 - the long stuff

(todo - as the short stuff is not really shorter – it worked better with longer original texts)

Thanks to

(todo - a very VERY long list mentioning all those people and projects that have already pushed the german language as far as now into this N-dimensional vectorized world)

LICENSING

(todo – something something hopefully based upon all licenses everyone can use)

Tests

(todo – but is a regular german ROUGE-test really the appropriate idea for that?)