IndoWebGen-7B

🇮🇩🌐🤖 IndoWebGen: LLM for Automated Website Generation Based on Indonesian Instructions

<video src="https://res.cloudinary.com/dn3srgvqk/video/upload/v1771360919/indowebgen-demo.mp4" controls="controls" style="max-width: 684px;"> </video>

Hugely inspired by Web App Factory.

Model Description:

• Base Model: codellama/CodeLlama-7b-hf [1]
• Finetuning Method: LoRA [2]
• Dataset: alxxtexxr/indowebgen-dataset

Finetuning Hyperparameters:

• Number of Epochs: 20
• Microbatch Size: 4
• Gradient Accumulation Step: 8
• LoRA Rank: 16
• LoRA Alpha: 32
• LoRA Target Modules: [q_proj, v_proj]

Inference:

<!-- Try the inference demo [here](https://indowebgen.alimtegar.my.id) or -->

Try running the inference code with the provided Google Colab notebook here. The inference code used is shown below:

# Install the required libraries
!pip install transformers bitsandbytes accelerate

# Import the neccessary modules
from transformers import AutoModelForCausalLM, AutoTokenizer

# Load the model and the tokenizer
model_id = 'alxxtexxr/indowebgen-7b'
model = AutoModelForCausalLM.from_pretrained(
  model_id, 
  load_in_8bit=True,
  # load_in_4bit=True, # for low memory
  device_map='auto',
)
tokenizer = AutoTokenizer.from_pretrained(model_id)

# Initialize the prompt
prompt_template = '''Berikut adalah instruksi pembuatan website beserta output-nya yang berupa kode HTML dari website yang dibuat:
    
### Instruksi:
{instruction}
    
### Output:
<!DOCTYPE html>
<html lang="id">'''

# INSERT YOUR OWN INDONESIAN INSTRUCTION BELOW
instruction = 'Buatlah website portfolio untuk Budi'

prompt = prompt_template.format(instruction=instruction)

# Generate the output
input_ids = tokenizer(prompt, return_tensors='pt').input_ids.to(model.device)
outputs = model.generate(
  input_ids, 
  max_new_tokens=2400,
  do_sample=True, 
  temperature=1.0,
  top_k=3, 
  top_p=0.8,
  repetition_penalty=1.1,
  pad_token_id=tokenizer.unk_token_id,
)
print(tokenizer.batch_decode(outputs, skip_special_tokens=True)[0])

Limitations

• The dataset used in training is limited to only 500 data, so the model performance may still not be optimal.
• The model is designed to generate single-page static websites, constructed using HTML with internal CSS.
• The content of the generated websites is dummy (including the images), so the users need to further customize the websites.
• The generated websites leverage Bootstrap for the styling, Font Awesome for the icons, and dummyimage.com images for the dummy images.