Downloads · 30 days
12
12% of all-time downloads
mohummadmahad/pakistani-fashion-generator-model
pakistani-fashion-generator-model is a text-to-image model from mohummadmahad. Use it when you need an image from a text prompt. It is set up for diffusers. The card lists the license as creativeml-openrail-m.
should probably proofread and complete it, then remove this comment. --
Downloads · 30 days
12
12% of all-time downloads
All-time downloads
97
Public
Parameters
860M
414 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors211 GB · 51%
From the Hugging Face model README
This pipeline was finetuned from stable-diffusion-v1-5/stable-diffusion-v1-5 on the mohummadmahad/pakistani_fashion_dataset dataset. Below are some example images generated with the finetuned pipeline using the following prompts: ['Pakistani Dress']:

You can try out a demo here.
You can use the pipeline like so:
from diffusers import DiffusionPipeline
import torch
pipeline = DiffusionPipeline.from_pretrained("mohummadmahad/pakistani-fashion-generator-model-2", torch_dtype=torch.float16)
prompt = "Pakistani Dress"
image = pipeline(prompt).images[0]
image.save("my_image.png")
Please credit me if you end up using this model! :')
These are the key hyperparameters used during training:
A pakistani fashion clothing site crawler developed using the typescript library Crawlee to crawl Pakistani clothing sites to create a dataset of clothing images and their descriptions for fine-tuning a text to image model such as Stable Diffusion.
Extra care was taken to exclude data from:
Once all the data was crawled, we got json files from each website with the image urls and descriptions. These were all concatenated using some simple scripts into a single folder called 'json-files'.
These were then processed using 'crawlee-dataset-to-csv.ts' to generate a 'pakistani-fashion-dataset.csv' file with website names, product urls, product image urls and product descriptions.
Then using the 'process_jsons.sh' script, we downloaded the first image of each product and placed in a folder called train while also generating a metadata.jsonl file in which we appended in a row its filename and description text.
Now we had a folder called 'train' with all the product images. We zipped this folder and then made a jupyter notebook in which we used Salesforces BLIP2 model to generate normalized text descriptions of all our images and also generate our dataset which included the previously mentioned train folder as well as a new metadata.csv file with all the fileneames and normalized text descriptions. This dataset was uploaded to hugging face and can be found here.
Then by using huggingface's diffusers library, we finetuned the Stable Diffusion text to image model using our newly created dataset. The notebook for that can be found here. This process took a lot of time and GPU resources for which Google Colab was not enough. So I signed up for Digital Ocean's Paperspace cloud computing platform and rented some GPUs there which cost me about $200.
After all was said and done, here is the model I came up with. Pretty good for a first try I might say though of course not perfect.
Here I will make a note of some challenges I faced during this whole process as well as where the model falls short.
Overall, despite the various complications I had during the whole process and the sometimes lacklustre performance of the model, it was a really good learning experience for me, the lessons of which I hope to carry forward as I continue on my machine learning journey.
<!-- ## Intended uses & limitations --> <!-- #### How to use --> <!-- ```python # TODO: add an example code snippet for running this diffusion pipeline ``` --> <!-- #### Limitations and bias --> <!-- [TODO: provide examples of latent issues and potential remediations] --> <!-- ## Training details --> <!-- [TODO: describe the data used to train the model] -->