llava-v1.6-34b-finetune

Model details

This model is trained by fine-tuning LLaVA-1.6-34b on multimodal instruction-following data, with a focus on enhancing detail capturing and summarizing.

It is an auto-regressive language model based on the transformer architecture.

Due to unclean conversation data, the model will attempt to repeat its words at the end of the output. This model is not well-trained and is just for demonstrating the potential of fine-tuning.

Base Model:

• NousResearch/Nous-Hermes-2-Yi-34B
• liuhaotian/llava-v1.6-34b

Training dataset

• 50K GPT-4V multimodal instruction-following data.
• 1K anime-captioning-task-oriented VQA data mixture.

Samples

<details> <summary>Demo 1</summary>

</details>

original 34b

The image presents an enchanting night scene set in a grassy field under a starry sky. The sky is a mesmerizing gradient of blue hues, transitioning from a deeper blue at the top to a lighter one near the horizon. The stars above twinkle brightly, creating a captivating celestial spectacle.

In the midst of this tranquil setting, a lone figure stands still, their gaze directed upwards towards the sky. They are positioned on the left side of the image, adding a sense of depth and perspective to the scene. To the right of the figure, a solitary tree stands tall, its silhouette stark against the vibrant backdrop of the night sky. The tree's presence adds an element of nature to the scene, further emphasizing the outdoor setting.

The overall composition of the image, with its central figure and the tree on the right, creates a harmonious balance. The positioning of the elements and the use of color contribute to a serene and peaceful atmosphere. The image captures a moment of quiet contemplation under the vast expanse of the night sky.

finetuned

In the image, a solitary figure stands on a grassy hill under the vast expanse of the night sky. The person, shrouded in darkness, is illuminated only by the soft glow of a long, white scarf that cascades down their back. They are positioned close to a solitary tree trunk, their stance suggesting a moment of quiet contemplation.

Above them, the sky is a canvas of celestial beauty, painted in shades of blue. Radiating from the top right corner of the image is the radiant Milky Way galaxy, its star-studded expanse seemingly in motion with the appearance of blurred lines. This cosmic spectacle serves as a striking backdrop to the tranquil scene on the hill below. The image captures a moment of serene solitude, the person and the tree standing as silent observers to the grandeur of the universe unfolding above them.

---

<details> <summary>Demo 2</summary>

</details>

original 34b

The image is a digital artwork in a realistic style. It depicts a young female character with a fantasy aesthetic, lying on her back in a tranquil setting. She has long, flowing blonde hair and striking blue eyes. Her attire is a white, translucent dress with lace details, and she is holding her right arm up, resting her head on her hand.

The character is surrounded by a serene environment. The ground appears to be water, as suggested by the reflections and the way the light is refracted. The color palette is dominated by shades of blue and white, creating a calm and ethereal atmosphere.

Scattered around the character are blue flowers and floating lanterns, adding to the dreamlike quality of the scene. The lighting is soft and ambient, filtering through the water and illuminating the surrounding area. The overall mood of the artwork is peaceful and introspective. There is no visible text on the image.

finetuned

In the image, a young woman is the central figure, lying down on a blue-green background that gives the impression of a serene body of water. Her attire is a white dress, adorned with gold accents, which contrasts beautifully with the background. Her hair, a vibrant shade of blonde, falls around her, adding to the overall composition.

Scattered around her are three gold lanterns, their light adding a touch of warmth to the image. These lanterns are not merely decorative elements; they seem to serve as points of illumination, casting a soft glow on the surrounding water.The woman's expression is one of contemplation. She rests her head on her hand, her gaze directed upwards, as if lost in thought. This pose, combined with the overall composition, creates a sense of tranquility and introspection.

The image does not contain any discernible text. The relative positions of the objects are such that the woman is at the center, with the lanterns surrounding her at various distances. The dress and the lanterns share a common color scheme of white and gold, which ties the image together. The blue-green background serves to highlight the woman and the lanterns, drawing the viewer's attention to them.

Overall, the image is a harmonious blend of color, light, and composition, with each element playing a part in conveying a sense of peace and introspection. The absence of any explicit action or movement further emphasizes the contemplative mood of the image.