followyourpose-charades

Model Details

The model in this repository is fine-tuned based on the charades datasets with different human actions. Each model in the folder is a checkpoint for the fine-tuned process.

Model Metadata

---

id:

• Unique identifier for each video
• 0MK2C

subject:

• Unique identifier for each subject in the dataset
• DXDI

scene:

• One of the 15 indoor scenes in the dataset
• Stairs

quality:

• Quality of the video judged by an annotator (7-point scale, 7 = high quality)
• 7

relevance:

• Relevance of the video to the script judged by an annotator (7-point scale, 7 = very relevant)
• 7

verified:

• Yes - if an annotator successfully verified that the video matches the script, else No
• "Yes"

script:

• The human-generated script used to generate the video
• "A person is running up the stairs holding a pair of shoes. The person goes through a door."

objects:

• List of objects identified in the video
• ["doorway", "shoe", "stairs"]

descriptions:

• List of descriptions by annotators watching the video
• "A person runs up the stairs carrying a pair of shoes and opens a door."

actions:

• Consists of actions - human actions in the video and timings - timing of the action happening in the video
• actions: [{ action: "run up the stairs", timing: ["1.50", "8.80"] }]

---

• Developed by: ICT3104-Team06-2023
• Finetuned from Model: stable-diffusion-v1-4
• Stable Diffusion Model Repository: https://huggingface.co/YueMafighting/FollowYourPose_v1
• Training Data Source: https://prior.allenai.org/projects/charades