SixthSense

SixthSense 👁️

👉 Click here to try the Live Interactive Demo on Hugging Face Spaces

Privacy-conscious, on-device activity & productivity tracker that uses computer vision, audio classification, and LLM summarization to understand your work patterns — without streaming or cloud storage of raw data.

Architecture

Webcam Frame
    │
    ├─► YOLOv8 + ByteTrack ──► Object detection & persistent tracking
    ├─► MediaPipe Pose ────────► 33-landmark skeleton estimation
    ├─► Depth Anything V2 ─────► Monocular relative depth (foreground vs background)
    │
    └─► Rule-based State Machine ──► Activity label
              │                        (WORKING / USING PHONE / THINKING / PRESENT / AWAY)
              ▼
        SQLite + JSONL ──► Analytics Engine ──► Groq LLM Recap

Setup

# 1. Clone & install dependencies
git clone https://github.com/avneetsingh7102/SixthSense.git && cd SixthSense
pip install -r requirements.txt

# 2. Download model weights (auto-downloaded on first run)
#    - yolov8n.pt            (YOLOv8 nano)
#    - pose_landmarker_full.task  (MediaPipe pose)

# 3. Configure Groq API key
cp .env.example .env
# Edit .env and set GROQ_API_KEY=your_key_here

Usage

# Run the main activity logger (opens webcam)
python periodic_logger.py

# Controls:
#   'c' → force an immediate snapshot
#   'q' → quit

# Run temporal analytics on logged data
python analytics.py

# Generate a daily LLM recap
python vlm_recapper.py

# Preview depth estimation standalone
python depth_estimator.py

Privacy

• No cloud streaming — all processing happens on-device.
• Periodic sampling — captures discrete snapshots at configurable intervals (default: 60s), not continuous video.
• Local storage only — all data stays in sixthsense.db and snapshots/.
• API key in .env — never committed to version control (.gitignore).

Technical Highlights

License

MIT