README.md

---
title: FocusGuard
sdk: docker
app_port: 7860
---

# FocusGuard

Webcam-based focus detection: MediaPipe face mesh -> 17 features (EAR, gaze, head pose, PERCLOS, etc.) -> MLP or XGBoost for focused/unfocused. React + FastAPI app with WebSocket video.

## Project layout

```
├── data/                 collected_<name>/*.npz
├── data_preparation/     loaders, split, scale
├── notebooks/            MLP/XGB training + LOPO
├── models/               face_mesh, head_pose, eye_scorer, train scripts
│   ├── gaze_calibration.py   9-point polynomial gaze calibration
│   ├── gaze_eye_fusion.py    Fuses calibrated gaze with eye openness
│   └── L2CS-Net/              In-tree L2CS-Net repo with Gaze360 weights
├── checkpoints/          mlp_best.pt, xgboost_*_best.json, scalers
├── evaluation/           logs, plots, justify_thresholds
├── ui/                   pipeline.py, live_demo.py
├── src/                  React frontend
│   ├── components/
│   │   ├── FocusPageLocal.jsx      Main focus page (camera, controls, model selector)
│   │   └── CalibrationOverlay.jsx  Fullscreen calibration UI
│   └── utils/
│       └── VideoManagerLocal.js    WebSocket client, frame capture, canvas rendering
├── static/               built frontend (after npm run build)
├── main.py, app.py       FastAPI backend
├── requirements.txt
└── package.json
```

## Setup

```bash
python -m venv venv
source venv/bin/activate
pip install -r requirements.txt
```

To rebuild the frontend after changes:

```bash
npm install
npm run build
mkdir -p static && cp -r dist/* static/
```

## Run

**Web app:** Use the venv and run uvicorn via Python so it picks up your deps (otherwise you get `ModuleNotFoundError: aiosqlite`):

```bash
source venv/bin/activate
python -m uvicorn main:app --host 0.0.0.0 --port 7860
```

Then open http://localhost:7860.

**Frontend dev server (optional, for React development):**

```bash
npm run dev
```

**OpenCV demo:**

```bash
python ui/live_demo.py
python ui/live_demo.py --xgb
```

**Train:**

```bash
python -m models.mlp.train
python -m models.xgboost.train
```

## Data

9 participants, 144,793 samples, 10 features, binary labels. Collect with `python -m models.collect_features --name <name>`. Data lives in `data/collected_<name>/`.

## Models

| Model | What it uses | Best for |
|-------|-------------|----------|
| **Geometric** | Head pose angles + eye aspect ratio (EAR) | Fast, no ML needed |
| **XGBoost** | Trained classifier on head/eye features (600 trees, depth 8) | Balanced accuracy/speed |
| **MLP** | Neural network on same features (64->32) | Higher accuracy |
| **Hybrid** | Weighted MLP + Geometric ensemble | Best head-pose accuracy |
| **L2CS** | Deep gaze estimation (ResNet50, Gaze360 weights) | Detects eye-only gaze shifts |

## Model numbers (15% test split)

| Model | Accuracy | F1 | ROC-AUC |
|-------|----------|-----|---------|
| XGBoost (600 trees, depth 8) | 95.87% | 0.959 | 0.991 |
| MLP (64->32) | 92.92% | 0.929 | 0.971 |

## L2CS Gaze Tracking

L2CS-Net predicts where your eyes are looking, not just where your head is pointed. This catches the scenario where your head faces the screen but your eyes wander.

### Standalone mode
Select **L2CS** as the model - it handles everything.

### Boost mode
Select any other model, then click the **GAZE** toggle. L2CS runs alongside the base model:
- Base model handles head pose and eye openness (35% weight)
- L2CS handles gaze direction (65% weight)
- If L2CS detects gaze is clearly off-screen, it **vetoes** the base model regardless of score

### Calibration
After enabling L2CS or Gaze Boost, click **Calibrate** while a session is running:
1. A fullscreen overlay shows 9 target dots (3x3 grid)
2. Look at each dot as the progress ring fills
3. The first dot (centre) sets your baseline gaze offset
4. After all 9 points, a polynomial model maps your gaze angles to screen coordinates
5. A cyan tracking dot appears on the video showing where you're looking

## Pipeline

1. Face mesh (MediaPipe 478 pts)
2. Head pose -> yaw, pitch, roll, scores, gaze offset
3. Eye scorer -> EAR, gaze ratio, MAR
4. Temporal -> PERCLOS, blink rate, yawn
5. 10-d vector -> MLP or XGBoost -> focused / unfocused

**Stack:** FastAPI, aiosqlite, React/Vite, PyTorch, XGBoost, MediaPipe, OpenCV, L2CS-Net.