# Frozen video embeddings for movement alignment

Local experiment recorded on **23 August 2026**, summarized for the landing page on 15 September. This is an alignment experiment: finding corresponding moments in two performances. It is not a benchmark of action-classification accuracy or a controlled study of training-set size.

## What was compared

- Frozen V-JEPA 2.1 ViT-L/16 at 384 resolution, checkpoint hash prefix `7ea9b7cb4a75d106`, with sampling based on accumulated movement.
- Frozen DINOv3 ViT-L/16, timm port, hash prefix `45172f209c9583c4`, applied to individual frames.
- The production posture-based alignment method, using its first returned match.

The generated report is timestamped `2026-08-23T02:32:22.662Z`. The table below transcribes its **“all (66 pairs)”** table, rather than the older prose verdict appended to that report, which contains some different aggregate values.

| Method | Median alignment error | Cases within the 200 ms threshold |
|---|---:|---:|
| Production posture matcher | 59 ms | 65/66 |
| V-JEPA 2.1 with movement-based sampling | 445 ms | 14/66 |
| DINOv3 frame features | 694 ms | 10/66 |

These numbers measure agreement with the recorded correspondences. Most were marked using the posture matcher's own similarity display, and some started as its saved output. **That favors the posture baseline.** The harness also measures a discretized alignment; continuous-warp errors use a different calculation and must not be mixed into this table.

## What the failure analysis suggested

The saved analysis identifies selecting the wrong repetition as a recurring failure, and long flat alignment paths around nearly static postures. Appearance similarity alone does not indicate which repetition or moment is intended. This supports using explicit movement features for this application; it does not isolate background, clothing or identity as the cause.

Sampling according to accumulated movement helped handle playback-speed differences. It did not consistently solve repetition selection. No fine-tuning, background-removal ablation or controlled increase in labeled training data was evaluated here. The experiment therefore cannot establish that video embeddings generally fail with small datasets or that a trained video model could not improve.

The separate recognition experiment compared predictions with existing automated swing labels. Agreement between those systems is not human-verified correctness and cannot populate a “both right / both wrong” gallery.

## Original local records

- `embedding-eval-artifacts/RESULTS.md`: generated tables, per-case errors and timestamp.
- `embedding-eval-artifacts/VERDICT.md`: qualitative interpretation and failure mechanisms; its older aggregate figures are not used in the table above.
- `embedding-eval-artifacts/model_info.json`: model names, output shapes and hash prefixes.
- `embedding-eval-artifacts/MOTION-PARAMETERIZATION.md`: movement-based sampling design.
- `scripts/lib/videoEmbeddingEval/extract_embeddings.py`: `motion_curve` and `embed_vjepa_motion` implementations.

The results are local observations, not independent validation of a general advantage over V-JEPA or DINO.
