# Tier-1 skeleton benchmarks — LegibleTCN

Historical run record (July–August 2026). NTU-60 lists 40,091 clips without a
complete train/test count breakdown. This may be a training-only count.
Full official dataset coverage has NOT been established for that run, so its
score must not be presented as a verified full-benchmark result. PKU-MMD
phase 1 records 944 train / 132 test videos. Published comparison numbers were checked against the
original papers (see per-row Source). The historical commands document the
intended recipes; reproducing them requires the matching code and assets.
Preprocessing has since changed: current PYSKL
code uses image-centered coordinates, confidence masking and original actor
order, whereas the August record below describes hip centering and motion
ranking. New runs must be reported separately, not treated as reproductions
of the archived recipe. NTU-120 used a rented RTX 4090 and three seeds.

September 2026 causality audit: `pkummdBenchmark.normalize_track` fills leading
missing frames from the first later valid pose. Network-only causality checks
do not establish causality of that complete preprocessing-to-alert path.
Treat the archived streaming figures as causal-network experiments pending a
fresh end-to-end audit. Median latency is conditional on onset-matched alerts
inside the stated ±250 ms tolerance, not all actions or all alerts.

The model: **LegibleTCN** — legible-by-construction temporal conv net
(per-joint grouped causal conv → pointwise named mixing → depthwise temporal
conv → linear head), GroupNorm (per-timestep stats in causal mode so framewise
logits have provably zero lookahead — verified exact), human labels only, no
distillation. `scripts/lib/skeletonBenchCommon.py`.

## PYSKL 2D-pose benchmarks (COCO-17 HRNet) — NTU-120 X-Sub (2026-08-28)

The Kinect benchmarks above parse raw 25-joint `.skeleton` files. This section
runs the SAME template on **PYSKL's released 2D HRNet pose pickles** (COCO-17,
the format our DETRPose pipeline emits — no pose extraction of our own). Only
the model INPUT DIMS change (V=17, C_IN=5: `x, y, conf, dx, dy`); the template
architecture, GroupNorm, and the AdamW/cosine recipe are untouched. Thin
adapter `scripts/lib/pysklBench.py`; runner `scripts/lib/pysklBenchmark.py`;
sanity-render `scripts/lib/pysklRender.py`. Run on a rented RTX 4090 (the tiny
model trains at ~5 s/epoch; 3 seeds 42/43/44, mean ± sd — the new measurement
rule). x,y normalized by img_shape then translation-removed at the hip-midpoint;
confidence fed as its own channel (not differenced).

Protocol: official X-Sub split (63,026 train / 50,919 test — canonical counts,
matched exactly); top-2 bodies by motion energy, train body-1, eval = max over
the two single-body passes (masked for empty 2nd body); 64-frame uniform
sampling; standard recipe (AdamW 2e-3 wd 1e-4 cosine, batch 64, class-balanced,
label-smoothing 0.1), 80 epochs. No per-benchmark architecture tuning.
Command: `python3 pysklBenchmark.py --dataset ntu120 --data-dir <dir> --seeds 42,43,44`

| Model | XSub top-1 | Params | Nameable | Source |
|---|---|---|---|---|
| ST-GCN joint-only | 82.9% | 3.1M | no | arXiv:2005.03259 / PYSKL zoo |
| CTR-GCN joint-only | 84.9% | 1.46M | no | arXiv:2107.12213 T5 (joint stream) |
| **LegibleTCN (2D COCO-17, 3 seeds)** | **65.94% ± 0.63** | **32.7k** | yes (all) | this repo |

Per-seed body1: 65.05 / 66.40 / 66.37. Max-over-2-bodies: 65.92% ± 0.59
(interactions don't help — as on NTU-60).

**Registered stopping rule TRIGGERED → suite halted after NTU-120.** The task
pre-registered: run NTU-120 first alone; if the 3-seed mean lands **>10 points
below ST-GCN joint-only, stop before spending the FineGym + Kinetics runs — the
template question is answered.** 65.94% is **16.96 points** below ST-GCN
joint-only (82.9%), clearly out of family, so FineGym Gym99 and
Kinetics-Skeleton were **not run**. (Kinetics was independently data-blocked:
PYSKL's public `k400_hrnet.pkl` is an index only — the keypoint shards
`kpfiles/kp_*.pkl` are 404 on the openmmlab mirror.)

**Honest reading.** The registered prediction ("within 5 pts of ST-GCN
joint-only") was over-optimistic for a **95×-smaller** model (32.7k vs 3.1M).
The result is coherent with NTU-60, where LegibleTCN was 71.8% vs ST-GCN 81.5%
(−9.7 pts, just inside a 10-pt band). Going to **NTU-120 the relative deficit
WIDENS to −17 pts** — the tiny legible template's gap to single-stream ST-GCN
grows with class count. It remains the legibility-per-parameter point on the
curve, not a single-stream contender on the 120-way board. A LR/epoch sweep
(the only tuning the protocol allowed) cannot close a 17-pt gap, so the sweep
was not spent.

## NTU RGB+D 60 — cross-subject (XSub), top-1

Protocol: official XSub split (train subjects {1,2,4,5,8,9,13,14,15,16,17,18,
19,25,27,28,31,34,35,38}); 302 missing-skeleton samples dropped (official
list); bodies = top-2 by motion energy, train on body-1, eval = elementwise
max of the two single-body passes; normalization = subtract first valid
frame's spine-mid (joint 1), metric coordinates, channels xyz+dxyz; 64-frame
uniform sampling (offline row); AdamW 2e-3 wd 1e-4 cosine, batch 64,
class-balanced sampling, label smoothing 0.1, seed 42.
Command: `python3 scripts/lib/ntuBenchmark.py --data-dir <dir> [--causal]`

| Model | XSub top-1 | Params | Causal | Nameable units | Source |
|---|---|---|---|---|---|
| ST-GCN (AAAI 2018) | 81.5% | 3.10M | no | no | arXiv:1801.07455 T3; params: arXiv:2106.15125 TVII |
| 2s-AGCN ensemble (CVPR 2019) | 88.5% | ~6.94M | no | no | arXiv:1805.07694 T3 |
| CTR-GCN 4-stream (ICCV 2021) | 92.4% | 1.46M/stream | no | no | arXiv:2107.12213 T6 |
| PoseC3D J+L (CVPR 2022) | 94.1% (93.7% joint-only) | 2.0M | no | no | arXiv:2104.13586 T3/T8 |
| **LegibleTCN offline** | **71.8%** | 37.9k | no | yes (all) | this repo |
| **LegibleTCN causal** | **70.4%** | 37.9k | **yes** | yes (all) | this repo |

All LegibleTCN numbers are body-1-only top-1 (the clean headline). Masked
max-over-two-bodies did NOT help (offline eval, corrected to skip empty second
bodies: ≈69% — slightly below body-1, since the single-actor forward on the
second body occasionally wins the max even on interaction classes), so we
report body-1-only. Causality costs only 1.4pp (71.8→70.4), matching the
BoxingVI streaming result. At approximately 82× fewer params than ST-GCN and 39× fewer
than CTR-GCN, this is the legibility-per-parameter point on the curve, not a
SOTA-chase. Reproduce offline: `python3 scripts/lib/ntuBenchmark.py --data-dir
<skeletons> --epochs 80` (add `--causal` for the streaming row); ~8 min on an
M-series MPS, 40,091 clips after denoising.

Flagged (not first-party): joint-stream-only 2s-AGCN ≈85.8% XSub and
joint-stream CTR-GCN ≈89.7–90.0% XSub are reproduction numbers, not in the
original papers — quote only with that caveat.

## PKU-MMD phase 1 — continuous detection, cross-subject

Protocol: official cross-subject split (57 train / 9 test subjects; 944/132
videos); metric = mAP_action at IoU θ=0.5 computed by a verbatim python3 port
of the official `evaluate.py` (github.com/ECHO960/PKU-MMD); 30 FPS.
Command: `python3 scripts/lib/pkummdBenchmark.py --data-dir <dir> [--causal]`

| Method | mAP@0.5 (CS) | Causal | Nameable | Source |
|---|---|---|---|---|
| SVM baseline | 0.021 | no | no | arXiv:1703.07475 T4 |
| BLSTM baseline | 0.130 | no | no | arXiv:1703.07475 T4 |
| JCRRNN baseline | 0.325 | no | no | arXiv:1703.07475 T4 |
| CNN window proposal (Li 2017) | 0.904 | no | no | arXiv:1704.07595 T3 |
| HCN (IJCAI 2018) | 0.926 | no | no | arXiv:1804.06055 T4 |
| LegibleTCN offline, rf=17 (37k) | 0.144 | no | yes | this repo (superseded) |
| LegibleTCN causal, rf=17 (37k) | 0.141 | yes | yes | this repo (superseded) |
| **LegibleTCN offline, dilated rf=129 (40k)** | **0.377** | no | yes (all) | this repo |
| **LegibleTCN causal, dilated rf=129 (40k)** | **0.213** | **yes** | yes (all) | this repo |

**Root-cause fix (2026-07-06): receptive field.** The rf=17 model saw only
~0.57s; PKU-MMD actions have a median duration of **98 frames (3.3s)** and 99%
exceed 17 frames — the model integrated ~17% of a median action. Replacing the
single depthwise temporal conv with a **legible WaveNet-style dilated stack**
(dilations 1,2,4,8, k=9 → 129-frame / 4.3s receptive field, +2.6k params;
each depthwise dilated conv stays a named per-unit temporal template, and
causality is verified exact — perturbing the last frame moves only the last
frame's logits) lifted offline mAP **0.144→0.377 (+162%)** and causal mAP
**0.141→0.213 (+51%)**. `--dilations 1,2,4,8 --window 256`.

Note the flip: at rf=17, offline≈causal (0.144≈0.141) — the window was too
short for future context to matter. At rf=129, offline (0.377) far exceeds
causal (0.213): a 4.3s window lets the bidirectional model see the action
*complete*, which the zero-lookahead model cannot. That gap (0.164 mAP) is the
honest, now-visible **cost of causality**, which the tiny old window was hiding.
Still not competitive with the 1–10M-param published detectors on raw mAP — the
framing remains "legible, causal, 40k params," and the causal streaming row is
the contribution.

**Causal streaming row** (the differentiator — nobody in the table above
publishes a causal number): left-pad-only convs + per-timestep GroupNorm
(zero lookahead, verified exact), hysteresis event extraction, one-to-one greedy
matching, onset tolerance ±250ms, latency = arm-frame − GT-start at 30 fps.
Hysteresis was tuned on a 150-video pilot (arm p≥**0.25** for 3 frames, release
<0.2) and applied fixed to the full test set — the default 0.5 arm rarely clears
a 52-way softmax for 3 straight frames, starving recall. `--arm-thr 0.25
--rel-thr 0.2`.

| Model | event-F1@250ms | precision | recall | median latency | Params |
|---|---|---|---|---|---|
| LegibleTCN causal, rf=17, arm0.5 (37k) | 0.126 | 0.123 | 0.129 | 167 ms | 37k |
| **LegibleTCN causal, dilated, tuned (40k)** | **0.190** | 0.151 | **0.255** | 200 ms | 40k |

The two levers stack: dilation (see the whole action) + hysteresis tuned to the
52-way softmax (arm 0.5→0.25) took causal streaming F1 **0.126→0.190 (+50%)**,
roughly doubling recall (0.129→0.255) at 40k params, still causal, still named.
Pilot ladder that predicted it: baseline 0.044 → dilated 0.057 → dilated+tuned
0.079.
