A drone-based computer vision platform for Toledo Football that converts overhead practice footage into clip-linked player tracking, football labels, self-scout insights, and coach-correctable analytics.
Toledo Football IQ is an active computer vision and sports analytics platform for University of Toledo Football. It is designed to convert drone-based overhead practice footage into structured, coach-verifiable football intelligence.
The core principle is evidence-first analytics. Every output connects to the exact clip, overlay, confidence score, correction path, and model version. If a coach cannot verify it, correct it, and teach from it — it should not be treated as a production metric.
The system is built as a coach-in-the-loop football intelligence platform. It starts with reliable practice-film infrastructure — video ingestion, play segmentation, field calibration, player tracking, clip-linked metrics, coach correction — and grows toward formation and motion analysis, route and coverage analytics, self-scout, pose-lite biomechanics, player development profiles, similar-rep search, zero-shot concept discovery, and same-session feedback.
Product north star: a Toledo-owned CV platform that converts drone and practice film into verified football intelligence — faster coaching feedback, better self-scout, better opponent prep, objective player development, and health-aware workload context.
Football programs already use film heavily. Coaches and analysts manually tag plays, identify formations, evaluate routes, track player effort, study tendencies, and create cutups. This works — but it is time-intensive, and the bottleneck gets worse as the volume of drone and practice footage grows.
The challenge is not just automation. The challenge is trust. A football computer vision system must handle:
Overhead drone footage involves dynamic homography — the camera moves, drifts, and reorients, meaning field calibration must be recomputed continuously, not once.
Players in crowded line-of-scrimmage situations overlap heavily. Standard detection and tracking methods degrade significantly in high-occlusion football scenarios.
Players wear helmets and similar uniforms. Jersey numbers blur at distance or are never visible from overhead. Identity association requires multiple signal types.
Generic CV labels like "mesh" or "outside zone" may not match Toledo's internal call language. A system that speaks wrong terminology loses coach adoption immediately.
A model that reports confident metrics on a poorly calibrated clip is worse than no model. Calibration confidence must gate downstream analytics.
Effort, load, and biomechanics outputs touch athlete welfare. Role-based access, conservative labeling, and no automated medical inference are non-negotiable.
Commercial tools can provide broad AI tagging, but Toledo can create a local advantage through overhead drone footage, Toledo-specific terminology, a coach-correction flywheel, and athlete-development integration. That private, corrected, domain-specific data is the moat.
Probe FPS, resolution, codec, duration, corruption, and metadata. Store raw video and queue processing jobs.
Split continuous practice video into per-play clips with editable boundaries. Manual correction is a first-class feature, not an afterthought.
Detect sidelines, yard lines, hash marks, field numbers, and boundaries. Estimate homography into standardized football field coordinates. Output calibration confidence and analytics-safe status — weak calibration suppresses downstream metrics.
YOLO or RF-DETR detects players, officials, and the ball. Detection confidence feeds the calibration-safe gating layer.
ByteTrack for speed and throughput. BoT-SORT for identity-aware tracking. SAM + CSRT-style approaches for heavy occlusion cases at the line of scrimmage.
Combine jersey OCR, roster priors, height/weight/position priors, equipment cues, biometric ratios, and movement signatures. Every identity assignment carries a confidence score and a correction path.
Detect snap, motion start/end, handoff, throw, catch, contact, tackle, and end-of-play events. Manual correction is supported early — automated event detection improves as corrections accumulate.
Propose offensive formation, defensive front, coverage shell, motion, shift, route family, and pressure candidates. Separate generic labels from Toledo-specific terminology.
Compute cushion, separation, distance traveled, max speed, return-to-huddle speed, time to throw, dropback depth, gap width proxy, pass-set depth, pursuit effort, and downfield blocking participation.
Render clips with field grids, tracks, route paths, motion arrows, coverage shells, confidence warnings, and metric callouts. Index for dashboard search and coach review.
Every layer of the stack is chosen to serve the evidence-first contract: every metric must be traceable to a clip, a model version, a confidence score, and a correction path.
Every table answers one question for any output: where did this metric come from, what model produced it, what evidence supports it, and has a coach corrected it?
The coach_corrections table is the flywheel engine — every correction to any output type is recorded, timestamped, and exported to training_datasets for model retraining. The metrics.suppressed flag prevents analytics from surfacing when calibration confidence is below threshold.
Coaches and analysts correct play boundaries, formation labels, route labels, coverage labels, player identity, and metrics. Every correction becomes structured Toledo-specific training data. This is the competitive moat: private, corrected, domain-specific data beats any off-the-shelf model at Toledo football terminology.
The platform separates generic football labels from Toledo-specific call language. A concept can have a generic label like "mesh" while also mapping to Toledo's internal terminology. This keeps ML portable while making the product feel native to staff.
Each clip receives an analytics-safe score based on field visibility, homography stability, player scale, motion blur, drone drift, and occlusion density. Weak calibration suppresses precise speed, separation, and distance metrics before they can mislead coaching decisions.
Identifies what Toledo is unintentionally revealing: formation-to-play tendencies, motion-to-play tendencies, field-zone tendencies, personnel tendencies, and pre-snap tells. Shows what a prepared opponent would see.
Coarse, coach-useful measures from overhead footage: pad level, hip flexion, torso angle, pass-set depth, strike timing, shoulder-over-knee alignment, deceleration step count, hip sink depth, stride symmetry, QB shoulder-hip separation. These remain experimental until reviewed by the relevant position coach.
Longitudinal development profiles: roster bio, role summary, development goals, coach notes, best teaching clips, best recruiting clips, corrected metrics, position benchmarks, player-facing summaries, workload visibility controls, and weekly snapshots. This turns the platform into a player-development system, not just a film tagger.
Coaches select a clip and find similar reps based on movement, spacing, formation, route concept, or play embeddings (pgvector). Valuable when labels are incomplete or when scouting opponent concepts.
A later phase targets period-break feedback: clip-ready metrics to position coaches within 5–10 minutes between periods. Full sessions processed within 60 minutes of practice ending. Lightweight models run during practice; full-quality runs run overnight.
A coach who does not trust the system will not use it. Building trust — through evidence links, correction paths, confidence scores, and conservative suppression — is more important than adding advanced features early.
The coach-correction flywheel must exist before pose-lite biomechanics, similar-rep search, or zero-shot discovery. Without corrections, there is no Toledo-specific training data. Without Toledo data, the models are generic.
A model that reports confident metrics on a poorly calibrated clip is actively harmful. Suppressing metrics when calibration is weak is a feature, not a limitation.
Dynamic homography, camera drift, high-occlusion football scenarios, and helmeted players with similar uniforms make drone footage harder than broadcast footage. These constraints need to be part of the design, not discovered during development.
A system that takes 24 hours to process a practice session cannot change practice behavior. Same-session feedback — even lightweight — is worth more than overnight perfect analysis.
Frontier experiments — zero-shot discovery, quantum AI sandbox, advanced health fusion — need isolation from the production coaching workflow. Experimental outputs must be clearly labeled and role-gated.
Effort, load, and biomechanics data touch athlete welfare. Role-based access, conservative labeling, and a strict no-automated-medical-inference rule are non-negotiable from the start, not retrofitted later.
In football analytics, trust is the product.
Toledo Football IQ demonstrates that computer vision in sports is not a model problem — it is a product problem. The best moat is private, corrected, domain-specific data. Every coach correction is a training example. Every trusted output extends what the system can do next.