# **AIPT: A Real-Time Vision-Based System for Injury-Aware Exercise Coaching** ## Mitul Aggarwal # Abstract {#abstract} Access to personalised exercise supervision is limited, particularly for individuals recovering from injury or returning to training without professional oversight. While recent advances in computer vision enable real-time human pose estimation using consumer-grade cameras, translating raw pose data into meaningful, safety-oriented feedback remains a challenge. This paper presents my project, AI Personal Trainer (AIPT), an end-to-end system that provides real-time, injury-aware exercise coaching using a standard laptop RGB camera, with future extensibility to mobile platforms. AIPT combines monocular pose estimation with deterministic biomechanical analysis, exercise-specific feature extraction, and temporal smoothing to deliver live corrective feedback, repetition counting, and form scoring without wearables or laboratory equipment. The system computes joint kinematics through vector-based geometry, adapts thresholds based on camera orientation, and stabilises noisy signals using exponential moving averages and multi-frame phase validation. We evaluate the system through quantitative analysis of joint-angle stability, repetition detection accuracy, and real-time performance, demonstrating improved robustness without sacrificing responsiveness. The camera input stream runs at 30 FPS (1440p), while the end-to-end coaching pipeline sustains approximately 18-20 FPS on consumer laptop hardware, and supports multiple rehabilitation and strength-training exercises. The project was developed as part of the ETH Zurich KI Challenge and was awarded first place in the “AI for Good” category for its accessibility and potential impact. This work demonstrates the feasibility of reliable, explainable, camera-only AI coaching for unsupervised exercise, while establishing a foundation for future deployment on mobile devices. # Table of Contents {#table-of-contents} [**Abstract 1**](#abstract) [**Table of Contents 2**](#table-of-contents) [**Introduction 3**](#introduction) [**Related Work 4**](#related-work) [Pose Estimation Frameworks for Real-Time Human Motion Sensing 4](#pose-estimation-frameworks-for-real-time-human-motion-sensing) [Exercise Assessment, Repetition Counting, and Real-Time Feedback 5](#exercise-assessment,-repetition-counting,-and-real-time-feedback) [Commercial Computer-Vision Fitness Platforms 5](#commercial-computer-vision-fitness-platforms) [Author Prior Work: Video Analysis for Sports Motion Tracking 5](#author-prior-work:-video-analysis-for-sports-motion-tracking) [**System Overview 6**](#system-overview) [System Architecture 6](#system-architecture) [User Profiling and Personalised Plan Generation 7](#user-profiling-and-personalised-plan-generation) [Camera Setup and Real-Time Constraints 7](#camera-setup-and-real-time-constraints) [Pose Estimation Layer 7](#pose-estimation-layer) [Data Flow and Adaptive Feedback Loop 8](#data-flow-and-adaptive-feedback-loop) [Design for Extensibility 8](#design-for-extensibility) [Privacy and Data Handling 8](#privacy-and-data-handling) [**Pose Estimation and Kinematic Analysis 9**](#pose-estimation-and-kinematic-analysis) [Pose Landmark Extraction 9](#pose-landmark-extraction) [Joint Angle Computation 9](#joint-angle-computation) [Pose Geometry Utilities 10](#pose-geometry-utilities) [Orientation Estimation and Viewpoint Handling 11](#orientation-estimation-and-viewpoint-handling) [Exercise-Specific Feature Extraction 11](#exercise-specific-feature-extraction) [From Kinematics to Temporal Analysis 12](#from-kinematics-to-temporal-analysis) [**Temporal Smoothing, Phase Detection, and Repetition Analysis 12**](#temporal-smoothing,-phase-detection,-and-repetition-analysis) [Joint Angle Smoothing 13](#joint-angle-smoothing) [Phase Detection and Temporal Validation 13](#phase-detection-and-temporal-validation) [Repetition Boundary Detection 14](#repetition-boundary-detection) [Quantitative Evaluation of Temporal Refinement 15](#quantitative-evaluation-of-temporal-refinement) [Implications for Real-Time Feedback and Coaching 16](#implications-for-real-time-feedback-and-coaching) [**Feedback Generation, Scoring, and User Interaction 16**](#feedback-generation,-scoring,-and-user-interaction) [Feedback Design Principles 16](#feedback-design-principles) [Real-Time Corrective Cues 17](#real-time-corrective-cues) [Form Scoring and Progress Indicators 18](#form-scoring-and-progress-indicators) [User Interface and System Integration 19](#user-interface-and-system-integration) [End-to-End Interaction Flow 19](#end-to-end-interaction-flow) [**Evaluation and Limitations 20**](#evaluation-and-limitations) [Evaluation Scope and Methodology 20](#evaluation-scope-and-methodology) [Signal Stability and Temporal Robustness 20](#signal-stability-and-temporal-robustness) [Repetition Detection Accuracy 20](#repetition-detection-accuracy) [Real-Time Performance 21](#real-time-performance) [Limitations 21](#limitations) [Summary of Evaluations and Limitations 22](#summary-of-evaluations-and-limitations) [**Discussion and Future Work 22**](#discussion-and-future-work) [Deterministic Reasoning versus Data-Driven Models 22](#deterministic-reasoning-versus-data-driven-models) [Pose Estimation as a Sensing Layer 23](#pose-estimation-as-a-sensing-layer) [Temporal Reasoning as a First-Class Design Concern 23](#temporal-reasoning-as-a-first-class-design-concern) [Implications for Accessibility and Injury-Aware Coaching 23](#implications-for-accessibility-and-injury-aware-coaching) [Future Work 24](#future-work) [Summary of Discussions and Future Work 24](#summary-of-discussions-and-future-work) [**Conclusion 24**](#conclusion) [**References 25**](#references) # Introduction {#introduction} Unsupervised exercise presents a significant challenge for individuals recovering from injury or beginning strength training without professional guidance. Physiotherapists and personal trainers play a critical role in correcting technique, preventing reinjury, and building user confidence; however, access to such supervision is often limited by cost, availability, or geography. As a result, many individuals rely on static instructional videos or written exercise plans, which provide no real-time feedback and cannot adapt to user-specific movement patterns. Recent advances in computer vision have enabled real-time human pose estimation using consumer-grade cameras, including those embedded in laptops and mobile devices. These developments create opportunities for accessible, software-only coaching systems that do not require specialised hardware. However, while pose estimation models can reliably detect body landmarks, converting raw pose data into interpretable, safety-focused feedback remains nontrivial. Exercise technique depends not only on joint positions but also on temporal dynamics, orientation relative to the camera, and exercise-specific biomechanical constraints. Naïve implementations are often susceptible to landmark jitter, false repetition detection, and misleading feedback, particularly in uncontrolled home environments. This paper introduces my project, AI Personal Trainer (AIPT), a real-time, vision-based system designed to bridge the gap between pose detection and practical exercise supervision using a laptop-based camera setup. AIPT tracks full-body motion, computes joint kinematics, and evaluates technique against deterministic, explainable biomechanical criteria. Rather than relying on opaque machine-learning classifiers, the system prioritises interpretability and reliability through explicit angle computation, per-exercise feature selection, and temporal smoothing. Although the current implementation is designed for laptop use, the system architecture and computational constraints were chosen with future mobile deployment in mind, enabling potential extension to smartphone-based coaching without fundamental changes to the underlying logic. The primary contributions of this work are: 1. An end-to-end, real-time architecture for camera-only exercise coaching on consumer laptop hardware. 2. A deterministic kinematic analysis pipeline that converts pose landmarks into exercise-specific, interpretable metrics. 3. A temporal refinement strategy combining exponential smoothing and multi-frame phase validation to improve robustness and repetition accuracy. 4. A fully integrated system demonstrating live feedback, form scoring, and user interaction at real-time frame rates, with a clear path toward mobile adaptation. The full implementation and demo are available at the project repository: [https://github.com/MitulAggarwal1/AI-Personal-Trainer](https://github.com/MitulAggarwal1/AI-Personal-Trainer). # Related Work {#related-work} Camera-based exercise coaching sits at the intersection of real-time pose estimation, biomechanics-inspired feature engineering, and feedback systems for human movement. Prior work spans (i) pose estimation frameworks that enable robust landmark extraction from RGB video, (ii) academic systems for repetition counting and posture assessment, and (iii) commercial fitness platforms that deliver real-time coaching at scale. This section situates AIPT within that landscape and motivates its emphasis on interpretable, real-time kinematics and temporal stability. ## Pose Estimation Frameworks for Real-Time Human Motion Sensing {#pose-estimation-frameworks-for-real-time-human-motion-sensing} Recent advances in human pose estimation have enabled practical, camera-only movement analysis by mapping RGB frames to a sparse set of body landmarks. AIPT builds on the premise that pose estimation should be treated as a sensing layer rather than an end-to-end solution: raw landmarks must be converted into biomechanically meaningful signals (e.g., joint angles, alignment) and stabilised over time to support reliable feedback. A widely used approach for building perception pipelines is MediaPipe, which provides modular graph-based processing for real-time inference and cross-platform deployment \[1\]. For single-person pose tracking on commodity devices, BlazePose (used within the MediaPipe Pose pipeline) is a lightweight architecture designed for on-device, real-time performance, producing 33 body keypoints and enabling fitness-oriented use cases \[2\]. In contrast, OpenPose introduced bottom-up multi-person pose estimation using Part Affinity Fields and remains influential for pose estimation research and applications \[3\]. While OpenPose-style methods can be powerful, AIPT prioritises single-person, real-time coaching on consumer hardware, making a lightweight, on-device-oriented pose pipeline more appropriate. ## Exercise Assessment, Repetition Counting, and Real-Time Feedback {#exercise-assessment,-repetition-counting,-and-real-time-feedback} A major challenge in exercise coaching systems is translating frame-level pose measurements into stable, actionable feedback. Research systems commonly focus on (a) recognising exercises, (b) counting repetitions, and (c) assessing form. A recurring theme is that naive thresholding on raw angles is brittle: landmark jitter and brief occlusions can produce unstable phase detection and false repetition events. Several academic works combine pose estimation with temporal logic or learning-based models to recognise exercises and count repetitions in real time. For example, Alatiah and Chen propose a real-time system combining pose estimation with deep learning for repetition analysis and feedback \[4\]. More recent work explicitly targets on-device repetition counting on smartphones, highlighting practical constraints (latency, compute limits, robustness) that overlap with AIPT’s deployment goals \[5\]. Collectively, this literature supports the approach that real-time systems must incorporate temporal reasoning-filtering, state validation, and debouncing-to convert noisy kinematics into stable coaching signals. AIPT’s contribution is aligned with this direction but intentionally emphasises interpretability: rather than relying on opaque classifiers, it uses deterministic, exercise-specific kinematic checks (joint angles, alignment proxies) coupled with temporal validation to generate explainable cues and scores. ## Commercial Computer-Vision Fitness Platforms {#commercial-computer-vision-fitness-platforms} Commercial platforms demonstrate user demand for camera-assisted strength training and form feedback, but their internal algorithms and datasets are rarely transparent. Peloton introduced Peloton Guide as a camera-enabled strength product that uses machine learning to enhance strength training experiences \[6\], and Peloton’s broader “AI-enabled” planning features illustrate how coaching is increasingly paired with personalised training roadmaps \[7\]. Tempo markets computer-vision-driven training with real-time tracking and guidance \[8\], and technical coverage describes Tempo’s use of a 3D motion-tracking camera for scanning and coaching users during workouts \[9\]. Tonal has publicly described “Form Feedback” as a core coaching feature, emphasising technique correction alongside repetition tracking \[10\]. These systems support the relevance of camera-based coaching but also motivate AIPT’s focus on auditable, biomechanics-linked feedback that can be inspected, tuned, and communicated clearly in a research setting. ## Author Prior Work: Video Analysis for Sports Motion Tracking {#author-prior-work:-video-analysis-for-sports-motion-tracking} AIPT was also informed by earlier work applying computer vision to motion analysis in a different domain: a golf launch monitor that estimated ball metrics from video. That project strengthened core skills directly transferable to AIPT, including managing noisy video signals, building reliable OpenCV pipelines, and debugging “real-world failure modes” caused by lighting changes, camera geometry, and imperfect detections. This prior experience shaped AIPT’s design philosophy: in practice, performance improvements often come less from adding complexity and more from identifying instability sources and designing robust signal processing and validation around them. # System Overview {#system-overview} AIPT is an end-to-end, real-time exercise coaching system designed to operate using a standard laptop RGB camera. The system combines user-specific contextual input with live computer-vision-based movement analysis to deliver personalised, injury-aware exercise guidance. Its architecture separates user profiling, sensing, kinematic analysis, decision logic, and feedback, prioritising robustness, explainability, and responsiveness on consumer hardware while maintaining extensibility for future mobile deployment. In this context, “injury-aware” refers to user-declared constraints (pain location, triggers, recovery stage) that restrict exercise selection and tighten technique thresholds; it does not constitute a medical diagnosis. ## System Architecture {#system-architecture} At a high level, AIPT consists of six primary components: (1) user profiling and personalisation, (2) video acquisition, (3) pose estimation, (4) kinematic feature extraction, (5) temporal analysis and exercise evaluation, and (6) feedback and user interface. Before exercise execution begins, users input a profile describing relevant contextual factors such as injury type, pain triggers, recovery stage, and training goals. This information is processed by a rule-based decision system that generates a personalised workout plan, including exercise selection, repetition ranges, and cautionary constraints. The output of this module defines which exercises are available for real-time monitoring and which biomechanical metrics should be prioritised during evaluation. During execution, live video frames are captured from the laptop camera and processed in real time. Each frame is passed to a pose estimation module based on MediaPipe, which outputs 2D body landmarks with associated confidence values. These landmarks form the sensing layer of the system and are intentionally decoupled from higher-level reasoning to preserve modularity and interpretability. The extracted landmarks are then processed by a custom kinematic analysis layer that computes joint angles, relative limb orientations, and normalised geometric features. These metrics are fed into an exercise-specific evaluation module that determines movement phase, repetition count, and form quality in real time. Results are communicated to the user through a graphical interface that overlays visual cues and textual feedback onto the live video stream. ## User Profiling and Personalised Plan Generation {#user-profiling-and-personalised-plan-generation} Personalisation in AIPT begins before any movement is analysed. Users provide structured input describing their physical context, including injury location, pain sensitivity, recovery duration, and movement limitations. Rather than relying on opaque machine-learning recommendations, AIPT employs a deterministic decision-tree-based logic to generate personalised workout plans. This decision tree maps user inputs to exercise eligibility, intensity constraints, and cautionary rules. For example, users reporting knee pain may receive reduced squat depth recommendations or be redirected to alternative lower-impact movements. Each generated plan includes: * a tailored set of exercises, * recommended repetition and set ranges, * and exercise-specific safety constraints. This design ensures that personalisation is transparent, explainable, and aligned with rehabilitation principles. Importantly, the personalisation logic directly informs the downstream analysis pipeline by activating only the relevant kinematic features and thresholds for each selected exercise. ## Camera Setup and Real-Time Constraints {#camera-setup-and-real-time-constraints} The system is designed to function under realistic home-use conditions, assuming a single, fixed laptop camera positioned approximately at torso height. No depth sensors, external cameras, or wearable devices are required. Users may face the camera directly or perform exercises side-on, depending on space constraints. Because real-time feedback is essential for coaching utility, the system is optimised to maintain interactive responsiveness. All processing stages run in a single loop and are able to process frames in real time on consumer laptop hardware. All experiments were run on an Intel Core i7 with 16GB RAM and Windows 11, using the built-in RGB camera at 1440p and an input stream of approximately 30 FPS. The pipeline was implemented in Python using OpenCV and MediaPipe Pose \[1\], \[2\], with all processing performed locally. Typical capture conditions involved indoor ambient lighting and a camera-to-subject distance of approximately 1.5-3 m. Lightweight geometric computation and deterministic logic were selected to minimise latency and prevent dropped frames during extended exercise sessions. ## Pose Estimation Layer {#pose-estimation-layer} AIPT uses MediaPipe for full-body pose estimation, extracting key landmarks including shoulders, elbows, hips, knees, wrists, and ankles. These landmarks are represented as normalised image coordinates and updated for each frame. Pose estimation serves strictly as a sensing layer. Raw landmarks are not used directly for evaluation; instead, they are transformed into biomechanical representations that better reflect human movement constraints. To accommodate different user orientations, the system estimates whether the user is facing the camera or positioned side-on by comparing left-right shoulder depth and horizontal displacement. This orientation estimate determines which joints and angles are prioritised during analysis. ## Data Flow and Adaptive Feedback Loop {#data-flow-and-adaptive-feedback-loop} Once pose landmarks are detected, joint kinematics are computed and passed to the exercise evaluation logic. Movement phase (e.g., contraction vs extension), repetition boundaries, and form quality are assessed continuously using exercise-specific criteria derived from the personalised plan. Feedback is generated in real time and displayed through an overlay that includes: * skeletal visualisation, * textual corrective cues, * repetition count, * and a dynamic form score. Crucially, feedback adapts dynamically to both the user’s live movement and their profile constraints. If unsafe patterns persist or pain-sensitive thresholds are exceeded, the system modifies cues in real time, encouraging correction or termination of the exercise as appropriate. This closed-loop interaction enables users to adjust their technique mid-repetition, reinforcing safe movement habits rather than relying on retrospective correction. ## Design for Extensibility {#design-for-extensibility} Although the current implementation targets laptop-based use, the system architecture was designed with future mobile deployment in mind. The reliance on monocular RGB input, lightweight computation, and modular processing stages allows the personalisation logic, kinematic analysis, and feedback pipeline to be transferred to mobile platforms without fundamental restructuring. By decoupling user profiling, pose sensing, and biomechanical reasoning, AIPT establishes a flexible foundation that can support expanded exercise libraries, adaptive threshold learning, or data-driven personalisation in future iterations. ## Privacy and Data Handling {#privacy-and-data-handling} AIPT processes video frames locally to extract pose landmarks and compute kinematic metrics. The system does not require uploading footage to external servers for core functionality. Any recording of video for debugging or evaluation is optional and user-controlled. # Pose Estimation and Kinematic Analysis {#pose-estimation-and-kinematic-analysis} This section describes how raw video input is transformed into stable, interpretable biomechanical representations suitable for real-time exercise evaluation. AIPT combines monocular pose estimation with custom geometric analysis and orientation-aware logic to ensure robustness across viewpoints and exercises. ## Pose Landmark Extraction {#pose-landmark-extraction} AIPT uses MediaPipe’s full-body pose model to extract 2D landmarks corresponding to major anatomical joints, including the shoulders, elbows, hips, knees, wrists, and ankles. Each landmark is represented as a normalised coordinate relative to the image frame and updated for every video frame captured by the laptop camera. Pose estimation operates continuously as the system’s sensing layer and is intentionally isolated from higher-level decision logic. Raw landmarks are not directly used for form assessment; instead, they serve as inputs to a downstream kinematic transformation pipeline. This separation ensures that transient pose noise or partial occlusions do not immediately propagate into incorrect feedback. Figure 1\. Full-body pose landmark extraction using a laptop RGB camera *(Real-time pose estimation output showing detected body landmarks and skeletal connections for major joints, including shoulders, elbows, hips, knees, wrists, and ankles. This overlay represents the sensing layer of AIPT, where pose landmarks are extracted from monocular video and passed to downstream kinematic analysis.)* ## ## ## Joint Angle Computation {#joint-angle-computation} To convert pose landmarks into meaningful biomechanical metrics, AIPT computes joint angles using vector-based geometry. For a joint of interest defined by three landmarks, a, b, and c, two vectors are constructed: u=a-b, v=c-b This formulation yields numerically stable joint angles in the range 0°-180°, avoiding discontinuities associated with inverse cosine-based methods. Angles exceeding 180° due to noise are automatically wrapped to preserve biomechanical validity. This approach enables consistent measurement of elbow flexion, knee flexion, shoulder extension, and hip hinge angles across exercises. Because all downstream evaluation logic depends on these kinematic signals, numerical stability and interpretability were prioritised over computational complexity. Figure 2 illustrates the direct correspondence between the mathematical formulation and its implementation in code, showing how raw pose landmarks are converted into stable joint-angle measurements in real time. *![][image1]* Figure 2\. Vector-based joint angle computation implementation *(Code snippet illustrating the calculate\_angle(a, b, c) function used to compute joint angles from pose landmarks. The implementation applies an arctan2-based formulation to produce stable angles in the range 0 °- 180°, forming the mathematical backbone of all kinematic analysis in AIPT.)* ## Pose Geometry Utilities {#pose-geometry-utilities} Beyond raw joint angles, AIPT employs a set of lightweight geometric helper functions to capture exercise-relevant constraints without assuming fixed camera distance or calibration. These include: * Vertical alignment metrics, used to assess whether joints remain level relative to gravity (e.g., wrist-shoulder alignment during presses), * Horizontal displacement measures, used to detect lateral drift or imbalance, * Normalised offsets, which allow feedback such as “keep wrists level with shoulders” to remain valid across different camera positions. These utilities translate abstract pose geometry into intuitive coaching cues while remaining robust to variations in camera placement. By operating on normalised landmark coordinates, the system maintains consistent behaviour across different users and environments. ## Orientation Estimation and Viewpoint Handling {#orientation-estimation-and-viewpoint-handling} Camera viewpoint significantly affects the reliability of kinematic measurements. To address this, AIPT estimates user orientation by comparing left-right shoulder displacement and relative depth cues derived from pose landmarks. If the user is detected to be side-on, the system prioritises sagittal-plane joints such as the knee, hip, and back angle, which are most informative from that viewpoint. For front-facing orientations, analysis emphasises symmetry and vertical alignment. Orientation estimation directly determines which kinematic features are activated and which evaluation thresholds are applied. This viewpoint-aware logic reduces false feedback caused by depth ambiguity and ensures that joint measurements are interpreted within the appropriate biomechanical context. ## Exercise-Specific Feature Extraction {#exercise-specific-feature-extraction} Rather than evaluating all joints for every movement, AIPT employs an exercise-specific feature extraction strategy. For each exercise, a compact set of task-relevant kinematic metrics is selected and passed to the evaluation logic. Examples include: * Squats and deadlifts: knee angle, back angle, torso uprightness, neck alignment * Bicep curls: elbow angle, upper-arm verticality, torso stability * Shoulder presses: arm vertical alignment, shoulder symmetry This design decouples sensing from judgment: downstream logic consumes small, interpretable feature vectors (e.g., `{elbow_angle: 54.3°, back_angle: 168.1°}`) rather than raw pose landmarks. This improves explainability, simplifies debugging, and allows new exercises to be added without restructuring the entire pipeline. ![][image2] Figure 3\. Exercise-specific kinematic feature selection *(Example of task-specific feature extraction, where raw pose landmarks are transformed into compact, interpretable kinematic vectors tailored to individual exercises. This abstraction enables scalable and explainable evaluation logic.)* ## From Kinematics to Temporal Analysis {#from-kinematics-to-temporal-analysis} The kinematic metrics described above form the input to the system’s temporal reasoning layer, which determines movement phase, repetition boundaries, and form stability over time. However, raw joint-angle signals exhibit frame-to-frame noise due to pose jitter and minor detection errors. To ensure reliable interpretation, AIPT applies temporal smoothing and multi-frame validation before deriving higher-level movement events. These techniques transform raw kinematic streams into stable signals suitable for repetition counting and safety evaluation. The temporal refinement methods used to achieve this robustness are described in the following section. # Temporal Smoothing, Phase Detection, and Repetition Analysis {#temporal-smoothing,-phase-detection,-and-repetition-analysis} Although joint-angle computation provides meaningful kinematic descriptors of human movement, raw pose-derived signals are inherently noisy when extracted from monocular RGB video. Small variations in landmark placement, partial occlusions, and frame-to-frame detection uncertainty introduce high-frequency fluctuations that are insignificant biomechanically but problematic for real-time analysis. If left unaddressed, this noise leads to unstable feedback, false repetition counts, and inconsistent exercise evaluation. AIPT addresses these challenges through a layered temporal reasoning pipeline that stabilises kinematic signals, enforces phase consistency, and robustly detects repetition boundaries. Rather than relying on data-driven sequence models, the system employs lightweight, deterministic temporal logic that is transparent, computationally efficient, and well-suited for real-time execution on consumer hardware. ## Joint Angle Smoothing {#joint-angle-smoothing} Raw joint-angle time series extracted from pose landmarks exhibit jitter that does not correspond to true joint motion. This noise is particularly pronounced at joint extrema, where small landmark errors can produce large angular deviations. In preliminary testing, such jitter resulted in oscillating feedback cues and unstable repetition detection, even when users performed movements smoothly. To address this, AIPT applies an exponential moving average (EMA) to all joint-angle signals prior to evaluation. For a raw joint angle t at time step t, the smoothed angle is tcomputed as: t=t+(1-)t-1 where ∈ (0, 1\) and controls the balance between responsiveness and noise attenuation. EMA was chosen over alternative filters such as sliding-window averages or Kalman filters due to its minimal computational overhead and ease of real-time integration. Empirically, a smoothing factor of \=0.2 was found to effectively suppress high-frequency noise while preserving the temporal responsiveness necessary for real-time coaching. Importantly, the smoothing is applied uniformly across joints, ensuring consistent behaviour across different exercises and movement speeds. ## Phase Detection and Temporal Validation {#phase-detection-and-temporal-validation} Exercise movements are inherently cyclical, typically alternating between contraction and extension phases. Reliable phase detection is critical for identifying correct movement execution, repetition boundaries, and form deviations. However, naïve phase detection based on instantaneous threshold crossings is highly susceptible to pose jitter, especially near angular boundaries. AIPT employs a state-based phase detection model in which phase transitions are determined by smoothed joint angles crossing exercise-specific thresholds derived from biomechanical constraints. To prevent spurious transitions, a phase change is only confirmed if the threshold condition persists for a predefined number of consecutive frames. This multi-frame validation introduces temporal consistency without adding perceptible delay. It ensures that detected phase transitions reflect deliberate user motion rather than transient measurement error. The result is a stable representation of movement progression that remains reliable across varying execution speeds and camera conditions. By explicitly modelling phase state rather than inferring it implicitly, the system maintains interpretability and avoids the need for sequence learning or recurrent models, which would require larger datasets and introduce opacity into the decision process. ![][image3] Figure 4\. Phase determination and repetition logic with temporal validation *(Code excerpt illustrating the phase-detection and repetition-counting logic used in AIPT for lower- and upper-body exercises. Smoothed joint angles (e.g., knee or elbow) are used to determine movement phase (“up” vs. “down”) based on exercise-specific thresholds. A repetition is registered only after a validated transition through the full movement cycle, with temporal buffering and threshold separation preventing false positives caused by pose jitter or brief pauses.)* ## Repetition Boundary Detection {#repetition-boundary-detection} Repetition counting is a core requirement for exercise coaching systems, yet it is particularly vulnerable to noise-induced errors. In early system iterations, small oscillations near joint-angle extrema frequently led to duplicate counts or missed repetitions, especially during slow or controlled movements. To address this, AIPT defines a repetition as a complete, validated traversal through the full movement cycle, rather than as a simple threshold event. Repetition boundaries are detected by monitoring validated phase transitions and enforcing both directional consistency and temporal separation between successive repetitions. This approach prevents common failure modes, including: * double-counting near threshold boundaries, * false positives caused by brief pauses, * missed repetitions during low-amplitude or rehabilitation-focused movements. By integrating repetition detection with phase state and temporal validation, the system achieves reliable counting across a wide range of exercise styles without relying on heuristic timing assumptions. ## Quantitative Evaluation of Temporal Refinement {#quantitative-evaluation-of-temporal-refinement} To evaluate the effectiveness of temporal smoothing and validation, joint-angle trajectories and repetition detection outputs were compared before and after refinement. Figure 5 presents the knee-angle time series recorded during squat execution for both system versions. In the unrefined system, raw angle traces exhibit high-frequency oscillations that result in unstable phase detection and occasional duplicate repetition markers. After applying EMA smoothing and multi-frame phase confirmation, the signal becomes visibly stable, and repetition markers align consistently with true movement cycles. Crucially, these improvements did not compromise real-time performance. Frame-rate monitoring confirmed that the end-to-end pipeline sustained approximately 18-20 FPS throughout execution, demonstrating that increased robustness was achieved without sacrificing responsiveness. This balance is essential for maintaining a natural feedback experience during exercise. ![][image4] Figure 5\. Effect of temporal smoothing and phase validation on repetition detection *(Comparison of knee-angle trajectories during squat execution before (top) and after (bottom) temporal refinement. Smoothing and multi-frame validation reduce noise, false phase transitions, and align repetition markers with actual movement cycles while preserving real-time performance.)* ## Implications for Real-Time Feedback and Coaching {#implications-for-real-time-feedback-and-coaching} Temporal refinement directly influences the quality and credibility of real-time feedback. Stable kinematic signals enable consistent form scoring, prevent oscillating cues, and ensure that corrective feedback reflects genuine movement errors rather than sensor noise. By combining smoothing, phase validation, and repetition logic, AIPT transforms raw pose data into reliable coaching signals that users can act upon mid-exercise. This reliability is particularly important in injury-aware contexts, where false alerts or inconsistent feedback may reduce confidence or encourage unsafe behaviour. More broadly, these results highlight that effective camera-only exercise coaching depends as much on temporal reasoning as on pose estimation accuracy. The techniques described here convert noisy kinematic streams into structured, interpretable representations suitable for real-time human interaction. # Feedback Generation, Scoring, and User Interaction {#feedback-generation,-scoring,-and-user-interaction} Reliable kinematic and temporal analysis is only useful if it can be translated into feedback that users can understand and act upon in real time. This section describes how AIPT converts stabilised movement signals into interpretable coaching cues, quantifies movement quality through a form score, and integrates these outputs into a responsive user interface suitable for unsupervised exercise and rehabilitation contexts. ## Feedback Design Principles {#feedback-design-principles} The feedback mechanisms in AIPT were designed around three guiding principles: interpretability, timeliness, and safety. Feedback must clearly explain *what* is incorrect, appear quickly enough to allow mid-movement correction, and avoid overwhelming users with excessive or contradictory cues. Rather than providing continuous commentary on all joints, AIPT prioritises exercise-relevant faults identified through the personalised plan and kinematic evaluation pipeline. This selective approach reduces cognitive load and mirrors how human coaches focus attention on the most critical aspects of technique at any given moment. Feedback is generated deterministically from kinematic thresholds and phase state rather than probabilistic classifiers. This ensures that each cue can be traced directly to a biomechanical condition (e.g., insufficient elbow extension or excessive back flexion), increasing transparency and user trust, particularly important in injury-aware scenarios. ## Real-Time Corrective Cues {#real-time-corrective-cues} Corrective feedback is delivered through contextual text overlays displayed alongside the live camera feed. These cues are triggered when stabilised joint angles or alignment metrics violate exercise-specific safety or technique thresholds for a sustained duration. For example, during bicep curls, insufficient elbow extension or excessive shoulder drift prompts a cue to encourage controlled extension. During lower-body exercises, cues may target squat depth, torso angle, or knee alignment. Because all feedback is driven by temporally validated signals, cues remain stable and do not oscillate in response to transient pose noise. This design allows users to correct technique *during* execution rather than after completing a repetition or set, reinforcing safe movement patterns in real time. Figure 6\. Real-time corrective feedback during bicep curl execution *(Live camera feed showing full-body pose tracking with overlaid kinematic feedback during a bicep curl. The system displays the current repetition count, movement phase, camera orientation, and a real-time form score. A targeted corrective cue (“Arm not curling enough”) is triggered when the elbow angle fails to reach the exercise-specific extension threshold, illustrating how stabilised kinematic signals are translated into actionable, joint-level feedback.)* ## Form Scoring and Progress Indicators {#form-scoring-and-progress-indicators} In addition to discrete corrective cues, AIPT provides a continuous form score to summarise overall movement quality. The form score aggregates multiple kinematic checks, such as joint range, alignment, and movement consistency, into a single interpretable metric displayed alongside the video feed. Scores are computed deterministically from validated kinematic signals and normalised to a fixed scale, allowing users to track consistency across repetitions and sessions. Importantly, the score is not intended as a competitive metric, but rather as a feedback signal for self-monitoring, particularly useful in rehabilitation contexts where consistency and control matter more than intensity. By exposing both granular cues and a holistic score, AIPT balances detailed guidance with high-level progress tracking. ![][image5] Figure 7\. Rule-based form scoring using exercise-specific kinematic penalties *(Code excerpt illustrating the deterministic form-scoring mechanism used in AIPT. For each exercise, a set of kinematic constraints is defined as a set of penalty rules specifying a metric, a comparison operator, a threshold, and a score deduction. During execution, stabilised joint-angle and alignment metrics are evaluated against these rules, and the form score is reduced when unsafe or suboptimal technique is detected. This rule-based design ensures that form scores are interpretable, exercise-specific, and directly traceable to biomechanical conditions.)* ## User Interface and System Integration {#user-interface-and-system-integration} The feedback and scoring mechanisms are integrated into a graphical user interface that also supports personalised workout plan execution. Users first generate a plan based on injury and profile inputs, then select exercises directly from the interface to begin monitored execution. Integrating real-time computer vision with a graphical interface required careful coordination between the UI event loop and the backend processing pipeline. Exercise selection triggers backend execution of the corresponding monitoring logic, while pose processing and feedback rendering run continuously to maintain responsiveness. Thread management and data sharing were designed to prevent frame drops or UI freezing, ensuring that visual feedback remains synchronised with user motion at real-time frame rates. ![][image6] Figure 8: GUI integration before and after real-time monitoring *(Comparison of the standalone recovery-plan interface (left) and the integrated system (right), where selecting an exercise launches live pose tracking and feedback.)* ## End-to-End Interaction Flow {#end-to-end-interaction-flow} From the user’s perspective, interaction with AIPT follows a closed feedback loop: profile input informs exercise selection and safety constraints; live execution generates kinematic data; temporal reasoning stabilises signals; and feedback is presented visually and textually in real time. This end-to-end interaction confirms that the system functions not merely as a pose visualiser, but as an interactive coaching tool that connects personalisation, biomechanics, and human-computer interaction. Observing the system respond predictably and consistently to movement validated the architectural decisions made throughout development. # Evaluation and Limitations {#evaluation-and-limitations} This section evaluates the performance and reliability of AIPT as a real-time, camera-only exercise coaching system and discusses the limitations of the current implementation. Given the system’s focus on accessibility and explainability, evaluation prioritised signal stability, responsiveness, and behavioural correctness rather than clinical efficacy. ## Evaluation Scope and Methodology {#evaluation-scope-and-methodology} The evaluation focused on three core system properties critical for real-time coaching: 1. Kinematic signal stability, assessed through joint-angle smoothness and phase consistency during exercise execution. 2. Repetition detection reliability, evaluated by examining alignment between detected repetitions and observed movement cycles. 3. Real-time performance, measured through sustained frame rate during live execution. Evaluation was conducted during repeated execution of supported exercises (e.g., squats, bicep curls, push-ups) under realistic home-use conditions using a laptop camera. Rather than relying on pre-recorded datasets, the evaluation emphasised live system behaviour to reflect the intended deployment context. ## Signal Stability and Temporal Robustness {#signal-stability-and-temporal-robustness} As shown in the Temporal Smoothing section, raw joint-angle signals extracted from pose landmarks exhibited high-frequency noise that interfered with phase detection and repetition counting. After applying exponential moving-average smoothing and multi-frame phase validation, joint-angle trajectories became visibly stable, and phase transitions aligned consistently with true movement cycles. Quantitative comparison of smoothed and unsmoothed signals demonstrated fewer spurious phase transitions and more stable repetition boundaries. These improvements were achieved without introducing perceptible latency, confirming that temporal refinement was effective for real-time use. ## Repetition Detection Accuracy {#repetition-detection-accuracy} Repetition-counting accuracy was evaluated by comparing detected repetitions against manual counting across repeated trials under typical indoor conditions. The state-based repetition logic successfully avoided common failure modes such as double-counting near joint-angle extrema and unstable counts during slow or controlled movements. Because repetition detection is based on validated phase transitions rather than instantaneous thresholds, the system remained robust across varying movement speeds and execution styles. Table 1 demonstrates the testing of the repetition counting. | Exercise | Number of Trials (Sets) | True Reps | Detected Reps | False Positives | False Negatives | | :---- | :---- | :---- | :---- | :---- | :---- | | Squat | 4 | 40 | 40 | 0 | 0 | | Bicep Curl | 4 | 40 | 38 | 0 | 2 | | Push-up | 4 | 40 | 37 | 0 | 3 | Table 1: Repetition counting accuracy *(Reports aggregated repetition-count results across four 10-rep trials per exercise under typical indoor conditions. Errors are reported as false positives (FP) and false negatives (FN) relative to manual counting.)* Across these trials, repetition-count agreement with manual counting was 100% (squat), 95% (bicep curl), and 92.5% (push-up), with errors dominated by false negatives, suggesting that some repetitions failed to fully satisfy the phase-completion thresholds (e.g., reduced range of motion or brief occlusions), rather than being double-counted. ## Real-Time Performance {#real-time-performance} Maintaining interactive responsiveness is essential for effective coaching feedback. Frame-rate monitoring during extended execution confirmed that AIPT consistently operated at approximately 18-20 frames per second on consumer laptop hardware. Importantly, adding temporal smoothing, phase validation, and scoring logic did not result in measurable performance degradation. This confirms that the system’s deterministic design choices, such as lightweight geometry and rule-based evaluation, are well-suited for real-time operation without specialised hardware. ## Limitations {#limitations} Despite these encouraging results, several limitations should be acknowledged. First, the system has not been clinically validated against expert physiotherapist assessments or motion-capture ground truth. While biomechanical thresholds were informed by established exercise principles, the system is not intended to replace professional supervision and should be viewed as a supportive coaching tool. Second, the evaluation was conducted by a single person, and performance may degrade under extreme camera angles, poor lighting, or significant occlusion. Although orientation-aware logic mitigates some viewpoint issues, monocular RGB input inherently limits depth perception. Third, the current personalisation logic relies on deterministic decision trees and fixed thresholds. While this ensures explainability, it does not adapt automatically to individual biomechanics or long-term user progress. More advanced personalisation would require additional data collection and validation. Finally, the current implementation is laptop-based, and mobile deployment introduces additional constraints related to camera placement, computational resources, and user interaction that have not yet been evaluated. ## Summary of Evaluations and Limitations {#summary-of-evaluations-and-limitations} Overall, the evaluation demonstrates that AIPT achieves its primary design goals: stable kinematic analysis, reliable repetition detection, and real-time feedback on consumer hardware. At the same time, the limitations outlined above highlight opportunities for future work, particularly in clinical validation, adaptive personalisation, and mobile deployment. # Discussion and Future Work {#discussion-and-future-work} This work demonstrates that reliable, injury-aware exercise coaching can be achieved using a single RGB camera and deterministic biomechanical reasoning, provided that pose estimation is paired with careful temporal analysis and interpretable evaluation logic. Beyond the specific implementation of AIPT, the system highlights broader design considerations for camera-based human movement analysis and suggests several directions for future research. ## Deterministic Reasoning versus Data-Driven Models {#deterministic-reasoning-versus-data-driven-models} A central design decision in AIPT was the deliberate use of deterministic, rule-based kinematic reasoning rather than end-to-end machine-learning classifiers. While data-driven approaches have shown success in activity recognition and pose classification, they often require large labelled datasets and provide limited transparency into how decisions are made. In safety-critical or injury-aware contexts, this opacity can reduce user trust and make incorrect feedback difficult to diagnose or correct. By contrast, AIPT’s rule-based approach allows every feedback cue and score adjustment to be traced directly to a biomechanical condition, such as insufficient joint range or unsafe alignment. This transparency is particularly valuable in rehabilitation settings, where users may already be cautious about movement and benefit from clear, explainable guidance. The results suggest that for structured, cyclical exercises with well-defined biomechanical constraints, deterministic reasoning can achieve reliable real-time performance without the overhead or interpretability challenges of complex learning-based models. Importantly, this does not imply that learning-based methods are unnecessary. Instead, the system points toward a hybrid paradigm, where deterministic biomechanics establish safe operating bounds and data-driven models are used to personalise thresholds, adapt to individual movement patterns, or predict fatigue over time. ## Pose Estimation as a Sensing Layer {#pose-estimation-as-a-sensing-layer} A key insight from this work is the importance of treating pose estimation as a sensing layer rather than a semantic representation. While modern pose models provide accurate landmark detection, raw landmark coordinates are insufficient for reasoning about movement quality, safety, or intent. Without additional structure, they are highly sensitive to noise, camera orientation, and user-specific variation. By explicitly transforming pose landmarks into joint angles, alignment metrics, and exercise-specific feature vectors, AIPT bridges the gap between low-level vision output and high-level biomechanical interpretation. This abstraction layer enables consistent reasoning across exercises and viewpoints and allows the system to remain robust even when individual landmark estimates fluctuate. This design choice suggests a broader principle for applied computer vision systems that interact with humans in real time: pose estimation should provide inputs to structured reasoning, not act as the final representation on which decisions are made. ## Temporal Reasoning as a First-Class Design Concern {#temporal-reasoning-as-a-first-class-design-concern} The evaluation highlights that accurate pose estimation alone is insufficient for reliable exercise analysis. Temporal instability, manifested as landmark jitter, oscillating angles, and false phase transitions, can severely degrade the user experience if left unaddressed. By incorporating temporal smoothing, phase validation, and state-based repetition logic, AIPT demonstrates that temporal reasoning is as critical as spatial accuracy. These techniques transform noisy, frame-level measurements into stable, interpretable signals suitable for real-time feedback. More broadly, the results suggest that future camera-based coaching systems should treat temporal modelling not as a post-processing step, but as a core component of system design. ## Implications for Accessibility and Injury-Aware Coaching {#implications-for-accessibility-and-injury-aware-coaching} AIPT was designed to operate on consumer laptop hardware without wearables, depth sensors, or specialised equipment. This focus on accessibility reflects the system’s underlying motivation: supporting individuals who lack consistent access to professional supervision but still require reliable guidance during exercise. The combination of personalised plan generation, real-time corrective feedback, and interpretable scoring offers a practical middle ground between static instructional content and in-person coaching. While the system is not a substitute for professional care, it demonstrates how applied AI can reduce uncertainty, reinforce safe technique, and increase confidence during unsupervised training, particularly during early stages of injury recovery or reintroduction to physical activity. These findings suggest that accessible, camera-only systems can play a meaningful role in widening access to safe movement guidance when designed with appropriate safeguards and transparency. ## Future Work {#future-work} Several avenues exist for extending this work. First, clinical validation remains an important next step. Comparing system feedback against expert physiotherapist assessments or motion-capture ground truth would enable quantitative evaluation of biomechanical accuracy and safety. Such studies could also help refine thresholds and scoring logic. Second, the personalisation pipeline could be enhanced through adaptive or hybrid approaches. Deterministic rules could establish baseline safety constraints, while learning-based models personalise feedback based on individual biomechanics, injury history, or longitudinal performance data. Third, mobile deployment represents a natural extension of the current system. Although the architecture is compatible with mobile platforms, further work is required to address challenges related to camera placement, limited field of view, computational constraints, and user interaction design on smartphones. Finally, expanding the exercise library and incorporating longitudinal tracking would allow the system to monitor progress over time, identify trends, and support more comprehensive rehabilitation or training programs. ## Summary of Discussions and Future Work {#summary-of-discussions-and-future-work} Overall, this work highlights the value of combining computer vision with explicit biomechanical reasoning and temporal analysis to produce reliable, explainable, and accessible exercise coaching systems. The insights gained from AIPT extend beyond a single application and contribute to a broader understanding of how real-time human motion analysis can be deployed responsibly and effectively in everyday environments. # Conclusion {#conclusion} This paper presented AIPT, a real-time, camera-based AI personal trainer designed to provide accessible, injury-aware exercise coaching using a single RGB camera. By combining pose estimation with explicit biomechanical reasoning, temporal signal refinement, and interpretable feedback logic, the system demonstrates that reliable movement analysis and coaching can be achieved on consumer hardware without specialised sensors or opaque machine-learning models. The key contributions of this work include: (1) a structured pipeline that transforms raw pose landmarks into stable, exercise-specific kinematic representations; (2) a deterministic temporal reasoning framework that enables robust phase detection and repetition counting in real time; (3) an interpretable, rule-based scoring and feedback system that prioritises safety and transparency; and (4) an integrated interface that links personalised plan generation with live coaching feedback. Together, these components show how applied computer vision can be deployed responsibly in unsupervised training and rehabilitation contexts. Beyond the specific implementation, this work highlights broader design principles for camera-only human motion analysis systems, including the importance of treating pose estimation as a sensing layer, elevating temporal reasoning to a first-class concern, and prioritising explainability in safety-critical applications. While further validation and expansion are needed, particularly in clinical settings and mobile deployment, the results demonstrate the feasibility and potential impact of accessible, AI-driven exercise coaching. The AIPT system, which won the ETH Zurich KI Challenge in the category “AI for Good,” represents an early but meaningful step toward democratising safe movement guidance and reducing barriers to effective physical training and rehabilitation. # References {#references} \[1\] C. Lugaresi et al., “MediaPipe: A Framework for Building Perception Pipelines,” arXiv:1906.08172, 2019\. \[2\] V. Bazarevsky et al., “BlazePose: On-device Real-time Body Pose tracking,” arXiv:2006.10204, 2020\. \[3\] Z. Cao, G. Hidalgo, T. Simon, S.-E. Wei, and Y. Sheikh, “Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields,” in Proc. CVPR, 2017\. \[4\] T. Alatiah and C. Chen, “Recognizing Exercises and Counting Repetitions in Real Time,” arXiv:2005.03194, 2020\. \[5\] A. Sinclair, K. Kautai, and S. R. Shahamiri, “Pūioio: On-device Real-Time Smartphone-Based Automated Exercise Repetition Counting System,” arXiv:2308.02420, 2023\. \[6\] Peloton Investor Relations, “Peloton Introduces Peloton Guide, First Connected Strength Product,” 2021\. \[7\] Peloton, “Peloton IQ: AI-powered personalized workout planner.” \[8\] Tempo, “Award-Winning AI-powered Home Gym Membership.” \[9\] TechCrunch, “Home gym startup Tempo raises $220M…,” 2021\. \[10\] Tonal, “Introducing Form Feedback,” 2021\. [image1]: [image2]: [image3]: [image4]: [image5]: [image6]: