Master of Graphic Design

THESIS

As part of my Master’s degree at North Carolina State University, I conducted a research project focused on the integration of advanced technologies to enhance athletic training and rehabilitation. The project developed a system combining body sensors, mixed reality, and real-time visual feedback to improve exercise performance, reduce injury risk, and optimize recovery. The full documentation of this research was published as my thesis, which is available for access through the NCSU Libraries repository. You can view the complete thesis at the following link: Thesis Document.

 

Problem Space

Each year, approximately 30 million young athletes in the United States participate in competitive sports. Unfortunately, nearly 90% of these athletes will experience a sports-related injury at some point. The recovery process for these athletes is highly individualized, influenced by various factors such as their unique anatomical characteristics, the nature of the injury, and personal recovery needs. This creates a significant opportunity to enhance the rehabilitation process through the integration of emerging technologies.

In my final thesis project, I investigate how mixed reality (MR) experiences can support injured athletes in their recovery journey. By providing real-time feedback on physical components and delivering personalized, data-driven insights, MR technology has the potential to offer a more tailored and effective approach to rehabilitation. This project explores how the design of such immersive experiences can optimize recovery outcomes and assist athletes in their return to peak performance.

main research question

Scenario

Jackie Nelson, a high school junior, is in the process of researching and applying to various college programs as she looks ahead to continuing her education. A key factor in her decision-making process is the opportunity to play her sport at the collegiate level. Jackie is committed to her sport and aspires to develop into a competitive athlete. However, during pre-season training, Jackie sustained an injury due to a combination of untreated muscle weakness and the physical impact of a game. As a result, she dislocated her knee cap.

Currently, Jackie is undergoing rehabilitation with her athletic trainer, focusing on strengthening key muscle groups to aid in her recovery. Due to staffing limitations in the athletic training department at her school, Jackie has been given a set of exercises to complete independently. While following her rehabilitation plan, Jackie seeks a deeper understanding of her injury and its impact on her body.

Her rehabilitation is structured in three distinct phases that reflect three visual studies:

  • Phase 1 focuses on the psychological aspect of Jackie’s recovery, particularly her initial lack of understanding about the nature of her injury. This phase is designed to help Jackie learn about her condition at her own pace, without the pressure of athletic trainers or the confusion that often arises from medical terminology. By empowering Jackie to explore her injury in a way that feels manageable and less overwhelming, this phase aims to alleviate emotions such as frustration and anxiety, while providing her with a clearer understanding of the recovery process.

  • Phase 2 centers on the physical rehabilitation of her knee injury. During this phase, Jackie will receive support as she completes her prescribed exercises, ensuring proper form and gradual strength building.

  • Phase 3 will assist Jackie in her return to play, emphasizing education about injury prevention, progress tracking, and visualizations of her rehabilitation milestones.

STUDY 01

How can an interactive system, using augmented reality and wearable technology, help an injured athlete understand the nature and extent of their injury at their own pace, while providing personalized insights to alleviate confusion and reduce the psychological stress often associated with recovery?

One key conclusion from Study One was the concept of making the invisible visible to the user. Through the use of mixed reality glasses, the system enables the athlete to visually observe the impact of rehabilitation exercises on their body, including the specific areas being targeted. This enhanced visibility allows the athlete to track muscle contractions and better understand the physiological effects of their exercises. For example, they can see a visual representation of what is happening to their knee during recovery.

In addition to providing real-time feedback, the system empowers the athlete by offering them greater independence and control over their recovery process. By actively engaging with the system, athletes can make informed decisions about their rehabilitation, track progress, and adjust their routines based on personalized insights. This autonomy fosters a sense of ownership in the recovery process, which can be crucial for maintaining motivation and reducing psychological stress.

The system also has the capability to detect elements within the surrounding environment, providing real-time simulations and feedback to the user. A consistent visual treatment across all studies is the use of color to differentiate between the real-world environment and the system-generated overlays. This color-coding ensures clarity, allowing the user to easily distinguish between actual physical space and virtual data.

I also explored how users can interact with and manipulate the visual overlays presented within the system. Through gesture recognition, users can adjust their viewpoint and engage with various elements in their environment, allowing them to control the aspects of their recovery they wish to focus on. This feature provides athletes with an added sense of responsibility and autonomy in managing their rehabilitation process.

Incorporating additional features such as a voice assistant, body visualizations, and data tracking, the system is capable of collecting and compiling information tailored to each individual user. These personalized insights help guide the user through their recovery, enhancing the overall effectiveness and relevance of the rehabilitation experience.

Study 02

How can augmented reality overlays provide additional feedback about an athlete's injury through the rehabilitation process by correcting and analyzing body movements in real-time?

Ensuring proper form during exercises is essential for targeting the correct muscles and performing movements safely and effectively. The form error detection system provides visual overlays on the user’s body, enabling them to correct their posture in real time. In this example, Jackie’s squat form is identified as incorrect, and the system offers immediate corrective feedback. As Jackie progresses through her exercises, the system continuously tracks her movements, providing real-time adjustments, guidance, and overall feedback.

Additionally, the body mapping feature offers personalized insights tailored to Jackie’s specific movements, especially when performing exercises independently. Maintaining proper form is a critical component of the rehabilitation process, and this system allows users to visually track the boundaries they must stay within to ensure correct posture. As the user completes their exercises, the system dynamically adjusts these boundaries, providing ongoing support to help maintain optimal form throughout the session

An added feature of this system is the ability for users to learn how specific exercises impact their body. For example, Jackie is curious about the benefits of leg raises for her recovery. Through the system, she can access educational modules that show which muscles are targeted during this exercise and explain how it contributes to her overall recovery process

.

Study 03

How can a personal data system augmented with educational modules help a recovering athlete safely return to play and learn more about their injury within their sport by utilizing customization and personalized exercises?

As Jackie progresses through the game-like drills, it's important to acknowledge that she may not yet be performing at her full capacity. To mitigate risks and optimize her training, mixed reality modules are provided. For instance, Jackie can practice offensive drills with a simulated defender. These simulations not only replicate real-game scenarios but also offer performance-enhancing guidelines, such as suggestions for effective dodges and movement techniques, to support her recovery and improve her performance.

In this study, I examined how various components of the system work together to enhance the user experience. For instance, body sensors attached directly to the athlete provide accurate measurements and data as they progress through different exercises. I also explored the concept of multiple modes within the system. For example, Jackie can view real-time data on the screen interface and see performance metrics within a mixed-reality environment, offering a more immersive and interactive experience. Additionally, a key feature of the system is the mirroring of Jackie’s body within the interface. This allows her to see form corrections and performance adjustments in real time, with movements and corrections represented visually. By using visual representations that closely mimic the athlete’s own body, the system fosters better spatial awareness and greater familiarity with the exercise environment.