IntroductionMachine Learning is fun with sensors and sports. Today's data scientist is out there, on the ski slopes, or surfing the waves, and best way to apply machine learning is real life scenarios of sports. What can we do if we had the best, the ultimate model of our body and health monitoring us constantly? So, when we wanted to start a new sport, for example skiing or surfing, our personal body assistant could give us suggestions, like a personal coach. With machine learning and AI methods, imagine having a coach next to you 24/7.
Part I: Sensors
Chapter 1: Getting StartedWhy are sensors important for health and fitness? For coaches, athletes and health professionals, they provide and objective picture of your activity. It's often impossible to capture micro-movements and forces of a downhill racer, moving at 100 mph down a winding ski trail, but when equipped with sensors, every aspect of that movement can be captured, analyzed and studied. In this book we'll use various IoT devices that can be used for sports data collection: inertial measurement units (IMUs), attitude and heading reference systems (AHRS), inertial navigation systems (INS/GPS), pressure sensors and others.1. Types of sensors and what they measurea. IMUs, AHRSb. INS/GPSc. Pressure sensorsd. Heart ratee. Vision and camera2. Sport science and dataa. Why is data frequency so important? A typical GPS device in your mobile phone works at 1Hz, that is one reading per second. Why isn't this enough for most sports applications?b. Machine Learning really cares about data frequencies, as a rule of thumb we will use 100 Hz for most sensor data we collect3. How can Machine Learning help?a. Problems solved by machine learning for human movement, health and fitness applications4. Visualizing sports from sensor dataProject: First look at athlete movement analysis with a sample sensor data set
Chapter 2: Sensor HardwareIt turns out they don't sell sensors with built in machine learning at convenience stores just yet! So, we made some. We go over some sport specific requirements for sensors, where and how sensors are placed on the body and equipment. In this chapter we will cover choices for sensor hardware, communication from sensors for data collection and data choices for IoT devices. 1) Sensor IoT devices: IMU, AHRS, INS/GPS, Pressure, Proximity2) Sensor communication3) Data choices for IoT devicesProject: Learning to work with a sample SensorKit dataset
Chapter 3: Sensor SoftwareOur sensor is operating at a relatively high frequency of 100 samples per second (100 Hz). We need a special software to connect our sensor to the app. In this chapter we include a practical project on how to connect our sensor via a protocol like Bluetooth Low Energy to a mobile device and transfer data to the cloud.1) Sensor firmware2) Algorithms for sensor data processing3) Connecting with the app and the SDKProject: Writing the code to connect from sensor to the cloud
Chapter 4: 3D Printing SensorsProject: 3D printing is a fantastic technology for custom applications like sports! In this chapter I included a fun project on designing the case for our sensor, using 3D design software like Fusion 360 and 3D printing our sensor.1) Designing sensor casing model for sports
2) Printing the sensor3) Every sport is different!Project: Designing a case and 3D printing our sensor
Part II: Sensor DataSensors generate an enormous amount of data! In this part we learn about different types of sensor data, how to parse it, store it, transfer between IoT devices and the cloud.
Chapter 5: Collecting sensor dataThis is where we sports scientists have most fun: data science on the ski slopes and surfing the waves! In this chapter I included a project.1) Sports and sensor placement2) Designing sports experiments3) Software and mobile devices for sports4) Sensor data for MLProject: Collecting dribble data from a basketball sensor
Chapter 6: Storing and parsing dataStoring sensor data is an interesting subject: at 100 Hz we have a lot of data from sports!1) Data frequency and aggregation decisions2) What to calculate on the sensors3) Sending data to the cloudProject: Writing code to parse and store sensor data
Chapter 7: Managing and streaming IoT data in the cloudAn overview of modern IoT data technologies for the cloud, this chapter is about managing and streaming IoT data in the cloud.1) Non-relational databases for sensor data 2) Streaming IoT data: (Spark, Kafka, Azure Stream Analytics)3) Data pipelines for IoTProject: Storing and streaming IoT data in the cloud
Part III: Machine Learning for Health, Fitness and SportsFrom sensor data to physics of sports, movement analysis and machine learning models.
Chapter 8: Physics of sportsSports scientists believe that each sport can be described mathematically with physics, let's dive into sport science! In this chapter we'll have a physics project to help us better understand the models.1) Physics of movement2) Sensors and physicsProject: Calculating forces for an athlete, using physics
Chapter 9: Machine Learning modelsMachine Learning models for sports. This chapter defines reasoning behind various algorithms for machine learning in sports, as applied to sensor data.1) Raw sensor data2) Clean and transform the data3) Engineering features4) Supervised Learning5) Unsupervised Learning6) Reinforcement LearningProject: Creating a machine learning model from our experiments
Chapter 10: Applying Machine Learning for various activitiesIn this chapter we look at some applications of sensors for sports, fitness and health. 1) Skiing and snowboarding2) Basketball3) Tennis4) Diving5) Javelin6) Surfing
Part III: Visualizing Sensors Using computer vision and visualizing sports data in 3D and VR.
Chapter 11: Computer visionComputer vision is an important way of tracking athletes in real time.1) Computer vision for sports overview
2) 3D body rendering3) Problems with computer vision vs sensors (occlusion)4) Winning scenario: combining sensors with computer vision5) Project: using computer vision for athletic performanceProject: using computer vision for athletic performance
Chapter 12: Visualizing athlete in 3D, Holograms and VRIn this chapter we'll touch the holy grail of sports science: visualizing athlete in full 3D, as a holographic avatar.1) Methods and requirements for 3D visualization2) Using Unity to visualize data Chapter 13: Vision and SensorsThis chapter is about combining vision and sensors. Imagine, if we had to bring visual and sensor data together, then we have a tool that can provide both a near-real time visual feedback and video analysis.1) Combining sensor and video dataProject: Combining sensor and video data for analysis
Part V: What the Coach needsFrom individual athletes to the team: this chapter would make the coach happy! Often, tracking an individual athlete with sensors is not enough: coaches or health professionals deal with teams they need to analyze.
Chapter 14: Coach and team view on the dataWorking with coaches on US Olympic Team, WTA, WNBA, professional ski and snowboard instructors, I learned a lot about requirements that coaches have on the sensors, data, analytics and presentation of the data.1) Coaches and teams view2) Looking across the entire team3) Coach dashboard (PowerBI)Project: Creating a coach dashboard with PowerBI
Chapter 15: Connected sensors and sports teamsFrom individual athletes and sports, to connected experiences.1) Sensor data from the team prospective2) Connected team
Conclusion: What's nextThis book provides a toolkit, a foundation for a sports scientist or a data professional to use sensors and machine learning for insights about athlete performance and injury prevention.
PROJECTS1) First look at athlete movement analysis with a sample sensor data set
2) Learning to work with a sample sport dataset 3) Writing the code to connect from sensor to the cloud4) Writing code to parse and store sensor data5) Storing and streaming IoT data in the cloud6) Designing a case and 3D printing our sensor7) Collecting dribble data from a basketball sensor8) Calculating forces for an athlete, using physics9) Creating a machine learning model from our experiments 10) Using computer vision for athletic performance11) Combining sensor and video data for analysis12) Creating a coach dashboard with PowerBI for the team
