How I Turned My Fitness Data Into a Visual Story Using AI Agents

I'll cover:

• The problem: fitness trackers give data but miss context
• The solution: combining structured data with AI vision
• How the pipeline works (my own interpretation)
• The coding agent as a productivity multiplier
• Lessons learned and future possibilities

Let's be honest: fitness apps are great at tracking numbers, but terrible at telling stories. Your running app knows you ran 42 kilometers last month. What it doesn't know is that you were training for your first half-marathon, or that those sessions were part of a carefully structured periodization plan you photographed from your gym's whiteboard.

This disconnect between data and context became the catalyst for a small side project that perfectly illustrates why AI coding agents are becoming indispensable for personal automation workflows.

The Problem With Metrics Alone

I use Samsung Fit 3 to log gym sessions and Strava for my runs. The setup works beautifully—my fitness tracker syncs with Strava automatically, giving me all the standard activity metadata: duration, distance, elevation, heart rate zones, timestamps.

But here's what Strava will never tell you: whether that "weight training" session was a strength day focused on compound lifts or a mobility-focused recovery workout. It knows I did something for 45 minutes. It has no idea I was working through a goblet squat progression with 24 kilograms while my trainer scribbled the session structure on a whiteboard I never thought to photograph until recently.

The context lived exclusively in photos I'd been casually attaching to my Strava posts. The gym's whiteboard documented the actual workout structure—movements, rep ranges, focus areas, and the crucial distinction between recovery days and intense training blocks.

Strava captured the quantity. The photos told the quality.

Building a Personal Data Pipeline

The solution wasn't a single tool—it was a pipeline that let multiple systems play to their strengths. Here's the architecture I landed on:

Fetch → Process → Extract → Visualize

First, a simple Python script pulled all my Strava activities along with their associated photos. The script maintained a local cache, which meant it could skip already-downloaded media and resume from where it left off if interrupted.

Next came the interesting part: extracting workout context from those gym whiteboard photos.

I used Claude Code as a coding agent with vision capabilities. For each gym board photograph, I'd prompt the agent to generate a structured JSON description of the workout:

{
  "type": "workout",
  "exercises": [
    {
      "name": "Romanian deadlift",
      "sets": 4,
      "reps": "8-10",
      "weight": "60kg",
      "muscle_groups": ["posterior chain", "glutes"]
    }
  ],
  "focus": "strength",
  "notes": "Felt strong today, increased weight from last week"
}

The agent-in-the-loop approach proved remarkably effective. If the extracted data looked wrong, I'd simply delete that JSON file and regenerate it. If an image was already processed, the script skipped it. The workflow was resumable, fixable, and—crucially—iterative.

Once I had structured workout data combined with Strava's metrics, rendering the final infographic became straightforward. A simple HTML template pulled everything together into a shareable PNG poster.

Why Classification Was Simpler Than Expected

My Strava activity tags weren't exactly consistent. Some strength sessions were labeled "weight training," others just said "workout." Mobility days sometimes appeared as generic activities. The chaos was entirely my own doing—three years of casual tagging with no standardized system.

Rather than building a sophisticated classification model (which would have been overkill for a personal project), I used a pragmatic approach: combine Strava's activity type, title keywords, and the extracted photo data to make a best-guess classification.

The logic looked something like this:

• Keywords like "squat," "deadlift," "bench" + strength-focused photo data = strength workout
• "Mobility," "stretch," "recovery" + flexibility-oriented photo data = mobility session
• Distance + duration + no gym photo = running (Strava's metadata is reliable here)

This hybrid classification was good enough for a personal recap. The final poster felt accurate because it leveraged what each data source did well.

The MCP Consideration

I briefly experimented with Strava's Model Context Protocol (MCP) server, which promised to simplify the API integration. It worked well for fetching activity metadata, but it couldn't retrieve photos attached to activities. For projects requiring only numbers, MCP would streamline development significantly. For this use case, the manual script remained the better choice since photo access was non-negotiable.

What This Reveals About AI Coding Agents

The real value in this project wasn't the infographic itself—it was demonstrating how AI agents can serve as flexible middleware between data sources and desired outputs.

Traditional automation tools follow rigid logic: if this, then that. Coding agents can reason across modalities. They can look at an image, understand its context, and generate structured data. They can adapt when edge cases emerge. They can help you build the automation itself rather than just execute within it.

For developers and technical entrepreneurs, this represents a fundamental shift. The question is no longer "Can I automate this?" but rather "How quickly can I iterate on the automation?" A project that might have taken a dedicated weekend can now be prototyped in an afternoon, with the agent handling boilerplate and helping you think through edge cases.

The Takeaway

The output was a shareable training recap that actually told the story of my fitness journey. But the process pointed toward something bigger: the era of personal data empowerment.

Your fitness data, your code projects, your business metrics—most of it sits in fragmented systems that don't talk to each other. AI coding agents are becoming the connective tissue that lets you build custom views of your own information without enterprise-scale resources.

I may explore running a local model for this workflow next time, keeping everything on-device for privacy. If you experiment with similar personal automation projects or have fitness-agent workflows of your own, I'd genuinely love to hear about them.

Sometimes the most satisfying projects aren't the ones that scale—they're the ones that finally make your own data tell your own story.