The Frustration of Describing UIs to Your AI Coding Buddy

Let's be real: CLI-based coding assistants like Claude Code, Aider, and Codex CLI are absolute powerhouses when it comes to writing and editing code. But here's the thing—they're essentially blind. They can't look at your screen and see what you see.

So when something breaks in your interface, you're stuck typing descriptions like "that blue button in the middle of the settings page that's showing the wrong text." Your AI agent does its best to picture what you're describing, but it's basically guessing.

This is the problem SlimSnap tackles head-on. It's a lightweight macOS app that turns any screenshot into clean, structured JSON that your CLI agent can actually work with.

Three Steps: Grab, Mark, Go

The whole process takes seconds:

1. Screenshot - Hit ⌘⇧S, then drag across whatever part of your screen you need. It taps into native macOS APIs, so nothing extra to install.

2. Markup - Draw arrows, add callouts, highlight problem areas. Point directly at what needs attention.

3. Export JSON - One click transforms your annotated screenshot into structured data you can paste straight into your terminal.

The resulting JSON is thoughtfully designed. It captures the timestamp, which app you were in, screen dimensions, and—most useful—an organized list of detected UI elements. Each element tells you its type (button, input, label, etc.), the actual text it contains, and where it sits on screen using normalized coordinates between 0 and 1.

Why does this matter? When your agent knows "the checkbox in the bottom-left of the permissions dialog," it can make surgical changes rather than fumbling around in the dark.

The Real Win: Massive Token Savings

Here's where SlimSnap really proves its value, especially during long debugging sessions: token efficiency.

When you paste a screenshot into ChatGPT's web interface, it gets compressed. But with CLI agents that accept images, you're still burning through your token budget. A standard screenshot might cost you around 1,568 tokens on Sonnet models, or nearly 4,784 tokens on Opus 4.7/4.8.

SlimSnap's JSON output? Usually 600-800 tokens. That's roughly half the cost on Sonnet, and 85% cheaper on Opus models. Multiply that across dozens of back-and-forth exchanges during a complex refactoring session, and the numbers add up fast.

More tokens staying in your context window means more space for actual code, error output, and your agent's reasoning chain. Your budget goes further.

Keeps Everything on Your Machine

Privacy-conscious developers, take note: all OCR processing happens locally on your Mac. Screenshots never touch external servers. No account creation, no cloud dependency. This is especially valuable when you're working with confidential applications or proprietary codebases.

The JSON specification itself is fully open under the MIT license and available on GitHub. Feel free to validate against it, build your own exporters, or even generate valid JSON from other OCR tools to integrate into your workflow.

For Claude Code Fans

SlimSnap includes a dedicated skill for Claude Code users. It watches for new captures, writes a small config file to ~/.slimsnap/config.json, and automatically loads the latest JSON into your agent's context. No file-hopping, no manual paste commands—just smooth handoffs between your screenshot and your coding session.

When to Use JSON vs. Pixels

Let's be balanced: JSON export isn't always the answer.

For element-level tasks—fixing that misaligned button, correcting a form label, adjusting spacing—structured data wins. Your agent gets exact positions and extracted text instead of squinting at pixels.

But for holistic checks—does this page feel cohesive? Are the colors working together?—you might still want to paste the raw image. Nothing stops you from sending both: JSON for precise manipulation, image for visual assessment.

Wrapping Up

SlimSnap addresses a genuine bottleneck in the AI-assisted development workflow. If you're heavily invested in CLI-based agents and regularly need to communicate about user interfaces, this tool deserves a spot in your stack. The token savings alone make it worthwhile, and local processing means privacy concerns disappear.

Currently macOS only, but since the JSON schema is open source, it's only a matter of time before someone builds compatible bridges for other platforms.

Give it a look at slimsnap.ai—particularly if your work involves heavy UI debugging or iterative interface work.

Typing "the red error message in the top-right corner" gets old fast. Sometimes the best workflow is just showing your agent what you see.