I have a problem: I go to a boba or coffee shop, I order something incredible, and I customize it perfectly (less ice, 50% sweetness, oat milk). Three months later, I'm standing at the same counter with absolutely no idea what I ordered last time.
So I built a system to fix that.
The Idea
The vision was simple: photograph the drink label that comes with my order, have my phone automatically parse it, and log it to a database I can query later.
The full pipeline looks like this:
- Apple Shortcuts captures a photo of the drink label
- Apple Intelligence (OCR + AI parsing) extracts the drink details from the label
- The parsed data gets submitted to a Google Form
- Google Form responses flow into a Google Sheet that serves as my database
- Claude reads the sheet and answers questions like "what did I order at Da Vien?"
Simple in theory. Deeply humbling in practice.
Building the Shortcut
I built the Shortcut step by step with Claude's help. The basic flow was:
- Take a photo
- Extract text from the image using Apple's built-in OCR
- Feed that text into an AI prompt that parses it into structured fields: cafe, drink name, size, ice level, sweetness, milk type, add-ons, notes
- Clean up each parsed value
- Encode the values into a Google Form submission URL
- Submit it silently in the background
What sounds like six steps was actually about 40 actions in Shortcuts. And every single one had a gotcha.
- The URL encoding problem. Google Forms doesn't accept spaces in URLs. My first fix was to replace spaces with %20... But then %20 showed up literally in my spreadsheet responses. The actual fix was to use Shortcuts' built-in URL Encode action on each individual field value before assembling the URL, not on the whole URL after the fact.
- The photo keeps getting sent to Google problem. Shortcuts has this maddening behavior where each action automatically grabs the output of the previous one. So my Get Contents of URL action kept latching onto the photo instead of my form URL. The fix was switching to Open URLs instead, which is less elegant but actually does what you tell it to.
- The "Set Variable" vs "Set Dictionary Value" problem. These two actions have almost identical names and I kept adding the wrong one. One stores a named value you can reference later. The other stores a key-value pair inside a Dictionary object. I needed the first one. I kept getting the second one.
Honestly, building in Shortcuts felt like giving instructions to someone who technically does exactly what you say but somehow always misunderstands you.
The Rating System
I also added a ❤️ rating to each log. The Shortcut shows a menu:
- ❤️❤️❤️❤️❤️
- ❤️❤️❤️❤️
- ❤️❤️❤️
- ❤️❤️
- ❤️
Each selection sets a variable to a number (5 through 1) which then gets submitted with the rest of the form data.
Connecting Claude to the Sheet
The last piece was connecting Claude to my Google Sheet so I could just ask it questions in plain language.
My first entry:
Da Vien — Strawberry Cream Oat Latte No ice, 50% sweetness, cream oat milk Notes: 50% sweetness was perfect! Rating: ❤️❤️❤️❤️❤️
Now I can ask Claude "what did I order at Da Vien?" and it reads the sheet and tells me exactly what I got and how I customized it.
What I'd Do Differently
I'd spend more time upfront testing the AI parsing prompt with real label photos before building the whole Shortcut around it. The OCR output from real drink labels is messy and the prompt needs to be robust enough to handle all of it.
Was It Worth It?
Absolutely. Not because it's the most efficient system in the world, but because now I'll never forget that Da Vien's Strawberry at 50% sweetness with cream oat milk is a perfect drink.
And honestly? Building it was fun. There's something satisfying about stitching together a bunch of tools that weren't designed to talk to each other and making them work anyway.