Most of my hobby projects die in a TODO.md. This one didn’t — because the users were my family, and they’d tell me at dinner when it broke.
Over four weekends I built FamilyLine: a bilingual AI agent that lives inside my family’s LINE group chat. It writes a warm morning briefing, tracks our expenses and reminders, plans our trips, helps my two daughters study, and — my favourite part — teaches them something new every morning and then talks with them about it. This is the story of how it came together, what’s under the hood, and the one lesson that kept surprising me: building for people you love is the most honest product feedback loop there is.
Why a family bot, and why LINE
Our family runs on a group chat. Schedules, receipts, “who’s picking up the kids”, homework panic, holiday planning — it all happens there. So instead of building yet another app nobody would open, I put the agent where the conversation already lives. On LINE, it’s just another member of the chat: you @-mention it, or reply to one of its messages, and it responds.
The stack is deliberately boring where it can be and interesting where it matters: Python 3.12 (fully async), Postgres for state, Cloud Run for the service, Terraform for the infra, the LINE Messaging API for the chat surface, and a set of open-source models I run myself — a small local model (Mistral-class) for the everyday digest, a Hermes model for the planning brain, and a larger local model for the parts that need more spark. Everything is bilingual by default: English and 简体中文, because that’s how my family actually talks.
Weekend 1 — Foundations: a bot with a warm voice
The first weekend was about earning a place in the chat. Not features — tone. A family assistant that sounds like a robot gets muted on day one.
So the first thing I built was the digest: every few hours the bot quietly reads the group’s activity and, twice a day, writes a short, cheerful briefing — today’s events, open to-dos, the weather, a nudge about a reminder. Around that I wired the practical stuff: add a calendar event in plain language (“dinner with grandma Friday 7pm”), log an expense (or just photograph a receipt and reply receipt), set a reminder (“remind me in 2h to take the soup out”). Foreign-currency expenses auto-convert to SGD. Every reply is localised, never hardcoded English.
The engineering discipline that saved me later started here: strict typing, a real test suite, database migrations, and a CI pipeline that refused to deploy broken code. A hobby project, run like production — because the “production incident” was my wife not getting her morning briefing.
Weekend 2 — Memory and governance: from chatbot to agent
A chatbot answers. An agent notices things and acts. Weekend two was the jump.
I gave the bot a memory: it extracts durable facts and commitments from the chat (“I’ll book the dentist”, “the car’s due for servicing”) and can surface them later. Then a small planner started proposing actions — “want me to set a reminder for the dentist?” — instead of waiting to be asked.
Here’s the part I’m proudest of, and the pattern I’d reuse in any agent: it never acts on its own. Every side-effect the agent wants to take flows through a permission gate — propose → the family taps Approve → then it acts. The model can suggest anything; it can do only what a human okayed. That single seam is what makes an autonomous-ish agent safe to hand to your kids. Suggestions are cheap and reversible; actions are gated and logged.
Weekend 3 — The travel agent: find, surface, hand off — never transact
We’re a Singapore family; we travel on school holidays. So weekend three was a travel assistant, built in slices: create a trip and it counts down to it; add itinerary items; it proposes a pre-trip checklist (passports, insurance, check-in) as the date nears; it finds real flight and hotel deals; it checks live flight status; and after the trip it tallies the spend and works out who owes whom.
The deal-finder taught me a design principle I now apply everywhere: find → surface → hand off, never transact. The bot searches, shows the best few options with prices, and gives you a “book here” link — and then gets out of the way. It never holds a card, never books on your behalf. An agent that surfaces is useful and safe; an agent that transacts is a liability. Knowing where to stop is a feature.
The nicest touch came last: you don’t need the commands. Someone can just ask, in the group, “hot spring hotel in Taipei? price” — and the bot recognises it as a travel question, searches, and answers with options, then remembers the thread so “flights too?” continues the same conversation. Natural language in; a real answer out.
Weekend 4 — Delight, and a bot that teaches
By the fourth weekend the bot was useful. The last weekend was about making it loved — and this is where my daughters took over as product managers.
They told me the morning jokes were repeating (they were — the model kept landing on the same puns). So I moved joke generation to a stronger local model with a “don’t repeat these” memory and a retry loop, added a daily word of the day, and then the feature I like best: a “Did you know?” that draws a fact from their actual Secondary 1 syllabus — Science, Geography, History, Maths, English, 华文, and Visual Arts — rotating the subject each day, bilingual, with a playful question at the end.
And then the magic: when a kid replies to the fact, the bot continues as a tutor. Reply “oxygen!” and it says “Yes — and did you know most of Earth’s oxygen comes from ocean algae? What do plants make first?” Ask “why are leaves green?” in Chinese and it answers in Chinese, about chlorophyll, and asks what happens in autumn. It matches their language, stays on the topic but follows their curiosity, and keeps going as long as they’re engaged. A morning fact became a conversation. That’s the whole point.
What four weekends taught me about agentic AI
- Put the agent where the conversation already is. The best interface is often no new interface.
- Propose, don’t act. A permission gate between “the model wants to” and “it did” is the difference between a helpful agent and a scary one.
- Know where to stop. Find and surface; let the human transact. Restraint is a feature, not a limitation.
- Tone is a technical requirement. If it doesn’t sound human, it gets muted — and then none of the clever engineering matters.
- Real users are ruthless and right. “The jokes repeat” and “it’s not friendly” were the best bug reports I’ve ever received.
None of this is specific to a family bot. Grounding decisions in real data, keeping a human in the loop, and knowing where an AI should stop are the same instincts that make any serious AI or data product trustworthy — I just got to learn them somewhere the stakes were dinner, not revenue.
FamilyLine isn’t a startup. It’s a bot that tells my family good morning, helps plan our holidays, and teaches my kids something new over breakfast — and occasionally chats with them about photosynthesis. Four weekends, one family agent, and a reminder of why I got into building products and digital platforms in the first place.
This is the first in a short series. Next: a weekend-by-weekend deep dive — starting with how to give a bot a warm, bilingual voice that a family won’t mute.
Building an AI Agent for My Family — a seven-part build log
- I Built an AI Agent for My Family — you are here
- A Voice Nobody Wants to Mute
- Propose, Don’t Act
- Find, Surface, Hand Off
- A Bot That Teaches
- The Homelab Behind the Family AI
- Homework Stays Private
Related: Agentic AI Is Not Just LLMs and Tokens.
A personal DIY hobby, tinkered together on weekends for my own family — just for the fun and the learning. Built entirely with open-source, self-hosted models.
Leave a Reply