One of the things that bothers me about most AI assistants is that you have no idea what they actually know about you. Your conversations disappear. The context resets. And whatever the system has learned about your preferences, your schedule, your life — it lives in a black box somewhere on someone else's server.

When I set up my own AI assistant, I decided that wasn't going to work for me. I wanted to see what it knows. I wanted to be able to edit it, delete it, and take it with me. So I gave it a memory system built on Obsidian — and it changed everything.

What the Problem Actually Is

When you chat with a commercial AI assistant, context is ephemeral. Start a new session, and it's forgotten you exist. Some systems have "memory" features, but they're opaque — you don't really know what's been retained, in what form, or how it influences responses. You're trusting a company to manage your personal context responsibly.

That's a lot of trust.

My assistant (Servo — a sarcastic horse with an old soul, if you're wondering) doesn't store anything in a cloud service I don't control. Everything he knows about me lives in plain Markdown files in an Obsidian vault on my own machine.

How It Works

Obsidian is a note-taking app that stores everything as plain .md files. There's no proprietary database, no lock-in. Your notes are just files — readable by any text editor, syncable with any tool, available on every device.

Servo reads and writes to this vault directly. When I tell him to add a task, he writes it to the right project file. When he takes a note about a decision we made, it goes into my daily notes. When something is worth remembering long-term, it goes into a curated memory file — think of it like his long-term memory, written in plain language he can read next session.

The result: I can open Obsidian on my phone, my tablet, my laptop — and see exactly what Servo knows. His task lists. His notes. His memory. All of it.

The Transparency Advantage

This matters more than it sounds.

Last week, Servo made a mistake categorizing some tasks. I opened Obsidian, saw exactly what had happened, and fixed it in thirty seconds. No support ticket. No mystery. Just a text file.

When I want to see what's on my plate, I open my Dashboard note and it's all there — rendered, organized, searchable. Dataview queries pull tasks from across every project into a single view. I didn't build a custom app. I didn't sync to yet another service. It's all just Markdown.

And when I want to forget something — when I want to remove personal information from Servo's context — I just delete the file. Done. No data retention policy to worry about.

The Structure That Makes It Work

A few things I landed on after some trial and error:

Projects over inbox. Tasks go directly into the relevant project file — not a dumping ground. Servo knows to put consulting work in the consulting project, family stuff in the family project. The inbox is only for things that genuinely don't have a home yet.

Daily notes as a log. Every day gets a note: goals, events, decisions, things that happened. It's a running record that Servo can reference. It's also useful for me — I actually remember what I did last Tuesday.

Fleeting notes for captures. Quick ideas, reference info, things to process later — all go in a Fleeting folder. It's like an inbox for thoughts. Low friction in, reviewed periodically.

Memory files as curated context. The difference between raw notes and memory: notes are everything, memory is what matters. Servo maintains a long-term memory file with the distilled context he actually needs — my preferences, ongoing projects, important decisions. It's concise by design.

What Syncs Where

Because Obsidian uses plain files, sync is solved by whatever you already use. I use iCloud — every note is on my Mac, my iPhone, my iPad instantly. No extra service, no subscription, no API to configure.

I can be on my phone, open a project, see the tasks Servo added this morning, add a note of my own, and it's all there next time Servo opens that file. Seamless.

The Privacy Angle

I'll be direct: I chose this approach partly because I'm cautious about where my personal context lives.

Servo knows a lot about my life. My schedule. My finances. My family. My projects. I'm not comfortable with that context living in a commercial cloud service under someone else's terms of service.

Plain files on my own machine — synced via a service I trust, backed up where I control — feels different. It's not paranoia. It's just being intentional about where your personal data lives.

What's Next

I'm still refining the system. The structure is solid, but I'm finding new ways to use it every week. Next I'm thinking about how to use Dataview more aggressively to surface things that need attention — a kind of ambient awareness layer on top of the vault.

If you use Obsidian and you're thinking about pairing it with an AI assistant, I'd start with the memory and daily notes structure. Get that right and the rest follows naturally.