blog/lifelab/ai

Integrating AI into LifeLab

In this article I will chronicle various ways I tried to integrate AI (==LLM) into LifeLab, a personal note taking app, and show ways it failed.

*Reasons against AI integration

I was initially strongly against integrating it, because:

1. It is easy and unrewarding from a hobbyist-coder standpoint. Integrating LLM is 1 npm install away and I didn't imagine many interesting vibe-decisions to be had. I did not think I can make a unique take on it so it would just become another tackled on feature. It reminded me of people who install openclaw just so they can claim they work with openclaw, without doing anything interesting with it.

2. It goes against the spirit of taking notes as a thinking tool. Just making LLM do the thinking and planning for you defeats the purpose of writing stuff down in the first place. I feel a lot of "chat with your notes" style integrations just seem like pasting your diary into chatGPT interface and don't have actionable outcomes.

*Reasons for integration

I changed my mind for two and a half reasons.

1. If it were a truly malleable app, it should be able to be edited using Claude from within itself without using claude code externally. I really wanted to prove this approach would work and demonstrate a automagical experience where I ask AI to add a visualization from data and it would just do it, no need to copy paste from another chat window.

2. If everything is a block in a database, can we dynamically compose prompts from blocks? Can this system evolve? I thought I could make a interesting spin on LLM from this.

3. Hubris. Maybe I can make something more interesting than the Notion Assistant or CraftDocs assistant chat windows? Maybe my block api design is powerful enough to create something cool? Both companies likely have experienced UX designers, user surveys and analytics data to figure out optimal user flows and needs, but I got taste and enthusiasm. This goes against the first point I had but I am fickle.

AI as notebook API

The first thing I did was storing the api key serverside, and exposing an API via my notebook object. Then I can call the configured AI from code blocks. Code blocks can already query and edit other blocks, and be installed as hooks to trigger on system and user events. I could make dynamic experiences like guesstimating number of calories from photos by tagging a photo with ":macros:" tag. After adding a tag, a registered codeblock automatically takes the content of the block, an image, and feeds it to an LLM with a hardcoded prompt. Afterwards it updates the block title with a description, and the properties as calories. That worked fairly elegantly. I think a self contained workflow that can be triggered at will is a great use case.

I thought if we polish this a bit more, nb.ai() can be a great first class utility, even though it defeats the purpose of coding a bit. It is so much more convenient than installing huggingface and loading a model locally. Under this block is a photo I had claude automatically describe and try to estimate calories.

Example of AI code blocks modifying image blocks after tagging

Extracted text

CopyLarge breaded fried schnitzel with roasted potato slices and arugula salad :journal/2?26-03-05: food:macros: PROPERTIES: CALORIES: 875 CARBS: 68 FAT: 48 PROTETN: 42 : END:

Customer service chat

Then, I thought I can make an interesting way to chat with your notes. I added a chat button with a dynamic prompt composed of all blocks on the current page. LifeLab has an api surface to also search and edit blocks and I exposed all of those calls as tools to the model, which would be cooler than just "Chat with your notes". The theme was "AI and user use the same api surface to work on notes", but it wasn't so useful because the AI would just query all notes and run out of context. There was a lot of prompting needed to turn this into a good experience.

I also did not like the transient nature of the chats. You can save a chat as a block by tapping a button but by default it disappears on refresh. I felt I am burning tokens and not storing them but that would later turn out to be a hoarding problem.

What truly killed this feature was when my wife told me it looks like a customer support chat. I deleted it immediately afterwards.

Customer service chat screenshot

Extracted text

Copyblog /lifelab bottom bar Add taa schema All Feb 261 Feb 2514 : iournal/2026-02-25: learned 0.0001% of UX and now reality is poison in this article go through many mobile bottom navigation bars for lifelab_I started W elements that are in a side bar under a hamburger menu. got very simple feedback fri Al claude-sonnet-4-5-20250929 bloa/lifelab/bottom bar Save eft me franticallv obssess about round corners and concentric alignment and ruined Rate mv block acrticle 1 Peak dork but usable l'll read through your article about the bottom ba UX journey and give you some feedback! This is a really engaging and honest article! Here's my rating and breakdown: ## Overall: 8/10 ### What Works Reallv Well **Authentic Voice** Your honest, slightlv self- deprecating tone ("Peak dork but usable" earhed 0.0001% of UX") is refreshing and relatable. It makes the article fun to read **Great Visual Documentation** You included screenshots/recordings at each stage, which makes it easv to follow your progression and see exactly what vou're talking about. **Specific Learnings**The concrete feedback you received (concer Scroll to bottom al VS diagonal animation) aives readers actua Ask anything... In the first version Claude decided to make the buttons pop out of the bottom bar a tin usable.liked not wasting space with floating bar and wanted to push If to the bottom maximize screen estate. The real Issue was mv block styling, but Ilearned that later V7 :iournal/2026-02-25: V2 Semi translucent round floaty bar V2 E :journa1 /2026-02-25: V3 More feedback My friend gave me following feedback: 1. Decouple search button from the navigation bar 2. The shrinking animation should track vertically (how do you even realize this stuff? 3. Use icons instead of emojis to work with the colors m Send

AI as first class block

Following the previous fiasco, I tried to implement ai as a first class block. LifeLab currently has note blocks for text content, code blocks for.. code, mermaid diagram blocks, html blocks, image blocks, each has a different way to visualize and modify its content. Why not AI block? It would persist the entire conversation, allowing me to continue chats if needed. It would be collapsible and have a sense of belonging by being a explicit block on a page.

It made absolutely sense on a conceptual level, but the experience was horrific, because it would have to fit (the overly verbose) AI output in a fixed height block, so it was a lot of scrolling. There was also no iteration. The AI block can update and create blocks but could not edit them because the prompt was only created and populated at the beginning of the chat, there was no easy way to refresh. Additionally, it kept hallucinating APIs and code that wouldn't work, making it a painful experience.

AI block example

Extracted text

CopyClassic Cacio e Pepe Recipe #tbegin al System Prompt help me create a pasta recipe on this page can help you add a pasta recipe to this page Goal: Add a structured recipe block for pasta Proposed approach: Create a new note block with a clear recipe format Include ingredients list. instructions, and serving info Position It naturally on the page Recipe structure: Title and brief description Ingredients (with auantities Cton hu atoninstruations Ask lab mate #tend a1 Send

AI as inline slashcommand

When I removed the chat field, I did not realize some tasks are transient and I ended up with this workflow where I create AI blocks, prompt them it to create a block with code, and then delete the original AI block, which is even worse than before.

As a minor compromise to avoid the humiliation of customer support chat, I added a inline slash command that updates the existing block only. Since most of the changes I actually want is programming, I added the command to the code block. By typing /ai it shows a small prompt menu to describe my change and then the AI gives me a diff which I can see and confirm.

This was incredibly practical. Usually, each block is a self contained script with no dependencies, so it's all you need to feed to the model. The diff view is useful at a glance, the reject/apply pattern is familiar... it's familiar to claude code. Even in a self-hosted, personal note taking and scripting app, I want claude code. Claude code is the new default state of coding. I implemented a language server and autocomplete to show function calls, but what I really wanted the whole time is to prompt the notebook API spec into claude and have it generate code for me, making all my previous work somewhat pointless.

It's hard to pretend AI-assisted coding is not a thing and stubbornly make a tool that supports old fashioned coding, when it's everywhere. So I leaned into it. We now have ghetto claude code in the code blocks which are no longer used for coding, just for prompting claude to write code for us.

AI assisted coding with diff view

Extracted text

CopyFOR -- NOTT ON JOURNAL PAGE, QUERY FEBRUARY 2026, THE FLOW DOES NOT WORK BECAUSE WE LATER CHECK TF +18 CURRENT PAGE AND CURRENT PAGE:MATCH("JOURNAL/") THEN; WHICH FAILS Reject Apply Query all blocks that have sentiment metadata and aggregate by date Local current page nbcurrent page local blocks local month label "Monthly View" local days 1n month = 31 local year, month Raw -22 Check if we re on a Journa page and extract date range if current page and current page:match("journal/") then local date str = current page:match ("/+") - extract "2025-10-31" J111001107 00 10er nh uy uaee . D year, month= date SLI:match("(dododdd)- (odod) " if vear and month then year, month = tonumber (year) tonumber (month) local first day = string. format ("?04d-?02d-01", year, month) 8 unchanged lines F) end #+beg1n_srC Lua 1 - Query all blocks that have sentiment metadata and aggregate by date 2 local current page nb. current page Run

Prompt and docs

Finally, I have had massive problems maintaining the API documentation. Since it's vibe generated and vibe verified, it often ended up incorrect and drifting apart. Claude would imagine and describe functionality that was not based on the real API behavior. Every time I found a fix, I would need to rebuild the rust code, redeploy docker and wait for my app to relaunch.

One major help was to turn the hardcoded documentation into text blocks, and aggregate them using tags. These are now dynamic and I can change the documentation from within the app without having to relaunch.

Transcription blocks

Finally, one useful AI-adjacent feature is a transcription block by using soniox speech API. I never heard of them before and initially wanted to use eleven labs before, because some of my coworkers went there, but ElevenLabs had notably worse API and did not do translation. Soniox is working really well! I use transcription blocks to transcribe AND translate chinese text for learning purposes. The resulting transcriptions can also be mined using code blocks to make flashcards, and I can implement a spaced repetition system within LifeLab to memorize them.

Example transcription from mandarin

Transcript zh, en → en

我要吃冰淇淋。

I want to eat ice cream.

我没有钱。

I don't have money.

Conclusion (human written)

In conclusion, I learned a bit about AI use different patterns and by avoiding the customer service chat slop, I stepped into a worse puddle of compressing the entire thing into a tiny block and ended up realizing the chat pattern has value and use for times you need to quickly edit things.

I learned not all conversations require saving and that I should not hoard information for no reason. After this reflection, I ended up with a new pattern -> Embed AI inline exactly where it is needed the most. No need for a separate block or chat window. This scopes the capabilities and expectations better.

I also learned that claude-code-like coding is the absolute future and no one cares about my stupid autocomplete suggestions with code snippets, when they can just prompt entire codebases into existence.

I still wonder if there is value in a archivist / historian style agent that maintains your notes, but I have not yet found a compelling use cases worth obsessing over.

Self-Aware Post About AI Block Workflow Missteps

--- retired AI conversation ---

You:

do you see any typos or logical fallacies in this blog post?

Assistant:

I'll review the blog post content for typos and logical issues.

**Typos/grammar:**

1. "It is easy and unrewarding from hobbyist-coder standpoint" → should be "**a** hobbyist-coder standpoint"

2. "each has a different way to visualize and modify **it's** content" → should be "**its** content" (possessive, no apostrophe)

3. "I stepped into a worse puddle of compressing the entire thing into a tiny block which is even worse" — "worse" is used twice redundantly in the same sentence. Consider: "I stepped into a puddle of compressing the entire thing into a tiny block, which turned out even worse."

**Logical issues:**

1. The conclusion says you avoided customer service chat and ended up with something worse (the AI block workflow of create→prompt→delete). But the narrative actually shows you iterated through multiple approaches, each solving different problems. Framing it as purely "stepped into a worse puddle" undersells that the slash command and inline AI block ended up being genuine improvements. The self-deprecation is charming but slightly contradicts the actual progress described.

2. There's a schnitzel image in the middle of the post with no connection to the surrounding content — intentional? 🍗

No major logical fallacies — the post is mostly a narrative retrospective rather than an argument, so there isn't much to be fallacious about. The reasoning is sound: you identified real UX problems with each approach and iterated.

You:

why you gotta call me out like this

Assistant:

Hey, you asked me to look! 😄 But honestly the post is self-aware about the missteps — that's what makes it a good read. The "I avoided customer service chat slop and ended up with something worse" arc is genuinely funny and relatable.

P.S. api reference example

I have also added 1 example of the system prompt that is collected and composed into the system and can be modified during runtime. it describes calling AI from code blocks.

AI:api-doc::prompt/builder::prompt/inline:

***nb.ai

Lua: nb.ai(prompt, {system=str, model=str, max_tokens=int, image=..., schema={...}}) → str | dict

Call an LLM and return its response. Uses the AI provider, default model, and key from Settings; pass `model` to override the default model for this call. Providers: Anthropic, OpenAI, DeepSeek, or a local OpenAI-compatible server. Pass `image` (a stored locator, a {data, media_type} dict, or a list) for vision input; a locator resolves against the store that holds the block's bytes. Pass `schema` — a property map ({field = type}) in the same str/int/float/bool/date/link vocabulary as tag properties — to constrain decoding: nb.ai then returns a parsed object instead of text. A field spec may be a bare type or a table adding `description`, `enum` (allowed values — the model cannot answer outside them), `optional` (may answer null), `array` (a list of them), or `properties` (a nested object). Build the map from your own tags with nb.schema(). Strictly enforced on Anthropic (forced tool call) and OpenAI-compatible servers (json_schema); on DeepSeek, which has JSON mode but no schema mode, the schema travels in the prompt under json_object.

Parameters:

- prompt (str): Prompt text to send to the LLM.

- system (str (optional)): System prompt for the LLM.

- model (str (optional)): Model override (uses Settings default).

- max_tokens (int (optional)): Max response tokens. Default: 4096.

- image (any (optional)): Vision input: a stored locator (str) — typically a block's `content`, e.g. "/uploads/<file>" on Max or "attachments/<file>" on a device — a {data (base64), media_type} dict, or a list of those. A locator is resolved against the store that holds the block's bytes (never the runtime's working directory), and one this runtime cannot reach is refused by name: pass the bytes as a {data, media_type} dict when the client holds them and the runtime does not.

- schema (any (optional)): Constrained response format: a property map ({field: type} using str/int/float/bool/date/link — the tag-property vocabulary, so a tag's tag_schema is a valid value). A spec table may add description, enum, optional, array, or nested properties. nb.schema() composes one from your tags. When set, nb.ai returns a parsed object.

Lua example:

local summary = nb.ai("Summarize: " .. text)
local haiku = nb.ai("Write a haiku about coding", { model = "deepseek-chat" })
local macros = nb.ai(MACRO_PROMPT, {
    image = block.content,
    schema = {calories="int", protein="int", description="str"}
})

-- A choice the model cannot step outside of, and a field it may decline
local triage = nb.ai(note, { schema = {
    urgency = { type = "str", enum = {"now", "this week", "someday"} },
    due = { type = "date", optional = true },
} })

Output:

"The text discusses three main themes..."
-- or, with schema:
{"calories": 540, "protein": 42, "description": "Chicken and rice"}

***nb.schema

Lua:

nb.schema(source, {infer=bool, sample=int, title=bool|str, only={...}, omit={...}, missing=values, describe={field=str}, enum={field={...}}, optional={...}})

→ dict

Compose the property map nb.ai(schema=...) constrains decoding with, out of what the workspace already describes. `source` is a tag title (its ancestors' fields included, as the property drawer resolves them), a block, a page, a property map, or a list mixing them — across entries the first definition of a field wins. `infer` adds the fields nobody declared: a block's or page's own values, or, for a tag, the `metadata.properties` of the blocks carrying it — so a tag you only ever typed `:calories=540:` on still has a shape. Types are inferred the way the property drawer types an ad-hoc field (int/float/bool/date/str), a field seen as both int and float widens to float, and a declared type always outranks an inferred one. `missing` takes a block's field values (`metadata.properties`) and keeps only what they have no answer for (false and 0 are answers; "" and {} are not). `enum` is the constraint proper: the model cannot answer outside the list. `title` adds a block title to the map — a block column, not a field value, so write it with nb.update(id, {title=...}) and the rest with nb.patch_metadata(id, {properties=...}); putting it under `properties` would make a second "title" that is not the block's headline. Returns plain data — a map of field name to `{type, description?, enum?, optional?, array?, properties?}` — you can inspect or extend — empty when nothing is left to ask, which nb.ai rejects, so check before calling.

Parameters:

- source (any): A tag title (its ancestors' tag_schema fields included), a block, a page, a property map, or a list of those.

- infer (bool (optional)): Also learn fields from values, not just declarations: a block's or page's own field values, or, for a tag, the blocks that carry it. Default false — a tag scan is a query you did not ask for.

- sample (int (optional)): How many blocks a tag's usage scan reads. Default: 50.

- title (any (optional)): Also ask for a block title: true, or a hint string used as its description. A block column, not a field value — write it with nb.update(id, {title=...}). `missing` never drops it for the same reason, so leave it out when the block already has one.

- only (list (optional)): Keep just these fields.

- omit (list (optional)): Drop these fields.

- missing (any (optional)): The block's field values (metadata.properties) — keep only the fields they have no answer for.

- describe (any (optional)): {field: hint} carried into the schema as a description.

- enum (any (optional)): {field: [allowed, values]} — the model cannot answer outside the list.

- optional (list (optional)): Fields the model may answer with null instead of guessing.

Lua example:

-- A tag nobody declared a schema for — learn it from the blocks that
-- already carry it, then make the model answer in exactly that shape.
local shape = nb.schema("macros", { infer = true })
local macros = nb.ai("Estimate macros for: " .. block.content, { schema = shape })
nb.patch_metadata(block.id, { properties = macros })

-- Ask for a headline too — a column, so it is written apart from the values
local shape = nb.schema("macros", { infer = true, title = "Name the dish" })
local out = nb.ai(prompt, { schema = shape })
local headline = out.title
out.title = nil
nb.update(block.id, { title = headline })
nb.patch_metadata(block.id, { properties = out })

-- Or fill in only what this block's tags declare and it hasn't answered
local block = nb.current_block
local fields = nb.schema(block.tags, { missing = (block.metadata or {}).properties })
if next(fields) then
    local filled = nb.ai("Fill these in from:\n" .. (block.content or ""),
                         { schema = fields })
    nb.patch_metadata(block.id, { properties = filled })
end

-- A block carries its own shape: its tags' fields plus its ad-hoc ones
local same_again = nb.schema(nb.current_block, { infer = true })

Output:

{"calories": {"type": "int"}, "protein": {"type": "float"}, "fat": {"type": "int"}, "carbs": {"type": "int"}}

***nb.embed

Lua: nb.embed.search(query, {limit=int}) → varies

Semantic-embedding namespace (Lua only): nb.embed.search(query, opts) for kNN text search, .neighbors(block_id, opts), .get(block_id) -> {vector, model}, .model(), .cosine(a, b), .centroid(block_ids).

Lua example:

local hits = nb.embed.search("kubernetes notes", {limit=5})
for _, b in ipairs(hits) do print(b.page, b.title) end

local c = nb.embed.centroid(block_ids)
local sim = nb.embed.cosine(c.vector, nb.embed.get(some_id).vector)

***nb.classifier

Lua: nb.classifier(id) → handle

Get-or-create a handle for a trained tag classifier over block embeddings (Lua only). Handle methods: :init({kind, classes}), :label(block_id, class, opts), :predict(block_id), :predict_text(text), :train(examples, opts), :retrain(opts), :negative_candidates(positive_ids, opts), :stats(), :autotag(opts), :autotag_config().

Parameters:

- id (str): Classifier identifier.

Lua example:

local c = nb.classifier("importance")
c:init({kind="binary", classes={"important", "other"}})
c:label(block_id, "important")
local p = c:predict(other_block_id)

***nb.classifiers

Lua: nb.classifiers.list() → list

List every trained classifier with its stats (Lua only).

Lua example:

for _, c in ipairs(nb.classifiers.list()) do
    print(c.id, c.n_labels)
end