Jennifer Naomi Nguyen · personalization study plan · updated September 11, 2026

I built Attractor to help long-running AI conversations pick up where they left off. It keeps a weighted map of recurring ways of working—research, systems design, creative work—and updates that map as conversations develop. I can see what changed and turn the state into context for the next session.

The idea is to carry forward something useful about how a person approaches their work. Someone who often thinks through experimental controls, for example, may benefit from a response that builds on that approach. This study asks how much of that context helps, how to keep it concise, and when to leave it out.

The project overview includes a runnable example. The research notes explore how recurring conversation patterns develop.

How it works

Attractor summarizes a conversation, asks a model to propose changes to its map, applies the update, and saves the result. Each area of the map, called a basin, has a label, description, weight, keywords, and connections to other basins.

Local runs save the state in a JSON file. Hosted routes use Cloudflare Workers KV. The attractor context command prints a text block that a calling application can include in its next model request.

The current renderer includes active basin descriptions and connections, dormant basin names, and emerging patterns. Its length follows the state. For this study, I plan to add a renderer that selects context within a token budget, so I can compare shorter and longer versions of the same memory.

The mechanics explain the weights and update rules. The running guide covers storage, API setup, and application integration.

What I want to learn

I want context to save people from explaining the same background each session while leaving room for a conversation to go somewhere new. The study looks at three questions:

1. Usefulness: which responses help the person accomplish their task?
2. Size: how much context is needed to preserve that benefit?
3. Relevance: when does personal context fit the question, and when does it get in the way?

A later comparison will look at working styles alongside factual memory about projects and preferences.

The experiment

The planned evaluation gives each test prompt to the same response model under several memory conditions. Task instructions and generation settings stay fixed while the supplied context changes.

The next implementation step is the budgeted renderer and evaluation harness. The renderer will use a defined selection order and record the text and token count delivered for each response. Even one basin can have a long description, so the budget applies to the actual text.

Build memory from the past

Choose a cutoff date, build the memory from earlier conversations, and freeze it. Use later conversations for test prompts. Apply the same cutoff to factual memory so every condition has access to the same history.

Keep the summary and update models fixed while building the state. The existing compare command is useful for comparing the full summary-and-update pipeline; this experiment holds that pipeline steady to study the context it produces. See the comparison notes.

Compare usefulness and relevance

Show response pairs in randomized order with the condition labels hidden. Ask which response is more useful and allow ties. I can judge how well a response fits my own work; a separate reviewer can assess correctness and task completion where the prompt supports it.

Include unrelated prompts to check whether the model brings up personal history unnecessarily. The rubric will look for irrelevant use of personal context, rather than counting every word that happens to match a basin label.

A model judge can provide a second reading using the same rubric. Present pairs in both orders, inspect disagreements, and report prompt counts, ties, failed runs, and uncertainty with the preference rates.

Keep a record of each run

Record the frozen state version, delivered context, block hash, token count, condition, model identifiers, and date. This makes it possible to trace a response back to the exact context it received.

For hosted runs, record the state actually read from storage. Workers KV is eventually consistent, so evaluation will use frozen state and explicit version records. The running guide covers the storage details.

Start small, then expand

The first pass will compare three conditions: no memory, the full block, and one budgeted block. The proposed sample is 20–30 prompts where context could help, plus 10 unrelated prompts. This pilot will help refine the rubric and identify failures before expanding the comparison.

A later live experiment could vary whether context is supplied at session start. That would explore the feedback loop: a response changes the conversation, and the conversation changes the next memory update. The design will account for context carried over from earlier sessions.

Study scope and data

The first study focuses on my own conversations and preferences. References to familiar projects can make conditions recognizable even with their labels hidden. Recording the model family, history length, summary choices, and memory wording will help interpret the results.

Public examples will use synthetic or reviewed material. Private transcripts and run logs stay in the research workspace. The stored state is also personal data, so it gets the same care as the conversations it describes.

Further reading

Background for the study design:

• Salemi et al., LaMP: When Large Language Models Meet Personalization.
• Park et al., Generative Agents.
• Packer et al., MemGPT.
• Zheng et al., Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena.

Project overview · All projects