Attractor — Knowledge Graph - Architecture
README · ARCHITECTURE · TECHNICAL · source
The pipeline
Two kinds of model call, and they do different jobs. Extraction is per conversation and independent. Consolidation is global and runs once. That split is the architecture.
Why consolidation exists
Each conversation is analysed in isolation, which is what keeps extraction cheap and parallelisable. The cost of that isolation is that the same idea comes back phrased differently every time: local JSON state, local JSON state persistence, local JSON storage, local JSON state management — four nodes for one idea.
Because those four share neighbours, the force-directed layout correctly stacks them on top of each other, and the graph becomes a pile of overlapping labels. No amount of layout tuning fixes it. The duplicates have to be merged before the graph is built.
The consolidation prompt is deliberately conservative: merge only genuine synonyms, and when in doubt leave a name alone. "local JSON state" and "local JSON state persistence" are the same idea; "model routing" and "model consistency" are not. An over-eager merge destroys the distinction the graph exists to show, and unlike a missed merge it is invisible in the output.
If consolidation fails it is caught and the run continues with unmerged concepts. A messy graph is a worse graph; no graph is a worse afternoon.
The noise filter
is_noisy drops conversations that are mostly terminal output or error logs
before they reach the API, so you are not paying to extract concepts from a stack
trace. Two thresholds:
- more than 55% of lines matching noise patterns → skip
- fewer than 500 characters of real content → skip
Both are tuned against my own corpus and are the first things to change on someone else's. The filter runs before the API call rather than after, because the saving is the point.
Layout stability
The layout is computed once, on the complete graph, with a fixed seed, and then held. The animation reveals nodes into positions that were decided in advance.
The alternative — recomputing layout per frame as nodes arrive — produces a graph that writhes. Every node moves every frame, so nothing can be tracked visually and the animation shows the layout algorithm rather than the history. Fixing the layout means a node's position carries the same meaning in every frame, which is the only reason the GIF is readable at all.
The seed is 42 so the same graph structure renders reproducibly. That removes
random layout drift within one graph. When extraction changes the nodes or edges,
the structure changes too, so positions can still move; the saved concept counts
are the cleaner comparison between the Haiku and Opus outputs.
Output paths
| Renderer | Library | Notes |
|---|---|---|
render_static | matplotlib | One PNG of the full graph |
render_gif | matplotlib + Pillow | Frames drawn in chronological order, assembled into a GIF |
render_html | hand-written JS in a template | Standalone file, no CDN, no build step — a small force simulation runs in the page |
The HTML output embeds its own simulation rather than pulling in a graph library, so the file works offline and forever. It is one file you can email.
The palette is shared with the Attractor's viewer so the two projects' graphs read as the same family of object.
Snapshot
This walkthrough follows revision e87a11c. The retained outputs come from 55
conversations, and the concept counts were checked against the saved JSON. They
are a recorded run, not a reproducibility study.