Jennifer Nguyen

Bootwitch

Scientist building AI and research tools
10+ years in immunology research · Building with AI since 2024
Bootwitch / Interactive demo
Simulated session
Files
Terminal
Welcome to Bootwitch.
Projects / runtime-observability-architecture

Systems Logging and Telemetry - Architecture

runtime-observabilityobservability-and-subsystems

README · ARCHITECTURE · TECHNICAL · source


Where the signals come from

signal capturesignal capture

Three extraction paths exist because providers disagree about where the model id lives: model for most, modelVersion for Google Gemini, meta.model for Cohere, and inside message_start for Anthropic streaming. Each falls through to "unknown" rather than throwing — an unidentifiable response is still a response, and losing it entirely would bias the counts toward the providers that are easy to parse.

model:gossip only records changes. A thousand identical calls write nothing to it. That makes it a drift log rather than a request log, and small enough to keep in one KV value.


Family, not exact id

Counts are keyed by family — opus, sonnet, haiku — rather than by the full dated id.

claude-opus-4-6            → opus
claude-sonnet-4-20250514   → sonnet
gpt-4o-2024-08-06          → gpt-4o     (date suffix stripped)
grok-3                     → grok-3

A dated snapshot rolling forward would otherwise fragment one model's history into unrelated buckets, and the question worth asking of these counters is "how much am I spending on large-tier calls", not "how much on the 2024-08-06 build".

Non-Claude ids keep their name with a trailing date suffix stripped, which is a looser rule and is acknowledged as such: it works on the providers here and is a guess about naming conventions that are not contracts.


Normalising a generation

Journal Club's Generation is the richer half of the story, because it must hold four providers' disagreements without flattening them.

generation normalisationgeneration normalisation

Every optional field distinguishes None from 0. None means the provider does not report this; 0 means it reported none. When a provider includes reasoning-token metadata, had_thinking can use it as a graded, provider-native signal. Those values help interpret a response, but they are not treated as directly comparable measurements across providers.


Four kinds of "the response is short"

These are different facts and the dataclass keeps them apart:

FieldMeans
finish_reason == "stop"The model finished
finish_reason == "length"hit_token_capThe provider cut it off
truncated_by_usOur own display limit clipped it
finish_reason == "error"The call failed; this is still a row

raw_chars records length before any truncation of ours, so the measured length is the model's, not the display layer's. The experiment passes max_len=None to disable our truncation entirely — the 1900-character cut exists for Discord, and letting a presentation constraint into a dataset whose measures include response length would manufacture a finding.


Storage housekeeping

The workspace separates archiving, backup, and cleanup because each job understands a different part of the data. The useful rule is simple: cleanup should act only on objects it can cross-reference confidently. Anything outside that scope should be left alone.

Backups have a different job. A partial backup and a failed backup are different outcomes, so coverage needs to stay visible instead of being flattened into one success signal. The detailed failure-path review stays in the private engineering notes.


Rate limiting

Some model routes need a spend ceiling. The current limiter is intentionally lightweight: it adds useful friction around a small personal system, but it is not an exact quota or a multi-user authorization boundary. Describing it as approximate is part of describing it accurately.


What is deliberately not recorded

The counters and drift history described on this page store operational metadata, not conversation content. Backups are separate and private because their purpose is to preserve application data rather than measure system behavior.


Project overview · All projects