How we built Lumen's tone detection engine

Building a grammar checker is relatively straightforward. There are strict rules for grammar, spelling, and basic syntax. But teaching an AI to understand and modify the *tone* of a team—to know the difference between "wordy," "assertive," and "collaborative"—requires a completely different engineering approach.

When we set out to build Lumen, our goal was clear: we needed a tone matching engine that was highly accurate, customized to individual organizations, and capable of returning recommendations in under 250 milliseconds. Here is how we built it.

Quantifying Writing Style as a Vector

The first challenge was representing "voice" mathematically. We started by extracting over 50 linguistic dimensions from text inputs: passive verb ratios, average sentence length distributions, lexical diversity, structural nesting depth, and semantic density.

By plotting these dimensions, we can create a "style profile" for a team's writing. When a team uploads their best historical documentations, Lumen builds a multi-dimensional vector representing their ideal writing style.

"AI-assisted writing should not feel like an external editor checking your grammar. It should feel like your own best self looking over your shoulder, helping you write at your highest level."

Model Fine-Tuning and Latency Optimization

We quickly realized that generic off-the-shelf LLMs are too slow and generic for inline writing suggestions. A writer cannot wait 2 seconds for a suggestion; it breaks their creative flow. We needed to optimize inference latency down to the millisecond.

We began by distil-training custom 3B parameter models on thousands of documentation pairs. By focusing the model strictly on technical and product style transformations, we kept the model size small and fast. We optimized GPU inference using key-value context caching and tensor parallelism, successfully reducing latency from 800ms down to **210ms**.

Retrieval-Augmented Guidelines (RAG)

To make suggestions feel native to a specific team, we integrated a real-time vector search index. When a user drafts a document, Lumen searches their company's index for historical writing examples that match the current context.

If the user writes a spec for a billing system, Lumen retrieves billing-related specs that have been verified as high-quality by team leads, using those documents as real-time context. The recommendations generated are not generic grammar fixes—they are contextual style suggestions that match the team's vocabulary.