Skip to content
(aparté)
v0.19.3 · beta

Keep a long conversation inside the model's window

Show how much of the context window a conversation uses, and summarise the older turns when it fills up — automatically, from a button, or your own way.

On this page
Open
The context gauge past its two thresholds: warn, then danger.

Install and set up

Section titled “Install and set up”
bash
npm install @aparte/plugin-compaction
ts
import { setupCompaction } from '@aparte/plugin-compaction';

const compaction = setupCompaction();
html
<aparte-chat id="main-chat">
  <aparte-chat-viewport></aparte-chat-viewport>
  <aparte-composer>
    <div class="aparte-composer-shell">
      <div class="aparte-composer-row">
        <aparte-composer-input></aparte-composer-input>
        <aparte-composer-send></aparte-composer-send>
      </div>
      <aparte-composer-toolbar>
        <aparte-context auto-compact></aparte-context>
      </aparte-composer-toolbar>
    </div>
  </aparte-composer>
</aparte-chat>

<aparte-context> measures how much of the model’s window the conversation uses, from the usage each reply reports: it shows nothing until a reply reports its usage — a provider that reports none leaves it empty — and goes blank again after a compaction, until the next reply. It turns warn at 75 % of the window and danger at 90 %, which its warn and danger attributes change. With auto-compact, a turn that ends in danger asks for a compaction; once that request is answered, the next turn still in danger asks again. setupCompaction() answers: it summarises the turns that no longer fit and keeps the recent ones as they are. Without the plugin, the gauge only measures. The gauge goes in the composer’s toolbar, which the composer toolbar places in plain HTML and in each framework.

The window is the current model’s contextWindow. A window attribute sizes the gauge alone: when the model declares no window, give the compaction the same number through its selector, or it keeps the last four messages and summarises the rest.

Compact from a button

Section titled “Compact from a button”
ts
document.querySelector('#compact')?.addEventListener('click', async () => {
  const outcome = await compaction.compact();   // never throws
  if (!outcome.ok) console.warn(outcome.error);
});

compact() compacts the chat whose id it is given, or the first on the page, and resolves to what happened: { ok: true, skipped: false, summary, kept, dropped } when it did; { ok: true, skipped: true, reason } when there was nothing to do — 'empty', 'nothing-to-drop', or 'running' and 'streaming' when a compaction or a reply is under way; and { ok: false, error } when it could not — no chat or provider found, a summary that failed or was cancelled, a conversation that changed meanwhile. A button can also send what the gauge sends: window.dispatchEvent(new CustomEvent('aparte-compact', { detail: { targetId: 'main-chat' } })). A host that only calls compact() itself passes listen: false, and the setup listens to no event.

Show progress, and cancel

Section titled “Show progress, and cancel”
ts
declare const spinner: { show(): void; hide(): void; fail(error: unknown): void };   // yours

window.addEventListener('aparte-compact-start', () => spinner.show());
window.addEventListener('aparte-compact-done', () => spinner.hide());
window.addEventListener('aparte-compact-error', (event) => spinner.fail(event.detail.error));

A compaction is a model call, so it takes a moment. The events go out on window, each with the chat’s targetId; aparte-compact-done carries { summary, kept, dropped }, or { skipped: true, reason }. compaction.running is true meanwhile, and compaction.abort(), or an aparte-abort sent for the chat, cancels it and leaves the conversation as it was; the cancel arrives as aparte-compact-error with error: 'Compaction aborted', and with listen: false only abort() cancels. A summary that comes back after the conversation was switched is not applied. compaction.dispose() removes the setup.

Choose what stays

Section titled “Choose what stays”
ts
import { aparteGlobalConfig } from '@aparte/core';
import { setupCompaction, createCompactionSelector } from '@aparte/plugin-compaction';

setupCompaction({
  selector: createCompactionSelector({
    contextWindow: 128_000,
    systemPrompt: () => aparteGlobalConfig.resolveSystemPrompt(),
    tools: () => aparteGlobalConfig.getTools(),
    minKeep: 6,   // never summarise the last three exchanges
  }),
});

By default the newest turns that fit the current model’s window stay word for word, after room for the system prompt, the tools, the reply, a buffer of a tenth of the window and a safety margin, and the older ones are summarised. A compaction at the default 90 % therefore frees a few turns; a larger autocompactBufferPct in the selector’s config, or a lower danger, frees more. The selector above does the same against a window you choose, with minKeep as a floor; any function from the messages to { keep, drop } can be a selector. When the model declares no window, the last four messages stay, which keepWithoutWindow changes.

There is one setup per configuration: a second setupCompaction call replaces the first. Pass the selector, the prompt and every other option in one call.

Summarise your way

Section titled “Summarise your way”
ts
import { setupCompaction } from '@aparte/plugin-compaction';

setupCompaction({
  prompt: 'Summarise in French, for a support agent picking up the ticket. Do not answer the conversation, do not continue it, and do not call a tool.',
  keyResolver: (providerId) => localStorage.getItem(`key:${providerId}`),
});

The summary is asked of the current model, through the same transport as the chat, and the request carries _meta: { compaction: true }, so a server of yours can send it to a cheaper model. It holds the summarised turns as text, their tool calls and errors included. prompt replaces the instruction: keep its last sentence, or a conversation that ends in a question can come back answered instead of summarised. keyResolver is asked for the provider’s key first, and the configuration’s key is used when it returns nothing, as with AparteClient. summarize: (request, signal) => Promise<string> replaces the call altogether — your own endpoint, any model — and needs no provider at all.

Several chats, or a transcript in a store

Section titled “Several chats, or a transcript in a store”

One setup answers every chat on the page: the gauge’s request names its chat, and that one is compacted. A gauge names its chat when it sits in a chat that has an id, or carries a target naming one; with neither, the first chat on the page is the one compacted. A request that arrives while another compaction runs is skipped with reason: 'running', and the gauge asks again on its next turn in danger.

A chat with a configuration of its own takes a setup on that configuration, scoped to the chat’s id — setupCompaction({ scopeToTargetId: 'support' }, support) — so it answers that chat alone and compact() with no id compacts it. When your transcript lives in a store rather than in the page, resolveTarget: (targetId) => ({ getMessages, clearAll, appendMessage }) hands the plugin your own message list.

What the person and the model see

Section titled “What the person and the model see”
Open
The summary in the conversation, where the older turns were.

The summarised turns are replaced by one notice, headed by the locale’s compactionSummaryTitle — “Conversation summary” in English — with the turns that stayed after it, and whatever arrived while the summary was being written. Only the versions on screen stay: the other versions of every answer, and their pickers, are dropped. The notice is a message with compaction: true, drawn apart from the others, without avatar or buttons. AparteClient sends it to the model as a summary of what came before; a summary you make by other means gets the same treatment from the same flag.

Reference

Section titled “Reference”

The default selector estimates tokens as characters ÷ 3.8 (estimateTokens). Its budget is the window, less the system prompt, the tools, the reserves for thinking and for the reply, a buffer, a safety margin and a slot for the summary.

ExportWhat it is
computeHistoryBudget, splitHistoryBudgetthe budget with its breakdown (BudgetResult, BudgetBreakdown), and its split into room for the summary and the verbatim window (SplitBudget)
estimateTokens, estimateTokensJsonthe estimate, for text and for a JSON value
DEFAULT_COMPACTION_CONFIGthe reserves and ratios a CompactionConfig overrides
transcriptForSummary, messageText, DEFAULT_COMPACTION_PROMPTthe transcript and the instruction the default summary uses, for a summarize of your own
CompactionSetupOptions, CompactionControllerwhat setupCompaction takes and returns
CompactionOutcome, CompactionSkipReasonwhat compact() resolves to
CompactionMessageSelector, CompactionSelectorOptions, CompactionSelector, CompactionSelection, CompactableMessagethe selector and its parts
CompactionSummarizer, CompactionKeyResolver, CompactionTargetthe summarize, keyResolver and resolveTarget options

The events are typed in core’s event map — AparteCompactEventDetail, AparteCompactStartEventDetail, AparteCompactDoneEventDetail and AparteCompactErrorEventDetail — and listed in the events reference.

Next

Section titled “Next”