dsh-continual-harness

English | 中文

A continual self-refinement plugin for DeepSeek Harness: one plugin gives the agent a closed loop of persistent memory + periodic review-and-refine + cross-session shared knowledge + automatic rollback on failure (plan → validate → apply → rollback), implemented through dsh's plugin mechanisms (session events, agent-scoped events, pre-step waterfall, tools service).

The design is inspired by the open-source prime-agent from Prime Intellect, a self-improving coding harness.

One plugin is enough

There is no need to split into multiple packages: this plugin is a single npm package (dsh-continual-harness) that takes effect through the following extension points once mounted:

Capability	Mechanism
State projection (inject harness context each step)	agent/pre-step waterfall listener; incremental injection when the content digest changes
Review and automatic refinement	session/event listener on turn interval / compaction end; runs LLM review → plan → apply automatically
Manual refinement tool	Registers the harness_refine tool (directly callable by the LLM, supports rollback)
In-session review trajectory	Rebuilt from session logs (tail-biased truncation)
Invariant guard	harness/refinement event validation + batched failure reporting

Architecture

src/
  domain.ts      event declaration merging (SessionEventMap / MessageSourceMap / cordis Events)
  types.ts       HarnessState / RefinementProposal / RefinementResult and other types
  storage.ts     disk read/write of state and history (atomic writes, corruption degradation, local/global merge, jsonl history)
  refine.ts      validation, application, rollback (baseline conflict detection, version increments, growth limit)
  skills.ts      SKILL.md rendering + file reconciliation (generated skills are real dsh skills)
  render.ts      model-facing overview / summary / history rendering
  planner.ts     LLM planning prompts and JSON parsing (plan / auto-refine review prompts)
  store.ts       HarnessStore: combined storage + event publishing (session events + agent-scoped events)
  complete.ts    completeViaAgent: completion through ctx.get('llm')
  tool.ts        harness_refine tool
  projection.ts  pre-step projection (digest dedup, <harness_state> injection)
  driver.ts      automatic refinement driver (turn-interval gate / compaction gate / cooldown / re-entry guard)
  invariant.ts   runtime invariant plugin
  index.ts       plugin entry and Config
tests/           12 specs, 120 cases (storage / store / refine / rules / planner / driver / approval / audit / logfile / skills / invariant / plugin integration)

Data layout

<harnessRoot>/                      shared ESP experience root; defaults to ~/.dsh/harness/
  harness_state.json                cross-session global state (ESP)
  refinements.jsonl                 global refinement history (append-only, ESP)
  reviews.jsonl                     cross-batch gate/audit history (ESP extension)
  continual-harness.log             continual-harness implementation log (JSONL, 0600)
  continual-harness.log.1           rotated continual-harness log
  sessions/<sessionKey>/
    harness_state.json              session-local state (shadows same-id global entries)
    refinements.jsonl               session refinement history

• Entries are stored in four kinds — prompt / memory / skill / subagent — each with a version (incremented on every update).
• Merged view: local entries win; a shadowed global entry remains visible under the local:<id> prefix.
• Baseline validation on apply: an edit is rejected if the entry changed concurrently during planning (entry changed during refinement planning).
• base_system_prompt is a protected id; any edit to it is rejected.
• No auto-migration from the legacy layout: installs that predate the flat layout (state under ~/.dsh/harness/harness/ and sessions/<id>/harness/) are not auto-migrated — move the state files into the flat layout above (or re-seed) to keep using the harness. New installs are unaffected.
• Skills are real dsh skills. Every applied skill edit materializes the effective merged entry as a <name>/SKILL.md bundle (YAML name + description frontmatter, kebab-case id) under Config.skillsDir (default $DSH_HOME/skills), where dsh's filesystem skill provider (dsh-skill-filesystem) discovers it live and dsh-tool-skill exposes it to the model. Deletes remove the bundle; rollbacks restore it. Only ids touched by a commit are written or removed, so user-owned skills in the same directory are never touched. Each bundle stamps a metadata provenance block (author: dsh-continual-harness, source: esp) so generated skills are distinguishable from hand-written ones.

Experience Solidification Protocol (ESP)

The Experience Solidification Protocol (ESP) is the protocol surface of this capability set, decoupled from this package's implementation:

Protocol element	Carrier	Description
Experience state schema	harness_state.json (schemaVersion: 1)	Four kinds of entries — prompt / memory / skill / subagent — each with id / kind / version / content / updatedAt
Experience history	refinements.jsonl (append-only)	One RefinementResult record per apply/rollback; rollback by id
Refinement event	session event harness/refinement	Written to the session log on apply/rollback (model-visible ⟺ logged)
Refinement notification	agent event harness/refined	Payload {agent, result}; subscribable by invariant and other plugins
Experience injection	message source harness-state (carries digest)	Pre-injected into the model context; deduplicated by digest change

Any dsh plugin can read and write experience through this protocol (write state files, append history, publish events, inject messages); this package is the protocol's reference implementation and primary consumer (planning / refinement / projection / automatic gate). If the experience read/write layer is ever extracted into a standalone reusable protocol package, dsh-esp can be split out along these lines, with the harness degrading to a consumer of ESP.

Events and message sources

• Session event harness/refinement (RefinementResult) — written to the session log on every apply/rollback (model-visible ⟺ logged).
• Agent-scoped event harness/refined (payload {agent, result}) — subscribable by invariant and other plugins.
• Pre-injected message source.kind === 'harness-state', carrying a digest for deduplication.

Mounting (dsh profile)

Install into a profile in one line (published to npm):

dsh plugin --profile <name> add dsh-continual-harness

The package declares dsh.bundle, so dsh plugin installs it as a profile layer: the dependency is added and its cordis.patch.yml is applied as that bundle's patch. The plugin's runtime imports of @deepseek-ai/* resolve through the profile's flat fallback node_modules directory. Update with dsh plugin --profile <name> update dsh-continual-harness@latest.

Manual overlay (before publish, or to pin a local checkout): apply cordis.patch.yml onto the profile, e.g. ~/.dsh/profiles/<name>/cordis.patch.yml; a patch layer must be a top-level YAML array (insert rows append plugin entries; id-targeted rows override an existing row):

- insert:
    - id: continual-harness
      name: dsh-continual-harness
      config:
        defaultGlobal: true

Prerequisites: the tools, agents, session, llm, systemPrompt capability plugins must load before this plugin (its inject declaration enforces that; mounting is deferred until they load).

Config

Field	Default	Description
harnessRoot	dsh data dir harness/	State root directory (temporary dir in tests)
skillsDir	$DSH_HOME/skills	Directory where skill entries materialize as dsh SKILL.md bundles (dsh's user skill root)
defaultGlobal	required	Target scope when the tool call omits global
maxTrajectoryChars	80000	Max characters of the review trajectory (tail-biased truncation)
plannerMaxTokens	32000	Max tokens for the planner LLM call
autoRefine	{turnInterval: 25, compact: true, cooldownMs: 1200000}	Auto-refine: turn-interval gate, compaction-end gate, cooldown, disable switch
requireGlobalApproval	false	Require explicit human approval before a global write commits (conservative mode)
auditReviews	true	Append every gate verdict to reviews.jsonl under the harness root
logToFile	true	Persist harness logs to continual-harness.log (JSONL, 0600, rotated)
logMaxBytes	5242880 (5 MB)	Rotation cap for the harness log file
maxEntryGrowth	0.5	Per-commit entry growth fraction cap; 0 disables the check
protectedKinds	['skill']	Kinds the automatic path may not modify (reserved; per-entry protection is the enforced guard)

Governance

Every write path — the harness_refine tool and the automatic gate — funnels through a rule layer with three tiers, plus a reversibility backstop:

1. Impact minimization — every edit is validated against a fixed contract before any write. create may omit a reason; update/delete must carry a one-line reason (a missing one rejects the edit with edit "<id>"缺 reason被拒绝，请补充 reason后重新提交). Optional blastRadius (general | project | session) defaults to general. base_system_prompt is immutable. maxEntryGrowth (default 0.5) caps how much an update may grow an entry in one commit (条目增长率超过 maxEntryGrowth上限; 0 disables the check).
2. Legality hard rejects — Protected entries (those carrying protection) are immutable on the automatic path (受保护条目仅显式用户会话可改); during a local refinement the global store is read-only, so touching an unshadowed global entry requires creating a local shadow first (global条目在 local精修期间只读，请创建 local遮蔽条目).
3. Necessity soft gate — before any automatic refinement the review gate decides whether persisting now is worthwhile; a declined review never reaches the store, and every verdict is audited.

Reversibility is the backstop: every committed refinement rolls back by id, and rollbacks carry a system-generated rollback:<id> reason.

Global writes are zero-approval by default: the tool commits a global refinement without consulting any approval service. Set requireGlobalApproval: true for the conservative mode, in which a global write first asks the user through the dsh-user-questions service and is skipped on rejection (global写入未获批：<error>).

The gate and the plugin keep two artifacts under the harness root: every gate verdict is appended to reviews.jsonl (outcomes approved | declined | assessed | failed), and harness log lines from the harness / continual-harness loggers are appended to continual-harness.log (JSONL, 0600, rotated to .1 once logMaxBytes is exceeded).

Watch the plugin log live with:

tail -f ~/.dsh/harness/continual-harness.log

(A dedicated tool entry for governance is deferred.)

Development

The plugin is self-contained: devDependencies pin the published @deepseek-ai/* packages (rc versions), so pnpm install, pnpm run typecheck, pnpm test (120 cases), and pnpm run build (tsc emits lib/types/*.js + *.d.ts; the "." and "./invariant" exports point at the artifacts) all work in a clean checkout — CI and the OIDC release workflow run the same steps. peerDependencies declare the semver ranges consumers (host dsh installations) must satisfy.

Known Limitations and Deferred Work

• No end-to-end tests with a real LLM: completeViaAgent depends on the loaded llm capability and provider/model configuration; tests cover the planning/review paths with a stub Complete. Real e2e requires DEEPSEEK_API_KEY.
• compaction/end is not part of the plugin's type union; the driver triggers it via string comparison after type narrowing, and the gate is silently skipped when the compaction capability is not loaded.
• Projection dedup is an in-process WeakMap<Agent, digest>: the first step after a session restart re-injects (stateless and idempotent, but one extra injection).
• Concurrent writes are last-writer-wins: multiple processes refining the same directory concurrently may overwrite each other; baseline conflict detection during planning can only catch read-after-write races, not serialize them.
• A failed automatic refinement degrades silently (only logged) and never interrupts the session.
• A content-shrink guard (rejecting updates that shrink an entry too far in one commit) is a planned follow-up and is not yet implemented; today only maxEntryGrowth caps how much an update may grow an entry.