Memory rows naming gotoi object IDs go stale silently #91

New issue

Closed

opened 2026-10-08 09:22:01 +02:00 by EduarD3V · 4 comments

EduarD3V commented 2026-10-08 09:22:01 +02:00

Owner

Copy link

Labels: memory, priority/low

remember stores one-line facts per host. Nothing invalidates them when the referenced object is deleted.

Live example, on adam-slave-0:
code

[id f88f134f-d212-4015-864c-f76b35c6081b]
[note] Remediation plan 0c52ac87-8ab6-42cb-a072-b02bc116e6f0: fix /home rsnapshot
self-nesting on slave-0 + prune container storage on master /.

That plan was deleted at the operator's hand during this session. The row now points at nothing and will keep steering future sessions toward a plan that does not exist. Same shape risk for rule IDs, execution IDs, and incident IDs.

Fix: a ref memory kind carrying {kind, id}. On delete of that object, mark the row stale rather than leaving it authoritative. recall surfaces stale rows distinctly.

Acceptance criteria

remember supports a typed object reference.
Deleting the object marks the memory row stale.
recall distinguishes stale rows from live ones.

Labels: memory, priority/low remember stores one-line facts per host. Nothing invalidates them when the referenced object is deleted. Live example, on adam-slave-0: code [id f88f134f-d212-4015-864c-f76b35c6081b] [note] Remediation plan 0c52ac87-8ab6-42cb-a072-b02bc116e6f0: fix /home rsnapshot self-nesting on slave-0 + prune container storage on master /. That plan was deleted at the operator's hand during this session. The row now points at nothing and will keep steering future sessions toward a plan that does not exist. Same shape risk for rule IDs, execution IDs, and incident IDs. Fix: a ref memory kind carrying {kind, id}. On delete of that object, mark the row stale rather than leaving it authoritative. recall surfaces stale rows distinctly. Acceptance criteria remember supports a typed object reference. Deleting the object marks the memory row stale. recall distinguishes stale rows from live ones.

EduarD3V added the

ai-assist

label 2026-10-08 09:22:39 +02:00

gotoi-agent referenced this issue 2026-10-08 20:59:45 +02:00

create_plan / create_match_rule guards are too strict and opaque #88

gotoi-agent added reference issue/91-memory-row-invalidation 2026-10-09 13:07:42 +02:00

gotoi-agent commented 2026-10-09 13:26:51 +02:00

Collaborator

Copy link

Issue #91 plan — memory rows pointing at deleted gotoi objects go stale

Branch: issue/91-memory-row-invalidation (base agent-develop @ 3122e89). Worktree: /home/eduardez/workspace/gotoi-wt/issue-91.

Restated goal

remember stores one-line facts per host. A row that names a plan / rule / execution / incident
keeps steering future control sessions at that object even after the object is deleted. Deliver:

1. A typed reference side-band on the memory row: ref?: { kind: 'plan'|'rule'|'execution'|'incident'; id }.
   content stays free-form; untyped rows are unchanged.
2. On delete of the referenced object the row is flipped stale: true — kept (audit retention), not
   deleted, and no longer authoritative.
3. recall (and the control-chat memory prompt block) surface stale rows distinctly:
   - [stale: plan 0c52ac87 deleted] <content> (id …), sorted after live rows.

Verified base facts (this worktree, 3122e89)

Fact	Evidence
AgentMemory has no ref/stale	middleware/src/types.ts:588-598
Table agent_memories has 9 columns, no ref/stale	middleware/src/db/schema.ts:485-495, middleware/drizzle/0025_control_plan_mode_memories.sql
addMemory / memoriesFor / updateMemory / removeMemory shape	middleware/src/agent-memory.ts:21-132
Prompt block renders - [kind] content (id …)	middleware/src/agent-memory.ts:44-56
recall renders the same line inline	middleware/src/control-agent.ts:1309-1315
Plan delete sites	middleware/src/routes/plans.ts:654, middleware/src/control-agent.ts:1145 (plan_delete)
Rule delete sites	middleware/src/routes/rules.ts:98, middleware/src/control-agent.ts:1265 (rule_delete)
No runtime execution delete	only pruneExecutions (middleware/src/ops.ts:117), a pure helper with no caller except its test
No runtime incident delete	IncidentStore (middleware/src/incidents.ts:15-123) has no remove/delete method
Latest migration	0034_execution_trigger; next free number = 0035
Live example id from the issue body	not present in the repo (runtime data only)
Free UC id	UC-MW-178 (max existing = 177)

Project fact that narrows the task file

Task step 3 says "plans/rules/executions/incidents on_delete calls". Executions and incidents have
no delete path at runtime — nothing tombstones them (ops.ts:117 is an unused pure helper; the
incident store has no removal method). So the invalidation hook exists for all four kinds, but only
plan and rule deletes can call it today. Recorded, not silently dropped: a typed ref of kind
execution/incident is storable and will go stale the moment such a delete path appears.

Scope

In:

• middleware/src/types.ts — MemoryRefKind, MemoryRef, AgentMemory.ref?, AgentMemory.stale?.
• middleware/src/db/schema.ts — ref jsonb, stale boolean not null default false.
• middleware/drizzle/0035_memory_ref_stale.sql + .down.sql.
• middleware/src/db/persist.ts — write ref / stale back on flush.
• middleware/src/agent-memory.ts — addMemory accepts ref; markMemoriesStale(stores, kind, id);
  stale-last sort in memoriesFor / memoriesForAgent; stale marker in memoryPromptBlock.
• middleware/src/control-agent.ts — remember gains ref_kind / ref_id; recall stale marker;
  markMemoriesStale after plan_delete and rule_delete.
• middleware/src/routes/plans.ts, middleware/src/routes/rules.ts — markMemoriesStale after delete.
• middleware/src/routes/agents.ts — optional ref on POST /agents/:id/memories.
• dashboard/src/lib/types.ts — mirror ref + stale (type-mirror gate).
• middleware/tests/uc-mw-178-memory-ref-staleness.test.ts — NEW (red first).
• docs/use-cases/middleware/UC-MW-178-memory-ref-staleness.md — NEW; UC-MW-153 related link.

Out: agent/**; plans/ (product content); executions/incidents store classes (no delete to hook);
any dashboard page work (types mirror only — the FE page already lists rows and will show the flag
whenever it renders it; no new UI surface is added).

UC decision

• UC-MW-178-memory-ref-staleness.md — "memory rows referencing a deleted object go stale".
  related: [UC-MW-153] (bidirectional: add UC-MW-178 to UC-MW-153's YAML + Related heading).
  Implementation cites middleware/src/agent-memory.ts, middleware/src/routes/plans.ts,
  middleware/src/routes/rules.ts, middleware/src/control-agent.ts.
  Tests cite middleware/tests/uc-mw-178-memory-ref-staleness.test.ts.

Ordered tasks

#	Task	Verify
1	Write the red test file (delete-marking + distinct recall + untyped rows untouched)	cd middleware && bun test tests/uc-mw-178-memory-ref-staleness.test.ts → red
2	Commit red: test: repro memory-ref staleness (red) Refs #91	git log -1
3	Types + schema + migration 0035 + persist write-back	bun test tests/uc-mw-db.test.ts
4	agent-memory.ts: ref input, markMemoriesStale, stale-last sort, prompt marker	bun test tests/uc-mw-153*.test.ts tests/uc-mw-control-hardening.test.ts
5	Delete-site wiring: plans/rules routes + control-agent plan_delete/rule_delete	targeted tests
6	remember params + recall marker + POST /agents/:id/memories ref	red test → green
7	Dashboard types mirror	python3 scripts/check-type-mirror.py
8	UC-MW-178 + UC-MW-153 related link	python3 docs/use-cases/_check.py
9	Gates, commit (feat(memory): … Refs #91), STOP	gate list in AGENTS.md

Open questions and the assumption taken

• Does stale need its own column or ride inside the ref jsonb? Two columns (ref, stale): the
  flag is queryable and a DB default covers old rows. Assumed.
• Should a stale row be revived if later edited? No — deletion is terminal; updateMemory does not
  clear the flag. Assumed.
• Should forget still delete a stale row? Yes (unchanged). Assumed.

Risk and boring fallback

Risk: a store delete path added later (e.g. execution pruning) forgets the hook. Boring fallback if
event-driven marking proves unreliable: derive staleness at read time from stores.<kind>.has(id) in
memoriesFor. Not implemented now — the task froze the delete-time hook and the persisted flag.

# Issue #91 plan — memory rows pointing at deleted gotoi objects go stale Branch: `issue/91-memory-row-invalidation` (base `agent-develop` @ 3122e89). Worktree: `/home/eduardez/workspace/gotoi-wt/issue-91`. ## Restated goal `remember` stores one-line facts per host. A row that names a plan / rule / execution / incident keeps steering future control sessions at that object even after the object is deleted. Deliver: 1. A typed reference side-band on the memory row: `ref?: { kind: 'plan'|'rule'|'execution'|'incident'; id }`. `content` stays free-form; untyped rows are unchanged. 2. On delete of the referenced object the row is flipped `stale: true` — kept (audit retention), not deleted, and no longer authoritative. 3. `recall` (and the control-chat memory prompt block) surface stale rows distinctly: `- [stale: plan 0c52ac87 deleted] <content> (id …)`, sorted after live rows. ## Verified base facts (this worktree, 3122e89) | Fact | Evidence | |---|---| | `AgentMemory` has no ref/stale | `middleware/src/types.ts:588-598` | | Table `agent_memories` has 9 columns, no ref/stale | `middleware/src/db/schema.ts:485-495`, `middleware/drizzle/0025_control_plan_mode_memories.sql` | | `addMemory` / `memoriesFor` / `updateMemory` / `removeMemory` shape | `middleware/src/agent-memory.ts:21-132` | | Prompt block renders `- [kind] content (id …)` | `middleware/src/agent-memory.ts:44-56` | | `recall` renders the same line inline | `middleware/src/control-agent.ts:1309-1315` | | Plan delete sites | `middleware/src/routes/plans.ts:654`, `middleware/src/control-agent.ts:1145` (`plan_delete`) | | Rule delete sites | `middleware/src/routes/rules.ts:98`, `middleware/src/control-agent.ts:1265` (`rule_delete`) | | **No runtime execution delete** | only `pruneExecutions` (`middleware/src/ops.ts:117`), a pure helper with no caller except its test | | **No runtime incident delete** | `IncidentStore` (`middleware/src/incidents.ts:15-123`) has no remove/delete method | | Latest migration | `0034_execution_trigger`; next free number = **0035** | | Live example id from the issue body | not present in the repo (runtime data only) | | Free UC id | `UC-MW-178` (max existing = 177) | ## Project fact that narrows the task file Task step 3 says "plans/rules/executions/incidents `on_delete` calls". Executions and incidents have **no delete path at runtime** — nothing tombstones them (`ops.ts:117` is an unused pure helper; the incident store has no removal method). So the invalidation hook exists for all four kinds, but only plan and rule deletes can call it today. Recorded, not silently dropped: a typed ref of kind `execution`/`incident` is storable and will go stale the moment such a delete path appears. ## Scope In: - `middleware/src/types.ts` — `MemoryRefKind`, `MemoryRef`, `AgentMemory.ref?`, `AgentMemory.stale?`. - `middleware/src/db/schema.ts` — `ref jsonb`, `stale boolean not null default false`. - `middleware/drizzle/0035_memory_ref_stale.sql` + `.down.sql`. - `middleware/src/db/persist.ts` — write `ref` / `stale` back on flush. - `middleware/src/agent-memory.ts` — `addMemory` accepts `ref`; `markMemoriesStale(stores, kind, id)`; stale-last sort in `memoriesFor` / `memoriesForAgent`; stale marker in `memoryPromptBlock`. - `middleware/src/control-agent.ts` — `remember` gains `ref_kind` / `ref_id`; `recall` stale marker; `markMemoriesStale` after `plan_delete` and `rule_delete`. - `middleware/src/routes/plans.ts`, `middleware/src/routes/rules.ts` — `markMemoriesStale` after delete. - `middleware/src/routes/agents.ts` — optional `ref` on `POST /agents/:id/memories`. - `dashboard/src/lib/types.ts` — mirror `ref` + `stale` (type-mirror gate). - `middleware/tests/uc-mw-178-memory-ref-staleness.test.ts` — NEW (red first). - `docs/use-cases/middleware/UC-MW-178-memory-ref-staleness.md` — NEW; `UC-MW-153` related link. Out: `agent/**`; `plans/` (product content); executions/incidents store classes (no delete to hook); any dashboard page work (types mirror only — the FE page already lists rows and will show the flag whenever it renders it; no new UI surface is added). ## UC decision - `UC-MW-178-memory-ref-staleness.md` — "memory rows referencing a deleted object go stale". `related: [UC-MW-153]` (bidirectional: add `UC-MW-178` to UC-MW-153's YAML + Related heading). Implementation cites `middleware/src/agent-memory.ts`, `middleware/src/routes/plans.ts`, `middleware/src/routes/rules.ts`, `middleware/src/control-agent.ts`. Tests cite `middleware/tests/uc-mw-178-memory-ref-staleness.test.ts`. ## Ordered tasks | # | Task | Verify | |---|---|---| | 1 | Write the red test file (delete-marking + distinct recall + untyped rows untouched) | `cd middleware && bun test tests/uc-mw-178-memory-ref-staleness.test.ts` → red | | 2 | Commit red: `test: repro memory-ref staleness (red) Refs #91` | `git log -1` | | 3 | Types + schema + migration 0035 + persist write-back | `bun test tests/uc-mw-db.test.ts` | | 4 | `agent-memory.ts`: ref input, `markMemoriesStale`, stale-last sort, prompt marker | `bun test tests/uc-mw-153*.test.ts tests/uc-mw-control-hardening.test.ts` | | 5 | Delete-site wiring: plans/rules routes + control-agent `plan_delete`/`rule_delete` | targeted tests | | 6 | `remember` params + `recall` marker + `POST /agents/:id/memories` ref | red test → green | | 7 | Dashboard types mirror | `python3 scripts/check-type-mirror.py` | | 8 | UC-MW-178 + UC-MW-153 related link | `python3 docs/use-cases/_check.py` | | 9 | Gates, commit (`feat(memory): … Refs #91`), STOP | gate list in AGENTS.md | ## Open questions and the assumption taken - *Does `stale` need its own column or ride inside the ref jsonb?* Two columns (`ref`, `stale`): the flag is queryable and a DB default covers old rows. Assumed. - *Should a stale row be revived if later edited?* No — deletion is terminal; `updateMemory` does not clear the flag. Assumed. - *Should `forget` still delete a stale row?* Yes (unchanged). Assumed. ## Risk and boring fallback Risk: a store delete path added later (e.g. execution pruning) forgets the hook. Boring fallback if event-driven marking proves unreliable: derive staleness at read time from `stores.<kind>.has(id)` in `memoriesFor`. Not implemented now — the task froze the delete-time hook and the persisted flag.

gotoi-agent commented 2026-10-09 14:57:14 +02:00

Collaborator

Copy link

RUNNER-TIMEOUT — wall cap reached (2026-10-09 14:57, 3 h). State for the next runner:

• Coding session gotoi-91 FINISHED: recap posted in pane, 2 commits on branch issue/91-memory-row-invalidation (c916504 test: repro red; 914bcff feat(memory): typed refs go stale when the referenced object is deleted), tree clean. Base 3122e89 (current agent-develop). Its own gate claims are NOT verified.
• NOT done: §4.6 independent review, §5 orchestrator gates, push, PR, DONE comment. The idle tmux session gotoi-91 can be killed (transcript stale since 14:34) — worker is at its idle prompt, coding complete.
• Plan: comment-936-adjacent (13:26:51) on this thread + meta/issue-91.plan.md. Scope: MemoryRef side-band {kind,id}, stale flag, migration 0035_memory_ref_stale, markMemoriesStale on plan/rule delete sites (plans.ts:654, rules.ts:98, control-agent plan_delete/rule_delete), recall + prompt-block render stale rows last. Executions/incidents have no runtime delete path (documented in plan).
• Next run: kill idle gotoi-91 tmux, run reviewer session (model 9router/reviewer) on the 2-commit diff, then §5 gates, push, PR against agent-develop, DONE comment.

Refs #91

RUNNER-TIMEOUT — wall cap reached (2026-10-09 14:57, 3 h). State for the next runner: - Coding session gotoi-91 FINISHED: recap posted in pane, 2 commits on branch issue/91-memory-row-invalidation (c916504 test: repro red; 914bcff feat(memory): typed refs go stale when the referenced object is deleted), tree clean. Base 3122e89 (current agent-develop). Its own gate claims are NOT verified. - NOT done: §4.6 independent review, §5 orchestrator gates, push, PR, DONE comment. The idle tmux session gotoi-91 can be killed (transcript stale since 14:34) — worker is at its idle prompt, coding complete. - Plan: comment-936-adjacent (13:26:51) on this thread + meta/issue-91.plan.md. Scope: MemoryRef side-band {kind,id}, stale flag, migration 0035_memory_ref_stale, markMemoriesStale on plan/rule delete sites (plans.ts:654, rules.ts:98, control-agent plan_delete/rule_delete), recall + prompt-block render stale rows last. Executions/incidents have no runtime delete path (documented in plan). - Next run: kill idle gotoi-91 tmux, run reviewer session (model 9router/reviewer) on the 2-commit diff, then §5 gates, push, PR against agent-develop, DONE comment. Refs #91

gotoi-agent commented 2026-10-09 15:34:14 +02:00

Collaborator

Copy link

REVIEW: CLEAN — findings: none; all 3 ACs pass (typed ref in remember + POST /agents/:id/memories, markMemoriesStale on plan/rule delete sites, recall + prompt render [stale: kind id8 deleted] with stale rows last); plan Scope 14/14 files, no out-of-scope edits; checkers exit 0 (_check.py, check-ai-headers.py, check-type-mirror.py, no 'Showing lines' artifacts); targeted tests 5+10+19 pass (uc-mw-178, uc-mw-db clean-state, uc-mw-169+control-hardening); zero test lines deleted/skipped. Note: executions/incidents have no runtime delete path (hook exists for all 4 kinds, documented in plan). UNFIXED: none. Refs #91

REVIEW: CLEAN — findings: none; all 3 ACs pass (typed ref in remember + POST /agents/:id/memories, markMemoriesStale on plan/rule delete sites, recall + prompt render [stale: kind id8 deleted] with stale rows last); plan Scope 14/14 files, no out-of-scope edits; checkers exit 0 (_check.py, check-ai-headers.py, check-type-mirror.py, no 'Showing lines' artifacts); targeted tests 5+10+19 pass (uc-mw-178, uc-mw-db clean-state, uc-mw-169+control-hardening); zero test lines deleted/skipped. Note: executions/incidents have no runtime delete path (hook exists for all 4 kinds, documented in plan). UNFIXED: none. Refs #91

EduarD3V referenced this issue from a commit 2026-10-09 15:57:48 +02:00

test: repro memory-ref staleness (red) Refs #91

EduarD3V referenced this issue from a commit 2026-10-09 15:57:48 +02:00

feat(memory): typed refs go stale when the referenced object is deleted Refs #91

gotoi-agent referenced this issue 2026-10-09 16:01:03 +02:00

Issue #91: memory rows naming gotoi object IDs go stale on delete #104

gotoi-agent commented 2026-10-09 16:01:24 +02:00

Collaborator

Copy link

DONE — branch issue/91-memory-row-invalidation, PR #104 (#104) targeting agent-develop. 2 commits (c916504 red repro, 914bcff feat): MemoryRef side-band {kind,id} on remember, migration 0035_memory_ref_stale, markMemoriesStale wired to plan/rule delete sites (routes + control-agent), recall + prompt block render stale rows last. Executions/incidents have no runtime delete path (plan documents this). Review: CLEAN (independent reviewer, no findings). Gates: UC catalog ok, ai-headers 582 ok, type-mirror ok, Showing-lines empty, dashboard test+check GREEN, middleware full suite 538 pass / 1 fail — UC-MW-154 secrets GET-org, verified pre-existing (fails identically on clean origin/agent-develop baseline; not touched by this diff). Refs #91

DONE — branch issue/91-memory-row-invalidation, PR #104 (https://git.eduard3v.com/EduarD3V/gotoi/pulls/104) targeting agent-develop. 2 commits (c916504 red repro, 914bcff feat): MemoryRef side-band {kind,id} on remember, migration 0035_memory_ref_stale, markMemoriesStale wired to plan/rule delete sites (routes + control-agent), recall + prompt block render stale rows last. Executions/incidents have no runtime delete path (plan documents this). Review: CLEAN (independent reviewer, no findings). Gates: UC catalog ok, ai-headers 582 ok, type-mirror ok, Showing-lines empty, dashboard test+check GREEN, middleware full suite 538 pass / 1 fail — UC-MW-154 secrets GET-org, verified pre-existing (fails identically on clean origin/agent-develop baseline; not touched by this diff). Refs #91

gotoi-agent referenced this issue from a commit 2026-10-09 16:08:17 +02:00

Merge pull request 'Issue #91: memory rows naming gotoi object IDs go stale on delete' (#104) from issue/91-memory-row-invalidation into agent-develop

EduarD3V closed this issue 2026-10-09 18:39:09 +02:00

Sign in to join this conversation.

issue/91-memory-row-invalidation

Branches Tags

Clear current reference

main

develop

agent-develop

hotfix/latest-tag

docs/reviewer-static-checkers

issue/93-table-fitting

issue/91-memory-row-invalidation

issue/90-requirements-hostnames

issue/89-typed-steps

issue/88-create-guards-explicit

issue/87-plan-rule-update

issue/86-abort-false-green

issue/85-typed-object-tools

issue/71-content-width

chore/integration-branch

docs/agent-review-guard

issue/72-datatable-row-menu

issue/66-docs-sync

issue/73-theme-selector

issue/70-personalization-move

issue/68-security-review

issue/67-bug-hunt

issue/58-tool-endpoint-scoping

issue/45-guided-setup-new-source

issue/46-accent-colour

chore/marketing-flat-docs

issue/44-hidden-plans

issue/43-plan-download

issue/38-applications-aiconfig

issue/41-bulk-decommission

chore/internal-forgejo-url

issue/42-source-list-text

issue/39-machines-details

issue/28-production-selling-audit

issue/27-hero-console-mock

issue/26-docs-unsell-selfhost

issue/22-source-type-badge

issue/21-alert-source-fields

chore/agent-queue-plan

develop-ed

chore/agent-phase3

fix/corpus-config-changed

fix/enforcement-retest

fix/verifier-observability

chore/agent-phase2

fix/issueflow-manual-never-merge

chore/agent-phase1

docs/agent-pipeline

chore/agent-phase0

chore/marketing-console-demo

feat/docs-site

chore/marketing-docs-production

fix/uc-mw-137-session-transcript

Clear current reference

v0.2.1

v0.2.0

agent-catalog

v0.1.1

v0.1.0

Labels

Clear labels   

ai-assist

Eligible for the autonomous issue pipeline — the runner only works issues carrying this label

  

needs_attention

Waiting on an operator decision. The runner skips this issue while the label is present; the operator answers with a DECISION: comment and the runner then applies it and clears this label.

  

verified

Operator verified the fix in production — runner closes it and does not re-work it

No labels

ai-assist

needs_attention

verified

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

2 participants

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

EduarD3V/gotoi#91

No description provided.