From 14ad31da4699e457c61d58426e01467668a96d11 Mon Sep 17 00:00:00 2001 From: Rene Fichtmueller Date: Wed, 29 Apr 2026 23:06:01 +0200 Subject: [PATCH] sync: refresh complete chat handoff with magatama state --- sync/CURRENT.md | 39 ++++++++-- .../2026-04-29-codex-complete-chat-sync.md | 72 +++++++++++++++++++ 2 files changed, 107 insertions(+), 4 deletions(-) diff --git a/sync/CURRENT.md b/sync/CURRENT.md index 2708c26..37c8890 100644 --- a/sync/CURRENT.md +++ b/sync/CURRENT.md @@ -1,6 +1,6 @@ # Current TIP Sync State -Updated: 2026-04-29 21:02 UTC +Updated: 2026-04-29 21:15 UTC ## Active Policy @@ -27,6 +27,18 @@ When work touches TIP, Magatama, LLM Gateway, bridges, auth, or shared Erik infr ## Latest Work +- MAGATAMA cross-repo state from the same chat is now synced into this handoff: + - Compliance framework cards in MAGATAMA are clickable and open per-framework requirement details. + - MAGATAMA training status was corrected so `New Since Last Training` no longer falsely shows `0`. + - Live verified/deduped MAGATAMA training state after the fix: + - `collectedExamples: 49` + - `rawExamples: 58` + - `duplicateExamples: 9` + - `effectiveExamples: 49` + - `newSinceLastTraining: 49` + - MAGATAMA now filters training metrics to verified/trainable examples only. + - Failed/escalated MAGATAMA remediation records should go to `errors.jsonl`, not the main `fixes.jsonl`, so the next MagatamaLLM run does not train on junk. + - Gitea-backed training pool remains the default target for training writes. - Complete Codex chat sync was added: - `sync/history/2026-04-29-codex-complete-chat-sync.md` - captures Ghost/blog updates, LinkedIn voice preferences, LPO/AI-fabric blog edits, Rest-Is-Not-Laziness scheduling replacement, and security notes. @@ -74,6 +86,19 @@ When work touches TIP, Magatama, LLM Gateway, bridges, auth, or shared Erik infr - `/tmp/tip-training-data/qa-pairs/robot-control-high.jsonl` - `/tmp/tip-training-data/robot-experiences/2026-04-29.jsonl` +## MAGATAMA Training / Operations State + +- Relevant local repo: + - `/Users/renefichtmueller/Desktop/Claude Code/magatama` +- Latest confirmed live MAGATAMA training metric after dashboard fix: + - `newSinceLastTraining: 49` +- Meaning: + - the old `0` was incorrect. + - the currently visible trainable MAGATAMA corpus is based on verified and deduplicated examples only. +- Important training integrity rule: + - report-only or failed/escalated records must not be treated as verified training fixes. + - keep them separated from the main verified training corpus. + ## Erik Status - Synced TIPLLM robot/training code to `/opt/tip`. @@ -84,6 +109,9 @@ When work touches TIP, Magatama, LLM Gateway, bridges, auth, or shared Erik infr - `/opt/tip/packages/scraper/src/scrapers/scheduler.ts` - `/opt/tip/packages/scraper/src/vendor-discovery-crawler.ts` - `tip-api` and `tip-scraper-daemon` are online. +- Shared Erik note from the same chat: + - MAGATAMA dashboard/core were redeployed during compliance/training fixes. + - TIP crawler policy remains unchanged: Erik is controller/light runner only, not heavy crawl execution host. ## Last Live Verification Snapshot @@ -104,7 +132,9 @@ From 2026-04-29: 3. For BlogLLM work, treat `fo-blog-v7` as Adapter Bridge / PEFT adapter, not as a `~/.ollama` GGUF model. 4. Also read `llm-gateway/sync/CURRENT.md` when work touches shared Erik infrastructure, LLM routing, bridges, auth, TIPLLM, or crawler orchestration. 5. For TIP robot/crawler planning, use TIPLLM only. Do not route this lane through external AI providers. -6. If testing robots, start with dry runs only: +6. When training pools or model stats look suspicious, prefer verified-only counts and check whether failed/escalated rows polluted the corpus. +7. For MAGATAMA-adjacent work, keep writing learnings back into the Gitea-backed pool and avoid training on report-only pseudo-fixes. +8. If testing robots, start with dry runs only: ```bash npm run robots:verification -w packages/scraper -- --status @@ -112,7 +142,7 @@ npm run robots:verification -w packages/scraper -- --tipllm-plan --limit=3 npm run robots:verification -w packages/scraper -- --enqueue=details-fast-lane --profile=erik-safe --dry-run ``` -7. Only dispatch real crawl work after deciding the target host: +9. Only dispatch real crawl work after deciding the target host: - Erik: `erik-safe`, tiny batches only. - Pi: `pi-fetch`. - Proxmox: `proxmox-heavy`. @@ -125,4 +155,5 @@ There are existing uncommitted changes outside `sync/`. Some are Codex work from - `6c42ca7 docs: add shared agent sync handoff` - `8e7c5aa docs: link llm-gateway sync handoff` -- Pending after this update: complete Codex chat sync in `sync/history/`. +- Pending after this update: + - push the refreshed complete-chat sync including MAGATAMA training/compliance state. diff --git a/sync/history/2026-04-29-codex-complete-chat-sync.md b/sync/history/2026-04-29-codex-complete-chat-sync.md index 3582ff9..ceccd2a 100644 --- a/sync/history/2026-04-29-codex-complete-chat-sync.md +++ b/sync/history/2026-04-29-codex-complete-chat-sync.md @@ -223,6 +223,73 @@ Codex read: - `sync/history/2026-04-29-codex-full-session-handoff.md` - sibling `llm-gateway/sync/CURRENT.md` +## MAGATAMA / Training / Compliance State Added From The Same Chat + +The same cross-repo chat also included live MAGATAMA work that should remain part of the shared handoff state. + +### Compliance UX + +- MAGATAMA compliance cards are now expected to be clickable. +- The detail view should show, per framework: + - fulfilled controls. + - partially fulfilled controls. + - open / not fulfilled controls. + - manual-review controls. + - not-relevant controls. +- This work was completed and deployed on Erik in the MAGATAMA dashboard. + +### Training Metric Correction + +Rene reported that MAGATAMA showed: + +```text +New Since Last Training: 0 +``` + +This was wrong. + +Root cause found in the chat: + +- Erik did not have a trusted `last_run.json` baseline for MAGATAMA training state. +- Dashboard fallback logic treated the current corpus as if it had already been trained. +- The MAGATAMA corpus also contained failed/escalated non-verified rows that should not count as trainable fixes. + +Corrected live MAGATAMA state after the fix: + +- `collectedExamples: 49` +- `rawExamples: 58` +- `duplicateExamples: 9` +- `effectiveExamples: 49` +- `newSinceLastTraining: 49` +- `hasTrustedTrainingBaseline: false` + +Interpretation: + +- the old `0` was a false UI result. +- MAGATAMA does have verified trainable material. +- the currently visible trainable set is the verified/deduped effective corpus. + +### Training Integrity Decision + +The chat established a strong rule for MAGATAMA learning: + +- do not train on report-only pseudo-fixes. +- do not mix failed/escalated remediation rows into the main verified fix corpus. +- keep the Gitea-backed learning pool as the default destination for training writes. + +Implementation expectation from the chat: + +- verified/trainable fixes stay in the main fix corpus. +- failed/escalated/non-verified records belong in a separate error lane such as `errors.jsonl`. + +### Shared Erik Safety + +- MAGATAMA dashboard/core were redeployed during these compliance/training fixes. +- This does not change the TIP crawler rule: + - TIPLLM-only for robot/crawler planning. + - Erik remains controller/light `erik-safe` only. + - heavy crawler execution belongs on Proxmox/Pi workers. + ## Current Worktree / Git Decision The TIP repo has many existing non-sync changes and untracked files. These are outside this chat-sync task. @@ -245,6 +312,7 @@ No Erik deployment or load test was performed. - Rotate Ghost Admin API key. - Continue to use `sync/` as the handoff source before any TIP/LLM/crawler work. +- Keep cross-repo MAGATAMA notes in sync here until MAGATAMA has its own equivalent binding sync workflow. - Before any real crawler dispatch, choose the target: - Erik: `erik-safe`, tiny batches only. - Pi: `pi-fetch`. @@ -253,3 +321,7 @@ No Erik deployment or load test was performed. - For future BlogLLM work, remember from current sync: - active model is `fo-blog-v7`. - it is an Adapter Bridge / PEFT adapter, not a normal `~/.ollama` GGUF manifest. +- For future MAGATAMA training work: + - verify whether the baseline is trusted before interpreting `newSinceLastTraining`. + - prefer verified-only counts. + - keep failed/escalated rows out of the main trainable corpus.