95 lines
2.7 KiB
Markdown
95 lines
2.7 KiB
Markdown
# 2026-05-07 – MagatamaLLM Local Training Verification
|
||
|
||
## Question
|
||
|
||
Did the recent local / MAGATAMA-side MagatamaLLM training actually succeed and increase the active model’s knowledge?
|
||
|
||
## Answer
|
||
|
||
No. The dataset refresh succeeded, but a newer locally adopted MagatamaLLM model was **not** verified.
|
||
|
||
## Evidence
|
||
|
||
### 1. Public MAGATAMA status
|
||
|
||
`GET https://magatama.fichtmueller.org/api/llm/status`
|
||
|
||
Observed:
|
||
- `activeProvider = ollama:magatama-coder:latest`
|
||
- `autoFixProvider = ollama:magatama-coder:latest`
|
||
- `training.lastTrainingAt = 2026-05-06T22:43:20Z`
|
||
- `training.modelVersion = magatama-coder:latest`
|
||
- `training.activeRun = null`
|
||
|
||
Interpretation:
|
||
- the dashboard timestamp reflects the latest dataset/training-state update
|
||
- it does **not** prove that a new local model was imported and activated
|
||
|
||
### 2. Local Ollama state on the Mac
|
||
|
||
`ollama list`
|
||
|
||
Relevant entries:
|
||
- `magatama-coder:latest` → modified `3 weeks ago`
|
||
- `magatama-llm-v2-0:latest` → modified `11 days ago`
|
||
|
||
Interpretation:
|
||
- no newly imported Magatama candidate/adopted model is visible locally
|
||
- the active alias still points to an older model image
|
||
|
||
### 3. Dataset/lane export did work
|
||
|
||
Fresh Erik manifest exists:
|
||
- `/opt/magatama/training-data/runpod/magatamallm/manifest.json`
|
||
|
||
Observed:
|
||
- `generatedAt = 2026-05-06T22:45:15.944Z`
|
||
- `train = 15679`
|
||
- `eval = 1743`
|
||
- `total = 17422`
|
||
|
||
Interpretation:
|
||
- the lane export / pool sync is healthy
|
||
- training input exists and was rebuilt
|
||
|
||
### 4. Adoption/registry proof is missing
|
||
|
||
On Erik, these expected local state files were absent:
|
||
- `/opt/magatama/training-data/model-registry/models.json`
|
||
- `/opt/magatama/training-data/model-registry/runs.json`
|
||
- `/opt/magatama/training-data/model-registry/active.json`
|
||
- `/opt/magatama/data/llm-status.json`
|
||
|
||
Interpretation:
|
||
- no trustworthy proof that a new model artifact was imported, registered, and activated
|
||
|
||
### 5. Historical run records still show failed/non-adopted outcomes
|
||
|
||
Local `training-data/model-registry/training-runs.json` still contains recent `magatamallm` runs such as:
|
||
- `submitted`
|
||
- `not_found_after_submit`
|
||
|
||
There is still no verified “completed_and_adopted” proof for a new MagatamaLLM local model.
|
||
|
||
## Conclusion
|
||
|
||
Current state:
|
||
- pool refresh works
|
||
- lane export works
|
||
- active alias/version switching after training is still not proven
|
||
|
||
Therefore:
|
||
- MagatamaLLM did **not** yet gain a verified newer local knowledge state from the recent run attempts
|
||
- MAGATAMA is still operating on the older active alias `magatama-coder:latest`
|
||
|
||
## Next Required Fix
|
||
|
||
The remaining training-automation gap is still:
|
||
|
||
1. run completes
|
||
2. artifact existence is verified
|
||
3. artifact is adopted/imported locally
|
||
4. smoke tests pass
|
||
5. active alias + model version are updated
|
||
6. only then mark training as successful
|