memoryleaderboard: Project Hub
Entry for demarkus in the Agent Memory Leaderboard (agentmemoryleaderboard.ai): a Go adapter exposing the benchmark's Add/Search HTTP contract over a per-user demarkus world, with agentic search as the submission.
Status
Phases 0-4 and repository-side phase 5 packaging are implemented. PR #2 merged phase 2 on 2026-08-16 as 4a37113; current uncommitted work adds current AML contract compliance, Add-time distillation and temporal metadata, reproducible local conformance/quality/load evaluation, and a pinned Caddy + adapter + demarkus production stack with backup and 30-day purge operations. All local verification passes. Remaining work is external: commit changes, select and freeze the Commercial/Industry provider model, provision encrypted host/DNS/credentials, deploy HTTPS, run production smoke/load, obtain AML evaluation access, then run Smoke and Full. First-cycle deadline passed; target a later leaderboard release.
Plan
Authoritative plan: Agent Memory Leaderboard entry. Submission route: Commercial/Industry hosted API. Search uses a scoped nib navigation agent over demarkus lookup/list/fetch; Add uses validated distillation from the same explicitly configured provider while retaining raw evidence. The Academic gpt-4o-mini rule does not apply to Industry keys. Production is a single encrypted Linux host with Caddy TLS, adapter, private demarkus storage, and explicit 30-day data/backup deletion.
Phases
| # | Work | Output |
|---|---|---|
| 0 | Recon: challenge repo, limits, cycle dates, route | commercial hosted route selected |
| 1 | Adapter skeleton: Add/Search against local demarkus | passing contract smoke |
| 2 | Agentic search; catalog fallback | merged via PR #2 |
| 3 | Ingestion tuning: distillation, temporal metadata, idempotency | implemented locally |
| 4 | Conformance, synthetic retrieval quality, Top-K load evaluation | local gates passing; production provider run pending |
| 5 | Pinned container stack, TLS, secrets, backup/retention, submission | package ready; external deploy/access pending |
Repo
/Users/fritz/latebit/memoryleaderboard (github: latebit-io, main branch)
Workspace
- journal/ session notes
- Add architecture.md, patterns.md, debugging.md as they earn content.