soul.demarkus.io:6309/memoryleaderboard/index.md/v15 draft reader meta

memoryleaderboard: Project Hub

Entry for demarkus in the Agent Memory Leaderboard: a Go adapter exposing the benchmark's Add/Search HTTP contract over a per-user demarkus world, with agentic search as the submission.

Status

The repository is public at latebit-io/memoryleaderboard, with main fixed at Academic-readiness merge 9f4ff7e. An OpenAI key is bootstrapped, but live gpt-4o-mini validation is blocked by OpenAI 429 insufficient_quota; billing or prepaid credits must be enabled. The attempt exposed masked async provider errors, fixed with regression coverage on fix/nav-provider-errors. Full race tests, vet, shell checks, and diff checks pass.

Plan

Authoritative plan: Agent Memory Leaderboard entry. Search uses a scoped nib navigation agent over Demarkus lookup/list/fetch; Add uses gpt-4o-mini distillation while retaining raw evidence. Production is a single encrypted Linux host with Caddy TLS, adapter, private Demarkus storage, and explicit 30-day data/backup deletion. The Academic submission uses Coding Memory and the hosted Add/Search API route.

Echo v5 is the Go HTTP framework used by the adapter, not an LLM provider.

Phases

# Work Output
0 Recon: challenge repo, limits, cycle dates, route Academic hosted API route selected
1 Adapter skeleton: Add/Search against local Demarkus merged
2 Agentic search; catalog fallback merged via PR #2
3 Ingestion tuning: distillation, temporal metadata, idempotency merged via PR #3
4 Conformance, synthetic retrieval quality, Top-K load evaluation local gates pass; live provider run pending
5 Public release, provider validation, host deploy, Smoke, Full public release complete; OpenAI quota needed

Repo

github.com/latebit-io/memoryleaderboard is public. Local checkout: /Users/fritz/latebit/memoryleaderboard, branch fix/nav-provider-errors over main at 9f4ff7e.

Workspace

trail
  1. soul.demarkus.io:6309 v15