soul.demarkus.io:6309/memoryleaderboard/index.md/v19 active reader meta

memoryleaderboard: Project Hub

Entry for demarkus in the Agent Memory Leaderboard: a Go adapter exposing the benchmark's Add/Search HTTP contract over a per-user demarkus world, with agentic search as the submission.

Status

The hosted API is live at https://benchmark.demarkus.io on DigitalOcean Toronto, running main commit 874c86f. Public health, TLS, auth, live Add/Search smoke, conformance, backup, isolated restore, and 16-worker capacity gates pass. The deployed 32-Add/64-Search run had zero HTTP, schema, isolation, or visibility failures with Search p95 4.63 seconds. Daily local backup and 30-day retention are scheduled. Encrypted off-host backup remains pending. Docker data lives on the Droplet root ext4 filesystem after the user declined an encrypted block volume, leaving the runbook's explicit storage-encryption control unmet.

Plan

Authoritative plan: Agent Memory Leaderboard entry. Deployment record: production deployment. The Academic submission uses Coding Memory and the hosted Add/Search API route with OpenAI gpt-4o-mini.

Echo v5 is the Go HTTP framework used by the adapter, not an LLM provider.

Phases

# Work Output
0 Recon: challenge repo, limits, cycle dates, route Academic hosted API route selected
1 Adapter skeleton: Add/Search against local Demarkus merged
2 Agentic search; catalog fallback merged via PR #2
3 Ingestion tuning: distillation, temporal metadata, idempotency merged via PR #3
4 Conformance, synthetic retrieval quality, Top-K load evaluation live provider gates complete; repeated-memory Top-K gap recorded
5 Public release, provider validation, host deploy, Smoke, Full host deployed and locally validated; off-host backup and platform evaluation pending

Repo

github.com/latebit-io/memoryleaderboard is public. Local checkout: /Users/fritz/latebit/memoryleaderboard, clean main at 874c86f matching origin/main.

Workspace

trail
  1. soul.demarkus.io:6309 v19