Home-Directory Consolidation: Analysis & Plan¶
Historical consolidation proposal
The topology and line-number observations below describe the earlier migration plan. They are not current deployment instructions. Current deployment preserves the upstream Hermes service's configured profile; do not move its home merely to satisfy this proposal. PostgreSQL is the live ledger authority. See Hermes deployment readiness and production deployment.
Status: plan (approved target = single authoritative root under $HOME).
Scope: first-party MAC only (src/mac/, deploy/, scripts/, Makefile).
The former in-tree Hermes snapshot is gone (PR #377); do not exclude a path
that is no longer in the tree. Every claim below is grounded in
file:line evidence gathered by a three-way exploration of the .mac,
.hermes, and .openclaw/.nemoclaw namespaces.
Motivation¶
An agent's "personal data" and the control-plane's operational state are
currently spread across multiple sibling directories under $HOME. This leaves
the system open to metadata duplication, abandoned metadata, and a
POLA violation: anyone inspecting an agent's data finds it scattered in a
confusing way, with the same logical datum (secrets, identity/soul) living in
several places. Example that surfaced this: an agent described its dream logs as
living in ~/.hermes/dream_logs/ — a location MAC does not own — even though we
have migrated the gateway off Hermes onto OpenClaw.
1. Actual topology — four homes, not three¶
| Home | Role | Location knob | Owns (authoritative) |
|---|---|---|---|
~/.mac |
Control plane / hub | MAC_HOME (leaky, §3) |
mac.db ledger, fleets.yaml, mac.env (hub secrets), client-principals.json, qdrant/ (L2 memory), installed src/venv/hermes-agent, bin/, OpenShell policies |
~/.hermes |
Gateway / agent-personal | HERMES_HOME (default pinned by one line) |
SOUL.md/USER.md/MEMORY.md/memories/, mood, config.yaml, .env (chat+Slack secrets), auth.json, state.db, sessions/, skills/, plugins/, cron/, logs, dream logs |
~/.hermes-nemoclaw |
NemoClaw gateway | NEMOCLAW_HERMES_HOME |
A second full Hermes home (a clone), used only by the nemoclaw pilot/compose deployment |
~/.openclaw |
— | — | Not a host home. OpenClaw's real home is ~/.mac/openclaw/ (already under .mac). ~/.openclaw exists only as a container symlink → /sandbox/state. .nemoclaw as a dotdir does not exist — it is only an env-var/label prefix. |
Key structural insight: consolidation is already half-done. The newer
components — OpenClaw home ($MAC_HOME/openclaw,
deploy/openclaw/install-openclaw-gateway.sh:16), OpenShell policy
(src/mac/executor_sandbox.py:667), installed source, venv — all live under
~/.mac. The only real holdout is the legacy Hermes home at the $HOME/.hermes
sibling (plus its .hermes-nemoclaw clone). This is not "merge three peers";
it is "pull the last legacy home under the root everything else already uses."
2. Split-brain instances (the concrete duplication / POLA violation)¶
- Secrets are tri-located. The hub
MAC_API_TOKENand Slack tokens live in.mac/mac.envand.hermes/.envand (client-scoped).mac/.env. The hub deliberately reads bothmac.envand.hermes/.envas one merged pool (src/mac/api.py:2755-2768), and deploy actively copies secrets frommac.env→.hermes/.env(deploy/fleet-node-install.sh:8623-8700). Ascrub_spoke_provider_secretsroutine (fleet-node-install.sh:10523-10560) exists only to clean provider keys that leak into.hermes/.env. - Identity/soul is triplicated.
SOUL.md/USER.md/MEMORY.mdare authoritative in.hermes(src/mac/soul_snapshot.py:33-34,journal.py:29), byte-copied into.mac/journal/<date>/(backup,journal.py:88-98), and migrated into.mac/openclaw/workspace/(deploy/openclaw/migrate-hermes-continuity.py:203-215). NemoClaw adds a fourth copy in.hermes-nemoclaw. - Reverse leakage: MAC writes its own artifacts —
mac-runtime-context.json/.md,mac-memory-topology.json— into the gateway's.hermeshome (src/mac/deploy_env.py:437-443). Control-plane data misfiled in the agent-personal home. slack_home_channels.jsonlives in.hermesand is copied to.mac/openclaw(install-openclaw-gateway.sh:767-815).
None of these are live two-writer conflicts today (copies are one-way, source-authoritative). The risk is drift, staleness, and "where does my data actually live?" — precisely POLA.
3. Why relocation is currently unsafe — leaky resolvers¶
Neither root has a single honored resolver, which is the root cause:
.hermes— the read path is clean: everything resolves through$HERMES_HOME(src/mac/journal.py:39-40et al.), pinned by one line (src/mac/deploy_env.py:429) plus three derived paths (:437,438,443). Genuinely hard-wired to the literal.hermes(ignore the env): the SSH soul transport (src/mac/soul_snapshot.py:81), the SSH migrate tar/verify (src/mac/agent_migrate.py:46-58,211,407,504,512), and a host probe (deploy/fleet-node-install.sh:10341)..mac—MAC_HOMEis a leaky knob. The canonicalmac_home()(src/mac/client_principals.py:62) honors it, but dozens of hot-path modules re-implement the path and hard-code~/.macignoringMAC_HOME: the ledger (src/mac/dispatch.py:2811), fleets registry (src/mac/fleet_creds.py:121), journal, most ofsrc/mac/cli.py, OpenShell. SettingMAC_HOMEtoday would relocate some data and orphan the rest — a data-loss trap.
Conclusion: the prerequisite for any relocation is a single, reliably-honored resolver per root. That is Phase 0 and must land first.
4. Target shape (approved: single authoritative root)¶
$MAC_HOME (default ~/.mac; XDG-aware) ← the ONE authoritative root
├── ledger/ mac.db, backups, archive
├── secrets/ mac.env, .env, client-principals.json ← single secret source
├── fleet/ fleets.yaml, specs
├── runtime/ mac-runtime-context.*, memory-topology, journal/
├── gateway/ (← today's ~/.hermes: SOUL, memory, sessions, skills, cron, dream logs)
│ └── openclaw/ (already here as .mac/openclaw)
└── toolchain/ src, venv, bin, hermes-agent
The control-plane vs agent-personal trust boundary is preserved by subdir +
0700 perms, not by separate top-level directories. ~/.hermes and
~/.hermes-nemoclaw become compatibility symlinks into $MAC_HOME/gateway*
so nothing external breaks during the transition.
5. Phased plan (each phase independently shippable and reversible)¶
Phase 0 — One resolver, no data moves (prerequisite)¶
Introduce src/mac/mac_paths.py as the only sanctioned resolver
(mac_home(), gateway_home(), secrets_dir(), ledger_path(), …). Route
every hard-coded ~/.mac and ~/.hermes site through it (see §3 for the site
lists). Add a test/lint that fails on any new literal
Path.home() / ".mac" | ".hermes". Moves zero data; closes the leaky-knob
trap and makes relocation possible. Highest value / lowest risk.
Phase 1 — De-duplicate before relocating¶
Fix the split-brain writes so we do not relocate duplicated data:
- Make one secret file authoritative; render .hermes/.env as a
projection/symlink of mac.env and delete the copy-into-both
(fleet-node-install.sh:8623-8700).
- Move MAC's mac-runtime-context.* / mac-memory-topology.json out of
.hermes into $MAC_HOME/runtime/ (deploy_env.py:437-443); update readers.
- Keep .mac/journal/ (a legitimate dated backup).
Phase 2 — Relocate the gateway home under the root¶
Flip the one pin (deploy_env.py:429) and the three hard-wired literals
(soul_snapshot.py:81, agent_migrate.py, fleet-node-install.sh:10341) to
gateway_home() = $MAC_HOME/gateway. Migrate on-disk ~/.hermes →
$MAC_HOME/gateway with an idempotent, checksum-verified,
permission-preserving move, leaving ~/.hermes as a compat symlink. Fold
.hermes-nemoclaw the same way.
Phase 3 — Retire .hermes (post-migration-off-Hermes)¶
Once OpenClaw is the sole gateway (its live data already under .mac/openclaw),
drop the legacy subtree entirely and remove the compat symlinks.
Cross-cutting — accuracy & safety¶
- Build a
mac_home_auditthat asserts the canonical unified layout and flags orphans/drift (built-in abandoned-metadata detection), wired to a caller in the same change. The earliersrc/mac/hermes_home_audit.py(with its 66-entry canonical allow-list for~/.hermes) was written for this roadmap item but was never called by anything, so it was deleted as dead code (task_1db5aa70); recover it from git history if the allow-list is useful as a starting point. - Every move idempotent + dated-backup (
.mac/backups) + reversible. - Secrets never printed;
0600/0700perms preserved across moves. - Roll out as a deploy epoch with Phase 0 landing first in a backward-compatible read-old / write-new / symlink-bridge mode so no worker breaks mid-fleet.
5b. Dream-cycle learning: two systems, one orphan (executed)¶
Investigation of the dream/learning capability found two dream systems:
- MAC's durable pipeline (correct, alive).
nap_consolidatorwalks each agent's recentmemory_records, writesnap_summary+ typeddream:*artifacts (schemamac.dream.v1) back intomemory_recordsviaadd_memory;dream_scanner/dream_cycle_classifier/dream_repair_tasksturn failure candidates into follow-up tasks. Live on the hub:nap_runs≈1789, latest today;dream:failure_pattern≈109,752,nap_summary≈109,803. This is the authoritative, durable location. (An earlier note that naps were dead is stale — the pipeline is running.) - The gateway's
~/.hermes/dream_logs/(orphan). A host-side cron (~/.hermes/scripts/dream_cycle.py, not MAC code) writes human-readabledream_YYYYMMDD_HHMMSS.mdreports. No first-party MAC code reads this directory — the learning captured there (error patterns, human corrections, near-failure skill correlations, action items) never reached MAC's durable store. On the live host: 334 reports spanning Jul 5→now.
Fix (shipped): src/mac/dream_log_import.py merges those orphaned reports
into memory_records as dream:imported_report memories. It is idempotent,
skips empty "No … detected" reports, and — because the gateway writes hourly and
mostly repeats findings — dedups on the findings (section content), not the
file, so the 334 reports collapse to 65 unique findings-sets rather than 333
timestamp-noise copies. The source directory resolves only through
mac_paths.dream_logs_dir(). Same tool consolidates any other
single-use-but-wrong-path metadata: point it at the misplaced directory.
5c. Host script jobs: one home per artefact class (executed 2026-08-21)¶
The first live instance of the split-brain, and the only one where a single
process straddled both homes. ~/.mac/bin/mac-cron-script-runner
(deploy/openclaw/run-script-cron-job.py) read its pre-run scripts from
~/.hermes/scripts (:390) and wrote its output to
~/.mac/openclaw/script-jobs/output (:408), so a job's code, its schedule and
its output lived in different trees. Visible consequences: Hermes-named reports
(hermes-fleet-drift-check-*.md) accumulating under the OpenClaw tree, and no
single answer to "where does this job live?".
Decision — every artefact a MAC-owned runner touches lives under
script_jobs_dir() = $MAC_OPENCLAW_HOST_DIR/script-jobs:
| Artefact class | Home | Resolver |
|---|---|---|
| Job scripts (the pre-run code) | $MAC_OPENCLAW_HOST_DIR/script-jobs/scripts |
mac_paths.script_jobs_scripts_dir() |
| Job definitions | $MAC_OPENCLAW_HOST_DIR/host-script-jobs.json |
mac_paths.openclaw_home() (unchanged) |
| Output | $MAC_OPENCLAW_HOST_DIR/script-jobs/output |
mac_paths.script_jobs_output_dir() |
| Schedule | launchd plist / systemd --user timer | host supervisor — no home |
Gateway session DB, credentials, config.yaml |
$HERMES_HOME (Phase 2 moves them wholesale) |
mac_paths.gateway_home() |
The last row is deliberate: the session DB and credentials are the gateway's state, read by the job scripts themselves, not by MAC's runner. Splitting them out per-file would create a fifth home; they move as one tree in Phase 2. The runner reads neither — after this change its only gateway-home reference is the named read-only fallback below.
Rollout is read-old / write-new, which is what makes flipping the default
safe. select_scripts_dir() prefers the sanctioned home and consults
legacy_gateway_scripts_dir() ($HERMES_HOME/scripts) only for a script the new
home does not have, so the three enabled jobs on a host that has not been
re-installed keep running instead of failing silently — the parent task's stated
risk. Every fallback is reported in the runner's result as
legacy_scripts_home: true, so the fleet can be swept for stragglers rather than
depending on the old home indefinitely, and
MAC_OPENCLAW_LEGACY_SCRIPTS_DIR=none turns it off once a host is clean.
MAC_HERMES_SCRIPTS_DIR remains honored (plists written before this change set
it) but is deprecated; the installer now exports
MAC_OPENCLAW_SCRIPT_JOB_SCRIPTS_DIR instead.
The on-disk move is deploy/openclaw/relocate-script-job-home.py, installed
as $MAC_HOME/bin/mac-relocate-script-job-home and invoked by
install-openclaw-gateway.sh on every install. It is idempotent and
digest-verified, never deletes, leaves ~/.hermes/scripts as a compat symlink,
and reports a conflict (two differing files claiming one name) rather than
picking a winner. A conflict does not fail the install: the read-only fallback
keeps the job running until an operator resolves it.
The config.yaml backup pile (nine variants on the hub — .bak,
.bak-mac-home-sync, .bak-mac-shutdown-quench, .chatbak, .provbak.*,
.mac-redaction-backup-*) is resolved by the same tool's config-backups
operation. config.yaml is kept because it is the only name the gateway reads —
determined, not guessed. Every variant moves to
$MAC_HOME/backups/hermes-config-<UTC date>/ with a WHICH-WAS-LIVE.md note
recording which file was live, its digest, and each variant's digest, size, mtime
and whether it was byte-identical to the live file. If config.yaml is missing
the tool refuses rather than promote a backup — at that point authority is
genuinely undeterminable from the tree, which is the failure mode this whole
exercise is about. This operation is not run automatically by the installer; it
is an explicit operator step (mac-relocate-script-job-home --apply
config-backups) because it touches the gateway's config directory.
Guard: tests/test_script_job_home.py asserts that with a clean environment
no live runner path (scripts, output, agent/message wrappers, job definitions)
resolves into a Hermes home, that the two offending literals are gone from the
source, that the installer no longer defaults to or exports the Hermes path, and
that the stdlib-only mirrors of mac_paths in the two deploy scripts agree with
the real resolver under both defaults and relocation. Note that the pre-existing
ratchet (tests/test_mac_paths_no_hardcode.py) could not have caught this: it
scans only src/mac/*.py, and both offenders lived in deploy/.
Not covered here: removing the Hermes code (task_2a7df680) and moving the
gateway tree itself (Phase 2). This change makes the first one safe by ending the
runner's dependency on ~/.hermes/scripts.
6. Risks¶
- Fleet-wide blast radius: home resolution runs in every worker + the hub + deploy. Phase 0 must be backward-compatible and land as a coordinated epoch.
- OpenClaw (and any remaining external Hermes home) expects its internal
home layout; we relocate the root via
HERMES_HOME/ the OpenClaw home under$MAC_HOME/openclaw, never rename the gateway's internal structure. - Secrets migration is the highest-care step — verify perms and that no secret is duplicated or dropped; do it with checksums and a reversible backup.