docs: add workspace architecture design and README
Captures the approved design for expanding notfiles into a Cargo workspace (notcore, notfiles, notsecrets, nothooks, notstrap) with full new-machine bootstrap story and gradual migration path from the dotfiles shell script ecosystem. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Joseph O'Brien
244
README.md
Normal file
244
README.md
Normal file
@@ -0,0 +1,244 @@
|
||||
# notfiles
|
||||
|
||||
A pure-Rust dotfiles manager — and eventually, a complete new-machine bootstrap system.
|
||||
|
||||
`notfiles` started as a Rust replacement for [GNU Stow](https://www.gnu.org/software/stow/). It's growing into a **Cargo workspace** of focused crates that together replace an entire shell-script-based dotfiles ecosystem.
|
||||
|
||||
---
|
||||
|
||||
## Quick Start
|
||||
|
||||
```bash
|
||||
# Symlink your dotfiles
|
||||
notfiles link
|
||||
|
||||
# Check status
|
||||
notfiles status
|
||||
|
||||
# Remove symlinks
|
||||
notfiles unlink
|
||||
```
|
||||
|
||||
On a new machine (once `notstrap` ships):
|
||||
|
||||
```bash
|
||||
cargo install notstrap
|
||||
notstrap run
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Workspace Architecture
|
||||
|
||||
```
|
||||
notfiles/
|
||||
├── crates/
|
||||
│ ├── notcore/ # shared types, config, paths, errors
|
||||
│ ├── notfiles/ # symlink engine (stow replacement) ← you are here
|
||||
│ ├── notsecrets/ # age key retrieval + sops decrypt
|
||||
│ ├── nothooks/ # hook execution engine
|
||||
│ └── notstrap/ # new-machine bootstrap orchestrator
|
||||
├── notfiles.toml # symlink package config
|
||||
└── notstrap.toml # bootstrap hook/phase config
|
||||
```
|
||||
|
||||
### Crate responsibilities
|
||||
|
||||
| Crate | Type | Does |
|
||||
|-------|------|------|
|
||||
| `notcore` | lib | Shared types, config, paths, errors — no deps on other crates |
|
||||
| `notfiles` | lib + bin | Symlink/copy packages into `$HOME`, track state |
|
||||
| `notsecrets` | lib | Retrieve age key → sops decrypt → inject secrets |
|
||||
| `nothooks` | lib + bin | Run bootstrap hooks in phases, skip already-run setup hooks |
|
||||
| `notstrap` | bin | Orchestrate everything on a fresh machine |
|
||||
|
||||
### Dependency graph
|
||||
|
||||
```
|
||||
notstrap
|
||||
├── notfiles
|
||||
│ └── notcore
|
||||
├── notsecrets
|
||||
│ └── notcore
|
||||
└── nothooks
|
||||
└── notcore
|
||||
```
|
||||
|
||||
`notcore` is the only shared dependency. No circular deps.
|
||||
|
||||
---
|
||||
|
||||
## How `notfiles` Works
|
||||
|
||||
Each subdirectory of your dotfiles repo is a **package**. `notfiles link` walks each package and symlinks its contents into a target directory (default: `$HOME`), mirroring the directory structure.
|
||||
|
||||
```
|
||||
dotfiles/
|
||||
└── zsh/
|
||||
└── .zshrc → symlink → ~/.zshrc
|
||||
|
||||
dotfiles/
|
||||
└── git/
|
||||
└── .config/
|
||||
└── git/
|
||||
└── config → symlink → ~/.config/git/config
|
||||
```
|
||||
|
||||
State is tracked in `.notfiles-state.toml` so `unlink` and `status` know exactly what was linked, when, and how.
|
||||
|
||||
### Link flow
|
||||
|
||||
```
|
||||
notfiles link
|
||||
│
|
||||
├─ resolve_packages() discover subdirs or validate requested names
|
||||
│
|
||||
├─ collect_files() recursive walk, apply ignore patterns (globset)
|
||||
│
|
||||
├─ conflict_check() existing file? symlink to wrong target?
|
||||
│
|
||||
└─ linker::link_package() create symlinks (or copies), write state
|
||||
```
|
||||
|
||||
### State file
|
||||
|
||||
`.notfiles-state.toml` records every linked file:
|
||||
|
||||
```toml
|
||||
[[entries]]
|
||||
source = "/Users/joe/dotfiles/zsh/.zshrc"
|
||||
target = "/Users/joe/.zshrc"
|
||||
method = "symlink"
|
||||
package = "zsh"
|
||||
linked_at = "2026-03-31T10:00:00Z"
|
||||
```
|
||||
|
||||
This powers `status` (diff expected vs actual) and `unlink` (clean removal with empty-parent cleanup).
|
||||
|
||||
---
|
||||
|
||||
## New Machine Bootstrap (notstrap)
|
||||
|
||||
The hardest part of a new machine is the chicken-and-egg problem: you need secrets to set up the machine, but secrets live in an encrypted file that requires a key you haven't retrieved yet.
|
||||
|
||||
`notstrap` solves this with a staged bootstrap:
|
||||
|
||||
```
|
||||
notstrap run
|
||||
│
|
||||
├─ 1. Prerequisites check
|
||||
│ Is bw/sops/age available? Print exactly what's missing and stop.
|
||||
│
|
||||
├─ 2. notsecrets — retrieve age key
|
||||
│ ├─ try: Bitwarden CLI (bw unlock)
|
||||
│ ├─ fallback: --key-file <path> (USB drive)
|
||||
│ └─ fallback: interactive prompt (paste key)
|
||||
│ └─ sops decrypt secrets.sops.env → env injected
|
||||
│ (now op, bw, github, openai, anthropic tokens are live)
|
||||
│
|
||||
├─ 3. Clone dotfiles repo (if not present)
|
||||
│
|
||||
├─ 4. notfiles link — stow all packages
|
||||
│
|
||||
├─ 5. nothooks --phase dot
|
||||
│ shell config, git config, AI tool configs (~seconds, re-runnable)
|
||||
│
|
||||
├─ 6. nothooks --phase setup
|
||||
│ Homebrew/Nix packages, mise runtimes, dev tools, op install (~minutes, once)
|
||||
│
|
||||
└─ 7. Report
|
||||
✓ linked 142 files ✓ 3 dot hooks ✓ 7 setup hooks
|
||||
```
|
||||
|
||||
Note: 1Password (`op`) is installed as a **hook** in phase `setup` — after secrets are already available via sops. It takes over secret management for day-to-day use once the machine is live.
|
||||
|
||||
---
|
||||
|
||||
## Secrets Bootstrap Detail
|
||||
|
||||
`notsecrets` implements an `AgeKeySource` trait with three sources tried in order:
|
||||
|
||||
```
|
||||
AgeKeySource
|
||||
├── BitwardenSource bw unlock → session token → bw get item age-key
|
||||
├── FileSource read from --key-file path (USB, etc.)
|
||||
└── PromptSource read from stdin (paste)
|
||||
```
|
||||
|
||||
Once the age key is retrieved, it's written to `~/.config/sops/age/keys.txt` and `sops` decrypts `secrets.sops.env`. The decrypted file contains all critical bootstrap credentials (op, bw, github, openai, anthropic, etc.) and is injected into the environment for subsequent hooks.
|
||||
|
||||
---
|
||||
|
||||
## Hook Phases
|
||||
|
||||
`nothooks` runs hooks in two phases defined in `notstrap.toml`:
|
||||
|
||||
| Phase | Speed | Re-runnable | Examples |
|
||||
|-------|-------|-------------|---------|
|
||||
| `dot` | ~seconds | Yes | shell config, git config, AI tool configs |
|
||||
| `setup` | ~minutes | No (tracked) | Homebrew packages, mise runtimes, op install |
|
||||
|
||||
`setup` hooks are tracked in `.nothooks-state.toml` — already-run hooks are skipped unless `--force` is passed. `dot` hooks always re-run (they're idempotent by design).
|
||||
|
||||
---
|
||||
|
||||
## Configuration
|
||||
|
||||
### `notfiles.toml` — symlink config
|
||||
|
||||
```toml
|
||||
[defaults]
|
||||
method = "symlink"
|
||||
target = "~"
|
||||
|
||||
[packages.secrets]
|
||||
method = "copy" # copy instead of symlink for sensitive files
|
||||
|
||||
[packages.work]
|
||||
target = "~/work" # different target dir
|
||||
ignore = ["*.local"]
|
||||
```
|
||||
|
||||
### `notstrap.toml` — bootstrap config
|
||||
|
||||
```toml
|
||||
[bootstrap]
|
||||
dotfiles_repo = "git@github.com:you/dotfiles.git"
|
||||
dotfiles_dir = "~/dotfiles"
|
||||
|
||||
[[hooks]]
|
||||
name = "shell-config"
|
||||
script = "scripts/setup-git-config.sh"
|
||||
phase = "dot"
|
||||
|
||||
[[hooks]]
|
||||
name = "homebrew-packages"
|
||||
script = "scripts/setup-packages.sh"
|
||||
phase = "setup"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Migration from dotfiles/
|
||||
|
||||
Migration is gradual — shell scripts are replaced as Rust equivalents ship:
|
||||
|
||||
| Phase | Ships | Replaces |
|
||||
|-------|-------|---------|
|
||||
| 1 | `notfiles` workspace (now) | GNU Stow |
|
||||
| 2 | `notsecrets` + `notstrap` skeleton | `setup-secrets.sh`, `install.sh` |
|
||||
| 3 | `nothooks` | `bootstrap.sh` hook runner |
|
||||
| 4 | Hook-by-hook Rust rewrites | Individual `setup-*.sh` scripts |
|
||||
| 5 | `dotfiles/` archived | — |
|
||||
|
||||
---
|
||||
|
||||
## Development
|
||||
|
||||
```bash
|
||||
cargo build # build all crates
|
||||
cargo test # run all tests
|
||||
cargo test -p notfiles # test one crate
|
||||
cargo clippy --workspace # lint everything
|
||||
cargo fmt --check # format check
|
||||
```
|
||||
@@ -0,0 +1,195 @@
|
||||
# notfiles Workspace Architecture Design
|
||||
|
||||
**Date:** 2026-03-31
|
||||
**Status:** Approved
|
||||
|
||||
## Overview
|
||||
|
||||
`notfiles` is a pure-Rust dotfiles manager replacing GNU Stow. This design expands it into a **Cargo workspace** of focused crates that together replace the entire `dotfiles/` shell script ecosystem — including bootstrapping a new machine from scratch.
|
||||
|
||||
The primary pain points being solved:
|
||||
|
||||
- New machine setup requires too many manual steps before the bootstrap script even runs
|
||||
- Secrets/1Password setup is fragile — keys aren't available early enough
|
||||
- Too many shell scripts that are hard to maintain and test
|
||||
- Nix/Homebrew split is unclear; Rust tools are preferred
|
||||
|
||||
---
|
||||
|
||||
## Workspace Structure
|
||||
|
||||
```
|
||||
notfiles/ # Cargo workspace root
|
||||
├── Cargo.toml # [workspace] members
|
||||
├── notfiles.toml # dotfiles symlink config
|
||||
├── notstrap.toml # bootstrap hook/phase config
|
||||
├── crates/
|
||||
│ ├── notcore/ # shared types, config, paths, errors
|
||||
│ ├── notfiles/ # symlink engine (stow replacement)
|
||||
│ ├── notsecrets/ # age key retrieval + sops decrypt
|
||||
│ ├── nothooks/ # hook execution engine
|
||||
│ └── notstrap/ # new-machine bootstrap orchestrator
|
||||
└── tests/ # workspace-level integration tests
|
||||
```
|
||||
|
||||
### Crate Summary
|
||||
|
||||
| Crate | Type | Role |
|
||||
|-------|------|------|
|
||||
| `notcore` | lib | Shared types, config, paths, errors |
|
||||
| `notfiles` | lib + bin | Symlink engine (stow replacement) |
|
||||
| `notsecrets` | lib | Age key retrieval → sops decrypt |
|
||||
| `nothooks` | lib + bin | Hook execution, phase tracking |
|
||||
| `notstrap` | bin | Orchestrates everything on a new machine |
|
||||
|
||||
---
|
||||
|
||||
## Crate Designs
|
||||
|
||||
### `notcore` — Shared Foundation
|
||||
|
||||
Pure library. No binary. Every other crate depends on it; it depends on nothing in the workspace.
|
||||
|
||||
**Contents:**
|
||||
- `NotfilesError` — unified error enum via `thiserror`
|
||||
- `Config` — parses `notfiles.toml` + `notstrap.toml`
|
||||
- `Paths` — `expand_tilde`, `dotfiles_dir()`, standard path resolution
|
||||
- `HookSpec` — name, script path, phase (`dot` | `setup`)
|
||||
- `PackageSpec` — name, link method (`symlink` | `copy`), target dir
|
||||
- `Report` / `Step` — shared types for the end-of-run summary table
|
||||
|
||||
**Migration from current code:** `config.rs`, `paths.rs`, `error.rs` move from the current `notfiles/src/` into `notcore`.
|
||||
|
||||
---
|
||||
|
||||
### `notfiles` — Symlink Engine
|
||||
|
||||
Library + binary. The stow replacement. Operates on a dotfiles directory of "packages" (subdirectories) and symlinks their contents into a target (typically `$HOME`).
|
||||
|
||||
**Responsibilities:**
|
||||
- `link` — create symlinks or copies from package dirs to target
|
||||
- `unlink` — remove symlinks, clean up empty parent dirs
|
||||
- `status` — diff expected state (config + disk) vs actual state (symlinks + state file)
|
||||
- `State` — serialize/deserialize `.notfiles-state.toml` (tracks every linked file)
|
||||
- Conflict detection and `--force` backup handling
|
||||
- Glob-based ignore matching via `globset`
|
||||
|
||||
**Migration from current code:** `linker.rs`, `package.rs`, `ignore.rs`, `status.rs` stay in `notfiles`. Config/paths/error types are imported from `notcore`.
|
||||
|
||||
Used as a library by `notstrap` (`notfiles::link()` called directly).
|
||||
|
||||
---
|
||||
|
||||
### `notsecrets` — Secrets Bootstrap
|
||||
|
||||
Library. Retrieves the age private key and decrypts `secrets.sops.env` before anything else runs.
|
||||
|
||||
**Age key sources (tried in order):**
|
||||
|
||||
1. **Bitwarden CLI** (`bw`) — non-interactive if session is cached; prompts for master password if not
|
||||
2. **File** — `--key-file <path>` (USB drive, external storage)
|
||||
3. **Interactive prompt** — user pastes the age key directly
|
||||
|
||||
Implemented as a port: `AgeKeySource` trait with three implementations (`BitwardenSource`, `FileSource`, `PromptSource`). `notstrap` iterates sources until one succeeds.
|
||||
|
||||
Once the age key is in hand: writes it to `~/.config/sops/age/keys.txt`, then runs `sops --decrypt secrets.sops.env` to produce a live env file. The decrypted env contains all critical credentials: op, bw, github, openai, anthropic, etc.
|
||||
|
||||
1Password (`op`) is NOT used at this stage — it's installed later as a hook, after secrets are already available.
|
||||
|
||||
---
|
||||
|
||||
### `nothooks` — Hook Execution Engine
|
||||
|
||||
Library + binary. Replaces `run_hook`, `run_dot_hooks`, `run_setup_hooks` from `bootstrap.sh`.
|
||||
|
||||
**Hook phases:**
|
||||
- `dot` — fast, re-runnable (shell config, git config, AI tool configs)
|
||||
- `setup` — slow, run-once (package installs, language runtimes, dev tools)
|
||||
|
||||
**Responsibilities:**
|
||||
- Read hook specs from `notstrap.toml` — ordered list with name, script path, phase
|
||||
- Execute hooks in declared order, capturing stdout/stderr
|
||||
- Track which `setup` hooks have already run in `.nothooks-state.toml` — skip on re-run unless `--force`
|
||||
- Feed per-hook pass/skip/fail results into `Report` from `notcore`
|
||||
|
||||
**Does NOT handle:** package management (invoked as hooks), secret handling (done before hooks run).
|
||||
|
||||
CLI: `nothooks run --phase dot` or `nothooks run --phase setup`
|
||||
|
||||
---
|
||||
|
||||
### `notstrap` — New Machine Orchestrator
|
||||
|
||||
Binary only. The single entry point on a fresh machine.
|
||||
|
||||
**Install:**
|
||||
```bash
|
||||
cargo install notstrap
|
||||
notstrap run
|
||||
```
|
||||
|
||||
**Execution order:**
|
||||
|
||||
1. Check prerequisites (`op`, `bw`, `sops`, `age` — print exactly what's missing and stop)
|
||||
2. Run `notsecrets` — retrieve age key → decrypt `secrets.sops.env` → inject into env
|
||||
3. Clone dotfiles repo if not present (URL from `notstrap.toml`)
|
||||
4. Run `notfiles link` — stow all packages
|
||||
5. Run `nothooks run --phase dot` — fast hooks
|
||||
6. Run `nothooks run --phase setup` — slow hooks (includes `op` install + sign-in)
|
||||
7. Print `Report` summary table
|
||||
|
||||
---
|
||||
|
||||
## Day-One Flow
|
||||
|
||||
```
|
||||
cargo install notstrap
|
||||
notstrap run
|
||||
└─ notsecrets
|
||||
├─ try: bw unlock → age key
|
||||
├─ fallback: --key-file <path>
|
||||
└─ fallback: interactive prompt
|
||||
└─ sops decrypt secrets.sops.env → env injected
|
||||
└─ notfiles link (all packages)
|
||||
└─ nothooks --phase dot
|
||||
└─ shell config, git config, AI tool configs (~seconds)
|
||||
└─ nothooks --phase setup
|
||||
└─ Homebrew/Nix packages, mise runtimes, dev tools, op install (~minutes)
|
||||
└─ Report: ✓ linked 142 files, ✓ 3 dot hooks, ✓ 7 setup hooks
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Migration Strategy
|
||||
|
||||
Migration from `dotfiles/` is gradual — shell scripts die as their Rust equivalents ship:
|
||||
|
||||
| Phase | What ships | What it replaces |
|
||||
|-------|-----------|-----------------|
|
||||
| 1 | `notfiles` workspace (this design) | GNU Stow |
|
||||
| 2 | `notsecrets` + `notstrap` skeleton | `setup-secrets.sh`, `install.sh` |
|
||||
| 3 | `nothooks` | `bootstrap.sh` hook runner |
|
||||
| 4 | Hook-by-hook Rust rewrites | Individual `setup-*.sh` scripts |
|
||||
| 5 | `dotfiles/` repo archived | — |
|
||||
|
||||
Existing `.rs` scripts (`drift-check.rs`, `redact-audit.rs`, `claude-sessions.rs`, etc.) can migrate into `nothooks` hook scripts or dedicated crates as needed.
|
||||
|
||||
---
|
||||
|
||||
## Dependencies
|
||||
|
||||
| Crate | Key dependencies |
|
||||
|-------|-----------------|
|
||||
| `notcore` | `serde`, `toml`, `thiserror`, `dirs`, `anyhow` |
|
||||
| `notfiles` | `notcore`, `clap`, `globset`, `chrono` |
|
||||
| `notsecrets` | `notcore`, `clap`, `rpassword`, `which` |
|
||||
| `nothooks` | `notcore`, `clap`, `serde`, `toml` |
|
||||
| `notstrap` | `notcore`, `notfiles`, `notsecrets`, `nothooks`, `clap` |
|
||||
|
||||
---
|
||||
|
||||
## Open Questions
|
||||
|
||||
- Should `notstrap` be published to crates.io, or installed from the dotfiles repo directly via `cargo install --path`?
|
||||
- Should `notsecrets` support a fourth source: a second Bitwarden-compatible backend (Vaultwarden self-hosted)?
|
||||
- Hook scripts: keep as shell scripts invoked by `nothooks`, or migrate each to a Rust binary over time?
|
||||
Reference in New Issue
Block a user