docs: add integration tests design spec
Some checks failed
Public Repo Readiness / Check Public Repo Readiness (push) Failing after 4s
Some checks failed
Public Repo Readiness / Check Public Repo Readiness (push) Failing after 4s
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
186
docs/superpowers/specs/2026-04-01-integration-tests-design.md
Normal file
186
docs/superpowers/specs/2026-04-01-integration-tests-design.md
Normal file
@@ -0,0 +1,186 @@
|
|||||||
|
---
|
||||||
|
title: Integration Tests — notstrap + cross-crate
|
||||||
|
date: 2026-04-01
|
||||||
|
status: approved
|
||||||
|
---
|
||||||
|
|
||||||
|
# Integration Tests Design
|
||||||
|
|
||||||
|
## Goal
|
||||||
|
|
||||||
|
Validate the layered workspace architecture end-to-end: `notstrap` orchestrating `notfiles`, `nothooks`, and `notsecrets` against real temp directories with real Nu scripts and a `FileSource` age key. Catch regressions across crate boundaries that unit tests cannot.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Part 1: notstrap library extraction
|
||||||
|
|
||||||
|
### Problem
|
||||||
|
|
||||||
|
All orchestration logic lives in `crates/notstrap/src/main.rs`. There is no library surface — nothing to import in tests. `crates/notstrap/src/lib.rs` is a placeholder.
|
||||||
|
|
||||||
|
### Change
|
||||||
|
|
||||||
|
Move orchestration logic from `main()` into `lib.rs`. Public surface:
|
||||||
|
|
||||||
|
```rust
|
||||||
|
pub struct BootstrapOptions {
|
||||||
|
pub config: PathBuf,
|
||||||
|
pub force: bool,
|
||||||
|
pub key_file: Option<PathBuf>,
|
||||||
|
pub dotfiles: Option<PathBuf>,
|
||||||
|
/// Optional override for the sops decrypt step.
|
||||||
|
/// None = skip env injection (used in tests).
|
||||||
|
/// Some(f) = call f(sops_path) and inject the result into env.
|
||||||
|
pub env_injector: Option<Box<dyn Fn(&Path) -> anyhow::Result<String>>>,
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn run(opts: BootstrapOptions) -> anyhow::Result<Report>
|
||||||
|
```
|
||||||
|
|
||||||
|
`main()` becomes:
|
||||||
|
|
||||||
|
```rust
|
||||||
|
fn main() -> Result<()> {
|
||||||
|
let cli = Cli::parse();
|
||||||
|
let Cmd::Run { config, force, key_file, dotfiles } = cli.command;
|
||||||
|
let opts = BootstrapOptions {
|
||||||
|
config, force, key_file, dotfiles,
|
||||||
|
env_injector: Some(Box::new(|p| notsecrets::decrypt_sops(p))),
|
||||||
|
};
|
||||||
|
let report = notstrap::run(opts)?;
|
||||||
|
if report.has_failures() { std::process::exit(1); }
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
`NotstrapConfig`, `BootstrapSection`, `prereqs`, and `repo` modules stay in `notstrap` (not moved to notcore — they are bootstrap-specific).
|
||||||
|
|
||||||
|
The `[lib]` section in `crates/notstrap/Cargo.toml` stays; the placeholder comment in `lib.rs` is replaced with real code.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Part 2: Workspace-level integration test crate
|
||||||
|
|
||||||
|
### Location
|
||||||
|
|
||||||
|
```
|
||||||
|
tests/
|
||||||
|
└── integration/
|
||||||
|
├── Cargo.toml
|
||||||
|
└── tests/
|
||||||
|
├── bootstrap.rs
|
||||||
|
└── cross_crate.rs
|
||||||
|
```
|
||||||
|
|
||||||
|
Not under `crates/` — this is a test-only crate at the workspace root level. Added to `members` in root `Cargo.toml`.
|
||||||
|
|
||||||
|
### Cargo.toml
|
||||||
|
|
||||||
|
```toml
|
||||||
|
[package]
|
||||||
|
name = "integration"
|
||||||
|
version = "0.1.0"
|
||||||
|
edition = "2024"
|
||||||
|
license.workspace = true
|
||||||
|
publish = false
|
||||||
|
|
||||||
|
[dev-dependencies]
|
||||||
|
notstrap = { path = "../../crates/notstrap" }
|
||||||
|
notfiles = { path = "../../crates/notfiles" }
|
||||||
|
nothooks = { path = "../../crates/nothooks" }
|
||||||
|
notsecrets = { path = "../../crates/notsecrets" }
|
||||||
|
notcore = { path = "../../crates/notcore" }
|
||||||
|
tempfile = { workspace = true }
|
||||||
|
assert_fs = { workspace = true }
|
||||||
|
anyhow = { workspace = true }
|
||||||
|
```
|
||||||
|
|
||||||
|
No `[lib]` or `[[bin]]` — test-only crate. Run with `cargo nextest run -p integration`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Part 3: Test scenarios
|
||||||
|
|
||||||
|
### `tests/bootstrap.rs` — full flow via `notstrap::run()`
|
||||||
|
|
||||||
|
All tests use a helper `fn make_dotfiles_dir() -> TempDir` that creates:
|
||||||
|
- A valid `notfiles.toml` (symlink method, target = tempdir home)
|
||||||
|
- A `notstrap.toml` with a fake `dotfiles_repo` and the tempdir as `dotfiles_dir`
|
||||||
|
- A `scripts/` directory with `.nu` hook scripts
|
||||||
|
- A `age.key` file with a fake key string
|
||||||
|
|
||||||
|
**Test 1: `test_full_bootstrap_dot_hooks_only`**
|
||||||
|
|
||||||
|
Setup: dotfiles dir with one dot hook (`print hello`) and one file to link. Pass `key_file` pointing to `age.key`, `env_injector: None`.
|
||||||
|
|
||||||
|
Assert:
|
||||||
|
- `Report` has no failures
|
||||||
|
- The dot hook ran (hook state not written — dot hooks don't track)
|
||||||
|
- The target file is symlinked into the temp home dir
|
||||||
|
|
||||||
|
**Test 2: `test_setup_hooks_skipped_on_rerun`**
|
||||||
|
|
||||||
|
Setup: dotfiles dir with one setup hook. Run bootstrap twice with same `dotfiles_dir`.
|
||||||
|
|
||||||
|
Assert:
|
||||||
|
- First run: setup hook `StepStatus::Ok`
|
||||||
|
- Second run: setup hook `StepStatus::Skipped`
|
||||||
|
- `.nothooks-state.toml` exists after first run
|
||||||
|
|
||||||
|
**Test 3: `test_bootstrap_fails_fast_on_bad_key`**
|
||||||
|
|
||||||
|
Setup: `key_file` points to a non-existent path.
|
||||||
|
|
||||||
|
Assert:
|
||||||
|
- `Report` has a failure on the `"age key"` step
|
||||||
|
- No files linked (bootstrap stopped before step 6)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### `tests/cross_crate.rs` — crate boundary tests
|
||||||
|
|
||||||
|
**Test 4: `test_nothooks_notsecrets_independent`**
|
||||||
|
|
||||||
|
Exercise the `notsecrets → nothooks` boundary without going through notstrap.
|
||||||
|
|
||||||
|
Setup:
|
||||||
|
- Write an age key to a tempfile via `FileSource`
|
||||||
|
- Resolve it with `resolve_age_key()`
|
||||||
|
- Write a `.nu` hook script that checks `$env.TEST_KEY` is set (set manually in test env)
|
||||||
|
- Run it via `HookRunner`
|
||||||
|
|
||||||
|
Assert: `HookResult::Ok`
|
||||||
|
|
||||||
|
**Test 5: `test_notfiles_respects_default_ignore`**
|
||||||
|
|
||||||
|
Exercise the `notcore` ignore list fix (from audit finding).
|
||||||
|
|
||||||
|
Setup:
|
||||||
|
- Create a package dir containing `foo.txt`, `.notfiles-state.toml`, `.nothooks-state.toml`
|
||||||
|
- Run `notfiles::link()` targeting a temp home dir
|
||||||
|
|
||||||
|
Assert:
|
||||||
|
- `foo.txt` is linked
|
||||||
|
- `.notfiles-state.toml` is NOT linked
|
||||||
|
- `.nothooks-state.toml` is NOT linked
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Running tests
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Run integration tests only
|
||||||
|
cargo nextest run -p integration
|
||||||
|
|
||||||
|
# Run all workspace tests
|
||||||
|
cargo nextest run --workspace
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Non-goals
|
||||||
|
|
||||||
|
- No sops binary required — `env_injector: None` in all tests
|
||||||
|
- No Bitwarden CLI required — all tests use `FileSource`
|
||||||
|
- No network access
|
||||||
|
- No testing of `notstrap`'s CLI arg parsing (that's thin and not worth testing)
|
||||||
Reference in New Issue
Block a user