v0.9.7 - Python Engine, Gitignore Anchor Fix, Engine Lifecycle & Security Hardening

Highlights

docmd v0.9.7 introduces the Python Engine, resolves a critical gitignore regression introduced in v0.9.6, unifies engine process lifecycle management, and ships security hardening improvements across the CLI and dev server.

• Python Engine (@docmd/engine-python): A brand new engine adapter utilizing Python 3 (>= 3.8) to accelerate file discovery, Git history parsing, and search indexing via a high-performance stdio worker pool.
• Unified Engine Lifecycle (@docmd/api, @docmd/core): Implemented standardized shutdown() and destroy() methods across all engines (js, rust, python) with automatic worker idle management and clean process exits.
• docmd-search 0.1.6 Integration: Updated search integration and plugin peer dependencies to require docmd-search >=0.1.6 with native Python engine support.
• Gitignore Anchored Pattern Fix (#244): Anchored patterns (e.g. /airo) are now matched relative to the project root, not the absolute filesystem path. Previously, if the pattern matched any ancestor folder name in the absolute path, every page was silently excluded and builds reported Generated 0 pages.
• MCP config.exclude Enforcement (#245): search_docs, list_docs, and validate_docs now honour config.exclude. Previously, excluded files (such as draft or archived pages) were still exposed to AI agents via MCP tools.
• Shell Injection Prevention: Replaced execSync shell interpolation with spawnSync and explicit argument arrays in doctor and search indexing to eliminate command injection vectors.
• Dev Server XSS & Path Traversal Hardening: URL reflection in 404/500 error pages is now HTML-escaped, and redirect targets are strictly validated to prevent reflected XSS and path traversal in docmd dev.
• Asset Path Canonicalisation: Live server asset resolution now enforces a canonical safe path boundary, preventing traversal outside the output directory.

Python Engine (@docmd/engine-python)

v0.9.7 brings Python into docmd’s multi-engine architecture alongside JavaScript and Rust:

• Auto-Detection & Zero-Config: If Python 3 is installed on your PATH, setting "engine": "python" in docmd.config.json automatically activates the Python engine. If missing, docmd falls back cleanly to the JS engine.
• Fast Persistent Worker: Operates via an optimized stdio JSON-RPC worker pool with idle cleanup, removing subprocess startup overhead across tasks.
• Plugin Integration: Directly powers Git log extraction, search indexing, and batch file scanning.

Engine Lifecycle & Process Management (@docmd/api, @docmd/core)

• Standardized Shutdown: Added shutdown() and destroy() hooks to the Engine interface across all engine implementations (@docmd/engine-js, @docmd/engine-rust, @docmd/engine-python).
• Resource Cleanup: Core build and dev pipelines now systematically call shutdownEngines() on completion to cleanly release subprocess pipes, memory, and native bindings.
• Idle Worker Pool: Python engine worker terminates automatically on idle, preventing orphaned background listeners.

Gitignore Anchored Pattern Regression (@docmd/core)

Bug fixed (#244): Since v0.9.6 introduced .gitignore awareness for page discovery, anchored patterns with a leading / were stripped of that slash and matched against the complete absolute path of each file. This meant that /airo (intended to ignore a binary at the repo root) would also match every file under /Users/someone/github/airo/docs/ because the absolute path contains the segment /airo/.

Root cause: isExcludedPath stripped the leading / and ran normalizedPath.includes('/airo/') against the full absolute path rather than the path relative to the .gitignore directory.

Fix: Anchored patterns are now matched only against the path relative to the top-level source directory passed into findFilesRecursive. Non-anchored patterns (no leading /) retain their existing anywhere-in-path matching behaviour. The findFilesRecursive function now tracks and propagates the original project root through all recursive calls.

Workaround that existed (no longer needed): Writing the pattern as /[a]iro (a bracket expression that git treats identically but the old matcher would not match literally).

Security Hardening (@docmd/core, @docmd/plugins-search, @docmd/live)

• doctor command: execSync replaced with spawnSync(['npm', 'doctor']) — arguments passed as an array with shell: false, removing the shell injection surface.
• Search indexing subprocess: Index spawn similarly hardened from a template-string shell command to an explicit argument array.
• Dev server (docmd dev): req.url reflected in 404 and 500 error HTML is now escaped via a minimal escapeHtml helper. Redirect targets are validated against a strict allowlist before being emitted as Location headers.
• Live server asset resolution: All asset paths are now resolved through canonicalSafePath, ensuring requests cannot traverse outside the designated output directory.

MCP config.exclude Enforcement (@docmd/core)

Bug fixed (#245): search_docs, list_docs, and validate_docs in the MCP server were ignoring config.exclude. The internal findMarkdownFiles() function only skipped node_modules and dot-directories — it never consulted the exclude patterns added in v0.9.6. This meant AI agents querying the MCP server could discover and read documentation that was deliberately excluded from the published site (e.g. archived specs, internal drafts).

Fix: search_docs and list_docs now use findFilesRecursive() (the same file walker used by the build pipeline), which already applies config.exclude, .gitignore, and all standard skip rules. validate_docs forwards config.exclude into the link-validator’s file walk. Build output and MCP tool results now agree on what counts as published documentation.

What to Expect in 0.9.8

• Package integrity check after pre-flight batch installs to detect half-extracted packages before they are falsely treated as resolved.
• Further improvements to the gitignore matching engine (negation patterns !, per-subdirectory .gitignore files).