ADR-009: Python Plugin SDK via Embedded PyO3¶

Status: Accepted — plugin-trust position superseded by ADR-018 (2026-08-15) Date: 2026-05-18 Deciders: Mohsen Seyedkazemi Ardebili

Read this first. The decision below that "plugins run in the daemon process and are trusted (no sandboxing in v1.0)" was made for a Rust core where plugin support sits behind a python-plugins cargo feature — a build-time gate, so a build without it cannot load foreign code at all. That core was never built; YazSes shipped as a Python daemon with no such gate, where any plugin mechanism would be live for every install. ADR-018 reverses this and declines third-party plug-in loading, with the reasoning and with what would change it. Everything else here — the plugin type taxonomy, the PyO3 measurements — stands as the record of that design.

Context¶

ADR-001 commits the core to Rust, but the plugin ecosystem benefits from Python: community familiarity with Whisper, Pydantic AI, and sentence-transformers; a lower contributor barrier than Rust; and mature Python clients for many editor bridge integrations (e.g., pynvim). Some deployments — enterprise or regulated-industry environments — prefer to run without any Python dependency. The design must serve both: a Python-capable build for plugin authors, and a Python-free build for users who want a minimal footprint.

Spike S-6 (2026-05-18) measured PyO3 call overhead at 0.6 µs P50 and P90 in a release build on an i7-1370P — 1,600× under the 1 ms per-call budget. Hot-reload via importlib.reload() was confirmed to work correctly.

Decision¶

A Python plugin SDK is provided via embedded PyO3, gated behind a python-plugins cargo feature. If no plugins are installed or the binary was built without the feature, YazSes operates Python-free.

Plugins are Python packages installed to ~/.local/share/yazses/plugins/ via yazses plugin install <name> (a thin wrapper around pip install --target). Each plugin declares a yazses_plugin.json manifest with its entry points; the daemon scans and imports these at startup. Plugins run in the daemon process and are trusted (no sandboxing in v1.0; v2 will add restricted sub-interpreters).

v1.0 plugin Protocol types: - EditorBridge plugins — community editor bridges beyond the Neovim and VS Code implementations that ship in-core. - ToolGrammar plugins — domain-specific extensions to the core ≤ 20-tool registry.

Reserved for v2+: LoRAAdapter, InputBackend, and MemoryPostProcessor plugin types.

Required Cargo dependency: pyo3 = { version = "0.23", features = ["auto-initialize"] }. Production call pattern per utterance:

Python::with_gil(|py| {
    registry.editor_bridge.call_method0("get_context")?.extract()
})

Consequences¶

Positive: - Community contributors who know Python can extend YazSes without learning Rust. - The editor-bridge and tool-grammar plugin ecosystems can grow faster than Rust-only would allow. - The Rust core stays minimal and fast for users who do not install plugins. - Existing v0.4 Python code (e.g., commands/grammar.py extensions) can be repackaged as v1.0 plugins, softening the migration.

Negative / trade-offs: - Embedded Python adds 15–25 MB to binaries built with the python-plugins feature. - Each loaded Python plugin adds approximately 20–50 MB RSS; users with many plugins should expect higher memory use. - No plugin sandboxing in v1.0 — a malicious plugin has daemon-level access. Mitigation: plugin manifests declare capabilities; users review before installing. - Cross-compiling with an embedded Python interpreter across all five target triples requires careful CI configuration.

Implementation¶

The python-plugins feature is declared in yazses-plugins-python/Cargo.toml. The plugin loader is in yazses-plugins-python/src/loader.rs. The yazses-plugin-sdk Python package (published to PyPI) defines the Protocol base classes that plugin authors depend on. The reference plugin is NeovimBridge, located in plugins/neovim-bridge/.