meta_package_manager.execution module

CLI-execution engine shared by every package manager.

Runs one manager’s CLI in one subprocess: the meta_package_manager.execution.CLIExecutor mixin (which meta_package_manager.manager.PackageManager inherits) locates the binary and runs it, the meta_package_manager.execution.CLIError exception carries a failed call’s result, and meta_package_manager.execution.highlight_cli_name() themes a binary’s name.

Scheduling many managers at once is the next altitude up, and lives in meta_package_manager.dispatch: the concurrent fan-out primitives, the lock families and the shared ✓/✗ trail. The sudo machinery that cuts across both altitudes (credential priming, the keepalive, the hidden-prompt stall watchdog) lives in meta_package_manager.sudo: this module only consumes it, to wrap escalated commands and diagnose their failures.

Note

The name and intent mirror click_extra.execution from the sibling click-extra project, where the generic layers now live: the concurrency primitives (run_jobs/run_lanes driven by mpm --jobs), the single-subprocess engine (click_extra.execution.run_cli(), which disclosed invocations and streams output to the logs), and the Ctrl+C machinery (click_extra.execution.install_interrupt_handler() terminating the in-flight children registered by run_cli). This module keeps what is package-manager policy: per-operation timeouts, sudo escalation, cooldown enforcement and dry-run.

meta_package_manager.execution.DIAGNOSIS_TAIL_LINES: Final = 10

Trailing lines of a failed command’s report relayed at WARNING.

CLIs conclude with their actual error, so the tail is where the diagnosis lives; the cap keeps a verbose failure (a source build’s compiler spew) from flooding the default view. The raw streams are always available in full, live, at DEBUG.

meta_package_manager.execution.WIN_DEFAULT_PATHEXT: Final = '.COM;.EXE;.BAT;.CMD;.VBS;.JS;.WS;.MSC'

Executable suffixes to search on Windows when %PATHEXT% is unset.

A copy of CPython’s own shutil._WIN_DEFAULT_PATHEXT, which CLIExecutor.search_all_cli() reads to mirror shutil.which()’s behaviour. That attribute is private, so it can be renamed or dropped by any release without notice, and reading it unguarded made every manager detection on Windows one AttributeError away from failing. This constant is the fallback; test_windows_pathext_tracks_cpython compares the two on every platform and every interpreter the matrix covers, so a divergence surfaces as a named test failure rather than as a Windows-only crash.

exception meta_package_manager.execution.CLIError(code, output, error)

Bases: Exception

An error occurred when running package manager CLI.

The exception internally keeps the result of CLI execution.

property diagnosis: str

The command’s own account of its failure, capped for log relay.

Prefers <stderr>, the conventional stream for error reporting, and falls back on <stdout> for the tools that report failures there (steamcmd); a command that died silently is reduced to its exit code. Only the last DIAGNOSIS_TAIL_LINES lines are kept, behind a counter of the truncated ones: errors conclude streams.

meta_package_manager.execution.VERSION_PROBE: Final = 'version'

Pseudo-operation stamped on CLIExecutor._active_operation during version detection.

Not a member of meta_package_manager.capabilities.Operations (no subcommand routes it), but it participates in the same per-operation machinery: OPERATION_TIMEOUTS binds it to the short read-only cap, and CLIExecutor.run() demotes its command disclosure to DEBUG so the per-candidate probes cannot drown the INFO narration.

meta_package_manager.execution.format_plan_command(cmd_args, extra_env=None)

Render a captured mpm --plan command as a copy-pasteable shell line.

Unlike click_extra.execution.format_cli_prompt() (styled, and prefixed with a $ prompt sigil for logs and dry-runs), this returns a plain, unstyled, shell-quoted line: the forced environment assignments followed by the resolved binary and its arguments, ready to paste into a terminal or pipe into a shell. See the plan-mode branch of CLIExecutor.run().

Return type:

str

meta_package_manager.execution.PLAN_RECORDER: Final = <meta_package_manager.execution._PlanRecorder object>

Process-wide sink for CLIExecutor.run()’s plan-mode captures.

A module-level singleton because run executes in the fan-out’s worker threads, where the click context is not reliably reachable. See _PlanRecorder.

meta_package_manager.execution.highlight_cli_name(path, match_names)

Highlight the binary name in the provided path.

The name is only highlighted when it matches one of the recognized match_names, so an unrecognized binary stays plain. Matching is insensitive to case on Windows and case-sensitive on other platforms, thanks to os.path.normcase.

The rendering is delegated to click_extra.execution.highlight_bin_name(), the same helper behind the $-prompt and spawn-trace log lines, so the mpm managers table and the logs can never drift apart.

Return type:

str | None

meta_package_manager.execution.READ_ONLY_TIMEOUT: Final = 120

Default timeout (seconds) for read-only probes and queries.

These operations only inspect state, so a short cap lets a wedged binary fail fast instead of stalling the whole run. The value is generous enough for legitimately slow scans (a freshly-pulled guix search walking every package’s metadata) while still being far below MUTATING_TIMEOUT.

meta_package_manager.execution.MUTATING_TIMEOUT: Final = 500

Default timeout (seconds) for operations that change system state.

Installs, upgrades, removals, channel syncs and cleanups routinely build from source, download large archives or pull entire channels, so they need a long cap. Kept identical to the historical global default so these operations behave exactly as before when no explicit --timeout is given.

meta_package_manager.execution.DEFAULT_TIMEOUT: Final = 500

Fallback timeout (seconds) for a CLI call whose operation is unknown.

Defaults to the conservative MUTATING_TIMEOUT: when in doubt, wait rather than risk killing a legitimate long-running command.

meta_package_manager.execution.OPERATION_TIMEOUTS: Final[dict[str, int]] = {'cleanup': 500, 'doctor': 500, 'install': 500, 'installed': 120, 'orphans': 120, 'outdated': 120, 'remove': 500, 'search': 120, 'sync': 500, 'upgrade': 500, 'upgrade_all': 500, 'version': 120}

Per-operation timeout defaults, applied only when the user has set no explicit --timeout (or per-manager timeout override).

Keyed by the meta_package_manager.capabilities.Operations member name, plus the special "version" detection probe. The keys are validated against the Operations enum by the test suite so the two never drift apart. An operation absent from this map resolves to DEFAULT_TIMEOUT.

meta_package_manager.execution.SPINNER_DELAY: Final = 0.1

Seconds a CLI call must run before its progress spinner appears.

Kept short so the spinner surfaces almost immediately on any call that is not instant: prompt feedback makes mpm feel responsive from the start rather than stalled during the first second. Only the quickest calls (cached version probes, trivial metadata queries) finish within this delay and stay silent; anything slower (a guix search, a source build) shows the spinner right away.

class meta_package_manager.execution.CLIExecutor

Bases: object

Locate a manager’s CLI on the system and run it.

Mixin inherited by meta_package_manager.manager.PackageManager. Owns the CLI-invocation configuration (names, search paths, environment, arguments, timeout) and the engine that searches for the binary, executes it, captures and normalizes its output, accumulates errors, and parses its self-reported version.

Initialize cli_errors list.

cli_names: tuple[str, ...]

List of CLI names the package manager is known as.

This list of recognized CLI names is ordered by priority. That way we can influence the search of the right binary.

..hint::

This was helpful in the case of the Python transition from 2.x to 3.x, where multiple versions of the same executable were named python or python3.

By default, this property’s value is derived from the manager’s ID (see the MetaPackageManager.__init__ method above).

cli_search_path: tuple[str, ...] = ()

List of additional path to help mpm hunt down the package manager CLI.

Must be a list of strings whose order dictates the search sequence.

Most of the time unnecessary: meta_package_manager.execution.CLIExecutor.cli_path works well on all platforms.

extra_env: ClassVar[Mapping[str, str | None] | None] = None

Additional environment variables to add to the current context.

Automatically applied on each meta_package_manager.execution.CLIExecutor.run_cli() calls.

pre_cmds: tuple[str, ...] = ()

Global list of pre-commands to add before before invoked CLI.

Automatically added to each meta_package_manager.execution.CLIExecutor.run_cli() call.

Used to prepend sudo or other system utilities.

pre_args: tuple[str, ...] = ()

post_args: tuple[str, ...] = ()

Global list of options used before and after the invoked package manager CLI.

Automatically added to each meta_package_manager.execution.CLIExecutor.run_cli() call.

Essentially used to force silencing, low verbosity or no-color output.

version_cli_options: tuple[str, ...] = ('--version',)

CLI options used to produce the version of the package manager.

The raw output produced by the package manager CLI will be parsed with the version_regexes below to extract the version number.

version_cli: str | None = None

Alternate binary probed for the manager’s version, instead of the main CLI.

Some manager suites expose no version flag on any of their own binaries (OpenBSD’s pkg_add/pkg_info, Solaris’ pkgadd/pkginfo): they ship with the base system and are versioned with the OS itself. Naming a version_cli (like uname) makes the version probe run that binary with version_cli_options and parse its output with version_regexes, while every operation keeps using the manager’s own cli_path. The binary is resolved with which(); the version resolves to None (manager not fresh) when it is not found.

version_regexes: tuple[str, ...] = ('(?P<version>\\S+)',)

Regular expressions used to extract the version number.

This property must be a tuple of strings, each of which is a valid regular expression that must contain a group named <version>.

The first of these regexes producing a match and returning non-empty <version> group will be used as the version string of the package manager.

That version string will then be sanitized and normalized by meta_package_manager.execution.CLIExecutor.version.

By default match the first part that is space-separated.

Caution

These regexes are compiled with re.MULTILINE only. They are not compiled with re.VERBOSE, so literal whitespace in the pattern is significant and matches whitespace in the CLI output.

stop_on_error: bool = False

Tell the manager to either raise or continue on errors.

dry_run: bool = False

Do not actually perform any action, just simulate CLI calls.

plan: bool = False

Capture state-changing CLI calls for inspection instead of running them.

Set by mpm --plan. Unlike dry_run (which simulates every call, read-only queries included), plan mode lets the read-only queries (installed, outdated, search) run for real so the resolved plan reflects actual system state, and records only the state-changing commands (see _MUTATING_OPERATIONS) into PLAN_RECORDER.

timeout: int | None = None

Maximum number of seconds to wait for a CLI call to complete.

None means the user expressed no explicit preference: the effective cap is then resolved per-operation by _resolve_timeout() from OPERATION_TIMEOUTS. A non-None value (the --timeout flag or a per-manager override) wins for every operation.

progress: bool = False

Whether CLI calls may show a progress spinner while they block.

Set by the CLI to an interactive, human-facing run only (a TTY, no serialized output, not at DEBUG verbosity). Even when True the spinner still self-suppresses off a TTY: see _make_spinner(). Defaults to False so programmatic use stays silent.

cooldown: timedelta | None = None

Minimum age a release must have before it can be installed or upgraded.

When set, the manager refuses to bring in any package version published more recently than cooldown ago. This is a mitigation against supply-chain attacks: a malicious release is typically detected and pulled within days of publication, so a waiting period keeps freshly-published (and potentially compromised) versions out of the system. None disables the gate.

Only managers able to enforce a release-age limit honor this, natively through cooldown_env_var or through the per-package release-date probe of release_date(); see supports_cooldown.

cooldown_policy: CooldownPolicy | None = None

Per-manager enforcement posture of an active release-age cooldown.

None (the default) inherits the policy resolved from --cooldown and the [mpm.cooldown] configuration, which the pool’s selection applies to every manager per run. Pin it explicitly through a [mpm.managers.<id>] override, which keeps precedence over the global resolution: enforce holds the manager fail-closed even under a best-effort run, best-effort waives the requirement and runs it without the safeguard, and off exempts it from the gate entirely, suppressing the cutoff injection even where the manager could honor one.

sudo: bool | None = None

User escalation policy: run this manager’s privileged commands with sudo.

None (the default) means the user expressed no preference, so the built-in default_sudo decides. True/False force escalation on or off for every operation this manager marks privileged (a build_cli(..., sudo=True) call). Set globally by mpm --sudo / mpm --no-sudo and per manager by the [mpm.managers.<id>] sudo config key, the latter winning (see meta_package_manager.pool.ManagerPool._select_managers()).

Only privileged operations on UNIX are ever escalated. A manager that escalates internally (internal_sudo) has no such markers and is never wrapped in sudo by mpm: its own sudo reuses the credential cache when prime_sudo() finds it already warm, and is otherwise covered by the silent-call notice in run().

default_sudo: bool = False

Built-in escalation default, used when sudo is None.

False on the base: most managers install into user-writable trees and never need root. The system package managers whose privileged operations require root (apt, dnf, pacman, zypper, …) set this to True so their build_cli(..., sudo=True) operations escalate out of the box, while staying switchable off through sudo (--no-sudo or config) for rootless setups.

internal_sudo: bool = False

Marks a manager whose CLI invokes sudo itself mid-run.

Homebrew cask runs it from installer artifacts, fink re-execs its root commands through it, and the AUR helpers call sudo pacman for their install steps. mpm never wraps such a manager’s commands: either none of its operations carry a build_cli(..., sudo=True) marker (cask, fink), or its default_sudo = False policy leaves the markers it inherits unescalated (the AUR helpers). Running the tool under sudo is often forbidden outright (brew refuses root, makepkg refuses to build). Consumed by prime_sudo(), whose opportunistic probe keeps an already-warm credential cache alive for these internal escalations, and by the silent-call notice in run(), which flags a possibly-hidden password prompt on a cold cache.

Forcing sudo = true on such a manager (config key or --sudo) still never wraps its commands, but does promote it into the up-front prompt path of prime_sudo().

cooldown_env_var: ClassVar[str | None] = None

Environment variable this manager reads to honor a cooldown.

None (the default) means the manager has no native release-age mechanism to feed through the environment; it may still honor a cooldown through the per-package release-date probe (see _probes_release_date()). A subclass that sets this string advertises native support (see supports_cooldown); the value produced by cooldown_env_value() is then injected into the environment of every CLI call.

windows_creation_flags: int = 0

Additional Windows process creation flags OR-ed with CREATE_NO_WINDOW.

Use this on individual managers to control how their subprocess is attached to the calling process’s console. For example, setting this to subprocess.DETACHED_PROCESS (0x8) fully detaches the child from the parent’s console. Any grandchild process (like a COM server or installer EXE) that calls GenerateConsoleCtrlEvent(0) on exit will then fail silently because there is no console to broadcast to.

No-op on non-Windows platforms (getattr returns 0 for Windows-only flags).

windows_processes_to_cleanup: tuple[str, ...] = ()

Windows process image names to forcibly terminate after each CLI call.

When a package manager spawns grandchild processes that outlive the direct subprocess (like winget’s WindowsPackageManagerServer.exe COM server), those orphans can linger and consume resources. List the image names here so they are killed after communicate() returns.

No-op on non-Windows platforms.

run_cache: dict[tuple, tuple[int, str, str]] | None = None

Optional cache that de-duplicates identical CLI runs across a lane’s managers.

None by default, which disables caching: every run() call spawns its own subprocess. Two callers install a shared dict, each for the duration of one lane:

• meta_package_manager.dispatch.dispatch(), on every multi-manager lock-family lane (see meta_package_manager.dispatch.SHARED_LOCK_FAMILIES), so brew and cask both running brew update for mpm sync spawn it once.

• meta_package_manager.dispatch.warm_availability(), on every version-probe lane (see meta_package_manager.dispatch.merge_into_probe_lanes()), so brew and cask both probing brew --version spawn it once too.

The replay still walks run()’s logging and failure gate, so a failed shared command is attributed to every member. Keyed on the resolved command line and its environment, so only genuinely identical invocations collapse.

Caution

Not thread-safe, and it does not need to be: both callers bind a cache to a lane, and click_extra.execution.run_lanes() runs a lane’s items serially on a single worker. Handing one dict to managers that run concurrently would race two peers into spawning the same command anyway, losing the de-duplication rather than corrupting anything, so the lane is what makes the cache work at all.

cli_errors: list[CLIError]

Accumulate all CLI errors encountered by the package manager.

Every CLIError produced by run() lands here, whether or not it is also raised: a failure the caller goes on to swallow (installed_or_empty() and its peers) is still a failure this manager committed, and the end-of-run summary, the serialized errors payload and the ✓/✗ trail all read this list to say so.

Recording only the non-raising half made a manager’s visibility hinge on whether its query happened to pass must_succeed: mpm list printed “Could not list installed packages.” and still scored the manager ✓, leaving it out of the closing count.

property supports_cooldown: bool

Whether this manager can enforce a release-age cooldown.

Either natively, by injecting cooldown_env_var into every CLI call, or through the per-package release-date probe advertised by _probes_release_date().

cooldown_env_value()

Render cooldown as the value of cooldown_env_var.

Defaults to the RFC 3339 timestamp of the most recent release date still allowed, i.e. now minus the cooldown. Managers whose environment variable expects another format (a number of minutes, a bare day count, …) override this.

Return type:

str

cooldown_rounded_up(unit_seconds)

Render cooldown as an integer count of unit_seconds-long units, rounded up.

Helper for the cooldown_env_value() overrides of managers whose native release-age knob expects a unit count rather than the default RFC 3339 timestamp (npm’s day-based min-release-age, pnpm’s minute-based minimumReleaseAge). Sub-unit cooldowns round up so the gate over-protects rather than silently collapsing to 0 (the “no cooldown” sentinel).

Return type:

str

cooldown_env()

Environment fragment enforcing the cooldown, empty when inactive.

Returns an empty mapping unless a cooldown is set and the manager supports it and its cooldown_policy has not exempted it (off). Merged into the environment of every run() call.

Return type:

Mapping[str, str | None]

search_all_cli(cli_names, env=None)

Search for all binary files matching the CLI names, in all environment path.

This is like our own implementation of shutil.which(), with the difference that it is capable of returning all the possible paths of the provided file names, in all environment path, not just the first one that match. And on Windows, prevents matching of CLI in the current directory, which takes precedence on other paths.

Returns all files matching any cli_names, by iterating over all folders in this order:

• folders provided by cli_search_path,

• then in all the default places specified by the environment variable (i.e. os.getenv("PATH")).

Only returns files that exists and are not empty.

Caution

Symlinks are not resolved, because some manager like Homebrew on Linux relies on some sort of symlink-based trickery to set environment variables.

Return type:

Generator[Path, None, None]

which(cli_name)

Emulates the which command.

Based on the search_all_cli() method.

Return type:

Path | None

sibling_cli(name, *, same_dir=False)

Resolve the path of a sibling binary of the manager’s main CLI.

Some managers ship as a suite of binaries (xbps-install/xbps-query, pkg_add/pkg_info, emerge’s qlist): an operation then runs a sibling instead of the main CLI. By default the sibling is searched like the main CLI itself (which(), honoring cli_search_path), and a missing binary raises FileNotFoundError rather than silently falling back to the wrong program.

same_dir=True instead takes the sibling from the directory of cli_path, without an existence probe: suites installing all their binaries side by side (XBPS, Nix) guarantee the neighbor, and resolving it from the same directory can never mix two installations. A genuinely missing file then surfaces at spawn time.

Return type:

Path

property cli_path: Path | None

Fully qualified path to the canonical package manager binary.

Try each CLI names provided by cli_names, in each system path provided by cli_search_path. In that order. Then returns the first match.

Executability of the CLI will be separately assessed later by the meta_package_manager.execution.CLIExecutor.executable property below.

property version: TokenizedString | None

Invoke the manager and extract its own reported version string.

Returns a parsed and normalized version in the form of a meta_package_manager.version.TokenizedString instance.

Skipped on platforms where the manager is not supported, even if cli_path resolved to an executable: that binary almost certainly belongs to a different tool that happens to share the same name (e.g. GNU make on macOS getting matched by the FreeBSD ports manager), so probing it would either misreport the version or surface confusing error output.

property executable: bool

Is the package manager CLI can be executed by the current user?

acting_as(operation=None, *, stop_on_error=None)

Temporarily adjust the manager’s execution state, restoring it on exit.

operation re-stamps _active_operation (the per-operation timeout and watchdog key) for the duration of the block; None leaves the current stamp untouched. stop_on_error likewise overrides the failure policy when set: the per-package state changers run their action under stop_on_error=True so a botched operation raises and is recorded by the caller instead of being silently accumulated.

The public seam for callers needing a scoped state override: the CLI layer must never poke _active_operation or stop_on_error directly.

Return type:

Iterator[None]

run(*args, extra_env=None, must_succeed=False)

Run a shell command, return the output and accumulate error messages.

args is allowed to be a nested structure of iterables, in which case it will be recursively flatten, then None will be discarded, and finally each item casted to strings.

Running commands with that method takes care of:

• disclosing the invocation at INFO (the reproducible $-prompt line with forced environment variables) and streaming the raw output live to DEBUG, prefixed with the manager ID, via click_extra.execution.run_cli()

• flagging, on a terminal, the mutating call of an internal escalator that goes silent on a cold credential cache and may be blocked on a hidden password prompt (see _StallWatchdog)

• detaching every other call into its own POSIX session and process group, so a timeout or Ctrl+C reaps the whole process tree and a wedged grandchild cannot linger as an orphan; the flagged call above keeps the controlling terminal so its sudo prompt stays answerable

• removing ANSI escape codes from subprocess.CompletedProcess.stdout and subprocess.CompletedProcess.stderr

• returning ready-to-use normalized strings (dedented and stripped)

• letting mpm --dry-run and mpm --stop-on-error have expected effect on execution

Parameters:

must_succeed (bool) – if True, raise meta_package_manager.execution.CLIError when the command fails, regardless of the user-facing stop_on_error preference, rather than accumulating the error for an end-of-run summary. Use for calls whose output is parsed (JSON, XML, regex), where a swallowed failure would be indistinguishable from empty results. A non-zero exit that leaves <stderr> empty is tolerated as a benign status code (npm and pnpm outdated exit 1 when updates exist); only the per-package state changers, which run under a patched stop_on_error, treat every non-zero exit as a failure. See the failure gate below for details.

Return type:

str

build_cli(*args, auto_pre_cmds=True, auto_pre_args=True, auto_post_args=True, override_pre_cmds=None, override_cli_path=None, override_pre_args=None, override_post_args=None, sudo=False)

Build the package manager CLI by combining the custom *args with the package manager’s global parameters.

Returns a tuple of strings.

Helps the construction of CLI’s repeating patterns and makes the code easier to read. Just pass the specific *args and the full CLI string will be composed out of the globals, following this schema:

$ [<pre_cmds>|sudo --non-interactive] <cli_path> <pre_args> <*args> <post_args>

Return type:

tuple[str, ...]

• self.pre_cmds is added before the CLI path.

• self.cli_path is used as the main binary to execute.

• self.pre_args and self.post_args globals are added before and after the provided *args.

Each additional set of elements can be disabled with their respective flag:

• auto_pre_cmds=False to skip the automatic addition of self.pre_cmds

• auto_pre_args=False to skip the automatic addition of self.pre_args

• auto_post_args=False to skip the automatic addition of self.post_args

Each global set of elements can be locally overridden with:

• override_pre_cmds=tuple()

• override_cli_path=str

• override_pre_args=tuple()

• override_post_args=tuple()

On UNIX, an operation marked privileged (sudo=True) is escalated only when the per-manager policy opts in (sudo, falling back to default_sudo). It is then run through sudo with --non-interactive (it spends the credential cache warmed by prime_sudo() and fails fast rather than blocking on a password prompt). When escalation applies, override_pre_cmds is not allowed to be set and auto_pre_cmds is forced to False. A non-UNIX host never escalates.

run_cli(*args, auto_extra_env=True, auto_pre_cmds=True, auto_pre_args=True, auto_post_args=True, override_extra_env=None, override_pre_cmds=None, override_cli_path=None, override_pre_args=None, override_post_args=None, force_exec=False, must_succeed=False, sudo=False)

Build and run the package manager CLI by combining the custom *args with the package manager’s global parameters.

After the CLI is built with the meta_package_manager.execution.CLIExecutor.build_cli() method, it is executed with the meta_package_manager.execution.CLIExecutor.run() method, augmented with environment variables from self.extra_env.

All parameters are the same as meta_package_manager.execution.CLIExecutor.build_cli(), plus:

• auto_extra_env=False to skip the automatic addition of self.extra_env

• override_extra_env=dict() to locally overrides the later

• force_exec ignores the mpm --dry-run, mpm --stop-on-error and mpm --plan options to force the execution and completion of the command. It is used for reads whose output is needed regardless (version detection, yarn global dir), which must run for real even when the user asked to simulate or to only plan mutations.

• must_succeed raises on non-zero exit regardless of mpm --stop-on-error. See run() for details.

Return type:

str