Lifeguard: A static analyzer for Python lazy imports compatibility

本文为原文前 6,000 字符的节选翻译,完整内容请查看原文。

Lifeguard: A static analyzer for Python lazy imports compatibility

Lifeguard for Lazy Imports A fast static analysis tool to aid adoption of Lazy Imports in Python. What are Lazy Imports? In Python, every import statement executes immediately when a module is loaded. This overhead is incurred regardless of whether that import is actually used. PEP 810 introduces explicit Lazy Imports to Python, which defer the actual loading of a module until the imported name is first accessed.

Lifeguard 是一个用于 Python 延迟导入(Lazy Imports)的快速静态分析工具,旨在帮助开发者在项目中采用延迟导入。什么是延迟导入?在 Python 中,每个 import 语句在模块加载时都会立即执行。无论该导入是否被实际使用,这种开销都会产生。PEP 810 为 Python 引入了显式的延迟导入,将模块的实际加载推迟到首次访问导入名称时。

Lazy Imports can significantly reduce memory usage, startup times, and import overhead, especially in large codebases with deep dependency trees. However, some Python patterns depend on imports executing immediately. For example: Module-level side effects — a module that registers a handler or modifies global state at import time will behave differently if that import is deferred. The registry pattern — a module that registers itself (e.g., adding to a global dict) when imported will silently fail to register under Lazy Imports. sys.modules manipulation — code that reads or writes sys.modules assumes prior imports have already executed. Metaclasses and init_subclass — class creation side effects may depend on imports being resolved.

延迟导入可以显著减少内存占用、启动时间和导入开销,特别是在具有深层依赖树的大型代码库中。然而,一些 Python 模式依赖于立即执行的导入。例如:模块级副作用——在导入时注册处理程序或修改全局状态的模块,如果导入被推迟,其行为将会改变。注册模式——在导入时自动注册(例如添加到全局字典)的模块,在延迟导入下将无法静默注册。sys.modules 操作——读取或写入 sys.modules 的代码假设之前的导入已经执行。元类和 init_subclass——类创建的副作用可能依赖于导入的解析。

Adapting an existing codebase to use Lazy Imports can be a daunting task, especially at scale. Lifeguard identifies these incompatible patterns so you can adopt Lazy Imports with confidence. How does Lifeguard work? Lifeguard analyzes Python source files for a given project in parallel. It walks each module’s AST to detect effects and maps Lazy-Imports-incompatible effects to errors. The analyzer takes a conservative approach towards its analysis: any module that cannot be programmatically determined to be safe to import lazily is marked unsafe by default. This means Lifeguard will err on the side of marking potentially compatible modules as incompatible, leaving potential performance optimizations on the table in favor of production safety. For a deeper look at the analysis pipeline and architecture, see docs/architecture.md.

将现有的代码库适配为使用延迟导入可能是一项艰巨的任务,尤其是在大规模项目中。Lifeguard 可以识别这些不兼容的模式,以便您可以放心地采用延迟导入。Lifeguard 是如何工作的?Lifeguard 并行分析给定项目的 Python 源文件。它遍历每个模块的抽象语法树(AST)来检测副作用,并将与延迟导入不兼容的效应映射为错误。该分析器采取保守的分析方法:任何无法通过程序确定为延迟导入安全的模块,默认都会被标记为不安全。这意味着 Lifeguard 会倾向于将潜在兼容的模块标记为不兼容,从而为了生产环境的安全性而放弃部分性能优化。有关分析流水线和架构的更深入了解,请参阅 docs/architecture.md。

Project Stage: Beta Lifeguard is in active development. We are aiming to be ready for general use by the Python 3.15 final release. Items on our roadmap We’ve tested and support Python 3.12 and 3.14. Other versions may also work. To analyze the explicit lazy import syntax from PEP 810, pass —python-version 3.15. We are actively developing a standalone linter output mode to help users identify which specific lines in their codebase are incompatible with Lazy Imports. We plan to add support for easy ingestion of Lifeguard’s output to drive Lazy Imports enablement for advanced users (see Using the Output).

项目阶段:Beta。Lifeguard 处于活跃开发阶段。我们的目标是在 Python 3.15 正式发布时准备好供通用。路线图项目:我们已经测试并支持 Python 3.12 和 3.14。其他版本也可能有效。要分析 PEP 810 中的显式延迟导入语法,请传入 —python-version 3.15。我们正在积极开发一种独立的 linter 输出模式,以帮助用户识别代码库中哪些具体行与延迟导入不兼容。我们计划增加对 Lifeguard 输出的轻松摄取支持,以推动高级用户启用延迟导入(请参阅“使用输出”)。

Install from PyPI Lifeguard is published on PyPI with prebuilt wheels for Linux, macOS, and Windows (x86-64 and ARM64). It requires Python 3.12 or newer and no Rust toolchain: pip install lifeguard-lazy-imports lifeguard run-tree /path/to/project output.json —verbose-output verbose.txt python -m lifeguard_lazy_imports is equivalent to the lifeguard command. The cargo run — examples below build and run the tool from source; with the installed package, replace cargo run — with lifeguard. PyPI releases are cut manually and can lag behind the main branch. Run lifeguard —help to see what your installed version supports.

从 PyPI 安装:Lifeguard 已发布在 PyPI 上,并提供适用于 Linux、macOS 和 Windows (x86-64 和 ARM64) 的预构建 wheel 包。它需要 Python 3.12 或更高版本,且不需要 Rust 工具链:pip install lifeguard-lazy-imports。lifeguard run-tree /path/to/project output.json —verbose-output verbose.txt。python -m lifeguard_lazy_imports 等同于 lifeguard 命令。下方的 cargo run — 示例用于从源码构建并运行工具;如果使用已安装的包,请将 cargo run — 替换为 lifeguard。PyPI 发布是手动进行的,可能会滞后于主分支。运行 lifeguard —help 以查看您安装的版本支持的功能。

Prerequisites for Building from Source Rust (nightly) — install via rustup. Cargo uses the nightly pinned in rust-toolchain.toml; there is no need to change your global default toolchain. Git — clone with submodules: git clone —recurse-submodules https://github.com/facebook/Lifeguard.git If you already cloned without —recurse-submodules, run git submodule update —init —recursive.

从源码构建的先决条件:Rust (nightly) — 通过 rustup 安装。Cargo 使用 rust-toolchain.toml 中固定的 nightly 版本;无需更改您的全局默认工具链。Git — 使用子模块克隆:git clone —recurse-submodules https://github.com/facebook/Lifeguard.git。如果您在克隆时未使用 —recurse-submodules,请运行 git submodule update —init —recursive。

Quick Start The fastest way to try Lifeguard is the run-tree subcommand, which discovers .py files under a directory and follows resolvable top-level imports. File and directory names below the input root must be ASCII Python identifiers; other paths are skipped. cargo run — run-tree <INPUT_DIR> <OUTPUT_PATH> For example, using the bundled sample project: cargo run — run-tree testdata/sample_project output.json For a full walkthrough including interpreting the output, see GETTING_STARTED.md.

快速入门:尝试 Lifeguard 最快的方法是使用 run-tree 子命令,它会发现目录下的 .py 文件并跟踪可解析的顶级导入。输入根目录下的文件和目录名称必须是 ASCII Python 标识符;其他路径将被跳过。cargo run — run-tree <INPUT_DIR> <OUTPUT_PATH>。例如,使用捆绑的示例项目:cargo run — run-tree testdata/sample_project output.json。有关包括解释输出在内的完整演练,请参阅 GETTING_STARTED.md。

Running Lifeguard For larger projects where you need more control, you can generate a source DB — a JSON file that tells Lifeguard the full set of Python files in your project and their module paths (see Input Format for details). Follow these steps: Generate the source DB. We provide a subcommand to start this file for you, but you may need to tune it by hand. (As the project matures, we hope to make this process smoother.) cargo run — gen-source-db <INPUT_DIR> <OUTPUT_PATH> Optionally, if your project has library dependencies, you can point Lifeguard at your site-packages by adding a lifeguard section to your pyproject.toml: [lifeguard] site_packages = “/path/to/site-packages” You can find out your site-packages path via python -m site. Both gen-source-db and run-tree read this section from <INPUT_DIR>/pyproject.toml. Relative site_packages paths are resolved against INPUT_DIR. You can override the setting with —site-packages /path/to/site-packages.

运行 Lifeguard:对于需要更多控制的大型项目,您可以生成一个源数据库(source DB)——这是一个 JSON 文件,告诉 Lifeguard 项目中所有 Python 文件的集合及其模块路径(详情请参阅“输入格式”)。请遵循以下步骤:生成源数据库。我们提供了一个子命令来为您启动此文件,但您可能需要手动调整它。(随着项目的成熟,我们希望使此过程更加顺畅。)cargo run — gen-source-db <INPUT_DIR> <OUTPUT_PATH>。可选地,如果您的项目有库依赖,您可以通过在 pyproject.toml 中添加一个 lifeguard 部分来将 Lifeguard 指向您的 site-packages:[lifeguard] site_packages = “/path/to/site-packages”。您可以通过 python -m site 找到您的 site-packages 路径。gen-source-db 和 run-tree 都会从 <INPUT_DIR>/pyproject.toml 读取此部分。相对 site_packages 路径将相对于 INPUT_DIR 进行解析。您可以使用 —site-packages /path/to/site-packages 覆盖此设置。

Note: Discovery follows top-level import statements and may not discover all dependencies, such as imports nested in functions or conditional blocks outside the input tree. If Lifeguard reports missing modules, you may need to manually add entries to the generated source DB. For explicit lazy syntax, pass —python-version 3.15 to both source discovery and analysis. Run Lifeguard in one of two modes: Default: Prints a high-level analysis of your codebase (% of compatible files, top errors, etc.) and writes the JSON output to OUTPUT_PATH. cargo run — <DB_PATH> <OUTPUT_PATH> Verbose mode: Also writes a human-readable report showing which specific lines in each module cause incompatibility. cargo run — <DB_PATH> <OUTPUT_PATH> —verbose-output <VERBOSE_OUTPUT_PATH> Example Verbose Output: ## example.module.foo ### Errors ImportedModuleAssignment (1) Line 17 - sys UnsafeFunctionCall (1) Line 38

注意:发现过程遵循顶级 import 语句,可能无法发现所有依赖项,例如嵌套在函数中或输入树之外的条件块中的导入。如果 Lifeguard 报告缺少模块,您可能需要手动向生成的源数据库添加条目。对于显式延迟语法,请在源发现和分析阶段都传入 —python-version 3.15。以两种模式之一运行 Lifeguard:默认模式:打印代码库的高级分析(兼容文件百分比、主要错误等)并将 JSON 输出写入 OUTPUT_PATH。cargo run — <DB_PATH> <OUTPUT_PATH>。详细模式:还会编写一份人类可读的报告,显示每个模块中导致不兼容的具体行。cargo run — <DB_PATH> <OUTPUT_PATH> —verbose-output <VERBOSE_OUTPUT_PATH>。详细输出示例:## example.module.foo ### Errors ImportedModuleAssignment (1) Line 17 - sys UnsafeFunctionCall (1) Line 38