Pytest pt2 - Mastering Pytest Fixtures

查看原文 / View Original 编辑 / Edit

Pytest pt2 - Mastering Pytest Fixtures

Pytest 系列第二部分：精通 Pytest Fixtures

Mastering Pytest Fixtures Fixtures are the backbone of pytest testing. They set up objects — like a SparkSession, a sample DataFrame, or a database connection — that your tests can use. Instead of constructing the same input data in ten different test functions, you define a fixture once and inject it where needed. In the previous post we covered the fundamentals of testing and mocking; now it’s time to eliminate duplication and make your test suite faster and easier to maintain. This post covers everything you need to know about fixtures: how to define them, the different scopes available, how to use yield for setup and teardown, when to reach for autouse=True, and how conftest.py acts as a shared repository for fixtures across your entire test suite.

精通 Pytest Fixtures。Fixtures 是 pytest 测试的基石。它们负责设置测试所需的各种对象，例如 SparkSession、示例 DataFrame 或数据库连接。与其在十个不同的测试函数中重复构建相同的输入数据，不如定义一次 fixture，然后在需要的地方注入它。在上一篇文章中，我们介绍了测试和 Mock 的基础知识；现在是时候消除重复代码，让你的测试套件运行更快且更易于维护了。本文涵盖了关于 fixtures 你需要了解的一切：如何定义它们、可用的不同作用域（scopes）、如何使用 yield 进行设置（setup）和清理（teardown）、何时使用 autouse=True，以及 conftest.py 如何作为整个测试套件的共享 fixture 仓库。

Defining a Fixture and Scoping It

定义 Fixture 及其作用域

A fixture is just a function decorated with @pytest.fixture. You can request it in a test by including its name as a parameter:

Fixture 本质上就是一个被 @pytest.fixture 装饰的函数。你可以在测试中通过将 fixture 名称作为参数传入来调用它：

import pytest
from pyspark.sql import SparkSession

@pytest.fixture(scope="session")
def spark():
    return (SparkSession.builder
            .master("local[2]")
            .appName("test")
            .getOrCreate())

Notice the line continuation uses parentheses instead of backslashes — it’s cleaner and avoids accidental white‑space issues. The scope parameter tells pytest how long the fixture should live:

注意，这里的换行使用了圆括号而不是反斜杠——这样更整洁，也能避免意外的空格问题。scope 参数告诉 pytest 该 fixture 的生命周期：

  • function (default): created anew for each test function. Safest when a test might mutate the fixture.

  • class: created once per test class. Good when you have several tests inside a class that share setup.

  • module: created once per test file.

  • package: created once per test package (a directory with __init__.py or conftest.py).

  • session: created once for the entire test run. Use for very expensive setup like a SparkSession.

  • function (默认): 每个测试函数都会重新创建。当测试可能会修改 fixture 时最安全。

  • class: 每个测试类创建一个。适用于类中多个测试共享同一设置的情况。

  • module: 每个测试文件创建一个。

  • package: 每个测试包（包含 __init__.py 或 conftest.py 的目录）创建一个。

  • session: 整个测试运行期间只创建一次。适用于像 SparkSession 这样开销极大的设置。

Scopes let you trade isolation for speed. I use session for the SparkSession because starting a new Spark context for every test is painfully slow. But if a test writes to a temporary table or changes a config, that state leaks to later tests — a conflict. For mutable fixtures, stick with function or module scope and clean up after yourself via a yield or a finalizer.

作用域让你可以在隔离性和速度之间进行权衡。我为 SparkSession 使用 session 作用域，因为为每个测试启动一个新的 Spark 上下文实在太慢了。但如果某个测试写入了临时表或更改了配置，该状态会泄露给后续测试，从而引发冲突。对于可变（mutable）的 fixture，请坚持使用 function 或 module 作用域，并通过 yield 或终结器（finalizer）进行清理。

Setup and Teardown with yield

使用 yield 进行设置与清理

Fixtures aren’t just about creating objects — they can also run cleanup after a test finishes. Use yield to separate setup from teardown:

Fixtures 不仅仅用于创建对象，它们还可以在测试结束后运行清理工作。使用 yield 来分隔设置（setup）和清理（teardown）：

@pytest.fixture(scope="function")
def temp_table(spark):
    spark.sql("CREATE OR REPLACE TEMP VIEW test_view AS SELECT 1 AS id")
    yield
    spark.sql("DROP VIEW IF EXISTS test_view")

Everything before yield runs before the test. Everything after yield runs after the test, even if the test fails. This is the cleanest way to ensure resources get released.

yield 之前的所有代码在测试前运行。yield 之后的所有代码在测试后运行，即使测试失败也会执行。这是确保资源被释放的最干净的方法。

Autouse Fixtures: Implicit Setup and Teardown

Autouse Fixtures：隐式设置与清理

Sometimes you want every test to begin in a known state without having to explicitly request a fixture in each test. That’s exactly what autouse=True is for. An autouse fixture runs automatically for every test in its scope, making it ideal for tasks like cleaning up files, resetting environment variables, or preparing test data.

有时你希望每个测试都在已知的状态下开始，而无需在每个测试中显式调用 fixture。这正是 autouse=True 的用途。autouse fixture 会自动为作用域内的每个测试运行，非常适合清理文件、重置环境变量或准备测试数据等任务。

from pathlib import Path
import pytest

LOG_FILE = Path("test.log")

@pytest.fixture(autouse=True)
def remove_test_log():
    # Setup: start each test with no log file
    LOG_FILE.unlink(missing_ok=True)
    yield
    # Teardown: remove any log file the test created
    LOG_FILE.unlink(missing_ok=True)

The code before yield is the setup phase. It runs before each test and deletes test.log if it’s present, guaranteeing that every test starts with a clean environment. Once the test completes, execution resumes after yield. This is the teardown phase, which removes the log file again in case the test created or modified it. Because the fixture uses autouse=True, you don’t need to include it as a parameter in your test functions — it runs automatically for every test, helping keep your test suite isolated and deterministic.

yield 之前的代码是设置阶段。它在每个测试前运行并删除已存在的 test.log，确保每个测试都在干净的环境中开始。测试完成后，执行会恢复到 yield 之后。这是清理阶段，它会再次删除日志文件，以防测试创建或修改了它。由于该 fixture 使用了 autouse=True，你不需要在测试函数中将其作为参数包含进去——它会自动为每个测试运行，有助于保持测试套件的隔离性和确定性。

Sharing Fixtures with conftest.py

使用 conftest.py 共享 Fixtures

The conftest.py file is the central repository for fixtures and plugin configuration. Any fixture defined in a conftest.py at the root of your test directory is automatically available to all tests in that directory and its subdirectories — no imports needed. I use conftest.py for shared Spark sessions, test data generators, and mock helpers.

conftest.py 文件是 fixture 和插件配置的中心仓库。在测试目录根目录下的 conftest.py 中定义的任何 fixture，都会自动提供给该目录及其子目录中的所有测试使用，无需导入。我使用 conftest.py 来存放共享的 Spark 会话、测试数据生成器和 Mock 辅助工具。

import pytest
from pyspark.sql import Row

@pytest.fixture(scope="module")
def sample_transactions(spark):
    data = [
        Row(id=1, amount=100.0, currency="USD"),
        Row(id=2, amount=200.0, currency="EUR"),
        Row(id=3, amount=0.0, currency="USD"),
        Row(id=4, amount=None, currency=None),
    ]
    return spark.createDataFrame(data)

Because the spark fixture is session‑scoped, it will be created once and reused across all tests in the session. The sample_transactions fixture is module‑scoped, so it’s created once per test file that uses it.

由于 spark fixture 是 session 作用域的，它会被创建一次并在整个会话的所有测试中重用。sample_transactions fixture 是 module 作用域的，因此每个使用它的测试文件只会创建一次。

Fixture Best Practices

Fixture 最佳实践

A few guidelines I follow after watching test suites spiral out of control: 在目睹了测试套件变得难以控制后，我遵循以下几条准则：

  • Keep fixtures close to where they’re used. Put shared, general-purpose fixtures in conftest.py. Put specific fixtures in the test module that needs them. Don’t turn conftest.py into a dumping ground.

  • Match scope to the cost of creation. A SparkSession is expensive; make it session‑scoped. A list of numbers is cheap; function scope is fine and safer.

  • Use yield for any fixture that acquires a resource. Files, database connections, Spark temporary views — all should be cleaned up after the test, even if it fails.

  • Watch for fixture conflicts. A session‑scoped fixture that gets mutated by one test will affect every subsequent test. If you need mutation, drop the scope or reset the state in a teardown step.

  • 让 fixture 靠近它们被使用的地方。 将共享的、通用的 fixture 放在 conftest.py 中。将特定的 fixture 放在需要它们的测试模块中。不要把 conftest.py 变成垃圾堆。

  • 根据创建成本匹配作用域。 SparkSession 开销很大，设为 session 作用域；数字列表开销很小，使用 function 作用域既合适又安全。

  • 对于获取资源的任何 fixture，请使用 yield。 文件、数据库连接、Spark 临时视图——所有这些都应该在测试后清理，即使测试失败也是如此。

  • 注意 fixture 冲突。 一个被某个测试修改过的 session 作用域 fixture 会影响后续的所有测试。如果确实需要修改，请降低作用域，或者在清理步骤中重置状态。

Conclusion

总结

Fixtures let you stop copy‑pasting setup code and start composing reusable, well‑scoped building blocks. You’ve seen how to define them, control their lifetime with scopes, separate setup…

Fixtures 让你不再需要复制粘贴设置代码，转而开始构建可重用、作用域明确的组件。你已经了解了如何定义它们、通过作用域控制它们的生命周期、分离设置……

← 返回新闻列表 / Back to news list