Reclaim the terminal

Reclaim the terminal / 重获终端控制权

Pipe a file into less and it still responds when you press j. But if less is reading the file from stdin, where is it reading your keystrokes from? I ran into the same question while connecting piped input to an interactive program. By the time the interactive program started, fd 0 was an exhausted pipe. Tools like less and fzf showed that getting the keyboard back was possible: fzf reads its entire candidate list from a pipe and still lets you type to filter it. But I did not know how they did it. Here’s what I found.

将文件通过管道传给 less,当你按下 j 键时它依然能响应。但如果 less 正从标准输入(stdin)读取文件,它又是从哪里读取你的按键输入的呢?我在将管道输入连接到一个交互式程序时也遇到了同样的问题。当交互式程序启动时,文件描述符 0(fd 0)已经是一个耗尽的管道了。像 lessfzf 这样的工具表明,重新获取键盘输入是可能的:fzf 从管道读取完整的候选列表,但仍然允许你输入内容进行过滤。但我不知道它们是如何做到的。以下是我发现的内容。

The experiment / 实验

The program is one binary piped into itself four times: ./target/debug/feat-test | ./target/debug/feat-test | ./target/debug/feat-test | ./target/debug/feat-test Each stage waits for its turn, reads one line from the terminal, and passes the accumulated lines downstream. The final stage prints all four lines with the pid of the process that read each one. Before typing anything, run ps in another window: all four processes already exist. The shell does not launch stage 2 when stage 1 finishes. It launches the whole pipeline at once, and the final stage is alive and waiting before you have pressed a single key.

该程序是一个二进制文件,通过管道自身连接了四次: ./target/debug/feat-test | ./target/debug/feat-test | ./target/debug/feat-test | ./target/debug/feat-test 每一级都会等待轮到自己,从终端读取一行,并将累积的行传递给下游。最后一级会打印所有四行内容,并附上读取每一行的进程 PID。在输入任何内容之前,在另一个窗口运行 ps:所有四个进程都已经存在了。Shell 并不是在第一级完成后才启动第二级,而是一次性启动整个管道,因此在按下任何按键之前,最后一级就已经处于存活并等待的状态了。

That leaves three questions. The processes are all instances of the same binary, so how does each one know whether it is first, last, or somewhere in the middle? After the first stage, stdin carries data from a pipe, so how does a process get back to the keyboard? And since all four are running from the start, what stops them from trying to read your input at once?

这留下了三个问题:这些进程都是同一个二进制文件的实例,那么每个进程如何知道自己是处于第一级、最后一级还是中间某处?在第一级之后,stdin 承载的是来自管道的数据,那么进程如何重新获取键盘输入?既然所有四个进程从一开始就在运行,是什么阻止它们同时尝试读取你的输入?

How does each process know where it is? / 每个进程如何知道自己的位置?

It is easy to picture a pipeline as having one stdin at the beginning and one stdout at the end. But stdin and stdout belong to processes, not pipelines. Every process has its own fd 0 and fd 1. The shell simply connects them to different things. For the first process, fd 0 still points to the terminal. For every later process, it points to the previous stage’s pipe. At the other end, the final process’s fd 1 points to the terminal while every earlier process writes to a pipe.

很容易将管道想象成在开头有一个 stdin,在结尾有一个 stdout。但 stdin 和 stdout 属于进程,而不是管道。每个进程都有自己的 fd 0 和 fd 1。Shell 只是将它们连接到不同的地方。对于第一个进程,fd 0 仍然指向终端;对于后续的每个进程,它指向的是上一级的管道。在另一端,最后一个进程的 fd 1 指向终端,而所有之前的进程都写入管道。

Rust exposes the relevant check through IsTerminal:

let lines: Vec<Entry> = if stdin.is_terminal() {
    // First stage: read from stdin, which is the terminal.
} else {
    // Later stage: drain the pipe, then read from the terminal.
};

if stdout.is_terminal() {
    // Last stage: print for the user.
} else {
    // Earlier stage: serialize for the next process.
}

The binary does not need a stage number. It can infer its position from what its own stdin and stdout are connected to.

Rust 通过 IsTerminal 暴露了相关的检查:

let lines: Vec<Entry> = if stdin.is_terminal() {
    // 第一级:从 stdin 读取,即终端。
} else {
    // 后续级:排空管道,然后从终端读取。
};

if stdout.is_terminal() {
    // 最后一级:为用户打印。
} else {
    // 之前级:为下一个进程序列化。
}

该二进制文件不需要阶段编号,它可以通过自身 stdin 和 stdout 的连接情况推断出自己的位置。

What tells the next stage to start? / 是什么告诉下一级开始运行?

I initially expected the stages to need a separate coordination channel. They do not. A downstream stage starts by draining its stdin: stdin.read_to_string(&mut buf)?; At first, this looks like ordinary data loading. But read_to_string does not return just because the pipe is empty. An empty pipe means there is nothing to read yet; EOF means nothing can ever arrive again. As long as stage 1 is alive, stage 2 waits inside that call. When stage 1 exits, its end of the pipe closes. Only then does stage 2’s read return. The pipe itself provides the handoff: there is no separate “your turn” message.

我最初以为这些阶段需要一个单独的协调通道,但其实不需要。下游阶段通过排空其 stdin 来开始工作: stdin.read_to_string(&mut buf)?; 起初,这看起来像是普通的数据加载。但 read_to_string 不会仅仅因为管道为空就返回。管道为空意味着暂时没有数据可读;而 EOF(文件结束符)意味着再也不会有数据到达了。只要第一级进程存活,第二级就会一直阻塞在那个调用中。当第一级退出时,它那一端的管道关闭,只有这时第二级的读取操作才会返回。管道本身就提供了交接机制:不需要单独的“轮到你了”消息。

How does a piped process get the keyboard back? / 管道进程如何重新获取键盘输入?

But stage 2 now has a different problem. It is finally awake, and fd 0 is an exhausted pipe. It still has no apparent way to reach the keyboard. After draining stdin, a downstream stage still has the exhausted pipe on fd 0. It opens its controlling terminal separately: let term = File::open("/dev/tty")?; This does not restore fd 0. It creates a new file descriptor, usually the next free slot, that refers to the same terminal. The program can read from that descriptor directly. There is no ceremony involved: no permission to request, no coordination with the shell. Any process with a controlling terminal can open it at any time.

但第二级现在面临另一个问题:它终于被唤醒了,但 fd 0 仍然是一个耗尽的管道。它似乎仍然无法触及键盘。在排空 stdin 后,下游阶段的 fd 0 上仍然是那个耗尽的管道。它会单独打开其控制终端: let term = File::open("/dev/tty")?; 这并不会恢复 fd 0。它创建了一个新的文件描述符(通常是下一个空闲槽位),指向同一个终端。程序可以直接从该描述符读取数据。这不需要任何繁琐的步骤:无需请求权限,也无需与 Shell 协调。任何拥有控制终端的进程都可以随时打开它。

This is where my mental model had been backwards. Your keystrokes never go “to stdin.” They go to the terminal, and fd 0 is just a descriptor that usually happens to point there. When the shell pointed fd 0 at a pipe instead, the keyboard did not go anywhere; the process only lost its usual pointer to it. Opening /dev/tty makes a new one. This is what less is doing when you press j: the file arrives on stdin, and your keystrokes come from the terminal.

这就是我之前思维模型出错的地方。你的按键输入从来不是“发送到 stdin”,它们是发送到终端的,而 fd 0 只是一个通常恰好指向终端的描述符。当 Shell 将 fd 0 指向管道时,键盘输入并没有消失;进程只是失去了指向它的常规指针。打开 /dev/tty 会创建一个新的指针。这就是 less 在你按下 j 时所做的事情:文件通过 stdin 到达,而你的按键输入则来自终端。

/dev/tty does not name a particular device such as /dev/ttys003. It resolves to the controlling terminal of the calling process. The processes in this pipeline belong to the terminal session created by the shell, so the same path works for every stage.

/dev/tty 并不指向某个特定的设备(如 /dev/ttys003),它会解析为调用进程的控制终端。此管道中的所有进程都属于 Shell 创建的终端会话,因此相同的路径对每一级都有效。

What if two processes read the terminal at once? / 如果两个进程同时读取终端会怎样?

They compete. Terminal input is a queue, not a broadcast: whichever read gets there first consumes the line, and it is gone. The terminal does not know the pipeline exists and does not assign turns. The race never occurs here, but not because anything prevents it. Every downstream process is still blocked on its pipe; the ordering comes from the program’s reads, not from the terminal.

它们会竞争。终端输入是一个队列,而不是广播:谁先读取,谁就消耗掉那一行,数据随之消失。终端不知道管道的存在,也不会分配轮次。这里从未发生竞争,但这并不是因为有什么机制阻止了它。每个下游进程都阻塞在自己的管道上;顺序是由程序的读取操作决定的,而不是由终端决定的。

Ctrl-C may look like an exception, but it takes a different path. The terminal driver turns it into SIGINT for the foreground process group. Input goes to one reader; terminal-generated signals go to the group.

Ctrl-C 看起来可能是一个例外,但它走的是另一条路径。终端驱动程序将其转换为发送给前台进程组的 SIGINT 信号。输入发送给单个读取者,而终端生成的信号则发送给整个进程组。