Deploying Redpanda Kafka-Compatible Streaming Platform on Ubuntu 24.04

在 Ubuntu 24.04 上部署兼容 Kafka 的流处理平台 Redpanda

Redpanda is a Kafka-API-compatible streaming platform written in C++ with no JVM and no ZooKeeper. This guide installs Redpanda on Ubuntu 24.04, secures it with a Let’s Encrypt certificate and SASL/SCRAM authentication, tunes the kernel for production, verifies with a producer/consumer test, and exposes Redpanda Console behind Nginx basic auth. By the end, you’ll have a secured, production-tuned single-node Redpanda cluster with a web console. Prerequisite: Ubuntu 24.04 server sized per Redpanda’s CPU/memory requirements, non-root sudo user, and a domain A record (e.g. redpanda.example.com).

Redpanda 是一个兼容 Kafka API 的流处理平台，使用 C++ 编写，无需 JVM 和 ZooKeeper。本指南将介绍如何在 Ubuntu 24.04 上安装 Redpanda，通过 Let’s Encrypt 证书和 SASL/SCRAM 身份验证进行加固，针对生产环境优化内核，通过生产者/消费者测试进行验证，并配置 Nginx 基本身份验证以暴露 Redpanda 控制台。完成本指南后，你将拥有一个经过生产环境调优、安全且带有 Web 控制台的单节点 Redpanda 集群。前提条件：符合 Redpanda CPU/内存要求的 Ubuntu 24.04 服务器、非 root 的 sudo 用户，以及一个域名 A 记录（例如 redpanda.example.com）。

Install Redpanda

安装 Redpanda

$ sudo apt update
$ curl -1sLf 'https://dl.redpanda.com/nzc4ZYQK3WRGd9sy/redpanda/cfg/setup/bash.deb.sh' | sudo -E bash

Warning: Only run vendor setup scripts you trust — piped curl | sudo bash runs with root privileges. 警告：仅运行你信任的供应商安装脚本——通过管道传输的 curl | sudo bash 会以 root 权限运行。

$ sudo apt install redpanda -y
$ rpk --version

Open the Firewall

配置防火墙

Port	Service	Purpose
9092	Kafka API	Producer/consumer traffic
8082	Pandaproxy (HTTP)	REST access for non-Kafka clients
8081	Schema Registry	Avro/Protobuf schema versioning
9644	Admin API	Monitoring, config, health checks
33145	Internal RPC	Inter-node communication

端口	服务	用途
9092	Kafka API	生产者/消费者流量
8082	Pandaproxy (HTTP)	非 Kafka 客户端的 REST 访问
8081	Schema Registry	Avro/Protobuf 模式版本控制
9644	Admin API	监控、配置、健康检查
33145	Internal RPC	节点间通信

$ sudo ufw allow 9092,8082,8081,9644,33145/tcp
$ sudo ufw allow 80/tcp
$ sudo ufw allow 443/tcp
$ sudo ufw reload

Issue a Let’s Encrypt Certificate

签发 Let’s Encrypt 证书

Redpanda ships with plaintext networking by default, fine for a lab, not for anything else. Redpanda 默认使用明文网络传输，这仅适用于实验环境，不建议用于其他场景。

$ sudo apt install certbot -y
$ DOMAIN=redpanda.example.com
$ EMAIL=admin@example.com
$ sudo certbot certonly --standalone -d $DOMAIN --non-interactive --email $EMAIL

Certbot stores certs under /etc/letsencrypt/live, readable only by root. Redpanda runs as its own redpanda user, so copy the certs into a dedicated directory: Certbot 将证书存储在 /etc/letsencrypt/live 下，仅 root 可读。Redpanda 以其专属的 redpanda 用户身份运行，因此需要将证书复制到专用目录：

$ sudo mkdir /etc/redpanda/certs
$ sudo cp /etc/letsencrypt/live/$DOMAIN/fullchain.pem /etc/redpanda/certs/node.crt
$ sudo cp /etc/letsencrypt/live/$DOMAIN/privkey.pem /etc/redpanda/certs/node.key
$ sudo cp /etc/letsencrypt/live/$DOMAIN/chain.pem /etc/redpanda/certs/ca.crt
$ sudo chown -R redpanda:redpanda /etc/redpanda/certs/
$ sudo chmod 400 /etc/redpanda/certs/node.key
$ sudo chmod 444 /etc/redpanda/certs/node.crt /etc/redpanda/certs/ca.crt

Automate renewal — Let’s Encrypt certs expire every 90 days, so wire a deploy hook that refreshes Redpanda’s copies and restarts the service: 自动化续期——Let’s Encrypt 证书每 90 天过期，因此需要设置一个部署钩子（deploy hook）来刷新 Redpanda 的证书副本并重启服务：

$ sudo nano /etc/letsencrypt/renewal-hooks/deploy/redpanda-renewal.sh

(Add the following content to the script) (将以下内容添加到脚本中)

#!/bin/bash
DOMAIN="redpanda.example.com"
cp /etc/letsencrypt/live/$DOMAIN/fullchain.pem /etc/redpanda/certs/node.crt
cp /etc/letsencrypt/live/$DOMAIN/privkey.pem /etc/redpanda/certs/node.key
cp /etc/letsencrypt/live/$DOMAIN/chain.pem /etc/redpanda/certs/ca.crt
chown -R redpanda:redpanda /etc/redpanda/certs/
chmod 400 /etc/redpanda/certs/node.key
chmod 444 /etc/redpanda/certs/node.crt /etc/redpanda/certs/ca.crt
systemctl restart redpanda

$ sudo chmod +x /etc/letsencrypt/renewal-hooks/deploy/redpanda-renewal.sh
$ sudo certbot renew --dry-run

Tune for Production

生产环境调优

$ sudo rpk redpanda mode production
$ sudo rpk redpanda tune all
$ sudo systemctl start redpanda-tuner

tune all optimizes I/O schedulers and CPU settings, takes ~30 seconds. The tuner service reapplies these on every reboot since kernel tuning doesn’t persist otherwise. tune all 会优化 I/O 调度器和 CPU 设置，耗时约 30 秒。由于内核调优在重启后不会自动保留，因此 tuner 服务会在每次重启时重新应用这些设置。

Bootstrap Authentication

引导身份验证

Set security policy before the first startup so it’s enforced from the very first boot. 在首次启动前设置安全策略，以便从第一次启动时就开始强制执行。

$ sudo nano /etc/redpanda/.bootstrap.yaml

enable_sasl: true
admin_api_require_auth: true
superusers:
  - admin

Note: This file is only read on first startup. Later changes go through the Admin API or rpk cluster config. 注意：此文件仅在首次启动时读取。后续更改需通过 Admin API 或 rpk 集群配置进行。

Set the superuser’s password: 设置超级用户密码：

$ echo "RP_BOOTSTRAP_USER=admin:STRONG_PASSWORD:SCRAM-SHA-256" | sudo tee -a /etc/default/redpanda

Configure the Node

配置节点

$ sudo cp /etc/redpanda/redpanda.yaml /etc/redpanda/redpanda-backup.yaml
$ sudo nano /etc/redpanda/redpanda.yaml

(Replace the contents with your domain configuration) (将内容替换为你的域名配置)

redpanda:
  data_directory: /var/lib/redpanda/data
  seed_servers: []
  rpc_server:
    address: 0.0.0.0
    port: 33145
  advertised_rpc_api:
    address: redpanda.example.com
    port: 33145
  kafka_api:
    - name: tls_listener
      address: 0.0.0.0
      port: 9092
      authentication_method: sasl
  advertised_kafka_api:
    - name: tls_listener
      address: redpanda.example.com
      port: 9092
  kafka_api_tls:
    - name: tls_listener
      key_file: /etc/redpanda/certs/node.key
      cert_file: /etc/redpanda/certs/node.crt
      truststore_file: /etc/redpanda/certs/ca.crt
      enabled: true
      require_client_auth: false
  admin:
    - address: 0.0.0.0
      port: 9644
  admin_api_tls:
    - enabled: true
      key_file: /etc/redpanda/certs/node.key
      cert_file: /etc/redpanda/certs/node.crt
      truststore_file: /etc/redpanda/certs/ca.crt
  rpk:
    tune_network: true
    tune_disk_scheduler: true
    tune_disk_nomerges: true
    tune_disk_write_cache: true
    tune_disk_irq: true
    tune_cpu: true
    tune_aio_events: true
    tune_clocksource: true
    tune_swappiness: true
    coredump_dir: /var/lib/redpanda/coredump
    tune_ballast_file: true

This enables SASL-over-TLS on the Kafka API (9092), TLS on the Admin API (9644), advertises both under your public domain, and applies the full rpk tuning profile. 这将在 Kafka API (9092) 上启用 SASL-over-TLS，在 Admin API (9644) 上启用 TLS，在你的公共域名下发布这两个 API，并应用完整的 rpk 调优配置。

$ sudo systemctl start redpanda
$ sudo systemctl status redpanda

Create an rpk Profile

创建 rpk 配置文件

Run rpk as your regular user (not sudo) so it picks up your profile. 以普通用户身份（非 sudo）运行 rpk，以便它能加载你的配置文件。

$ nano ~/rpk-profile.yaml

kafka_api:
  sasl:
    user: admin
    password: STRONG_PASSWORD
    mechanism: SCRAM-SHA-256
  tls:
    enabled: true
    ca_file: /etc/redpanda/certs/ca.crt
  brokers:
    - redpanda.example.com:9092
admin_api:
  tls:
    enabled: true
    ca_file: /etc/redpanda/certs/ca.crt
  addresses:
    - redpanda.example.com:9644

$ rpk profile create production --from-profile ~/rpk-profile.yaml
$ rpk cluster health

Test with a Producer and Consumer

生产者与消费者测试

$ rpk topic create test-topic
$ echo "Hello Redpanda" | rpk topic produce test-topic