Go back

Zero to production in Rust An Option 笔记

Updated:
Edit page

## ZLD 的配置

Rust 编译大部分耗时在 linker 阶段, 所以文中给与 ZLD 配置, 但是注意后面的配置路径有点问题, homebrew Apple silicon 默认的安装位置在/opt/homebrew/bin/

# .cargo/config.toml
# On Windows
# ```
# cargo install -f cargo-binutils
# rustup component add llvm-tools-preview
# ```
[target.x86_64-pc-windows-msvc]
rustflags = ["-C", "link-arg=-fuse-ld=lld"]
[target.x86_64-pc-windows-gnu]
rustflags = ["-C", "link-arg=-fuse-ld=lld"]
# On Linux:
# - Ubuntu, `sudo apt-get install lld clang`
# - Arch, `sudo pacman -S lld clang`
[target.x86_64-unknown-linux-gnu]
rustflags = ["-C", "linker=clang", "-C", "link-arg=-fuse-ld=lld"]
# On MacOS, `brew install michaeleisel/zld/zld`
[target.x86_64-apple-darwin]
rustflags = ["-C", "link-arg=-fuse-ld=/usr/local/bin/zld"]
[target.aarch64-apple-darwin]
- rustflags = ["-C", "link-arg=-fuse-ld=/usr/local/bin/zld"]
+ rustflags = ["-C", "link-arg=-fuse-ld=/opt/homebrew/bin/zld"]

提到了新出来的 mold 会更加好一点, 有机会再试试

update:

总结: 不要花时间在 linker 上

配置

CI

Makefile

SQLx

  1. 首先 cargo install slqx-cli 如果不指定版本, 它会使用最新的 0.7.0 版本, 但是这个版本不兼容 0.6.3 所以无法得到 offline 的 sqlx-data.json (这个在新版本中已经被取消了)
  2. 新版本(0.7.0)取消了 runtime-actix-rustls 使得整个项目都无法正常运行, 所以暂时无法简单通过升级版本解决上面的问题
  3. 所以使用固定 install 的写法 cargo install sqlx-cli@0.6.3

技巧

使用 night 且不改变项目本身的 toolchain

使用 +night 不需要在项目中设置 tool-chain, 当然要注意下编译使用的版本需要和项目的一致, 比如 https://github.com/BurtonQin/lockbud

# Use the nightly toolchain just for this command invocation
cargo +nightly expand

Actix-Web 的一些技巧

actix-web 支持共享 app 配置了4

fn create_app() -> App<
    impl ServiceFactory<
        ServiceRequest,
        Config = (),
        Response = ServiceResponse<impl MessageBody>,
        Error = Error,
        InitError = (),
    >,
> {
    App::new()
        .route("/", web::get().to(greet))
        .route("/health_check", web::get().to(health_check))
        .route("/{name}", web::get().to(greet))
}

Debug

解引用的疑惑

按照 https://course.rs/advance/smart-pointer/deref.html 一文所述, String 会自动解引用

#[stable(feature = "rust1", since = "1.0.0")]
impl ops::Deref for String {
    type Target = str;

    #[inline]
    fn deref(&self) -> &str {
        unsafe { str::from_utf8_unchecked(&self.vec) }
    }
}

但是我遇到一个情况是, 需要使用 &*String 而不是 &String 来赋值5

    let mut connection = PgConnection::connect(&config.connection_string_without_db())
        .await
        .expect("Failed to connect to Postgres.");

    connection
+        .execute(&*format!(r#"CREATE DATABASE "{}";"#, config.database_name))
-        .execute(format!(r#"CREATE DATABASE "{}";"#, config.database_name).as_str())
        .await
        .expect("Failed to create database.");

按理来说应该是可以自动转换的, 比如: let tmp: &str = &String.

也许可能的原因: trait bound 高于解引用

实现 dyn trait 的疑惑

我发现 rust 在同一个 create 下是可以 impl trait for dyn trait 不报错的, 但是只要把 define trait 放在其它 create 下就不行

update: 我突然想到是不是我没有 use 的缘故

其它资料:

指针的优美之处

  fn read(&mut self, buf: &mut [u8]) -> io::Result<usize> {
        for slot in &mut *buf {
            *slot = self.byte;
        }
        Ok(buf.len())
    }

RwLock 的坑记录

In particular, a writer which is waiting to acquire the lock in write might or might not block concurrent calls to read6

代码 https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=18024235e93eeb6f580eea8770167d63 在 playground 上无法运行, 但是在 apple silicon 上运行正常, 所以有的时候也无法保证 write 一定会 block read…

update: 它使用 semaphore 来控制 read/write, 所以


// Thread 1             |  // Thread 2
let _rg = lock.read();  |
                        |  // will block
                        |  let _wg = lock.write();
// may deadlock         |
let _rg = lock.read();  |

会出现死锁的情况, 但是先 write 的话就会先占用所有 semaphore 量就不会出现循环等待 (所以我觉得这种放在 init pool manager 是一个不错的选择, 当然使用 try_writer 是另一个好的选择)

加快 docker 的构建速度

我之前知道使用 sccache 来缓存编译和使用 release 来缩小编译体积7, 但这一般是构建宿主机上的优化手段, 从来没考虑过加速构建 docker, 所幸现在学习到了 🥇

前置知识

我的理解:

  1. 首先我们需要知道 rust 没有像 npm install 那样直接根据依赖文件直接安装的功能[^8]
  2. 所以我们需要手动先编译仅含有 Cargo.toml Cargo.lock 和 empty src/main.rs src/lib.rs(这里是因为这个项目是 bin 类型)8
  3. 继承这个镜像然后 copy 所有的 src 文件再编译, 这样就可以利用之前的 /usr/local/cargo 和 target cache 了

实战(不考虑第一次构建耗时)修改 main.rs 然后重新构建计算耗时:

手动复制 Cargo.toml 多阶段构建 1m50s 
# Set the base image
FROM rust:1.69 AS toolchain
WORKDIR /app
RUN apt update && apt install lld clang -y

# Fetch all the carte source file
FROM toolchain AS bare-source
COPY Cargo.toml Cargo.lock /app/
RUN \
    mkdir /app/src && \
    echo 'fn main() {}' > /app/src/main.rs && \
    touch /app/src/lib.rs && \
    cargo build --release && \
    rm -Rvf /app/src

FROM bare-source AS builder
COPY . .
ENV SQLX_OFFLINE true
# Build the project
RUN cargo clean && cargo build --release --bin zero2prod

# Runtime stage
FROM debian:bullseye-slim AS runtime
WORKDIR /app
# Install OpenSSL - it is dynamically linked by some of our dependencies
# Install ca-certificates - it is needed to verify TLS certificates
# when establishing HTTPS connections
RUN apt-get update -y \
    && apt-get install -y --no-install-recommends openssl ca-certificates \
    # Clean up
    && apt-get autoremove -y \
    && apt-get clean -y \
    && rm -rf /var/lib/apt/lists/*
COPY --from=builder /app/target/release/zero2prod zero2prod
COPY configuration configuration
ENV APP_ENVIRONMENT production
ENTRYPOINT ["./zero2prod"]
使用cargo-chef 多阶段构建 1m52s 
FROM rust:1.69 AS chef
WORKDIR /app
RUN cargo install cargo-chef

FROM chef AS planner
# Copy the whole project
COPY . .
# Prepare a build plan ("recipe")
RUN cargo chef prepare --recipe-path recipe.json

FROM chef AS builder
# Copy the build plan from the previous Docker stage
COPY --from=planner /app/recipe.json recipe.json
# Build dependencies - this layer is cached as long as `recipe.json`
# doesn't change.
RUN cargo chef cook --recipe-path recipe.json
COPY . .
ENV SQLX_OFFLINE true
# Build the project
RUN cargo build --release --bin zero2prod

# Runtime stage
FROM debian:bullseye-slim AS runtime
WORKDIR /app
# Install OpenSSL - it is dynamically linked by some of our dependencies
# Install ca-certificates - it is needed to verify TLS certificates
# when establishing HTTPS connections
RUN apt-get update -y \
    && apt-get install -y --no-install-recommends openssl ca-certificates \
    # Clean up
    && apt-get autoremove -y \
    && apt-get clean -y \
    && rm -rf /var/lib/apt/lists/*
COPY --from=builder /app/target/release/zero2prod zero2prod
COPY configuration configuration
ENV APP_ENVIRONMENT production
ENTRYPOINT ["./zero2prod"]

总结: 还是和书上一致, 使用 chef (Better support of Docker layer caching in Cargo 一文说的很清楚了, 各有利弊)

update: 未来需要看看 sccache 来加速编译速度, 我目前的瓶颈卡在 cargo build --release --bin zero2prod 上

部署

port: Internal port to connect to. Needs to be available on 0.0.0.0. Required.

underscore pattern 的错误理解

之前我讲 _ 和 _xx 当做同样的事情来看, 但是写下面代码的时候死活都无法触发一次请求

let _ = Mock::given(path("/emails"))
        .and(method("POST"))
        .respond_with(ResponseTemplate::new(200))
        .named("Create unconfirmed subscriber")
		.except(1)
        .mount_as_scoped(&app.email_server)
        .await;

然后将 except 去掉的时候看日志报 404 错误, 就知道这个 guard 自动 drop 掉导致服务没有正确 mount 上

总结: _ 会立刻 drop, 并不是和 _xx 一样随作用域结束来 drop 的13

_ 和 let _ 的区别就是 _ = expr; 是 { let _ = expr; }14

Footnotes

  1. https://eisel.me/lld ↩

  2. https://www.reddit.com/r/rust/comments/11h28k3/faster_apple_builds_with_the_lld_linker/ ↩

  3. https://github.com/BurtonQin/lockbud/issues/44 ↩

  4. https://github.com/actix/actix-web/issues/1147#issuecomment-1509937750 ↩

  5. https://users.rust-lang.org/t/what-is-the-difference-between-as-ref/76059 ↩

  6. https://doc.rust-lang.org/std/sync/struct.RwLock.html ↩

  7. https://github.com/johnthagen/min-sized-rust#strip-symbols-from-binary ↩

  8. https://github.com/rust-lang/cargo/issues/2644 ↩

  9. https://www.lpalmieri.com/posts/fast-rust-docker-builds/ ↩

  10. https://registry.terraform.io/ ↩

  11. https://developer.hashicorp.com/vault/docs/what-is-vault ↩

  12. https://www.bilibili.com/video/BV1L34y1B7PT ↩

  13. https://stackoverflow.com/questions/76311007/what-happens-when-assigning-to-the-underscore-pattern ↩

  14. https://t.me/c/2261788729/77332 ↩

Edit page
Share this post on:
Share this post via WhatsAppShare this post on FacebookShare this post on XShare this post via TelegramShare this post on PinterestShare this post via email
Next Post
QNAP 技巧分享