seg fault on docker container deployment

When I’m using flyctl launch my build works just fine and the site is deployed, however, when I use flyctl deploy I get a seg fault. I’m not sure what is causing the differing behavior, but because it works on one command and not the other I believe it may be an issue on the fly side and not the Rust side.

FROM rust:latest as builder
RUN apt-get update && apt-get -y install ca-certificates cmake musl-tools libssl-dev && rm -rf /var/lib/apt/lists/*
COPY . .
RUN rustup default stable && rustup update
RUN rustup target add x86_64-unknown-linux-musl
ENV PKG_CONFIG_ALLOW_CROSS=1
RUN cargo build --target x86_64-unknown-linux-musl --release

FROM scratch
COPY --from=builder /target/x86_64-unknown-linux-musl/release/rusty_portfolio .
COPY --from=builder /templates templates
EXPOSE 8000
#10 122.8 /usr/local/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-1bde28880ea8bde8.so(+0x4e54e3)[0x40021864e3]
#10 122.8 /lib/x86_64-linux-gnu/libpthread.so.0(+0x14140)[0x4005fbe140]
#10 122.8 /usr/local/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/rustc(+0x488a7)[0x40000488a7]
#10 122.8 /usr/local/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/rustc(+0x9841)[0x4000009841]
#10 122.8 /usr/local/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/../lib/libLLVM-14-rust-1.62.1-stable.so(_ZN4llvm9MCContext5resetEv+0x55c)[0x400935b49c]
#10 122.8 /usr/local/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/../lib/libLLVM-14-rust-1.62.1-stable.so(_ZN4llvm28MachineModuleInfoWrapperPass14doFinalizationERNS_6ModuleE+0x35)[0x400935af05]
#10 122.8 /usr/local/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/../lib/libLLVM-14-rust-1.62.1-stable.so(_ZN4llvm6legacy15PassManagerImpl3runERNS_6ModuleE+0xb0)[0x4009441660]
#10 122.8 /usr/local/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-1bde28880ea8bde8.so(+0x2634c75)[0x40042d5c75]
#10 122.8 /usr/local/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-1bde28880ea8bde8.so(+0x262ddb7)[0x40042cedb7]
#10 122.8 /usr/local/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-1bde28880ea8bde8.so(+0x26310c6)[0x40042d20c6]
#10 122.8 /usr/local/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-1bde28880ea8bde8.so(+0x262be78)[0x40042cce78]
#10 122.8 /usr/local/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-1bde28880ea8bde8.so(+0x2625d43)[0x40042c6d43]
#10 122.8 /usr/local/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-1bde28880ea8bde8.so(+0x25f553f)[0x400429653f]
#10 122.8 /usr/local/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-1bde28880ea8bde8.so(+0x2617e93)[0x40042b8e93]
#10 122.8 /usr/local/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/libstd-69edc9ac8de4d39c.so(rust_metadata_std_8996aeeb2a45fe64+0xab3c3)[0x4005ccb3c3]
#10 122.8 /lib/x86_64-linux-gnu/libpthread.so.0(+0x8ea7)[0x4005fb2ea7]
#10 122.8 /lib/x86_64-linux-gnu/libc.so.6(clone+0x3f)[0x40060d4def]
#10 122.8 qemu: uncaught target signal 11 (Segmentation fault) - core dumped
#10 122.8 error: could not compile `typenum`
#10 122.8
#10 122.8 Caused by:
#10 122.8   process didn't exit successfully: `rustc --crate-name build_script_main --edition=2018 /usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/typenum-1.15.0/build/main.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off -C metadata=ee6956477f56ac1c -C extra-filename=-ee6956477f56ac1c --out-dir /target/release/build/typenum-ee6956477f56ac1c -L dependency=/target/release/deps --cap-lints allow` (signal: 11, SIGSEGV: invalid memory reference)
#10 122.8 warning: build failed, waiting for other jobs to finish...

Can you paste the full output of the build? This can happen if you’re running the build locally on a Mac M1, which has trouble with cross-compiling for Linux. Looking at the full build output, we could see if you’re using a remote builder or not.

It is an m1, though I admittedly don’t know how to tell if I’m using a remote builder or not.

fly deploy
==> Verifying app config
--> Verified app config
==> Building image
==> Creating build context
--> Creating build context done
==> Building image with Docker
--> docker host: 20.10.17 linux aarch64
Sending build context to Docker daemon  185.9kB
[+] Building 167.4s (10/12)
 => [internal] load remote build context                                                       0.0s
 => copy /context /                                                                            0.1s
 => [internal] load metadata for docker.io/library/rust:latest                                 2.5s
 => [buildy 1/6] FROM docker.io/library/rust:latest@sha256:2cd5c8517d0640bd287b2cfb4c0e62f1a8  0.0s
 => CACHED [buildy 2/6] RUN apt-get update && apt-get -y install ca-certificates cmake musl-t  0.0s
 => CACHED copy /context /                                                                     0.0s
 => [buildy 3/6] COPY . .                                                                      0.0s
 => [buildy 4/6] RUN rustup default stable && rustup update                                   26.4s
 => [buildy 5/6] RUN rustup target add x86_64-unknown-linux-musl                              11.2s
 => ERROR [buildy 6/6] RUN cargo build --target x86_64-unknown-linux-musl --release          127.1s
------
 > [buildy 6/6] RUN cargo build --target x86_64-unknown-linux-musl --release:
#10 2.397     Updating crates.io index
#10 108.2  Downloading crates ...
#10 111.0   Downloaded binascii v0.1.4
#10 111.1   Downloaded devise_codegen v0.3.1
#10 111.1   Downloaded devise v0.3.1
#10 111.1   Downloaded chrono-tz-build v0.0.3
#10 111.1   Downloaded block-buffer v0.10.2
#10 111.1   Downloaded either v1.7.0
#10 111.1   Downloaded digest v0.10.3
#10 111.2   Downloaded cfg-if v1.0.0
#10 111.2   Downloaded bitflags v1.3.2
#10 111.2   Downloaded devise_core v0.3.1
#10 111.2   Downloaded globwalk v0.8.1
#10 111.2   Downloaded cpufeatures v0.2.2
#10 111.2   Downloaded ctr v0.8.0
#10 111.2   Downloaded crypto-common v0.1.6
#10 111.2   Downloaded crossbeam-utils v0.8.11
#10 111.2   Downloaded cipher v0.3.0
#10 111.2   Downloaded try-lock v0.2.3
#10 111.2   Downloaded fastrand v1.8.0
#10 111.3   Downloaded subtle v2.4.1
#10 111.3   Downloaded slug v0.1.4
#10 111.3   Downloaded slab v0.4.7
#10 111.3   Downloaded futures-io v0.3.21
#10 111.3   Downloaded stable-pattern v0.1.0
#10 111.3   Downloaded getrandom v0.2.7
#10 111.3   Downloaded httparse v1.7.1
#10 111.3   Downloaded tempfile v3.3.0
#10 111.3   Downloaded spin v0.9.4
#10 111.3   Downloaded thiserror v1.0.32
#10 111.4   Downloaded pear_codegen v0.2.3
#10 111.4   Downloaded phf_codegen v0.11.1
#10 111.4   Downloaded pest_derive v2.2.1
#10 111.4   Downloaded humansize v1.1.1
#10 111.4   Downloaded http-body v0.4.5
#10 111.4   Downloaded hmac v0.12.1
#10 111.4   Downloaded ghash v0.4.4
#10 111.4   Downloaded glob v0.3.0
#10 111.4   Downloaded typenum v1.15.0
#10 111.4   Downloaded state v0.5.3
#10 111.4   Downloaded time-macros v0.2.4
#10 111.5   Downloaded tracing-attributes v0.1.22
#10 111.5   Downloaded tower-service v0.3.2
#10 111.5   Downloaded tokio-macros v1.8.0
#10 111.5   Downloaded num_cpus v1.13.1
#10 111.5   Downloaded mime v0.3.16
#10 111.5   Downloaded tracing v0.1.36
#10 111.5   Downloaded tokio-stream v0.1.9
#10 111.5   Downloaded log v0.4.17
#10 111.5   Downloaded toml v0.5.9
#10 111.6   Downloaded ucd-trie v0.1.4
#10 111.6   Downloaded tokio-util v0.7.3
#10 111.6   Downloaded lazy_static v1.4.0
#10 111.6   Downloaded pest_generator v2.2.1
#10 111.6   Downloaded polyval v0.5.3
#10 111.6   Downloaded num-traits v0.2.15
#10 111.6   Downloaded uncased v0.9.7
#10 111.6   Downloaded ubyte v0.10.2
#10 111.7   Downloaded pest v2.2.1
#10 111.7   Downloaded parking_lot_core v0.9.3
#10 111.7   Downloaded once_cell v1.13.0
#10 111.7   Downloaded lazycell v1.3.0
#10 111.7   Downloaded parse-zoneinfo v0.3.0
#10 111.7   Downloaded memchr v2.5.0
#10 111.7   Downloaded pest_meta v2.2.1
#10 111.7   Downloaded proc-macro2-diagnostics v0.9.1
#10 111.8   Downloaded remove_dir_all v0.5.3
#10 111.8   Downloaded walkdir v2.3.2
#10 111.8   Downloaded unic-common v0.9.0
#10 111.8   Downloaded unic-char-range v0.9.0
#10 111.8   Downloaded unic-char-property v0.9.0
#10 111.8   Downloaded unicode-xid v0.2.3
#10 111.8   Downloaded want v0.3.0
#10 111.8   Downloaded rand v0.8.5
#10 111.9   Downloaded universal-hash v0.4.1
#10 111.9   Downloaded yansi v0.5.1
#10 111.9   Downloaded sha-1 v0.10.0
#10 111.9   Downloaded rocket_http v0.5.0-rc.2
#10 111.9   Downloaded unic-ucd-segment v0.9.0
#10 111.9   Downloaded rand_core v0.6.3
#10 111.9   Downloaded signal-hook-registry v1.4.0
#10 111.9   Downloaded sha2 v0.10.2
#10 111.9   Downloaded serde_derive v1.0.142
#10 111.9   Downloaded ryu v1.0.11
#10 112.0   Downloaded proc-macro2 v1.0.43
#10 112.0   Downloaded serde v1.0.142
#10 112.0   Downloaded aes v0.7.5
#10 112.0   Downloaded version_check v0.9.4
#10 112.0   Downloaded tokio v1.20.1
#10 112.1   Downloaded same-file v1.0.6
#10 112.1   Downloaded unic-ucd-version v0.9.0
#10 112.2   Downloaded serde_json v1.0.83
#10 112.2   Downloaded rocket_codegen v0.5.0-rc.2
#10 112.2   Downloaded regex-syntax v0.6.27
#10 112.2   Downloaded rocket v0.5.0-rc.2
#10 112.3   Downloaded quote v1.0.21
#10 112.3   Downloaded unic-segment v0.9.0
#10 112.3   Downloaded aho-corasick v0.7.18
#10 112.3   Downloaded unicode-ident v1.0.3
#10 112.3   Downloaded aead v0.4.3
#10 112.3   Downloaded async-stream-impl v0.3.3
#10 112.3   Downloaded async-stream v0.3.3
#10 112.4   Downloaded encoding_rs v0.8.31
#10 112.5   Downloaded libc v0.2.127
#10 112.6   Downloaded scopeguard v1.1.0
#10 112.6   Downloaded rocket_dyn_templates v0.1.0-rc.2
#10 112.6   Downloaded regex v1.6.0
#10 112.7   Downloaded ref-cast-impl v1.0.9
#10 112.7   Downloaded rand_chacha v0.3.1
#10 112.7   Downloaded hkdf v0.12.3
#10 112.7   Downloaded mio v0.8.4
#10 112.7   Downloaded http v0.2.8
#10 112.7   Downloaded num-integer v0.1.45
#10 112.7   Downloaded mio-extras v2.0.6
#10 112.7   Downloaded mio v0.6.23
#10 112.7   Downloaded ref-cast v1.0.9
#10 112.7   Downloaded phf v0.11.1
#10 112.8   Downloaded percent-encoding v2.1.0
#10 112.8   Downloaded tracing-core v0.1.29
#10 112.8   Downloaded syn v1.0.99
#10 112.8   Downloaded phf_shared v0.11.1
#10 112.8   Downloaded notify v4.0.17
#10 112.8   Downloaded net2 v0.2.37
#10 112.8   Downloaded inotify v0.7.1
#10 112.8   Downloaded itoa v1.0.3
#10 112.8   Downloaded inlinable_string v0.1.15
#10 112.8   Downloaded iovec v0.1.4
#10 112.8   Downloaded inotify-sys v0.1.5
#10 112.9   Downloaded indexmap v1.9.1
#10 112.9   Downloaded hyper v0.14.20
#10 112.9   Downloaded phf_generator v0.11.1
#10 112.9   Downloaded num_threads v0.1.6
#10 112.9   Downloaded time v0.3.12
#10 112.9   Downloaded pin-utils v0.1.0
#10 112.9   Downloaded parking_lot v0.12.1
#10 112.9   Downloaded bstr v0.2.17
#10 113.0   Downloaded pin-project-lite v0.2.9
#10 113.0   Downloaded h2 v0.3.13
#10 113.0   Downloaded lock_api v0.4.7
#10 113.0   Downloaded thread_local v1.1.4
#10 113.0   Downloaded siphasher v0.3.10
#10 113.0   Downloaded opaque-debug v0.3.0
#10 113.0   Downloaded thiserror-impl v1.0.32
#10 113.0   Downloaded socket2 v0.4.4
#10 113.0   Downloaded globset v0.4.9
#10 113.1   Downloaded tera v1.16.0
#10 113.1   Downloaded generic-array v0.14.6
#10 113.1   Downloaded filetime v0.2.17
#10 113.1   Downloaded pear v0.2.3
#10 113.1   Downloaded normpath v0.3.2
#10 113.1   Downloaded multer v2.0.3
#10 113.1   Downloaded ignore v0.4.18
#10 113.1   Downloaded chrono-tz v0.6.3
#10 113.2   Downloaded chrono v0.4.20
#10 113.2   Downloaded smallvec v1.9.0
#10 113.2   Downloaded futures-util v0.3.21
#10 113.3   Downloaded hashbrown v0.12.3
#10 113.3   Downloaded fnv v1.0.7
#10 113.3   Downloaded futures v0.3.21
#10 113.3   Downloaded futures-channel v0.3.21
#10 113.3   Downloaded httpdate v1.0.2
#10 113.3   Downloaded futures-task v0.3.21
#10 113.3   Downloaded futures-core v0.3.21
#10 113.3   Downloaded futures-sink v0.3.21
#10 113.3   Downloaded ppv-lite86 v0.2.16
#10 113.3   Downloaded aes-gcm v0.9.4
#10 113.3   Downloaded bytes v1.2.1
#10 113.4   Downloaded atomic v0.5.1
#10 113.4   Downloaded async-trait v0.1.57
#10 113.4   Downloaded cfg-if v0.1.10
#10 113.4   Downloaded atty v0.2.14
#10 113.4   Downloaded figment v0.10.6
#10 113.4   Downloaded cookie v0.16.0
#10 113.4   Downloaded deunicode v0.4.3
#10 113.4   Downloaded autocfg v1.1.0
#10 113.4   Downloaded base64 v0.13.0
#10 114.1    Compiling version_check v0.9.4
#10 114.1    Compiling libc v0.2.127
#10 114.1    Compiling proc-macro2 v1.0.43
#10 114.1    Compiling unicode-ident v1.0.3
#10 116.4    Compiling quote v1.0.21
#10 118.6    Compiling syn v1.0.99
#10 118.7    Compiling autocfg v1.1.0
#10 119.4    Compiling typenum v1.15.0
#10 120.0    Compiling cfg-if v1.0.0
#10 121.9    Compiling memchr v2.5.0
#10 122.2    Compiling log v0.4.17
#10 122.8 /usr/local/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-1bde28880ea8bde8.so(+0x4e54e3)[0x40021864e3]
#10 122.8 /lib/x86_64-linux-gnu/libpthread.so.0(+0x14140)[0x4005fbe140]
#10 122.8 /usr/local/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/rustc(+0x488a7)[0x40000488a7]
#10 122.8 /usr/local/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/rustc(+0x9841)[0x4000009841]
#10 122.8 /usr/local/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/../lib/libLLVM-14-rust-1.62.1-stable.so(_ZN4llvm9MCContext5resetEv+0x55c)[0x400935b49c]
#10 122.8 /usr/local/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/../lib/libLLVM-14-rust-1.62.1-stable.so(_ZN4llvm28MachineModuleInfoWrapperPass14doFinalizationERNS_6ModuleE+0x35)[0x400935af05]
#10 122.8 /usr/local/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/../lib/libLLVM-14-rust-1.62.1-stable.so(_ZN4llvm6legacy15PassManagerImpl3runERNS_6ModuleE+0xb0)[0x4009441660]
#10 122.8 /usr/local/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-1bde28880ea8bde8.so(+0x2634c75)[0x40042d5c75]
#10 122.8 /usr/local/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-1bde28880ea8bde8.so(+0x262ddb7)[0x40042cedb7]
#10 122.8 /usr/local/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-1bde28880ea8bde8.so(+0x26310c6)[0x40042d20c6]
#10 122.8 /usr/local/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-1bde28880ea8bde8.so(+0x262be78)[0x40042cce78]
#10 122.8 /usr/local/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-1bde28880ea8bde8.so(+0x2625d43)[0x40042c6d43]
#10 122.8 /usr/local/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-1bde28880ea8bde8.so(+0x25f553f)[0x400429653f]
#10 122.8 /usr/local/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-1bde28880ea8bde8.so(+0x2617e93)[0x40042b8e93]
#10 122.8 /usr/local/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/libstd-69edc9ac8de4d39c.so(rust_metadata_std_8996aeeb2a45fe64+0xab3c3)[0x4005ccb3c3]
#10 122.8 /lib/x86_64-linux-gnu/libpthread.so.0(+0x8ea7)[0x4005fb2ea7]
#10 122.8 /lib/x86_64-linux-gnu/libc.so.6(clone+0x3f)[0x40060d4def]
#10 122.8 qemu: uncaught target signal 11 (Segmentation fault) - core dumped
#10 122.8 error: could not compile `typenum`
#10 122.8
#10 122.8 Caused by:
#10 122.8   process didn't exit successfully: `rustc --crate-name build_script_main --edition=2018 /usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/typenum-1.15.0/build/main.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off -C metadata=ee6956477f56ac1c -C extra-filename=-ee6956477f56ac1c --out-dir /target/release/build/typenum-ee6956477f56ac1c -L dependency=/target/release/deps --cap-lints allow` (signal: 11, SIGSEGV: invalid memory reference)
#10 122.8 warning: build failed, waiting for other jobs to finish...
------
Error failed to fetch an image or build from source: error building: executor failed running [/bin/sh -c cargo build --target x86_64-unknown-linux-musl --release]: exit code: 101

It does seem to be building locally, and I just found while looking at the builder instance I can use the --remote-only flag so let me give that a shot real quick.

Using --remote-only worked. Didn’t realize that even with the builder instance that it would default to building locally.
Thanks for the hint!

1 Like

It should be defaulting to remote deploys. What does flyctl version say?

1 Like

0.330…
v0.0.367
No need to ask for --remote-only anymore

Looks like I should update… lol