diff --git a/.gitignore b/.gitignore index 5abb451..44a2af1 100644 --- a/.gitignore +++ b/.gitignore @@ -17,6 +17,10 @@ third_party/src/ /docs/* !/docs/investigating-builds.md !/docs/ripgrep-tutorial.md +!/docs/usage.md +!/docs/build-systems.md +!/docs/troubleshooting.md +!/docs/internals.md !/docs/assets/ /docs/assets/* !/docs/assets/ripgrep-release-clean.png diff --git a/README.md b/README.md index dccbdd0..e1cdd2e 100644 --- a/README.md +++ b/README.md @@ -13,25 +13,30 @@

Quick start · Try the demo · - Investigation guide · - Install + Features · + Install · + Documentation

## What is Buildprof? -Buildprof traces every process a Linux build launches and turns the recording -into an interactive timeline you can explore in the browser. Put `buildprof --` -in front of your build command to get started. - -- **See the whole build.** Find expensive commands, gaps in parallelism, and - work that starts unexpectedly late across Make, Ninja, CMake, Meson, Cargo, - Go, and shell scripts. -- **Follow the files.** Inspect commands, working directories, and exit statuses, - then follow inputs back to the processes that produced them. -- **Look inside compilers.** Optionally add internal timing data from Clang, - LLD, and nightly Rust alongside the process timeline. +Buildprof traces **every process** a Linux build launches and turns the +recording into an **interactive timeline** you can explore in the browser. Put +`buildprof --` in front of your build command to get started. + +- **See the whole build.** Every command the build ran, however it was + launched, with its command line, working directory, exit status, and place in + the process tree. +- **Follow the files.** File opens and renames are recorded alongside the + processes, so the recording links a file's producer to everything that + consumed it. That link is usually how you find the step waiting on work it + did not need. +- **Whatever your build system.** Buildprof follows processes rather than build + systems, so Make, Ninja, CMake, Meson, Cargo, Go, npm, Bazel and Buck2 all + work, as do the shell scripts, code generators and wrapper scripts they + launch. Recording requires Linux. You can explore recordings on any platform in the [web UI](https://buildprof.lalitm.com); trace data stays in your browser. @@ -67,9 +72,8 @@ buildprof -- make -j8 ``` Replace `make -j8` with your build command, such as `cargo build` or -`ninja -C out`. For Bazel, Gradle, Buck2, and other daemon build systems, see -[daemon builds](#bazel-gradle-and-other-daemon-build-systems); for Docker, see -[container builds](#builds-inside-docker-containers). +`ninja -C out`. Bazel, Gradle and Docker hand work to a daemon, which needs +[a little more care](docs/troubleshooting.md). ### 3. Explore the recording @@ -81,327 +85,130 @@ localhost. Nothing is ever uploaded. Start with the longest commands and gaps in parallelism. The [investigation guide](docs/investigating-builds.md) walks through finding bottlenecks, following file dependencies, and checking whether a change helped. -For builds over SSH, see [Builds on a remote machine](#builds-on-a-remote-machine). + +## Features + +- **Look inside compilers.** `--compiler-traces` records what a compiler did + internally, as per-thread phase tracks under the process that produced them, + so you can go from "this rustc took 40 seconds" to which part of it did. + ([details](docs/usage.md#compiler-details)) + - Clang `-ftime-trace`: parsing, template instantiation, optimisation. + - LLD `--time-trace`, when the build selects LLD explicitly. + - Nightly Rust self-profile data, per compiler query. +- **Record in CI.** The GitHub Action records a build and attaches the + recording to the job summary, including when the build fails, which is the + quickest way to find out why CI is slower than your laptop. + ([details](docs/usage.md#recording-in-github-actions)) +- **Record over SSH.** On a build host there is no browser to launch, so + Buildprof prints the port forward to run from your own machine and waits for + it. ([details](docs/usage.md#builds-on-a-remote-machine)) +- **Turn collection down.** `--no-file-events` keeps the process timeline and + skips filesystem interception entirely, for builds where that overhead + matters. ([details](docs/usage.md#collection-options)) +- **Keep your build to yourself.** buildprof.lalitm.com delivers the UI and + nothing else: your browser fetches the recording from localhost and processes + it in the page. Recordings do contain command lines and paths, so review one + before sending it to anyone. +- **Check the conformance suite.** Every change records a real build with each + supported build system and checks the resulting trace. + ([details](docs/build-systems.md)) ## Why use Buildprof? -Build tools generally explain only the work they manage themselves. Cargo -timings cannot break down an arbitrary `build.rs` script; Ninja cannot see -inside commands it launches; compiler traces describe one compiler invocation -rather than the build around it. +Build tools generally explain only the work they manage themselves: -Buildprof follows the complete process tree, so the same view includes the -build system, compilers, linkers, code generators, and arbitrary tools launched -along the way. You can see how their work fits together and where the build -spends its time. +- **Cargo timings** cannot break down an arbitrary `build.rs` script. +- **Ninja** cannot see inside the commands it launches. +- **Compiler traces** describe a single compiler invocation rather than the + build around it. + +Buildprof follows the complete process tree instead, so the build system, +compilers, linkers, code generators and whatever else the build launches all +land in one timeline, and you can see how their work fits together. ## Install Install on your Linux build machine using whichever method you prefer. -**Shell installer**: +
+ +Shell installer + +

```bash curl -fsSL https://buildprof.lalitm.com/install.sh | sh ``` +
-**Homebrew**: +
+ +Homebrew, mise, or Cargo + +

```bash brew install lalitmaganti/tap/buildprof -``` - -**mise**: - -```bash mise use -g github:LalitMaganti/buildprof +cargo install --locked buildprof # builds from source; needs Rust 1.91 or newer ``` +
-**Cargo** (builds from source; needs Rust 1.91 or newer): +
+ +Debian, Ubuntu, Fedora, and other .deb or .rpm distributions + +

-```bash -cargo install --locked buildprof -``` - -**Debian, Ubuntu, Fedora, and other `.deb` or `.rpm` distributions**: download -the package for your architecture from the -[latest release](https://github.com/LalitMaganti/buildprof/releases/latest) -and install it with `apt install ./buildprof_*.deb` or -`dnf install ./buildprof-*.rpm`. - -**Tarballs**: the same release page carries prebuilt binaries for x86_64 and -aarch64 Linux, both glibc and static musl. - -### Requirements - -**Recording builds is currently supported on Linux only.** The kernel or -container configuration must permit tracing child processes: Docker needs -`--cap-add SYS_PTRACE`, `kernel.yama.ptrace_scope` must be below 3, and -gVisor-style sandboxes cannot trace at all. Installing from source needs Rust -1.91 or newer. - -Existing recordings can be viewed on any platform in the -[web UI](https://buildprof.lalitm.com), without installing Buildprof. Viewing a -recording does not require the same operating system it was recorded on. - -## Usage - -### Recording a build - -Put `buildprof --` in front of the build command. Choose another output path -or disable automatic opening when needed: +Download the package for your architecture from the +[latest release](https://github.com/LalitMaganti/buildprof/releases/latest): ```bash -buildprof -o clean-build.buildprof --no-open -- ninja -C out -``` - -### Recording in GitHub Actions - -On a Linux runner, install your build tools, then record the build: - -```yaml -steps: - - uses: actions/checkout@v7 - - uses: LalitMaganti/buildprof@v0.2.6 - with: - command: | - cmake -S . -B build - cmake --build build -j2 +apt install ./buildprof_*.deb +dnf install ./buildprof-*.rpm ``` +
-The job summary links to the recording artifact, including when the build -fails. Download it and open it in [the web UI](https://buildprof.lalitm.com). -See [action inputs](action.yml) for version and retention settings; use a -unique `artifact-name` for each matrix entry. For upgrades, use release tags -or release SHAs with Dependabot (see [action releases](RELEASING.md#github-action-releases)). +
+ +Tarballs + +

-### Opening recordings +The [release page](https://github.com/LalitMaganti/buildprof/releases/latest) +carries prebuilt binaries for x86_64 and aarch64 Linux, both glibc and static +musl. The musl build is the one to mount into a container. +
-Open an existing recording later with: - -```bash -buildprof open clean-build.buildprof -``` - -buildprof.lalitm.com only delivers the UI itself. Your browser fetches the -recording from localhost and processes it entirely in the page; no trace data -leaves your machine. - -Because the page comes from buildprof.lalitm.com and the recording from -localhost, the browser asks once whether the site may access other apps and -services on this device. Allow it. If you block it, the trace never loads. To -recover, allow it again in the site settings next to the address bar, under -"Apps on device" in Chrome or "Access this device" in Firefox, then run -`buildprof open` again. - -Recordings contain command lines and filesystem paths. Review them before -sending them to anyone. - -### Builds on a remote machine - -Over SSH there is no browser to launch, so Buildprof prints the port forward -to run from your own machine instead and waits for the browser to fetch the -trace: - -```bash -ssh -L 9001:127.0.0.1:9001 user@buildhost -``` - -VS Code Remote and JetBrains Gateway forward the port automatically. The wait -gives up after ten minutes; adjust it with `--wait `, where `0` waits -forever. Alternatively, copy the recording to your own machine and open it -in the [web UI](https://buildprof.lalitm.com). No local installation is needed -for viewing. - -### Collection options - -Process creation, commands, and timing are always recorded. File opens and -renames are also recorded by default; to reduce overhead on builds with lots -of filesystem activity, disable that layer: - -```bash -buildprof --no-file-events -- make -j6 -``` - -The process timeline remains available, but file lists and producer/consumer -links are unavailable. This skips filesystem interception itself, rather than -collecting and discarding events. The UI identifies recordings made this way. - -### Compiler details - -Process timing is usually the right level for understanding a build. When a -particular compiler or linker invocation needs a closer look, enable compiler -tracing: - -```bash -buildprof --compiler-traces -- cargo build -``` - -Buildprof currently imports Clang `-ftime-trace`, explicitly selected LLD -`--time-trace`, and nightly Rust self-profile data. These events appear as a -summary of active compiler threads with expandable per-thread phase tracks. -Compiler tracing can be combined with process-only recording: - -```bash -buildprof --no-file-events --compiler-traces -- ninja -C build -``` - -A build which invokes Clang through an absolute path currently bypasses -compiler tracing. Buildprof will still record the compiler process, but its -Clang and LLD internal phases will be absent. - -Compiler tracing can also change compiler cache keys or turn cache hits into -misses. Existing Rust compiler wrappers remain in the invocation chain, but -cache preservation is not guaranteed in this mode. - -## Build systems - -Buildprof follows the process tree, so it does not need to understand the -build system. These are exercised by the conformance suite on every change: - -- **Make**: compiles, archiving, linking, and renamed outputs. -- **CMake with Ninja**: the configure step and the Ninja build. -- **Meson with Ninja**: the setup step and the Ninja build. -- **Cargo**: `rustc` invocations and linking; nightly self-profile data with - `--compiler-traces`. -- **Go**: compile, assemble, and link. -- **npm**: offline build scripts, generated JavaScript, and renamed outputs. -- **Bazel**: local actions and generated-file dependencies in batch mode. -- **Buck2**: local actions and artifact flow through an isolated build daemon. - -Anything else that runs as a child process is recorded the same way: shell -scripts, code generators, wrapper scripts, and tools launched by the build. - -## Troubleshooting - -Buildprof only records the process tree under the command it runs. Work -outside that tree is not recorded. It usually appears as one long command -with nothing underneath it. - - -### Bazel, Gradle, and other daemon build systems - -Buildprof cannot see work sent to an existing daemon or a remote executor. -An existing daemon leaves only the client in the recording. If the recorded -command starts the daemon, recording continues until the daemon exits. -Disable the daemon or stop it before and after the build: - -```sh -bazel shutdown && buildprof -- sh -c 'bazel build //... ; bazel shutdown' -buildprof -- ./gradlew --no-daemon build -buck2 kill && buildprof -- sh -c 'buck2 build //... ; buck2 kill' -sccache --stop-server && buildprof -- sh -c 'cargo build; sccache --stop-server' -``` - - -### Builds inside Docker containers - -`docker run` hands the container to the Docker daemon. The container's -processes are outside the recorded tree. You can record inside the container -or use Podman. - -#### Record inside the container - -Download the static musl build on the host and mount it into the container. -It runs in any Linux image without changing the image. Use `aarch64` instead -of `x86_64` for ARM images, including Docker Desktop on Apple Silicon: - -```sh -mkdir -p ~/.cache/buildprof-linux -curl -fsSL https://github.com/LalitMaganti/buildprof/releases/latest/download/buildprof-x86_64-unknown-linux-musl.tar.xz \ - | tar -xJ -C ~/.cache/buildprof-linux --strip-components=1 -``` - -Run the build through the mounted binary. Docker needs `--cap-add SYS_PTRACE` -to allow tracing. The recording goes in the mounted project directory: - -```sh -docker run --rm --cap-add SYS_PTRACE \ - -v ~/.cache/buildprof-linux:/opt/buildprof:ro \ - -v "$PWD:/src" -w /src my-build-image \ - /opt/buildprof/buildprof -o build.buildprof --no-open -- cargo build -buildprof open build.buildprof -``` - -Use `--no-open` because the host browser cannot reach the container's -localhost, where the recording would be served. Open the recording with -`buildprof open` on the host. To make the binary a permanent part of the image, -copy it in the Dockerfile. - - -#### Use Podman - -Podman has no daemon. Container processes are descendants of `podman run`, -so Buildprof records them without image changes. Podman accepts the same -`run` options as Docker: - -```sh -podman info > /dev/null -buildprof -- podman run --rm -v "$PWD:/src" -w /src my-build-image cargo build -``` - -> [!IMPORTANT] -> Run `podman info` before recording. The first Podman command after a reboot -> or `podman system migrate` sets up the rootless user namespace using setuid -> `newuidmap`. It cannot gain privileges while traced, causing -> `newuidmap: write to uid_map failed: Operation not permitted`. -> Any Podman command run outside the recording sets up the namespace. -> Later commands reuse it. - -Commands and file paths from inside the container are recorded as the -container sees them, such as `/src/target/...` rather than the host path. - -## How it works - -On Linux, Buildprof launches the command under `ptrace` and follows process -creation, execution, and exit through the complete descendant tree. A seccomp -filter lets it stop only for the filesystem operations it records instead of -paying the cost of intercepting every system call. - -The recorder writes a Perfetto protobuf trace directly. Perfetto provides the -storage format, query engine, and core timeline interactions; Buildprof adds -the build-specific view on top, including process ancestry, command types, -concurrency, file relationships, and optional compiler timing data. - -### Backwards compatibility - -Before 1.0, the CLI and the UI move in lockstep at the minor version: a -recording is meant to be viewed in a UI from the same 0.x series, and patch -releases never change the trace format. Every UI version stays deployed under -its own path, the CLI opens the one matching its version, and the UI links to -the matching series when it is handed a recording from a different one, so -nothing stops working, but the trace format may change between minor -releases. - -From 1.0 onward, the trace format is stable and compatibility is permanent: -any recording opens in every later UI, and newer UIs simply add features on -top of older recordings. - -### Self-hosting the UI - -Each release attaches `buildprof-ui-v.tar.zst`, the complete UI as -static files. Serve its contents from any web server and point the CLI at it: - -```bash -buildprof open --url https://ui.example.internal/v0.2.0 clean-build.buildprof -``` - -## Development - -See [CONTRIBUTING.md](CONTRIBUTING.md) for the layout and the UI workflow. -Run the complete conformance suite in the Linux development container: - -```bash -just bootstrap -just test -``` - -For a quick host-side check of formatting, lints, unit tests, and package -contents: +### Requirements -```bash -just release-check -``` +- **Recording needs Linux**, with a kernel and container configuration that + permits tracing child processes: + - Docker needs `--cap-add SYS_PTRACE`. + - `kernel.yama.ptrace_scope` must be below 3. + - gVisor-style sandboxes cannot trace at all. +- **Viewing needs nothing.** Recordings open on any platform in the + [web UI](https://buildprof.lalitm.com), whatever they were recorded on, with + Buildprof installed or not. +- **Building from source needs Rust 1.91 or newer.** + +## Documentation + +- Reading a recording: + - [Investigation guide](docs/investigating-builds.md): finding bottlenecks, + following dependencies, and checking whether a change helped. + - [Guided tour](docs/ripgrep-tutorial.md): the example recording, explained. +- Recording one: + - [Recording and opening builds](docs/usage.md): GitHub Actions, remote + machines, collection options, compiler details. + - [Build systems](docs/build-systems.md): what the conformance suite covers. + - [Troubleshooting](docs/troubleshooting.md): daemon build systems and + containers. +- Under the hood: + - [How it works](docs/internals.md): tracing, the trace format, its + compatibility rules, and self-hosting the UI. + - [CONTRIBUTING.md](CONTRIBUTING.md): building, testing, and the UI + workflow. ## License diff --git a/RELEASING.md b/RELEASING.md index 092c27b..eeaa69b 100644 --- a/RELEASING.md +++ b/RELEASING.md @@ -23,7 +23,7 @@ Also create the empty `homebrew-tap` repository. Re-record `examples/*.buildprof` with this version if the trace format changed (see `examples/README.md`). 2. Run `just release-prepare` to update the action's default recorder version - and README release-tag example from `Cargo.toml`. Run `cargo check` to + and the docs' release-tag example from `Cargo.toml`. Run `cargo check` to refresh `Cargo.lock`. Then run `just release-check`, commit, push, and wait for CI. 3. Tag and push: `git tag v0.2.2 && git push origin v0.2.2`. @@ -60,7 +60,7 @@ Manual follow-ups: ## GitHub Action releases The action ships under the crate's `vX.Y.Z` tags. `just release-prepare` -updates its recorder default and README example from `Cargo.toml`; CI and +updates its recorder default and the docs example from `Cargo.toml`; CI and release checks catch drift. Keep release tags fixed. Consumers can use tags or release SHAs with Dependabot for upgrade PRs. @@ -78,10 +78,10 @@ tests the released action's default recorder on x86_64 and ARM64 after publicati cannot express are edited in by hand, and `allow-dirty = ["ci"]` in `dist-workspace.toml` stops dist from overwriting them: -- the release metadata check validates the action default and README example; +- the release metadata check validates the action default and the docs example; - the GitHub release is created as a draft titled `Buildprof `; - `infra/release-notes` replaces the install section of dist's notes with - the README's commands; the changelog and the download table stay as dist + the documented commands; the changelog and the download table stay as dist wrote them. To upgrade dist: bump `cargo-dist-version`, run `dist generate` with diff --git a/docs/build-systems.md b/docs/build-systems.md new file mode 100644 index 0000000..d2264e9 --- /dev/null +++ b/docs/build-systems.md @@ -0,0 +1,17 @@ +# Build systems + +Buildprof follows the process tree, so it does not need to understand the +build system. These are exercised by the conformance suite on every change: + +- **Make**: compiles, archiving, linking, and renamed outputs. +- **CMake with Ninja**: the configure step and the Ninja build. +- **Meson with Ninja**: the setup step and the Ninja build. +- **Cargo**: `rustc` invocations and linking; nightly self-profile data with + `--compiler-traces`. +- **Go**: compile, assemble, and link. +- **npm**: offline build scripts, generated JavaScript, and renamed outputs. +- **Bazel**: local actions and generated-file dependencies in batch mode. +- **Buck2**: local actions and artifact flow through an isolated build daemon. + +Anything else that runs as a child process is recorded the same way: shell +scripts, code generators, wrapper scripts, and tools launched by the build. diff --git a/docs/internals.md b/docs/internals.md new file mode 100644 index 0000000..bbb5a61 --- /dev/null +++ b/docs/internals.md @@ -0,0 +1,34 @@ +# How Buildprof works + +On Linux, Buildprof launches the command under `ptrace` and follows process +creation, execution, and exit through the complete descendant tree. A seccomp +filter lets it stop only for the filesystem operations it records instead of +paying the cost of intercepting every system call. + +The recorder writes a Perfetto protobuf trace directly. Perfetto provides the +storage format, query engine, and core timeline interactions; Buildprof adds +the build-specific view on top, including process ancestry, command types, +concurrency, file relationships, and optional compiler timing data. + +## Backwards compatibility + +Before 1.0, the CLI and the UI move in lockstep at the minor version: a +recording is meant to be viewed in a UI from the same 0.x series, and patch +releases never change the trace format. Every UI version stays deployed under +its own path, the CLI opens the one matching its version, and the UI links to +the matching series when it is handed a recording from a different one, so +nothing stops working, but the trace format may change between minor +releases. + +From 1.0 onward, the trace format is stable and compatibility is permanent: +any recording opens in every later UI, and newer UIs simply add features on +top of older recordings. + +## Self-hosting the UI + +Each release attaches `buildprof-ui-v.tar.zst`, the complete UI as +static files. Serve its contents from any web server and point the CLI at it: + +```bash +buildprof open --url https://ui.example.internal/v0.2.0 clean-build.buildprof +``` diff --git a/docs/investigating-builds.md b/docs/investigating-builds.md index 4fbe6bb..cac4159 100644 --- a/docs/investigating-builds.md +++ b/docs/investigating-builds.md @@ -29,7 +29,7 @@ Look for long-running commands, gaps before compilation starts, and processes which keep running after everything else finishes. Click a process to inspect its command line, working directory, lifetime, and exit status. For example, a long linker invocation is a reason to inspect its LTO flags and then look -inside the linker with [compiler tracing](#compiler-details). +inside the linker with [compiler tracing](usage.md#compiler-details). A parent's lifetime includes time spent waiting for children: follow the process tree down to see which command is still running. Process duration @@ -160,7 +160,7 @@ inputs, and full argument list, including libraries and any LTO flags. Keep response files referenced by `@file` available too. Both `clang` and `clang++` must be on `PATH` for Buildprof to install its Clang wrappers. -See [compiler support and limitations](../README.md#compiler-details) for +See [compiler support and limitations](usage.md#compiler-details) for supported compilers and the effects on compiler caches. ## Check whether a change helped diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md new file mode 100644 index 0000000..54ced21 --- /dev/null +++ b/docs/troubleshooting.md @@ -0,0 +1,78 @@ +# Troubleshooting + +Buildprof only records the process tree under the command it runs. Work +outside that tree is not recorded. It usually appears as one long command +with nothing underneath it. + + +## Bazel, Gradle, and other daemon build systems + +Buildprof cannot see work sent to an existing daemon or a remote executor. +An existing daemon leaves only the client in the recording. If the recorded +command starts the daemon, recording continues until the daemon exits. +Disable the daemon or stop it before and after the build: + +```sh +bazel shutdown && buildprof -- sh -c 'bazel build //... ; bazel shutdown' +buildprof -- ./gradlew --no-daemon build +buck2 kill && buildprof -- sh -c 'buck2 build //... ; buck2 kill' +sccache --stop-server && buildprof -- sh -c 'cargo build; sccache --stop-server' +``` + + +## Builds inside Docker containers + +`docker run` hands the container to the Docker daemon. The container's +processes are outside the recorded tree. You can record inside the container +or use Podman. + +## Record inside the container + +Download the static musl build on the host and mount it into the container. +It runs in any Linux image without changing the image. Use `aarch64` instead +of `x86_64` for ARM images, including Docker Desktop on Apple Silicon: + +```sh +mkdir -p ~/.cache/buildprof-linux +curl -fsSL https://github.com/LalitMaganti/buildprof/releases/latest/download/buildprof-x86_64-unknown-linux-musl.tar.xz \ + | tar -xJ -C ~/.cache/buildprof-linux --strip-components=1 +``` + +Run the build through the mounted binary. Docker needs `--cap-add SYS_PTRACE` +to allow tracing. The recording goes in the mounted project directory: + +```sh +docker run --rm --cap-add SYS_PTRACE \ + -v ~/.cache/buildprof-linux:/opt/buildprof:ro \ + -v "$PWD:/src" -w /src my-build-image \ + /opt/buildprof/buildprof -o build.buildprof --no-open -- cargo build +buildprof open build.buildprof +``` + +Use `--no-open` because the host browser cannot reach the container's +localhost, where the recording would be served. Open the recording with +`buildprof open` on the host. To make the binary a permanent part of the image, +copy it in the Dockerfile. + + +## Use Podman + +Podman has no daemon. Container processes are descendants of `podman run`, +so Buildprof records them without image changes. Podman accepts the same +`run` options as Docker: + +```sh +podman info > /dev/null +buildprof -- podman run --rm -v "$PWD:/src" -w /src my-build-image cargo build +``` + +> [!IMPORTANT] +> Run `podman info` before recording. The first Podman command after a reboot +> or `podman system migrate` sets up the rootless user namespace using setuid +> `newuidmap`. It cannot gain privileges while traced, causing +> `newuidmap: write to uid_map failed: Operation not permitted`. +> Any Podman command run outside the recording sets up the namespace. +> Later commands reuse it. + +Commands and file paths from inside the container are recorded as the +container sees them, such as `/src/target/...` rather than the host path. diff --git a/docs/usage.md b/docs/usage.md new file mode 100644 index 0000000..0aca291 --- /dev/null +++ b/docs/usage.md @@ -0,0 +1,112 @@ +# Recording and opening builds + +Everything `buildprof` can do beyond `buildprof -- `, which is +covered by the [quick start](../README.md#quick-start). + +## Recording a build + +Put `buildprof --` in front of the build command. Choose another output path +or disable automatic opening when needed: + +```bash +buildprof -o clean-build.buildprof --no-open -- ninja -C out +``` + +## Recording in GitHub Actions + +On a Linux runner, install your build tools, then record the build: + +```yaml +steps: + - uses: actions/checkout@v7 + - uses: LalitMaganti/buildprof@v0.2.6 + with: + command: | + cmake -S . -B build + cmake --build build -j2 +``` + +The job summary links to the recording artifact, including when the build +fails. Download it and open it in [the web UI](https://buildprof.lalitm.com). +See [action inputs](../action.yml) for version and retention settings; use a +unique `artifact-name` for each matrix entry. For upgrades, use release tags +or release SHAs with Dependabot (see [action releases](../RELEASING.md#github-action-releases)). + +## Opening recordings + +Open an existing recording later with: + +```bash +buildprof open clean-build.buildprof +``` + +buildprof.lalitm.com only delivers the UI itself. Your browser fetches the +recording from localhost and processes it entirely in the page; no trace data +leaves your machine. + +Because the page comes from buildprof.lalitm.com and the recording from +localhost, the browser asks once whether the site may access other apps and +services on this device. Allow it. If you block it, the trace never loads. To +recover, allow it again in the site settings next to the address bar, under +"Apps on device" in Chrome or "Access this device" in Firefox, then run +`buildprof open` again. + +Recordings contain command lines and filesystem paths. Review them before +sending them to anyone. + +## Builds on a remote machine + +Over SSH there is no browser to launch, so Buildprof prints the port forward +to run from your own machine instead and waits for the browser to fetch the +trace: + +```bash +ssh -L 9001:127.0.0.1:9001 user@buildhost +``` + +VS Code Remote and JetBrains Gateway forward the port automatically. The wait +gives up after ten minutes; adjust it with `--wait `, where `0` waits +forever. Alternatively, copy the recording to your own machine and open it +in the [web UI](https://buildprof.lalitm.com). No local installation is needed +for viewing. + +## Collection options + +Process creation, commands, and timing are always recorded. File opens and +renames are also recorded by default; to reduce overhead on builds with lots +of filesystem activity, disable that layer: + +```bash +buildprof --no-file-events -- make -j6 +``` + +The process timeline remains available, but file lists and producer/consumer +links are unavailable. This skips filesystem interception itself, rather than +collecting and discarding events. The UI identifies recordings made this way. + +## Compiler details + +Process timing is usually the right level for understanding a build. When a +particular compiler or linker invocation needs a closer look, enable compiler +tracing: + +```bash +buildprof --compiler-traces -- cargo build +``` + +Buildprof currently imports Clang `-ftime-trace`, explicitly selected LLD +`--time-trace`, and nightly Rust self-profile data. These events appear as a +summary of active compiler threads with expandable per-thread phase tracks. +Compiler tracing can be combined with process-only recording: + +```bash +buildprof --no-file-events --compiler-traces -- ninja -C build +``` + +A build which invokes Clang through an absolute path currently bypasses +compiler tracing. Buildprof will still record the compiler process, but its +Clang and LLD internal phases will be absent. + +Compiler tracing can also change compiler cache keys or turn cache hits into +misses. Existing Rust compiler wrappers remain in the invocation chain, but +cache preservation is not guaranteed in this mode. diff --git a/infra/buildprof.lalitm.com/assemble-site b/infra/buildprof.lalitm.com/assemble-site index 8765049..819ba1d 100755 --- a/infra/buildprof.lalitm.com/assemble-site +++ b/infra/buildprof.lalitm.com/assemble-site @@ -123,13 +123,15 @@ def main() -> int: # A short, stable installer URL; GitHub's "latest" redirect keeps it # pointing at the newest release without a redeploy. The /diagnose/ links # are printed by the CLI, so installed binaries keep working when the - # README's headings change; update the anchors here instead. - readme = "https://github.com/LalitMaganti/buildprof" + # troubleshooting headings change; update the anchors here instead. + troubleshooting = ( + "https://github.com/LalitMaganti/buildprof/blob/main/docs/troubleshooting.md" + ) (site / "_redirects").write_text( "/install.sh https://github.com/LalitMaganti/buildprof/releases/latest/download/buildprof-installer.sh 302\n" - f"/diagnose/containers {readme}#builds-inside-docker-containers 302\n" - f"/diagnose/daemons {readme}#bazel-gradle-and-other-daemon-build-systems 302\n" - f"/diagnose/podman {readme}#use-podman 302\n" + f"/diagnose/containers {troubleshooting}#builds-inside-docker-containers 302\n" + f"/diagnose/daemons {troubleshooting}#bazel-gradle-and-other-daemon-build-systems 302\n" + f"/diagnose/podman {troubleshooting}#use-podman 302\n" ) if args.headers is not None: diff --git a/infra/prepare-action-release b/infra/prepare-action-release index 643d86a..776c851 100644 --- a/infra/prepare-action-release +++ b/infra/prepare-action-release @@ -30,17 +30,17 @@ def replace_once(text: str, pattern: str, replacement: str) -> str: def prepare(root: Path, *, check: bool, allow_preview: bool) -> list[str]: package = tomllib.loads((root / "Cargo.toml").read_text())["package"] version = "v" + package["version"] - readme = root / "README.md" + usage = root / "docs/usage.md" reference = version if ( allow_preview and version == PREVIEW_VERSION - and f"{PREVIEW_REF} {PREVIEW_NOTE}" in readme.read_text() + and f"{PREVIEW_REF} {PREVIEW_NOTE}" in usage.read_text() ): reference = f"{PREVIEW_REF} {PREVIEW_NOTE}" replacements = { root / "action.yml": (r"(?<= default: )v[^\n]+", version), - readme: ( + usage: ( r"(?<= - uses: LalitMaganti/buildprof@)[^\n]+", reference, ), diff --git a/justfile b/justfile index 7e5f675..3609c52 100644 --- a/justfile +++ b/justfile @@ -38,7 +38,7 @@ test *args: check: uv run dev/test --collect-only -# Update the action default and README example after changing Cargo.toml's version. +# Update the action default and docs example after changing Cargo.toml's version. release-prepare: python3 infra/prepare-action-release