aya/test/README.md

# Aya Integration Tests

The aya integration test suite is a set of tests to ensure that
common usage behaviours work on real Linux distros

## Prerequisites

You'll need:

1. `rustup toolchain install nightly`
1. `rustup target add {aarch64,x86_64}-unknown-linux-musl`
1. `cargo install bpf-linker`
1. `libelf-dev` (`libelf-devel` on rpm-based distros)
1. `llvm` (for `llvm-objcopy`)
1. (virtualized only) `qemu`

## Usage

From the root of this repository:

### Native

```bash
cargo xtask integration-test local
```

### Virtualized

```bash
cargo xtask integration-test vm <KERNEL IMAGE>
```

### Writing an integration test

Tests should follow these guidelines:

- Rust eBPF code should live in `integration-ebpf/${NAME}.rs` and included in
  `integration-ebpf/Cargo.toml` and `integration-test/src/lib.rs` using
  `include_bytes_aligned!`.
- C eBPF code should live in `integration-test/bpf/${NAME}.bpf.c`. It should be
  added to the list of files in `integration-test/build.rs` and the list of
  constants in `integration-test/src/lib.rs` using `include_bytes_aligned!`.
- Tests should be added to `integration-test/tests`.
- You may add a new module, or use an existing one.
- Test functions should not return `anyhow::Result<()>` since this produces
  errors without stack traces. Prefer to `panic!` instead.
.github: Add markdownlint This adds a linter to catch common markdown formatting errors. The linter used is markdownlint-cli2 which is available on all platforms and has an associated Github Action to automate these checks in CI. Configuration is checked in at .markdownlint-cli2.yaml. You may run the check locally using `markdownlint-cli2`. Or you may install the extension for VSCode: DavidAnson.vscode-markdownlint Signed-off-by: Dave Tucker <dave@dtucker.co.uk> 1 year ago			`# Aya Integration Tests`
test: Add regression tests This uses a mix of rust-script, bash, qemu and a test runner called RTF to add a regression test suite... and wires it into GitHub Actions Signed-off-by: Dave Tucker <dave@dtucker.co.uk> 3 years ago
test: Replace RTF with Rust This commit replaces the existing RTF test runner with a simple rust binary package called - integration-test. integration-test depends on integration-ebpf, which contains test eBPF code written in Rust and C. `cargo xtask build-integration-test-ebpf` can be used to build this code and supress rust-analyzer warnings. It does require `bpf-linker`, but that is highly likely to be available to developers of Aya. It also requires a checkout of `libbpf` to extract headers like bpf-helpers.h. Since everything is compiled into a single binary, it can be run be run locally using `cargo xtask integration-test` or remotely using `./run.sh` which re-uses the bash script from the old test framework to spawn a VM in which to run the tests. Signed-off-by: Dave Tucker <dave@dtucker.co.uk> 2 years ago			`The aya integration test suite is a set of tests to ensure that`
test: Add regression tests This uses a mix of rust-script, bash, qemu and a test runner called RTF to add a regression test suite... and wires it into GitHub Actions Signed-off-by: Dave Tucker <dave@dtucker.co.uk> 3 years ago			`common usage behaviours work on real Linux distros`

integration-test: Implement running on VMs Implements running integration tests on multiple VMs with arbitrary kernel images using `cargo xtask integration-test vm ...`. This changes our coverage from 6.2 to 6.1 and 6.4. 1 year ago			`## Prerequisites`
test: Add regression tests This uses a mix of rust-script, bash, qemu and a test runner called RTF to add a regression test suite... and wires it into GitHub Actions Signed-off-by: Dave Tucker <dave@dtucker.co.uk> 3 years ago
integration-test: Implement running on VMs Implements running integration tests on multiple VMs with arbitrary kernel images using `cargo xtask integration-test vm ...`. This changes our coverage from 6.2 to 6.1 and 6.4. 1 year ago			`You'll need:`
ci: try running regression tests in a container Signed-off-by: Dave Tucker <dave@dtucker.co.uk> 3 years ago
integration-test: Implement running on VMs Implements running integration tests on multiple VMs with arbitrary kernel images using `cargo xtask integration-test vm ...`. This changes our coverage from 6.2 to 6.1 and 6.4. 1 year ago			1. `rustup toolchain install nightly`
			1. `rustup target add {aarch64,x86_64}-unknown-linux-musl`
test: Replace RTF with Rust This commit replaces the existing RTF test runner with a simple rust binary package called - integration-test. integration-test depends on integration-ebpf, which contains test eBPF code written in Rust and C. `cargo xtask build-integration-test-ebpf` can be used to build this code and supress rust-analyzer warnings. It does require `bpf-linker`, but that is highly likely to be available to developers of Aya. It also requires a checkout of `libbpf` to extract headers like bpf-helpers.h. Since everything is compiled into a single binary, it can be run be run locally using `cargo xtask integration-test` or remotely using `./run.sh` which re-uses the bash script from the old test framework to spawn a VM in which to run the tests. Signed-off-by: Dave Tucker <dave@dtucker.co.uk> 2 years ago			1. `cargo install bpf-linker`
tests: update instructions on setting up and running tests - Update the list of required packages. - Update the command for executing tests in a VM. 10 months ago			1. `libelf-dev` (`libelf-devel` on rpm-based distros)
			1. `llvm` (for `llvm-objcopy`)
integration-test: Implement running on VMs Implements running integration tests on multiple VMs with arbitrary kernel images using `cargo xtask integration-test vm ...`. This changes our coverage from 6.2 to 6.1 and 6.4. 1 year ago			1. (virtualized only) `qemu`
ci: try running regression tests in a container Signed-off-by: Dave Tucker <dave@dtucker.co.uk> 3 years ago
test: Add regression tests This uses a mix of rust-script, bash, qemu and a test runner called RTF to add a regression test suite... and wires it into GitHub Actions Signed-off-by: Dave Tucker <dave@dtucker.co.uk> 3 years ago			`## Usage`

test: Replace RTF with Rust This commit replaces the existing RTF test runner with a simple rust binary package called - integration-test. integration-test depends on integration-ebpf, which contains test eBPF code written in Rust and C. `cargo xtask build-integration-test-ebpf` can be used to build this code and supress rust-analyzer warnings. It does require `bpf-linker`, but that is highly likely to be available to developers of Aya. It also requires a checkout of `libbpf` to extract headers like bpf-helpers.h. Since everything is compiled into a single binary, it can be run be run locally using `cargo xtask integration-test` or remotely using `./run.sh` which re-uses the bash script from the old test framework to spawn a VM in which to run the tests. Signed-off-by: Dave Tucker <dave@dtucker.co.uk> 2 years ago			`From the root of this repository:`
test: Add regression tests This uses a mix of rust-script, bash, qemu and a test runner called RTF to add a regression test suite... and wires it into GitHub Actions Signed-off-by: Dave Tucker <dave@dtucker.co.uk> 3 years ago
test: Replace RTF with Rust This commit replaces the existing RTF test runner with a simple rust binary package called - integration-test. integration-test depends on integration-ebpf, which contains test eBPF code written in Rust and C. `cargo xtask build-integration-test-ebpf` can be used to build this code and supress rust-analyzer warnings. It does require `bpf-linker`, but that is highly likely to be available to developers of Aya. It also requires a checkout of `libbpf` to extract headers like bpf-helpers.h. Since everything is compiled into a single binary, it can be run be run locally using `cargo xtask integration-test` or remotely using `./run.sh` which re-uses the bash script from the old test framework to spawn a VM in which to run the tests. Signed-off-by: Dave Tucker <dave@dtucker.co.uk> 2 years ago			`### Native`
test: Add regression tests This uses a mix of rust-script, bash, qemu and a test runner called RTF to add a regression test suite... and wires it into GitHub Actions Signed-off-by: Dave Tucker <dave@dtucker.co.uk> 3 years ago
.github: Add markdownlint This adds a linter to catch common markdown formatting errors. The linter used is markdownlint-cli2 which is available on all platforms and has an associated Github Action to automate these checks in CI. Configuration is checked in at .markdownlint-cli2.yaml. You may run the check locally using `markdownlint-cli2`. Or you may install the extension for VSCode: DavidAnson.vscode-markdownlint Signed-off-by: Dave Tucker <dave@dtucker.co.uk> 1 year ago			```bash
integration-test: Implement running on VMs Implements running integration tests on multiple VMs with arbitrary kernel images using `cargo xtask integration-test vm ...`. This changes our coverage from 6.2 to 6.1 and 6.4. 1 year ago			`cargo xtask integration-test local`
test: Add regression tests This uses a mix of rust-script, bash, qemu and a test runner called RTF to add a regression test suite... and wires it into GitHub Actions Signed-off-by: Dave Tucker <dave@dtucker.co.uk> 3 years ago			```
test: Replace RTF with Rust This commit replaces the existing RTF test runner with a simple rust binary package called - integration-test. integration-test depends on integration-ebpf, which contains test eBPF code written in Rust and C. `cargo xtask build-integration-test-ebpf` can be used to build this code and supress rust-analyzer warnings. It does require `bpf-linker`, but that is highly likely to be available to developers of Aya. It also requires a checkout of `libbpf` to extract headers like bpf-helpers.h. Since everything is compiled into a single binary, it can be run be run locally using `cargo xtask integration-test` or remotely using `./run.sh` which re-uses the bash script from the old test framework to spawn a VM in which to run the tests. Signed-off-by: Dave Tucker <dave@dtucker.co.uk> 2 years ago
			`### Virtualized`

.github: Add markdownlint This adds a linter to catch common markdown formatting errors. The linter used is markdownlint-cli2 which is available on all platforms and has an associated Github Action to automate these checks in CI. Configuration is checked in at .markdownlint-cli2.yaml. You may run the check locally using `markdownlint-cli2`. Or you may install the extension for VSCode: DavidAnson.vscode-markdownlint Signed-off-by: Dave Tucker <dave@dtucker.co.uk> 1 year ago			```bash
tests: update instructions on setting up and running tests - Update the list of required packages. - Update the command for executing tests in a VM. 10 months ago			`cargo xtask integration-test vm <KERNEL IMAGE>`
test: Add regression tests This uses a mix of rust-script, bash, qemu and a test runner called RTF to add a regression test suite... and wires it into GitHub Actions Signed-off-by: Dave Tucker <dave@dtucker.co.uk> 3 years ago			```
integration-test: Remove integration-test-macros This doesn't add any value; use `cargo build --tests` with `--message-format=json` instead; parse the output using `cargo_metadata` to discover the location of the test binary. Move test/integration-test/src/tests -> test/integration-test/tests to conform to https://doc.rust-lang.org/book/ch11-03-test-organization.html#integration-tests. 1 year ago
			`### Writing an integration test`
test: Add regression tests This uses a mix of rust-script, bash, qemu and a test runner called RTF to add a regression test suite... and wires it into GitHub Actions Signed-off-by: Dave Tucker <dave@dtucker.co.uk> 3 years ago
test: Replace RTF with Rust This commit replaces the existing RTF test runner with a simple rust binary package called - integration-test. integration-test depends on integration-ebpf, which contains test eBPF code written in Rust and C. `cargo xtask build-integration-test-ebpf` can be used to build this code and supress rust-analyzer warnings. It does require `bpf-linker`, but that is highly likely to be available to developers of Aya. It also requires a checkout of `libbpf` to extract headers like bpf-helpers.h. Since everything is compiled into a single binary, it can be run be run locally using `cargo xtask integration-test` or remotely using `./run.sh` which re-uses the bash script from the old test framework to spawn a VM in which to run the tests. Signed-off-by: Dave Tucker <dave@dtucker.co.uk> 2 years ago			`Tests should follow these guidelines:`
test: Add regression tests This uses a mix of rust-script, bash, qemu and a test runner called RTF to add a regression test suite... and wires it into GitHub Actions Signed-off-by: Dave Tucker <dave@dtucker.co.uk> 3 years ago
integration-test: Remove integration-test-macros This doesn't add any value; use `cargo build --tests` with `--message-format=json` instead; parse the output using `cargo_metadata` to discover the location of the test binary. Move test/integration-test/src/tests -> test/integration-test/tests to conform to https://doc.rust-lang.org/book/ch11-03-test-organization.html#integration-tests. 1 year ago			- Rust eBPF code should live in `integration-ebpf/${NAME}.rs` and included in
integration-test: compile Rust probes using build.rs 1 year ago			`integration-ebpf/Cargo.toml` and `integration-test/src/lib.rs` using
			`include_bytes_aligned!`.
integration-test: compile C probes using build.rs - Add libbpf as a submodule. This prevents having to plumb its location around (which can't be passed to Cargo build scripts) and also controls the version against which codegen has run. - Move bpf written in C to the integration-test crate and define constants for each probe. - Remove magic; each C source file must be directly enumerated in the build script and in lib.rs. 1 year ago			- C eBPF code should live in `integration-test/bpf/${NAME}.bpf.c`. It should be
			added to the list of files in `integration-test/build.rs` and the list of
			constants in `integration-test/src/lib.rs` using `include_bytes_aligned!`.
integration-test: Remove integration-test-macros This doesn't add any value; use `cargo build --tests` with `--message-format=json` instead; parse the output using `cargo_metadata` to discover the location of the test binary. Move test/integration-test/src/tests -> test/integration-test/tests to conform to https://doc.rust-lang.org/book/ch11-03-test-organization.html#integration-tests. 1 year ago			- Tests should be added to `integration-test/tests`.
			`- You may add a new module, or use an existing one.`
.github: Add markdownlint This adds a linter to catch common markdown formatting errors. The linter used is markdownlint-cli2 which is available on all platforms and has an associated Github Action to automate these checks in CI. Configuration is checked in at .markdownlint-cli2.yaml. You may run the check locally using `markdownlint-cli2`. Or you may install the extension for VSCode: DavidAnson.vscode-markdownlint Signed-off-by: Dave Tucker <dave@dtucker.co.uk> 1 year ago			- Test functions should not return `anyhow::Result<()>` since this produces
			errors without stack traces. Prefer to `panic!` instead.