"integration tests" as defined by Cargo produce a binary per file in the
tests directory. This is really not what we want and has a number of
downsides, but the main one is binary size.
Before:
tamird@pc:~/src/aya$ cargo xtask build-integration-test | xargs ls -lah
Finished dev [unoptimized + debuginfo] target(s) in 0.05s
Running `target/debug/xtask build-integration-test`
Compiling integration-test v0.1.0 (/home/tamird/src/aya/test/integration-test)
Finished dev [unoptimized + debuginfo] target(s) in 0.68s
-rwxrwxr-x 1 tamird tamird 34M Jul 12 15:21 /home/tamird/src/aya/target/debug/deps/bpf_probe_read-e03eb905a5e6209c
-rwxrwxr-x 1 tamird tamird 35M Jul 12 15:21 /home/tamird/src/aya/target/debug/deps/btf_relocations-57a4fbb38bf06064
-rwxrwxr-x 1 tamird tamird 31M Jul 12 15:21 /home/tamird/src/aya/target/debug/deps/elf-98b7a32d6d04effb
-rwxrwxr-x 1 tamird tamird 6.9M Jul 12 15:21 /home/tamird/src/aya/target/debug/deps/integration_test-0dd55ce96bfdad77
-rwxrwxr-x 1 tamird tamird 34M Jul 12 15:21 /home/tamird/src/aya/target/debug/deps/load-0718562e85b86d03
-rwxrwxr-x 1 tamird tamird 40M Jul 12 15:21 /home/tamird/src/aya/target/debug/deps/log-dbf355f9ea34068a
-rwxrwxr-x 1 tamird tamird 36M Jul 12 15:21 /home/tamird/src/aya/target/debug/deps/rbpf-89a1bb848fa5cc3c
-rwxrwxr-x 1 tamird tamird 34M Jul 12 15:21 /home/tamird/src/aya/target/debug/deps/relocations-cfe655c3bb413d8b
-rwxrwxr-x 1 tamird tamird 34M Jul 12 15:21 /home/tamird/src/aya/target/debug/deps/smoke-ccd3974180a3fd29
After:
tamird@pc:~/src/aya$ cargo xtask build-integration-test | xargs ls -lah
Finished dev [unoptimized + debuginfo] target(s) in 0.05s
Running `target/debug/xtask build-integration-test`
Compiling integration-test v0.1.0 (/home/tamird/src/aya/test/integration-test)
Finished dev [unoptimized + debuginfo] target(s) in 0.90s
-rwxrwxr-x 1 tamird tamird 47M Jul 12 15:21 /home/tamird/src/aya/target/debug/deps/integration_test-0dd55ce96bfdad77
Since we plan to run these tests in a VM, copying 10x fewer bytes seems
like a win.
Use the environment variable AYA_BUILD_INTEGRATION_BPF to indicate to
the build script that it should *actually* build bpf, otherwise emitting
empty files.
This allows metadata builds to skip costly build steps without
sacrificing ergonomics; all compile-time tools such as cargo clippy work
out of the box.
Cargo even gives each of these builds (depending on the value of the
environment variable) its own cache key, so they do not invalidate each
other when the user alternates between metadata and real builds.
This allows the lint action to move out of the VM.
- Add libbpf as a submodule. This prevents having to plumb its location
around (which can't be passed to Cargo build scripts) and also
controls the version against which codegen has run.
- Move bpf written in C to the integration-test crate and define
constants for each probe.
- Remove magic; each C source file must be directly enumerated in the
build script and in lib.rs.
Replace all `assert!(matches!(..))` with `assert_matches!(..)`.
Remove the now-unused build-integration-test xtask command whose logic
doesn't match that of the build-and-run command.
This doesn't add any value; use `cargo build --tests` with
`--message-format=json` instead; parse the output using `cargo_metadata`
to discover the location of the test binary.
Move test/integration-test/src/tests -> test/integration-test/tests to
conform to
https://doc.rust-lang.org/book/ch11-03-test-organization.html#integration-tests.
This change does a few things:
- it fixes a bug in the wrappers, where we were expecting the kernel to
return len=1 for b"\0" where it instead returns 0 and doesn't write
out the NULL terminator
- it makes the helpers more robust by hardcoding bound checks in
assembly so that LLVM optimizations can't transform the checks in a
way that the verifier can't understand.
- it adds integration tests
Before this change:
```
error[E0382]: use of moved value: `no_copy`
--> test/integration-ebpf/src/log.rs:35:9
|
33 | let no_copy = NoCopy {};
| ------- move occurs because `no_copy` has type `NoCopy`, which does not implement the `Copy` trait
34 |
35 | debug!(&ctx, "{:x}", no_copy.consume());
| ^^^^^^^^^^^^^^^^^^^^^-------^---------^
| | | |
| | | `no_copy` moved due to this method call
| | use occurs due to use in closure
| value used here after move
|
note: `NoCopy::consume` takes ownership of the receiver `self`, which moves `no_copy`
--> test/integration-ebpf/src/log.rs:28:24
|
28 | fn consume(self) -> u64 {
| ^^^^
= note: this error originates in the macro `debug` (in Nightly builds, run with -Z macro-backtrace for more info)
For more information about this error, try `rustc --explain E0382`.
error: could not compile `integration-ebpf` (bin "log") due to previous error
```
This fix aya wrong logic causing non entrypoint functions to not have
any BTF relocations working.
Also fix missing section_offset computation for instruction offset in
multiple spots.
Having separate format hints and tokens per IP address family is
unnecessary, since they are represented by different types and we handle
format hints for each type separately. So we can just have one format
hint.
Also, we should be consistent with the format strings grammar in
Rust[0]. The `type` token, which is mapped to formatting traits, usually
consists of one letter[1] (and optional `?` for `Debug` trait, but that
doesn't matter for us). It shouldn't consist of multiple letters. Our
`:ipv4` and `:ipv6` tokens were clearly breaking that convention, so we
should rather switch to something with one letter - hence `:i`.
[0] https://doc.rust-lang.org/std/fmt/#syntax
[1] https://doc.rust-lang.org/std/fmt/#formatting-traits
Now that bpf-linker uses llvm 16, the easiest way is to use Fedora 38
Beta with the testing repos as they have it, without resorting to
Rawhide.
See https://packages.fedoraproject.org/pkgs/llvm/llvm/.
This commit adds from_pin() which allows the creation of a Program
from a path on bpffs. This is useful to be able to call `attach` or
other APIs for programs that are already loaded to the kernel.
This differs from #444 since it implements this on the concrete program
type, not the Program enum, allowing the user to pass in any additional
context that isn't available from bpf_prog_info.
Signed-off-by: Dave Tucker <dave@dtucker.co.uk>
Before this chane, the check was always negative if the minor version
was less then 9. So, for example, the smoke test was skipped for kernel
6.1:
```
skipping as 6.1 does not meet version requirement of 5.9
```
Signed-off-by: Michal Rostecki <vadorovsky@gmail.com>
Simplifiy the relocation tests build process by removing the need for libbpf
at runtime. Its usage is replaced with local `__builtin_*` attributes.
This removes the need for the `LIBBPF_INCLUDE` env variable.
This commit adds documentation on how program names are parsed from
section names, as is used by `aya_obj::Object.programs` as HashMap keys,
and updates the examples into using program names.
Instead of returning anyhow>>Result<()> handle errors
'in-place' with unwrap or panic, for more informative
and user-friendly error messages on test failures.
Fixes#421.
Signed-off-by: Dmitry Savintsev <dsavints@gmail.com>
switch map() and map_mut() from returning a
`Result` to an `Option` since it's just getting
a value from a Hashmap, and to stay in line with
the Programs API.
Remove `MapError::MapNotFound`
Signed-off-by: Andrew Stoycos <astoycos@redhat.com>
Build completing tests passing
Refactor the Map API to better align
with the aya programs API. Specifically
remove all internal locking mechanisms
and custom Deref/DerefMut implementations.
They are replaced with a Map enum
and AsRef/AsMut implementations.
All Try_From implementations have been moved
to standardized enums, with a slightly
special one for PerfEventArray's.
Also cleanup/fix all associated tests and
documentation.
Signed-off-by: Andrew Stoycos <astoycos@redhat.com>
We often need to wait a bit until the program or link gets unloaded
after dropping.
Also, using a macro makes it clear in which particular test the panic
happened.
Signed-off-by: Michal Rostecki <vadorovsky@gmail.com>
libbpf 1.0 doesn't support multimaps defined in `maps` section, it
supports only BTF maps.
At the same time, we already test multimaps loading with the Rust
example (`integration-ebpf/src/bpf/map_test.rs`), so we can just remove
the failing C test.
Signed-off-by: Michal Rostecki <vadorovsky@gmail.com>
1. Removes OwnedLink
2. Allows Links to be converted into FdLink
3. Introduces a PinnedLink type to handle wrap FdLink when pinned and
support un-pinning
Signed-off-by: Dave Tucker <dave@dtucker.co.uk>
This commit moves the aya-log projects from the subtree and adds them to
the main cargo workspace. It also brings the BPF crates into the
workspace and moves the macro crates up a level since they aren't BPF
code.
Miri was disabled for aya-bpf as the previous config wasn't actually
checking anything.
CI, clippy, fmt and release configurations have all been adjusted
appropriately.
CI was not properly running for other supported arches which was also
ixed here.
Signed-off-by: Dave Tucker <dave@dtucker.co.uk>
This commit allows for BTF maps in the .maps ELF section to be parsed.
It reads the necessary information from the BTF section of the ELF file.
While the btf_ids of Keys and Values types are stored, they are not (yet)
used.
When creating a BTF map, we pass the btf_key_type_id and
btf_value_type_id.
Signed-off-by: Dave Tucker <dave@dtucker.co.uk>
This commit replaces the existing RTF test runner with a simple rust
binary package called - integration-test.
integration-test depends on integration-ebpf, which contains test eBPF
code written in Rust and C. `cargo xtask build-integration-test-ebpf`
can be used to build this code and supress rust-analyzer warnings. It
does require `bpf-linker`, but that is highly likely to be available to
developers of Aya. It also requires a checkout of `libbpf` to extract
headers like bpf-helpers.h.
Since everything is compiled into a single binary, it can be run
be run locally using `cargo xtask integration-test` or remotely using
`./run.sh` which re-uses the bash script from the old test framework
to spawn a VM in which to run the tests.
Signed-off-by: Dave Tucker <dave@dtucker.co.uk>
This forces all maps to the maps section so we remain compatible with
libbpf. This requires #181 to avoid breaking userspace.
Signed-off-by: Dave Tucker <dave@dtucker.co.uk>
This commit uses the symbol table to discover all maps inside an ELF
section. Instead of doing what libbpf does - divide the section data
in to equal sized chunks - we read in to section data using the
symbol address and offset, thus allowing us to support definitions
of varying lengths.
Signed-off-by: Dave Tucker <dave@dtucker.co.uk>
This allows for C code to be compiled and used in tests!
It does require libbpf on the host machine though...
Also moved removal of the `user` and `ebpf` dirs created by rust-script
into `lib.sh` and parameterized bits of the XDP smoke test for ease of
copying
Signed-off-by: Dave Tucker <dave@dtucker.co.uk>
This uses a mix of rust-script, bash, qemu and a test runner called RTF
to add a regression test suite... and wires it into GitHub Actions
Signed-off-by: Dave Tucker <dave@dtucker.co.uk>
Tamir Duberstein
Alessandro Decina
dependabot[bot]
Mary
Michal Rostecki
Tuetuopay
Dave Tucker
Michal Rostecki
Matteo Nardi
Shenghui Ye
Dmitry Savintsev
Andrew Stoycos
abhijeetbhagat
Tatsuyuki Ishi
Davide Bertola