- integration-ebpf: add artifact dependency on bpf-linker.
- integration-ebpf: remove manual rerun-if-changed=bpf-linker.
- integration-ebpf: make bpf-linker availble to rustc in build.rs.
- xtask: filter for test binaries now that the build also produces the
bpf-linker binary.
- github: remove `cargo install bpf-linker`.
- integration-test: remove bpf-linker installation instructions.
Blockers:
- https://github.com/rust-lang/cargo/issues/12385 (artifact = "bin"
makes entire workspace nightly-only).
This implements the userspace binding for RingBuf.
Instead of streaming the samples as heap buffers, the process_ring
function takes a callback to which we pass the event's byte region,
roughly following [libbpf]'s API design. This avoids a copy and allows
marking the consumer pointer in a timely manner.
[libbpf]: https://github.com/libbpf/libbpf/blob/master/src/ringbuf.c
Additionally, integration tests are added to demonstrate the usage
of the new APIs and to ensure that they work end-to-end.
Co-authored-by: William Findlay <william@williamfindlay.com>
Co-authored-by: Tatsuyuki Ishi <ishitatsuyuki@gmail.com>
Some of these functions fail to compile when not inlined, so we should
be explicit.
Before deciding on this approach I tried various ways of making all
these functions #[inline(never)] to save instructions but I ran into
blockers:
- These functions currently return Result, which is a structure. This is
not permitted in BPF.
- I tried inventing a newtype that is a #[repr(transparent)] wrapper of
u16, and having these functions return that; however it seems that
even if the object code is legal, the verifier will reject such
functions because the BTF (if present, and it was in my local
experiments) would indicate that the return is a structure.
- I tried having these functions return a plain u16 where 0 means error,
but the verifier still rejected the BTF because the receiver (even if
made into &self) is considered a structure, and forbidden.
We can eventually overcome these problems by "lying" in our BTF once
support for it matures in the bpf-linker repo (e.g. Option<NonZeroU16>
should be perfectly legal as it is guaranteed to be word-sized), but we
aren't there yet, and this is the safest thing we can do for now.
The aya-bpf-macros needed refactoring for:
1. Ease of testing
2. To be consistent with when we use K/V args vs. idents
3. To deprecate the use of `name` to change the exported name of a
function - we now use the symbol table.
Signed-off-by: Dave Tucker <dave@dtucker.co.uk>
This makes a few changes to the way that Aya reads the ELF object
files.
1. To find programs in a section, we use the symbols table. This allows
for cases where multiple programs could appear in the same section.
2. When parsing our ELF file we build symbols_by_section_index as an
optimization as we use it for legacy maps, BTF maps and now programs.
As a result of theses changes the "NAME" used in `bpf.prog_mut("NAME")`
is now ALWAYS the same as the function name in the eBPF code, making the
user experience more consistent.
Signed-off-by: Dave Tucker <dave@dtucker.co.uk>
Remove the manual dependency tracking machinery in
integration-test/build.rs in favor of a build-dependency on
integration-ebpf. This required adding an empty lib.rs to create the
library target.
This allows integration-test/build.rs to be ignorant of bpf-linker.
Remove that in favor of the logic now in integration-ebpf.
Extract the symlink-to-bpf-linker logic from integration-test to xtask
and use it in a new build script in integration-ebpf, causing ebpf
probes to be rebuilt when bpf-linker changes. Previously bpf-linker
changes would rebuild integration-test, but not integration-ebpf,
resulting in stale tests.
Note that this still doesn't address the possibility that a new
bpf-linker is added to the PATH ahead of the cached one. Solving this in
the general case would require rebuild-if-changed-env=PATH *and*
rebuild-if-changed={every-directory-in-PATH} which would likely mean far
too much cache invalidation.
The ebpf probes require a nightly compiler. Before this change, if you
ran `cargo xtask integration-test` with a stable compiler toolchain as
default, or you ran `cargo +stable xtask integration-test`, you would
have seen an error like the one below. This is now fixed by running the
cargo build command in the integration-ebpf directory and making sure
to clear the RUSTUP_TOOLCHAIN env var.
```
--- stderr
/home/ajwerner/src/github.com/aya-rs/aya/test/integration-test/bpf/ring_buf_sched_tracepoint.bpf.c:18:21: warning: declaration of 'struct switch_args' will not be visible outside of this function [-Wvisibility]
int bpf_prog(struct switch_args* ctx)
^
1 warning generated.
error: the `-Z` flag is only accepted on the nightly channel of Cargo, but this is the `stable` channel
See https://doc.rust-lang.org/book/appendix-07-nightly-rust.html for more information about Rust release channels.
thread 'main' panicked at '"cargo" "build" "-p" "integration-ebpf" "-Z" "build-std=core" "--release" "--message-format=json" "--target" "bpfel-unknown-none" "--target-dir" "/home/ajwerner/src/github.com/aya-rs/aya/target/debug/build/integration-test-9bbcb3db5e9f8f57/out/integration-ebpf" exited
with status code 101:
', test/integration-test/build.rs:219:25
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Error: error while building userspace application
Caused by:
Child { stdin: None, stdout: None, stderr: None, .. } exited with status code 101:
```
Add integration testing for link pinning and
loading/unloading of tracepoint, kprobe, and
uprobe programs.
Redo how we utilize bpftool to verify that programs
are loaded to be explicit with names. Also add a helper
to verify that a program is loaded AND linked.
Signed-off-by: Andrew Stoycos <astoycos@redhat.com>
- Add libbpf as a submodule. This prevents having to plumb its location
around (which can't be passed to Cargo build scripts) and also
controls the version against which codegen has run.
- Move bpf written in C to the integration-test crate and define
constants for each probe.
- Remove magic; each C source file must be directly enumerated in the
build script and in lib.rs.
Replace all `assert!(matches!(..))` with `assert_matches!(..)`.
Remove the now-unused build-integration-test xtask command whose logic
doesn't match that of the build-and-run command.
This change does a few things:
- it fixes a bug in the wrappers, where we were expecting the kernel to
return len=1 for b"\0" where it instead returns 0 and doesn't write
out the NULL terminator
- it makes the helpers more robust by hardcoding bound checks in
assembly so that LLVM optimizations can't transform the checks in a
way that the verifier can't understand.
- it adds integration tests
Before this change:
```
error[E0382]: use of moved value: `no_copy`
--> test/integration-ebpf/src/log.rs:35:9
|
33 | let no_copy = NoCopy {};
| ------- move occurs because `no_copy` has type `NoCopy`, which does not implement the `Copy` trait
34 |
35 | debug!(&ctx, "{:x}", no_copy.consume());
| ^^^^^^^^^^^^^^^^^^^^^-------^---------^
| | | |
| | | `no_copy` moved due to this method call
| | use occurs due to use in closure
| value used here after move
|
note: `NoCopy::consume` takes ownership of the receiver `self`, which moves `no_copy`
--> test/integration-ebpf/src/log.rs:28:24
|
28 | fn consume(self) -> u64 {
| ^^^^
= note: this error originates in the macro `debug` (in Nightly builds, run with -Z macro-backtrace for more info)
For more information about this error, try `rustc --explain E0382`.
error: could not compile `integration-ebpf` (bin "log") due to previous error
```
Having separate format hints and tokens per IP address family is
unnecessary, since they are represented by different types and we handle
format hints for each type separately. So we can just have one format
hint.
Also, we should be consistent with the format strings grammar in
Rust[0]. The `type` token, which is mapped to formatting traits, usually
consists of one letter[1] (and optional `?` for `Debug` trait, but that
doesn't matter for us). It shouldn't consist of multiple letters. Our
`:ipv4` and `:ipv6` tokens were clearly breaking that convention, so we
should rather switch to something with one letter - hence `:i`.
[0] https://doc.rust-lang.org/std/fmt/#syntax
[1] https://doc.rust-lang.org/std/fmt/#formatting-traits
libbpf 1.0 doesn't support multimaps defined in `maps` section, it
supports only BTF maps.
At the same time, we already test multimaps loading with the Rust
example (`integration-ebpf/src/bpf/map_test.rs`), so we can just remove
the failing C test.
Signed-off-by: Michal Rostecki <vadorovsky@gmail.com>
This commit moves the aya-log projects from the subtree and adds them to
the main cargo workspace. It also brings the BPF crates into the
workspace and moves the macro crates up a level since they aren't BPF
code.
Miri was disabled for aya-bpf as the previous config wasn't actually
checking anything.
CI, clippy, fmt and release configurations have all been adjusted
appropriately.
CI was not properly running for other supported arches which was also
ixed here.
Signed-off-by: Dave Tucker <dave@dtucker.co.uk>
This commit allows for BTF maps in the .maps ELF section to be parsed.
It reads the necessary information from the BTF section of the ELF file.
While the btf_ids of Keys and Values types are stored, they are not (yet)
used.
When creating a BTF map, we pass the btf_key_type_id and
btf_value_type_id.
Signed-off-by: Dave Tucker <dave@dtucker.co.uk>
This commit replaces the existing RTF test runner with a simple rust
binary package called - integration-test.
integration-test depends on integration-ebpf, which contains test eBPF
code written in Rust and C. `cargo xtask build-integration-test-ebpf`
can be used to build this code and supress rust-analyzer warnings. It
does require `bpf-linker`, but that is highly likely to be available to
developers of Aya. It also requires a checkout of `libbpf` to extract
headers like bpf-helpers.h.
Since everything is compiled into a single binary, it can be run
be run locally using `cargo xtask integration-test` or remotely using
`./run.sh` which re-uses the bash script from the old test framework
to spawn a VM in which to run the tests.
Signed-off-by: Dave Tucker <dave@dtucker.co.uk>
Tamir Duberstein
Alessandro Decina
Andrew Werner
Tuetuopay
Dave Tucker
Andrew Stoycos
Michal Rostecki
Michal Rostecki