Commit Graph

124 Commits (104f449c0ff8fe072f53c1d7a35e3ae51b394797)

Author SHA1 Message Date
tyrone-wu 86e279ef0a
codegen: add bpf_stats_type
Added `bpf_stats_type` enum to codegen bindings.

Refs: #959
2 months ago
Dave Tucker b20b1f1b0a chore(xtask): Create bindings dir
If the bindings directory doesn't exist then create it.

Signed-off-by: Dave Tucker <dave@dtucker.co.uk>
3 months ago
Billy McFall 62efed1853 chore(xtask): Add s390x and powerpc64 to codegen
This commit adds the s390x and powerpc architectures
to codegen. This will enable an upcoming PR to add
support for aya to support theses architectures
in both aya and aya-ebpf.

Co-authored-by: Dave Tucker <dave@dtucker.co.uk>
Signed-off-by: Billy McFall <22157057+Billy99@users.noreply.github.com>
3 months ago
tyrone-wu b23973dd9c xtask: corrected bpf to ebpf for path to aya-ebpf-bindings in codegen
Currently, when running `cargo +nightly xtask codegen` locally and in
the codegen GHA workflow, an error occurs with only "Error: bindgen
failed" displayed.

This was due to a path using "bpf/..." instead of "ebpf/...". It is now
corrected to "ebpf", and bindgen fails should now display a more direct
message on why it failed.

Fixes: #914
6 months ago
Dave Tucker 41c61560ea chore(aya-ebpf): Rename bpf -> ebpf
Signed-off-by: Dave Tucker <dave@dtucker.co.uk>
7 months ago
Dave Tucker 70ac91dc1e chore(aya-ebpf-bindings): Rename bpf -> ebpf
Signed-off-by: Dave Tucker <dave@dtucker.co.uk>
7 months ago
Dave Tucker 21f570a19c chore(aya-ebpf-cty): Rename bpf -> ebpf
Signed-off-by: Dave Tucker <dave@dtucker.co.uk>
7 months ago
Dave Tucker 0c58bb66b6 integration-test: Fix build.rs output
The cargo::warning seems to ignore output after a newline.
Iterate over the entire rendered message and print it line-by-line.

Signed-off-by: Dave Tucker <dave@dtucker.co.uk>
7 months ago
Tamir Duberstein 5041aa41de
xtask: allow public-api regen on aarch64-apple-darwin
```
CARGO_CFG_BPF_TARGET_ARCH=x86_64 cargo +nightly xtask public-api --bless --target x86_64-unknown-linux-gnu
```
9 months ago
Tamir Duberstein 7a4c6153a0
xtask: terminate QEMU on "RCU grace-period kthread stack dump"
See
https://github.com/aya-rs/bpf-linker/actions/runs/7067897954/job/19241830198
10 months ago
Tamir Duberstein 42cc175aeb
xtask: attempt to open /dev/kvm
We're seeing test failures where KVM is present but we aren't able to use it.

See https://github.com/actions/runner-images/issues/7670#issuecomment-1760184162.
11 months ago
Tamir Duberstein 5392f9821d
xtask: terminate QEMU on "BUG: soft lockup"
See
https://github.com/aya-rs/aya/actions/runs/6329828507/job/17190931752.

Unlike on kernel panic this message prints periodically so it is
permitted to print twice before QEMU is killed.
12 months ago
Tamir Duberstein d16e607fd4
rustfmt: group_imports = "StdExternalCrate"
High time we stop debating this; let the robots do the work.
12 months ago
Tamir Duberstein fe13b2eb41
xtask: remove useless copy
/tmp/initrd.img is unused.
12 months ago
Tamir Duberstein cbe9cd9133
xtask: appease new clippy warning
```
warning: unnecessary hashes around raw string literal
  --> xtask/src/docs.rs:70:17
   |
70 |           indoc! {r#"
   |  _________________^
71 | |     User-Agent:*
72 | |     Disallow: /
73 | |     "#},
   | |______^
   |
   = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#needless_raw_string_hashes
   = note: `#[warn(clippy::needless_raw_string_hashes)]` on by default
help: remove all the hashes around the literal
   |
70 ~         indoc! {r"
71 |     User-Agent:*
72 |     Disallow: /
73 ~     "},
   |
```

This false negative was fixed in
https://github.com/rust-lang/rust-clippy/pull/11518 (the title
incorrectly says false positive).
12 months ago
Tamir Duberstein 782eb85d4a
xtask: extract `Errors` type
This produces better errors in `cargo xtask public-api` because before
this change we'd print errors as display, which didn't show context.

Before:
```
Error: public API errors:
aya failed to check public API: public_api::Builder::build
aya-obj failed to check public API: public_api::Builder::build
aya-log failed to check public API: public_api::Builder::build
aya-log-common failed to check public API: public_api::Builder::build
aya-log-parser failed to check public API: public_api::Builder::build
aya-tool failed to check public API: public_api::Builder::build
aya-bpf failed to check public API: public_api::Builder::build
aya-bpf-bindings failed to check public API: public_api::Builder::build
aya-bpf-cty failed to check public API: public_api::Builder::build
aya-bpf-macros failed to check public API: public_api::Builder::build
aya-log-ebpf-macros failed to check public API: public_api::Builder::build
```

After:
```
Error: aya failed to check public API

Caused by:
    0: public_api::Builder::build
    1: unknown variant `type_alias`, expected one of `module`, `extern_crate`, `import`, `struct`, `struct_field`, `union`, `enum`, `variant`, `function`, `typedef`, `opaque_ty`, `constant`, `trait`, `trait_alias`, `impl`, `static`, `foreign_type`, `macro`, `proc_attribute`, `proc_derive`, `assoc_const`, `assoc_type`, `primitive`, `keyword` at line 1 column 3082463
aya-obj failed to check public API

Caused by:
    0: public_api::Builder::build
    1: unknown variant `type_alias`, expected one of `module`, `extern_crate`, `import`, `union`, `struct`, `struct_field`, `enum`, `variant`, `function`, `trait`, `trait_alias`, `impl`, `typedef`, `opaque_ty`, `constant`, `static`, `foreign_type`, `macro`, `proc_macro`, `primitive`, `assoc_const`, `assoc_type` at line 1 column 129355
aya-log failed to check public API

Caused by:
    0: public_api::Builder::build
    1: unknown variant `type_alias`, expected one of `module`, `extern_crate`, `import`, `struct`, `struct_field`, `union`, `enum`, `variant`, `function`, `typedef`, `opaque_ty`, `constant`, `trait`, `trait_alias`, `impl`, `static`, `foreign_type`, `macro`, `proc_attribute`, `proc_derive`, `assoc_const`, `assoc_type`, `primitive`, `keyword` at line 1 column 311910
aya-log-common failed to check public API

Caused by:
    0: public_api::Builder::build
    1: unknown variant `type_alias`, expected one of `module`, `extern_crate`, `import`, `union`, `struct`, `struct_field`, `enum`, `variant`, `function`, `trait`, `trait_alias`, `impl`, `typedef`, `opaque_ty`, `constant`, `static`, `foreign_type`, `macro`, `proc_macro`, `primitive`, `assoc_const`, `assoc_type` at line 1 column 130456
aya-log-parser failed to check public API

Caused by:
    0: public_api::Builder::build
    1: unknown variant `type_alias`, expected one of `module`, `extern_crate`, `import`, `struct`, `struct_field`, `union`, `enum`, `variant`, `function`, `typedef`, `opaque_ty`, `constant`, `trait`, `trait_alias`, `impl`, `static`, `foreign_type`, `macro`, `proc_attribute`, `proc_derive`, `assoc_const`, `assoc_type`, `primitive`, `keyword` at line 1 column 204930
aya-tool failed to check public API

Caused by:
    0: public_api::Builder::build
    1: unknown variant `type_alias`, expected one of `module`, `extern_crate`, `import`, `struct`, `struct_field`, `union`, `enum`, `variant`, `function`, `typedef`, `opaque_ty`, `constant`, `trait`, `trait_alias`, `impl`, `static`, `foreign_type`, `macro`, `proc_attribute`, `proc_derive`, `assoc_const`, `assoc_type`, `primitive`, `keyword` at line 1 column 234519
aya-bpf failed to check public API

Caused by:
    0: public_api::Builder::build
    1: unknown variant `type_alias`, expected one of `module`, `extern_crate`, `import`, `struct`, `struct_field`, `union`, `enum`, `variant`, `function`, `typedef`, `opaque_ty`, `constant`, `trait`, `trait_alias`, `impl`, `static`, `foreign_type`, `macro`, `proc_attribute`, `proc_derive`, `assoc_const`, `assoc_type`, `primitive`, `keyword` at line 1 column 741697
aya-bpf-bindings failed to check public API

Caused by:
    0: public_api::Builder::build
    1: unknown variant `type_alias`, expected one of `module`, `extern_crate`, `import`, `union`, `struct`, `struct_field`, `enum`, `variant`, `function`, `trait`, `trait_alias`, `impl`, `typedef`, `opaque_ty`, `constant`, `static`, `foreign_type`, `macro`, `proc_macro`, `primitive`, `assoc_const`, `assoc_type` at line 1 column 112445
aya-bpf-cty failed to check public API

Caused by:
    0: public_api::Builder::build
    1: unknown variant `type_alias`, expected one of `module`, `extern_crate`, `import`, `union`, `struct`, `struct_field`, `enum`, `variant`, `function`, `trait`, `trait_alias`, `impl`, `typedef`, `opaque_ty`, `constant`, `static`, `foreign_type`, `macro`, `proc_macro`, `primitive`, `assoc_const`, `assoc_type` at line 1 column 315
aya-bpf-macros failed to check public API

Caused by:
    0: public_api::Builder::build
    1: unknown variant `type_alias`, expected one of `module`, `extern_crate`, `import`, `struct`, `struct_field`, `union`, `enum`, `variant`, `function`, `typedef`, `opaque_ty`, `constant`, `trait`, `trait_alias`, `impl`, `static`, `foreign_type`, `macro`, `proc_attribute`, `proc_derive`, `assoc_const`, `assoc_type`, `primitive`, `keyword` at line 1 column 186226
aya-log-ebpf-macros failed to check public API

Caused by:
    0: public_api::Builder::build
    1: unknown variant `type_alias`, expected one of `module`, `extern_crate`, `import`, `struct`, `struct_field`, `union`, `enum`, `variant`, `function`, `typedef`, `opaque_ty`, `constant`, `trait`, `trait_alias`, `impl`, `static`, `foreign_type`, `macro`, `proc_attribute`, `proc_derive`, `assoc_const`, `assoc_type`, `primitive`, `keyword` at line 1 column 158055
```
1 year ago
Tamir Duberstein ca2e43d318
integration-test: Remove cargo symlink workaround
https://github.com/rust-lang/cargo/pull/12369 fixed this bug and was
picked up in https://github.com/rust-lang/rust/pull/114027.
1 year ago
Tamir Duberstein 7e14214f47
xtask: add noapic to kernel parameters
This might prevent some kernel panics.
1 year ago
Tamir Duberstein de65ba0067
xtask: watch for kernel panic in stdout too
Seems unclear whether kernel panics go to stdout or stderr, so do them
both.
1 year ago
Tamir Duberstein 58ba66c003
xtask: watch for kernel panics on stderr
It seems these go to stderr, not stdout. Use `Ctrl-A x` to shut QEMU
down if a panic is seen.
1 year ago
Tamir Duberstein 82a77bc83d
integration-test: Implement running on VMs
Implements running integration tests on multiple VMs with arbitrary
kernel images using `cargo xtask integration-test vm ...`.

This changes our coverage from 6.2 to 6.1 and 6.4.
1 year ago
Tamir Duberstein dc9f72adf0
test,xtask: Simplify ExitStatus handling 1 year ago
Tamir Duberstein 8d7545aff8
xtask: reverse public API diff direction
This is currently producing inverse diffs.
1 year ago
Tamir Duberstein 321bda7539
public-api: simplify and improve output 1 year ago
Dave Tucker f833f1fc1d xtask: Add cargo-public-api
Adds cargo-public-api. This allows for public API changes to get caught
in CI, requiring new changes to be "blessed" by using:

  cargo xtask public-api --bless

When this file is changed for aya, Alessandro will need to review the PR.

Signed-off-by: Dave Tucker <dave@dtucker.co.uk>
1 year ago
Tamir Duberstein a6b1fb9a1e xtask: allow arbitrary flags to cargo build 1 year ago
Tamir Duberstein 74fc50bf7e
integration-test: shuttle stdio to user
Trampoline cargo-in-cargo stdio through cargo:warning to ensure the user
sees all the output.

Use bpf-linker from git in CI so we can see what's going on there.
1 year ago
Tamir Duberstein e276c07f73
integration-ebpf: invalidate on bpf-linker
Extract the symlink-to-bpf-linker logic from integration-test to xtask
and use it in a new build script in integration-ebpf, causing ebpf
probes to be rebuilt when bpf-linker changes. Previously bpf-linker
changes would rebuild integration-test, but not integration-ebpf,
resulting in stale tests.

Note that this still doesn't address the possibility that a new
bpf-linker is added to the PATH ahead of the cached one. Solving this in
the general case would require rebuild-if-changed-env=PATH *and*
rebuild-if-changed={every-directory-in-PATH} which would likely mean far
too much cache invalidation.
1 year ago
Andrew Werner d5259ce789 xtask: move libbpf submodule 1 year ago
Andrew Werner 15b3c459ae xtask: ensure libbpf submodule is initialized
Libbpf is used by xtasks, in the command, ensure that the submodules
are initialized. This eases the user-experience so that users don't
need to think about the submodule, while retaining all the benefits
of using a submodule vs forcing the user to manually check out libbpf
and stick it in some pre-defined place.

We use the symbol pointing to libbpf in xtask in the build script
to avoid repeating this constant.

Also, we install git in the vm so that we can init the submodule
when we build in the vm.
1 year ago
Tamir Duberstein 40f24fe8e1
site: generate docs for logging crates 1 year ago
Tamir Duberstein b86d42d1b0
xtask: remove assumptions from docs command
This slightly changes the site layout: crate documentation is now flat
rather than being nested under "user" and  "bpf".

- Run `cargo clean --doc` before generating docs to ensure hermiticity.
- Generate header.html into a temporary directory.
- Remove "site" on each run to ensure hermiticity.
- Invoke cargo only once.
- Avoid editing sources.
1 year ago
Tamir Duberstein 6ac1320707
integration-test: build "fake" by default
Use the environment variable AYA_BUILD_INTEGRATION_BPF to indicate to
the build script that it should *actually* build bpf, otherwise emitting
empty files.

This allows metadata builds to skip costly build steps without
sacrificing ergonomics; all compile-time tools such as cargo clippy work
out of the box.

Cargo even gives each of these builds (depending on the value of the
environment variable) its own cache key, so they do not invalidate each
other when the user alternates between metadata and real builds.

This allows the lint action to move out of the VM.
1 year ago
Tamir Duberstein 9086265ac0
xtask: reimplement build-integration-test
This command builds the integration test binaries and prints their paths
to stdout.
1 year ago
Tamir Duberstein 3d463a3610
integration-test: compile Rust probes using build.rs 1 year ago
Tamir Duberstein bc9f059d53
xtask: inline build_ebpf 1 year ago
Tamir Duberstein 8c61fc9ea6
integration-test: compile C probes using build.rs
- Add libbpf as a submodule. This prevents having to plumb its location
  around (which can't be passed to Cargo build scripts) and also
  controls the version against which codegen has run.
- Move bpf written in C to the integration-test crate and define
  constants for each probe.
- Remove magic; each C source file must be directly enumerated in the
  build script and in lib.rs.
1 year ago
Tamir Duberstein e621a09181
Clippy over tests and integration-ebpf
Replace all `assert!(matches!(..))` with `assert_matches!(..)`.

Remove the now-unused build-integration-test xtask command whose logic
doesn't match that of the build-and-run command.
1 year ago
Tamir Duberstein b0a4ab5f20
xtask: Standardize command logging
Don't run `cargo build --verbose`; it's too noisy.
1 year ago
Tamir Duberstein 27120b328a
aya: don't allocate static strings 1 year ago
Tamir Duberstein 9ca0af1a79
integration-test: Remove integration-test-macros
This doesn't add any value; use `cargo build --tests` with
`--message-format=json` instead; parse the output using `cargo_metadata`
to discover the location of the test binary.

Move test/integration-test/src/tests -> test/integration-test/tests to
conform to
https://doc.rust-lang.org/book/ch11-03-test-organization.html#integration-tests.
1 year ago
Tamir Duberstein ecc03ecfad
xtask: destructure 1 year ago
Tamir Duberstein de1fe1f5e1
xtask: Avoid lossy conversions, spurious vectors 1 year ago
Tamir Duberstein 25859e4e33
xtask: emit clang command on failure 1 year ago
Tamir Duberstein 7067db450a
xtask: Avoid lossy string conversion 1 year ago
Tamir Duberstein 5a2906a6c9
test,xtask: Replace lazy_static with OnceCell
This doesn't affect MSRV because this is testing code.
1 year ago
Tamir Duberstein ff86f1385c
Remove dependency on bpftool in integration tests 1 year ago
Kenjiro Nakayama 8c8d236f42 xtask: codegen: generate bindings for user_regs_struct
This patch adds `user_regs_struct`.

riscv provides struct user_regs_struct instead of struct pt_regs to userspace.
After bindings generates the code, adding the riscv support in `bpf/aya-bpf/src/args.rs`
then aya-bpf can be built for riscv.
1 year ago
Michal Rostecki ed9c2a1780 integration-tests: Build eBPF programs always with release profile
Also, add the `codegen-unit` option to the profile.
1 year ago
Tuetuopay 23e7c65244 xtask: generate userspace bindings for bpf_{cpu,dev}map_val 2 years ago