Commit Graph

227 Commits (3aa274597236e8c9938e3626346e2d23119e7ed6)

Author SHA1 Message Date
Dave Tucker ec8293ab86 aya: Implement XDP Map Types
This commit adds implementations for:
- xskmap
- devmap
- devmap_hash
- cpumap

Which can all be used to redirect XDP packets to various different
locations

Signed-off-by: Dave Tucker <dave@dtucker.co.uk>
2 years ago
Dave Tucker 938f979fe7 aya: Make MapData::pin pub
This is to solve a use-case where a user (in this case bpfd) may want
to:

- MapData::from_pin to open a pinned map from bpffs
- MapData::pin to pin that object into another bpffs

Both operations should be easily accomplished without needing to cast
a MapData into a concrete Map type - e.g aya::maps::HashMap.

Signed-off-by: Dave Tucker <dave@dtucker.co.uk>
2 years ago
Dave Tucker 0f4021ec89 aya: Remove MapData::pinned
BPF objects can be pinned multiple times, to multiple different places.
Tracking whether or not a map is pinned in a bool is therefore not sufficient.
We could track this in a HashSet<PathBuf>, but there is really no reason
to track it at all.

Signed-off-by: Dave Tucker <dave@dtucker.co.uk>
2 years ago
Tamir Duberstein 0dacb34d44
maps: fix typos, avoid fallible conversions 2 years ago
Tamir Duberstein b4d5a1e8db
maps: MapData::{obj, fd} are private 2 years ago
Tamir Duberstein f41592663c
maps: `MapFd` and `SockMapFd` are owned
`MapData::fd` is now a `MapFd`. This means that `MapData` now closes the
file descriptor on drop. In the future we might consider making `MapFd`
hold a `BorrowedFd` but this requires API design work due to overlapping
borrows.

Since `SockMapFd` is no longer `Copy`, attach methods to take it by
reference to allow callers to use it multiple times as they are
accustomed to doing.

`SockMapFd` implements `try_clone`. `MapFd` and `SockMapFd` are now
returned by reference to allow callers to avoid file descriptor cloning
when desired.

This is an API breaking change.

Updates #612.
2 years ago
Tamir Duberstein 92d3056db3
Merge pull request #775 from aya-rs/perf-as-raw-fd
async_perf_event_array: access inner through async
2 years ago
Andrew Werner 172859c66b aya/maps: support TryFrom for LRU hash maps
The macro to implement TryFrom for MapData didn't have the ability to
specify that more than one variant of MapData can be valid for a single
map implementation. Support for new syntax was added to the macro so that
the implementation can succeed for both valid variants in the HashMap
and PerCpuHashMap impl.

Fixes #636
2 years ago
Andrew Werner 2a1bf609b2 aya/maps: rework TryFrom macros
The old macros were repetitive and inflexible. This unifies the various
macros used to generate TryFrom implementations for map implementations
from the relevant map enum variants.

Cleanup in anticipation of fixing #636.

The API changes are just about renaming the return to Self and
Self::Error; they are not real changes.
2 years ago
Tamir Duberstein 8b0c7f1204
async_perf_event_array: access inner through async
Avoid holding onto raw file descriptors.

Remove some implied bounds (BorrowMut implies Borrow).
2 years ago
Andrés Medina 6895b1e2ed
aya: Use AsFd when attaching fds to programs
This is a breaking change but adds another level of safety to ensure
the file descriptor we receive is valid. Additionally, this allows
aya to internally easily duplicate this file descriptor using std
library methods instead of manually calling `dup` which doesn't
duplicate with the CLOSE_ON_EXEC flag that is standard pratice to
avoid leaking the file descriptor when exec'ing.
2 years ago
Tamir Duberstein 0bba9b14b0
maps,programs: avoid path UTF-8 assumptions 2 years ago
Tamir Duberstein abda239d63
aya: deny various allow-by-default lints
Notably:
- clippy::use_self: replaced many T with Self.
- single_use_lifetimes: removed some single use lifetimes.
- unreachable_pub: removed some unreachable pub items.
- unused_crate_dependencies: removed unused futures,parking_lot deps.
- unused_qualifications: found a potential `crate` vs `$crate` bug.
- let_underscore_drop: not enabled, seems to trigger false positives.
- missing_copy_implementations: not enabled, unclear if we want this.
- unsafe_op_in_unsafe_fn: not enabled, unclear if we want this.
- unused_results: not enabled, needs many fixes (but I think wanted).
2 years ago
Tamir Duberstein 9ff1bf3d3b
aya: fix docs build
Appease the new lint rustdoc::redundant_explicit_links that was added in
https://github.com/rust-lang/rust/pull/113167.
2 years ago
Tamir Duberstein a31544b6e7
maps: BloomFilter::insert takes &mut self
This is consistent with all the other maps.
2 years ago
Tamir Duberstein 89bc255f1d
aya: MapData::fd is non-optional
The primary driver of change here is that `MapData::create` is now a
factory function that returns `Result<Self, _>` rather than mutating
`&mut self`. The remaining changes are consequences of that change, the
most notable of which is the removal of several errors which are no
longer possible.
2 years ago
Tamir Duberstein 504fd1df0a
programs: `ProgramFd` is owned
`ProgramData::fd` is now a `ProgramFd`. This means that `ProgramData`
now closes the file descriptor on drop. In the future we might consider
making `ProgramFd` hold a `BorrowedFd` but this requires API design work
due to overlapping borrows.

Since `ProgramFd` is no longer `Copy`, update methods to take it by
reference to allow callers to use it multiple times as they are
accustomed to doing.

`ProgramFd` is now returned by reference and implements `try_clone` to
allow callers to avoid file descriptor cloning when desired.

This is an API breaking change.

Updates #612.
2 years ago
Dave Tucker db975e9778 aya: Don't store bpf_fd in MapData
This is only used in create and therefore can be passed
as a parameter.

Signed-off-by: Dave Tucker <dave@dtucker.co.uk>
2 years ago
Addison Crump ed777273b1
nuclear option: no symbol resolution in the crate 2 years ago
Andrés Medina 8ebf0ac327
aya: Use OwnedFd in FdLink. 2 years ago
Addison Crump d8709de9f2
Extract trait SymbolResolver 2 years ago
Tamir Duberstein b1404e9a73
sys: push error construction up 2 years ago
Tamir Duberstein de8519a380
sys: extract common SyscallError
We currently have 4 copies of this.
2 years ago
Andrés Medina dbfba18dac aya: Return `OwnedFd` for `perf_event_open`.
This fixes a file descriptor leak when creating a link of
BPF_PERF_EVENT attach type.
2 years ago
Tamir Duberstein 17f25a6793
all: better panic messages
Always include operands in failing assertions. Use assert_matches over
manual match + panic.
2 years ago
Andrés Medina ea96c29ccb aya: Use Arc<OwnedFd> when loading BTF fd
This fixes an existing file descriptor leak when there is BTF data in
the loaded object.

To avoid lifetime issues while having minimal impact to UX the
`OwnedFd` returned from the BPF_BTF_LOAD syscall will be wrapped in an
`Arc` and shared accross the programs and maps of the loaded BPF
file.
2 years ago
Andrés Medina 683a1cf2e4 aya: Make SysResult generic on Ok variant 2 years ago
Andrés Medina c63d9904f7 Replace std::os::unix::io for std::os::fd
This is just taking https://github.com/aya-rs/aya/pull/633 to its
logical conclusion. Because `std::os::fd` was only introduced as a
module in Rust v1.66.0 I have also updated the `Cargo.toml` of the
`aya` package to reflect the true MSRV. Note that this commit is *not*
the cause for this MSRV bump, that was done by a previous commit, this
commit is just making it explicit in the `Cargo.toml`
2 years ago
Dave Tucker 764eb309b0 Clippy fixes for latest nightly
Signed-off-by: Dave Tucker <dave@dtucker.co.uk>
2 years ago
Tamir Duberstein 961f45da37
Replace matches with assert_matches
The matches crate has been archived now that `matches!` is in std.
However `assert_matches!` is still unstable in std, and the
assert_matches crate provides a more expressive form:

```
assert_matches!(foo, Ok(bar) => {
  assert_eq!(bar, baz);
});
```
2 years ago
Tamir Duberstein fa91fb4f59
Remove "async" feature
This feature is equivalent to async_tokio || async_std; removing it
avoids warnings emitted during `cargo hack check --feature-powerset`
where async is selected without either of the other features.

Use cargo hack to ensure clippy runs on the powerset of features.
2 years ago
Alessandro Decina eb60d65613
Merge pull request #520 from astoycos/unsupported-map
Add Unsupported Map type
2 years ago
Tamir Duberstein e621a09181
Clippy over tests and integration-ebpf
Replace all `assert!(matches!(..))` with `assert_matches!(..)`.

Remove the now-unused build-integration-test xtask command whose logic
doesn't match that of the build-and-run command.
2 years ago
Tamir Duberstein cc2bc0acc1
Remove procfs dependency 2 years ago
Tamir Duberstein b611038d5b
Use procfs crate for kernel version parsing
This allows the logic to be shared between aya and the integration tests
without exposing additional public API surface.
2 years ago
Tamir Duberstein 27120b328a
aya: don't allocate static strings 2 years ago
Alessandro Decina 76d35d10ce
Merge pull request #526 from dave-tucker/trie
aya: Remove iter_key from LPM Trie API
2 years ago
Dave Tucker 00c480d2f9 aya: Remove iter_key from LPM Trie API
Based on the discussion in Discord we've decided to drop the
iter_key() API for LPM Trie. According to the kernel self-tests and
experimentation done in Aya, providing a key into bpf_map_get_next_id
will either:

- If key is an EXACT match, proceed iterating through all keys in the
trie from this point
- If key is NOT an EXACT match, proceed iterating through all keys in
the trie starting at the leftmost entry.

An API in Aya could be crafted that gets the LPM match + less specific
matches for a prefix using these semantics BUT it would only apply to
userspace. Therefore we've opted out of fixing this.

Signed-off-by: Dave Tucker <dave@dtucker.co.uk>
2 years ago
ajwerner 65d10f9ffc aya: replace os::unix::prelude with os::fd 2 years ago
Andrew Stoycos 17930a88c5
Fixups in response to alessandrod review
Move BpfError::UnsupportedMap into MapError and add a map_type field to
the error.

Update the `allow_unsupported_maps` function comment.

Update the logic to check if any unsupported maps are being used. More
specifically only search a program's maps if `allow_unsupported_maps` is
set and then use the iter function `try_for_each` with a match statement
which allows us to return the inner map type if it's unsupported.

Signed-off-by: Andrew Stoycos <astoycos@redhat.com>
2 years ago
Andrew Stoycos b5719c5b3f
Add Unsupported Map type
Just because aya doesn't support working with some map
types doesn't mean we should't allow them to be loaded.

Add a new `Unsupported` map type to facilitate this,

Next add a user configurable option `allow_unsupported_maps()`
which can be called against the `bpfLoader`.  Additionally
add a nice warning log message when a program is being loaded
by Aya and contains an unsupported map type.

Signed-off-by: Andrew Stoycos <astoycos@redhat.com>
2 years ago
William Batista 3d1013d729 Fixed a typo in the per_cpu_hashmap documentation 2 years ago
Hanaasagi 0e4aec475f fix(lint): remove useless `any` `all` in cfg. 2 years ago
Dave Tucker ed14751c79
Merge pull request #525 from dave-tucker/borrow
aya: MapData should be Borrow, not AsRef
3 years ago
Alessandro Decina 401ea5e848 aya, aya-obj: refactor map relocations
Clearly split the code between `.maps`, `maps` and data maps (bss, data,
rodata). Sprinkle comments.

Remove MapKind which was effectively only needed since we used to have
one variant - BpfSectionKind::Data - to represent all data maps. Instead
add explicit BpfSectionKind::{Data, Rodata, Bss} variants and match on
those when we initialize maps.
3 years ago
Mary 94049ec661 aya: Fix MapData Clone implementation
The Clone implementation of MapData was previously not storing the
result of the dup operation.
3 years ago
Dave Tucker b1a70fc6e4 aya: MapData should be Borrow, not AsRef
We don't ever do ref-to-ref conversion for MapData so Borrow should
suffice.

Signed-off-by: Dave Tucker <dave@dtucker.co.uk>
3 years ago
Alessandro Decina c22014c757 aya: fix Lru and LruPerCpu hash maps
They were broken by https://github.com/aya-rs/aya/pull/397
3 years ago
Shenghui Ye e52497cb9c aya-obj: add basic documentation to public members
Types relevant to maps are moved into aya_obj::maps.
Some members are marked `pub(crate)` again.

Refs: #473
3 years ago
Shenghui Ye 81bc307dce aya-obj: migrate bindgen destination
Aya::obj depends on bindgen generated files, and we start
by migrating bindgen generated files.

This commit adds the new aya-obj crate to the workplace
and migrates generated files into the crate. We use core
instead of std in an effort to make the final crate no_std.

Bindgen was run against libbpf v1.0.1.

Refs: #473
3 years ago
Michal Rostecki 9f5d157628
Merge pull request #461 from FallingSnow/main
Add ability to iterate over LpmTrie keys and matches
3 years ago
Ayrton Sparling 10ac5957c1
Fix LpnTrieKeys -> LpmTrieKeys typo 3 years ago
Ayrton Sparling a44f054bec fix formatting 3 years ago
Ayrton Sparling 1368eb94e7 Remove old test 3 years ago
Ayrton Sparling 9a3682e793 Add ability to iterate over lpmtrie key matches 3 years ago
Ayrton Sparling 8fe64aef1f Fix lpmtrie iter returning nothing 3 years ago
Alexis Bauvin 51bb50ed8e maps: add missing TryFrom<Map> for HashMap, PerCpuHashMap and LpmTrie 3 years ago
Ayrton Sparling e4182a9eab Iterate lpmtrie 3 years ago
Alessandro Decina 88d7777553
Merge pull request #431 from 0b01/refs
aya: use impl Borrow<T> instead of T for maps
3 years ago
0b01 76e417a474 Fix formatting 3 years ago
Michal Rostecki e0a9895260 maps: Fix the error message in `MapData::pin()`
The syscall name is `BPF_OBJ_PIN`, not `BPF_OBJ_GET`.

Signed-off-by: Michal Rostecki <vadorovsky@gmail.com>
3 years ago
Ricky Han 6ce60ad21d make sure everything is marked correctly 3 years ago
Ricky Han 9525b1a370 fix array 3 years ago
Ricky Han 575fea4cb9 fix wrong bounds 3 years ago
Ricky Han fbfbedb6a8 cargo fmt 3 years ago
Ricky Han 9991ffb093 Use & 3 years ago
Ricky Han e9ec257328 Add test case 3 years ago
Ricky Han 1247ffc19b Use Borrow<T> instead 3 years ago
Andrew Stoycos 82edd681c3 Fix doc links, update rustdoc args
Fix some broken rust doc links.

Make sure rustdoc build fail on warnings
so we catch these broken links in CI.

Signed-off-by: Andrew Stoycos <astoycos@redhat.com>
3 years ago
Andrew Stoycos f3262e87bd Make map APIs return an option
switch map() and map_mut() from returning a
`Result` to an `Option` since it's just getting
a value from a Hashmap, and to stay in line with
the Programs API.

Remove `MapError::MapNotFound`

Signed-off-by: Andrew Stoycos <astoycos@redhat.com>
3 years ago
Andrew Stoycos 4ddf2600b4 Fixups4
Remove From method and replace with internal
helper function.

Signed-off-by: Andrew Stoycos <astoycos@redhat.com>
3 years ago
Andrew Stoycos 440097d7bc Fixups 3
Remove MapError::UnexpectedMapType

Add Macro for converting from aya::Map to
u32 (map type) for use in
`MapError::InvalidMapType { map_type: x }`

Signed-off-by: Andrew Stoycos <astoycos@redhat.com>
3 years ago
Andrew Stoycos 939d16cce5 Fixups 2
Respond to more review comments:

Revert to try_from in doctests so we don't need
to explicitly specify type parameters.

Fixup some documentation

Remove explit types in `try_from` methods

Signed-off-by: Andrew Stoycos <astoycos@redhat.com>
3 years ago
Andrew Stoycos 8009361694 Fixups
Respond to review comments, specifically:

- Remove Map::map_type()

- Update some comments

- remove `docs` from feature macros

- generalize check_bounds, check_kv_size,
and check_v_size functions to remove
duplicate code

Signed-off-by: Andrew Stoycos <astoycos@redhat.com>
3 years ago
Andrew Stoycos 893f9f44a2 Implement Copy for MapData
Implement Copy for MapData so that
when `take_map` is used we create a
1 to 1 mapping of MapData to internal
FileDescriptor.  This will ensure
that when MapData is used in multiple
tasks that we don't drop the FD before
all tasks are done using it.

Signed-off-by: Andrew Stoycos <astoycos@redhat.com>
3 years ago
Andrew Stoycos 898a14d425 Use SockMapFd
Create a new type called `SockMapFd` which is
solely used when a program needs to attach
to a socket map. In the future this same
tatic could be used for other use cases
so we may make this more generic.

Signed-off-by: Andrew Stoycos <astoycos@redhat.com>
3 years ago
Andrew Stoycos 1aefa2e5e6 Core refactor of Map API
Build completing tests passing

Refactor the Map API to better align
with the aya programs API.  Specifically
remove all internal locking mechanisms
and custom Deref/DerefMut implementations.
They are replaced with a Map enum
and AsRef/AsMut implementations.

All Try_From implementations have been moved
to standardized enums, with a slightly
special one for PerfEventArray's.

Also cleanup/fix all associated tests and
documentation.

Signed-off-by: Andrew Stoycos <astoycos@redhat.com>
3 years ago
abhijeetbhagat 6c813b8c38 fix all clippy warnings 3 years ago
Dave Tucker 95e8c78db8 docs: Add labels for optional features
Following the lead of crates like tokio and nix, we now annotate APIs
that require optional features. This helps in cases where a user wants
to have an `AsyncPerfEventArray` which is documented on crates.io, but
it's not obvious that you have to enable the `async` feature.

Signed-off-by: Dave Tucker <dave@dtucker.co.uk>
3 years ago
Michal Rostecki 43aff57793 maps: Disable miri warnings about integer-to-pointer conversions
`override_syscall` performs integer-to-pointer conversion. This is
considered harmful on the newest Rust nightly which provides
`ptr::from_exposed_addr`, but there is no other way on Rust stable than
doing `as *const T`, which is what miri is unhappy about.

Signed-off-by: Michal Rostecki <vadorovsky@gmail.com>
3 years ago
Dave Tucker 5693fb9941 aya: Rename from_pinned and from_path to from_pin
Signed-off-by: Dave Tucker <dave@dtucker.co.uk>
3 years ago
Dave Tucker de6fa98963 aya: Fix review comments from #387
Signed-off-by: Dave Tucker <dave@dtucker.co.uk>
3 years ago
Andrew Stoycos 8a9cbf179f Add `from_pinned` and `from_fd` methods
Add `from_pinned` to allow loading BPF maps
from pinned points in the bpffs and
`from_fd` to allow loading BPF maps from
RawFds aquired via some other means eg
a unix socket.

These functions return an
aya::Map which has not been used previously
but will be the future abstraction once
all bpf maps are represented as an enum.

Signed-off-by: Andrew Stoycos <astoycos@redhat.com>
3 years ago
Michal Rostecki 944d6b8a16 Change from Rust edition 2018 to 2021
Rust 2021 adds more core prelude imports, including `TryFrom` and
`TryInto`.

Signed-off-by: Michal Rostecki <vadorovsky@gmail.com>
3 years ago
Dave Tucker 03a15b9864 aya: Remove MapError::InvalidPinPath
Signed-off-by: Dave Tucker <dave@dtucker.co.uk>
3 years ago
Dave Tucker 34ba2bc048 aya: Use PinError for all pinning errors
Signed-off-by: Dave Tucker <dave@dtucker.co.uk>
3 years ago
Dave Tucker c9e70a8758 aya: Fix rlimit warning on for 32bit systems
Signed-off-by: Dave Tucker <dave@dtucker.co.uk>
3 years ago
Dave Tucker f976229477 Support BTF Maps
This commit allows for BTF maps in the .maps ELF section to be parsed.
It reads the necessary information from the BTF section of the ELF file.
While the btf_ids of Keys and Values types are stored, they are not (yet)
used.

When creating a BTF map, we pass the btf_key_type_id and
btf_value_type_id.

Signed-off-by: Dave Tucker <dave@dtucker.co.uk>
3 years ago
Michal Rostecki 3d592d0f29 aya: Raise the RLIMIT_MEMLOCK warning only if failed to create a map
Also, mention that setting the RLIMIT_MEMLOCK to a higher value is an
option.

Signed-off-by: Michal Rostecki <vadorovsky@gmail.com>
3 years ago
Dave Tucker 336faf553e clippy: Fix latest nightly lints
Signed-off-by: Dave Tucker <dave@dtucker.co.uk>
3 years ago
Alessandro Decina a301a56316
Merge pull request #328 from drewkett/map-update-no-key
Have bpf_map_update_elem take Option<&K> for key
3 years ago
Dave Tucker b4413322e3 aya: Replace ProgramFd trait with struct
This removes the ProgramFd trait with a struct that wraps a RawFd.
Program::fd() has been implemented as well as fd() for each Program
Type. This allows for a better API than requiring the use of the
ProgramFd trait.

Signed-off-by: Dave Tucker <dave@dtucker.co.uk>
3 years ago
Andrew Burkett 36edf09254 Have bpf_map_update_elem take Option<&K> for key
bpf_map_update_elem is used in lieu of bpf_map_push_elem to maintain support for kernel version < 4.20. The kernel expects a null pointer for the key for this use case. With this change, if you pass None as key to `bpf_map_update_elem`, it will pass null as key.
3 years ago
Dave Tucker 623579a47f aya: Add Map::fd() function to return a MapFd
Signed-off-by: Dave Tucker <dave@dtucker.co.uk>
3 years ago
Kenjiro Nakayama c192817a59 Fix typo, take & to query the value 3 years ago
Kenjiro Nakayama c4262f793d Add support for BPF_MAP_TYPE_BLOOM_FILTER
This patch adds support for `BPF_MAP_TYPE_BLOOM_FILTER`.
3 years ago
Dave Tucker 4a32e7d985 clippy: fix new lints on nightly
Signed-off-by: Dave Tucker <dave@dtucker.co.uk>
3 years ago
Dave Tucker d1f2215193 aya: Relocate maps using symbol_index
Since we support multiple maps in the same section, the section_index is
no longer a unique way to identify maps. This commit uses the symbol
index as the identifier, but falls back to section_index for rodata
and bss maps since we don't retrieve the symbol_index during parsing.

Signed-off-by: Dave Tucker <dave@dtucker.co.uk>
4 years ago
Alessandro Decina ad1636d2e7 aya: perf_buffer: call BytesMut::reserve() internally
This changes PerfBuffer::read_events() to call BytesMut::reserve()
internally, and deprecates PerfBufferError::MoreSpaceNeeded.

This makes for a more ergonomic API, and allows for a more idiomatic
usage of BytesMut. For example consider:

    let mut buffers = vec![BytesMut::with_capacity(N), ...];
    loop {
        let events = oob_cpu_buf.read_events(&mut buffers).unwrap();
        for buf in &mut buffers[..events.read] {
            let sub: Bytes = buf.split_off(n).into();
            process_sub_buf(sub);
        }
        ...
    }

This is a common way to process perf bufs, where a sub buffer is split
off from the original buffer and then processed. In the next iteration
of the loop when it's time to read again, two things can happen:

- if processing of the sub buffer is complete and `sub` has been
dropped, read_events() will call buf.reserve(sample_size) and hit a fast
path in BytesMut that will just restore the original capacity of the
buffer (assuming sample_size <= N).

- if processing of the sub buffer hasn't ended (eg the buffer has been
stored or is being processed in another thread),
buf.reserve(sample_size) will actually allocate the new memory required
to read the sample.

In other words, calling buf.reserve(sample_size) inside read_events()
simplifies doing zero-copy processing of buffers in many cases.
4 years ago
Alessandro Decina 9a642d373f aya: fix lint errors 4 years ago