Commit Graph

2077 Commits (89205f25fa25bc93ddb46412a9c453848ef029e0)
 

Author SHA1 Message Date
Krish Sharma 89205f25fa docs: add traffic monitor development history to CLAUDE.md
Development Documentation:
- Complete development history and commit reference
- Four-phase implementation breakdown with detailed descriptions
- Key commit hashes with functional descriptions
- Project structure overview with file descriptions
- Build and test command reference

Reproducibility Guide:
- Step-by-step development phases documented
- Commit-by-commit feature implementation tracking
- Docker and testing infrastructure setup
- Usage examples for all major features
- Troubleshooting and architecture guidance

This documentation allows future developers or Claude instances to:
- Understand the complete development process
- Reproduce the implementation step-by-step
- Navigate the codebase structure effectively
- Run tests and demonstrations independently
- Extend functionality with proper context

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
3 weeks ago
Krish Sharma 2f76f0e4e5 feat: add Docker containerization and demo infrastructure
Docker Setup:
- Dockerfile: Full development environment with eBPF support
- Dockerfile.simple: Lightweight testing environment
- Dockerfile.run: Production-ready container configuration
- Dockerfile.demo: Demonstration environment setup

Demo Infrastructure:
- demo.sh: Comprehensive demonstration script with system checks
- demo-cargo.toml: Standalone cargo configuration for demos
- demo-lib.rs: Library configuration for non-eBPF testing

Analysis Outputs:
- comprehensive_analysis.json: Detailed analysis report example
- threat_analysis.csv: CSV format threat analysis export

Docker Features:
- Multi-stage builds for optimized container sizes
- eBPF toolchain installation and configuration
- Network namespace setup for testing
- Automated testing and validation scripts

Demo Capabilities:
- System information gathering and display
- eBPF capability detection and validation
- Network interface enumeration
- Traffic generation and analysis examples
- Linux container environment verification

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
3 weeks ago
Krish Sharma ba73b4c2d1 docs: add comprehensive README with usage examples and architecture
README Documentation:
- Complete project overview and architecture description
- Detailed installation and setup instructions
- Comprehensive usage examples for all features
- Command-line reference with all options
- Structured logging format specifications

Key Documentation Sections:
- XDP vs other eBPF hook point comparison
- Performance considerations and tuning guide
- Security considerations and best practices
- Troubleshooting guide for common issues
- Development workflow and contribution guidelines

Usage Examples:
- Basic traffic monitoring setup
- Structured logging in multiple formats
- Log analysis and threat detection workflows
- Configuration management examples
- Docker and containerized deployments

Technical Details:
- eBPF program architecture and data flow
- Ring buffer communication mechanisms
- CIDR range configuration and validation
- Logging system design and performance
- Threat detection algorithms and thresholds

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
3 weeks ago
Krish Sharma 2e57c8da99 feat: add examples and sample data for testing and demonstration
Example Programs:
- standalone-demo.rs: Non-eBPF demo for testing core logic
- test-traffic.rs: Traffic generation for testing
- macos-demo.rs: macOS-compatible demonstration

Sample Log Data:
- sample_traffic.jsonl: Basic traffic examples in JSONL format
- sample_traffic.csv: CSV format examples for spreadsheet analysis
- comprehensive_traffic.jsonl: Complex traffic patterns with multiple protocols
- threat_traffic.jsonl: Examples triggering threat detection (port scanning, etc.)

Demonstration Features:
- Multi-protocol traffic examples (TCP, UDP, ICMP, GRE, ESP, AH)
- Port scanning simulation for threat detection testing
- High-volume traffic patterns for performance analysis
- Realistic IP addresses and network patterns
- Flow correlation examples with unique hashes

Use Cases:
- Development testing without requiring eBPF environment
- Log analysis script validation and testing
- Threat detection algorithm verification
- Performance benchmarking and optimization
- Documentation and training examples

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
3 weeks ago
Krish Sharma 8aef9d987e feat: add configuration examples and comprehensive test suite
Configuration Files:
- configs/default.json: Standard private network CIDR ranges
- configs/strict.json: Restrictive localhost-only configuration
- configs/logging-example.json: Example with logging settings

Test Suite (tests/integration_tests.rs):
- Comprehensive integration tests for all components
- CIDR parsing and IP matching validation
- Event handling and statistics verification
- Configuration loading and validation tests
- Error handling and edge case coverage

Features Tested:
- IP address parsing and CIDR range matching
- Traffic event processing and statistics
- Configuration file loading and validation
- Protocol detection and classification
- Event handler statistics and summaries

Quality Assurance:
- Edge case handling for malformed inputs
- Performance testing for high-volume scenarios
- Memory safety verification for eBPF integration
- Cross-platform compatibility testing

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
3 weeks ago
Krish Sharma 722d4c8d32 feat: add comprehensive traffic log analysis and threat detection
Analysis Script (scripts/analyze_logs.py):
- Multi-format log parser (JSON, JSONL, CSV with auto-detection)
- Comprehensive traffic analytics and statistical analysis
- Advanced threat detection and security intelligence
- Flexible export options for further analysis

Traffic Analytics:
- Traffic volume and bandwidth analysis
- Top source/destination IP identification
- Protocol distribution and temporal patterns
- Packet size analysis with percentiles
- Network flow correlation and tracking

Threat Detection Capabilities:
- Port scanning detection with configurable thresholds
- High-volume source identification using statistical analysis
- Unusual protocol detection for tunnel/VPN identification
- Suspicious traffic pattern recognition
- Repeated flow analysis for DDoS detection

Advanced Features:
- Statistical analysis with percentiles and distributions
- Temporal pattern analysis (hourly/daily trends)
- Security scoring and risk assessment
- Flow-based analysis with duration tracking
- Comprehensive reporting with multiple output formats

Export and Reporting:
- Human-readable console reports with emoji indicators
- JSON export for programmatic analysis
- CSV export for spreadsheet integration
- Detailed threat intelligence summaries

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
3 weeks ago
Krish Sharma e6bca88295 feat: implement userspace program with integrated logging
Main Program (main.rs):
- Command-line interface using clap with comprehensive options
- eBPF program lifecycle management (load, configure, attach)
- Integration with structured logging system
- Graceful shutdown handling with log flushing

Command-Line Options:
- Interface selection for XDP attachment
- Configuration file path for CIDR ranges
- Packet action control (log-only vs drop)
- Logging format selection (console/json/csv/jsonl)
- Output file specification and buffer size tuning
- Verbose logging for debugging

eBPF Integration:
- Automatic eBPF program loading and BTF initialization
- CIDR range configuration via eBPF maps
- Real-time event processing from ring buffers
- XDP attachment with fallback to SKB mode

Enhanced Features:
- Structured event logging with flow correlation
- Real-time statistics and periodic summaries
- Signal handling for clean shutdown
- Error handling with context preservation

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
3 weeks ago
Krish Sharma da81c221d3 feat: add comprehensive structured logging system
Structured Logging Module (logger.rs):
- Multiple output formats: JSON, CSV, JSONL, Console
- Configurable buffering and log rotation settings
- Thread-safe logging with Arc<Mutex<>> for concurrent access
- Rich log entries with metadata and flow correlation

Log Entry Features:
- Timestamp (Unix and ISO 8601 formats)
- Source/destination IP addresses and ports
- Protocol information (name and number)
- Packet size and action taken (LOG/DROP)
- Network interface and unique flow hash
- Structured serialization with serde

Output Format Support:
- Console: Human-readable real-time logging
- JSON: Structured array format for batch processing
- JSONL: Line-delimited JSON for streaming analytics
- CSV: Spreadsheet-compatible format with headers

Performance Optimizations:
- Buffered I/O with configurable buffer sizes
- Efficient serialization and string formatting
- Minimal allocation during high-throughput logging

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
3 weeks ago
Krish Sharma 21bd2041e7 feat: implement core eBPF traffic monitoring functionality
Core eBPF Program (traffic_monitor.bpf.rs):
- XDP-based packet processing for high performance
- IP header parsing and CIDR range matching
- Configurable packet dropping or logging
- Ring buffer event logging to userspace

Supporting Modules:
- config.rs: JSON configuration management for CIDR ranges
- ip_utils.rs: CIDR parsing and IP matching utilities
- event_handler.rs: Traffic event processing and statistics
- lib.rs: Module exports and shared structures

Key Features:
- Line-rate packet filtering in kernel space
- Support for up to 256 permitted CIDR ranges
- Real-time event streaming via ring buffers
- Protocol-aware logging (TCP/UDP/ICMP/etc.)

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
3 weeks ago
Krish Sharma c590290bdf feat: initialize eBPF traffic monitor project structure
- Add Cargo.toml with aya dependencies and project configuration
- Add build.rs with aya-build integration for eBPF compilation
- Configure workspace and example targets

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
3 weeks ago
Tamir Duberstein 44b26b3b50 integration-test: use only readable interest 3 weeks ago
Tamir Duberstein 5fa52ac5c1 integration-test: remove an allocation 3 weeks ago
Tamir Duberstein f08772ec2f test-distro: enable rust backtrace 3 weeks ago
dependabot[bot] cfff75166c build(deps): update public-api requirement in the cargo-crates group
Updates the requirements on [public-api](https://github.com/cargo-public-api/cargo-public-api) to permit the latest version.

Updates `public-api` to 0.47.1
- [Release notes](https://github.com/cargo-public-api/cargo-public-api/releases)
- [Changelog](https://github.com/cargo-public-api/cargo-public-api/blob/main/CHANGELOG.md)
- [Commits](https://github.com/cargo-public-api/cargo-public-api/compare/public-api-v0.47.0...public-api-v0.47.1)

---
updated-dependencies:
- dependency-name: public-api
  dependency-version: 0.47.1
  dependency-type: direct:production
  dependency-group: cargo-crates
...

Signed-off-by: dependabot[bot] <support@github.com>
1 month ago
Tamir Duberstein 30d5c9f53e Remove dead code expectation fixed upstream
See https://github.com/rust-lang/rust/commit/c93fac7d64394c6b926d863c19.
1 month ago
tamird bd0424ca61 aya-obj, aya-ebpf-bindings: regenerate
libbpf commit: 20ea95b4505c477af3b6ff6ce9d19cee868ddc5d
2 months ago
Tamir Duberstein eda29958f8 Use CRABBY_GITHUB_TOKEN for regen PRs
See https://github.com/peter-evans/create-pull-request/issues/48.
2 months ago
dependabot[bot] 6d5826fe99
Merge pull request #1277 from aya-rs/dependabot/cargo/cargo-crates-2948a9e609 2 months ago
dependabot[bot] f96250e80d
build(deps): bump the cargo-crates group with 2 updates
Updates the requirements on [bindgen](https://github.com/rust-lang/rust-bindgen) and [which](https://github.com/harryfei/which-rs) to permit the latest version.

Updates `bindgen` to 0.71.1
- [Release notes](https://github.com/rust-lang/rust-bindgen/releases)
- [Changelog](https://github.com/rust-lang/rust-bindgen/blob/main/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/rust-bindgen/compare/v0.71.0...v0.71.1)

Updates `which` to 7.0.3
- [Release notes](https://github.com/harryfei/which-rs/releases)
- [Changelog](https://github.com/harryfei/which-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/harryfei/which-rs/compare/7.0.0...7.0.3)

---
updated-dependencies:
- dependency-name: bindgen
  dependency-version: 0.71.1
  dependency-type: direct:production
  dependency-group: cargo-crates
- dependency-name: which
  dependency-version: 7.0.3
  dependency-type: direct:production
  dependency-group: cargo-crates
...

Signed-off-by: dependabot[bot] <support@github.com>
2 months ago
dependabot[bot] d1ed76e626
Merge pull request #1273 from aya-rs/dependabot/cargo/cargo-crates-af2cda06bf 2 months ago
dependabot[bot] a7e3e6d4d9
build(deps): bump the cargo-crates group with 2 updates
Updates the requirements on [cargo_metadata](https://github.com/oli-obk/cargo_metadata) and [object](https://github.com/gimli-rs/object) to permit the latest version.

Updates `cargo_metadata` to 0.19.2
- [Release notes](https://github.com/oli-obk/cargo_metadata/releases)
- [Changelog](https://github.com/oli-obk/cargo_metadata/blob/main/CHANGELOG.md)
- [Commits](https://github.com/oli-obk/cargo_metadata/compare/0.19.0...0.19.2)

Updates `object` to 0.36.7
- [Changelog](https://github.com/gimli-rs/object/blob/master/CHANGELOG.md)
- [Commits](https://github.com/gimli-rs/object/compare/0.36.0...0.36.7)

---
updated-dependencies:
- dependency-name: cargo_metadata
  dependency-version: 0.19.2
  dependency-type: direct:production
  dependency-group: cargo-crates
- dependency-name: object
  dependency-version: 0.36.7
  dependency-type: direct:production
  dependency-group: cargo-crates
...

Signed-off-by: dependabot[bot] <support@github.com>
2 months ago
dependabot[bot] e239f3abbd
Merge pull request #1269 from aya-rs/dependabot/github_actions/github-actions-b6130af3f2 2 months ago
dependabot[bot] e1fa363619
build(deps): bump DavidAnson/markdownlint-cli2-action
Bumps the github-actions group with 1 update: [DavidAnson/markdownlint-cli2-action](https://github.com/davidanson/markdownlint-cli2-action).


Updates `DavidAnson/markdownlint-cli2-action` from 19 to 20
- [Release notes](https://github.com/davidanson/markdownlint-cli2-action/releases)
- [Commits](https://github.com/davidanson/markdownlint-cli2-action/compare/v19...v20)

---
updated-dependencies:
- dependency-name: DavidAnson/markdownlint-cli2-action
  dependency-version: '20'
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: github-actions
...

Signed-off-by: dependabot[bot] <support@github.com>
2 months ago
Tamir Duberstein 37257fe7cd
Fix MD059/descriptive-link-text 2 months ago
Tamir Duberstein ccf6c4707f aya-ebpf: disable generic_const_exprs
This has recently regressed on nightly.

See https://github.com/rust-lang/rust/issues/141492.
2 months ago
Tamir Duberstein 4f654865e9 aya-ebpf: add a dedicated generic_const_exprs cfg 2 months ago
Tamir Duberstein 630a767117 Split relocation tests into multiple files
This avoids requiring kernel support for all types of relocations when
testing a specific type.
2 months ago
Tamir Duberstein c8e9037ca6 Avoid uncontrolled stdout into cargo
This fixes rebuild on change of C sources by black-holing:
- Output of `make` (harmless).
- Output of `llmv-objcopy` (harmful: binary data).
2 months ago
Tamir Duberstein 025b6eaa0d Use a macro to reduce boilerplate 2 months ago
Tamir Duberstein b6daf463e6 Remove stale comment since a1b46ece05 2 months ago
Mehnaz Yunus 4b5ba53a36 aya: implement TryFrom<[Program Type]> for FdLink for various program types
Implements TryFrom for FdLink for CgroupSkb, CgroupSock, CgroupSockAddr
and SockOps program types. This allows support for link pinning for
these program types, aligning with the documentation for FdLink.

Fixes: #739

Co-authored-by: Benjamin Barzen <bbarzen@amazon.com>
2 months ago
Tyrone Wu 7a0dabc295 aya: short-circuit info field if non-zero
Short-circuits `CACHE` to true if the field is non-zero.

This saves from executing the probing logic since the logic essentially
checks if the field can process (or doesn't error) non-zero value.
2 months ago
Tyrone Wu 23bc5b5836 aya,aya-obj: cache feat probed info fields
Cached probed for ProgramInfo fields instead of exposing it through
global FEATURE. Probing occurs on cache miss, which happens when first
accessing the field, *and* if the field is 0.
2 months ago
Tyrone Wu bd492860f5 aya: add feature probing for map type
Add API that probes whether kernel supports a map type.
2 months ago
Tyrone Wu ab77decd9a aya,aya-obj: add feature probing program type
Adds API that probes whether kernel supports a program type.
2 months ago
Dave Tucker f74a157907
Merge pull request #1262 from dave-tucker/stabilize-links
feat(aya): Make LinkInfo and loaded_links public
3 months ago
Dave Tucker 7dba5a41ad feat(aya): Make LinkInfo and loaded_links public
We have had loaded_links in the API as `#[doc(hidden)]` for a while.
I've been using it in bpfman and it's been fine. This commit does the
minimal work required to make the API stable.

We expose a `LinkInfo` type - similar to `ProgInfo` - which wraps the
generated type. In this case, `bpf_link_info`.

A few accessor functions have been added for `id`, `link_type` and
`program_id`. There are many more fields that could be (eventually)
made public.

As a convenience, `LinkInfo` can be retrieved from an existing FdLink
by using `FdLink::info()`.

Signed-off-by: Dave Tucker <dave@dtucker.co.uk>
3 months ago
dependabot[bot] 92c73a72e7
Merge pull request #1263 from aya-rs/dependabot/cargo/cargo-crates-b22b264ef4 3 months ago
dependabot[bot] 95c03e6600
build(deps): update nix requirement in the cargo-crates group
Updates the requirements on [nix](https://github.com/nix-rust/nix) to permit the latest version.

Updates `nix` to 0.29.0
- [Changelog](https://github.com/nix-rust/nix/blob/master/CHANGELOG.md)
- [Commits](https://github.com/nix-rust/nix/compare/v0.29.0...v0.29.0)

---
updated-dependencies:
- dependency-name: nix
  dependency-version: 0.29.0
  dependency-type: direct:production
  dependency-group: cargo-crates
...

Signed-off-by: dependabot[bot] <support@github.com>
3 months ago
Omri Steiner 33c9f2b2b2 aya: uprobe: use PathBuf for ResolveSymbolError::DebuglinkAccessError
Makes no sense to use a string, as it's a path.
This breaks the public API.
3 months ago
Omri Steiner 90d5604877 aya: uprobe attach: use mmap instead of reading whole binaries to memory 3 months ago
Omri Steiner 647100faa7 aya: clean up resolve_symbol a bit
Instead of using intermediate values to extend the lifetime of the
object::File, we just separate the branches.
3 months ago
Omri Steiner 3aded0e0a5 aya: move Mmap struct to aya::util 3 months ago
Tamir Duberstein 583709f6a0 appease `clippy::uninlined-format-args` 3 months ago
dependabot[bot] c65a200e9a
Merge pull request #1256 from aya-rs/dependabot/cargo/cargo-crates-9bb5a8e9c0 3 months ago
dependabot[bot] c0ff1aa0cc
build(deps): update network-types requirement in the cargo-crates group
Updates the requirements on [network-types](https://github.com/vadorovsky/network-types) to permit the latest version.

Updates `network-types` to 0.0.7
- [Release notes](https://github.com/vadorovsky/network-types/releases)
- [Changelog](https://github.com/vadorovsky/network-types/blob/main/CHANGELOG.md)
- [Commits](https://github.com/vadorovsky/network-types/compare/v0.0.7...v0.0.7)

---
updated-dependencies:
- dependency-name: network-types
  dependency-version: 0.0.7
  dependency-type: direct:production
  dependency-group: cargo-crates
...

Signed-off-by: dependabot[bot] <support@github.com>
3 months ago
Tamir Duberstein 6004fcdb0f aya-ebpf: put mem{set,move,cpy} behind cfg(target_arch = "bpf")
Address some lints while I'm here.
3 months ago
Tamir Duberstein eab5661a0e aya-build: enable BTF 3 months ago
Tamir Duberstein 0237e36dbe ci: test against 6.12 rather than 6.10
6.12 is an LTS kernel, while 6.10 is not. The latter has reached EOL and
no longer lives on debian mirrors.
3 months ago
Tamir Duberstein 5732b2c203 test-distro: build without cross toolchain
Make the xz2 dependency optional to allow building without a C cross
compiler. This allows clippy.sh to be used on e.g. macOS more easily:

```
./clippy.sh --target x86_64-unknown-linux-gnu --exclude-features xz2
```
4 months ago