eventmill_v01

Event Mill

Event record analysis platform for Security Operations and Detection Engineering teams.

License Python

What is Event Mill?

Event Mill is an open-source platform for analyzing unfamiliar event sources before committing to full SIEM integration. It lives upstream of the SIEM — in the gap between “we just got access to a new event source” and “we have a parser, field mappings, and detection rules in production.”

Value Propositions

  1. New source triage: Speed up initial analysis of unfamiliar event sources to determine whether they contain enough security-relevant information to warrant engineering investment.

  2. Incident-time analysis: During incidents, analysts receive event artifacts (logs, PCAPs, audit exports) for unfamiliar systems. Event Mill helps gain context quickly without requiring full knowledge of the event record structure.

What Event Mill is NOT

Architecture

Event Mill uses a three-layer architecture:

┌─────────────────────────────────────────────────────────────┐
│                     FRAMEWORK LAYER                          │
│  CLI • Session Management • LLM Orchestration • Routing     │
│  Artifact Registry • Plugin Lifecycle • Cloud Abstraction   │
└─────────────────────────────────────────────────────────────┘
                              │
┌─────────────────────────────────────────────────────────────┐
│                      PLUGIN LAYER                            │
│  Self-describing tools following EventMillToolProtocol      │
│  Organized by investigation pillar                          │
└─────────────────────────────────────────────────────────────┘
                              │
┌─────────────────────────────────────────────────────────────┐
│                     ROUTING LAYER                            │
│  Controls which plugins are visible to LLM per request      │
│  Prevents context bloat from full tool catalog              │
└─────────────────────────────────────────────────────────────┘

Investigation Pillars

Pillar Purpose Status
log_analysis Event source triage, threat intel ingestion, image analysis MVP
network_forensics PCAP triage, firewall log analysis MVP
threat_modeling Shostack 4-question framework, attack path visualization MVP
cloud_investigation Cloud audit log analysis Post-MVP
risk_assessment Risk scoring, control effectiveness Post-MVP

Providers and model tiers

Every LLM call is routed to one of two tiers, declared per plugin as model_tier in its manifest. Three providers can be bound at once, each offering both tiers:

Provider light heavy
gcp_gemini gemini-3.8-flash gemini-3.1-pro-preview
anthropic claude-sonnet-5 claude-opus-5
openai gpt-5.6-terra gpt-5.6-sol

light is bulk work — pattern summarization, IOC extraction, chunked reads. heavy is deep reasoning — threat modeling, risk assessment, synthesis. Within a provider the two tiers are capacity-identical, so the tier is a choice about reasoning depth and cost, never about how much fits. A plugin can override its manifest default per call with QueryHints, and the framework clamps output requests to what the selected model can actually emit.

Which vendor serves a tool is an operator decision, never a plugin’s. use <provider> sets the session default and use <provider> for <tool_name> overrides one module, which is how the same tool can be run on the same input past two vendors and compared — the prompt stays byte-identical across the swap and every response records the provider that served it. Automatic failover between vendors is deliberately not offered: it would send investigation data to a provider nobody selected and leave the result unattributable.

Model ids, token limits, and per-tier capabilities live in one declarative manifest per provider under framework/llm/providers/ rather than in code.

Quick Start

Installation

# Clone the repository
git clone https://github.com/eventmilldevops/eventmill_v01.git
cd eventmill_v01

# Install with pip
pip install -e ".[all]"

# Or install specific components
pip install -e ".[dev,plugins-log-analysis]"

Configuration

# Copy example environment file
cp .env.example .env

Event Mill talks to Gemini, Anthropic and OpenAI. Only Gemini splits keys by tier, so that high-volume light-tier traffic cannot exhaust the heavy tier’s quota; the other two issue one key per account:

GEMINI_FLASH_API_KEY=...   # light tier
GEMINI_PRO_API_KEY=...     # heavy tier
ANTHROPIC_API_KEY=...      # both tiers
OPENAI_API_KEY=...         # both tiers

You need only one of them. EVENTMILL_LLM_PROVIDERS names the providers a session may bind and defaults to all three; a provider whose key is absent is skipped at startup and reported as dormant, so setting one key and leaving the rest is a normal, working configuration. connect says which vendors bound and which did not.

A single GEMINI_API_KEY also works and binds to both Gemini tiers, so plugin manifests still drive model selection rather than every tool collapsing onto Flash. Gemini keys come from Google AI Studio.

Running

# Start the CLI
eventmill

# Or run directly
python -m framework.cli.shell

Inside the shell:

models              # list configured models and their tier
connect             # bind every available model (tiered auto-routing)
new                 # start an investigation session
pillar <name>       # set the investigation pillar
files [filters]     # list files in the pillar and common buckets
load <file|#N>      # register a file as an artifact
tools               # list available plugins and the name to invoke them by
help <tool_name>    # show a tool's arguments
run <tool_name> ... # run a tool
ask: <question>     # ask the LLM about the current session
providers           # which LLM providers are configured, keyed and bound
use <provider>      # choose the vendor that serves tools this session

files lists what a pillar can reach. On a large store, narrow it rather than scrolling it — --path <prefix>, --ext .log,.json, --newer 24h, --match "*auth*", --sort time|size|name, and --limit N:

files --ext .log --newer 24h
    #  Path                                     Source       Size  Modified
  ───  ──────────────────────────────────────── ─────── ─────────  ────────────
    1  linuxdroplettest/auth.log                pillar     2.1 MB  3h ago
    2  linuxdroplettest/auth.log.1              pillar   878.9 KB  9h ago

Rows are numbered, and #N stands in for a file wherever one is expected, so paths never have to be retyped:

load #2
run log_navigator --action read --path #2 --line_limit 100

#N refers to the listing you last saw. It is refused rather than guessed at if the pillar or workspace changed since, and run requires the file to be loaded first — downloading it is load’s job, not a side effect of running a tool.

Tools are always invoked through run, with arguments as --key value flags:

run threat_report_analyzer --action list_reports
run log_navigator --action read --path access.log --line_limit 100
run log_searcher --file_path access.log --query "Failed password" --context_lines 2

Flag values are typed from the tool’s input schema, so numbers and booleans arrive as numbers and booleans. A flag given without a value is a boolean and sets it true (--ai_analysis). A comma-separated value becomes a list (--ioc_types ip,domain).

Arguments that are lists of objects, or otherwise nested, cannot be expressed as flags. For those, pass a JSON payload instead — still after run <tool_name>:

run attack_path_visualizer {"format": "ascii", "stages": [{"name": "Initial Access"}]}

help <tool_name> lists every argument a tool accepts, with its type, default, and allowed values.

Directory Structure

eventmill_v01/
├── framework/              # Framework layer
│   ├── cli/               # Metasploit-style command shell
│   ├── session/           # Session management (SQLite)
│   ├── routing/           # Plugin routing and filtering
│   ├── llm/               # MCP client and LLM orchestration
│   ├── artifacts/         # Artifact registry
│   ├── plugins/           # Plugin lifecycle management
│   ├── reference_data/    # MITRE ATT&CK, attack chains, vetted sources
│   ├── logging/           # Structured logging
│   └── cloud/             # Cloud abstraction (GCP, local)
├── plugins/               # Plugin layer
│   ├── log_analysis/
│   ├── network_forensics/
│   ├── cloud_investigation/
│   ├── risk_assessment/
│   └── threat_modeling/
├── cloud_install/         # GCP provisioning + Cloud Run deployment
├── tests/                 # Test suites
├── scripts/               # CI and utility scripts
├── docs/                  # Documentation
│   ├── specs/            # Normative specifications
│   ├── guides/           # User guides
│   ├── change_log/       # Dated records of significant changes
│   └── reference/        # Reference documentation
├── AGENTS.md              # Day-one operational briefing
└── workspace/             # Runtime data (gitignored)

Deployment

Event Mill runs in production on Cloud Run, exposed as a browser terminal (ttyd). Provisioning and deployment are scripted:

export GOOGLE_CLOUD_PROJECT="your-project-id"
export CLOUD_RUN_REGION="us-central1"      # required — no default

bash cloud_install/provision-gcp-project.sh   # once per project
bash cloud_install/provision-secrets.sh       # set real secret values
bash cloud_install/deploy-cloudrun-secrets.sh # deploy

Provisioning is the only step that writes IAM, so the deploy path can run under a CI service account with no permission to change it. See cloud_install/README.md for the full guide, and AGENTS.md for the failure modes worth knowing before you start.

Plugin Development

Plugins are self-describing tools following the EventMillToolProtocol. Each plugin provides:

See Plugin Development Guide and Tool Plugin Spec.

Documentation

Document Purpose
Grounding Document Strategic context and MVP scope
Framework Architecture Component responsibilities and data flow
Tool Plugin Spec Normative plugin contract
Router Design Routing architecture and scoring
LLM Dispatcher Tiered routing and native document handling
Plugin Development Guide How to build a plugin, including model tier selection
Cloud Installation GCP provisioning and Cloud Run deployment
AGENTS.md Operational briefing — commands, layout, deployment traps
Change Log Dated records of significant changes

Contributing

Contributions welcome! Please read the plugin development guide before submitting new tools.

pip install -e ".[all]"              # dev + gcp + all plugin extras

pytest                               # collects tests/ and plugins/
ruff check .                         # line-length 88
black .
mypy framework plugins

python scripts/validate_manifests.py # plugin manifests
python scripts/validate_schemas.py   # JSON schemas

Maintainers

Event Mill is maintained by a small group of security practitioners focused on detection engineering, incident response, and cyber threat informed detection.

Current maintainers:

Please use GitHub Issues for bug reports, feature requests, and design discussions. Pull Requests are welcome, especially for new plugins, artifact parsers, investigation workflows, documentation improvements, and test coverage.

License

Apache License 2.0. See LICENSE for details.

This site is open source. Improve this page.