Logging & Monitoring

Log Configuration

Rivellum uses Rust's tracing framework. Configure log levels via the RUST_LOG environment variable:

# Default info level
RUST_LOG=info rivellum-node start

# Debug consensus and execution
RUST_LOG=info,rivellum_consensus=debug,rivellum_photon=debug rivellum-node start

# Trace everything (very verbose)
RUST_LOG=trace rivellum-node start

Log Levels

Level	Use
`error`	Failures requiring attention
`warn`	Degraded conditions (queue pressure, retries)
`info`	Normal operations (batches sealed, committed)
`debug`	Detailed per-intent and per-batch tracing
`trace`	Full protocol message dumps

Per-Module Filtering

Filter specific components:

RUST_LOG=info,rivellum_consensus=debug           # Consensus details
RUST_LOG=info,rivellum_admission=debug            # Admission pool flow
RUST_LOG=info,rivellum_cpc=debug                  # Cross-lane transfers
RUST_LOG=info,rivellum_pouw=debug                 # PoUW job lifecycle
RUST_LOG=info,rivellum_wal=debug                  # WAL writes/replays

Prometheus Metrics

The node exposes Prometheus-format metrics:

Endpoint	Format	Content
`GET /v1/metrics`	Prometheus text	Core node metrics
`GET /v1/pouw/metrics`	Prometheus text	PoUW job metrics
`GET /v1/pouw/metrics/json`	JSON	PoUW metrics (machine-readable)
`GET /v1/zk/metrics`	Prometheus text	ZK proof metrics
`GET /v1/zk/metrics/json`	JSON	ZK metrics (machine-readable)

Key Metrics

Queue depths: Admission pool size, lane buffer levels
Throughput: Intents processed per second, batches sealed per minute
Latency: P50, P95, P99 intent processing latency
Consensus: Proposals, votes, QCs per round
PoUW: Pending/completed/failed proof jobs
State: Block height, state root, lane commit counts

Prometheus Scrape Config

scrape_configs:
  - job_name: 'rivellum-node'
    scrape_interval: 15s
    static_configs:
      - targets: ['rpc.rivellum.network']
    metrics_path: '/v1/metrics'

Event Streaming

Real-time event monitoring via Server-Sent Events:

# Stream all events
curl -N https://rpc.rivellum.network/events/stream

# Query recent events
curl https://rpc.rivellum.network/v1/events/recent

Diagnostics Endpoints

Endpoint	Description
`GET /health`	Node health check
`GET /v1/chain`	Chain info (height, epoch, validator count)
`GET /v1/diagnostics/cpc`	Cross-lane transfer diagnostics
`GET /v1/admission/diagnostics`	Admission pool diagnostics
`GET /v1/state_roots`	Current state roots per lane

Testnet Log Inspection

When running a testnet, validator logs are in the testnet directory:

# Check committee 0, validator 0 logs
tail -100 testnet/logs/committee-0-validator-0.log

# Search for consensus events
grep -iE 'consensus|batch|proposal|vote|QC' testnet/logs/committee-0-validator-0.log

# Check all committee logs
for f in testnet/logs/committee-*-validator-0.log; do
    echo "=== $(basename $f) ==="
    grep -iE 'batch|consensus' "$f" | tail -20
done

For node setup and hardware requirements, see Node Setup.