Overview

The Data Collector is ravenxcope-sensor-suricata running in server mode (historically called the sensor-api). It is the gRPC entry point for sensor alert events and the bridge into the Kafka pipeline.

Responsibilities

1. Accept a gRPC stream of SensorEvent messages from sensors (RegisterSensorServiceServer).
2. Process/queue events (internal/processor, internal/queue).
3. Produce them to Kafka using an Avro schema registered in the Schema Registry (internal/kafka_producer).
4. Expose Prometheus metrics (internal/prometheus_exporter).

Configuration (server mode)

Setting	Flag / key	Default
gRPC port	--port / port	50051
Kafka brokers	--kafka-brokers	from env
Schema Registry URL	--schema-registry-url / schema_registry_url	http://localhost:8081
Kafka topic	--kafka-topic / kafka_topic	sensor_events
Max gRPC message size	GRPCMaxMsgSize	MB-based recv/send limit

The listener binds GRPCHost:GRPCPort and serves a standard gRPC server with the configured max message sizes.

Transition note: In the current operational deployment the sensor streams to this collector, which feeds the Kafka pipeline. The backend also exposes its own mTLS gRPC SensorEventStreamService, intended as the future ingestion path — see the Data Plane scenario for how the two relate.

Downstream

sensor_events is consumed by the Event Aggregator.