Architecture

Introduction

This document describes the layered architecture of Ravenxcope.Backend, the dependency direction between layers, the runtime initialization flow, and important design decisions that shape the codebase. It also highlights the fresh-deployment sensor architecture where a physical Sensor enrolls once through a host agent and each VirtualSensor is deployed independently onto that host.

---

Layering Approach

The backend is organized into distinct layers, each with a clear responsibility:

Layer	Directory	Responsibility
API	API/Controllers/	Thin HTTP/gRPC endpoint handlers, request routing, auth annotations
Application	Application/DTOs/	Request/response payload definitions, API envelope
Domain	Domain/Entities/	Core business entities persisted in PostgreSQL, including physical sensors, virtual sensors, and structured sensor interface inventory
Infrastructure	Infrastructure/	Services, repositories, DbContext, external integrations
Common	Common/Helpers/	Shared utilities used across all layers
Extensions	Extensions/	Startup composition (DI, middleware, migration, validation)

---

Dependency Direction

Current dependency rules:

• API depends on Application (DTOs), and Infrastructure (service interfaces). Controllers should delegate to services and avoid direct persistence access.
• Infrastructure depends on Domain (entities) and external SDKs (EF Core, Redis, InfluxDB, OpenSearch)
• Domain has no outward dependencies — pure entity definitions
• Common can be used by all layers — shared helpers
• Program.cs and Extensions orchestrate startup only

Note: All feature controllers now delegate to dedicated service interfaces (e.g., IAuthService, ISensorsService, IAssetService). All entities have corresponding repository abstractions under Infrastructure/Repositories/.

---

Runtime Architecture

The application progresses through three distinct phases during startup:

Phase 1: Builder (Service Registration)

1. Configure Logging: Initialize Serilog logging from appsettings
2. Validate Configuration: Ensure all required configuration keys are present
3. Security Check: Emit JWT secret length warning if < 32 characters
4. Log Settings: Log auto-migration configurations
5. Bind Typed Options: Map configuration sections to strongly-typed classes
6. Register Controllers: Configure API controllers with case-insensitive JSON
7. Register API Docs: Add EndpointsApiExplorer and Swagger
8. Register gRPC: Add gRPC services
9. Rate Limiting: Set auth-login (5/min) and auth-register (3/min)
10. Database: Configure PostgreSQL DbContext with Npgsql
11. Caching: Connect to Redis (eager connection)
12. Singleton Services: Register JwtService, InfluxDbService, OpenSearchService, RedisService, ResilientHttpService
13. Scoped Repositories: Register all entity repositories (e.g., UserRepository, SensorRepository)
14. Scoped Services: Register all business logic services (e.g., AuthService, SensorsService)
15. HTTP Client: Register generic HttpClient factory
16. Authentication: Configure JWT Bearer auth with token blacklist check
17. Authorization: Configure role/permission-based authorization policies
18. CORS: Register CORS policy

Phase 2: Health Checks & Migrations

1. Check PostgreSQL: Run connectivity check with retry mechanism
2. Check Redis: Run PING check with retry mechanism
3. Check InfluxDB: Run /health endpoint check with retry mechanism
4. Check OpenSearch: Run root endpoint check with retry mechanism
5. Fail-Fast: Halt startup immediately if any check fails
6. Apply Migrations: Execute EF Core database migrations (if AutoMigrate = true)

Phase 3: Middleware Pipeline

1. Request Logging: Serilog HTTP request logging
2. Exception Handling: Global exception handler (ApiExceptionMapper)
3. Swagger UI: Serve API documentation
4. CORS: Apply Cross-Origin Resource Sharing
5. HTTPS Redirection: Force secure connections
6. Rate Limiting: Apply endpoint limits
7. Authentication: Validate JWT tokens
8. Authorization: Enforce access policies
9. Map Controllers: Route HTTP requests to endpoints
10. Map gRPC: Route SensorHealthcheckService

---

Extension Method Architecture

Each startup responsibility is isolated in a dedicated extension method:

Extension Class	Method	Purpose
ServiceCollectionExtensions	AddBackendServicesAsync()	All DI registrations (services, repos, DB, auth, rate limiting)
ConfigurationValidationExtensions	ValidateRequiredConfiguration()	Startup config key validation
StartupDependencyHealthChecksExtensions	RunStartupDependencyHealthChecksAsync()	PostgreSQL, Redis, InfluxDB, OpenSearch checks
DatabaseMigrationExtensions	ApplyDatabaseMigrationsAsync()	Auto-migration with retry logic
WebApplicationExtensions	UseBackendPipeline()	Middleware pipeline composition
GlobalExceptionHandlingExtensions	UseGlobalExceptionHandling()	Global exception handler registration
CorsExtensions	AddCorsPolicy()	CORS policy registration

This gives Program.cs a clean, declarative bootstrap sequence:

var builder = WebApplication.CreateBuilder(args);
// ... logging, validation, warnings ...
await builder.Services.AddBackendServicesAsync(builder.Configuration);
var app = builder.Build();
await app.RunStartupDependencyHealthChecksAsync();
await app.ApplyDatabaseMigrationsAsync();
app.UseBackendPipeline();
await app.RunAsync();

---

Important Design Decisions

1. Service-first controller design — All controllers delegate business logic to dedicated service interfaces (e.g., IAuthService, ISensorsService, IAssetService). Controllers contain no business logic, only HTTP transport concerns.

2. Repository pattern for all entities — Every entity has a corresponding repository abstraction (IRepository<T> base + feature-specific interfaces). Services depend on repository interfaces, not ApplicationDbContext directly.

3. Singleton infrastructure services — JwtService, InfluxDbService, OpenSearchService, RedisService, and ResilientHttpService are registered as singletons because they manage their own connection state and are thread-safe.

4. Eager Redis connection — Redis ConnectionMultiplexer is connected during service registration (AddBackendServicesAsync), not lazily. This means a Redis failure blocks DI setup.

5. Fail-fast startup philosophy — The service deliberately crashes on startup for any configuration or dependency failure, rather than starting in a degraded state.

6. Typed options over ad-hoc reads — All configuration sections are bound to strongly-typed option classes (defined in BackendConfiguration.cs), avoiding scattered configuration["key"] calls.

7. Composite runtime options — SensorRuntimeOptions aggregates settings from multiple configuration sections into a single options class for convenience in sensor-related operations.

8. Standardized API envelope — All endpoints return ApiResponse<T> via ApiEnvelope.Success() and ApiEnvelope.Error(), providing a consistent { success, data, errors, traceId } response contract.

9. Global exception handling — GlobalExceptionHandlingExtensions catches unhandled exceptions and maps them via ApiExceptionMapper to appropriate HTTP status codes, eliminating manual try-catch patterns.

10. Resilient external calls — ResilientHttpService wraps outbound HTTP calls with retry (3 attempts with backoff) and circuit breaker (opens after 3 failures, 30s cooldown) patterns.

11. Rate limiting on auth endpoints — Login (5 req/min) and register (3 req/min) are protected by ASP.NET Core fixed-window rate limiters.

12. Host-agent-first sensor model — A physical Sensor is enrolled once through a one-time install token and a durable agent token. That host agent reports heartbeat, host metrics, and discovered network interfaces, while VirtualSensor records only workload-level deployment state and assigned capture interfaces.

13. Structured interface inventory — The backend persists sensor interfaces as first-class rows (SensorNetworkInterface) rather than runtime JSON blobs. This allows backend-side eligibility validation and exclusive interface assignment to virtual sensors.

14. Separated telemetry measurements — InfluxDB stores host presence in sensor_status and host metrics/interface counters in sensor_metrics, so online state is derived from heartbeat data rather than inferred from CPU series.

---

Known Architecture Debt

Area	Issue	Status	Impact
Test coverage breadth	Coverage is improving, but many services/controllers still lack deeper integration and contract tests	Open	Regression risk remains for less-tested flows
Analytics graceful fallback	Graceful fallback is now applied across analytics endpoints with stale-cache fallback patterns; response metadata contracts may still vary by endpoint shape	Partial	Consumers should still handle endpoint-specific fallback payload nuances
Analytics precompute/warming	Background cache warming exists for key dashboard queries, but coverage and schedule tuning are still basic	Partial	Remaining cold-start spikes possible on non-warmed query shapes

Resolved debt items (previously open):

• Audit logging in RoleService, UserService, and PermissionService.
• Analytics response caching for high-frequency endpoints.
• Hosted analytics cache warming service for key dashboard paths.