Observability Field Matrix¶

This document defines the canonical field inventory for the Observability domain.

The field matrix sits between the OpenTelemetry telemetry inventory and the canonical data model. It records how telemetry emitted by the Observability package is represented, what dimensions it carries, and where it is consumed.

The matrix is derived from:

Dashboard specifications
PromQL and LogQL references
The Observability Metrics Catalogue
The current OpenTelemetry resource, metric, log, and trace instrumentation

The field matrix does not define dashboard behaviour or PromQL. It defines the data fields available to downstream systems.

Purpose¶

The field matrix provides a stable mapping between telemetry produced by the Observability package and the canonical organizational data model.

It answers:

What telemetry fields exist?
Which instrumentation produces them?
What does each field represent?
At what scope does the field exist?
What dimensions distinguish its values?
Which dashboards consume it?
Which fields can be mapped directly into canonical concepts?
Which fields require derivation or further modelling?

Field Matrix¶

Service and Resource Identity¶

These fields identify the service, deployment, process, host, and runtime associated with telemetry.

Signal	Instrumentation	OTel Field	Field Type	Value Type	Scope	Description	Canonical ID	Mapping Type	Used By	Status
resource	Core	`service.name`	Resource Attribute	string	Resource	Logical name of the instrumented service	`observability.service.name`	direct	All dashboards	stable
resource	Core	`service.version`	Resource Attribute	string	Resource	Deployed service version	`observability.service.version`	direct	Service Overview	stable
resource	Core	`service.instance.id`	Resource Attribute	string	Resource	Unique identifier for a service instance	`observability.service.instance_id`	direct	Service Overview, Infrastructure	stable
resource	Core	`deployment.environment`	Resource Attribute	string	Resource	Deployment environment	`observability.environment`	direct	All dashboards	stable
resource	Core	`host.name`	Resource Attribute	string	Resource	Hostname on which the process runs	`observability.host.name`	direct	Infrastructure	stable
resource	Core	`host.arch`	Resource Attribute	string	Resource	Host CPU architecture	`observability.host.architecture`	direct	Infrastructure	stable
resource	Core	`os.type`	Resource Attribute	string	Resource	Operating system type	`observability.os.type`	direct	Infrastructure	stable
resource	Core	`process.pid`	Resource Attribute	integer	Resource	Operating system process identifier	`observability.process.pid`	direct	Infrastructure	stable
resource	Core	`process.runtime.name`	Resource Attribute	string	Resource	Runtime implementation used by the process	`observability.runtime.name`	direct	Infrastructure	stable
resource	Core	`process.runtime.version`	Resource Attribute	string	Resource	Runtime version	`observability.runtime.version`	direct	Infrastructure	stable

HTTP Server Metrics¶

These fields describe incoming HTTP traffic handled by an instrumented service.

Signal	Instrumentation	OTel Field	Prometheus Name	Field Type	Value Type	Scope	Unit	Dimensions	Description	Canonical ID	Mapping Type	Used By	Status
metric	FastAPI	`http.server.request.duration`	`http_server_duration_milliseconds`	Metric	Histogram	Datapoint	ms	`http_method`, `http_status_code`, `http_target`	Duration of incoming HTTP requests	`observability.http.request.duration`	direct	Service Overview, HTTP/API	stable
metric	FastAPI	`http.server.active_requests`	`http_server_active_requests`	Metric	Gauge	Datapoint	requests	`http_method`	Number of currently active HTTP requests	`observability.http.active_requests`	direct	Service Overview, HTTP/API	stable

HTTP Derived Fields¶

Some dashboard values are derived from the underlying HTTP metrics rather than emitted as independent telemetry.

Derived Field	Source Metric	Derivation	Canonical ID	Used By
Request Rate	`http_server_duration_milliseconds`	Rate of completed request observations	`observability.http.request_rate`	Service Overview, HTTP/API
Error Rate	`http_server_duration_milliseconds`	Ratio of 5xx responses to total responses	`observability.http.error_rate`	Service Overview, HTTP/API
Average Latency	`http_server_duration_milliseconds`	Duration sum divided by request count	`observability.http.average_latency`	Service Overview
P50 Latency	`http_server_duration_milliseconds`	Histogram quantile 0.50	`observability.http.p50_latency`	HTTP/API
P95 Latency	`http_server_duration_milliseconds`	Histogram quantile 0.95	`observability.http.p95_latency`	Service Overview, HTTP/API
P99 Latency	`http_server_duration_milliseconds`	Histogram quantile 0.99	`observability.http.p99_latency`	HTTP/API
Top Endpoints	`http_server_duration_milliseconds`	Aggregation by `http_target`	`observability.http.top_endpoints`	Service Overview, HTTP/API
Slowest Endpoints	`http_server_duration_milliseconds`	Latency aggregation by `http_target`	`observability.http.slowest_endpoints`	Service Overview, HTTP/API
Status Distribution	`http_server_duration_milliseconds`	Aggregation by `http_status_code`	`observability.http.status_distribution`	Service Overview, HTTP/API

These derived fields belong to the analytical layer rather than the raw telemetry inventory.

HTTP Client Metrics¶

These fields describe outbound HTTP requests generated by an instrumented service.

Signal	Instrumentation	OTel Field	Prometheus Name	Field Type	Value Type	Scope	Unit	Dimensions	Description	Canonical ID	Mapping Type	Used By	Status
metric	HTTPX	`http.client.request.duration`	`http_client_request_duration`	Metric	Histogram	Datapoint	ms	`http_request_method`, `http_response_status_code`, `server_address`	Duration of outbound HTTP requests	`observability.http.client.request.duration`	direct	External Services	stable
metric	HTTPX	`http.client.active_requests`	`http_client_active_requests`	Metric	Gauge	Datapoint	requests	`http_request_method`	Number of active outbound HTTP requests	`observability.http.client.active_requests`	direct	External Services	stable

HTTP Client Derived Fields¶

Derived Field	Source Metric	Derivation	Canonical ID	Used By
HTTP Client Request Rate	`http_client_request_duration`	Rate of outbound request observations	`observability.http.client.request_rate`	External Services
HTTP Client P95 Latency	`http_client_request_duration`	Histogram quantile 0.95	`observability.http.client.p95_latency`	External Services
Top Dependencies	`http_client_request_duration`	Aggregation by `server_address`	`observability.http.client.top_dependencies`	External Services

Database Metrics¶

These fields describe database operations performed by instrumented services.

Signal	Instrumentation	OTel Field	Prometheus Name	Field Type	Value Type	Scope	Unit	Dimensions	Description	Canonical ID	Mapping Type	Used By	Status
metric	SQLAlchemy / psycopg2	`db.client.operation.duration`	`db_client_operation_duration`	Metric	Histogram	Datapoint	ms	`db_operation`, `db_system_name`, `db_name`	Duration of database operations	`observability.database.operation.duration`	direct	Database	stable
metric	SQLAlchemy / psycopg2	`db.client.operation.active`	`db_client_operation_active`	Metric	Gauge	Datapoint	operations	`db_operation`	Number of currently active database operations	`observability.database.operation.active`	direct	Database	stable
metric	SQLAlchemy	`db.client.connections.usage`	`db_client_connections_usage`	Metric	Gauge	Datapoint	connections	`state`	Database connection usage by connection state	`observability.database.connections.usage`	direct	Database	stable
metric	SQLAlchemy	`db.client.connections.wait_time`	`db_client_connections_wait_time`	Metric	Histogram	Datapoint	ms	`db_operation`	Time spent waiting for database connections	`observability.database.connections.wait_time`	direct	Database	stable

Database Derived Fields¶

Derived Field	Source Metric	Derivation	Canonical ID	Used By
Database Request Rate	`db_client_operation_duration`	Rate of database operation observations	`observability.database.request_rate`	Database
Database P95 Latency	`db_client_operation_duration`	Histogram quantile 0.95	`observability.database.p95_latency`	Database
Database Connections	`db_client_connections_usage`	Aggregation across connection states	`observability.database.connections`	Database

Redis Metrics¶

Redis operations are represented using the database client metric family because Redis instrumentation produces database client telemetry.

Signal	Instrumentation	OTel Field	Prometheus Name	Field Type	Value Type	Scope	Unit	Dimensions	Description	Canonical ID	Mapping Type	Used By	Status
metric	Redis	`db.client.operation.duration`	`db_client_operation_duration`	Metric	Histogram	Datapoint	ms	`db_operation`, `db_system_name`	Duration of Redis operations	`observability.redis.operation.duration`	direct	External Services	stable
metric	Redis	`db.client.operation.active`	`db_client_operation_active`	Metric	Gauge	Datapoint	operations	`db_operation`	Number of active Redis operations	`observability.redis.operation.active`	direct	External Services	stable

The db_system_name dimension distinguishes Redis telemetry from other database systems.

Redis Derived Fields¶

Derived Field	Source Metric	Derivation	Canonical ID	Used By
Redis Request Rate	`db_client_operation_duration`	Rate filtered to Redis	`observability.redis.request_rate`	External Services
Redis P95 Latency	`db_client_operation_duration`	Histogram quantile filtered to Redis	`observability.redis.p95_latency`	External Services

Process Runtime Metrics¶

These fields describe the health and resource consumption of the application process.

Signal	Instrumentation	OTel Field	Prometheus Name	Field Type	Value Type	Scope	Unit	Dimensions	Description	Canonical ID	Mapping Type	Used By	Status
metric	System Metrics	`process.cpu.utilization`	`process_cpu_utilization_ratio`	Metric	Gauge	Datapoint	ratio	`exported_job`	CPU utilization of the process	`observability.process.cpu.utilization`	direct	Infrastructure	stable
metric	System Metrics	`process.cpu.time`	`process_cpu_time_seconds_total`	Metric	Counter	Datapoint	seconds	`exported_job`	Cumulative CPU time consumed by the process	`observability.process.cpu.time`	direct	Infrastructure	stable
metric	System Metrics	`process.memory.usage`	`process_memory_usage_bytes`	Metric	Gauge	Datapoint	bytes	`exported_job`	Memory used by the process	`observability.process.memory.usage`	direct	Infrastructure	stable
metric	System Metrics	`process.memory.virtual`	`process_memory_virtual_bytes`	Metric	Gauge	Datapoint	bytes	`exported_job`	Virtual memory allocated to the process	`observability.process.memory.virtual`	direct	Infrastructure	stable
metric	Process Runtime	`process.runtime.start_time`	`process_runtime_start_time`	Metric	Gauge	Datapoint	Unix timestamp	`exported_job`	Process start time	`observability.process.start_time`	direct	Service Overview, Infrastructure	stable
metric	Process Runtime	`process.thread.count`	`process_thread_count`	Metric	Gauge	Datapoint	threads	`exported_job`	Number of active process threads	`observability.process.thread_count`	direct	Infrastructure	stable
metric	Process Runtime	`process.open_file_descriptors`	`process_open_file_descriptor_count`	Metric	Gauge	Datapoint	descriptors	`exported_job`	Number of open file descriptors	`observability.process.open_file_descriptors`	direct	Infrastructure	stable

Python Runtime Metrics¶

These fields describe Python garbage collection activity.

Signal	Instrumentation	OTel Field	Prometheus Name	Field Type	Value Type	Scope	Unit	Dimensions	Description	Canonical ID	Mapping Type	Used By	Status
metric	System Metrics	`cpython.gc.collections`	`cpython_gc_collections_total`	Metric	Counter	Datapoint	collections	`exported_job`	Number of garbage collection cycles	`observability.python.gc.collections`	direct	Infrastructure	stable
metric	System Metrics	`cpython.gc.collected_objects`	`cpython_gc_collected_objects_total`	Metric	Counter	Datapoint	objects	`exported_job`	Number of objects reclaimed by garbage collection	`observability.python.gc.collected_objects`	direct	Infrastructure	stable
metric	System Metrics	`cpython.gc.uncollectable_objects`	`cpython_gc_uncollectable_objects_total`	Metric	Counter	Datapoint	objects	`exported_job`	Objects that could not be reclaimed by garbage collection	`observability.python.gc.uncollectable_objects`	direct	Infrastructure	stable

System Metrics¶

These fields describe host-level resource activity visible to the instrumented process.

Signal	Instrumentation	OTel Field	Prometheus Name	Field Type	Value Type	Scope	Unit	Dimensions	Description	Canonical ID	Mapping Type	Used By	Status
metric	System Metrics	`system.memory.utilization`	`system_memory_utilization_ratio`	Metric	Gauge	Datapoint	ratio	`exported_job`	System memory utilization	`observability.system.memory.utilization`	direct	Infrastructure	stable
metric	System Metrics	`system.network.io`	`system_network_io_bytes_total`	Metric	Counter	Datapoint	bytes	`direction`, `exported_job`	Network bytes transmitted or received	`observability.system.network.io`	direct	Infrastructure	stable
metric	System Metrics	`system.disk.io`	`system_disk_io_bytes_total`	Metric	Counter	Datapoint	bytes	`direction`, `exported_job`	Disk bytes read or written	`observability.system.disk.io`	direct	Infrastructure	stable

Logging Fields¶

Logs are exported using OTLP and stored in Loki.

The current observability package establishes the following fields as the minimum logging contract.

Signal	Instrumentation	OTel Field	Field Type	Value Type	Scope	Description	Canonical ID	Mapping Type	Used By	Status
log	Logging	`service.name`	Resource Attribute	string	Resource	Service that emitted the log	`observability.service.name`	direct	Logs	stable
log	Logging	`deployment.environment`	Resource Attribute	string	Resource	Environment in which the log was generated	`observability.environment`	direct	Logs	stable
log	Logging	`severity`	Log Field	string	Log Record	Log severity	`observability.log.severity`	direct	Logs	stable
log	Logging	`trace_id`	Log Field	string	Log Record	Trace identifier associated with the log	`observability.trace.id`	direct	Logs, Traces	stable
log	Logging	`span_id`	Log Field	string	Log Record	Span identifier associated with the log	`observability.span.id`	direct	Logs, Traces	stable

The current implementation also injects trace identifiers into standard Python logging records through LoggingInstrumentor.

Tracing Fields¶

Traces are exported through OTLP and visualized through Jaeger.

Signal	Instrumentation	OTel Field	Field Type	Value Type	Scope	Unit	Description	Canonical ID	Mapping Type	Used By	Status
trace	Core	`service.name`	Resource Attribute	string	Resource		Service producing the trace	`observability.service.name`	direct	Traces	stable
trace	Core	`span.name`	Span Attribute	string	Span		Name of the span operation	`observability.trace.span.name`	direct	Traces	stable
trace	Core	`trace_id`	Trace Field	string	Trace		Unique identifier for a distributed trace	`observability.trace.id`	direct	Logs, Traces	stable
trace	Core	`span_id`	Span Field	string	Span		Unique identifier for a span	`observability.span.id`	direct	Logs, Traces	stable
trace	Core	`duration`	Span Field	numeric	Span	time	Duration of the span	`observability.trace.span.duration`	direct	Traces	stable
trace	Core	`status`	Span Field	enum	Span		Span status	`observability.trace.span.status`	direct	Traces	stable

Instrumentation Coverage¶

The field matrix currently covers the following instrumentation sources.

Instrumentation	Signal	Domain Area	Status
FastAPI	Metrics, Traces	HTTP Server	stable
HTTPX	Metrics, Traces	HTTP Client	stable
SQLAlchemy	Metrics, Traces	Database	stable
psycopg2	Traces	Database	stable
Redis	Metrics, Traces	Redis	stable
System Metrics	Metrics	Process, Python, System	stable
Process Runtime	Metrics	Process	stable
Python Logging	Logs	Logging	stable
OpenTelemetry SDK	Metrics, Logs, Traces	Resource Identity	stable

Canonical Mapping Status¶

Canonical IDs in this matrix represent the current proposed namespace for the Observability domain.

They are intentionally scoped under:

observability.*

The IDs are not yet the final organizational ontology.

They provide stable identifiers for the field mapping layer while the minimum Observability ontology is being established.

The following mapping types are currently recognized:

Mapping Type	Meaning
`direct`	The telemetry field maps directly to a canonical field
`derived`	The value is calculated from one or more telemetry fields
`composite`	The canonical concept requires multiple telemetry fields
`provisional`	Mapping exists but requires further ontology review

Field Matrix Principles¶

The following rules apply to future additions.

Every dashboard-facing telemetry field should appear in this matrix.
Every field should have an identifiable OpenTelemetry source.
Prometheus names should only be recorded where the signal is exported through Prometheus-compatible metrics.
Resource attributes should not be treated as metric labels unless they are actually promoted into labels by the telemetry pipeline.
Dashboard-derived values should be distinguished from raw telemetry.
Canonical IDs should remain stable even when the underlying Prometheus representation changes.
Instrumentation-specific fields should map into shared canonical concepts where their semantics are equivalent.
Redis telemetry should remain distinguishable from general database telemetry through its database system dimension.
Logs and traces should remain separate signals even where they share correlation identifiers.
The field matrix should describe the telemetry contract, not the implementation details of individual Grafana panels.

Relationship to Other Observability Documents¶

The Observability domain uses the following documentation flow:

Dashboard Specifications
        ↓
PromQL / LogQL References
        ↓
Metrics Catalogue
        ↓
OTel Metric / Attribute Inventory
        ↓
Field Matrix
        ↓
Canonical Mappings
        ↓
Minimum Observability Ontology

The field matrix therefore acts as the boundary between the implementation telemetry model and the canonical organizational data model.