diff --git a/docs/event-telemetry.md b/docs/event-telemetry.md new file mode 100644 index 0000000000..6d709bc85d --- /dev/null +++ b/docs/event-telemetry.md @@ -0,0 +1,119 @@ +# Event measurements and relationships + +Any event type can carry numeric observations, categorical labels, an outcome, a detailed result, and relationships to other events. These are native event fields, independent of `data` and custom-field configuration. A producer can use `type:operation` for tests, benchmarks, jobs, imports, or other repeatable work; no particular type is required. + +```json +{ + "type": "operation", + "source": "Import.Accounts", + "date": "2026-10-01T12:00:00Z", + "reference_id": "invocation-1234", + "parent_reference_id": "suite-run-1234", + "root_reference_id": "build-run-1234", + "outcome": "success", + "result": "completed", + "labels": { + "branch": "main", + "runtime": "net10", + "scenario": "10000-records" + }, + "measurements": [ + { "name": "duration", "value": 12.4, "unit": "s" }, + { "name": "allocated", "value": 48192, "unit": "By" }, + { "name": "sql.calls", "value": 4, "unit": "{call}" }, + { "name": "redis.calls", "value": 0, "unit": "{call}" } + ] +} +``` + +## Field contract + +- **Measurements:** at most 32 uniquely named observations per event. Names start with a letter and contain letters, digits, dots, underscores, or hyphens (100 characters maximum). Values are finite doubles, including zero and fractional values. A missing value is invalid; a missing measurement is absent, not zero. Integers above 2^53 cannot be represented exactly. Units are required, case-sensitive strings of at most 32 characters. Follow UCUM/OTel conventions: prefer `s` for duration, `By` for bytes, `1` for dimensionless ratios, and annotations such as `{call}` or `{request}` for counts. Keep units separate from measurement names. Existing units such as `ms` remain valid; ingestion does not validate the full UCUM grammar. Units do not choose the numeric storage type. No automatic unit conversion occurs: select the exact unit when querying or charting. +- **Labels:** at most 32 name/string pairs, using the same name rules and values up to 256 characters. `"4.90"` and `"00123"` remain exact strings. Labels are categorical; numeric comparisons belong in measurements. There is no inference of numbers, dates, or booleans. +- **Outcome:** optional, with the ECS values `success`, `failure`, or `unknown`. Use `unknown` when an operation's outcome is unknown; omit the field when an outcome is not applicable. Values are lowercase and case-sensitive. +- **Result:** an optional detailed result of 1–100 characters, such as `completed`, `skipped`, `cancelled`, `aborted`, or `timed_out`. It is a keyword string, not a prescribed taxonomy. A result does not infer an outcome: a producer can report `outcome:failure` with `result:timed_out`, or just `result:skipped` when no success/failure outcome applies. +- **Relationships:** `parent_reference_id` identifies the immediate parent, and `root_reference_id` identifies the enclosing execution. References use the existing 8–100 character alphanumeric/hyphen contract and are scoped to the same project. The producer supplies these values. Parent events can arrive later or be outside retention; ingestion does not require them to exist. A direct self-parent is invalid. Navigation follows one link at a time and does not recursively expand a graph, so indirect cycles cannot trigger unbounded traversal. Existing `data["@ref:name"]` relationships keep their current meaning. + +Keep `source` and the stacking signature stable across executions to obtain the history of a repeatable operation. Give each execution a distinct reference; do not put a run ID in its source or stack signature. Reference IDs retain their existing deduplication semantics; they are not permanent uniqueness guarantees or an exactly-once ingestion API. + +Native fields do not change `value`, infer a new event type, or create stack signatures. Older unstructured root properties whose shapes differ from the new fields are retained in `data` by the inbound parser. Noncanonical root outcomes (for example `"outcome":"skipped"`) also remain legacy data; they are not native outcomes. Producers must use the canonical values to opt into outcome semantics. Existing stored `data` entries remain there. + +## Stacks and regression + +A stack groups repeated occurrences of the same operation across executions. Parent/root references link event instances within one execution. An operation can therefore retain one stack across many builds while each occurrence belongs to a different run hierarchy. Outcome, result, labels, and measurements do not automatically become part of the stack signature, so success and failure observations can share their history. + +For a fixed stack, `outcome:failure` is eligible for regression under the existing fixed-date or fixed-version rules. Success, unknown, and result-only events remain visible without regressing the stack or being discarded as old-version failures. Events with neither native outcome nor result keep legacy regression behavior. A successful observation does not automatically resolve an open or regressed stack. This is stack lifecycle behavior; detecting performance regressions or flaky tests still requires follow-up analysis. + +## ECS and OpenTelemetry alignment + +The native contract adopts useful shared semantics without claiming wire compatibility with ECS or OTLP: + +| Native field | Alignment and conversion boundary | +| --- | --- | +| `outcome` | Uses [ECS `event.outcome`](https://www.elastic.co/docs/reference/ecs/ecs-allowed-values-event-outcome). OTel span status needs an adapter: `Error` can indicate failure, but `Unset` must not blindly become unknown because successful spans commonly leave status unset. | +| `labels` | Follows the flat categorical semantics of [ECS labels](https://www.elastic.co/docs/reference/ecs/ecs-base). Exceptionless accepts strings only and indexes the collection as Elasticsearch `flattened`; numeric-looking values remain keyword values. These are not a replacement for OTel's typed attributes. | +| `measurements` | Uses [OTel instrument-unit conventions](https://opentelemetry.io/docs/specs/semconv/general/metrics/#instrument-units). Values are individual numeric observations. Full OTLP metric types, aggregation temporality, int64 precision, and histogram payloads are not represented by this contract. | +| A `duration` measurement | Prefer seconds. [ECS `event.duration`](https://www.elastic.co/docs/reference/ecs/ecs-event) is integer nanoseconds, so an adapter must explicitly convert values and units. The existing event `value` field retains its current semantics. | +| `result` | Exceptionless's generic detailed result; it has no direct ECS `event.result` counterpart. A future test/CI adapter can translate the relevant OTel semantic conventions. | +| `reference_id`, `parent_reference_id`, `root_reference_id` | Identify individual Exceptionless events. They are not `trace.id`, `span.id`, or OTel parent span identifiers. A trace can contain many spans, and multiple events can refer to the same span. A root event reference is not a trace ID. | + +First-class trace context, typed attributes, resource identity, and ECS/OTLP ingestion adapters are follow-up work. They should have native contracts rather than rely on `data` or configured custom fields. Keep the distinction between a resource (the entity producing telemetry) and attributes of an individual observation. Existing Exceptionless event types also retain their meaning; ECS event categorization is a different taxonomy. + +## Queries and charts + +Use existing event search and sort APIs: + +```text +type:operation outcome:failure +result:timed_out +labels.branch:main +measurement.duration@ms:>500 +project:PROJECT_ID parent_reference_id:suite-run-1234 +project:PROJECT_ID (reference:build-run-1234 OR root_reference_id:build-run-1234) +``` + +`sort=-measurement.duration@ms` orders individual events by a specific measurement and unit. The `measurement.NAME@UNIT` search alias binds the name, unit, and numeric value to the same nested observation. Normal numeric aggregation expressions can use this alias too. Use simple unit symbols in search aliases; the structured chart API accepts the full unit string. + +The events dashboard's **Configure chart** control selects a measurement/unit, aggregation (average, minimum, maximum, sum, count, p50, p95, p99), line/bar display, and split by source, stack, outcome, result, or a named label. With no measurement, bucketed counts can be split by outcome or result. Counts represent stored event observations, not measurement sums or the legacy duplicate `count` field. + +Charts inherit the view's filter and time range. They can show UTC time buckets or individual event observations. Bucketed measurements omit missing values and display gaps. Individual observations retain event IDs for drilldown. Bucket drilldown selects the series, measurement, unit, and interval. The expandable observations list provides keyboard access and separates events with identical timestamps. + +Bucket charts return at most 20 series and approximately 200 time buckets per series; terms ranking is by observation frequency, not metric value. Terms aggregation counts can be approximate across shards. Missing split fields are excluded from bucketed series. Individual-event mode returns the latest 1,000 observations and at most 20 series. The response and UI flag incomplete results; narrow the query for a complete view. Percentiles describe submitted observations and are approximate, not pooled percentiles of raw samples hidden inside pre-aggregated benchmark means. + +Chart configuration is a typed `chart` property on saved views. It is included in create, update, copy, predefined synchronization, and import/export. Omitting it preserves existing behavior; explicitly patching it to `null` restores the default event/stack count chart. Views without a chart retain their legacy predefined content hashes. Unsaved chart changes are carried in the `chart` URL parameter, including an explicit `null` override; URLs support refresh, sharing, and Back/Forward navigation. Other saved-view display drafts keep their existing storage behavior. + +The authenticated API provides: + +- `GET /api/v2/organizations/{organizationId}/events/measurements` with optional `filter`, `time`, and `offset`: bounded measurement-name/unit suggestions. +- `POST /api/v2/organizations/{organizationId}/events/chart` with `{ "chart": { "measurement": "duration", "unit": "ms", "aggregation": "avg", "group_by": "source", "mode": "buckets", "display": "line" }, "filter": "type:operation", "time": "[now-7d TO now]" }`: observations or aggregated series. + +These APIs enforce organization access, retention, suspension, and premium-search policies. They use repository queries and the existing event store. + +## MCP investigation tools + +MCP exposes the same native fields and chart query engine. `search_events`, `get_stack_events`, and `get_event` return measurements, labels, outcome, result, and parent/root references directly on each event result, including summary rows. Large legacy detail sections can still be omitted by the existing detail-size limit without hiding these native fields. + +Agents can use these generic tools for tests, benchmarks, jobs, or other event types: + +| Investigation | MCP call or query | +| --- | --- | +| Discover searchable fields and numeric sorts | `get_filter_fields`; telemetry prefixes are included in filter/sort metadata. | +| Discover measurement names and exact units | `get_event_measurements(projectId, filter, last)`; discovery is scoped to the project, filter, and time range. | +| Inspect a suite or run | `search_events(projectId, filter: "reference:suite-run-1234")`; use `parent_reference_id:suite-run-1234` for immediate children, or `(reference:build-run-1234 OR root_reference_id:build-run-1234)` for a whole run. Always keep references in the same project. | +| Find failing or slow individual observations | `search_events(projectId, filter: "outcome:failure AND labels.branch:main", sort: "-measurement.duration@s")`. Use a measurement threshold such as `measurement.duration@s:>1` when appropriate. | +| Follow one operation across executions | Use its `stackId` with `get_stack_events`, optionally filtering by outcome, result, labels, measurement, or time. `get_event` retrieves failure details for a returned event ID. | +| Count outcomes or detailed results | `count_events(projectId, filter, groupBy: "outcome")`, or group by `result`, `labels.runtime`, `parent_reference_id`, or `root_reference_id`. Optional `interval` adds time buckets. | +| Inspect duration, memory, or call-count trends | `get_event_chart(projectId, measurement: "duration", unit: "s", aggregation: "p95", groupBy: "source", filter, last: "7d")`. Use `allocated`/`By` or `sql.calls`/`{call}` with the same tool. | +| Retrieve individual measurement points | Use `get_event_chart` with `mode: "events"`; points contain event IDs for `get_event`. | + +The new discovery/history tools require `events:read` and the same plan access as the chart API. They default to 24 hours when no time is supplied, accept `last` or absolute `startUtc`/`endUtc`, and apply project access, organization access, suspension, and retention. They expose structured errors for invalid measurement/unit/grouping combinations and time ranges. Native fields do not require custom-field configuration. + +An agent must honor pagination and truncation. Search returns cursor-paged summaries; follow `pagination.after` with the same filter, sort, time, and limit for a complete history. Chart series are selected by observation frequency, not metric value, and discovery is also bounded. A `truncated` response includes a warning to narrow the query or page through events. Count groups are bounded and may omit missing or less frequent values; their totals are not necessarily the whole population. A sorted event list ranks individual observations, not per-operation averages. Compare a selected operation's average/percentiles through `get_event_chart`, or retrieve its complete observations for further analysis. + +For flakiness investigations, compare both success and failure observations with the same operation identity, run/attempt identity, parameters, and environment. Exclude unknown/result-only observations from a success/failure denominator, distinguish retries from independent runs, and avoid adding suite totals to child observations. A failure count by itself is not a flakiness classification. No reporter-specific event types or dedicated test/benchmark MCP tools are required. + +## Storage and rollout + +Elasticsearch uses a fixed `nested` mapping for `measurements` (`name` and `unit` keywords, `value` double), a `flattened` mapping for labels, and keyword mappings for outcome, result, and relationships. New names do not create new mappings. Each measurement does create an additional hidden nested document, so the per-event limit bounds storage and indexing overhead. Daily index creation includes these mappings; migration 10 adds them to retained partitions. It does not backfill historical custom data or move it into native fields. Deploy the mapping migration before enabling producers. + +CI/benchmark reporters, SDK convenience methods, multiple dashboard widgets, baseline comparisons, automatic flaky-test classification, and performance regression alerts are follow-up work. For tests, report passing and failing attempts, retries, and the logical execution identity; failure-only events cannot establish a flakiness rate. For benchmarks, use stable scenario/runtime/machine labels and compare equivalent environments. Keep suite totals separate from child observations in queries to avoid double-counting. Production enablement should measure actual ingest volume, nested-document/storage cost, event quotas, and retention before broad reporting. diff --git a/src/Exceptionless.Core/Bootstrapper.cs b/src/Exceptionless.Core/Bootstrapper.cs index 92c2ad4c43..0e43005e3c 100644 --- a/src/Exceptionless.Core/Bootstrapper.cs +++ b/src/Exceptionless.Core/Bootstrapper.cs @@ -186,6 +186,7 @@ public static void RegisterServices(IServiceCollection services, AppOptions appO services.AddSingleton(); services.AddSingleton(); services.AddSingleton(); + services.AddSingleton(); services.AddSingleton(); services.AddSingleton(); services.AddSingleton(); diff --git a/src/Exceptionless.Core/Migrations/010_AddEventTelemetryMappings.cs b/src/Exceptionless.Core/Migrations/010_AddEventTelemetryMappings.cs new file mode 100644 index 0000000000..60c1d3df3a --- /dev/null +++ b/src/Exceptionless.Core/Migrations/010_AddEventTelemetryMappings.cs @@ -0,0 +1,25 @@ +using Exceptionless.Core.Models; +using Exceptionless.Core.Repositories.Configuration; +using Foundatio.Repositories.Elasticsearch.Extensions; +using Foundatio.Repositories.Migrations; +using Microsoft.Extensions.Logging; + +namespace Exceptionless.Core.Migrations; + +/// Adds native telemetry mappings to retained event partitions without reindexing. +public sealed class AddEventTelemetryMappings(ExceptionlessElasticConfiguration configuration, ILoggerFactory loggerFactory) : MigrationBase(loggerFactory) +{ + public override int? Version => 10; + + public override async Task RunAsync(MigrationContext context) + { + var response = await configuration.Client.Indices.PutMappingAsync(d => d + .Indices($"{configuration.Events.Name}-v*-*") + .AllowNoIndices(true) + .IgnoreUnavailable(true) + .Properties(p => p.AddTelemetry()), context.CancellationToken); + _logger.LogRequest(response); + if (!response.IsValidResponse) + throw new InvalidOperationException("Unable to add event telemetry mappings: " + response.DebugInformation); + } +} diff --git a/src/Exceptionless.Core/Models/Event.cs b/src/Exceptionless.Core/Models/Event.cs index 84a96c7acd..f865789c42 100644 --- a/src/Exceptionless.Core/Models/Event.cs +++ b/src/Exceptionless.Core/Models/Event.cs @@ -56,6 +56,28 @@ public class Event : IData, IJsonOnDeserialized /// public int? Count { get; set; } + /// The ECS-aligned outcome: success, failure, or unknown. Omit when not applicable. + [StringLength(100, MinimumLength = 1), RegularExpression("^(success|failure|unknown)$")] + public string? Outcome { get; set; } + + /// An optional detailed result, such as skipped, cancelled, or timed_out. Does not imply an outcome. + [StringLength(100, MinimumLength = 1)] + public string? Result { get; set; } + + /// Reference of the immediate parent event in the same project. + public string? ParentReferenceId { get; set; } + + /// Reference of the root event in the same project. Supplied by the producer. + public string? RootReferenceId { get; set; } + + /// Up to 32 uniquely named numeric observations. Missing values are not zero. + [MaxLength(32)] + public List? Measurements { get; set; } + + /// Up to 32 categorical labels, indexed as exact strings without type inference. + [MaxLength(32), SkipRecursion] + public Dictionary? Labels { get; set; } + /// /// Optional data entries that contain additional information about this event. /// @@ -102,6 +124,9 @@ void IJsonOnDeserialized.OnDeserialized() Data ??= []; foreach (var kvp in ExtensionData) { + if (EventTelemetryReader.TryRead(this, kvp.Key, kvp.Value)) + continue; + object? value = JsonElementConverter.Convert(kvp.Value); EventDataNormalizer.Set(Data, kvp.Key, value); } @@ -112,7 +137,10 @@ void IJsonOnDeserialized.OnDeserialized() protected bool Equals(Event other) { - return String.Equals(Type, other.Type) && String.Equals(Source, other.Source) && Tags.CollectionEquals(other.Tags) && String.Equals(Message, other.Message) && String.Equals(Geo, other.Geo) && Value == other.Value && Equals(Data, other.Data); + return String.Equals(Type, other.Type) && String.Equals(Source, other.Source) && Tags.CollectionEquals(other.Tags) && String.Equals(Message, other.Message) && String.Equals(Geo, other.Geo) && Value == other.Value && Equals(Data, other.Data) + && Outcome == other.Outcome && Result == other.Result && ParentReferenceId == other.ParentReferenceId && RootReferenceId == other.RootReferenceId + && (Measurements ?? []).OrderBy(m => m?.Name, StringComparer.Ordinal).SequenceEqual((other.Measurements ?? []).OrderBy(m => m?.Name, StringComparer.Ordinal)) + && (Labels ?? []).OrderBy(d => d.Key, StringComparer.Ordinal).SequenceEqual((other.Labels ?? []).OrderBy(d => d.Key, StringComparer.Ordinal)); } public override bool Equals(object? obj) @@ -138,10 +166,22 @@ public override int GetHashCode() hashCode = (hashCode * 397) ^ (Geo?.GetHashCode() ?? 0); hashCode = (hashCode * 397) ^ Value.GetHashCode(); hashCode = (hashCode * 397) ^ (Data?.GetCollectionHashCode(_exclusions) ?? 0); + hashCode = (hashCode * 397) ^ HashCode.Combine(Outcome, Result, ParentReferenceId, RootReferenceId); + foreach (var measurement in (Measurements ?? []).OrderBy(m => m?.Name, StringComparer.Ordinal)) + hashCode = (hashCode * 397) ^ (measurement?.GetHashCode() ?? 0); + foreach (var label in (Labels ?? []).OrderBy(d => d.Key, StringComparer.Ordinal)) + hashCode = (hashCode * 397) ^ label.GetHashCode(); return hashCode; } } + public static class KnownOutcomes + { + public const string Success = "success"; + public const string Failure = "failure"; + public const string Unknown = "unknown"; + } + public static class KnownTypes { public const string Error = "error"; diff --git a/src/Exceptionless.Core/Models/EventChart.cs b/src/Exceptionless.Core/Models/EventChart.cs new file mode 100644 index 0000000000..82fd755bd0 --- /dev/null +++ b/src/Exceptionless.Core/Models/EventChart.cs @@ -0,0 +1,33 @@ +using System.ComponentModel.DataAnnotations; + +namespace Exceptionless.Core.Models; + +/// Configuration of a saved view's single event chart. Null retains the default event-count chart. +public sealed record EventChart : IValidatableObject +{ + [RegularExpression(EventMeasurement.NamePattern)] + public string? Measurement { get; set; } + + [StringLength(32, MinimumLength = 1)] + public string? Unit { get; set; } + + [Required, RegularExpression("^(avg|min|max|sum|p50|p95|p99|count)$")] + public string Aggregation { get; set; } = "avg"; + + [RegularExpression("^(source|stack|outcome|result|labels\\.[a-zA-Z][a-zA-Z0-9_.-]{0,99})$")] + public string? GroupBy { get; set; } + + [Required, RegularExpression("^(buckets|events)$")] + public string Mode { get; set; } = "buckets"; + + [Required, RegularExpression("^(line|bar)$")] + public string Display { get; set; } = "line"; + + public IEnumerable Validate(ValidationContext validationContext) + { + if (Measurement is null && (Aggregation != "count" || Mode == "events")) + yield return new ValidationResult("Select a measurement, or use bucketed event counts.", [nameof(Measurement)]); + if ((Measurement is null) != (Unit is null)) + yield return new ValidationResult("A measurement and its unit must be selected together.", [nameof(Unit)]); + } +} diff --git a/src/Exceptionless.Core/Models/EventChartResult.cs b/src/Exceptionless.Core/Models/EventChartResult.cs new file mode 100644 index 0000000000..1a4e06fe56 --- /dev/null +++ b/src/Exceptionless.Core/Models/EventChartResult.cs @@ -0,0 +1,14 @@ +namespace Exceptionless.Core.Models; + +public sealed record EventChartResult +{ + public List Series { get; set; } = []; + public long Total { get; set; } + public bool Truncated { get; set; } + public long? IntervalMilliseconds { get; set; } +} + +public sealed record EventChartSeries(string Name, List Points); +public sealed record EventChartPoint(DateTime Date, double? Value, long Count, string? EventId = null); +public sealed record EventMeasurementDescriptor(string Name, string Unit); +public sealed record EventMeasurementCatalog(List Measurements, bool Truncated); diff --git a/src/Exceptionless.Core/Models/EventMeasurement.cs b/src/Exceptionless.Core/Models/EventMeasurement.cs new file mode 100644 index 0000000000..087993cb75 --- /dev/null +++ b/src/Exceptionless.Core/Models/EventMeasurement.cs @@ -0,0 +1,25 @@ +using System.ComponentModel.DataAnnotations; + +namespace Exceptionless.Core.Models; + +/// A numeric observation on one event. Units describe values, not their storage type. +public sealed record EventMeasurement : IValidatableObject +{ + public const string NamePattern = "^[a-zA-Z][a-zA-Z0-9_.-]{0,99}$"; + + [Required, RegularExpression(NamePattern)] + public string Name { get; set; } = null!; + + [Required] + public double? Value { get; set; } + + /// Case-sensitive unit. Prefer UCUM/OTel symbols: s for duration, By for bytes, {call} for counts, or 1 for dimensionless values. + [Required, StringLength(32, MinimumLength = 1)] + public string Unit { get; set; } = null!; + + public IEnumerable Validate(ValidationContext validationContext) + { + if (Value is { } value && !Double.IsFinite(value)) + yield return new ValidationResult("Measurement values must be finite.", [nameof(Value)]); + } +} diff --git a/src/Exceptionless.Core/Models/PersistentEvent.cs b/src/Exceptionless.Core/Models/PersistentEvent.cs index 968a4acd37..813da626d8 100644 --- a/src/Exceptionless.Core/Models/PersistentEvent.cs +++ b/src/Exceptionless.Core/Models/PersistentEvent.cs @@ -1,5 +1,6 @@ using System.ComponentModel.DataAnnotations; using System.Diagnostics; +using Exceptionless.Core.Validation; using Exceptionless.Core.Attributes; using Exceptionless.Core.Extensions; using Foundatio.Repositories.Models; @@ -71,6 +72,9 @@ public IEnumerable Validate(ValidationContext validationContex yield return new ValidationResult("ReferenceId must contain between 8 and 100 alphanumeric or '-' characters.", [nameof(ReferenceId)]); } + foreach (var error in EventTelemetryValidation.ValidateRelationshipsAndLabels(this)) + yield return error; + // NOTE: We need to write a migration to cleanup all old events of 50 or more tags so there never is an error while saving. //if (ev.Tags.Count > 50) // yield return new ValidationResult("Tags can't include more than 50 tags.", nameof(Tags)); diff --git a/src/Exceptionless.Core/Models/SavedView.cs b/src/Exceptionless.Core/Models/SavedView.cs index 14527619c6..a657cc026b 100644 --- a/src/Exceptionless.Core/Models/SavedView.cs +++ b/src/Exceptionless.Core/Models/SavedView.cs @@ -54,6 +54,8 @@ public partial record SavedView : IOwnedByOrganizationWithIdentity, IHaveDates /// Whether the dashboard chart is shown for this view. Null means use the default. public bool? ShowChart { get; set; } + public EventChart? Chart { get; set; } + /// Stable identifier used to synchronize predefined saved views across organizations. [MaxLength(150)] public string? PredefinedKey { get; set; } diff --git a/src/Exceptionless.Core/Pipeline/007_ValidateEventTelemetryAction.cs b/src/Exceptionless.Core/Pipeline/007_ValidateEventTelemetryAction.cs new file mode 100644 index 0000000000..d66d1c6e4d --- /dev/null +++ b/src/Exceptionless.Core/Pipeline/007_ValidateEventTelemetryAction.cs @@ -0,0 +1,24 @@ +using Exceptionless.Core.Plugins.EventProcessor; +using Exceptionless.Core.Validation; +using Microsoft.Extensions.Logging; + +namespace Exceptionless.Core.Pipeline; + +/// Rejects invalid observations individually before stack assignment and batch persistence. +[Priority(7)] +public sealed class ValidateEventTelemetryAction(AppOptions options, ILoggerFactory loggerFactory) : EventPipelineActionBase(options, loggerFactory) +{ + public override Task ProcessAsync(EventContext ctx) + { + if (ctx.Event is { Outcome: null, Result: null, ParentReferenceId: null, RootReferenceId: null, Measurements: null, Labels: null }) + return Task.CompletedTask; + + var errors = EventTelemetryValidation.GetErrors(ctx.Event); + if (errors.Count > 0) + { + var error = new MiniValidatorException(errors); + ctx.SetError(error.Message, error); + } + return Task.CompletedTask; + } +} diff --git a/src/Exceptionless.Core/Pipeline/030_CheckForRegressionAction.cs b/src/Exceptionless.Core/Pipeline/030_CheckForRegressionAction.cs index f7a8c79a62..f751b48cf3 100644 --- a/src/Exceptionless.Core/Pipeline/030_CheckForRegressionAction.cs +++ b/src/Exceptionless.Core/Pipeline/030_CheckForRegressionAction.cs @@ -24,6 +24,8 @@ public override async Task ProcessBatchAsync(ICollection contexts) { var stacks = contexts .Where(c => c.Stack is { Status: StackStatus.Fixed, DateFixed: not null }) + // Explicit failures can regress. Events without outcome/result retain legacy behavior. + .Where(c => c.Event.Outcome == Event.KnownOutcomes.Failure || c.Event is { Outcome: null, Result: null }) .OrderBy(c => c.Event.Date) .GroupBy(c => c.Event.StackId); @@ -53,7 +55,7 @@ public override async Task ProcessBatchAsync(ICollection contexts) var version = _semanticVersionParser.Parse(versionGroup.Key, versionCache) ?? _semanticVersionParser.Default; if (version < fixedInVersion) { - foreach (var ctx in stackGroup.Where(s => s.Organization.HasPremiumFeatures)) + foreach (var ctx in versionGroup.Where(s => s.Organization.HasPremiumFeatures)) { _logger.LogDebug("Discarding fixed stack event: Version {Version} is older than fixed in version {FixedInVersion}", version, fixedInVersion); ctx.IsDiscarded = true; diff --git a/src/Exceptionless.Core/Plugins/EventParser/Default/JsonEventParserPlugin.cs b/src/Exceptionless.Core/Plugins/EventParser/Default/JsonEventParserPlugin.cs index 327c507c6b..67e30ce198 100644 --- a/src/Exceptionless.Core/Plugins/EventParser/Default/JsonEventParserPlugin.cs +++ b/src/Exceptionless.Core/Plugins/EventParser/Default/JsonEventParserPlugin.cs @@ -1,5 +1,7 @@ using System.Text.Json; using Exceptionless.Core.Extensions; +using System.Text.Json.Serialization.Metadata; +using Exceptionless.Core.Serialization; using Exceptionless.Core.Models; using Exceptionless.Core.Pipeline; using Microsoft.Extensions.Logging; @@ -17,6 +19,8 @@ public JsonEventParserPlugin(AppOptions options, JsonSerializerOptions jsonOptio // non-nullable properties. We must not reject structurally valid events; the pipeline // handles missing/null values gracefully downstream. _jsonOptions = new JsonSerializerOptions(jsonOptions) { RespectNullableAnnotations = false }; + _jsonOptions.TypeInfoResolver = (_jsonOptions.TypeInfoResolver ?? new DefaultJsonTypeInfoResolver()) + .WithAddedModifier(EventTelemetryReader.CaptureTelemetryProperties); } public List? ParseEvents(string input, int apiVersion, string? userAgent) diff --git a/src/Exceptionless.Core/Repositories/Configuration/Indexes/EventIndex.cs b/src/Exceptionless.Core/Repositories/Configuration/Indexes/EventIndex.cs index 9c7b77b4cd..0e6be0aaae 100644 --- a/src/Exceptionless.Core/Repositories/Configuration/Indexes/EventIndex.cs +++ b/src/Exceptionless.Core/Repositories/Configuration/Indexes/EventIndex.cs @@ -46,6 +46,7 @@ protected override void ConfigureQueryBuilder(ElasticQueryBuilder builder) var stacksRepository = _serviceProvider.GetRequiredService(); var cacheClient = _serviceProvider.GetRequiredService(); base.ConfigureQueryBuilder(builder); + builder.Register(new EventChartQueryBuilder()); builder.RegisterBefore(new EventStackFilterQueryBuilder(stacksRepository, cacheClient, _configuration.LoggerFactory)); } @@ -61,6 +62,7 @@ public override void ConfigureIndexMapping(TypeMappingDescriptor t.Match("*-s").Mapping(m => m.Keyword(s => s.IgnoreAbove(1024))))) .Properties(p => p .SetupDefaults() + .AddTelemetry() .Keyword(e => e.OrganizationId) .FieldAlias(Alias.OrganizationId, a => a.Path(f => f.OrganizationId)) .Keyword(e => e.ProjectId) @@ -154,6 +156,7 @@ protected override void ConfigureQueryParser(ElasticQueryParserConfiguration con EventIndexExtensions.DataPath(Event.KnownDataKeys.UserInfo, u => u.Name) ]) .AddQueryVisitor(new EventFieldsQueryVisitor()) + .UseNested() .UseFieldMap(new Dictionary { { Alias.BrowserVersion, EventIndexExtensions.DataDictionaryPath(Event.KnownDataKeys.RequestInfo, r => r.Data, RequestInfo.KnownDataKeys.BrowserVersion) }, { Alias.BrowserMajorVersion, EventIndexExtensions.DataDictionaryPath(Event.KnownDataKeys.RequestInfo, r => r.Data, RequestInfo.KnownDataKeys.BrowserMajorVersion) }, @@ -164,6 +167,12 @@ protected override void ConfigureQueryParser(ElasticQueryParserConfiguration con { Alias.OperatingSystemVersion, EventIndexExtensions.DataDictionaryPath(Event.KnownDataKeys.RequestInfo, r => r.Data, RequestInfo.KnownDataKeys.OSVersion) }, { Alias.OperatingSystemMajorVersion, EventIndexExtensions.DataDictionaryPath(Event.KnownDataKeys.RequestInfo, r => r.Data, RequestInfo.KnownDataKeys.OSMajorVersion) } }); + + var fieldResolver = config.FieldResolver; + config.UseFieldResolver((field, context) => MeasurementField.TryParse(field, out _, out _) + ? Task.FromResult("measurements.value") + : fieldResolver!(field, context)); + config.UseNestedFilter((path, original, resolved, context) => path == "measurements" ? MeasurementField.Filter(original) : null); } public ElasticsearchOptions Options => _configuration.Options; diff --git a/src/Exceptionless.Core/Repositories/Configuration/Indexes/EventTelemetryMapping.cs b/src/Exceptionless.Core/Repositories/Configuration/Indexes/EventTelemetryMapping.cs new file mode 100644 index 0000000000..5ce8954688 --- /dev/null +++ b/src/Exceptionless.Core/Repositories/Configuration/Indexes/EventTelemetryMapping.cs @@ -0,0 +1,18 @@ +using Elastic.Clients.Elasticsearch.Mapping; +using Exceptionless.Core.Models; + +namespace Exceptionless.Core.Repositories.Configuration; + +public static class EventTelemetryMapping +{ + public static PropertiesDescriptor AddTelemetry(this PropertiesDescriptor properties) => properties + .Keyword(e => e.Outcome) + .Keyword(e => e.Result) + .Keyword(e => e.ParentReferenceId) + .Keyword(e => e.RootReferenceId) + .Flattened(e => e.Labels, p => p.DepthLimit(1)) + .Nested(e => e.Measurements, n => n.Dynamic(DynamicMapping.False).Properties(p => p + .Keyword("name") + .Keyword("unit") + .DoubleNumber("value"))); +} diff --git a/src/Exceptionless.Core/Repositories/Queries/EventChartQuery.cs b/src/Exceptionless.Core/Repositories/Queries/EventChartQuery.cs new file mode 100644 index 0000000000..d4328e0f04 --- /dev/null +++ b/src/Exceptionless.Core/Repositories/Queries/EventChartQuery.cs @@ -0,0 +1,120 @@ +using Elastic.Clients.Elasticsearch.Aggregations; +using Elastic.Clients.Elasticsearch.QueryDsl; +using Exceptionless.Core.Models; +using Foundatio.Repositories; +using Foundatio.Repositories.Elasticsearch.Queries.Builders; +using Foundatio.Repositories.Options; + +namespace Exceptionless.Core.Repositories.Queries; + +public static class EventChartQueryExtensions +{ + internal const string ChartKey = "@EventChart"; + internal const string CatalogKey = "@EventMeasurementCatalog"; + public const int MaxSeries = 20; + public const int MaxPoints = 1000; + + public static T EventChart(this T query, EventChart chart, DateTime start, DateTime end) where T : IRepositoryQuery + => query.BuildOption(ChartKey, new ChartQuery(chart, GetInterval(start, end))); + + public static T MeasurementCatalog(this T query) where T : IRepositoryQuery => query.BuildOption(CatalogKey, true); + + // UTC fixed intervals keep a strict upper bound on buckets, even for very long retention periods. + public static long GetInterval(DateTime start, DateTime end) => Math.Max(1000, (long)Math.Ceiling((end - start).TotalMilliseconds / 200 / 1000) * 1000); + + internal sealed record ChartQuery(EventChart Chart, long IntervalMilliseconds); +} + +public sealed class EventChartQueryBuilder : IElasticQueryBuilder +{ + public Task BuildAsync(QueryBuilderContext ctx) where T : class, new() + { + if (ctx.Source.SafeGetOption(EventChartQueryExtensions.CatalogKey)) + { + ctx.Search.Aggregations(new Dictionary + { + ["measurements"] = new() + { + Nested = new NestedAggregation { Path = "measurements" }, + Aggregations = new Dictionary + { + ["names"] = new() + { + Terms = new TermsAggregation { Field = "measurements.name", Size = 100 }, + Aggregations = new Dictionary + { + ["units"] = new() { Terms = new TermsAggregation { Field = "measurements.unit", Size = 20 } } + } + } + } + } + }); + } + + var query = ctx.Source.SafeGetOption(EventChartQueryExtensions.ChartKey); + if (query is null) + return Task.CompletedTask; + + var chart = query.Chart; + Query? measurementFilter = null; + if (chart.Measurement is not null) + { + measurementFilter = new TermQuery { Field = "measurements.name", Value = chart.Measurement }; + measurementFilter &= new TermQuery { Field = "measurements.unit", Value = chart.Unit! }; + ctx.Filter &= new NestedQuery { Path = "measurements", Query = measurementFilter, IgnoreUnmapped = true }; + } + + if (chart.Mode == "events") + return Task.CompletedTask; + + var date = new Aggregation + { + DateHistogram = new DateHistogramAggregation + { + Field = "date", + FixedInterval = $"{query.IntervalMilliseconds}ms", + MinDocCount = 0 + } + }; + + if (measurementFilter is not null) + { + var metric = chart.Aggregation switch + { + "min" => new Aggregation { Min = new MinAggregation { Field = "measurements.value" } }, + "max" => new Aggregation { Max = new MaxAggregation { Field = "measurements.value" } }, + "sum" => new Aggregation { Sum = new SumAggregation { Field = "measurements.value" } }, + "count" => new Aggregation { ValueCount = new ValueCountAggregation { Field = "measurements.value" } }, + "p50" or "p95" or "p99" => new Aggregation { Percentiles = new PercentilesAggregation { Field = "measurements.value", Percents = [Double.Parse(chart.Aggregation[1..], System.Globalization.CultureInfo.InvariantCulture)] } }, + _ => new Aggregation { Avg = new AverageAggregation { Field = "measurements.value" } } + }; + date.Aggregations = new Dictionary + { + ["measurements"] = new() + { + Nested = new NestedAggregation { Path = "measurements" }, + Aggregations = new Dictionary + { + ["selected"] = new() + { + Filter = measurementFilter, + Aggregations = new Dictionary { ["value"] = metric } + } + } + } + }; + } + + var aggregations = new Dictionary { ["dates"] = date }; + if (chart.GroupBy is not null) + { + string field = chart.GroupBy switch { "source" => "source.keyword", "stack" => "stack_id", _ => chart.GroupBy }; + aggregations = new Dictionary + { + ["series"] = new() { Terms = new TermsAggregation { Field = field, Size = EventChartQueryExtensions.MaxSeries }, Aggregations = aggregations } + }; + } + ctx.Search.Aggregations(aggregations); + return Task.CompletedTask; + } +} diff --git a/src/Exceptionless.Core/Repositories/Queries/MeasurementField.cs b/src/Exceptionless.Core/Repositories/Queries/MeasurementField.cs new file mode 100644 index 0000000000..6612198cec --- /dev/null +++ b/src/Exceptionless.Core/Repositories/Queries/MeasurementField.cs @@ -0,0 +1,31 @@ +using System.Text.RegularExpressions; +using Elastic.Clients.Elasticsearch.QueryDsl; +using Exceptionless.Core.Models; + +namespace Exceptionless.Core.Repositories.Queries; + +/// Resolves measurement.duration@ms to a value and its nested name/unit scope. +public static class MeasurementField +{ + public static bool TryParse(string field, out string name, out string unit) + { + name = unit = String.Empty; + const string prefix = "measurement."; + if (!field.StartsWith(prefix, StringComparison.Ordinal)) + return false; + int separator = field.IndexOf('@'); + if (separator <= prefix.Length || separator == field.Length - 1) + return false; + name = field[prefix.Length..separator]; + unit = field[(separator + 1)..]; + return unit.Length <= 32 && Regex.IsMatch(name, EventMeasurement.NamePattern); + } + + public static Query? Filter(string originalField) + { + if (!TryParse(originalField, out string name, out string unit)) + return null; + Query filter = new TermQuery { Field = "measurements.name", Value = name }; + return filter & new TermQuery { Field = "measurements.unit", Value = unit }; + } +} diff --git a/src/Exceptionless.Core/Repositories/Queries/Validation/PersistentEventQueryValidator.cs b/src/Exceptionless.Core/Repositories/Queries/Validation/PersistentEventQueryValidator.cs index 5cd11109f5..3164e7c516 100644 --- a/src/Exceptionless.Core/Repositories/Queries/Validation/PersistentEventQueryValidator.cs +++ b/src/Exceptionless.Core/Repositories/Queries/Validation/PersistentEventQueryValidator.cs @@ -1,4 +1,5 @@ using Exceptionless.Core.Repositories.Configuration; +using Exceptionless.Core.Repositories.Queries; using Foundatio.Parsers.LuceneQueries; using Foundatio.Parsers.LuceneQueries.Visitors; using Microsoft.Extensions.Logging; @@ -36,6 +37,10 @@ public sealed class PersistentEventQueryValidator : AppQueryValidator }; private static readonly HashSet _allowedAggregationFields = new(StringComparer.OrdinalIgnoreCase) { + "outcome", + "result", + "parent_reference_id", + "root_reference_id", "date", "source", "tags", @@ -117,7 +122,9 @@ protected override QueryProcessResult ApplyAggregationRules(QueryValidationResul return new QueryProcessResult { Message = "Aggregation count exceeded" }; // Only allow fields that are numeric or have high commonality. - if (!result.ReferencedFields.All(_allowedAggregationFields.Contains)) + if (!result.ReferencedFields.All(field => _allowedAggregationFields.Contains(field) + || MeasurementField.TryParse(field, out _, out _) + || field.StartsWith("labels.", StringComparison.Ordinal))) return new QueryProcessResult { Message = "One or more aggregation fields are not allowed" }; // Distinct queries are expensive. diff --git a/src/Exceptionless.Core/Seed/PredefinedSavedViewConfiguration.cs b/src/Exceptionless.Core/Seed/PredefinedSavedViewConfiguration.cs index 95e4ce00da..8f3b512696 100644 --- a/src/Exceptionless.Core/Seed/PredefinedSavedViewConfiguration.cs +++ b/src/Exceptionless.Core/Seed/PredefinedSavedViewConfiguration.cs @@ -19,6 +19,7 @@ public static bool Apply(SavedView destination, SavedView source, string key, st changed |= SetColumnsIfChanged(destination, source.Columns); changed |= SetIfChanged(destination, source.ShowStats, static (view, value) => view.ShowStats = value, static view => view.ShowStats); changed |= SetIfChanged(destination, source.ShowChart, static (view, value) => view.ShowChart = value, static view => view.ShowChart); + changed |= SetIfChanged(destination, source.Chart is null ? null : source.Chart with { }, static (view, value) => view.Chart = value, static view => view.Chart); changed |= SetIfChanged(destination, source.Version, static (view, value) => view.Version = value, static view => view.Version); changed |= SetIfChanged(destination, PredefinedSavedViewContentHasher.GetContentHash(destination), static (view, value) => view.PredefinedContentHash = value, static view => view.PredefinedContentHash); diff --git a/src/Exceptionless.Core/Seed/PredefinedSavedViewContentHasher.cs b/src/Exceptionless.Core/Seed/PredefinedSavedViewContentHasher.cs index cd5b111718..da3605563a 100644 --- a/src/Exceptionless.Core/Seed/PredefinedSavedViewContentHasher.cs +++ b/src/Exceptionless.Core/Seed/PredefinedSavedViewContentHasher.cs @@ -21,7 +21,7 @@ public static string GetContentHash(SavedView savedView) savedView.FilterDefinitions, savedView.Columns, savedView.ShowStats, - savedView.ShowChart); + savedView.ShowChart, savedView.Chart); } public static string GetDefinitionsContentHash(IEnumerable definitions) @@ -43,7 +43,7 @@ public static string GetDefinitionsContentHash(IEnumerable? columns, bool? showStats, - bool? showChart) + bool? showChart, + EventChart? chart) { var content = new { @@ -75,7 +76,13 @@ private static string GetContentHash( showChart }; - return SerializeAndHash(content); + // Keep the exact legacy hash when no chart configuration exists. + if (chart is null) + return SerializeAndHash(content); + + var configured = JsonSerializer.SerializeToNode(content)!.AsObject(); + configured["chart"] = JsonSerializer.SerializeToNode(chart); + return configured.ToJsonString().ToSHA256(); } private static string SerializeAndHash(T content) diff --git a/src/Exceptionless.Core/Seed/PredefinedSavedViewsDataSeed.cs b/src/Exceptionless.Core/Seed/PredefinedSavedViewsDataSeed.cs index 6ff27831d1..556ee64958 100644 --- a/src/Exceptionless.Core/Seed/PredefinedSavedViewsDataSeed.cs +++ b/src/Exceptionless.Core/Seed/PredefinedSavedViewsDataSeed.cs @@ -143,6 +143,12 @@ private static bool ApplyDefinition(SavedView existing, PredefinedSavedViewDefin changed = true; } + if (existing.Chart != definition.Chart) + { + existing.Chart = definition.Chart is null ? null : definition.Chart with { }; + changed = true; + } + if (existing.ShowChart != definition.ShowChart) { existing.ShowChart = definition.ShowChart; @@ -204,6 +210,7 @@ internal static SavedView CreateSavedView( Columns = CopyColumns(definition.Columns), ShowStats = definition.ShowStats, ShowChart = definition.ShowChart, + Chart = definition.Chart is null ? null : definition.Chart with { }, Version = 1 }; @@ -271,4 +278,7 @@ public sealed record PredefinedSavedViewDefinition [JsonPropertyName("showChart")] public bool? ShowChart { get; init; } + + [JsonPropertyName("chart")] + public EventChart? Chart { get; init; } } diff --git a/src/Exceptionless.Core/Serialization/EventTelemetryReader.cs b/src/Exceptionless.Core/Serialization/EventTelemetryReader.cs new file mode 100644 index 0000000000..2a4cc0fe07 --- /dev/null +++ b/src/Exceptionless.Core/Serialization/EventTelemetryReader.cs @@ -0,0 +1,72 @@ +using System.Text.Json; +using System.Text.Json.Serialization.Metadata; +using System.Text.RegularExpressions; +using Exceptionless.Core.Models; + +namespace Exceptionless.Core.Serialization; + +/// Preserves older clients' unstructured root properties when they do not use the new native shapes. +internal static partial class EventTelemetryReader +{ + private static readonly JsonSerializerOptions _options = new() { PropertyNameCaseInsensitive = true }; + private static readonly HashSet _fields = new(StringComparer.OrdinalIgnoreCase) + { + "outcome", "result", "parent_reference_id", "root_reference_id", "measurements", "labels" + }; + + // Only the inbound parser uses this modifier. Native models retain their typed API and storage contracts. + public static void CaptureTelemetryProperties(JsonTypeInfo typeInfo) + { + if (!typeof(Event).IsAssignableFrom(typeInfo.Type)) + return; + + for (int i = typeInfo.Properties.Count - 1; i >= 0; i--) + if (_fields.Contains(typeInfo.Properties[i].Name)) + typeInfo.Properties.RemoveAt(i); + } + + public static bool TryRead(Event ev, string name, JsonElement value) + { + try + { + switch (name.Replace("_", "").ToLowerInvariant()) + { + case "outcome" when value.ValueKind == JsonValueKind.String && value.GetString() is Event.KnownOutcomes.Success or Event.KnownOutcomes.Failure or Event.KnownOutcomes.Unknown: + ev.Outcome = value.GetString(); + return true; + case "result" when value.ValueKind == JsonValueKind.String: + ev.Result = value.GetString(); + return true; + case "parentreferenceid" when value.ValueKind == JsonValueKind.String && IsReference(value.GetString()): + ev.ParentReferenceId = value.GetString(); + return true; + case "rootreferenceid" when value.ValueKind == JsonValueKind.String && IsReference(value.GetString()): + ev.RootReferenceId = value.GetString(); + return true; + case "measurements" when value.ValueKind == JsonValueKind.Array && value.EnumerateArray().All(IsMeasurement): + ev.Measurements = value.Deserialize>(_options); + return true; + case "labels" when value.ValueKind == JsonValueKind.Object && value.EnumerateObject().All(property => property.Value.ValueKind == JsonValueKind.String): + ev.Labels = value.Deserialize>(_options); + return true; + } + } + catch (JsonException) + { + // This property was previously extension data; preserve its original value. + } + + return false; + } + + public static bool IsReference(string? value) => value is not null && ReferencePattern().IsMatch(value); + + private static bool IsMeasurement(JsonElement element) => element.ValueKind == JsonValueKind.Null || + (element.ValueKind == JsonValueKind.Object && element.EnumerateObject().Any(property => + property.Name.Equals("name", StringComparison.OrdinalIgnoreCase) || + property.Name.Equals("value", StringComparison.OrdinalIgnoreCase) || + property.Name.Equals("unit", StringComparison.OrdinalIgnoreCase))); + + [GeneratedRegex("^[a-zA-Z0-9-]{8,100}$")] + private static partial Regex ReferencePattern(); +} diff --git a/src/Exceptionless.Core/Services/EventTelemetryService.cs b/src/Exceptionless.Core/Services/EventTelemetryService.cs new file mode 100644 index 0000000000..6d18152d88 --- /dev/null +++ b/src/Exceptionless.Core/Services/EventTelemetryService.cs @@ -0,0 +1,106 @@ +using Exceptionless.Core.Models; +using Exceptionless.Core.Repositories; +using Exceptionless.Core.Repositories.Queries; +using Foundatio.Repositories; +using Foundatio.Repositories.Extensions; +using Foundatio.Repositories.Models; + +namespace Exceptionless.Core.Services; + +/// Shared, bounded telemetry queries for the API and MCP. Callers authorize and validate inputs. +public sealed class EventTelemetryService(IEventRepository eventRepository) +{ + public async Task GetChartAsync(AppFilter systemFilter, EventChart chart, string? filter, DateTime start, DateTime end) + { + RepositoryQueryDescriptor query = q => q + .SortExpression("-date") + .AppFilter(systemFilter) + .FilterExpression(filter) + .EnforceEventStackFilter() + .DateRange(start, end, "date") + .Index(start, end) + .EventChart(chart, start, end); + + if (chart.Mode == "events") + { + // Load only the fields used to name series and build points; events can carry large error and request data. + var events = await eventRepository.FindAsync(q => query(q).Include(e => e.Id, e => e.Date, e => e.Source, e => e.StackId, + e => e.Outcome!, e => e.Result!, e => e.Labels!, e => e.Measurements!), o => o.PageLimit(EventChartQueryExtensions.MaxPoints)); + var groups = events.Documents.GroupBy(ev => GetSeriesName(ev, chart.GroupBy)).ToList(); + return new EventChartResult + { + Total = events.Total, + Truncated = events.HasMore || events.Total > events.Documents.Count || groups.Count > EventChartQueryExtensions.MaxSeries, + Series = groups.Take(EventChartQueryExtensions.MaxSeries).Select(group => new EventChartSeries(group.Key, + group.OrderBy(ev => ev.Date).Select(ev => new EventChartPoint(ev.Date.UtcDateTime, + ev.Measurements?.FirstOrDefault(m => m.Name == chart.Measurement && m.Unit == chart.Unit)?.Value, 1, ev.Id)).ToList())).ToList() + }; + } + + var result = await eventRepository.CountAsync(query); + var response = new EventChartResult + { + Total = result.Total, + IntervalMilliseconds = EventChartQueryExtensions.GetInterval(start, end) + }; + if (chart.GroupBy is null) + response.Series.Add(ReadSeries("All events", result.Aggregations, chart)); + else + { + var buckets = result.Aggregations.Terms("series")?.Buckets ?? []; + response.Truncated = buckets.Sum(b => b.Total ?? 0) < result.Total; + foreach (var bucket in buckets) + response.Series.Add(ReadSeries(bucket.Key, bucket.Aggregations, chart)); + } + return response; + } + + public async Task GetMeasurementsAsync(AppFilter systemFilter, string? filter, DateTime start, DateTime end) + { + var result = await eventRepository.CountAsync(q => q.AppFilter(systemFilter).FilterExpression(filter).EnforceEventStackFilter() + .DateRange(start, end, "date").Index(start, end).MeasurementCatalog()); + var measurements = GetBucket(result.Aggregations, "measurements"); + var names = measurements?.Aggregations.Terms("names")?.Buckets ?? []; + var values = new List(); + bool truncated = names.Sum(n => n.Total ?? 0) < (measurements?.Total ?? 0); + foreach (var name in names) + { + var units = name.Aggregations.Terms("units")?.Buckets ?? []; + truncated |= units.Sum(u => u.Total ?? 0) < (name.Total ?? 0); + values.AddRange(units.Select(u => new EventMeasurementDescriptor(name.Key, u.Key))); + } + return new EventMeasurementCatalog(values.OrderBy(m => m.Name, StringComparer.Ordinal).ThenBy(m => m.Unit, StringComparer.Ordinal).ToList(), truncated); + } + + private static string GetSeriesName(PersistentEvent ev, string? groupBy) => groupBy switch + { + null => "All events", + "source" => ev.Source ?? "(missing)", + "stack" => ev.StackId, + "outcome" => ev.Outcome ?? "(missing)", + "result" => ev.Result ?? "(missing)", + _ => ev.Labels?.GetValueOrDefault(groupBy["labels.".Length..]) ?? "(missing)" + }; + + private static SingleBucketAggregate? GetBucket(IReadOnlyDictionary aggregations, string name) + => aggregations.GetValueOrDefault(name) as SingleBucketAggregate; + + private static EventChartSeries ReadSeries(string name, IReadOnlyDictionary aggregations, EventChart chart) + { + var points = new List(); + foreach (var bucket in aggregations.DateHistogram("dates")?.Buckets ?? []) + { + var selected = GetBucket(bucket.Aggregations, "measurements") is { } nested ? GetBucket(nested.Aggregations, "selected") : null; + double? value = chart.Measurement is null ? bucket.Total ?? 0 : null; + if (selected is { Total: > 0 } && selected.Aggregations.TryGetValue("value", out var metric)) + value = metric switch + { + ValueAggregate numeric => numeric.Value, + PercentilesAggregate percentiles => percentiles.Items.FirstOrDefault()?.Value, + _ => null + }; + points.Add(new EventChartPoint(bucket.Date, value, bucket.Total ?? 0)); + } + return new EventChartSeries(name, points); + } +} diff --git a/src/Exceptionless.Core/Validation/EventTelemetryValidation.cs b/src/Exceptionless.Core/Validation/EventTelemetryValidation.cs new file mode 100644 index 0000000000..13d299ce96 --- /dev/null +++ b/src/Exceptionless.Core/Validation/EventTelemetryValidation.cs @@ -0,0 +1,40 @@ +using System.ComponentModel.DataAnnotations; +using System.Text.RegularExpressions; +using Exceptionless.Core.Models; +using Exceptionless.Core.Serialization; + +namespace Exceptionless.Core.Validation; + +public static class EventTelemetryValidation +{ + public static IEnumerable ValidateRelationshipsAndLabels(Event ev) + { + if (ev.ParentReferenceId is not null && (!EventTelemetryReader.IsReference(ev.ParentReferenceId) || ev.ParentReferenceId == ev.ReferenceId)) + yield return new ValidationResult("ParentReferenceId must be a valid reference to a different event.", [nameof(ev.ParentReferenceId)]); + if (ev.RootReferenceId is not null && !EventTelemetryReader.IsReference(ev.RootReferenceId)) + yield return new ValidationResult("RootReferenceId must be a valid event reference.", [nameof(ev.RootReferenceId)]); + if (ev.Measurements is not null && (ev.Measurements.Any(m => m is null) || ev.Measurements.Select(m => m?.Name).Distinct(StringComparer.Ordinal).Count() != ev.Measurements.Count)) + yield return new ValidationResult("Measurements must have unique names and cannot contain null entries.", [nameof(ev.Measurements)]); + if (ev.Labels is not null && ev.Labels.Any(d => !Regex.IsMatch(d.Key, EventMeasurement.NamePattern) || d.Value is null || d.Value.Length > 256)) + yield return new ValidationResult("Label names must be valid measurement names and values must be strings of at most 256 characters.", [nameof(ev.Labels)]); + } + + public static IDictionary GetErrors(Event ev) + { + var results = ValidateRelationshipsAndLabels(ev).ToList(); + ValidateProperty(ev, nameof(ev.Outcome), ev.Outcome, results); + ValidateProperty(ev, nameof(ev.Result), ev.Result, results); + ValidateProperty(ev, nameof(ev.Measurements), ev.Measurements, results); + ValidateProperty(ev, nameof(ev.Labels), ev.Labels, results); + foreach (var measurement in ev.Measurements ?? []) + if (measurement is not null) + Validator.TryValidateObject(measurement, new ValidationContext(measurement), results, validateAllProperties: true); + + return results.SelectMany(result => result.MemberNames.DefaultIfEmpty("Measurements").Select(member => (member, result.ErrorMessage))) + .GroupBy(result => result.member) + .ToDictionary(group => group.Key, group => group.Select(result => result.ErrorMessage ?? "Invalid telemetry.").ToArray()); + } + + private static void ValidateProperty(Event ev, string name, object? value, List results) + => Validator.TryValidateProperty(value, new ValidationContext(ev) { MemberName = name }, results); +} diff --git a/src/Exceptionless.Web/Api/Endpoints/EventEndpoints.cs b/src/Exceptionless.Web/Api/Endpoints/EventEndpoints.cs index 05ad674605..8027a8b881 100644 --- a/src/Exceptionless.Web/Api/Endpoints/EventEndpoints.cs +++ b/src/Exceptionless.Web/Api/Endpoints/EventEndpoints.cs @@ -25,6 +25,25 @@ public static IEndpointRouteBuilder MapEventEndpoints(this IEndpointRouteBuilder .AddEndpointFilter() .WithTags("Event"); + group.MapPost("organizations/{organizationId:objectid}/events/chart", async (string organizationId, EventChartRequest request, HttpContext httpContext, IMediator mediator, IMediatorResultMapper resultMapper) + => (await mediator.InvokeAsync>(new GetEventChart(organizationId, request, httpContext))).ToHttpResult(resultMapper)) + .RequireAuthorization(AuthorizationRoles.EventsReadPolicy) + .Produces() + .ProducesProblem(StatusCodes.Status400BadRequest) + .ProducesProblem(StatusCodes.Status404NotFound) + .ProducesValidationProblem(StatusCodes.Status422UnprocessableEntity) + .ProducesProblem(StatusCodes.Status426UpgradeRequired) + .WithSummary("Chart event measurements or counts"); + + group.MapGet("organizations/{organizationId:objectid}/events/measurements", async (string organizationId, HttpContext httpContext, IMediator mediator, IMediatorResultMapper resultMapper, string? filter = null, string? time = null, string? offset = null) + => (await mediator.InvokeAsync>(new GetEventMeasurements(organizationId, filter, time, offset, httpContext))).ToHttpResult(resultMapper)) + .RequireAuthorization(AuthorizationRoles.EventsReadPolicy) + .Produces() + .ProducesProblem(StatusCodes.Status400BadRequest) + .ProducesProblem(StatusCodes.Status404NotFound) + .ProducesProblem(StatusCodes.Status426UpgradeRequired) + .WithSummary("List measurement names and units in the selected events"); + // Count group.MapGet("events/count", async (HttpContext httpContext, IMediator mediator, IMediatorResultMapper resultMapper, string? filter = null, string? aggregations = null, string? time = null, string? offset = null, string? mode = null) => (await mediator.InvokeAsync>(new GetEventCount(filter, aggregations, time, offset, mode, httpContext))).ToHttpResult(resultMapper)) diff --git a/src/Exceptionless.Web/Api/Handlers/EventHandler.cs b/src/Exceptionless.Web/Api/Handlers/EventHandler.cs index a034c63691..373168ce00 100644 --- a/src/Exceptionless.Web/Api/Handlers/EventHandler.cs +++ b/src/Exceptionless.Web/Api/Handlers/EventHandler.cs @@ -39,6 +39,7 @@ public class EventHandler( IProjectRepository projectRepository, IStackRepository stackRepository, EventPostService eventPostService, + EventTelemetryService eventTelemetryService, IQueue eventUserDescriptionQueue, MiniValidationValidator miniValidationValidator, FormattingPluginManager formattingPluginManager, @@ -687,6 +688,46 @@ public async Task> Handle(DeleteEvents message) return results; } + public async Task> Handle(GetEventChart message) + { + var organization = await GetOrganizationAsync(message.OrganizationId, message.Context); + if (organization is null) + return Result.NotFound("Organization not found."); + if (organization.IsSuspended) + return PlanLimitResult("Unable to view events for the suspended organization."); + + var request = message.Request; + var validation = await validator.ValidateQueryAsync(request.Filter); + if (!validation.IsValid) + return Result.BadRequest(validation.Message ?? "Invalid filter."); + + var systemFilter = new AppFilter(organization) { UsesPremiumFeatures = true }; + if (ApiFilterPolicy.IsPremiumFeatureQueryBlocked(systemFilter)) + return PlanLimitResult(ApiFilterPolicy.PremiumSearchUpgradeMessage); + + var time = TimeRangeParser.GetTimeInfo(request.Time, request.Offset, timeProvider, _allowedDateFields, DefaultDateField, organization.GetRetentionUtcCutoff(appOptions.MaximumRetentionDays, timeProvider)); + return await eventTelemetryService.GetChartAsync(systemFilter, request.Chart, request.Filter, time.Range.UtcStart, time.Range.UtcEnd); + } + + public async Task> Handle(GetEventMeasurements message) + { + var organization = await GetOrganizationAsync(message.OrganizationId, message.Context); + if (organization is null) + return Result.NotFound("Organization not found."); + if (organization.IsSuspended) + return PlanLimitResult("Unable to view events for the suspended organization."); + var validation = await validator.ValidateQueryAsync(message.Filter); + if (!validation.IsValid) + return Result.BadRequest(validation.Message ?? "Invalid filter."); + + var systemFilter = new AppFilter(organization) { UsesPremiumFeatures = true }; + if (ApiFilterPolicy.IsPremiumFeatureQueryBlocked(systemFilter)) + return PlanLimitResult(ApiFilterPolicy.PremiumSearchUpgradeMessage); + + var time = TimeRangeParser.GetTimeInfo(message.Time, message.Offset, timeProvider, _allowedDateFields, DefaultDateField, organization.GetRetentionUtcCutoff(appOptions.MaximumRetentionDays, timeProvider)); + return await eventTelemetryService.GetMeasurementsAsync(systemFilter, message.Filter, time.Range.UtcStart, time.Range.UtcEnd); + } + #region Private Helpers private async Task> CountInternalAsync(AppFilter sf, TimeInfo ti, HttpContext httpContext, string? filter = null, string? aggregations = null, string? mode = null) diff --git a/src/Exceptionless.Web/Api/Handlers/SavedViewHandler.cs b/src/Exceptionless.Web/Api/Handlers/SavedViewHandler.cs index 45baae424c..35006d0a16 100644 --- a/src/Exceptionless.Web/Api/Handlers/SavedViewHandler.cs +++ b/src/Exceptionless.Web/Api/Handlers/SavedViewHandler.cs @@ -281,22 +281,23 @@ public async Task>> Ha } var savedViews = message.Definitions.Select(definition => new SavedView - { - OrganizationId = PredefinedSavedViewsDataSeed.SystemOrganizationId, - CreatedByUserId = GetCurrentUserId(), - PredefinedKey = definition.Key, - Name = definition.Name, - Slug = definition.Slug, - ViewType = definition.ViewType, - Filter = definition.Filter, - Time = definition.Time, - Sort = definition.Sort, - FilterDefinitions = definition.FilterDefinitions is { } filterDefinitions ? JsonSerializer.Serialize(filterDefinitions) : null, - Columns = CopyColumns(definition.Columns), - ShowStats = definition.ShowStats, - ShowChart = definition.ShowChart, - Version = 1 - }) + { + OrganizationId = PredefinedSavedViewsDataSeed.SystemOrganizationId, + CreatedByUserId = GetCurrentUserId(), + PredefinedKey = definition.Key, + Name = definition.Name, + Slug = definition.Slug, + ViewType = definition.ViewType, + Filter = definition.Filter, + Time = definition.Time, + Sort = definition.Sort, + FilterDefinitions = definition.FilterDefinitions is { } filterDefinitions ? JsonSerializer.Serialize(filterDefinitions) : null, + Columns = CopyColumns(definition.Columns), + ShowStats = definition.ShowStats, + ShowChart = definition.ShowChart, + Chart = definition.Chart is null ? null : definition.Chart with { }, + Version = 1 + }) .ToList(); foreach (var savedView in savedViews) @@ -364,6 +365,10 @@ public async Task> Handle(UpdateSavedViewMessage message) var changedNames = message.Changes.GetChangedPropertyNames(); message.Changes.Patch(original); + // Delta's cross-model copy skips null references. An explicit null chart restores the default. + if (changedNames.Contains(nameof(UpdateSavedView.Chart)) && message.Changes.TryGetPropertyValue(nameof(UpdateSavedView.Chart), out var chart)) + original.Chart = (EventChart?)chart; + if (changedNames.Contains(nameof(UpdateSavedView.Slug))) original.Slug = ToSlug(original.Slug); @@ -818,6 +823,7 @@ private SavedView CreatePredefinedSavedView(string organizationId, PredefinedSav Columns = CopyColumns(definition.Columns), ShowStats = definition.ShowStats, ShowChart = definition.ShowChart, + Chart = definition.Chart is null ? null : definition.Chart with { }, Version = 1 }; @@ -838,6 +844,7 @@ private static bool ApplyPredefinedSavedView(SavedView savedView, PredefinedSave changed |= SetColumnsIfChanged(savedView, definition.Columns); changed |= SetIfChanged(savedView, definition.ShowStats, static (view, value) => view.ShowStats = value, static view => view.ShowStats); changed |= SetIfChanged(savedView, definition.ShowChart, static (view, value) => view.ShowChart = value, static view => view.ShowChart); + changed |= SetIfChanged(savedView, definition.Chart is null ? null : definition.Chart with { }, static (view, value) => view.Chart = value, static view => view.Chart); changed |= SetIfChanged(savedView, 1, static (view, value) => view.Version = value, static view => view.Version); changed |= SetIfChanged(savedView, PredefinedSavedViewContentHasher.GetContentHash(savedView), static (view, value) => view.PredefinedContentHash = value, static view => view.PredefinedContentHash); @@ -943,7 +950,8 @@ private static PredefinedSavedViewDefinition ToPredefinedSavedView(SavedView sav FilterDefinitions = ParseFilterDefinitions(savedView.FilterDefinitions), Columns = savedView.Columns, ShowStats = savedView.ShowStats, - ShowChart = savedView.ShowChart + ShowChart = savedView.ShowChart, + Chart = savedView.Chart }; } diff --git a/src/Exceptionless.Web/Api/Messages/EventMessages.cs b/src/Exceptionless.Web/Api/Messages/EventMessages.cs index 6c1ac3bb95..eb50a0babf 100644 --- a/src/Exceptionless.Web/Api/Messages/EventMessages.cs +++ b/src/Exceptionless.Web/Api/Messages/EventMessages.cs @@ -18,6 +18,10 @@ public record GetEventsByStack(string StackId, string? Filter, string? Sort, str public record GetEventsByReferenceId(string ReferenceId, string? Offset, string? Mode, int? Page, int Limit, string? Before, string? After, string? Include, HttpContext Context); public record GetEventsByReferenceIdAndProject(string ReferenceId, string ProjectId, string? Offset, string? Mode, int? Page, int Limit, string? Before, string? After, string? Include, HttpContext Context); +// Measurements +public record GetEventChart(string OrganizationId, EventChartRequest Request, HttpContext Context); +public record GetEventMeasurements(string OrganizationId, string? Filter, string? Time, string? Offset, HttpContext Context); + // Sessions public record GetEventsBySessionId(string SessionId, string? Filter, string? Sort, string? Time, string? Offset, string? Mode, int? Page, int Limit, string? Before, string? After, string? Include, HttpContext Context); public record GetEventsBySessionIdAndProject(string SessionId, string ProjectId, string? Filter, string? Sort, string? Time, string? Offset, string? Mode, int? Page, int Limit, string? Before, string? After, string? Include, HttpContext Context); diff --git a/src/Exceptionless.Web/ClientApp/e2e/tests/saved-views.e2e.ts b/src/Exceptionless.Web/ClientApp/e2e/tests/saved-views.e2e.ts index 9de4671196..aaa7604dde 100644 --- a/src/Exceptionless.Web/ClientApp/e2e/tests/saved-views.e2e.ts +++ b/src/Exceptionless.Web/ClientApp/e2e/tests/saved-views.e2e.ts @@ -667,6 +667,15 @@ test('save completion does not overwrite a newly active saved view', async ({ e2 const firstViewSlug = savedViewSlug(firstViewName); const secondViewSlug = savedViewSlug(secondViewName); const authorizationHeaders = { Authorization: `Bearer ${e2eScenario.userToken}` }; + const configureChart = async (measurement: string, unit: string) => { + await page.getByRole('button', { name: 'Configure chart' }).click(); + const dialog = page.getByRole('dialog', { name: 'Configure chart' }); + await dialog.getByLabel('Measurement', { exact: true }).fill(measurement); + await dialog.getByLabel('Unit', { exact: true }).fill(unit); + await dialog.getByRole('button', { exact: true, name: 'Apply' }).click(); + await expect(dialog).toBeHidden(); + await expect.poll(() => JSON.parse(new URL(page.url()).searchParams.get('chart') ?? 'null')?.measurement).toBe(measurement); + }; const createView = (name: string, slug: string, time: string) => request.post(`/api/v2/organizations/${e2eScenario.organizationId}/saved-views`, { data: { @@ -701,6 +710,7 @@ test('save completion does not overwrite a newly active saved view', async ({ e2 await page.goto(`/event/${firstViewSlug}`); await expect(page.getByRole('heading', { name: firstViewName })).toBeVisible(); + await configureChart('duration', 'ms'); await page.getByRole('button', { name: /^Date/ }).filter({ visible: true }).first().click(); await page.getByRole('button', { name: 'Last 90 days' }).click(); await expect(page.getByLabel('Unsaved view changes')).toBeVisible(); @@ -739,13 +749,18 @@ test('save completion does not overwrite a newly active saved view', async ({ e2 await expect.poll(() => secondViewRequestTimes.some((time) => time.includes('now-1d'))).toBe(true); await page.getByRole('button', { name: /^Date/ }).filter({ visible: true }).first().click(); await page.getByRole('button', { name: 'Last 7 days' }).click(); + await configureChart('allocated', 'By'); const saveCompleted = page.waitForResponse( (response) => response.request().method() === 'PATCH' && new URL(response.url()).pathname === `/api/v2/saved-views/${firstView.id}` ); releaseSave(); - await saveCompleted; + const savedViewResponse = await saveCompleted; + expect((await savedViewResponse.json()).chart).toMatchObject({ measurement: 'duration', unit: 'ms' }); await expect.poll(() => secondViewRequestTimes.some((time) => time.includes('now-7d'))).toBe(true); + await expect(page.getByText('allocated · avg (By)', { exact: true })).toBeVisible(); + expect(JSON.parse(new URL(page.url()).searchParams.get('chart') ?? 'null')).toMatchObject({ measurement: 'allocated', unit: 'By' }); await page.reload(); + await expect(page.getByText('allocated · avg (By)', { exact: true })).toBeVisible(); await expect( page .getByRole('button', { name: /Date\s+Last 7 days/ }) diff --git a/src/Exceptionless.Web/ClientApp/src/lib/features/events/api.svelte.ts b/src/Exceptionless.Web/ClientApp/src/lib/features/events/api.svelte.ts index f14e187745..04a01614c6 100644 --- a/src/Exceptionless.Web/ClientApp/src/lib/features/events/api.svelte.ts +++ b/src/Exceptionless.Web/ClientApp/src/lib/features/events/api.svelte.ts @@ -3,6 +3,7 @@ import { createMutation, createQuery, keepPreviousData, QueryClient, useQueryCli import { SvelteSet } from 'svelte/reactivity'; import type { WebSocketMessageValue } from '#features/websockets/models.ts'; +import type { EventChartRequest, EventChartResult, EventMeasurementCatalog } from '#generated/api.ts'; import type { CountResult, WorkInProgressResult } from '#shared/models/index.ts'; import { accessToken } from '#features/auth/index.svelte.ts'; @@ -364,6 +365,47 @@ export function deleteEvent(request: DeleteEventsRequest) { })); } +export function getEventChartQuery(request: { body: EventChartRequest | undefined; enabled?: boolean; organizationId: string | undefined }) { + return createQuery(() => { + const organizationId = request.organizationId; + const body = request.body; + return { + enabled: !!accessToken.current && !!organizationId && !!body && (request.enabled ?? true), + queryFn: async ({ signal }) => { + const response = await useFetchClient().postJSON(`organizations/${organizationId}/events/chart`, body, { + signal + }); + return response.data!; + }, + queryKey: [...queryKeys.organizations(organizationId), 'chart', body], + staleTime: ORGANIZATION_EVENT_QUERY_STALE_TIME_MS + }; + }); +} + +export function getEventMeasurementsQuery(request: { enabled: boolean; filter?: null | string; organizationId: string | undefined; time?: null | string }) { + return createQuery(() => { + const organizationId = request.organizationId; + const params = { + filter: request.filter, + offset: DEFAULT_OFFSET, + time: request.time + }; + return { + enabled: !!accessToken.current && !!organizationId && request.enabled, + queryFn: async ({ signal }) => { + const response = await useFetchClient().getJSON(`organizations/${organizationId}/events/measurements`, { + params, + signal + }); + return response.data!; + }, + queryKey: [...queryKeys.organizations(organizationId), 'measurements', params], + staleTime: ORGANIZATION_EVENT_QUERY_STALE_TIME_MS + }; + }); +} + // Cacheable reads intentionally finish after their observer unmounts so navigation can reuse the result instead of aborting and restarting the request. export function getEventQuery(request: GetEventRequest) { return createQuery(() => ({ @@ -729,7 +771,7 @@ function isOrganizationEventDashboardQueryKey(queryKey: readonly unknown[]): boo return ( queryKey[0] === queryKeys.type[0] && queryKey[1] === 'organizations' && - (queryKey[3] === 'count' || queryKey[3] === 'events' || queryKey[3] === 'sessions') + (queryKey[3] === 'count' || queryKey[3] === 'events' || queryKey[3] === 'sessions' || queryKey[3] === 'chart' || queryKey[3] === 'measurements') ); } diff --git a/src/Exceptionless.Web/ClientApp/src/lib/features/events/components/event-chart-settings.svelte b/src/Exceptionless.Web/ClientApp/src/lib/features/events/components/event-chart-settings.svelte new file mode 100644 index 0000000000..6caefbe79b --- /dev/null +++ b/src/Exceptionless.Web/ClientApp/src/lib/features/events/components/event-chart-settings.svelte @@ -0,0 +1,269 @@ + + + + + {#snippet child({ props })}{/snippet} + + + + Configure chart + Chart a measurement in these events. Save the view to keep this configuration. + +
{ + event.preventDefault(); + event.stopPropagation(); + void form.handleSubmit(); + }} + > + + {#each textFields as item (item.key)} + + {#snippet children(field)} + + {item.label} + field.handleChange(event.currentTarget.value)} + onblur={field.handleBlur} + aria-invalid={ariaInvalid(field)} + aria-describedby={`chart-${item.key}-error`} + /> + + + {/snippet} + + {/each} + {#each selectFields as item (item.key)} + + {#snippet children(field)} + + {item.label} + + {item.options.find((option) => option.value === field.state.value)?.label} + {#each item.options as option (option.value)}{option.label}{/each} + + + {/snippet} + + {/each} + Use the recorded unit, such as s for seconds, By for bytes, or {'{call}'} for calls. Units are exact: ms and s are separate series. To count + all events, clear Measurement and Unit and select Count with Time buckets. + {#if catalog.error}{/if} + {#if catalog.data?.truncated}

+ Only the most common measurements are suggested. You can enter another name and unit. +

{/if} + + + + + +
+ + +{#each [...new Set(catalog.data?.measurements.map((m) => m.name) ?? [])] as name (name)}{/each} +{#each [...new Set(catalog.data?.measurements.map((m) => m.unit) ?? [])] as unit (unit)}{/each} + diff --git a/src/Exceptionless.Web/ClientApp/src/lib/features/events/components/event-chart-settings.svelte.test.ts b/src/Exceptionless.Web/ClientApp/src/lib/features/events/components/event-chart-settings.svelte.test.ts new file mode 100644 index 0000000000..c7b8c413e9 --- /dev/null +++ b/src/Exceptionless.Web/ClientApp/src/lib/features/events/components/event-chart-settings.svelte.test.ts @@ -0,0 +1,44 @@ +import { fireEvent, render, screen, waitFor } from '@testing-library/svelte'; +import { describe, expect, it, vi } from 'vitest'; + +import EventChartSettings from './event-chart-settings.svelte'; + +vi.mock('#features/organizations/context.svelte.ts', () => ({ organization: { current: 'organization-a' } })); +vi.mock('../api.svelte', () => ({ + getEventMeasurementsQuery: () => ({ data: { measurements: [{ name: 'duration', unit: 'ms' }], truncated: false }, error: null }) +})); + +describe('event chart settings', () => { + it('validates a measurement unit and applies a typed chart configuration', async () => { + const onApply = vi.fn(); + render(EventChartSettings, { chart: null, onApply }); + await fireEvent.click(screen.getByRole('button', { name: 'Configure chart' })); + await fireEvent.input(screen.getByLabelText('Measurement'), { target: { value: 'duration' } }); + await fireEvent.click(screen.getByRole('button', { name: 'Apply' })); + await screen.findByText('Select a measurement and its unit together.'); + expect(onApply).not.toHaveBeenCalled(); + await fireEvent.input(screen.getByLabelText('Unit'), { target: { value: 'ms' } }); + await fireEvent.input(screen.getByLabelText('Split by'), { target: { value: 'labels.branch' } }); + await fireEvent.click(screen.getByRole('button', { name: 'Apply' })); + + await waitFor(() => + expect(onApply).toHaveBeenCalledWith({ + aggregation: 'avg', + display: 'line', + group_by: 'labels.branch', + measurement: 'duration', + mode: 'buckets', + unit: 'ms' + }) + ); + }); + + it('explicitly restores the default chart', async () => { + const onApply = vi.fn(); + render(EventChartSettings, { chart: { aggregation: 'avg', display: 'line', measurement: 'duration', mode: 'buckets', unit: 'ms' }, onApply }); + await fireEvent.click(screen.getByRole('button', { name: 'Configure chart' })); + await fireEvent.click(screen.getByRole('button', { name: 'Default chart' })); + + expect(onApply).toHaveBeenCalledWith(null); + }); +}); diff --git a/src/Exceptionless.Web/ClientApp/src/lib/features/events/components/event-measurement-chart.stories.ts b/src/Exceptionless.Web/ClientApp/src/lib/features/events/components/event-measurement-chart.stories.ts new file mode 100644 index 0000000000..279e0c5ac5 --- /dev/null +++ b/src/Exceptionless.Web/ClientApp/src/lib/features/events/components/event-measurement-chart.stories.ts @@ -0,0 +1,47 @@ +import type { Meta, StoryObj } from '@storybook/sveltekit'; + +import EventMeasurementChart from './event-measurement-chart.svelte'; + +const meta = { + args: { + chart: { aggregation: 'avg', display: 'line', group_by: 'source', measurement: 'duration', mode: 'buckets', unit: 'ms' }, + isLoading: false, + onSelect: () => {}, + result: { + interval_milliseconds: 86400000, + series: [ + { + name: 'Account import', + points: [ + { count: 1, date: '2026-10-01T12:00:00Z', value: 120 }, + { count: 1, date: '2026-10-02T12:00:00Z', value: 100 }, + { count: 0, date: '2026-10-03T12:00:00Z', value: null }, + { count: 1, date: '2026-10-04T12:00:00Z', value: 90 }, + { count: 1, date: '2026-10-05T12:00:00Z', value: 95 } + ] + }, + { + name: 'Transaction import', + points: [ + { count: 1, date: '2026-10-01T12:00:00Z', value: 45 }, + { count: 1, date: '2026-10-02T12:00:00Z', value: 35 }, + { count: 1, date: '2026-10-03T12:00:00Z', value: 0 }, + { count: 1, date: '2026-10-04T12:00:00Z', value: 25 }, + { count: 1, date: '2026-10-05T12:00:00Z', value: 20 } + ] + } + ], + total: 8, + truncated: false + } + }, + component: EventMeasurementChart, + title: 'Components/Events/Measurement chart' +} satisfies Meta; + +export default meta; +type Story = StoryObj; +export const Line: Story = {}; +export const Bars: Story = { args: { chart: { ...meta.args.chart, display: 'bar' } } }; +export const Empty: Story = { args: { result: { series: [], total: 0, truncated: false } } }; +export const Limited: Story = { args: { result: { ...meta.args.result, truncated: true } } }; diff --git a/src/Exceptionless.Web/ClientApp/src/lib/features/events/components/event-measurement-chart.svelte b/src/Exceptionless.Web/ClientApp/src/lib/features/events/components/event-measurement-chart.svelte new file mode 100644 index 0000000000..9d4ea4f7b4 --- /dev/null +++ b/src/Exceptionless.Web/ClientApp/src/lib/features/events/components/event-measurement-chart.svelte @@ -0,0 +1,116 @@ + + + +

+ {chart.measurement ?? 'Events'} · {chart.mode === 'events' ? 'Individual events' : chart.aggregation} + {unit === '1' ? '' : `(${unit})`} +

+ {#if isLoading} + + {:else if !data.some((point) => point.value != null)} +

No matching observations in this time range.

+ {:else} + + {#if chart.display === 'bar'} + selectAt(details.data.date, details.series.key, details.data.value)} + > + {#snippet tooltip()}{@render chartTooltip()}{/snippet} + + {:else} + selectAt(details.data.x, details.series.key, details.data.y)} + > + {#snippet marks({ context })} + {#each context.series.visibleSeries as item (item.key)} + + + {/each} + {/snippet} + {#snippet tooltip()}{@render chartTooltip()}{/snippet} + + {/if} + +

Select a point to inspect its events. Missing measurements appear as gaps.

+
+ View observations +
+ {#each series as item (item.key)} +

{item.label}

+ {#each item.data as point, index (`${point.date.getTime()}-${index}`)} + + {/each} + {/each} +
+
+ {/if} + {#if result?.truncated}

+ Showing a limited set of observations or series. Narrow the filter or time range for a complete view. Events without the split field are excluded + from bucketed series. +

{/if} + + +{#snippet chartTooltip()} + formatDateLabel(value instanceof Date ? value : new Date(value as number | string))}> + {#snippet formatter({ name, value })}{name}{formatMeasurement(typeof value === 'number' ? value : null, unit)}{/snippet} + +{/snippet} diff --git a/src/Exceptionless.Web/ClientApp/src/lib/features/events/components/event-measurement-chart.svelte.test.ts b/src/Exceptionless.Web/ClientApp/src/lib/features/events/components/event-measurement-chart.svelte.test.ts new file mode 100644 index 0000000000..ebd81687a4 --- /dev/null +++ b/src/Exceptionless.Web/ClientApp/src/lib/features/events/components/event-measurement-chart.svelte.test.ts @@ -0,0 +1,56 @@ +import { fireEvent, render, screen } from '@testing-library/svelte'; +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; + +import type { EventChart, EventChartResult } from '#generated/api.ts'; + +import EventMeasurementChart from './event-measurement-chart.svelte'; + +const chart: EventChart = { aggregation: 'avg', display: 'line', measurement: 'duration', mode: 'events', unit: 'ms' }; + +describe('EventMeasurementChart', () => { + beforeEach(() => { + vi.stubGlobal( + 'ResizeObserver', + class { + public disconnect() {} + public observe() {} + public unobserve() {} + } + ); + }); + + afterEach(() => vi.unstubAllGlobals()); + + it('keeps coincident events separately accessible and preserves zeros and missing values', async () => { + const onSelect = vi.fn(); + const result: EventChartResult = { + series: [ + { + name: 'Import', + points: [ + { count: 1, date: '2026-10-01T12:00:00Z', event_id: 'zero', value: 0 }, + { count: 1, date: '2026-10-01T12:00:00Z', event_id: 'ten', value: 10 }, + { count: 0, date: '2026-10-01T12:01:00Z', value: null } + ] + } + ], + total: 2, + truncated: true + }; + + render(EventMeasurementChart, { chart, isLoading: false, onSelect, result }); + await fireEvent.click(screen.getByText('View observations')); + await fireEvent.click(screen.getByRole('button', { name: /10 ms/ })); + + expect(onSelect).toHaveBeenCalledWith(expect.objectContaining({ event_id: 'ten', value: 10 }), 'Import'); + expect(screen.getByText('0 ms').closest('button')).toBeTruthy(); + expect(screen.getByRole('button', { name: /—/ }).hasAttribute('disabled')).toBe(true); + expect(screen.getByRole('status').textContent).toContain('limited set'); + }); + + it('explains an empty measurement selection without rendering a false zero', () => { + render(EventMeasurementChart, { chart, isLoading: false, onSelect: vi.fn(), result: { series: [], total: 0, truncated: false } }); + expect(screen.getByText('No matching observations in this time range.')).toBeTruthy(); + expect(screen.queryByText('0 ms')).toBeNull(); + }); +}); diff --git a/src/Exceptionless.Web/ClientApp/src/lib/features/events/components/views/overview.svelte b/src/Exceptionless.Web/ClientApp/src/lib/features/events/components/views/overview.svelte index c57a181ad8..ea3a987109 100644 --- a/src/Exceptionless.Web/ClientApp/src/lib/features/events/components/views/overview.svelte +++ b/src/Exceptionless.Web/ClientApp/src/lib/features/events/components/views/overview.svelte @@ -10,6 +10,7 @@ import { Button } from '#comp/ui/button/index.ts'; import * as Table from '#comp/ui/table/index.ts'; import * as EventsFacetedFilter from '#features/events/components/filters/index.ts'; + import { formatMeasurement } from '#features/events/event-chart.ts'; import { getErrorType, getLocation, @@ -70,10 +71,58 @@ let requestUrl = $derived(getRequestInfoUrl(event)); let requestUrlPath = $derived(getRequestInfoPath(event)); let version = $derived(event.data?.['@version']); + + function relatedEventsHref(filter: string): string { + return `${resolve('/(app)/event')}?filter=${encodeURIComponent(`project:${event.project_id} AND (${filter})`)}&time=all`; + } + function relationshipHref(referenceId: string): string { + return `${resolve('/(app)/event/by-ref/[referenceId]', { + referenceId + })}?project=${encodeURIComponent(event.project_id)}`; + } + {#if event.outcome} + Outcome{event.outcome} + {/if} + {#if event.result} + Result{event.result} + {/if} + {#if event.parent_reference_id} + Parent event{event.parent_reference_id} + {/if} + {#if event.root_reference_id} + Root event{event.root_reference_id} · View related events + {/if} + {#if event.reference_id && (event.measurements?.length || event.outcome || event.result || event.parent_reference_id || event.root_reference_id)} + ChildrenView child events + {/if} + {#each event.measurements ?? [] as measurement (measurement.name)} + {measurement.name}{formatMeasurement(measurement.value, measurement.unit)} + {/each} + {#each Object.entries(event.labels ?? {}) as [name, value] (name)} + {name}{value} + {/each} {#if isSessionStart} Duration diff --git a/src/Exceptionless.Web/ClientApp/src/lib/features/events/event-chart.test.ts b/src/Exceptionless.Web/ClientApp/src/lib/features/events/event-chart.test.ts new file mode 100644 index 0000000000..ecb724036f --- /dev/null +++ b/src/Exceptionless.Web/ClientApp/src/lib/features/events/event-chart.test.ts @@ -0,0 +1,60 @@ +import { describe, expect, it } from 'vitest'; + +import type { EventChart } from '#generated/api.ts'; + +import { chartSeries, chartSignature, eventChartSchema, formatMeasurement, measurementUnit, parseChartParameter } from './event-chart'; + +const chart: EventChart = { aggregation: 'avg', display: 'line', measurement: 'duration', mode: 'buckets', unit: 'ms' }; + +describe('event charts', () => { + it('preserves missing values, real zero values, and coincident observations', () => { + const series = chartSeries({ + series: [ + { + name: 'Import', + points: [ + { count: 1, date: '2026-10-01T12:00:00Z', event_id: 'first', value: 0 }, + { count: 1, date: '2026-10-01T12:00:00Z', event_id: 'second', value: 12 }, + { count: 0, date: '2026-10-01T12:05:00Z', value: null } + ] + } + ], + total: 3, + truncated: false + }); + expect(series[0]!.data.map((point) => point.value)).toEqual([0, 12, null]); + expect(series[0]!.data.map((point) => point.event_id)).toEqual(['first', 'second', undefined]); + expect(formatMeasurement(0, 'ms')).toBe('0 ms'); + expect(formatMeasurement(null, 'ms')).toBe('—'); + }); + + it('distinguishes an explicit default chart from an absent or invalid URL override', () => { + expect(parseChartParameter(JSON.stringify(chart))).toEqual(chart); + expect(parseChartParameter('null')).toBeNull(); + expect(parseChartParameter(null)).toBeUndefined(); + expect(parseChartParameter('{invalid')).toBeUndefined(); + expect(parseChartParameter(JSON.stringify({ ...chart, unit: null }))).toBeUndefined(); + }); + + it('requires a unit for measurements and allows generic event counts without a measurement', () => { + expect(eventChartSchema.safeParse({ ...chart, unit: null }).success).toBe(false); + expect(eventChartSchema.safeParse({ ...chart, aggregation: 'count', measurement: null, unit: null }).success).toBe(true); + expect(eventChartSchema.safeParse({ ...chart, aggregation: 'count', measurement: null, mode: 'events', unit: null }).success).toBe(false); + expect(eventChartSchema.safeParse({ ...chart, group_by: 'labels.branch', unit: '{call}' }).success).toBe(true); + expect(eventChartSchema.safeParse({ ...chart, group_by: 'result' }).success).toBe(true); + expect(eventChartSchema.safeParse({ ...chart, group_by: 'dimensions.branch' }).success).toBe(false); + }); + + it('formats counts independently of the measured unit', () => { + expect(measurementUnit({ ...chart, aggregation: 'count' })).toBe('1'); + expect(measurementUnit({ ...chart, aggregation: 'count', mode: 'events' })).toBe('ms'); + }); + + it('compares chart configurations without depending on property order or absent optional fields', () => { + expect(chartSignature(chart)).toBe( + chartSignature({ aggregation: 'avg', display: 'line', group_by: null, measurement: 'duration', mode: 'buckets', unit: 'ms' }) + ); + expect(chartSignature(chart)).not.toBe(chartSignature({ ...chart, unit: 's' })); + expect(chartSignature(null)).toBe(chartSignature(undefined)); + }); +}); diff --git a/src/Exceptionless.Web/ClientApp/src/lib/features/events/event-chart.ts b/src/Exceptionless.Web/ClientApp/src/lib/features/events/event-chart.ts new file mode 100644 index 0000000000..5893a69582 --- /dev/null +++ b/src/Exceptionless.Web/ClientApp/src/lib/features/events/event-chart.ts @@ -0,0 +1,59 @@ +import type { EventChart, EventChartPoint, EventChartResult } from '#generated/api.ts'; + +import { EventChartSchema } from '#generated/schemas.ts'; + +export const eventChartSchema = EventChartSchema.superRefine((chart, context) => { + if (!chart.measurement && (chart.aggregation !== 'count' || chart.mode === 'events')) { + context.addIssue({ code: 'custom', message: 'Select a measurement, or use bucketed event counts.', path: ['measurement'] }); + } + if (!!chart.measurement !== !!chart.unit) { + context.addIssue({ code: 'custom', message: 'Select a measurement and its unit together.', path: ['unit'] }); + } +}); + +export type MeasurementPoint = Omit & { date: Date }; + +export function chartSeries(result: EventChartResult | undefined) { + return (result?.series ?? []).map((series, index) => { + const key = `series${index}`; + return { + color: `var(--chart-${(index % 5) + 1})`, + data: series.points.map((point) => ({ ...point, date: new Date(point.date), [key]: point.value })), + key, + label: series.name, + value: key + }; + }); +} + +export function chartSignature(chart: EventChart | null | undefined): string { + return JSON.stringify(chart ? [chart.measurement ?? null, chart.unit ?? null, chart.aggregation, chart.group_by ?? null, chart.mode, chart.display] : null); +} + +export function formatMeasurement(value: null | number | undefined, unit: string): string { + if (value == null || !Number.isFinite(value)) { + return '—'; + } + const number = new Intl.NumberFormat(undefined, { maximumSignificantDigits: 6 }).format(value); + return unit === '1' ? number : `${number} ${unit}`; +} + +export function measurementUnit(chart: EventChart): string { + return chart.mode !== 'events' && chart.aggregation === 'count' ? '1' : (chart.unit ?? '1'); +} + +export function parseChartParameter(value: null | string | undefined): EventChart | null | undefined { + if (!value) { + return undefined; + } + try { + const parsed: unknown = JSON.parse(value); + if (parsed === null) { + return null; + } + const result = eventChartSchema.safeParse(parsed); + return result.success ? result.data : undefined; + } catch { + return undefined; + } +} diff --git a/src/Exceptionless.Web/ClientApp/src/lib/features/saved-views/components/saved-view-picker.svelte b/src/Exceptionless.Web/ClientApp/src/lib/features/saved-views/components/saved-view-picker.svelte index 89b364e296..67c18a57a6 100644 --- a/src/Exceptionless.Web/ClientApp/src/lib/features/saved-views/components/saved-view-picker.svelte +++ b/src/Exceptionless.Web/ClientApp/src/lib/features/saved-views/components/saved-view-picker.svelte @@ -1,6 +1,8 @@ @@ -55,6 +57,7 @@ activeSavedView?: SavedView; autoFillColumnId: AutoFillColumnSelection; canModifySavedView?: boolean; + chart?: EventChart | null; columnOrder?: string[]; columnSizing?: Record; columnVisibility?: Record; @@ -93,6 +96,7 @@ activeSavedView, autoFillColumnId, canModifySavedView = true, + chart, columnOrder, columnSizing, columnVisibility, @@ -235,6 +239,7 @@ function getUpdateBody(): UpdateSavedView { return { + chart, columns: getSavedColumnSettings(), filter: currentFilterString || null, filter_definitions: serializeFilters(filters), @@ -308,6 +313,7 @@ const tour = savedViewCreateTour; const filterDefinitions = serializeFilters(filters); const body: NewSavedView = { + chart, columns: getSavedColumnSettings(), filter: currentFilterString || undefined, filter_definitions: filterDefinitions, diff --git a/src/Exceptionless.Web/ClientApp/src/lib/features/saved-views/use-saved-views.svelte.ts b/src/Exceptionless.Web/ClientApp/src/lib/features/saved-views/use-saved-views.svelte.ts index f38e289f60..9bfe213c50 100644 --- a/src/Exceptionless.Web/ClientApp/src/lib/features/saved-views/use-saved-views.svelte.ts +++ b/src/Exceptionless.Web/ClientApp/src/lib/features/saved-views/use-saved-views.svelte.ts @@ -7,6 +7,7 @@ import { tick, untrack } from 'svelte'; import { SvelteSet, SvelteURL } from 'svelte/reactivity'; import type { IFilter } from '#comp/faceted-filter/index.ts'; +import type { EventChart } from '#generated/api.ts'; import { applyTimeFilter, @@ -15,6 +16,7 @@ import { getFiltersFromCache, serializeFilters } from '#features/events/components/filters/helpers.svelte.ts'; +import { chartSignature } from '#features/events/event-chart.ts'; import { getVisibleUrl } from '#features/navigation/visible-url.ts'; import { organization } from '#features/organizations/context.svelte.ts'; import { updateShallowHistory } from '#features/shared/history-state.ts'; @@ -78,6 +80,7 @@ export interface UseSavedViewsOptions { defaultTime?: null | string; filterCacheKey: (filter: null | string) => string; getAvailableColumnIds?: () => string[]; + getChart?: () => EventChart | null; getColumnOrder?: () => ColumnOrderState; getColumnSizing?: () => ColumnSizingState; getColumnVisibility?: () => ColumnVisibilityState; @@ -89,6 +92,7 @@ export interface UseSavedViewsOptions { getTime?: () => null | string | undefined; normalizeSavedView?: (view: SavedView) => SavedView; queryParams: SavedViewQueryParams; + setChart?: (chart: EventChart | null) => void; setColumnOrder?: (order: ColumnOrderState) => void; setColumnSizing?: (sizing: ColumnSizingState) => void; setColumnVisibility?: (visibility: ColumnVisibilityState) => void; @@ -280,7 +284,7 @@ export function getSavedViewDraftIdentity( } export function getSavedViewStateSignature( - view: Pick + view: Pick ): string { const columns = Object.entries(view.columns ?? {}) .sort(([left], [right]) => left.localeCompare(right)) @@ -294,6 +298,7 @@ export function getSavedViewStateSignature( ]); return JSON.stringify({ + chart: chartSignature(view.chart), columns, filter: view.filter ?? null, filterDefinitions: view.filter_definitions ?? null, @@ -512,9 +517,10 @@ export function useSavedViews(options: UseSavedViewsOptions): UseSavedViewsRetur wrappedColumnIds = filterAvailableColumnIds(getSavedWrappedColumnIds(view), availableColumnIds); } - function applyDisplayState(view: Pick | undefined): void { + function applyDisplayState(view: Pick | undefined): void { options.setShowStats?.(view?.show_stats ?? true); options.setShowChart?.(view?.show_chart ?? true); + options.setChart?.(view?.chart ?? null); } function getDraftIdentity(view: SavedView): SavedViewDraftIdentity | undefined { @@ -679,6 +685,7 @@ export function useSavedViews(options: UseSavedViewsOptions): UseSavedViewsRetur options.setShowStats?.(draft && 'showStats' in draft ? draft.showStats! : (view.show_stats ?? true)); options.setShowChart?.(draft && 'showChart' in draft ? draft.showChart! : (view.show_chart ?? true)); + options.setChart?.(view.chart ?? null); if (draft && 'sort' in draft && !page.url.searchParams.has('sort') && !preservePendingSort) { setSortQueryParam(options.queryParams, getDraftSortQueryParam(view.sort ?? null, draft.sort ?? null), 'replace'); @@ -1095,6 +1102,10 @@ export function useSavedViews(options: UseSavedViewsOptions): UseSavedViewsRetur return true; } + if (options.getChart && chartSignature(options.getChart()) !== chartSignature(view.chart)) { + return true; + } + return false; }); diff --git a/src/Exceptionless.Web/ClientApp/src/lib/generated/api.ts b/src/Exceptionless.Web/ClientApp/src/lib/generated/api.ts index a5983264a5..ce2491c5b0 100644 --- a/src/Exceptionless.Web/ClientApp/src/lib/generated/api.ts +++ b/src/Exceptionless.Web/ClientApp/src/lib/generated/api.ts @@ -162,6 +162,73 @@ export interface CountResult { data?: null | object; } +/** Configuration of a saved view's single event chart. Null retains the default event-count chart. */ +export interface EventChart { + /** @pattern ^[a-zA-Z][a-zA-Z0-9_.-]{0,99}$ */ + measurement?: null | string; + unit?: null | string; + /** @pattern ^(avg|min|max|sum|p50|p95|p99|count)$ */ + aggregation: string; + /** @pattern ^(source|stack|outcome|result|labels\.[a-zA-Z][a-zA-Z0-9_.-]{0,99})$ */ + group_by?: null | string; + /** @pattern ^(buckets|events)$ */ + mode: string; + /** @pattern ^(line|bar)$ */ + display: string; +} + +export interface EventChartPoint { + /** @format date-time */ + date: string; + /** @format double */ + value?: null | number; + /** @format int64 */ + count: number; + event_id?: null | string; +} + +export interface EventChartRequest { + /** Configuration of a saved view's single event chart. Null retains the default event-count chart. */ + chart: EventChart; + filter?: null | string; + time?: null | string; + offset?: null | string; +} + +export interface EventChartResult { + series: EventChartSeries[]; + /** @format int64 */ + total: number; + truncated: boolean; + /** @format int64 */ + interval_milliseconds?: null | number; +} + +export interface EventChartSeries { + name: string; + points: EventChartPoint[]; +} + +/** A numeric observation on one event. Units describe values, not their storage type. */ +export interface EventMeasurement { + /** @pattern ^[a-zA-Z][a-zA-Z0-9_.-]{0,99}$ */ + name: string; + /** @format double */ + value?: null | number; + /** Case-sensitive unit. Prefer UCUM/OTel symbols: s for duration, By for bytes, {call} for counts, or 1 for dimensionless values. */ + unit: string; +} + +export interface EventMeasurementCatalog { + measurements: EventMeasurementDescriptor[]; + truncated: boolean; +} + +export interface EventMeasurementDescriptor { + name: string; + unit: string; +} + export interface EventSubmissionSettings { enabled: boolean; configured_enabled: boolean; @@ -266,6 +333,7 @@ export interface NewSavedView { columns?: null | Record; show_stats?: null | boolean; show_chart?: null | boolean; + chart?: null | EventChart; /** If true, the view will only be visible to the current user. Defaults to false. */ is_private?: null | boolean; } @@ -472,6 +540,21 @@ export interface PersistentEvent { * @format int32 */ count?: null | number; + /** + * The ECS-aligned outcome: success, failure, or unknown. Omit when not applicable. + * @pattern ^(success|failure|unknown)$ + */ + outcome?: null | string; + /** An optional detailed result, such as skipped, cancelled, or timed_out. Does not imply an outcome. */ + result?: null | string; + /** Reference of the immediate parent event in the same project. */ + parent_reference_id?: null | string; + /** Reference of the root event in the same project. Supplied by the producer. */ + root_reference_id?: null | string; + /** Up to 32 uniquely named numeric observations. Missing values are not zero. */ + measurements?: EventMeasurement[] | null; + /** Up to 32 categorical labels, indexed as exact strings without type inference. */ + labels?: null | Record; /** Optional data entries that contain additional information about this event. */ data?: null | Record; /** An optional identifier to be used for referencing this event instance at a later time. */ @@ -490,6 +573,7 @@ export interface PredefinedSavedViewDefinition { columns?: null | Record; showStats?: null | boolean; showChart?: null | boolean; + chart?: null | EventChart; } export interface ProblemDetails { @@ -686,6 +770,7 @@ export interface UpdateSavedView { columns?: null | Record; show_stats?: null | boolean; show_chart?: null | boolean; + chart?: EventChart | null; } export interface UpdateSavedViewDefault { @@ -943,6 +1028,7 @@ export interface ViewSavedView { columns?: null | Record; show_stats?: null | boolean; show_chart?: null | boolean; + chart?: null | EventChart; name: string; slug: string; time?: null | string; diff --git a/src/Exceptionless.Web/ClientApp/src/lib/generated/schemas.ts b/src/Exceptionless.Web/ClientApp/src/lib/generated/schemas.ts index e3a11cd9f8..2e8ad5dbbc 100644 --- a/src/Exceptionless.Web/ClientApp/src/lib/generated/schemas.ts +++ b/src/Exceptionless.Web/ClientApp/src/lib/generated/schemas.ts @@ -207,6 +207,109 @@ export const CountResultSchema = object({ }); export type CountResultFormData = Infer; +export const EventChartSchema = object({ + measurement: string() + .min(1, "Measurement is required") + .regex(/^[a-zA-Z][a-zA-Z0-9_.-]{0,99}$/, "Measurement has invalid format") + .nullable() + .optional(), + unit: string() + .min(1, "Unit is required") + .max(32, "Unit must be at most 32 characters") + .nullable() + .optional(), + aggregation: string() + .min(1, "Aggregation is required") + .regex( + /^(avg|min|max|sum|p50|p95|p99|count)$/, + "Aggregation has invalid format", + ), + group_by: string() + .min(1, "Group by is required") + .regex( + /^(source|stack|outcome|result|labels\.[a-zA-Z][a-zA-Z0-9_.-]{0,99})$/, + "Group by has invalid format", + ) + .nullable() + .optional(), + mode: string() + .min(1, "Mode is required") + .regex(/^(buckets|events)$/, "Mode has invalid format"), + display: string() + .min(1, "Display is required") + .regex(/^(line|bar)$/, "Display has invalid format"), +}); +export type EventChartFormData = Infer; + +export const EventChartPointSchema = object({ + date: iso.datetime(), + value: number().nullable(), + count: int(), + event_id: string().min(1, "Event id is required").nullable().optional(), +}); +export type EventChartPointFormData = Infer; + +export const EventChartRequestSchema = object({ + chart: lazy(() => EventChartSchema), + filter: string() + .min(1, "Filter is required") + .max(2000, "Filter must be at most 2000 characters") + .nullable() + .optional(), + time: string() + .min(1, "Time is required") + .max(100, "Time must be at most 100 characters") + .nullable() + .optional(), + offset: string() + .min(1, "Offset is required") + .max(20, "Offset must be at most 20 characters") + .nullable() + .optional(), +}); +export type EventChartRequestFormData = Infer; + +export const EventChartResultSchema = object({ + series: array(lazy(() => EventChartSeriesSchema)), + total: int(), + truncated: boolean(), + interval_milliseconds: int().nullable().optional(), +}); +export type EventChartResultFormData = Infer; + +export const EventChartSeriesSchema = object({ + name: string().min(1, "Name is required"), + points: array(lazy(() => EventChartPointSchema)), +}); +export type EventChartSeriesFormData = Infer; + +export const EventMeasurementSchema = object({ + name: string() + .min(1, "Name is required") + .regex(/^[a-zA-Z][a-zA-Z0-9_.-]{0,99}$/, "Name has invalid format"), + value: number().nullable(), + unit: string() + .min(1, "Unit is required") + .max(32, "Unit must be at most 32 characters"), +}); +export type EventMeasurementFormData = Infer; + +export const EventMeasurementCatalogSchema = object({ + measurements: array(lazy(() => EventMeasurementDescriptorSchema)), + truncated: boolean(), +}); +export type EventMeasurementCatalogFormData = Infer< + typeof EventMeasurementCatalogSchema +>; + +export const EventMeasurementDescriptorSchema = object({ + name: string().min(1, "Name is required"), + unit: string().min(1, "Unit is required"), +}); +export type EventMeasurementDescriptorFormData = Infer< + typeof EventMeasurementDescriptorSchema +>; + export const EventSubmissionSettingsSchema = object({ enabled: boolean(), configured_enabled: boolean(), @@ -353,6 +456,7 @@ export const NewSavedViewSchema = object({ .optional(), show_stats: boolean().nullable().optional(), show_chart: boolean().nullable().optional(), + chart: lazy(() => EventChartSchema).optional(), is_private: boolean().nullable().optional(), }); export type NewSavedViewFormData = Infer; @@ -597,6 +701,29 @@ export const PersistentEventSchema = object({ geo: string().min(1, "Geo is required").nullable().optional(), value: number().nullable().optional(), count: int32().nullable().optional(), + outcome: string() + .min(1, "Outcome is required") + .max(100, "Outcome must be at most 100 characters") + .regex(/^(success|failure|unknown)$/, "Outcome has invalid format") + .nullable() + .optional(), + result: string() + .min(1, "Result is required") + .max(100, "Result must be at most 100 characters") + .nullable() + .optional(), + parent_reference_id: string() + .min(1, "Parent reference id is required") + .nullable() + .optional(), + root_reference_id: string() + .min(1, "Root reference id is required") + .nullable() + .optional(), + measurements: array(lazy(() => EventMeasurementSchema)) + .nullable() + .optional(), + labels: record(string(), string()).nullable().optional(), data: record(string(), unknown()).nullable().optional(), reference_id: string() .min(1, "Reference id is required") @@ -622,6 +749,7 @@ export const PredefinedSavedViewDefinitionSchema = object({ .optional(), showStats: boolean().nullable().optional(), showChart: boolean().nullable().optional(), + chart: lazy(() => EventChartSchema).optional(), }); export type PredefinedSavedViewDefinitionFormData = Infer< typeof PredefinedSavedViewDefinitionSchema @@ -824,6 +952,7 @@ export const UpdateSavedViewSchema = object({ .optional(), show_stats: boolean().nullable().optional(), show_chart: boolean().nullable().optional(), + chart: lazy(() => EventChartSchema).optional(), }); export type UpdateSavedViewFormData = Infer; @@ -1109,6 +1238,7 @@ export const ViewSavedViewSchema = object({ .optional(), show_stats: boolean().nullable().optional(), show_chart: boolean().nullable().optional(), + chart: lazy(() => EventChartSchema).optional(), name: string().min(1, "Name is required"), slug: string().min(1, "Slug is required"), time: string().min(1, "Time is required").nullable().optional(), diff --git a/src/Exceptionless.Web/ClientApp/src/routes/(app)/event/+page.svelte b/src/Exceptionless.Web/ClientApp/src/routes/(app)/event/+page.svelte index bf58b7f9e3..2bcaae8ee0 100644 --- a/src/Exceptionless.Web/ClientApp/src/routes/(app)/event/+page.svelte +++ b/src/Exceptionless.Web/ClientApp/src/routes/(app)/event/+page.svelte @@ -1,4 +1,5 @@