Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions rust/crates/sift_cli/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,10 @@ This project adheres to [Semantic Versioning](http://semver.org/).

### What's New

- `get_data` accepts `include_received_at` (alias `includeReceivedAt`). When
set, and the data service returns `sift_received_at`, the Parquet file
includes a receipt-time column per channel in unix nanos next to the
generated `timestamp_unix_nanos`. `upload_dataset` skips those columns.
- `create_artifact` and the Sift skill now say which link relation to use:
`source` for the run or asset an artifact was built from, and `derived_from`
only for another artifact. Agents no longer send `derived_from` for a run and
Expand Down
7 changes: 7 additions & 0 deletions rust/crates/sift_cli/assets/skills/sift/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,13 @@ tool.
to enable them, then restart the MCP client.
- **Data:** `get_data` writes channel data to a Parquet file. `sql` queries
Parquet files. `upload_dataset` streams a Parquet dataset into Sift.
- **Compare generation time with receipt time.** Call `get_data` with
`include_received_at: true` and `sample_ms: 0`. When Sift has a receipt time,
the file includes an Int64 column whose name contains `received_at="true"`,
in unix nanos, aligned with that channel's samples. `timestamp_unix_nanos` is
still when the measurement was generated. The receipt column is absent when
the service has no receipt time, including enum and bit-field channels.
`upload_dataset` skips receipt columns.
- **Links:** `explore_url`.
- **Docs:** `search_docs`.
- **Writes:** `create_rule`, `update_rule`, `archive_rule`, `unarchive_rule`,
Expand Down
4 changes: 3 additions & 1 deletion rust/crates/sift_cli/assets/skills/sift/references/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -129,7 +129,9 @@ If the CLI does not support the user's file type:
Do not reach for the MCP `upload_dataset` tool to import a user's file. It
accepts only Sift's canonical Parquet schema — every column named
`<channel_name> {channel_id="<uuid>"}` — so it round-trips the output of
`get_data`, and rejects a foreign file with `missing attribute block`.
`get_data`, and rejects a foreign file with `missing attribute block`. A
`received_at="true"` column from `get_data` is a receipt time, not a channel,
and upload skips it.

### Stream data

Expand Down
44 changes: 41 additions & 3 deletions rust/crates/sift_mcp/src/service/common/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -74,16 +74,18 @@ pub fn secs_and_subsec_nanos_to_unix_nanos(sec: i64, subsec_nanos: i32) -> i64 {

/// A fully-qualified Arrow column name with the channel-specific fields the
/// data pipeline cares about. `Display` emits the canonical Parquet column-name
/// string: `<name> {channel_id="...", bit_field_element="...", run="...", units="..."}`.
/// Empty optional fields are omitted. Construct via [`ColumnName::builder`] or
/// `TryFrom<String>` (when re-hydrating a string from a Parquet schema).
/// string: `<name> {channel_id="...", bit_field_element="...", run="...", units="...", received_at="true"}`.
/// Empty optional fields are omitted. `received_at="true"` marks a receipt-time
/// column rather than the channel's samples. Construct via [`ColumnName::builder`] or
/// `TryFrom<&str>` (when re-hydrating a string from a Parquet schema).
#[derive(Debug, Clone, PartialEq, Eq, Hash)]
pub struct ColumnName {
name: String,
channel_id: String,
bit_field_element: Option<String>,
run: Option<String>,
units: Option<String>,
received_at: bool,
}

impl ColumnName {
Expand All @@ -94,6 +96,20 @@ impl ColumnName {
bit_field_element: None,
run: None,
units: None,
received_at: false,
}
}

/// The same channel identity, marked as that channel's receipt-time column.
/// Units are dropped: the column holds unix nanos, not the measurement unit.
pub fn with_received_at(&self) -> Self {
Self {
name: self.name.clone(),
channel_id: self.channel_id.clone(),
bit_field_element: self.bit_field_element.clone(),
run: self.run.clone(),
units: None,
received_at: true,
}
}

Expand All @@ -116,6 +132,10 @@ impl ColumnName {
pub fn units(&self) -> Option<&str> {
self.units.as_deref()
}

pub fn received_at(&self) -> bool {
self.received_at
}
}

impl fmt::Display for ColumnName {
Expand All @@ -130,6 +150,9 @@ impl fmt::Display for ColumnName {
if let Some(v) = &self.units {
write!(f, ", units=\"{v}\"")?;
}
if self.received_at {
write!(f, ", received_at=\"true\"")?;
}
write!(f, "}}")
}
}
Expand Down Expand Up @@ -157,6 +180,7 @@ impl TryFrom<&str> for ColumnName {
let mut bit_field_element: Option<String> = None;
let mut run: Option<String> = None;
let mut units: Option<String> = None;
let mut received_at = false;

for segment in attrs.split(',') {
let segment = segment.trim();
Expand All @@ -183,6 +207,12 @@ impl TryFrom<&str> for ColumnName {
"bit_field_element" => bit_field_element = Some(val),
"run" => run = Some(val),
"units" => units = Some(val),
"received_at" => {
if val != "true" {
bail!("`received_at` must be \"true\"");
}
received_at = true;
}
other => bail!("unknown attribute key `{other}`"),
}
}
Expand All @@ -196,6 +226,7 @@ impl TryFrom<&str> for ColumnName {
bit_field_element,
run,
units,
received_at,
})
}
}
Expand All @@ -210,6 +241,7 @@ pub struct ColumnNameBuilder<'a> {
bit_field_element: Option<String>,
run: Option<String>,
units: Option<String>,
received_at: bool,
}

impl<'a> ColumnNameBuilder<'a> {
Expand All @@ -228,13 +260,19 @@ impl<'a> ColumnNameBuilder<'a> {
self
}

pub fn received_at(mut self, value: bool) -> Self {
self.received_at = value;
self
}

pub fn build(self) -> ColumnName {
ColumnName {
name: self.name.to_string(),
channel_id: self.channel_id.to_string(),
bit_field_element: self.bit_field_element,
run: self.run,
units: self.units,
received_at: self.received_at,
}
}
}
27 changes: 27 additions & 0 deletions rust/crates/sift_mcp/src/service/common/test.rs
Original file line number Diff line number Diff line change
Expand Up @@ -137,6 +137,33 @@ fn column_name_try_from_missing_channel_id_errors() {
);
}

#[test]
fn column_name_received_at_round_trips_and_drops_units() {
let original = ColumnName::builder("temp", "c1")
.run(Some("r42"))
.units(Some("C"))
.build()
.with_received_at();
assert!(original.received_at());
assert_eq!(original.units(), None);
assert_eq!(
original.to_string(),
"temp {channel_id=\"c1\", run=\"r42\", received_at=\"true\"}"
);
let parsed = ColumnName::try_from(original.to_string().as_str()).expect("should parse");
assert_eq!(parsed, original);
}

#[test]
fn column_name_try_from_rejects_received_at_other_than_true() {
let err = ColumnName::try_from("temp {channel_id=\"c1\", received_at=\"false\"}")
.expect_err("only true is a receipt column");
assert!(
err.to_string().contains("`received_at` must be \"true\""),
"unexpected error: {err}"
);
}

#[test]
fn column_name_try_from_unknown_key_errors() {
let err = ColumnName::try_from("temp {channel_id=\"c1\", flavor=\"strawberry\"}")
Expand Down
Loading
Loading