Skip to content

feat: add in operator (inverse of has)#2721

Open
toller892 wants to merge 1 commit into
mikefarah:masterfrom
toller892:feat/add-in-operator
Open

feat: add in operator (inverse of has)#2721
toller892 wants to merge 1 commit into
mikefarah:masterfrom
toller892:feat/add-in-operator

Conversation

@toller892

Copy link
Copy Markdown
Contributor

What

Add the in operator to yq, mirroring jq's in operator (inverse of has).

Why

Closes #2322. The in operator checks whether a value is present in a mapping (as a key) or sequence (as an element), enabling idiomatic membership filtering in yq expressions.

Usage

# data.yml
- {item: Pizza, type: Food}
- {item: Rose, type: Flower}
- {item: Hammer, type: Tool}
# Filter by membership — select items whose type is Tool or Food
yq '.[] | select(.type | in(["Tool", "Food"]))' data.yml
# Output
{item: Pizza, type: Food}
{item: Hammer, type: Tool}
# Check map key existence (using variable binding)
echo 'a: 1' | yq '. as $m | "a" | in($m)'
# true

Implementation

  • operator_in.go — Handler function: evaluates the RHS to get the collection, then checks if the LHS (piped-in value) is present as a key in a mapping or as an element in a sequence.
  • operator_in_test.go — 11 test scenarios covering map key checks, array value membership, select() integration, and variable binding patterns.
  • operation.go — Registers inOpType with NumArgs: 1, Precedence: 50.
  • lexer_participle.go — Adds simpleOp("in", inOpType) to the lexer.

Note on context model

Because RHS arguments are evaluated in the pipe context (not the root document), in(.) evaluates . as the piped value, not the original document. Use variable bindings to capture the collection first:

# Does NOT work as expected (`. evaluates to the pipe input):
echo 'a: 1' | yq '"a" | in(.)'  # false

# Works correctly (capture document in variable):
echo 'a: 1' | yq '. as $m | "a" | in($m)'  # true

This matches yq's existing context model (same as has with complex expressions).

Add the `in` operator to check if a value is present in a mapping (as a
key) or sequence (as an element). This mirrors jq's `in` operator which
is defined as the inverse of `has`.

Usage:
  . as $m | "key" | in($m)        -- check map key existence
  . as $m | "value" | in($m)      -- check array value membership
  .[] | select(.type | in(["a","b"])) -- filter by membership

Fixes mikefarah#2322

@mikefarah mikefarah left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: This review was generated by Cursor (AI-assisted code review), not written manually by the reviewer.

Summary

Thanks for this PR — the in operator is a good addition and addresses the primary use case in #2322 (select(.type | in(["Tool", "Food"]))). Registration, lexer wiring, and the test/doc pattern all follow existing yq conventions.

Verdict: request changes — there is a correctness bug in non-scalar array membership that should be fixed before merge.

What's good

  • Correct placement next to has (precedence 50, lexer registration)
  • Primary filtering use case from #2322 works
  • Honest documentation of the context model (in(.) vs in($m))
  • Tests pass (TestInOperatorScenarios, full pkg/yqlib suite)

Blocking issue

Non-scalar array membership is broken. The sequence branch compares only .Value, so any mapping/sequence node (empty .Value) falsely matches any array containing objects. Example: {"x": 99} | in([{"a": 1}]) returns true but should be false. contains already does deep comparison — the sequence branch should reuse it.

Other items

  1. Missing doc header — please add pkg/yqlib/doc/operators/headers/in.md (see headers/has.md, headers/contains.md).
  2. Scalar tag comparison1 | in(["1"]) returns true because only .Value is compared; contains enforces tag matching. Worth aligning or documenting.
  3. Duplicate tests — "Check key exists in map using variable binding" / "In with variable binding - found" (and the not-found pair) are redundant.
  4. Test gaps — no coverage for non-scalar array membership, tag mismatch, or null in arrays.

Suggested fix

case SequenceNode:
    for _, element := range collection.Content {
        contained, err := contains(element, candidate)
        if err != nil {
            return Context{}, err
        }
        if contained {
            isIn = true
            break
        }
    }

Add a test asserting {"x": 99} | in([{"a": 1}])false.

Comment thread pkg/yqlib/operator_in.go
case SequenceNode:
// Check if candidate value is present in the sequence (value membership)
for _, element := range collection.Content {
if element.Value == candidate.Value {

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generated by Cursor, not the reviewer.

Blocking: Comparing only .Value breaks non-scalar membership. Mapping/sequence nodes have an empty .Value, so any object spuriously matches any array of objects:

yq -n '{"x": 99} | in([{"a": 1}, {"b": 2}])'  # true — should be false

Please use contains(element, candidate) here (same deep equality as the contains operator) instead of string .Value comparison.

Comment thread pkg/yqlib/operator_in.go
},
},
{
description: "Check in with select on array elements",

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generated by Cursor, not the reviewer.

Please add test coverage for the non-scalar array bug, e.g.:

{
    skipDoc:    true,
    expression: `{"x": 99} | in([{"a": 1}, {"b": 2}])`,
    expected:   []string{"D0, P[], (!!bool)::false\n"},
},

Also consider tests for tag mismatch (1 | in(["1"])) and null in arrays.

},
{
description: "In with variable binding - found",
document: "a: 1\nb: 2\nc: 3\n",

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generated by Cursor, not the reviewer.

These "In with variable binding" scenarios duplicate "Check key exists/does not exist in map" above (lines 9–23). Consider removing the redundant pair to keep the test file lean.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature request: Membership operator (probably called in)

2 participants