[C++] is_null(nan_is_null=true) ignores NaN and null values in dictionary-encoded float arrays #51127

Describe the bug, including details regarding any error messages, version, or platform.

is_null(arr, nan_is_null=True) does not mark NaN values as null when arr is dictionary-encoded with floating-point dictionary values.

IsNullExec decides whether to take the NaN path by looking at the top-level type id. For a dictionary-encoded array that id is DICTIONARY, so the floating-point check is skipped entirely and NaN entries in the dictionary are never marked null. The same call on the decoded (non-dictionary) array returns the expected result.

Related: null values inside the dictionary (as opposed to null indices) are also not reflected, so a valid index pointing at a null dictionary slot is reported as non-null.

Discussion and a first attempt at a fix are in #51000; @pitrou suggested computing is_null on the dictionary values, take-ing by the indices, and OR-ing the result with the index validity bitmap, which covers both the NaN case and null dictionary values in one pass.

Component(s)

C++