Repository navigation
feat: store %%sparksql results in _df by default - #203
Merged
Merged
Conversation
When a %%sparksql cell does not name a target variable, store the resulting un-truncated DataFrame in `_df` in the notebook namespace. Passing an explicit variable name stores the result there instead of `_df`, and the `%sparksql` line magic still returns the DataFrame directly without modifying the namespace. As before, the variable is only bound once execution and display succeed.
medb
approved these changes
Oct 15, 2026
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request updates the %%sparksql magic command to store the resulting DataFrame in a default variable named _df when no explicit variable name is provided in cell form. The documentation, implementation, and corresponding unit and local tests have been updated to reflect and verify this new default behavior. There are no review comments to address, and the changes look solid.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When a %%sparksql cell does not name a target variable, store the resulting un-truncated DataFrame in
_dfin the notebook namespace. Passing an explicit variable name stores the result there instead of_df, and the%sparksqlline magic still returns the DataFrame directly without modifying the namespace. As before, the variable is only bound once execution and display succeed.