Repository navigation
Parse adls.anon as a boolean in FsspecFileIO - #4154
Closed
kevinjqliu wants to merge 1 commit into
Closed
kevinjqliu wants to merge 1 commit into
kevinjqliu wants to merge 1 commit into
Conversation
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Explicit boolean False becomes None, allowing environment settings to override the requested authentication mode.
1 open issue
What changed in this PR
This PR corrects string parsing for adls.anon in FsspecFileIO, supporting the default-credential configuration introduced in #2661.
Changes:
- Parse configured
adls.anonstrings as booleans while retainingNonewhen unset. - Add regression coverage for
"false"and"true".
| File | Description |
|---|---|
| tests/io/test_fsspec.py | Tests boolean values passed to adlfs from string configuration. |
| pyiceberg/io/fsspec.py | Uses property_as_bool for the anonymous-access option. |
Review effort: Balanced
kevinjqliu
marked this pull request as draft
October 25, 2026 04:12
adls.anon was passed to adlfs as a string, and adlfs only uses default credentials when anon is False, so "false" silently disabled them. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
kevinjqliu
force-pushed
the
kevinjqliu-adls-anon-boolean-fix
branch
from
October 25, 2026 04:19
537ce44 to
7ba06d8
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rationale for this change
Follow-up to #2661, which added
adls.anon.adls.anonis passed to adlfs as-is. Config from.pyiceberg.yamlandPYICEBERG_*env vars is always a string, soadls.anon: falsereaches adlfs as"false". adlfs expects a bool (anon: bool = None) and only usesDefaultAzureCredentialwhenanon is False."false"is notFalse, so default credentials are skipped and requests go out anonymously.This parses
adls.anonwithproperty_as_bool. When unset, it still passesNoneso adlfs falls back to its default andAZURE_STORAGE_ANON.adls.anonmain"false"(YAML or env var)DefaultAzureCredentialFalse(Python)DefaultAzureCredentialDefaultAzureCredential"true"/True"maybe"ValueErrorAre these changes tested?
Yes,
test_adls_anon_parsed_as_boolcovers the cases above and fails onmain.uv run python -m pytest tests/io/test_fsspec.py -k "anon_parsed or adls_account_name or adls_does_not_mutate": 6 passedadls.anon=falseand anaz loginsession:mainsends an unauthenticated request, this PR authenticatesAre there any user-facing changes?
Yes, bug fix.
adls.anon=falseset from config now usesDefaultAzureCredentialinstead of anonymous access. Invalid values now raiseValueError.AI Disclosure: Assisted by GitHub Copilot.