FEAT: Add conda packaging recipes and publish tooling for mssql-python

[Jahnvi Thakkar (jahnvi480)]

Work Item / Issue Reference

AB#47315

---

Summary

This pull request introduces a complete, standalone Conda build and release pipeline for the mssql-python package, decoupling Conda packaging from the wheel/PyPI release process. The new pipelines ensure that Conda packages are built, validated, and published independently, with robust audit and gating steps to prevent regressions and incomplete releases.

Key changes include:

1. New Conda Build and Release Pipelines

• Added OneBranchPipelines/conda-build-pipeline.yml: Implements a full, per-platform Conda build pipeline, repackaging prebuilt wheels into Conda packages for all supported OS/architectures, with validation and artifact consolidation.
• Added OneBranchPipelines/conda-release-pipeline.yml: Introduces a dedicated release pipeline to validate the consolidated Conda packages and optionally publish them to Anaconda.org, with strict gating to ensure completeness and version consistency.

2. Automated Auditing on Pull Requests

• Added .github/workflows/conda-audit.yml: Sets up a GitHub Actions workflow that builds and audits a Linux Conda package on every relevant PR, automatically failing the PR if regressions are detected in binary dependencies or audit checks.

3. Improved Separation and Safety

• The Conda pipelines are now fully decoupled from the wheel/PyPI release pipeline, ensuring that Conda artifacts are not mixed or confused with wheel artifacts and that each release path is independently validated and gated.

4. Platform Coverage and Validation

• The build pipeline covers all major platforms (Windows, macOS Intel/ARM, Linux x86_64/aarch64), using native and cross-builds as appropriate, and includes runtime validation (e.g., import tests, TLS probes) to ensure package integrity.

5. Secure, Gated Publishing

• Publishing to Anaconda.org is strictly gated behind successful validation and is only enabled when explicitly requested, with secrets handled securely and never exposed in logs.

These changes significantly improve the reliability, auditability, and maintainability of the Conda packaging and release process for mssql-python.

[Copilot]

Pull request overview

This PR adds initial conda packaging assets for mssql-python (and its mssql-python-odbc companion) so the project can be published to the Microsoft-owned microsoft Anaconda channel, alongside the existing PyPI wheel distribution.

Changes:

• Adds conda recipes for mssql-python and mssql-python-odbc that repackage existing wheels (no compilation).
• Adds a local PowerShell harness to build both recipes, create a local channel, and validate import / optional live-connect.
• Adds a draft OneBranch stage to build/test/publish conda artifacts from signed release artifacts.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 6 comments.

File	Description
conda/onebranch-publish-conda-stage.yml	Draft OneBranch stage to build from signed artifacts, gate on smoke tests, and publish to the microsoft Anaconda channel.
conda/mssql-python/meta.yaml	Conda recipe for repackaging the mssql-python wheel and depending on a version-locked mssql-python-odbc.
conda/mssql-python-odbc/meta.yaml	Conda recipe for repackaging the proprietary driver wheel into a companion conda package.
conda/build_and_test_local.ps1	Local “test-before-live” harness to build, index, install, and smoke test the conda packages.

Suppressed comments (1)

conda/onebranch-publish-conda-stage.yml:133

• Same string-vs-boolean condition issue here: quoting the template expression turns it into a string, which can cause the publish step to be skipped unexpectedly even when publishToConda is true.

          - task: PowerShell@2
            displayName: 'Publish to anaconda.org/microsoft'
            condition: and(succeeded(), eq('${{ parameters.publishToConda }}', true))

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[github-actions]

📊 Code Coverage Report

🔥 Diff Coverage 100%

🎯 Overall Coverage 83%

📈 Total Lines Covered: 7959 out of 9522 📁 Project: mssql-python

---

Diff Coverage

Diff: main...HEAD, staged and unstaged changes

No lines with coverage information in this diff.

---

📋 Files Needing Attention

📉 Files with overall lowest coverage (click to expand)

mssql_python.pybind.logger_bridge.cpp: 57.9%
mssql_python.pybind.ddbc_bindings.h: 61.5%
mssql_python.pybind.logger_bridge.hpp: 70.8%
mssql_python.pybind.ddbc_bindings.cpp: 77.5%
mssql_python.row.py: 77.6%
mssql_python.pybind.connection.connection_pool.cpp: 81.6%
mssql_python.pybind.connection.connection.cpp: 84.4%
mssql_python.logging.py: 85.5%
mssql_python.helpers.py: 89.3%
mssql_python.pooling.py: 90.1%

---

🔗 Quick Links

⚙️ Build Summary	📋 Coverage Details
View Azure DevOps Build	Browse Full Coverage Report

[Copilot]

Pull request overview

Copilot reviewed 24 out of 24 changed files in this pull request and generated 3 comments.

Suppressed comments (4)

OneBranchPipelines/scripts/build-conda-packages.sh:173

• The verify env name only includes the Python version. On macOS this script is invoked twice on the same agent (native osx-arm64, then cross-target osx-64), so the second conda create -n verify_... can fail with an existing prefix or reuse the wrong-arch env. Include the effective subdir (native vs $CONDA_SUBDIR) in the env name to avoid collisions.

for py in $pyvers; do
  envName="verify_${py//./}"
  echo "=== [py $py] create verify env from local channel ==="

OneBranchPipelines/stages/build-linux-single-stage.yml:116

• This apt install runs before the later apt-get update in the job, so it can fail on agents with stale apt indices (common on hosted Ubuntu). Add an apt-get update (and ideally noninteractive) before installing qemu-user-static/libc6-arm64-cross.

              # qemu-user-static: run aarch64 ELF binaries on the x86_64 host.
              # libc6-arm64-cross: the aarch64 glibc runtime (loader + libc/libm/...)
              # under /usr/aarch64-linux-gnu so qemu can resolve /lib/ld-linux-aarch64.so.1
              # for the emulated aarch64 conda build/verify (QEMU_LD_PREFIX points here).
              sudo apt-get install -y qemu-user-static libc6-arm64-cross

OneBranchPipelines/steps/conda-publish-step.yml:91

• Installing anaconda-client from PyPI without pinning a version can make releases non-reproducible and may unexpectedly break publishing if a new release introduces behavioral changes. Consider pinning to a vetted version (or using a constraints file) so publish behavior is stable.

        $ErrorActionPreference = 'Stop'
        python -m pip install --upgrade pip
        python -m pip install anaconda-client
        # anaconda-client installs the `anaconda` console script onto PATH.
        anaconda --version

OneBranchPipelines/steps/conda-build-validate-step.yml:126

• The error message references $links (the wheel find-links directory), but conda packages are searched under ${{ parameters.outputDir }}/bld. If this trips, the message will mislead troubleshooting.

      if (-not $built) { Write-Error "No conda packages were produced under $($links)"; exit 1 }

[Copilot]

Pull request overview

Copilot reviewed 28 out of 28 changed files in this pull request and generated no new comments.

Suppressed comments (2)

OneBranchPipelines/scripts/build-conda-packages.sh:184

• The verify env name is only based on the Python version (e.g. "verify_310"). On macOS this script is invoked twice on the same agent (osx-64 and osx-arm64) sharing the same Miniforge install/outputDir, so the second invocation will fail when it tries to create an env that already exists. Include the target subdir in the env name (and/or delete any existing env before creating it) so the two runs don’t collide.

for py in $pyvers; do
  envName="verify_${py//./}"
  echo "=== [py $py] create verify env from local channel ==="
  # -c microsoft (ahead of conda-forge) so azure-core/azure-identity/msal resolve from the
  # lean `microsoft` channel, NOT conda-forge whose azure-core recipe over-declares flask/six
  # -> celery/boto3/botocore (~9 MB); see conda-forge/azure-core-feedstock#71.
  # --strict-channel-priority keeps the freshly built local companion + binding authoritative.
  "$conda" create -y -n "$envName" -c "$bld" -c microsoft -c conda-forge --strict-channel-priority --override-channels "python=$py" mssql-python

conda/validate_conda_release.py:67

• read_index_json() uses next(...) to locate the info-*.tar.zst member inside a .conda. If the archive is malformed (missing that member), this will raise StopIteration and produce a stack trace rather than a clear validation failure message. Handle the empty case and raise a ValueError with a helpful message instead.

        with zipfile.ZipFile(path) as zf:
            info_name = next(
                n for n in zf.namelist() if n.startswith("info-") and n.endswith(".tar.zst")
            )
            info_blob = zf.read(info_name)

[Copilot]

🟡 Changes recommended

A few pipeline/script robustness issues (notably YAML parameter quoting and zstd decompression exception handling) should be addressed to avoid fragile release behavior and confusing failures.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

• Files reviewed: 28/29 changed files
• Comments generated: 3
• Review effort level: Lite

[Copilot]

🟡 Changes recommended

The publish pipeline hardening is inconsistent (compile-time interpolation remains for required/allowed subdir lists) and the driver-load probe contains avoidable insecure flags/suppression that should be corrected before shipping.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

conda/driver_load_probe.py:106

• TrustServerCertificate=yes is unnecessary when Encrypt=no, and keeping it (plus the DevSkim suppression) can be misleading in a repo script. Since this is a DB-less probe against an unreachable local port, the connection string can omit TrustServerCertificate entirely and avoid needing the suppression.

• Files reviewed: 28/29 changed files
• Comments generated: 1
• Review effort level: Lite

[Copilot]

🔵 Needs a closer look

It introduces extensive new release/publish pipeline logic and binary-audit tooling where small mismatches (e.g., parameter semantics and artifact handling) can have high release impact and should be validated by a human owner.

Review details

Suppressed comments (7)

Previously missed (6) — in code that hasn't changed since the last review.

OneBranchPipelines/steps/conda-release-step.yml:168

• This template passes --mssql-python-odbc-version through to conda/validate_conda_release.py, but that script explicitly ignores the flag (self-contained model). Keeping a no-op parameter here can give a false sense that the ODBC version is being enforced.

Either drop the parameter from this step, or change the validator to warn/fail if the flag is supplied so callers can’t accidentally rely on it.
conda/validate_conda_release.py:295

• --mssql-python-odbc-version is accepted but ignored. If a caller supplies it (and conda-release-step.yml currently will when odbcVersion is set), the pipeline can appear to be version-pinned when it isn’t.

At minimum, emit an explicit warning to stderr when this flag is provided so the logs clearly show it has no effect (or fail closed if you want to forbid it entirely).
eng/scripts/assert_macho_arch.py:159

• members = list(_iter_payload_members(path)) forces the entire conda payload (including file bytes) into memory before the loop. Since artifacts can be large and this audit only needs to stream Mach-O members, this can unnecessarily increase peak memory usage.

Consider making _conda_pkg.iter_payload_members() validate/raise before iteration (or adding a dedicated streaming wrapper) so callers can iterate without list(...) while preserving the current malformed-archive handling.
eng/scripts/assert_pe_machine.py:102

• members = list(_iter_payload_members(path)) loads the entire conda payload into memory up front. These packages can be large, and the PE audit only needs to stream over members; pre-materializing all bytes increases peak memory significantly.

Consider adjusting _conda_pkg.iter_payload_members() (or adding a wrapper) so malformed-archive errors are raised before iteration, allowing callers to iterate directly without building a list while keeping the current fail-closed behavior.
eng/scripts/audit_bundled_binaries.py:335

• members = list(_iter_payload_members(path)) fully materializes every payload file (and its bytes) before processing. For real conda artifacts this can be very large and may cause high memory usage/OOM in CI; the audit only needs to stream members and can short-circuit when possible.

Consider refactoring _conda_pkg.iter_payload_members() (or adding a new helper) so validation errors are raised before iteration and callers can safely iterate without building a list.
mssql_python/cursor.py:1850

• This change alters legacy execute()-path type inference (setinputsizes shorter than params) so money-range Decimals bind as SQL_NUMERIC instead of the MONEY/SMALLMONEY VARCHAR shortcut (GH-740). There isn’t a test exercising the setinputsizes-shorter-than-params + money-range Decimal case, so this regression guard is currently untested.

Add a pytest case that sets setinputsizes() for only the first parameter, passes a money-range Decimal as an uncovered later parameter, and asserts the operation does not overflow and returns the expected semantics against a NUMERIC/DECIMAL column.

conda/validate_conda_release.py:46

• _zstd_decompress() swallows all exceptions from the stdlib backend and then unconditionally imports zstandard. If neither backend is available, users get a raw ModuleNotFoundError without guidance.

Prefer catching ImportError specifically and raising a clear RuntimeError instructing to use Python 3.14+ (compression.zstd) or pip install zstandard (similar to eng/scripts/_conda_pkg.py).

def _zstd_decompress(raw: bytes) -> bytes:
    """Decompress a zstandard blob, preferring the 3.14+ stdlib backend."""
    try:  # Python 3.14+
        from compression import zstd  # type: ignore

• Files reviewed: 28/29 changed files
• Comments generated: 0 new
• Review effort level: Lite

[Copilot]

🟡 Changes recommended

There are at least two correctness/reproducibility issues to address (un-pinned conda-build channels and Win-ARM64 conda install documentation mismatch) before this can be safely approved.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

• Files reviewed: 24/25 changed files
• Comments generated: 2
• Review effort level: Lite

[Copilot]

🟡 Changes recommended

It contains a YAML indentation error that makes OneBranchPipelines/conda-release-pipeline.yml invalid and a broad exception catch in _zstd_decompress() that can mask real decompression failures.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (2)

Previously missed (1) — in code that hasn't changed since the last review.

README.md:63

• This README blurb says conda resolves the ODBC driver native dependencies (krb5, openssl, libltdl) so no system packages are required. However, the conda recipe explicitly declares openssl/krb5/libtool only for Linux and notes that macOS encrypted connections still require system OpenSSL. The README should scope this claim to Linux and mention the macOS OpenSSL requirement to avoid misleading conda users.

conda/validate_conda_release.py:46

• _zstd_decompress() currently catches a broad Exception, which will also swallow real decompression errors (e.g., corrupt .tar.zst) and then fall back to zstandard, obscuring the true failure. It also raises a bare ModuleNotFoundError if neither backend is available. Narrow the exception to ImportError and raise a clear RuntimeError when no backend is present (similar to eng/scripts/_conda_pkg.py).

def _zstd_decompress(raw: bytes) -> bytes:
    """Decompress a zstandard blob, preferring the 3.14+ stdlib backend."""
    try:  # Python 3.14+
        from compression import zstd  # type: ignore

• Files reviewed: 24/25 changed files
• Comments generated: 1
• Review effort level: Lite