Fenrir fixes

[aidankeefe2022]

https://fenrir.wolfssl.com/finding/5020

https://fenrir.wolfssl.com/finding/5022

https://fenrir.wolfssl.com/finding/5019

https://fenrir.wolfssl.com/finding/5021

https://fenrir.wolfssl.com/finding/5018

https://fenrir.wolfssl.com/finding/5023

Added stdout configuration to rand when running on windows to make it behave like Posix

[wolfSSL-Fenrir-bot]

Fenrir Automated Review — PR #255

Scan targets checked: wolfclu-bugs, wolfclu-src

Findings: 1
1 finding(s) posted as inline comments (see file-level comments below)

This review was generated automatically by Fenrir. Findings are non-blocking.

[wolfSSL-Fenrir-bot]

Fenrir Automated Review — PR #255

Scan targets checked: wolfclu-bugs, wolfclu-src

Findings: 1
1 finding(s) posted as inline comments (see file-level comments below)

This review was generated automatically by Fenrir. Findings are non-blocking.

[wolfSSL-Fenrir-bot]

Fenrir Automated Review — PR #255

Scan targets checked: wolfclu-bugs, wolfclu-src

Findings: 1
1 finding(s) posted as inline comments (see file-level comments below)

This review was generated automatically by Fenrir. Findings are non-blocking.

[Copilot]

Pull request overview

This PR addresses multiple Fenrir findings by hardening CLI argument handling and file-length processing, adds regression tests for those cases, and fixes Windows rand stdout behavior to avoid output corruption.

Changes:

• Add regression tests for verify/dgst malformed-argument edge cases (missing trailing positional args) and a verify no-crash case.
• Switch Windows rand stdout to binary mode when emitting to stdout to prevent newline translation from corrupting output.
• Improve robustness around buffer sizing and OCSP index parsing (e.g., safer serial handling, larger size types, and negative-length checks).

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
tests/x509/x509-verify-test.py	Adds regression tests for verify malformed-argument and no-crash scenarios.
tests/dgst/dgst-test.py	Adds regression tests ensuring dgst detects missing trailing data file and doesn’t mis-flag valid invocations.
src/tools/clu_rand.c	Sets stdout to binary mode on Windows when writing to stdout to avoid data corruption.
src/sign-verify/clu_x509_verify.c	Adds malformed-argument detection before loading certs.
src/sign-verify/clu_verify.c	Uses long for file sizes and adds negative-size checks before allocating/reading.
src/sign-verify/clu_dgst_setup.c	Adjusts option parsing to avoid scanning the trailing positional and adds malformed-argument detection.
src/ocsp/clu_ocsp.c	Enlarges serial buffer and adds length validation + explicit NUL termination.
src/dh/clu_dh.c	Uses long for BIO length and casts for read/decode.
src/crypto/clu_evp_crypto.c	Uses long for BIO length and checks for negative length before base64 decode.

Comments suppressed due to low confidence (1)

src/dh/clu_dh.c:488

• wolfSSL_BIO_get_len() can return 0/negative for empty or unreadable input. With the current logic, that case silently skips parsing the input params and continues, which can lead to confusing downstream failures. Treat non-positive lengths as an error before proceeding.

        inSz = wolfSSL_BIO_get_len(bioIn);
        if (inSz > 0) {
            in = (byte*)XMALLOC(inSz, NULL, DYNAMIC_TYPE_TMP_BUFFER);
            if (in == NULL) {
                ret = WOLFCLU_FATAL_ERROR;

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.