Skip to content

fix: keep llama.cpp logs routed to the latest logger after recreating Llama - #690

Open
tomwangcn wants to merge 1 commit into
withcatai:masterfrom
tomwangcn:fix-addon-logger-replacement
Open

tomwangcn wants to merge 1 commit into
withcatai:masterfrom
tomwangcn:fix-addon-logger-replacement

Conversation

@tomwangcn

Copy link
Copy Markdown

Summary

Fixes #684.

Each new Llama() calls the addon's setLogger, which released the previous logger's thread-safe function before installing the new one. The old function's deferred finalizer then ran and cleared the global "JS logger callback set" flag, so from the second getLlama() call on, native llama.cpp log lines no longer reached any custom logger and were printed by the addon's fallback to stdout/stderr instead.

Changes

  • llama/addon/globals/addonLog.cpp: track the currently installed logger's context, install the new thread-safe function before releasing the previous one, and only clear the global callback flag when the finalized logger is still the current one. Single-instance behavior is unchanged.
  • test/modelDependent/llama/logger.test.ts: new regression test that loads a model through two sequential getLlama() instances with separate loggers and asserts both receive native llama.cpp log lines (llama_model_loader). Without the fix, the second logger only receives the few TypeScript-side lines, so the test fails.

Verification

  • Baseline (unpatched local build): first instance's logger receives 527 lines on model load; second instance's logger receives only 4 TypeScript-side lines while 44 native llama_model_loader lines leak to process stdout - matching the issue report.
  • Patched build: both loggers receive 527 lines including all native lines, zero leaked to stdout.
  • vitest run ./test/modelDependent/llama/logger.test.ts - passes (CPU build, NODE_LLAMA_CPP_GPU=false, bge-small-en-v1.5-q8_0.gguf).
  • tsc --noEmit --project tsconfig.json - clean
  • eslint on the changed files - clean

Tested on Linux x64 only; the change is platform-independent N-API ordering logic.

… `Llama`

Creating a new `Llama` instance released the previous logger's thread-safe
function before installing the new one. The old function's deferred finalizer
then cleared the global logger callback flag, so native llama.cpp log lines
stopped reaching the custom logger of the new instance and were printed to
the process stdout instead.

Install the new logger before releasing the previous one, and only clear the
callback flag when the finalized logger is still the current one.

Closes: withcatai#684

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: a custom logger stops receiving llama.cpp log lines after a second getLlama() call in the same process

1 participant