Report each warning once per channel; stop mutating global warning state (#362-#364)

docs/source/pcapkit/utilities/exceptions.rst

@@ -3,15 +3,47 @@ User Defined Exceptions
 
 .. module:: pcapkit.utilities.exceptions
 
-:mod:`pcapkit.exceptions` refined built-in exceptions.
-Make it possible to show only user error stack infomation [*]_,
+:mod:`pcapkit.utilities.exceptions` refined built-in exceptions.
+Make it possible to show only user error stack information [*]_,
 when exception raised on user's operation.
 
+Loud and Quiet Errors
+---------------------
+
+Raising a :class:`~pcapkit.utilities.exceptions.BaseError` is, by default, a
+**loud** act: the error is logged once at :data:`logging.CRITICAL` on the
+:data:`~pcapkit.utilities.logging.logger` logger, and outside development mode
+:data:`sys.tracebacklimit` is set to ``0``, which suppresses the traceback frames
+entirely so the user sees the exception line rather than a walk through
+:mod:`pcapkit`'s internals.
+
+``quiet=True`` marks an error that :mod:`pcapkit` raises as **internal control
+flow** and expects to catch itself -- the
+:exc:`~pcapkit.utilities.exceptions.MissingKeyError` behind
+:meth:`MultiDict.get <pcapkit.corekit.multidict.MultiDict.get>` is the archetype.
+Such an error emits nothing on any channel and touches no process-global state.
+It is still an ordinary exception carrying its message, so ``except`` clauses and
+:func:`repr` are unaffected.
+
+.. attention::
+
+   Up to and including v1.4.1, ``quiet=True`` meant "log at ``ERROR`` instead of
+   ``CRITICAL``" rather than "do not log", and :data:`sys.tracebacklimit` was set
+   on both paths. A single ``MultiDict.get()`` miss therefore produced an
+   ``ERROR`` record -- one per frame when parsing a capture containing
+   unfragmented IPv6 with reassembly enabled -- and truncated the tracebacks of
+   unrelated exceptions for the remainder of the process. A consumer who was
+   watching for those ``ERROR`` records will no longer see them; they never
+   corresponded to a fault. Anything that genuinely wants to observe internal
+   lookup misses should catch the exception rather than read the log.
+
 .. autoexception:: pcapkit.utilities.exceptions.BaseError
    :no-members:
    :show-inheritance:
 
-   :param quiet: If :data:`True`, suppress exception message.
+   :param quiet: If :data:`True`, the error is neither logged nor allowed to
+      alter :data:`sys.tracebacklimit`; it is raised silently, as internal
+      control flow.
    :param \*args: Arbitrary positional arguments.
    :param \*\*kwargs: Arbitrary keyword arguments.
 

docs/source/pcapkit/utilities/logging.rst

@@ -191,10 +191,9 @@ Compatibility Note
 
 .. note::
 
-   Two related issues are deliberately left alone for now:
-   :func:`pcapkit.utilities.warnings.warn` reports every warning twice, once
-   through :mod:`logging` and once through :mod:`warnings`; and
-   :class:`~pcapkit.utilities.warnings.BaseWarning` calls
-   :func:`warnings.simplefilter` outside development mode, which mutates the
-   process-wide warning filters. Both change observable behaviour well beyond
-   logging and belong in their own change.
+   The warning channel is documented separately, in
+   :doc:`warnings`. In short:
+   :func:`pcapkit.utilities.warnings.warn` reports each warning exactly once per
+   channel -- one :data:`logging.WARNING` record and one :func:`warnings.warn`
+   -- and constructing a warning no longer mutates the process-wide warning
+   filters, so suppression is the application's to configure.

docs/source/pcapkit/utilities/warnings.rst

@@ -3,14 +3,88 @@ User Defined Warnings
 
 .. module:: pcapkit.utilities.warnings
 
-:mod:`pcapkit.warnings` refined built-in warnings.
+:mod:`pcapkit.utilities.warnings` refined built-in warnings.
+
+How a Warning Is Reported
+-------------------------
+
+Every warning :mod:`pcapkit` reports goes through
+:func:`~pcapkit.utilities.warnings.warn`, which reports it **exactly once on each
+of two channels**:
+
+1. the :data:`~pcapkit.utilities.logging.logger` logger, at
+   :data:`logging.WARNING` level, unconditionally -- so a consumer watching the
+   ``pcapkit`` logger sees every complaint whatever the warning filters say;
+2. the standard :mod:`warnings` machinery, via :func:`warnings.warn`, subject to
+   the filters -- so :func:`warnings.filterwarnings`,
+   :func:`warnings.catch_warnings`, :mod:`pytest`'s ``filterwarnings``,
+   :option:`-W` and :envvar:`PYTHONWARNINGS` govern :mod:`pcapkit` warnings
+   exactly as they govern any other library's (with one wrinkle in naming a
+   category on the command line, noted below).
+
+The counts are the same in development mode as outside it:
+:envvar:`PCAPKIT_DEVMODE` and :envvar:`PCAPKIT_VERBOSE` change how much detail
+the log record carries, never how many records there are. Constructing a warning
+class is not an act of reporting one -- it emits nothing and has no side effects.
+
+Silencing pcapkit Warnings
+--------------------------
+
+Filter them like any other library's, at the level of granularity you want --
+:class:`~pcapkit.utilities.warnings.BaseWarning` for all of them, or a single
+category::
+
+   import warnings
+
+   from pcapkit.utilities.warnings import BaseWarning, SchemaWarning
+
+   warnings.filterwarnings('ignore', category=BaseWarning)    # all of them
+   warnings.filterwarnings('error', category=SchemaWarning)   # or turn one into an error
+
+From the command line, name one of the standard categories these are mixed with.
+:exc:`UserWarning` covers all of them, and :exc:`RuntimeWarning`,
+:exc:`ImportWarning`, :exc:`ResourceWarning` and :exc:`DeprecationWarning` each
+select a family::
+
+   python -W ignore::UserWarning ...      # every pcapkit warning, and everyone else's
+   python -W error::RuntimeWarning ...    # the RuntimeWarning family, as errors
+
+.. note::
+
+   :option:`-W` and :envvar:`PYTHONWARNINGS` cannot name a :mod:`pcapkit`
+   category directly: ``-W ignore::pcapkit.utilities.warnings.BaseWarning`` is
+   rejected with ``Invalid -W option ignored: invalid module name``. CPython
+   imports the category while parsing the option, and that happens before
+   :mod:`site` has added ``site-packages`` to :data:`sys.path`, so no installed
+   package's own category can be named there -- this is not specific to
+   :mod:`pcapkit`. Use a standard category on the command line, or install a
+   precise filter in code as above.
+
+All of that governs the :mod:`warnings` channel. The ``pcapkit`` logger is
+configured separately, through the :mod:`logging` module::
+
+   import logging
+
+   logging.getLogger('pcapkit').setLevel(logging.ERROR)   # drop WARNING records
+
+.. attention::
+
+   Up to and including v1.4.1, :class:`~pcapkit.utilities.warnings.BaseWarning`
+   installed an ``ignore`` filter for its own class as a side effect of being
+   constructed, so :mod:`pcapkit` warnings were invisible on the :mod:`warnings`
+   channel by default and a caller could not re-enable them. They are now
+   delivered, which means a consumer who was relying on that silence will start
+   seeing them; the first snippet above restores the old quiet. Since the filter
+   was installed at the *front* of the process-global :data:`warnings.filters`,
+   it also overrode the host application's own configuration for those
+   categories and made unrelated warnings re-fire, so it is not something that
+   can be kept.
 
 .. autoexception:: pcapkit.utilities.warnings.BaseWarning
    :no-members:
    :show-inheritance:
 
    :param \*args: Arbitrary positional arguments.
-   :param \*\*kwargs: Arbitrary keyword arguments.
 
 :exc:`ImportWarning` Category
 -----------------------------

pcapkit/utilities/exceptions.py

@@ -86,39 +86,43 @@ def stacklevel() -> 'int':
 class BaseError(Exception):
     """Base error class of all kinds.
 
-    Important:
-
-        * Turn off system-default traceback function by set :data:`sys.tracebacklimit` to ``0``.
-        * But bugs appear in Python 3.6, so we have to set :data:`sys.tracebacklimit` to ``None``.
-
-          .. note::
+    A loud error -- the default -- is reported once, at
+    :data:`logging.CRITICAL` level, on the
+    :data:`~pcapkit.utilities.logging.logger` logger. Outside development mode it
+    also sets :data:`sys.tracebacklimit` to ``0``, which suppresses the traceback
+    frames entirely, so a user sees the exception line rather than a walk through
+    :mod:`pcapkit`'s internals.
+
+    A **quiet** error (``quiet=True``) is one :mod:`pcapkit` raises as internal
+    control flow and expects to catch itself, such as the
+    :exc:`~pcapkit.utilities.exceptions.MissingKeyError` behind
+    :meth:`MultiDict.get <pcapkit.corekit.multidict.MultiDict.get>`. It is
+    therefore silent and free of side effects: nothing is logged, and
+    :data:`sys.tracebacklimit` is left alone. It is still a perfectly ordinary
+    exception, carrying its message for whoever catches it.
 
-             This note is deprecated since Python fixed the problem above.
+    Important:
 
-        * In Python 2.7, :func:`trace.print_stack(limit)` dose not support negative limit.
+        * :data:`sys.tracebacklimit` is process-global, so it is only set for a
+          loud error -- a quiet one used as control flow must not truncate the
+          tracebacks of unrelated exceptions for the rest of the process.
+        * In Python 2.7, :func:`trace.print_stack(limit)` does not support negative limit.
 
     See Also:
         :func:`pcapkit.utilities.exceptions.stacklevel`
 
     """
 
     def __init__(self, *args: 'Any', quiet: 'bool' = False, **kwargs: 'Any') -> 'None':
-        # log error
+        # log error -- a quiet error emits nothing and mutates nothing
         if not quiet:
             if DEVMODE:
                 logger.critical('%s: %s', type(self).__name__, str(self),
                                 exc_info=self if VERBOSE else False,
                                 stack_info=VERBOSE, stacklevel=-stacklevel())
             else:
-                logger.critical("%s: %s", type(self).__name__, str(self))
-
-            # logger.error('%s: %s', type(self).__name__, str(self), exc_info=self,
-            #              stack_info=True, stacklevel=-stacklevel())
-        else:
-            logger.error('%s: %s', type(self).__name__, str(self))
-
-        if not DEVMODE:
-            sys.tracebacklimit = 0
+                logger.critical('%s: %s', type(self).__name__, str(self))
+                sys.tracebacklimit = 0
         super().__init__(*args, **kwargs)
 
 

pcapkit/utilities/warnings.py

@@ -4,17 +4,62 @@
 
 .. module:: pcapkit.utilities.warnings
 
-:mod:`pcapkit.warnings` refined built-in warnings.
+:mod:`pcapkit.utilities.warnings` refined built-in warnings.
+
+Every warning :mod:`pcapkit` reports goes through :func:`warn`, which reports it
+**exactly once on each of two channels**:
+
+1. the :data:`~pcapkit.utilities.logging.logger` logger, at
+   :data:`logging.WARNING` level, unconditionally -- so a consumer watching the
+   ``pcapkit`` logger sees every complaint whatever the warning filters say;
+2. the standard :mod:`warnings` machinery, via :func:`warnings.warn`, subject to
+   the filters -- so :func:`warnings.filterwarnings`,
+   :func:`warnings.catch_warnings`, :mod:`pytest`'s ``filterwarnings``,
+   :option:`-W` and :envvar:`PYTHONWARNINGS` govern :mod:`pcapkit` warnings
+   exactly as they govern any other library's (see below for the one wrinkle in
+   naming a category on the command line).
+
+The counts are the same in development mode as outside it;
+:data:`~pcapkit.utilities.logging.DEVMODE` and
+:data:`~pcapkit.utilities.logging.VERBOSE` change how much detail the log record
+carries, never how many records there are. Constructing a warning class is not
+an act of reporting one, and has no side effects at all.
+
+To silence :mod:`pcapkit` warnings, filter them like any other category::
+
+    import warnings
+
+    from pcapkit.utilities.warnings import BaseWarning
+
+    warnings.filterwarnings('ignore', category=BaseWarning)
+
+From the command line, name one of the standard categories these are mixed with
+-- :exc:`UserWarning` covers all of them, and :exc:`RuntimeWarning`,
+:exc:`ImportWarning`, :exc:`ResourceWarning` and :exc:`DeprecationWarning` each
+select a family::
+
+    python -W ignore::UserWarning ...
+
+:option:`-W` and :envvar:`PYTHONWARNINGS` cannot name a :mod:`pcapkit` category
+directly -- ``-W ignore::pcapkit.utilities.warnings.BaseWarning`` is rejected
+with ``Invalid -W option ignored: invalid module name``. CPython imports the
+category while parsing the option, which happens before :mod:`site` has added
+``site-packages`` to :data:`sys.path`, so no installed package's own category can
+be named there; this is not specific to :mod:`pcapkit`.
+
+Note that all of the above governs the :mod:`warnings` channel only. The
+``pcapkit`` logger is configured separately, e.g. with
+``logging.getLogger('pcapkit').setLevel(logging.ERROR)``.
 
 """
 import warnings
 from typing import TYPE_CHECKING
 
 from pcapkit.utilities.exceptions import stacklevel as stacklevel_calculator
-from pcapkit.utilities.logging import DEVMODE, VERBOSE, get_logger
+from pcapkit.utilities.logging import VERBOSE, get_logger
 
 if TYPE_CHECKING:
-    from typing import Any, Optional, Type, Union
+    from typing import Optional, Type, Union
 
 __all__ = [
     'warn',
@@ -45,11 +90,23 @@ def warn(message: 'Union[str, Warning]', category: 'Type[Warning]',
          stacklevel: 'Optional[int]' = None) -> 'None':
     """Wrapper function of :func:`warnings.warn`.
 
+    The warning is reported once on the :data:`~pcapkit.utilities.logging.logger`
+    logger, then once through :func:`warnings.warn`. The logger call does not
+    consult :data:`warnings.filters`, so a log-based consumer sees the complaint
+    even when the application has filtered the category out; the
+    :func:`warnings.warn` call is filtered normally, so the application keeps
+    full control of that channel -- including turning the warning into an error
+    with :option:`-W error <-W>`.
+
     Args:
         message: Warning message.
         category: Warning category.
         stacklevel: Warning stack level.
 
+    See Also:
+        :mod:`pcapkit.utilities.warnings` for the emission model in full, and for
+        how to silence either channel.
+
     """
     if stacklevel is None:
         stacklevel = stacklevel_calculator()
@@ -65,21 +122,26 @@ def warn(message: 'Union[str, Warning]', category: 'Type[Warning]',
 
 
 class BaseWarning(UserWarning):
-    """Base warning class of all kinds."""
-
-    def __init__(self, *args: 'Any', **kwargs: 'Any') -> 'None':  # pylint: disable=useless-super-delegation
-        # log warning
-        if DEVMODE:
-            if VERBOSE:
-                logger.warning(str(self), exc_info=self, stack_info=True,
-                            stacklevel=stacklevel_calculator())
-            else:
-                logger.warning(str(self))
-        else:
-            warnings.simplefilter('ignore', type(self))
-
-        # warnings.simplefilter('default')
-        super().__init__(*args, **kwargs)
+    """Base warning class of all kinds.
+
+    Constructing one of these is deliberately free of side effects: it emits
+    nothing and mutates no process-global state. Reporting a warning is the job
+    of :func:`warn`, which is called once per complaint; a warning object may be
+    built without ever being reported, and the two must not be confused.
+
+    Note:
+        Up to and including v1.4.1, this constructor called
+        ``warnings.simplefilter('ignore', type(self))`` outside development
+        mode, which inserted an entry at the front of the process-global
+        :data:`warnings.filters` -- overriding whatever the host application,
+        :option:`-W` or the test runner had configured, and invalidating every
+        module's ``__warningregistry__`` so that unrelated warnings re-fired.
+        It also logged the warning a second time under
+        :data:`~pcapkit.utilities.logging.DEVMODE`. Both are gone; filtering is
+        the application's business, and is done through the standard
+        :mod:`warnings` machinery.
+
+    """
 
 
 ##############################################################################