From 0eb089dab71518c302b4ade21f67f704687500bf Mon Sep 17 00:00:00 2001 From: Kseniia Antonova Date: Wed, 11 Feb 2026 17:53:18 +0300 Subject: [PATCH 1/6] Add info about AUTH_DELAY_ERROR Fixes #5534 --- doc/book/admin/troubleshoot.rst | 80 ++++++++++++------ .../ru/LC_MESSAGES/book/admin/troubleshoot.po | 82 +++++++++++++++++++ 2 files changed, 138 insertions(+), 24 deletions(-) diff --git a/doc/book/admin/troubleshoot.rst b/doc/book/admin/troubleshoot.rst index a084f93ab5..b7b495d8d0 100644 --- a/doc/book/admin/troubleshoot.rst +++ b/doc/book/admin/troubleshoot.rst @@ -1,15 +1,14 @@ .. _admin-troubleshoot: .. _admin-troubleshooting-guide: -================================================================================ Troubleshooting guide -================================================================================ +===================== .. _admin-troubleshoot-memory-issues: --------------------------------------------------------------------------------- + Problem: INSERT/UPDATE-requests result in ER_MEMORY_ISSUE error --------------------------------------------------------------------------------- +--------------------------------------------------------------- **Possible reasons** @@ -63,9 +62,8 @@ Try either of the following measures: .. _admin-troubleshoot-cpu-load: --------------------------------------------------------------------------------- Problem: Tarantool generates too heavy CPU load --------------------------------------------------------------------------------- +----------------------------------------------- **Possible reasons** @@ -108,9 +106,8 @@ If the load is mostly generated by INSERT/UPDATE/DELETE requests, we recommend .. _admin-troubleshoot-query-timeout: --------------------------------------------------------------------------------- Problem: Query processing times out --------------------------------------------------------------------------------- +----------------------------------- **Possible reasons** @@ -165,9 +162,8 @@ Problem: Query processing times out .. _admin-troubleshoot-negative-lag-idle: --------------------------------------------------------------------------------- Problem: Replication "lag" and "idle" contain negative values --------------------------------------------------------------------------------- +------------------------------------------------------------- This is about ``box.info.replication.(upstream.)lag`` and ``box.info.replication.(upstream.)idle`` values in @@ -189,9 +185,8 @@ the local instance’s clock. .. _admin-troubleshoot-idle-grows-no-logs: --------------------------------------------------------------------------------- Problem: Replication "idle" keeps growing, but no related log messages appear --------------------------------------------------------------------------------- +----------------------------------------------------------------------------- This is about ``box.info.replication.(upstream.)idle`` value in :doc:`/reference/reference_lua/box_info/replication` section. @@ -211,9 +206,8 @@ the same replica UUID'``. .. _admin-troubleshoot-mr-odd-replication-stats: --------------------------------------------------------------------------------- Problem: Replication statistics differ on replicas within a replica set --------------------------------------------------------------------------------- +----------------------------------------------------------------------- This is about a replica set that consists of one master and several replicas. In a replica set of this type, values in @@ -231,9 +225,8 @@ Replication is broken. .. _admin-troubleshoot-mm-replication-stopped: --------------------------------------------------------------------------------- Problem: Master-master replication is stopped --------------------------------------------------------------------------------- +--------------------------------------------- This is about :doc:`box.info.replication(.upstream).status ` @@ -268,9 +261,8 @@ We also recommend using text primary keys or setting up .. _admin-troubleshoot-slow-tarantool: --------------------------------------------------------------------------------- Problem: Tarantool works much slower than before --------------------------------------------------------------------------------- +------------------------------------------------ **Possible reasons** @@ -308,15 +300,56 @@ recommend to optimize your Tarantool application code). If the value is greater than 0.01, your application definitely needs thorough code analysis aimed at optimizing memory usage. +.. _admin-troubleshoot-auth-delay: + +Problem: Adding a new replica set to a cluster results in ER_AUTH_DELAY error +----------------------------------------------------------------------------- + +There are instances in the cluster that are unable to connect to another node in the replica set due to exceeding +the number of authorization attempts. +On these instances, the ``Too many authentication attempts`` error is raised. + +**Possible reasons** + +1. Incorrect authentication credentials + + **Solution** + + In the cluster configuration, verify that the credentials the node is attempting to connect with are correct. + To do this, check the :ref:`replication ` parameter. + +2. Network issues + + **Solution** + + If you encounter network issues, restart the instance or re-add the replica set to the cluster. + +3. Tarantool instances are running on matching addresses + + **Solution** + + Identify the instance that other nodes in the replica set are unable to connect to. + Check the number of failed authorization attempts on the instance that was unable to connect to. + + .. code-block:: lua + + box.stat().AUTH + + If the number of failed attempts is increasing every second, check the list of nodes that are trying to authorize on this replica. + An increasing number of attempts may indicate there are some other Tarantool instances on the machine that have been + previously started on the same addresses. + In this case, the instance with the ``ER_AUTH_DELAY`` error and some old Tarantool nodes are both trying to + authorize on the same replica, and the first instance exceeds the authorization time limit on the replica. + + To resolve the problem, stop the old Tarantool instances and restart the replication. + .. _admin-troubleshoot-finalizer_yielding: --------------------------------------------------------------------------------- Problem: Fiber switch is forbidden in '__gc' metamethod --------------------------------------------------------------------------------- +------------------------------------------------------- -~~~~~~~~~~~~~~~~~~~~~~~~ Problem description -~~~~~~~~~~~~~~~~~~~~~~~~ +~~~~~~~~~~~~~~~~~~~ Fiber switch is forbidden in ``__gc`` metamethod since `this change `_ to avoid unexpected Lua OOM. @@ -325,9 +358,8 @@ for example, to close a socket. Below are examples of proper implementing such a procedure. -~~~~~~~~~~~~~~~~ Solution -~~~~~~~~~~~~~~~~ +~~~~~~~~ First, there come two simple examples illustrating the logic of the solution: diff --git a/locale/ru/LC_MESSAGES/book/admin/troubleshoot.po b/locale/ru/LC_MESSAGES/book/admin/troubleshoot.po index 384b943f27..3c2d6173dd 100644 --- a/locale/ru/LC_MESSAGES/book/admin/troubleshoot.po +++ b/locale/ru/LC_MESSAGES/book/admin/troubleshoot.po @@ -448,6 +448,88 @@ msgstr "" "Если значение больше 0,01, код приложения однозначно необходимо " "проанализировать на предмет оптимизации использования памяти." +msgid "" +"Problem: Adding a new replica set to a cluster results in " +"ER_AUTH_DELAY error" +msgstr "" +"Проблема: при добавлении нового набора реплик в кластер возникает ошибка " +"ER_AUTH_DELAY" + +msgid "" +"There are instances in the cluster that are unable to connect to another node " +"in the replica set due to exceeding the number of authorization attempts. " +"On these instances, the ``Too many authentication attempts`` error is raised." +msgstr "" +"В кластере есть экземпляры, которые могут подключиться к другому узлу " +"в наборе реплик из-за превышения числа попыток авторизации. " +"На таких узлах возникает ошибка ``Too many authentication attempts``." + +msgid "**Possible reasons**" +msgstr "**Возможные причины**" + +msgid "" +"Incorrect authentication credentials" +msgstr "" +"Некорректные данные аутентификации" + +msgid "Solution" +msgstr "Решение" + +msgid "" +"In the cluster configuration, verify that the credentials the node is attempting to connect with are correct. " +"To do this, check the :ref:`replication ` parameter." +msgstr "" +"Убедитесь, что в конфигурации кластера корректно указаны учетные данные, которые узел использует для подключения. " +"Для этого проверьте параметр :ref:`replication `." + +msgid "" +"Network issues" +msgstr "" +"Проблемы с сетью" + +msgid "Solution" +msgstr "Решение" + +msgid "" +"If you encounter network issues, restart the instance or re-add the replica set to the cluster." +msgstr "" +"При возникновении проблем с сетью перезапустите экземпляр или повторно добавьте набор реплик в кластер." + +msgid "" +"Tarantool instances are running on matching addresses" +msgstr "" +"Экземляры Tarantool запущены на совпадающих адресах" + +msgid "Solution" +msgstr "Решение" + +msgid "" +"Identify the instance that other nodes in the replica set are unable to connect to. " +"Check the number of failed authorization attempts on the instance that was unable to connect to." +msgstr "" +"Определите экземпляр, к которому не могут подключиться другие узлы в наборе реплик. " +"Проверьте, сколько было неудачных попыток авторизации на узле, к которому не удалось подключиться." + +msgid "" +"If the number of failed attempts is increasing every second, check the list of nodes " +"that are trying to authorize on this replica. " +"An increasing number of attempts may indicate there are some other Tarantool instances on the machine " +"that have been previously started on the same addresses. " +"In this case, the instance with the ``ER_AUTH_DELAY`` error and some old Tarantool nodes " +"are both trying to authorize on the same replica, and the first instance exceeds " +"the authorization time limit on the replica." +msgstr "" +"Если количество неудачных попыток растет с каждой секундой, проверьте список узлов, " +"которые пытаются авторизоваться на этой реплике. " +"Растущее число попыток может указывать на наличие на машине других экземпляров Tarantool, " +"которые ранее были запущены на тех же адресах. " +"В этом случае узел с ошибкой ``ER_AUTH_DELAY`` и некоторые старые узлы Tarantool " +"пытаются авторизоваться на одной и той же реплике, и узел с ошибкой превышает " +"лимит времени авторизации на реплике." + +msgid "To resolve the problem, stop the old Tarantool instances and restart the replication." +msgstr "Чтобы решить возникшую проблему, остановите старые экземпляры Tarantool и перезапустите репликацию." + msgid "Problem: Fiber switch is forbidden in '__gc' metamethod" msgstr "Проблема: Переключатель файберов запрещен в метаметоде ``__gc``" From 16ad40fc3fdd7419cd28a58ec4edad220abe4d3d Mon Sep 17 00:00:00 2001 From: Kseniia Antonova Date: Thu, 20 Aug 2026 19:23:13 +0300 Subject: [PATCH 2/6] Update errcodes --- doc/reference/reference_lua/errcodes.rst | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/doc/reference/reference_lua/errcodes.rst b/doc/reference/reference_lua/errcodes.rst index a7539cc4b9..9da132b11c 100644 --- a/doc/reference/reference_lua/errcodes.rst +++ b/doc/reference/reference_lua/errcodes.rst @@ -28,6 +28,10 @@ You can find a complete list of errors in the - box.error.NONMASTER - (In replication) A server instance cannot modify data unless it is a master. + * - ER_AUTH_DELAY + - box.error.AUTH_DELAY + - Too many authentication attempts. + * - ER_ILLEGAL_PARAMS - box.error.ILLEGAL_PARAMS - Illegal parameters. Malformed protocol message. From 09075bef6030c1d76094cbfa73dd9a7e8e327a67 Mon Sep 17 00:00:00 2001 From: Kseniia Antonova Date: Thu, 20 Aug 2026 19:32:21 +0300 Subject: [PATCH 3/6] Update text --- doc/book/admin/troubleshoot.rst | 6 ++++-- .../ru/LC_MESSAGES/book/admin/troubleshoot.po | 17 +++++++++++------ 2 files changed, 15 insertions(+), 8 deletions(-) diff --git a/doc/book/admin/troubleshoot.rst b/doc/book/admin/troubleshoot.rst index b7b495d8d0..f5e63fb3f9 100644 --- a/doc/book/admin/troubleshoot.rst +++ b/doc/book/admin/troubleshoot.rst @@ -329,13 +329,15 @@ On these instances, the ``Too many authentication attempts`` error is raised. **Solution** Identify the instance that other nodes in the replica set are unable to connect to. - Check the number of failed authorization attempts on the instance that was unable to connect to. + To do this, check the instance logs for authentication errors. + + On that instance, check the number of authentication attempts using ``box.stat().AUTH``: .. code-block:: lua box.stat().AUTH - If the number of failed attempts is increasing every second, check the list of nodes that are trying to authorize on this replica. + If the number of attempts is increasing every second, check the list of nodes that are trying to authorize on this replica. An increasing number of attempts may indicate there are some other Tarantool instances on the machine that have been previously started on the same addresses. In this case, the instance with the ``ER_AUTH_DELAY`` error and some old Tarantool nodes are both trying to diff --git a/locale/ru/LC_MESSAGES/book/admin/troubleshoot.po b/locale/ru/LC_MESSAGES/book/admin/troubleshoot.po index 3c2d6173dd..5b704b6fb8 100644 --- a/locale/ru/LC_MESSAGES/book/admin/troubleshoot.po +++ b/locale/ru/LC_MESSAGES/book/admin/troubleshoot.po @@ -460,7 +460,7 @@ msgid "" "in the replica set due to exceeding the number of authorization attempts. " "On these instances, the ``Too many authentication attempts`` error is raised." msgstr "" -"В кластере есть экземпляры, которые могут подключиться к другому узлу " +"В кластере есть экземпляры, которые не могут подключиться к другому узлу " "в наборе реплик из-за превышения числа попыток авторизации. " "На таких узлах возникает ошибка ``Too many authentication attempts``." @@ -498,20 +498,25 @@ msgstr "" msgid "" "Tarantool instances are running on matching addresses" msgstr "" -"Экземляры Tarantool запущены на совпадающих адресах" +"Экземпляры Tarantool запущены на совпадающих адресах" msgid "Solution" msgstr "Решение" msgid "" "Identify the instance that other nodes in the replica set are unable to connect to. " -"Check the number of failed authorization attempts on the instance that was unable to connect to." +"To do this, check the instance logs for authentication errors." msgstr "" "Определите экземпляр, к которому не могут подключиться другие узлы в наборе реплик. " -"Проверьте, сколько было неудачных попыток авторизации на узле, к которому не удалось подключиться." +"Для этого проверьте логи экземпляра на наличие ошибок аутентификации." msgid "" -"If the number of failed attempts is increasing every second, check the list of nodes " +"On that instance, check the number of authentication attempts using ``box.stat().AUTH``:" +msgstr "" +"На этом экземпляре проверьте число попыток аутентификации с помощью ``box.stat().AUTH``:" + +msgid "" +"If the number of attempts is increasing every second, check the list of nodes " "that are trying to authorize on this replica. " "An increasing number of attempts may indicate there are some other Tarantool instances on the machine " "that have been previously started on the same addresses. " @@ -519,7 +524,7 @@ msgid "" "are both trying to authorize on the same replica, and the first instance exceeds " "the authorization time limit on the replica." msgstr "" -"Если количество неудачных попыток растет с каждой секундой, проверьте список узлов, " +"Если количество попыток растет с каждой секундой, проверьте список узлов, " "которые пытаются авторизоваться на этой реплике. " "Растущее число попыток может указывать на наличие на машине других экземпляров Tarantool, " "которые ранее были запущены на тех же адресах. " From 6b8456d7ea23eee882257fcb249a80efd3b7ba4f Mon Sep 17 00:00:00 2001 From: Kseniia Antonova Date: Fri, 21 Aug 2026 17:04:50 +0300 Subject: [PATCH 4/6] Apply review suggestions --- doc/book/admin/troubleshoot.rst | 6 +++--- doc/reference/reference_lua/errcodes.rst | 2 +- locale/ru/LC_MESSAGES/book/admin/troubleshoot.po | 8 ++++---- 3 files changed, 8 insertions(+), 8 deletions(-) diff --git a/doc/book/admin/troubleshoot.rst b/doc/book/admin/troubleshoot.rst index f5e63fb3f9..12878dc490 100644 --- a/doc/book/admin/troubleshoot.rst +++ b/doc/book/admin/troubleshoot.rst @@ -307,7 +307,7 @@ Problem: Adding a new replica set to a cluster results in ER_AUTH_DELAY error There are instances in the cluster that are unable to connect to another node in the replica set due to exceeding the number of authorization attempts. -On these instances, the ``Too many authentication attempts`` error is raised. +On these instances, the ``Too many failed authentication attempts`` error is raised. **Possible reasons** @@ -338,8 +338,8 @@ On these instances, the ``Too many authentication attempts`` error is raised. box.stat().AUTH If the number of attempts is increasing every second, check the list of nodes that are trying to authorize on this replica. - An increasing number of attempts may indicate there are some other Tarantool instances on the machine that have been - previously started on the same addresses. + An increasing number of attempts may indicate there are some other Tarantool instances on the machine that are trying + to connect to the same replica address with wrong or outdated credentials. In this case, the instance with the ``ER_AUTH_DELAY`` error and some old Tarantool nodes are both trying to authorize on the same replica, and the first instance exceeds the authorization time limit on the replica. diff --git a/doc/reference/reference_lua/errcodes.rst b/doc/reference/reference_lua/errcodes.rst index 9da132b11c..28abaab814 100644 --- a/doc/reference/reference_lua/errcodes.rst +++ b/doc/reference/reference_lua/errcodes.rst @@ -30,7 +30,7 @@ You can find a complete list of errors in the * - ER_AUTH_DELAY - box.error.AUTH_DELAY - - Too many authentication attempts. + - Too many failed authentication attempts. * - ER_ILLEGAL_PARAMS - box.error.ILLEGAL_PARAMS diff --git a/locale/ru/LC_MESSAGES/book/admin/troubleshoot.po b/locale/ru/LC_MESSAGES/book/admin/troubleshoot.po index 5b704b6fb8..b3a463dd9d 100644 --- a/locale/ru/LC_MESSAGES/book/admin/troubleshoot.po +++ b/locale/ru/LC_MESSAGES/book/admin/troubleshoot.po @@ -458,11 +458,11 @@ msgstr "" msgid "" "There are instances in the cluster that are unable to connect to another node " "in the replica set due to exceeding the number of authorization attempts. " -"On these instances, the ``Too many authentication attempts`` error is raised." +"On these instances, the ``Too many failed authentication attempts`` error is raised." msgstr "" "В кластере есть экземпляры, которые не могут подключиться к другому узлу " "в наборе реплик из-за превышения числа попыток авторизации. " -"На таких узлах возникает ошибка ``Too many authentication attempts``." +"На таких узлах возникает ошибка ``Too many failed authentication attempts``." msgid "**Possible reasons**" msgstr "**Возможные причины**" @@ -519,7 +519,7 @@ msgid "" "If the number of attempts is increasing every second, check the list of nodes " "that are trying to authorize on this replica. " "An increasing number of attempts may indicate there are some other Tarantool instances on the machine " -"that have been previously started on the same addresses. " +"that are trying to connect to the same replica address with wrong or outdated credentials. " "In this case, the instance with the ``ER_AUTH_DELAY`` error and some old Tarantool nodes " "are both trying to authorize on the same replica, and the first instance exceeds " "the authorization time limit on the replica." @@ -527,7 +527,7 @@ msgstr "" "Если количество попыток растет с каждой секундой, проверьте список узлов, " "которые пытаются авторизоваться на этой реплике. " "Растущее число попыток может указывать на наличие на машине других экземпляров Tarantool, " -"которые ранее были запущены на тех же адресах. " +"которые пытаются подключиться к этому же адресу реплики с некорректными или устаревшими учетными данными. " "В этом случае узел с ошибкой ``ER_AUTH_DELAY`` и некоторые старые узлы Tarantool " "пытаются авторизоваться на одной и той же реплике, и узел с ошибкой превышает " "лимит времени авторизации на реплике." From 36cf02a22673a1e3f7b49bff7a19d9e91918e954 Mon Sep 17 00:00:00 2001 From: Kseniia Antonova Date: Fri, 21 Aug 2026 20:35:59 +0300 Subject: [PATCH 5/6] Apply review suggestions --- doc/book/admin/troubleshoot.rst | 8 +------- locale/ru/LC_MESSAGES/book/admin/troubleshoot.po | 13 ------------- 2 files changed, 1 insertion(+), 20 deletions(-) diff --git a/doc/book/admin/troubleshoot.rst b/doc/book/admin/troubleshoot.rst index 12878dc490..61d95a0ab5 100644 --- a/doc/book/admin/troubleshoot.rst +++ b/doc/book/admin/troubleshoot.rst @@ -318,13 +318,7 @@ On these instances, the ``Too many failed authentication attempts`` error is rai In the cluster configuration, verify that the credentials the node is attempting to connect with are correct. To do this, check the :ref:`replication ` parameter. -2. Network issues - - **Solution** - - If you encounter network issues, restart the instance or re-add the replica set to the cluster. - -3. Tarantool instances are running on matching addresses +2. Tarantool instances are running on matching addresses **Solution** diff --git a/locale/ru/LC_MESSAGES/book/admin/troubleshoot.po b/locale/ru/LC_MESSAGES/book/admin/troubleshoot.po index b3a463dd9d..ed6c2af6e4 100644 --- a/locale/ru/LC_MESSAGES/book/admin/troubleshoot.po +++ b/locale/ru/LC_MESSAGES/book/admin/troubleshoot.po @@ -482,19 +482,6 @@ msgstr "" "Убедитесь, что в конфигурации кластера корректно указаны учетные данные, которые узел использует для подключения. " "Для этого проверьте параметр :ref:`replication `." -msgid "" -"Network issues" -msgstr "" -"Проблемы с сетью" - -msgid "Solution" -msgstr "Решение" - -msgid "" -"If you encounter network issues, restart the instance or re-add the replica set to the cluster." -msgstr "" -"При возникновении проблем с сетью перезапустите экземпляр или повторно добавьте набор реплик в кластер." - msgid "" "Tarantool instances are running on matching addresses" msgstr "" From d595805aa80a9fe79ef8a9eba7458d1a97866681 Mon Sep 17 00:00:00 2001 From: Kseniia Antonova Date: Fri, 21 Aug 2026 20:37:05 +0300 Subject: [PATCH 6/6] Apply review suggestions --- doc/book/admin/troubleshoot.rst | 2 +- locale/ru/LC_MESSAGES/book/admin/troubleshoot.po | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/doc/book/admin/troubleshoot.rst b/doc/book/admin/troubleshoot.rst index 61d95a0ab5..edab7d6317 100644 --- a/doc/book/admin/troubleshoot.rst +++ b/doc/book/admin/troubleshoot.rst @@ -332,7 +332,7 @@ On these instances, the ``Too many failed authentication attempts`` error is rai box.stat().AUTH If the number of attempts is increasing every second, check the list of nodes that are trying to authorize on this replica. - An increasing number of attempts may indicate there are some other Tarantool instances on the machine that are trying + An increasing number of attempts may indicate there are some other Tarantool instances that are trying to connect to the same replica address with wrong or outdated credentials. In this case, the instance with the ``ER_AUTH_DELAY`` error and some old Tarantool nodes are both trying to authorize on the same replica, and the first instance exceeds the authorization time limit on the replica. diff --git a/locale/ru/LC_MESSAGES/book/admin/troubleshoot.po b/locale/ru/LC_MESSAGES/book/admin/troubleshoot.po index ed6c2af6e4..6c08cf823f 100644 --- a/locale/ru/LC_MESSAGES/book/admin/troubleshoot.po +++ b/locale/ru/LC_MESSAGES/book/admin/troubleshoot.po @@ -505,7 +505,7 @@ msgstr "" msgid "" "If the number of attempts is increasing every second, check the list of nodes " "that are trying to authorize on this replica. " -"An increasing number of attempts may indicate there are some other Tarantool instances on the machine " +"An increasing number of attempts may indicate there are some other Tarantool instances " "that are trying to connect to the same replica address with wrong or outdated credentials. " "In this case, the instance with the ``ER_AUTH_DELAY`` error and some old Tarantool nodes " "are both trying to authorize on the same replica, and the first instance exceeds " @@ -513,7 +513,7 @@ msgid "" msgstr "" "Если количество попыток растет с каждой секундой, проверьте список узлов, " "которые пытаются авторизоваться на этой реплике. " -"Растущее число попыток может указывать на наличие на машине других экземпляров Tarantool, " +"Растущее число попыток может указывать на наличие других экземпляров Tarantool, " "которые пытаются подключиться к этому же адресу реплики с некорректными или устаревшими учетными данными. " "В этом случае узел с ошибкой ``ER_AUTH_DELAY`` и некоторые старые узлы Tarantool " "пытаются авторизоваться на одной и той же реплике, и узел с ошибкой превышает "