Tree Trunk

docs/changes.rst

@@ -1,5 +1,12 @@
-1.2.3 (current, released 2026-9-08)
+1.2.4 (current, released 2026-9-24)
 -----------------------------------
+    * adding new tests for localtree and retry helpers.
+    * adding key_type parameter to get_hostkey to pull specific key.
+    * change hostkey verification comparison to pass bytes to hash function.
+    * fix remote path nesting and duplicate sub-directory names in localtree.
+
+1.2.3 (released 2026-9-08)
+--------------------------
     * adding new tests for connections, hash, drivepath and read-only servers.
     * reworking tests to incorporate self-cleaning of all artifacts.
     * fix for error handling catches in _set_authentication function.  

docs/conf.py

@@ -54,9 +54,9 @@
 # built documents.
 #
 # The short X.Y version.
-version = '1.2.3'
+version = '1.2.4'
 # The full version, including alpha/beta/rc tags.
-release = '1.2.3'
+release = '1.2.4'
 
 # The language for content autogenerated by Sphinx. Refer to documentation
 # for a list of supported languages.

docs/contributing.rst

@@ -21,7 +21,7 @@ Code
 
         a. Setup CI testing for your fork. Currently testing is done on Github Actions but feel free to use the framework of your choosing.
         b. Testing features that concern chmod, chown on Windows is NOT supported. Testing compression has to be ran against a local compatible sshd and not the pytest-sftpserver plugin as it does NOT support this feature.
-        c. You will need to setup an ssh daemon on your local machine and create a user: copy the contents of id_sftpretty.pub to the newly created user's authorized_keys file -- Tests that can only be ran locally are skipped using the @skip_if_ci decorator so they don't fail when the test suite runs on the CI server.
+        c. You will need to setup an ssh daemon on your local machine and create a user. Copy the contents of id_sftpretty.pub to the newly created user's authorized_keys file. Tests that can only be ran locally are skipped using the @SKIP_IF_CI decorator so they don't fail when the test suite runs on the CI server.
 
     #. Ensure that your name is added to the end of the :doc:`authors` file using the format Name <email@domain.com> (url), where the (url) portion is optional.
     #. Submit a Pull Request to the project.
@@ -36,9 +36,9 @@ This section lists the priority that will be assigned to an issue:
     #. Developer Issues
     #. Issues that have a pull request with a test(s) displaying the issue and code change(s) that satisfies the test suite
     #. Issues that have a pull request with a test(s) displaying the issue
-    #. Naked pull requests - a code change request with no accompaning test
+    #. Naked pull requests. A code change request with no accompaning test
     #. An issue without a pull request with a test displaying the issue
-    #. Badly documented issue with no code or test - sftpretty is not an end-user tool, it is a developer tool and it is expected that issues will be submitted like a developer and not an end-user. Issues in the realm of "the internet is broken" will be marked as invalid with a comment pointing the submitter to this section.
+    #. Badly documented issue with no code or test. sftpretty is not an end-user tool, it is a developer tool and it is expected that issues will be submitted like a developer and not an end-user. Issues in the realm of "the internet is broken" will be marked as invalid with a comment pointing the submitter to this section.
 
 Testing
 -------

docs/cookbook.rst

@@ -532,37 +532,145 @@ Don't like how we have modified a paramiko method? Use this attribute to get
 at the original version. Our goal is to augment not supplant paramiko.
 
 
-:func:`sftpretty.localtree`
----------------------------
+:func:`sftpretty.helpers._callback`
+-----------------------------------
+A progress reporter implementation :meth:`sftpretty.Connection.get` and
+:meth:`sftpretty.Connection.put` reach for when you hand them no ``callback``
+of your own. Paramiko calls it once per chunk with the bytes moved so far and
+the total. A large file can fill your terminal, give it a ``logger`` and let
+your handler save the transfer output to a preferred location. Roll your own
+callable that takes two integers if you dare.
+
+.. code-block:: python
+
+    >>> from logging import getLogger
+    >>> from sftpretty.helpers import _callback
+
+    >>> _callback('eels.txt', 512, 1024)
+    Transfer of File: [eels.txt] @ 50.0% 512:1024 bytes
+
+    >>> log = getLogger('LoggyMcLogs')
+    >>> _callback('eels.txt', 512, 1024, logger=log)
+
+
+:func:`sftpretty.helpers.drivepath`
+-----------------------------------
+A painful shim that attempts to convert Windows based pathing into a valid
+POSIX one, that the remote will accept. It's purely lexical, nothing is opened,
+resolved or checked for existence. A path already in POSIX form returns
+untouched.
+
+.. code-block:: python
+
+    >>> from sftpretty.helpers import drivepath
+
+    >>> drivepath('C:\\Users\\nick\\file.txt')
+    '/C:/Users/nick/file.txt'
+    >>> drivepath('C:tmp\\test.txt')
+    '/C:/tmp/test.txt'
+    >>> drivepath('\\\\server\\share\\file.txt')
+    '//server/share/file.txt'
+    >>> drivepath('/home/user/file.txt')
+    '/home/user/file.txt'
+
+
+:func:`sftpretty.helpers.hash`
+------------------------------
+
+One digest, five types of input. Give it a path, an open file object, a
+:class:`io.BytesIO`, some bytes or a string and get back the hexdigest. Anything
+other than the five input types digest as an empty buffer rather than
+complianing. A string that cannot be opened is digested as text rather than
+raising. Only the ``algorithm.name`` is read, so any spent hash object can be
+passed without remnants carrying over between calls. Files are read in
+``blocksize`` chunks, so size shouldn't be a concern.
+
+.. code-block:: python
+
+    >>> from hashlib import md5
+    >>> from pathlib import Path
+    >>> from sftpretty.helpers import hash
+
+    >>> Path('/tmp/eels.txt').write_text('My hovercraft is full of eels.')
+    30
+    >>> hash('/tmp/eels.txt') == hash('My hovercraft is full of eels.')
+    True
+    >>> hash(open('/tmp/eels.txt', 'rb')) == hash('/tmp/eels.txt')
+    True
+    >>> hash('/tmp/eels.txt', algorithm=md5())
+    '5d5bc914f200b729e1c64c927cafe8c3'
+    >>> hash('/tmp/eels.txt', blocksize=8192)
+    '4953167ab20a15c0...'
+
+
+:func:`sftpretty.helpers.localtree`
+-----------------------------------
 Similar to :meth:`sftpretty.Connection.remotetree` except that it walks a
 **local** directory structure. It has the same output format and likewise
-stores the resulting tree in a dictionary.
+stores the resulting tree in a dictionary. Each sub-directory is paired with
+its own finished path. This is the parent :meth:`sftpretty.Connection.put_d`
+appends a directory name to. Links are followed once per target, so a directory
+pointing back at one of its own parents is mapped rather than chased.
 
 .. code-block:: python
 
     import sftpretty
 
     >>> directories = {}
-    >>> sftpretty.localtree(directories, '/home/user/downloads', '/tmp')
+    >>> sftpretty.helpers.localtree(directories, '/home/user/downloads', '/tmp')
     >>> directories
-    {'/home/user/downloads': [('/home/user/downloads/percona', '/tmp/downloads/percona'),
-                              ('/home/user/downloads/wallstreet', '/tmp/downloads/wallstreet')
-                             ]
+    {'/home/user/downloads': [('/home/user/downloads/percona', '/tmp/downloads'),
+                              ('/home/user/downloads/wallstreet', '/tmp/downloads')
+                             ],
+     '/home/user/downloads/wallstreet': [('/home/user/downloads/wallstreet/bets',
+                                          '/tmp/downloads/wallstreet')
+                                        ]
     }
 
 
-:func:`sftpretty.st_mode_to_int`
---------------------------------
-Converts an octal mode result back to an integer representation. The information
-returned in SFTPAttribute object ``.stat(*fname*).st_mode`` contains extra
-things you probably don't care about, in a form that has been converted from
-octal to int so you won't recognize it at first. This function clips the extra
-bits and hands you the file mode in a way you'll recognize.
+:func:`sftpretty.helpers.retry`
+-------------------------------
+For the stubborn programmer in all of us. Calls sometimes fail for no good
+reason and work on subsequent attempts. Name the exceptions worth another
+attempt and wait ``delay`` seconds, multiplied by ``backoff``, then try again.
+Specify a *type* to catch that whole family or an *instance* to catch exactly
+one error while letting it siblings through. So ``IOError(errno.ECOMM)`` will
+sit out a comms failure while a missing file still fails. Set ``silent`` to not
+hear about it or ``logger`` to send the whole song and dance somewhere useful.
+A value of 0 or None for ``tries`` returns your function undecorated. Its count
+refers to total attempts rather than retries, so ``tries=3`` calls three times
+at most.
+
+.. code-block:: python
+
+    >>> from sftpretty.helpers import retry
+
+    >>> @retry(TimeoutError, tries=3, delay=1, backoff=2)
+    ... def flaky():
+    ...     return connection.read()
+
+    >>> flaky()
+    Retry (3/3):
+    connection reset
+    Retrying in 1 second(s)...
+    Retry (2/3):
+    connection reset
+    Retrying in 2 second(s)...
+    'connected'
+
+
+:func:`sftpretty.helpers.st_mode_to_int`
+----------------------------------------
+Converts an octal mode result back to an integer representation. The
+information returned in SFTPAttribute object ``.stat(*fname*).st_mode``
+contains extra things you probably don't care about, in a form that has been
+converted from octal to int so you won't recognize it at first. This function
+clips the extra bits and hands you the file mode in a way you'll recognize.
 
 .. code-block:: python
 
     >>> attr = sftp.stat('readme.txt')
     >>> attr.st_mode
     33188
-    >>> sftpretty.st_mode_to_int(attr.st_mode)
+    >>> sftpretty.helpers.st_mode_to_int(attr.st_mode)
     644

pyproject.toml

@@ -48,7 +48,7 @@ keywords = [
 name = 'sftpretty'
 readme = 'README.rst'
 requires-python = '>=3.6'
-version = '1.2.3'
+version = '1.2.4'
 
 [project.scripts]
 sftpretty = 'sftpretty:Connection'

sftpretty/__init__.py

@@ -145,11 +145,14 @@ def get_config(self, host):
         cval = self.ssh_config.lookup(host)
         return cval or {}
 
-    def get_hostkey(self, host):
+    def get_hostkey(self, host, key_type=None):
         '''Return the matching known hostkey to be used for verification or
         raise an SSHException.
 
         :param str host: The Hostname or IP of the remote machine.
+        :param str|None key_type: *Default: None* - Key type negotiated with
+            the remote such as ``ssh-ed25519``. When None the first hostkey
+            known for a host is returned.
 
         :returns: (obj) PKey - Public key(s) associated with host or None.
 
@@ -159,6 +162,12 @@ def get_hostkey(self, host):
         # None | {key_type: private_key}
         if kval is None:
             raise SSHException(f'No hostkey for host [{host}] found.')
+        if key_type is not None:
+            hostkey = kval.get(key_type)
+            if hostkey is None:
+                raise SSHException(f'No [{key_type}] hostkey for host '
+                                   f'[{host}] found.')
+            return hostkey
 
         # Return the public key from the dictionary
         return list(kval.values())[0]
@@ -495,7 +504,7 @@ def _start_transport(self, host, port):
 
             if self._transport.is_active():
                 remote_hostkey = self._transport.get_remote_server_key()
-                remote_fingerprint = hash(remote_hostkey)
+                remote_fingerprint = hash(remote_hostkey.asbytes())
                 log.info((f'[{host}] Host Key: \n\t'
                           f'Name: {remote_hostkey.get_name()}\n\t'
                           f'Fingerprint: {remote_fingerprint}\n\t'
@@ -507,8 +516,9 @@ def _start_transport(self, host, port):
                     else:
                         knownhost_name = host
                     log.debug(f'Hostkey Name: {knownhost_name}')
-                    user_hostkey = self._cnopts.get_hostkey(knownhost_name)
-                    user_fingerprint = hash(user_hostkey)
+                    user_hostkey = self._cnopts.get_hostkey(
+                        knownhost_name, remote_hostkey.get_name())
+                    user_fingerprint = hash(user_hostkey.asbytes())
                     log.info(f'Known Fingerprint: {user_fingerprint}')
                     if user_fingerprint != remote_fingerprint:
                         raise HostKeysException((f'{host} key verification: '
@@ -521,6 +531,7 @@ def _start_transport(self, host, port):
         except (AttributeError, gaierror, UnicodeError):
             raise ConnectionException(host, port)
         except Exception as err:
+            self.close()
             raise err
 
     def get(self, remotefile, localpath=None, callback=None,