Robur compatibility, async propfind, and url.encode-at

CHANGELOG.md

@@ -23,10 +23,17 @@ This project should adhere to [Semantic Versioning](https://semver.org/spec/v2.0
 * `caldav.config.extract_conn_params_from_section` is now public API (renamed from `_extract_conn_params_from_section`), so that downstream tools like plann can map plann-style config sections (`caldav_url`, `caldav_user`, `features`, etc.) to `DAVClient` parameters without duplicating the logic.
 * New compatibility feature `create-calendar.stable-url` (default `full`): whether a calendar, once created, remains addressable at the URL derived from the requested `cal_id`.  Some servers assign a different *canonical* URL: Zimbra relocates the collection to a display-name-derived path when a display name is set (a collection alias lingers at the `cal_id` and answers `PROPFIND`/`REPORT`, but a `GET` on a child object under it 404s, so the `cal_id` is not a usable address); OX always exposes an opaque `cal://0/NNN` (base64-segment) canonical URL.  Both are marked `create-calendar.stable-url: unsupported`.  For such servers `Calendar._create()` now discovers and adopts the canonical URL after creation (re-pointing `self.url`) instead of dropping the display name, so the calendar keeps its name *and* every later URL-based operation resolves — identical handling for Zimbra and OX.
 * `caldav[niquests]` is now a valid install target.  It changes nothing today - `niquests` is still an ordinary dependency - but v4.0 is planned to ship without a default HTTP library dependency, and `caldav[niquests]` is how the current behaviour will be kept.  Downstream projects can depend on it now and not have to change anything at that point.  See https://github.com/python-caldav/caldav/issues/611
+* `non-existing-raises-not-found` is now a grouping node with two sibling subfeatures, `non-existing-raises-not-found.object` and `.collection` (both default `full`): what a lookup of a non-existing calendar *object* raises, and what a lookup of a non-existing *calendar* raises.  Neither can be derived from the other, so neither is the other's parent.  Robur answers 403 for everything that does not exist, objects included, but `CalendarObjectResource.load()` retries a failed GET as a calendar-multiget REPORT against the existing parent calendar, where Robur reports the missing href with an inner 404 — so an object lookup still ends in `NotFoundError` while a calendar lookup surfaces `AuthorizationError`.  Robur's profile declares exactly that: `quirk` on the object level, with the rescue spelled out in `behaviour`, and `unsupported` on the collection.  An existing config naming the old `non-existing-raises-not-found` still works and now claims both.
+* New compatibility feature `url.encode-at`, with three subfeatures: `url.encode-at.identity` (the server treats `@` and `%40` as two resources — the RFC3986-conformant reading, §2.2), `url.encode-at.literal` (a literal `@` resolves) and `url.encode-at.encoded` (`%40` resolves).  Each is one thing a probe can observe on its own, and the client reads only these — the grouping parent's support level decides nothing.  **Nothing changes for a server you have not configured this for**: `.identity` defaults to `unsupported`, and where the two spellings name one resource the spelling carries no information, so every path is normalised exactly as it always was and an `@` in a UID still becomes `%40`.  Declaring `url.encode-at.identity: full` makes the spelling part of the resource name, and then the client stops normalising it anywhere: an href keeps the spelling the server sent, and the ownCloud calendar-home-set workaround (which has percent-encoded a relative home-set containing an `@` since 2021) switches off — unless `url.encode-at.literal` says the literal spelling is the one that server will not serve, which is the case that workaround was written for.  `url.encode-at.encoded: unsupported` is the only thing that changes which spelling the client *mints*.  The 3.x default is deliberately the non-conformant one: defaulting to conformance would change URL identity under every user of an unprobed server.  It is also the accurate reading — of the twelve test servers, every one that resolved both spellings served them as one.  4.0 should flip it round.  `caldav-server-tester` probes all three.  Not covered here: a server that stores under one spelling and reports back another, which belongs to `save-load.stable-url`.
+* New `multiget_fallback` parameter on `CalendarObjectResource.load()` (default `True` - unchanged behaviour).  When a GET on the object URL fails, `load()` retries it as a calendar-multiget REPORT against the parent collection.  That rescue is what makes servers that do not serve objects over GET usable at all, but it also hides what the server said: a server answering 403 rather than 404 for something non-existing (Robur) still ends up raising `NotFoundError`.  Pass `False` to get the server's own answer instead - `caldav-server-tester` uses it to tell a server that really answers 404 (`non-existing-raises-not-found: full`) from one that is only rescued into it (`quirk`).
 * New `compatibility_workarounds` parameter on `Calendar.search()` / `CalDAVSearcher.search()` / `async_search()`.  When `False`, all server-compatibility workarounds are disabled and the query is sent verbatim (a single REPORT, no comp-type splitting, no filter rewriting, no fallback retries).  Mainly for the server-compatibility checker, to observe raw server behaviour.
 
 ### Fixed
 
+* `collection.py`: the ownCloud `@`-quoting heuristic for a relative `calendar-home-set` lived in three copies — `Principal.calendar_home_set`, its async twin, and `_sanitize_calendar_home_set_url()` used by the PROPFIND extractor — and they had drifted: only the extractor's copy skipped a URL that already contains `%40`, so a home-set the server delivered part-encoded went through `quote()` a second time and came back with `%2540`.  All three are now the one helper.
+* `async_davclient.py` `AsyncDAVClient.propfind()`: a raw XML body passed as `props` - which the sync `DAVClient.propfind()` accepts, and which the async integration test used - was treated as a list of property names and iterated character by character, so the request sent to the server was an empty `<D:prop/>` PROPFIND.  Most servers answer that with a (useless but well-formed) multistatus, hiding the bug; Robur answers with an empty body.  `props` now rejects a string with a `TypeError` naming `body`, which is where a raw request belongs; the async client has had a dedicated `body` parameter from the start, so rather than inherit the sync client's legacy shape it refuses the mistake outright.  The async integration test passes `body=` accordingly.
+* `compatibility_hints.py`: the Zimbra and CCS profiles drop their stale `search.recurrences.expanded.todo: unsupported` declarations (both observed `full`, 2026-08-26).  The declarations dated from when the server-tester searched the *event* calendar for the recurring todo, so any server keeping tasks in a collection of their own could only ever come out `unsupported`.
+* `compatibility_hints.py`: the Robur profile is refreshed against the live server (2026-08-24).  `delete-calendar` (declared `unsupported`), `non-existing-raises-not-found` (declared `unsupported`, a leftover from the old `non_existing_raises_other` flag that was never re-probed) and `search.comp-type.optional` (declared `ungraceful`) are all observed as `full` and are dropped from the profile.  The stale `delete-calendar` entry made `Calendar.delete()` silently degrade to `delete(wipe=True)`, so nothing ever removed a calendar from a Robur account.
 * A response body that contains no iCalendar at all - an empty object, an HTML error page delivered with a 200, a notification carrying only headers - now raises `caldav.lib.error.ResponseError` naming the URL and quoting what arrived, instead of `ValueError: Found no components where exactly one is required` from inside the icalendar library.  Data that does contain an iCalendar component is unaffected and still parsed by icalendar as before.
 * The source distribution no longer ships stray local files.  hatchling's VCS-ignore support only honours the *root* `.gitignore`, so files hidden from `git status` by a nested `.gitignore` or by the packager's global git ignore file were invisible locally and packaged anyway — `caldav-3.2.1.tar.gz` contains `.claude/settings.json` and 1755 files under `venv/`, and the current tree would have added 443 files under `.prompts/`.  A new `package` tox environment (run in CI, and now part of the release procedure) builds both artifacts and fails if anything git does not track turns up in them.
 * Looking up a calendar or object that does not exist now raises `NotFoundError` also when the server reports the 404 inside a `207 Multi-Status` (a bare `<status>` on the `<response>` element, which RFC 4918 §14.24 allows) rather than as a plain HTTP 404.  Previously the property lookup found no `propstat` elements, ignored the 404, and returned `None` for every requested property — so e.g. `calendar.get_display_name()` on a non-existent calendar silently returned `None` while `calendar.get_events()` on the same calendar raised.  Observed against Xandikos.

caldav/async_davclient.py

@@ -104,7 +104,7 @@ def auth_flow(self, request):
 from caldav import __version__
 from caldav.base_client import BaseDAVClient
 from caldav.base_client import get_davclient as _base_get_davclient
-from caldav.compatibility_hints import FeatureSet
+from caldav.compatibility_hints import FeatureSet, at_spellings_are_aliased
 from caldav.lib import error
 from caldav.lib.python_utilities import to_wire
 from caldav.lib.url import URL
@@ -235,6 +235,11 @@ def __init__(
 
         # Parse and store URL
         self.url = URL.objectify(url_str)
+        ## Whether this server aliases the two "@" spellings travels with the
+        ## URL, and every URL the library builds is joined onto this one, so
+        ## setting it here reaches canonical(), __eq__ and __hash__ everywhere
+        ## without threading the feature set through them.
+        self.url = self.url.with_alias_at(at_spellings_are_aliased(self.features))
 
         # Combine credentials (explicit params take precedence).
         # An explicit username discards the URL credentials wholesale: they
@@ -535,12 +540,24 @@ async def propfind(
             depth: Maximum recursion depth.
             headers: Additional headers.
             props: List of property names to request (uses protocol layer).
+                A raw XML body belongs in ``body``, not here.
 
         Returns:
             DAVResponse with results attribute containing parsed PropfindResult list.
         """
         # Use protocol layer to build XML if props provided
         if props is not None and not body:
+            ## Guard the one mistake this signature invites.  A raw XML body
+            ## handed to _build_propfind_body() would be iterated character by
+            ## character and quietly produce an empty <D:prop/>, which most
+            ## servers answer with a well-formed but useless multistatus - so
+            ## the bug hides.  The sync client accepts a body here for
+            ## backward compatibility; this one is new and has ``body``.
+            if isinstance(props, str):
+                raise TypeError(
+                    "propfind(props=...) takes a list of property names; "
+                    "pass a raw XML request as propfind(body=...) instead"
+                )
             body = self._build_propfind_body(props).decode("utf-8")
 
         final_headers = self._build_method_headers("PROPFIND", depth, headers)

caldav/base_client.py

@@ -330,7 +330,7 @@ def _calendar_home_url(self, home_set_response: Any, principal: Any) -> str:
             _extract_calendar_home_set_from_results as extract_home_set,
         )
 
-        calendar_home_url = extract_home_set(home_set_response.results)
+        calendar_home_url = extract_home_set(home_set_response.results, features=self.features)
         if not calendar_home_url:
             calendar_home_url = str(principal.url)
         return self._make_absolute_url(calendar_home_url)
@@ -342,7 +342,7 @@ def _build_calendars_from_propfind(self, list_response: Any) -> list:
             _extract_calendars_from_propfind_results as extract_calendars,
         )
 
-        calendar_infos = extract_calendars(list_response.results)
+        calendar_infos = extract_calendars(list_response.results, features=self.features)
         return [
             Calendar(client=self, url=info.url, name=info.name, id=info.cal_id)
             for info in calendar_infos

caldav/calendarobjectresource.py

@@ -40,6 +40,7 @@
 from contextlib import contextmanager
 
 from .base_client import ICALH
+from .compatibility_hints import at_spelling_to_mint
 from .datastate import DataState, IcalendarState, NoDataState, RawDataState, VobjectState
 from .davobject import DAVObject
 from .elements import cdav, dav
@@ -51,13 +52,25 @@
 log = logging.getLogger("caldav")
 
 
-def _quote_uid(uid: str) -> str:
+def _quote_uid(uid: str, features: Any = None) -> str:
     """URL-quote a UID for use in a CalDAV object URL.
 
     Slashes are double-quoted (replaced with %2F before percent-encoding)
     per https://github.com/python-caldav/caldav/issues/143.
+
+    A UID that is an email address puts an ``@`` in the path, and this is one
+    of the two places where the client has no existing spelling to preserve
+    and must pick one.  It picks the literal ``@``: RFC3986 section 3.3 makes
+    it a legal ``pchar``, so encoding it is a rewrite nobody asked for.  Only
+    a server whose ``url.encode-at.literal`` is declared unsupported - the
+    ownCloud/Nextcloud case - gets ``%40`` instead.
+
+    This used to encode unconditionally.  An object whose UID contains an
+    ``@``, stored by an older caldav, therefore lives at the ``%40`` spelling;
+    ``load()`` finds it anyway through its multiget and by-UID fallbacks.
     """
-    return quote(uid.replace("/", "%2F"))
+    safe = "/@" if at_spelling_to_mint(features) == "@" else "/"
+    return quote(uid.replace("/", "%2F"), safe=safe)
 
 
 class CalendarObjectResource(DAVObject):
@@ -934,13 +947,26 @@ def copy(self, keep_uid: bool = False, new_parent: Any | None = None) -> Self:
 
     ## TODO: move get-logics to a load_by_get method.
     ## The load method should deal with "server quirks".
-    def load(self, only_if_unloaded: bool = False) -> "Self | Coroutine[Any, Any, Self]":
+    def load(
+        self, only_if_unloaded: bool = False, multiget_fallback: bool = True
+    ) -> "Self | Coroutine[Any, Any, Self]":
         """
         (Re)load the object from the caldav server.
 
         For sync clients, loads and returns self.
         For async clients, returns a coroutine that must be awaited.
 
+        :param only_if_unloaded: skip the server round-trip if the object
+            already carries data.
+        :param multiget_fallback: when the GET fails, retry it as a
+            calendar-multiget REPORT against the parent collection.  Some
+            servers do not serve calendar object resources over GET at all,
+            and some (Robur) answer 403 rather than 404 for anything that
+            does not exist, so the fallback is what makes them usable and is
+            on by default.  Pass False to see what the server itself
+            answered - a compatibility checker probing whether a missing
+            object really raises NotFoundError needs the unrescued error.
+
         Example (sync):
             obj.load()
 
@@ -961,7 +987,9 @@ def load(self, only_if_unloaded: bool = False) -> "Self | Coroutine[Any, Any, Se
 
         # Dual-mode support: async clients return a coroutine
         if self.is_async_client:
-            return self._async_load(only_if_unloaded=only_if_unloaded)
+            return self._async_load(
+                only_if_unloaded=only_if_unloaded, multiget_fallback=multiget_fallback
+            )
 
         if self.url is None:
             raise ValueError("Unexpected value None for self.url")
@@ -980,10 +1008,11 @@ def load(self, only_if_unloaded: bool = False) -> "Self | Coroutine[Any, Any, Se
             uid = self.id
             if uid:
                 # Fallback 1: try multiget (REPORT may work even when GET fails)
-                try:
-                    return self.load_by_multiget()
-                except Exception:
-                    pass
+                if multiget_fallback:
+                    try:
+                        return self.load_by_multiget()
+                    except Exception:
+                        pass
                 # Fallback 2: re-fetch by UID (server may have changed the URL)
                 if self.parent and hasattr(self.parent, "get_object_by_uid"):
                     try:
@@ -998,12 +1027,16 @@ def load(self, only_if_unloaded: bool = False) -> "Self | Coroutine[Any, Any, Se
                         pass
             raise
         except Exception:
+            if not multiget_fallback:
+                raise
             return self.load_by_multiget()
 
         self._update_tag_props(r)
         return self
 
-    async def _async_load(self, only_if_unloaded: bool = False) -> Self:
+    async def _async_load(
+        self, only_if_unloaded: bool = False, multiget_fallback: bool = True
+    ) -> Self:
         """Async implementation of load."""
         if only_if_unloaded and self.is_loaded():
             return self
@@ -1022,10 +1055,11 @@ async def _async_load(self, only_if_unloaded: bool = False) -> Self:
             uid = self.id
             if uid:
                 # Fallback 1: try multiget (REPORT may work even when GET fails)
-                try:
-                    return await self.load_by_multiget()
-                except Exception:
-                    pass
+                if multiget_fallback:
+                    try:
+                        return await self.load_by_multiget()
+                    except Exception:
+                        pass
                 # Fallback 2: re-fetch by UID (server may have changed the URL)
                 if self.parent and hasattr(self.parent, "get_object_by_uid"):
                     try:
@@ -1040,6 +1074,8 @@ async def _async_load(self, only_if_unloaded: bool = False) -> Self:
                         pass
             raise
         except Exception:
+            if not multiget_fallback:
+                raise
             return await self.load_by_multiget()
 
         self._update_tag_props(r)
@@ -1204,7 +1240,8 @@ def _generate_url(self):
         ## See https://github.com/python-caldav/caldav/issues/143 for the rationale behind double-quoting slashes
         ## TODO: should try to wrap my head around issues that arises when id contains weird characters.  maybe it's
         ## better to generate a new uuid here, particularly if id is in some unexpected format.
-        url = self.parent.url.join(_quote_uid(self.id) + ".ics")
+        features = self.client.features if self.client is not None else None
+        url = self.parent.url.join(_quote_uid(self.id, features) + ".ics")
         assert " " not in str(url)
         return url