Skip to content
Merged
60 changes: 59 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -185,6 +185,50 @@ async with await client.rent(schedule_id) as browser:
- Encrypt the blob before writing to disk if it contains sensitive credentials.
- `import_()` raises `ValueError` on `schema_version` mismatch (future-proofing).

## Browser Vault (server-stored sessions)

The **vault** stores a browser snapshot (cookies + per-origin localStorage/sessionStorage + fingerprint) on the Ceki API (`/api/vault/sessions`, encrypted server-side) so you can restore it on a **different** browser later — the same session profile across machines.

Two surfaces:

**`client.vault`** — HTTP CRUD on vault sessions (no live browser needed):

```python
# List your vault sessions
sessions = await client.vault.list()
for s in sessions:
print(s.id, s.label, s.urls)

# Fetch one session with its decrypted profile envelope
session = await client.vault.get(8)
data = session.data # {cookies, localStorage, sessionStorage, fingerprint, urls, collectedAt}
```

**`browser.vault`** — snapshot save / restore from a rented browser:

```python
# 1. On browser A — export the current state into a NEW vault session
vault_id = await browser.vault.save(label="vc.ru session")

# or overwrite the session you rented with (browser was rented with vault=8)
vault_id = await browser.vault.save() # PUT onto the bound id

# 2. On browser B (or the same one later) — rent WITH the vault profile
async with await client.rent(schedule_id, vault=vault_id) as browser:
# cookies applied immediately, localStorage/sessionStorage buffered by the
# extension and flushed on first navigation to each origin
await browser.send({"method": "Page.navigate", "params": {"url": "https://vc.ru"}})

# or restore the profile mid-session
await browser.vault.restore(vault_id)
```

Notes:
- Vault restore uses `session.configure(profile=...)` — the extension applies cookies first, then buffers localStorage/sessionStorage until the first navigation to each origin (Vault 3+ extension required).
- When `vault=<id>` is passed to `rent()`, the browser is bound to that vault session: a later `browser.vault.save()` overwrites it (PUT).
- The vault endpoints are guarded by Sanctum and resolve to a **user**; use a user token as `api_key` for vault operations.
- `browser.profile.export()` (local blob) and `browser.vault.save()` (server) are complementary: the first keeps the blob agent-side, the second stores it encrypted on the API.

## CDP Lifecycle

The relay maintains the CDP connection to the incognito browser tab. If the connection drops, it automatically reattaches with 1s/2s/4s exponential backoff. Commands during reattach are buffered (FIFO, max 50). If 3 reattach attempts fail, a new fallback tab is created. If that also fails, `cdp_unrecoverable` error is sent.
Expand Down Expand Up @@ -299,7 +343,7 @@ The CLI persists session state locally — after `rent` it saves the session ID
|---|---|
| `search [--limit N] [--filter K=V]…` | List available browsers |
| `my-browsers` | List browsers with pre-arranged rent contracts |
| `rent --schedule ID [--mode incognito\|main] [--fingerprint-from FILE]` | Rent a browser |
| `rent --schedule ID [--mode incognito\|main] [--fingerprint-from FILE] [--vault SESSION_ID]` | Rent a browser |
| `sessions [--all] [--limit N] [--json]` | List your sessions |
| `stop SID` | End a session |
| `wait SID` | Block until the session ends |
Expand Down Expand Up @@ -336,6 +380,20 @@ The CLI persists session state locally — after `rent` it saves the session ID
| `configure SID [--masking-mode VAL] [--fingerprint VAL]` | Toggle masking / fingerprint |
| `cdp SID --method METHOD [--params JSON]` | Raw CDP command |

#### Vault sessions

| Command | Description |
|---|---|
| `vault list [--json] [--per-page N]` | List vault sessions (id, label, urls, updated) |
| `vault get ID [--json] [-o FILE]` | Show a session; `--json` prints the decrypted profile, `-o` dumps it to a file |
| `vault save FILE [--id ID] [--label L]` | Create (or PUT-update with `--id`) a session from a profile JSON |
| `vault save --session SID [--id ID] [--label L] [--no-session-storage]` | Snapshot a live rental session into the vault |
| `vault apply ID --session SID` | Apply a vault profile into an existing rental (resume + restore) |
| `vault apply ID --schedule N` | Rent a fresh browser with the vault profile restored |
| `vault delete ID` | Delete a vault session |

Vault commands run over plain HTTP (no relay session) and are user/Sanctum-scoped — the same token caveat as `client.vault` applies (use a user token).

### Output and errors

Successful commands write a single JSON line to stdout. Errors go to stderr as `{"error": "...", "code": "..."}`. Pipe stdout through `jq` to chain commands.
Expand Down
6 changes: 5 additions & 1 deletion ceki_sdk/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,10 @@
from ._models import BrowserOption, ChatMessage, Match, ReadReceipt, SessionInfo, Snapshot
from ._profile import BrowserProfile
from ._provider import ProviderError, run_provider
from ._vault import BrowserVault, ClientVault, VaultSession
from .humanize import HumanProfile

__version__ = "2.37.0"
__version__ = "2.37.1"
__all__ = [
"connect",
"ConnectOptions",
Expand All @@ -47,6 +48,9 @@
"SessionInfo",
"Snapshot",
"BrowserProfile",
"BrowserVault",
"ClientVault",
"VaultSession",
"CekiError",
"HumanProfile",
"CaptchaResult",
Expand Down
46 changes: 38 additions & 8 deletions ceki_sdk/_browser.py
Original file line number Diff line number Diff line change
Expand Up @@ -105,9 +105,16 @@ def __init__(self, client: "Client", match: Match, *, human="natural") -> None:

from ._chat import BrowserChat
from ._profile import BrowserProfile
from ._vault import BrowserVault

self.chat = BrowserChat(self)
self.profile = BrowserProfile(self)
self.vault = BrowserVault(self)

# Bound vault session id — set when the browser was rented with
# vault=<id> or restored from one (BrowserVault.restore). BrowserVault.save
# PUTs onto this id on overwrite=True.
self._vault_session_id: int | None = None

env_profile = os.environ.get("CEKI_HUMAN_PROFILE")
env_path = os.environ.get("CEKI_HUMAN_PROFILE_PATH")
Expand Down Expand Up @@ -181,20 +188,30 @@ async def send(self, cdp: dict[str, Any], *, timeout: float = 60.0) -> dict[str,
# ConnectionError/OSError (DC broken): permanent WS fallback via
# _p2p_fallback to avoid 30s wait on every subsequent command.
try:
await asyncio.wait_for(p2p.wait_dc_open(), timeout=30.0)
# Short grace for the DC to open (ice gathering can take a
# second), then hard-fall back to WS for this session.
await asyncio.wait_for(p2p.wait_dc_open(), timeout=3.0)
await p2p.send_cdp({
"session_id": self.session_id,
"id": cdp_id,
"method": cdp["method"],
"params": cdp.get("params", {}),
})
except asyncio.TimeoutError:
# DC never opened (relay/extension never completed the
# signaling). Fall back to WS for good on THIS browser —
# otherwise every subsequent send() re-pays the 30s
# wait_dc_open() stall (test .send(timeout=5) would
# always time out).
log.warning(
"cdp: P2P DC not ready within 30s for cmd %d — WS fallback for this cmd",
cdp_id,
"cdp: P2P DC not ready within 30s — permanent WS fallback for this session",
)
self._p2p_fallback = True
fut._cdp_transport = 'ws' # type: ignore[attr-defined]
log.debug("cdp: WS fallback sending cmd %d session=%s method=%s", cdp_id, self.session_id, cdp["method"])
log.debug(
"cdp: WS fallback sending cmd %d session=%s method=%s",
cdp_id, self.session_id, cdp["method"],
)
await self._client._ws_send(
{
"type": "cdp",
Expand Down Expand Up @@ -958,7 +975,10 @@ async def _on_cdp_response(self, msg: dict[str, Any]) -> None:
# Skip the WS echo and wait for the DC response.
transport = getattr(fut, '_cdp_transport', 'ws')
is_from_ws = msg.get("type") == "cdp_response" or "session_id" in msg
log.debug("_on_cdp_response: transport=%s is_from_ws=%s skip=%s", transport, is_from_ws, transport == 'dc' and is_from_ws)
log.debug(
"_on_cdp_response: transport=%s is_from_ws=%s skip=%s",
transport, is_from_ws, transport == 'dc' and is_from_ws,
)
if transport == 'dc' and is_from_ws:
log.debug("cdp: skip WS echo for DC-sent command id=%s", cmd_id)
return
Expand All @@ -971,7 +991,10 @@ async def _on_cdp_response(self, msg: dict[str, Any]) -> None:
log.debug("_on_cdp_response: resolving future with error %s", err)
fut.set_exception(Exception(f"CDP error {err}"))
else:
log.debug("_on_cdp_response: id=%s NOT in pending (keys=%s) or None", cmd_id, list(self._pending_cdp.keys()))
log.debug(
"_on_cdp_response: id=%s NOT in pending (keys=%s) or None",
cmd_id, list(self._pending_cdp.keys()),
)

async def _on_cdp_event(self, msg: dict[str, Any]) -> None:
method = msg.get("method", "")
Expand All @@ -997,8 +1020,15 @@ async def _on_tab_opened(self, msg: dict[str, Any]) -> None:
asyncio.create_task(cast(Coroutine, cb(url)))

async def _on_session_ended(self, msg: dict[str, Any]) -> None:
reason = msg.get("reason", "completed")
self._ended_reason = reason
if self._ended.is_set():
# A terminal error (e.g. -1011 heartbeat_timeout) already ended the
# session with a precise reason. A later bare session_ended without
# an explicit reason must not clobber it with a generic "completed".
if msg.get("reason"):
self._ended_reason = msg["reason"]
else:
self._ended_reason = msg.get("reason", "completed")
reason = self._ended_reason
if reason == "provider_disconnected":
exc: Exception = ProviderDisconnected()
else:
Expand Down
33 changes: 29 additions & 4 deletions ceki_sdk/_client.py
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,10 @@ def __init__(
# shared WebSocket once the last session for a client is gone.
self._on_session_ended: Callable[[str], Awaitable[None]] | None = None

# Vault HTTP surface (see ceki_sdk/_vault.py)
from ._vault import ClientVault
self.vault = ClientVault(self)

# P2P WebRTC transport (primary, WS = fallback)
self._p2p: WebRTCTransport | None = None
self._p2p_init_lock = asyncio.Lock()
Expand Down Expand Up @@ -198,6 +202,7 @@ async def rent(
masking_mode: bool = True,
fingerprint: bool | dict | None = True,
pacing_profile: str | None = None,
vault: int | dict[str, Any] | None = None,
) -> Browser:
if mode not in ("incognito", "main"):
raise ValueError(f"mode must be 'incognito' or 'main', got {mode!r}")
Expand Down Expand Up @@ -227,16 +232,31 @@ async def rent(
# Wait for P2P WebRTC transport to initialize before returning Browser.
# Otherwise Browser.send() races with _init_p2p() — first CDP falls back
# to WS because self._p2p is still None.
if self._p2p_enabled and not self._p2p_ready.is_set():
#
# Only wait when P2P has actually been initiated (the relay answered
# with a webrtc.offer / the extension started a host-initiated P2P).
# If _p2p is still None the signaling never started (or failed) and the
# WS path is already the right one — waiting the full 15s here would
# stall every rent when the relay simply doesn't do P2P.
if self._p2p_enabled and not self._p2p_ready.is_set() and self._p2p is not None:
try:
await asyncio.wait_for(self._p2p_ready.wait(), timeout=15)
except asyncio.TimeoutError:
log.warning("P2P transport not ready within 15s, CDP will use WS path")

browser = Browser(client=self, match=match, human=human)
self._active_browsers[match.session_id] = browser
with_restored = False
# Vault profile restore — do it first so the rent() fingerprint branch
# below can't clobber a profile-supplied fingerprint. Profile cookies/
# storage go through session.configure(profile=...) (Vault 3+ extension).
if vault is not None:
await browser.vault.restore(vault)
with_restored = True
if not masking_mode:
await browser.configure(masking_mode=False)
if with_restored:
return browser
if isinstance(fingerprint, dict):
await browser.configure(fingerprint=fingerprint)
elif fingerprint is False or fingerprint is None:
Expand Down Expand Up @@ -589,9 +609,14 @@ async def _dispatch(self, msg: dict[str, Any]) -> None:
browser = self._active_browsers.get(session_id) if session_id else None
if browser is not None and msg.get("code", 0) in (-1011, -1018):
# Relay reports a session end as ``error -1011/-1018`` (provider
# death, grace expiry, admin kill). Clean up exactly like
# ``session_ended`` so the daemon never keeps a dead session.
await browser._on_session_ended(msg)
# death, grace expiry, admin kill). Route through
# ``_on_error`` first so the terminal reason (e.g.
# ``heartbeat_timeout`` for -1011) is preserved — calling
# ``_on_session_ended`` directly would clobber it with a
# generic ``completed`` when the message has no ``reason``
# field. Then clean up exactly like ``session_ended`` so the
# daemon never keeps a dead session.
await browser._on_error(msg)
hook = self._on_session_ended
if hook is not None:
try:
Expand Down
Loading
Loading