Skip to content

Add a counter example using hyperdrive and Durable Objects - #102

Merged
ryanking13 merged 3 commits into
mainfrom
gyeongjae/hyperdrive-example
Sep 7, 2026
Merged

Add a counter example using hyperdrive and Durable Objects#102
ryanking13 merged 3 commits into
mainfrom
gyeongjae/hyperdrive-example

Conversation

@ryanking13

Copy link
Copy Markdown
Contributor

This example shows creating a counter using Durable Objects and storing the data into PostgreSQL-backed Hyperdrive.

  1. there is one Durable objects per room that hold the counter, and multiple workers can add a counter or query the current counter from that DO.
  2. DO stores the counter in 3-levels.
  • In memory (caching)
  • sqlite3 (per room, faster and cheaper storage)
  • postgreSQL (hyperdrive, storing all counter for all rooms)
  1. DO stores counter in memory first ==> then stores the value to sqlite3.

  2. For every 5 seconds, it accumulates the counter and stores the value to Hyperdrive.

Screen.Recording.2026-09-02.at.6.22.12.PM.mov

@ryanking13
ryanking13 requested review from dom96 and hoodmane September 2, 2026 09:27
Comment thread hyperdrive-batched-counter/src/room.py
Comment thread batched-counter/src/constants.py Outdated
Comment thread batched-counter/src/room.py Outdated
Comment thread batched-counter/src/room.py Outdated
Comment thread batched-counter/src/room.py Outdated
Comment thread batched-counter/src/room.py Outdated
Comment thread batched-counter/src/room.py Outdated
Comment on lines +130 to +132
connection = None
try:
connection = self._connection()

@hoodmane hoodmane Sep 2, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
connection = None
try:
connection = self._connection()
with self._connection() as connection:
try:

Comment thread batched-counter/src/room.py Outdated
@hoodmane

hoodmane commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Perhaps we should include hyperdrive in the folder name?

@hoodmane

hoodmane commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

I tried it out and got a bunch of errors like AttributeError: module 'hashlib' has no attribute 'pbkdf2_hmac'. It says "Function only available when Python is compiled with OpenSSL."
https://docs.python.org/3/library/hashlib.html#hashlib.pbkdf2_hmac

I guess in Pyodide 0.29.x it is available, but in 314.x it has been removed.

Traceback

Exception in ASGI application after response started
Traceback (most recent call last):
  File "/session/metadata/python_modules/workers/asgi.py", line 283, in run_app
    await app(request_to_scope(req, env, state=state), receive, send)
  File "/session/metadata/python_modules/starlette/applications.py", line 96, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/session/metadata/python_modules/starlette/middleware/errors.py", line 186, in __call__
    raise exc
  File "/session/metadata/python_modules/starlette/middleware/errors.py", line 164, in __call__
    await self.app(scope, receive, _send)
  File "/session/metadata/python_modules/starlette/middleware/exceptions.py", line 63, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/session/metadata/python_modules/starlette/_exception_handler.py", line 53, in wrapped_app
    raise exc
  File "/session/metadata/python_modules/starlette/_exception_handler.py", line 42, in wrapped_app
    await app(scope, receive, sender)
  File "/session/metadata/python_modules/starlette/routing.py", line 670, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/session/metadata/python_modules/starlette/routing.py", line 690, in app
    await route.handle(scope, receive, send)
  File "/session/metadata/python_modules/starlette/routing.py", line 280, in handle
    await self.app(scope, receive, send)
  File "/session/metadata/python_modules/starlette/routing.py", line 65, in app
    await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  File "/session/metadata/python_modules/starlette/_exception_handler.py", line 53, in wrapped_app
    raise exc
  File "/session/metadata/python_modules/starlette/_exception_handler.py", line 42, in wrapped_app
    await app(scope, receive, sender)
  File "/session/metadata/python_modules/starlette/routing.py", line 62, in app
    response = await f(request)
               ^^^^^^^^^^^^^^^^
  File "/session/metadata/entry.py", line 28, in get_stats
    return JSONResponse(await stub.get_stats(room))
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/session/metadata/python_modules/workers/rpc.py", line 295, in await_and_convert
    return self._convert_result(await result)
                                ^^^^^^^^^^^^
pyodide.ffi.JsException: PythonError: Traceback (most recent call last):
  File "introspection.py", line 88, in wrapper_func
    return python_to_rpc(await result)
                         ^^^^^^^^^^^^
  File "/session/metadata/room.py", line 54, in get_stats
    connection = self._connection()
  File "/session/metadata/room.py", line 111, in _connection
    return pg8000.connect(
           ~~~~~~~~~~~~~~^
        host=hd.host,
        ^^^^^^^^^^^^^
    ...<4 lines>...
        ssl_context=False,
        ^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/session/metadata/python_modules/pg8000/__init__.py", line 112, in connect
    return Connection(
        user,
    ...<11 lines>...
        startup_params=startup_params,
    )
  File "/session/metadata/python_modules/pg8000/legacy.py", line 445, in __init__
    super().__init__(*args, **kwargs)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/session/metadata/python_modules/pg8000/core.py", line 398, in __init__
    raise e
  File "/session/metadata/python_modules/pg8000/core.py", line 391, in __init__
    self.message_types[code](_read(self._sock, data_len - 4), context)
    ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/session/metadata/python_modules/pg8000/core.py", line 618, in handle_AUTHENTICATION_REQUEST
    msg = self.auth.get_client_final().encode("utf8")
          ~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/session/metadata/python_modules/scramp/core.py", line 336, in get_client_final
    self.server_signature, cfinal = _get_client_final(
                                    ~~~~~~~~~~~~~~~~~^
        self.hf,
        ^^^^^^^^
    ...<7 lines>...
        self.gs2_header,
        ^^^^^^^^^^^^^^^^
    )
    ^
  File "/session/metadata/python_modules/scramp/core.py", line 838, in _get_client_final
    salted_password = _make_salted_password(hf, password, salt, iterations)
  File "/session/metadata/python_modules/scramp/core.py", line 455, in _make_salted_password
    return hashlib.pbkdf2_hmac(
           ^^^^^^^^^^^^^^^^^^^
AttributeError: module 'hashlib' has no attribute 'pbkdf2_hmac'

Comment thread hyperdrive-batched-counter/schema.sql
@ryanking13

Copy link
Copy Markdown
Contributor Author

I tried it out and got a bunch of errors like AttributeError: module 'hashlib' has no attribute 'pbkdf2_hmac'. It says "Function only available when Python is compiled with OpenSSL."

Right. pbkdf2_hmac comes from openssl which is not available in pyodide 314. I added md5 auth in docker compose file but adding it in the migration file sounds good.

@ryanking13
ryanking13 merged commit 5827797 into main Sep 7, 2026
7 checks passed
@ryanking13
ryanking13 deleted the gyeongjae/hyperdrive-example branch September 7, 2026 10:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants