AsyncDB

AsyncDB is a collection of different Database Drivers using asyncio-based connections and binary connectors (as asyncpg) but providing an abstraction layer to easily connect to different data sources, a high-level abstraction layer for various non-blocking database connectors, on other blocking connectors (like MS SQL Server) we are using ThreadPoolExecutors to run in a non-blocking manner.

Why AsyncDB?

The finality of AsyncDB is to provide us with a subset of drivers (connectors) for accessing different databases and data sources for data interaction. The main goal of AsyncDB is to use asyncio-based technologies.

Getting Started

Requirements

Python 3.9+

Installation

$ pip install asyncdb
---> 100%
Successfully installed asyncdb

Can also install only drivers required like:

$ pip install asyncdb[pg] # this install only asyncpg

Or install all supported drivers as:

$ pip install asyncdb[all]

Requirements

• Python >= 3.8
• asyncio (https://pypi.python.org/pypi/asyncio/)

Currently AsyncDB supports the following databases:

• PostgreSQL (supporting two different connectors: asyncpg or aiopg)
• SQLite (requires aiosqlite)
• mySQL/MariaDB (requires aiomysql and mysqlclient)
• ODBC (using aioodbc)
• JDBC(using JayDeBeApi and JPype)
• RethinkDB (requires rethinkdb)
• Redis (requires aioredis)
• Memcache (optional, requires aiomcache: pip install asyncdb[memcache])
• MS SQL Server (non-asyncio using freeTDS and pymssql)
• Apache Cassandra (requires official cassandra driver)
• InfluxDB (using influxdb)
• CouchBase (using aiocouch)
• MongoDB (using motor and pymongo)
• SQLAlchemy (requires sqlalchemy async (+3.14))
• Oracle (requires oracledb)
• Redpanda (Kafka-compatible, requires aiokafka)

Platform Support

AsyncDB's core package (pip install asyncdb) and the default extra (pip install asyncdb[default]) are supported on Linux, macOS, and Windows for Python 3.10–3.14. uvloop remains an optional performance extra (pip install asyncdb[uvloop]): it is only available on POSIX platforms and is never required to import or run AsyncDB. On Windows, uvloop activation is automatically and safely skipped, and the standard asyncio event loop policy is used instead.

The default extra covers SQLite (aiosqlite), RethinkDB, InfluxDB, Redis, MS SQL Server (pymssql), Delta Lake, and DuckDB, in addition to the PostgreSQL support (asyncpg) that ships with the core package.

Other provider extras (Cassandra, ScyllaDB, MySQL/MySQLdb, ODBC, Oracle, JDBC, ClickHouse, etc.) depend on native or platform-sensitive client libraries and are not guaranteed to be installable or fully supported on every platform, including Windows. Selecting a driver whose optional dependency is not installed raises a focused error naming the provider and the extra required to install it (e.g. pip install asyncdb[mysql]).

Quick Tutorial

from asyncdb import AsyncDB

db = AsyncDB('pg', dsn='postgres://user:password@localhost:5432/database')

# Or you can also passing a dictionary with parameters like:
params = {
    "user": "user",
    "password": "password",
    "host": "localhost",
    "port": "5432",
    "database": "database",
    "DEBUG": True,
}
db = AsyncDB('pg', params=params)

async with await db.connection() as conn:
    result, error = await conn.query('SELECT * FROM test')

And that's it!, we are using the same methods on all drivers, maintaining a consistent interface between all of them, facilitating the re-use of the same code for different databases.

Every Driver has a simple name to call it:

• pg: AsyncPG (PostgreSQL)
• postgres: aiopg (PostgreSQL)
• mysql: aiomysql (mySQL)
• influx: influxdb (InfluxDB)
• redis: redis-py (Redis)
• memcache: aiomcache (Memcache, async, no system libraries required)
• odbc: aiodbc (ODBC)
• oracle: oracle (oracledb)
• redpanda: Redpanda/Kafka (aiokafka)

Output Support

With Output Support results can be returned into a wide-range of variants:

from datamodel import BaseModel

class Point(BaseModel):
    col1: list
    col2: list
    col3: list

db = AsyncDB('pg', dsn='postgres://user:password@localhost:5432/database')
async with await d.connection() as conn:
    # changing output format to Pandas:
    conn.output_format('pandas')  # change output format to pandas
    result, error = await conn.query('SELECT * FROM test')
    conn.output_format('csv')  # change output format to CSV
    result, _ = await conn.query('SELECT TEST')
    conn.output_format('dataclass', model=Point)  # change output format to Dataclass Model
    result, _ = await conn.query('SELECT * FROM test')

Currently AsyncDB supports the following Output Formats:

• CSV (comma-separated or parametrized)
• JSON (using orjson)
• iterable (returns a generator)
• Recordset (Internal meta-Object for list of Records)
• Pandas (a pandas Dataframe)
• Datatable (Dt Dataframe)
• Dataclass (exporting data to a dataclass with -optionally- passing Dataclass instance)
• PySpark Dataframe

And others to come:

• Apache Arrow (using pyarrow)
• Polars (Using Python polars)
• Dask Dataframe

Contribution guidelines

Please have a look at the Contribution Guide

• Writing tests
• Code review

Who do I talk to?

• Repo owner or admin
• Other community or team contact

License

AsyncDB is copyright of Jesus Lara (https://phenobarbital.info) and is licensed under BSD. I am providing code in this repository under an open source licenses, remember, this is my personal repository; the license that you receive is from me and not from my employeer.