Add a TokenSource crate to the Rust SDKs by MaxHeimbrock · Pull Request #1274 · livekit/rust-sdks

MaxHeimbrock · 2026-07-22T09:29:13Z

Before you submit your PR

Make sure the following is true before submitting your PR:

[ x ] I have read the contributing guidelines and validated that this PR will be accepted.
[ x ] I have read and followed the principles regarding breaking changes, testing, and code quality.

PR description

This PR adds the Token Source concept as an independent crate.

Testing

Ideally, unit test the code you add, but ensure you're not repeating existing test cases. Use as many already written scaffolding, utilities as possible; write your own, when needed. If external services, APIs, tokens are required (e.g., running an LK server instance), provide the necessary information. Make sure your tests perform useful, context-aware assertions and do not simply emulate "happy paths".

Async

We want the project to be runtime-agnostic, so please reuse what's already in livekit-runtime and feel free to add anything missing. It's ok to use Tokio directly, when writing unit tests, if necessary. When testing, do not use artificial delays for the state to "catch up"; instead, respect the event flow and subscribe properly using channels or other mechanisms.

ladvoc

Left some initial comments, will return for a more thorough review when this is taken out of draft.

1egoman

I saw you opened this so I left some initial thoughts, feel free to ignore them for now if it's still early and not yet ready for a review.

MaxHeimbrock · 2026-07-24T14:11:24Z

Potential follow up: Mint tokens based on API secret

MaxHeimbrock · 2026-07-26T17:12:21Z

@ladvoc this thing is already minting tokens from api key and api secret right? Are we using this in the rust-dev-client?

github-actions · 2026-08-04T13:33:59Z

Changeset ✓

This PR includes a changeset covering all affected packages:

Package	Bump
`livekit-token-source`	`minor`

1egoman

Really nice work! I've left a bunch of small / nitpicky comments which I noticed when going through, but I think this generally makes sense to me.

One higher level / larger question: I see you decided to punt the TokenSourceMinter question to the future. Totally fine with me, but I am just raising that because I think @ladvoc might have a use case for it in livekit-capture. It might be worth chatting with him about it.

1egoman · 2026-08-05T14:22:00Z

+        let response = http_client.post(self.endpoint_url.clone(), headers, body).await?;
+
+        if !(200..300).contains(&response.status) {


thought: This isn't in this code, but seeing this makes me realize we really should have a .ok() method on response which encapsulates this (200..300).contains(&response.status) check. Is that something you'd be willing to add to livekit-net? Or I could maybe do it as a follow up if you want.

Maybe you can draft it?

I can do it as a follow up, np.

1egoman · 2026-08-05T14:37:22Z

+/// The return type of [`TokenSource::endpoint`].
+pub struct TokenSourceEndpoint {
+    endpoint_url: String,
+    headers: Vec<(String, String)>,
+}
+
+#[async_trait]
+impl TokenSourceConfigurable for TokenSourceEndpoint {
+    async fn fetch(
+        &self,
+        options: &TokenSourceFetchOptions,
+    ) -> TokenSourceResult<TokenSourceResponse> {
+        let request = TokenSourceRequest::from(options);
+


thought: In the web token source implementation, there is caching which is done to avoid re-fetching a new token when the previous token is valid. IMO, this is really useful, and can be a significant slowdown when a user is initially connecting to a room to not have this. It also allows for pre-emptive token fetching (either on app load, or after a disconnect to make repeated connect / disconnect cycles fast).

I know how exactly this caching is done differs between the implementations (Web does it within the TokenSourceEndpoint via an internal abstract class it extends, swift has a TokenSourceCached, etc) but IMO it would be worth coming up with a strategy of some sort on how to handle this (I don't have a strong preference how it should work).

t's somewhat arguable whether adding it would be a breaking change in the interface IMO, adding caching later on obviously wouldn't break the existing interface but also on the other hand, a user may build around assuming that a token will always be regenerated for each room connection and adding caching would break that assumption. That's why I would be in favor of adding this now, because then there's no risk of a user accidentally building around this assumption and making adding it later in a fully backwards compatible way challenging.

Thanks for your opinion, I wanted to implement it later. Let me see how I can add it now already though.

One thing I want to note is that for agent use cases, I have experienced caching causing issues. If you leave a room and there was an agent dispatched, calling again with caching active will put you in the same room. But because you left before, the agent is being torn down and is also not joining again. So for agent use cases, I think caching should be discouraged with our current logic on dispatch.

I think the way the syntax we mostly use is the livekit_token_source.development_token_server("id").cached(). I don't see this as a breaking change if we add that later. I will still see if I can add the caching in this PR already.

I stacked the .cached() implementation against this PR to keep things separate for easier reviews: #1318

ladvoc

Looking clean overall! Just some minor comments and suggestions.

ladvoc · 2026-08-05T19:48:21Z

Note: before publishing this crate, I need to setup trusted publishing to prevent a recurrence of last week's release issue; on my list, just haven't gotten around to it yet.

1egoman

Generally looks good to me!

ladvoc · 2026-08-10T17:42:24Z

@ladvoc this thing is already minting tokens from api key and api secret right? Are we using this in the rust-dev-client?

Yes, I definitely think we should integrate it. I am also planning to integrate this in the LiveKit Capture CLI I am currently working on (more details on that to come).

# This is the 1st commit message: Started with token source # This is the commit message #2: Before http

Before http Moved everything into a modular crate Without borrow seems to work Token source with tokio async Move reqwest into workspace Token fetching from sandbox works TokenSourceEndpoint works the same as sandbox Using composition for sandbox Error status codes are working Remove room mod change Version 1 done, works like Unity First round of review changes done Switching to livekit-net crate instead of reqwest Revert putting reqwest dependency in workspace Remove test errors Renaming to development token server

The versioned_files dependency entry named livekit-net, so releasing livekit-token-source would have rewritten the livekit-net version pin in the workspace Cargo.toml instead of its own. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…traits Mirrors the JS SDK's TokenSourceFixed / TokenSourceConfigurable split: fixed sources fetch without options, configurable sources take TokenSourceFetchOptions. Endpoint and DevelopmentTokenServer implement Configurable; Literal implements Fixed and now stores the response directly, removing the never-constructed Err arm. This gives generic call sites (a future Room::connect) and custom credential backends a common interface. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Matches the JS SDK's TokenSourceFetchOptions field name; the wire mapping (room_config.agents[0].deployment) is unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

… tolerate camelCase responses The documented endpoint contract is snake_case; serde aliases add the same leniency the JS SDK gets from proto3 fromJson, which also accepts camelCase field names. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Ports the JS SDK doc comments, including the production-use warning on the development token server, and adds crate-level docs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Repo convention (livekit, livekit-api) is to ship no TLS in defaults and let consumers opt in; the example now enables rustls-tls-native-roots explicitly and uses the workspace dependency like the other examples. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Removes the committed live sandbox id in favor of LIVEKIT_SANDBOX_ID and tidies the output. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Cargo.toml declares readme = "README.md" but the file was empty. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>

… sandbox id These belong to the base token-source work rather than the caching layer: README and doc links updated for the free-function factories, the README example fixed to the current endpoint() signature, a stale camelCase-alias comment removed, and the real sandbox id scrubbed from the example. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

@MaxHeimbrock

> [!IMPORTANT] > Merging this pull request will create these releases # livekit-ffi 0.12.75 (2026-08-10) ## Features - Add `other_sdks` field to propagate additional SDK metadata to the server. ## Fixes - Only advertise internal H264 decode formats if the decoder works - #1313 (@MaxHeimbrock) # webrtc-sys 0.3.42 (2026-08-10) ## Fixes - Only advertise internal H264 decode formats if the decoder works - #1313 (@MaxHeimbrock) # livekit-api 0.6.3 (2026-08-10) ## Features - Add `other_sdks` field to propagate additional SDK metadata to the server. # livekit 0.8.3 (2026-08-10) ## Features - Add `other_sdks` field to propagate additional SDK metadata to the server. ## Fixes - Only advertise internal H264 decode formats if the decoder works - #1313 (@MaxHeimbrock) # livekit-token-source 0.1.1 (2026-08-10) ## Features - Add the `livekit-token-source` crate: token sources for procuring LiveKit credentials, mirroring the JS SDK's `TokenSource` — `literal`, `endpoint` (standard token endpoint format), and `development_token_server`, plus `TokenSourceFixed` / `TokenSourceConfigurable` traits for custom backends. ## Fixes - Add a TokenSource crate to the Rust SDKs - #1274 (@MaxHeimbrock) # libwebrtc 0.3.45 (2026-08-10) ## Fixes - Only advertise internal H264 decode formats if the decoder works - #1313 (@MaxHeimbrock) Co-authored-by: knope-bot[bot] <152252888+knope-bot[bot]@users.noreply.github.com>

MaxHeimbrock mentioned this pull request Jul 22, 2026

Include new Rust token source livekit-examples/rust-dev-client#36

Open

ladvoc reviewed Jul 22, 2026

View reviewed changes

Comment thread livekit-token-source/src/request.rs

Comment thread livekit-token-source/Cargo.toml Outdated

Comment thread livekit-token-source/src/token_source.rs Outdated

1egoman reviewed Jul 22, 2026

View reviewed changes

MaxHeimbrock commented Jul 24, 2026

View reviewed changes

Comment thread examples/token_source/src/main.rs Outdated

MaxHeimbrock force-pushed the max/token-source branch 4 times, most recently from f690b60 to e63f0ef Compare August 3, 2026 10:09

MaxHeimbrock force-pushed the max/token-source branch from dea4dce to 1d0d082 Compare August 3, 2026 14:54

MaxHeimbrock force-pushed the max/token-source branch from 81e7a4d to 30aaa7e Compare August 4, 2026 13:53

MaxHeimbrock marked this pull request as ready for review August 5, 2026 11:35

This comment was marked as resolved.

Sign in to view

1egoman approved these changes Aug 5, 2026

View reviewed changes

ladvoc approved these changes Aug 5, 2026

View reviewed changes

MaxHeimbrock force-pushed the max/token-source branch from 38d82ab to 0ba1275 Compare August 6, 2026 14:00

This comment was marked as resolved.

Sign in to view

MaxHeimbrock force-pushed the max/token-source branch from 0ba1275 to 69e463a Compare August 6, 2026 14:22

This comment was marked as resolved.

Sign in to view

MaxHeimbrock force-pushed the max/token-source branch from c5a0f83 to fc4e8ff Compare August 6, 2026 14:55

1egoman approved these changes Aug 6, 2026

View reviewed changes

Comment thread examples/token_source/token.json

Comment thread livekit-token-source/README.md Outdated

MaxHeimbrock force-pushed the max/token-source branch 2 times, most recently from 1837f99 to 084ea8d Compare August 7, 2026 10:01

MaxHeimbrock added 2 commits August 10, 2026 10:52

# This is a combination of 2 commits.

930263d

# This is the 1st commit message: Started with token source # This is the commit message #2: Before http

MaxHeimbrock and others added 23 commits August 10, 2026 10:52

Using static factory pattern

8ea3a2e

Add license headers

f5eb077

Adding to knope.toml

243a369

Changeset

a8f9e6e

style: run cargo fmt on livekit-token-source and example

1ecbe14

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

refactor(token-source): rename agent_deployment option to deployment

c23509e

Matches the JS SDK's TokenSourceFetchOptions field name; the wire mapping (room_config.agents[0].deployment) is unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

docs(token-source): document the public API

57dc5ab

Ports the JS SDK doc comments, including the production-use warning on the development token server, and adds crate-level docs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

chore(token-source): clean up example, read sandbox id from env

9e6f873

Removes the committed live sandbox id in favor of LIVEKIT_SANDBOX_ID and tidies the output. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

test(token-source): cover agent options to room_config request nesting

8c55367

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

docs(token-source): write the crate README

c016e75

Cargo.toml declares readme = "README.md" but the file was empty. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

chore: add code owner for livekit-token-source

c1d5746

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

chore: add changeset for livekit-token-source

8356be5

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Update livekit-token-source/src/error.rs

72eee03

Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>

Update .github/CODEOWNERS

3927efd

Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>

First batch of comment addressing

3da9215

More comments addressed

eaccdf3

fmt

4b420a3

Addressing Ryans comments

0631484

ladvoc force-pushed the max/token-source branch from 084ea8d to 0631484 Compare August 10, 2026 17:52

ladvoc merged commit c5f92db into main Aug 10, 2026
24 of 25 checks passed

ladvoc deleted the max/token-source branch August 10, 2026 18:30

knope-bot Bot mentioned this pull request Aug 10, 2026

Release packages #1316

Merged

		let response = http_client.post(self.endpoint_url.clone(), headers, body).await?;

		if !(200..300).contains(&response.status) {

Uh oh!

Conversation

MaxHeimbrock commented Jul 22, 2026

Before you submit your PR

PR description

Testing

Async

Uh oh!

ladvoc left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

1egoman left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

MaxHeimbrock commented Jul 24, 2026

Uh oh!

MaxHeimbrock commented Jul 26, 2026

Uh oh!

github-actions Bot commented Aug 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changeset ✓

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

1egoman left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

1egoman Aug 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

MaxHeimbrock Aug 5, 2026

Choose a reason for hiding this comment

Uh oh!

1egoman Aug 5, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

1egoman Aug 5, 2026

Choose a reason for hiding this comment

Uh oh!

MaxHeimbrock Aug 5, 2026

Choose a reason for hiding this comment

Uh oh!

MaxHeimbrock Aug 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

MaxHeimbrock Aug 6, 2026

Choose a reason for hiding this comment

Uh oh!

ladvoc left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Aug 4, 2026 •

edited

Loading

1egoman Aug 5, 2026 •

edited

Loading

MaxHeimbrock Aug 6, 2026 •

edited

Loading

ladvoc commented Aug 5, 2026 •

edited

Loading