Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
70 commits
Select commit Hold shift + click to select a range
a1d7dc0
starting work on blob versioning. Added account model and modified cr…
rorozcov Aug 6, 2025
e394be7
working tests with successful blob create with versioning
rorozcov Aug 7, 2025
a84fc37
improved account model usage outside store
rorozcov Aug 8, 2025
2aa7cfb
working test. now I need to add more tests and develop rest of functi…
rorozcov Aug 8, 2025
d4a8080
went back and added versioning support to all of loki blob metadata s…
rorozcov Aug 9, 2025
0571013
updated files to get build working. looking getting db tests working
rorozcov Aug 9, 2025
70f15cb
basic tests and project builds. Must audit then make more tests
rorozcov Aug 9, 2025
ef14de9
ccaught bugs. Basic tests passing. Added tests to new utils
rorozcov Aug 9, 2025
fb63f7d
addressed bugs, added more versioning tests
rorozcov Aug 10, 2025
903fab5
more tests and bugs fixed. More work to do before API changes
rorozcov Aug 10, 2025
1279659
added disable to enable tests. Missing enable to disable and listblobs
rorozcov Aug 10, 2025
ed70b4c
squashed more bugs on blob operations. ready for v1. missing list blo…
rorozcov Aug 10, 2025
db20834
fixed bugs and added more tests to verify delete blob and snapshot logic
rorozcov Aug 10, 2025
94d91fb
updated APIs. Tests still pass. Time for APi testing
rorozcov Aug 11, 2025
056d639
fixing handler tests
rorozcov Aug 11, 2025
721e60d
fixed copy bugs. missing block blob operations bugs
rorozcov Aug 11, 2025
7b7f33a
fixed bugs and now all pre-existing block blob tests are passing
rorozcov Aug 12, 2025
8c57e6b
all tests passing
rorozcov Aug 19, 2025
7350ef8
Adding appendblob tests. They need auditing
rorozcov Aug 20, 2025
d24739c
added pageblob versioning tests, but only copy pasted non-versioning …
rorozcov Aug 20, 2025
fe3fafb
finished versioning enabled pageblob tests
rorozcov Aug 22, 2025
82354f4
prod comparison tests
rorozcov Aug 24, 2025
301b646
fixed bug in properties and added parity tests against storage account
rorozcov Aug 25, 2025
36bf787
adding mutually exclusive check and tests for it
rorozcov Aug 25, 2025
56e6f57
adding valid date checks
rorozcov Aug 25, 2025
25457ca
allow iso range due to js vs blob storage differences
rorozcov Aug 25, 2025
69cb5aa
renaming to prod
rorozcov Aug 25, 2025
0edbad6
fixed API return values and added parity tests on azurite
rorozcov Aug 25, 2025
63ae1a7
appendblob now checks versionId is undefined when versioning is off
rorozcov Aug 26, 2025
ac5867e
Adding basic blob tests against versionId being undefined when versio…
rorozcov Aug 26, 2025
d07c1ac
Revert "appendblob now checks versionId is undefined when versioning …
rorozcov Aug 26, 2025
a363e72
adding basic versionId undefined test
rorozcov Aug 26, 2025
9888af9
added listblob versioning logic. missing full test suite with version…
rorozcov Aug 26, 2025
bb18539
refactoring lokidb testing files. next step: add list blob files
rorozcov Aug 26, 2025
0625af0
added lokidb versioning tests for listBlobs
rorozcov Aug 26, 2025
481b636
massively improved block blob API tests with meaningful test cases
rorozcov Aug 26, 2025
8c63db7
cleaner append blob tests
rorozcov Aug 26, 2025
0378faa
same for pageblob
rorozcov Aug 26, 2025
d6f607f
removing settings.json
rorozcov Aug 26, 2025
4af1312
Switched completely to blob config filejson/path
rorozcov Aug 27, 2025
b6fbadc
adding comment about model compat
rorozcov Aug 27, 2025
e19ff17
fixing package.json
rorozcov Aug 27, 2025
eda253a
removing formatting changes to improve reviewability
rorozcov Aug 27, 2025
4d7b5ac
adding env functions tests
rorozcov Aug 27, 2025
44143e0
removing snapshot string
rorozcov Aug 27, 2025
5cf4060
adding new parity test. IfNoneMatch should always fail to overwrite, …
rorozcov Sep 7, 2025
c52aa46
Updating readme
rorozcov Sep 7, 2025
655468c
updating readme
rorozcov Sep 12, 2025
87e8d25
added versioning to paginated listBlobs and filterBlobs
rorozcov Sep 28, 2025
5b28f1b
Finished pagewithdelimeter logic
rorozcov Sep 29, 2025
966a38d
Multi-account support works in unit testing. Missing manual testing
rorozcov Dec 2, 2025
0413872
manually tested. Multi-account works
rorozcov Dec 3, 2025
d21d5ed
removing comment
rorozcov Dec 3, 2025
363339b
Adding comment back
rorozcov Dec 6, 2025
2c338c6
addressing api query param validation comment
rorozcov Dec 6, 2025
7fc72de
added comment on blob insertion
rorozcov Dec 6, 2025
4b13904
improving account model init, adding required comment
rorozcov Dec 6, 2025
b8e96f1
appendblob test comments addressed
rorozcov Dec 6, 2025
8e36ede
blob test comments addressed
rorozcov Dec 6, 2025
2dc9daf
addressing more comments
rorozcov Dec 7, 2025
d5318c2
updating client in use
rorozcov Dec 7, 2025
a24fa0c
adding docs
rorozcov Dec 7, 2025
29a371c
filter blobs ignore tags in previous versions and only uses current. …
rorozcov Dec 7, 2025
840fc68
matching prod behaviour
rorozcov Dec 7, 2025
18cdd51
verified deletion behaviours
rorozcov Dec 7, 2025
1f9f80d
addressing comments
rorozcov Dec 9, 2025
12a07e5
Merge upstream main into blob versioning branch
rorozcov Aug 13, 2026
5814af5
Merge latest upstream main
rorozcov Aug 13, 2026
72ac584
Improve blob versioning compatibility and parity
rorozcov Aug 13, 2026
bb5a781
Address blob versioning review feedback
rorozcov Aug 13, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
95 changes: 92 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -207,6 +207,8 @@ Following extension configurations are supported:
- `azurite.disableProductStyleUrl` Force parsing storage account name from request URI path, instead of from request URI host.
- `azurite.inMemoryPersistence` Disable persisting any data to disk. If the Azurite process is terminated, all data is lost.
- `azurite.extentMemoryLimit` When using in-memory persistence, limit the total size of extents (blob and queue content) to a specific number of megabytes. This does not limit blob, queue, or table metadata. Defaults to 50% of total memory.
- `azurite.accountConfigFilePath` Path to a JSON file containing AccountModel configuration for blob versioning settings. See [Use Blob Versioning](#use-blob-versioning) for details.
- `azurite.accountConfigAsJson` Inline JSON string containing AccountModel configuration for blob versioning settings. See [Use Blob Versioning](#use-blob-versioning) for details.
- `azurite.disableTelemetry` Disable telemetry data collection of this Azurite execution. By default, Azurite will collect telemetry data to help improve the product.

### [DockerHub](https://hub.docker.com/_/microsoft-azure-storage-azurite)
Expand Down Expand Up @@ -240,7 +242,7 @@ docker run -p 10000:10000 -p 10001:10001 -v c:/azurite:/data mcr.microsoft.com/a
#### Customize all Azurite V3 supported parameters for docker image

```bash
docker run -p 7777:7777 -p 8888:8888 -p 9999:9999 -v c:/azurite:/workspace mcr.microsoft.com/azure-storage/azurite azurite -l /workspace -d /workspace/debug.log --blobPort 7777 --blobHost 0.0.0.0 --blobKeepAliveTimeout 5 --queuePort 8888 --queueHost 0.0.0.0 --queueKeepAliveTimeout 5 --tablePort 9999 --tableHost 0.0.0.0 --tableKeepAliveTimeout 5 --loose --skipApiVersionCheck --disableProductStyleUrl --disableTelemetry
docker run -p 7777:7777 -p 8888:8888 -p 9999:9999 -v c:/azurite:/workspace mcr.microsoft.com/azure-storage/azurite azurite -l /workspace -d /workspace/debug.log --blobPort 7777 --blobHost 0.0.0.0 --blobKeepAliveTimeout 5 --queuePort 8888 --queueHost 0.0.0.0 --queueKeepAliveTimeout 5 --tablePort 9999 --tableHost 0.0.0.0 --tableKeepAliveTimeout 5 --loose --skipApiVersionCheck --disableProductStyleUrl --accountConfigAsJson "{\"isBlobVersioningEnabled\":true}" --disableTelemetry
```

Above command will try to start Azurite image with configurations:
Expand Down Expand Up @@ -273,6 +275,10 @@ Above command will try to start Azurite image with configurations:

`--disableProductStyleUrl` force parsing storage account name from request URI path, instead of from request URI host.

`--accountConfigFilePath /workspace/accountModel.json` configures blob versioning using an AccountModel JSON file mapped to the docker workspace. See [Use Blob Versioning](#use-blob-versioning) for details.

`--accountConfigAsJson "{\"isBlobVersioningEnabled\":true}"` configures blob versioning using an inline JSON string. See [Use Blob Versioning](#use-blob-versioning) for details.

`--azurite.disableTelemetry` disable telemetry data collection of this Azurite execution. By default, Azurite will collect telemetry data to help improve the product.

> If you use customized azurite parameters for docker image, `--blobHost 0.0.0.0`, `--queueHost 0.0.0.0` are required parameters.
Expand Down Expand Up @@ -504,6 +510,86 @@ server thus resetting the storage completely.
Note that if many hundreds of megabytes of content (queue message or blob content) are stored in-memory, it can take
noticeably longer than usual for the process to terminate since all the consumed memory needs to be released.

### Use Blob Versioning

#### How it works

Blob Versioning was implemented to follow the exact guidelines outlined in the [Azure Blob Storage versioning documentation](https://learn.microsoft.com/en-us/azure/storage/blobs/versioning-overview), excluding interactions with soft delete, blob expiration, and SAS URIs. For detailed implementation information, see the [blob versioning design document](docs/designs/2025-12-blob-versioning.md).

#### How to use it

##### Single Account support

Optional. By default, this is disabled. To enable it, there are two CLI args you can use: accountConfigFilePath, accountConfigAsJson.

Blob versioning is enabled by leveraging the [AccountModel](src/common/account/AccountModel.ts). The account model is an abstraction to configure the storage account. Currently, it only supports configuring blob versioning.

accountConfigFilePath lets you pass in the path to a json file modeled after the AccountModel, which is then used to configure Azurite.

```bash
azurite --accountConfigFilePath "./myAccountModel.json"
```

Example contents of `myAccountModel.json`:

```json
{
"isBlobVersioningEnabled": true
}
```

accountConfigAsJson allows you to pass a json string as a CLI arg to configure the account as well.

```bash
azurite --accountConfigAsJson "{ \"isBlobVersioningEnabled\": true }"
```

By default, Azurite will always use whatever version of the account model already exists in its databases. However, if any of these parameters are passed in and are valid, the existing account model will be overwritten.

##### Multi-account AccountModel support

Both `accountConfigFilePath` and `accountConfigAsJson` support configuring multiple accounts with different versioning settings.

**Using accountConfigFilePath with multiple accounts:**

```bash
azurite --accountConfigFilePath "account1:/path/to/config1.json,account2:/path/to/config2.json"
```

Where `config1.json` might contain:

```json
{
"isBlobVersioningEnabled": true
}
```

And `config2.json` might contain:

```json
{
"isBlobVersioningEnabled": false
}
```

**Using accountConfigAsJson with multiple accounts:**

```bash
azurite --accountConfigAsJson "account1:{\"isBlobVersioningEnabled\":true},account2:{\"isBlobVersioningEnabled\":false}"
```

**Backward compatibility:**

For single-account configuration, you can omit the account name prefix (defaults to `devstoreaccount1`):

```bash
azurite --accountConfigFilePath "./myAccountModel.json"
# or
azurite --accountConfigAsJson "{\"isBlobVersioningEnabled\":true}"
```

> **Important:** Declaring an account in the AccountModel configuration only sets the versioning behavior for that account. You still need to configure authentication for these accounts using the `AZURITE_ACCOUNTS` environment variable (see [Customized Storage Accounts & Keys](#customized-storage-accounts--keys-1)) to actually use them. Without proper authentication setup, requests to these accounts will fail authentication. **Furthermore, if you want to configure all accounts, you must configure each account individually. If you follow the single-account flow or configure only one account, the other accounts will not be configured.**

### Command Line Options Differences between Azurite V2

Azurite V3 supports SharedKey, Account Shared Access Signature (SAS), Service SAS, OAuth, and Public Container Access authentications, you can use any Azure Storage SDKs or tools like Storage Explorer to connect Azurite V3 with any authentication strategy.
Expand Down Expand Up @@ -1039,14 +1125,17 @@ Latest release targets **2025-11-05** API version **blob** service.
Detailed support matrix:

- Supported Vertical Features

- CORS and Preflight
- SharedKey Authentication
- OAuth authentication
- Shared Access Signature Account Level
- Shared Access Signature Service Level (Not support response header override in service SAS)
- Container Public Access
- Blob Tags (preview)
- Blob versioning (Only in LokiDb instances of Azurite, which is the default. Does not support SAS URIs)
- Supported REST APIs

- List Containers
- Set Service Properties
- Get Service Properties
Expand Down Expand Up @@ -1081,7 +1170,6 @@ Detailed support matrix:
- Soft delete & Undelete Blob
- Incremental Copy Blob
- Blob Query
- Blob Versions
- Blob Last Access Time
- Concurrent Append
- Blob Expiry
Expand All @@ -1092,6 +1180,7 @@ Detailed support matrix:
- Encryption Scope
- Get Page Ranges Continuation Token
- Blob Immutability Policy and Legal Hold
- SAS URIs for Blob Versions

Latest version supports for **2025-11-05** API version **queue** service.
Detailed support matrix:
Expand Down Expand Up @@ -1175,4 +1264,4 @@ provided by the bot. You will only need to do this once across all repos using o

This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/).
For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or
contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments.
contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments.
100 changes: 100 additions & 0 deletions docs/designs/2025-12-blob-versioning.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
# Add Blob Versioning Support

- Author Name: Rodolfo Orozco Vasquez ([@rorozcov](https://github.com/rorozcov))
- GitHub Issue: [Azure/Azurite#665](https://github.com/Azure/Azurite/issues/665)

## Summary

This design adds support for Azure Blob Storage versioning to Azurite, allowing blobs to maintain previous versions automatically when they are modified or deleted. Versioning is implemented following the [Azure Blob Storage versioning guidelines](https://learn.microsoft.com/en-us/azure/storage/blobs/versioning-overview) as closely as possible, with some limitations based on features not yet supported in Azurite.

## Motivation

Blob versioning is a critical feature in Azure Blob Storage that automatically maintains previous versions of a blob. This enables users to:

- Recover from accidental blob modifications or deletions
- Maintain a history of blob changes over time
- Access and restore previous versions of blobs

Without blob versioning support in Azurite, developers cannot fully test applications that rely on this feature locally, limiting their ability to validate version-aware workflows before deploying to Azure.

## Explanation

### Functional explanation

Blob versioning in Azurite is controlled through the `AccountModel` abstraction, which allows configuration of storage account-level settings. When enabled, blob versioning automatically creates a new version of a blob whenever it is modified or overwritten.

Two command line options are available to configure blob versioning:

1. **`--accountConfigFilePath`** - Path to a JSON configuration file
2. **`--accountConfigAsJson`** - Inline JSON string configuration

We also support multi-account configuration since Azurite supports multiple accounts.

### Technical explanation

Blob versioning is implemented using the `AccountModel` type which is stored in the metadata database:

```typescript
export interface AccountModel {
key: string;
isBlobVersioningEnabled: boolean;
}
```

When versioning is enabled for an account:

- **For block blobs:** All write operations trigger the creation of a new version, except for the Put Block operation
- **For page blobs and append blobs:** Only a subset of write operations triggers version creation:
- Put Blob
- Put Block List
- Set Blob Metadata
- Copy Blob
- **Operations that do NOT trigger version creation:**
- Put Page (page blob)
- Append Block (append blob)
- Each version is assigned a unique version ID in ISO 8601 date-time format
- Azurite emits the same 7 fractional digits used by Azure. The first 3 digits come from JavaScript millisecond precision, and the final 4 digits provide a per-blob sub-millisecond counter so version IDs remain unique.
- Previous versions are immutable and can be accessed using the version ID
- The `List Blobs` operation can include versions when the `includeVersions` parameter is set to true
- Specific versions can be retrieved, downloaded, or deleted using the `versionId` query parameter

The configuration is parsed through `EnvironmentFunctions.parseAccountModelFlags()` which supports:

- Single account configuration
- Multi-account configuration with comma-separated entries
- Both file-based and inline JSON configurations
- Proper JSON parsing with support for nested objects and escaped characters

### Integration with Authentication

> **Important:** The `AccountModel` configuration only controls the versioning behavior. To actually use the configured accounts, they must also be set up in the `AZURITE_ACCOUNTS` environment variable for authentication. See [Customized Storage Accounts & Keys](https://github.com/Azure/Azurite#customized-storage-accounts--keys-1) for details.

### Limitations

The following Azure Blob Storage versioning features are **not** currently supported:

- Soft delete integration with versioning
- Blob expiration with versioning
- SAS URIs for specific blob versions
- Version-level immutability policies (Version Level WORM)

### VS Code Extension Support

Similar configuration options are available in the VS Code extension settings:

- `azurite.accountConfigFilePath` - Path to account configuration file
- `azurite.accountConfigAsJson` - Inline JSON configuration string

## Azure Documentation on Blob Versioning

This implementation follows the Azure Blob Storage versioning specification as documented in the official Microsoft documentation:

- [Blob versioning overview](https://learn.microsoft.com/en-us/azure/storage/blobs/versioning-overview)
- [Enable and manage blob versioning](https://learn.microsoft.com/en-us/azure/storage/blobs/versioning-enable)

The design aligns with Azure's behavior where:

- Versioning is a storage account-level setting
- Version IDs are automatically assigned timestamps
- Previous versions are immutable
- The current version is mutable
10 changes: 10 additions & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -275,6 +275,16 @@
"type": "boolean",
"default": false,
"description": "Disable telemetry data collection of this Azurite execution. By default, Azurite will collect telemetry data to help improve the product."
},
"azurite.accountConfigFilePath": {
"type": "string",
"default": null,
"description": "Path to the account configuration file. This file contains account-specific settings. Mutually exclusive with azurite.accountConfigAsJson."
},
"azurite.accountConfigAsJson": {
"type": "string",
"default": null,
"description": "Account configuration settings in JSON format. Mutually exclusive with azurite.accountConfigFilePath."
}
}
}
Expand Down
12 changes: 11 additions & 1 deletion src/azurite.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ import { dirname, join } from "path";

// Load Environment before BlobServerFactory to make sure args works properly
import Environment from "./common/Environment";
import LokiAccountModelStore from "./common/account/LokiAccountModelStore";
// tslint:disable-next-line:ordered-imports
import { BlobServerFactory } from "./blob/BlobServerFactory";

Expand All @@ -18,6 +19,7 @@ import {
} from "./queue/utils/constants";
import SqlBlobServer from "./blob/SqlBlobServer";
import BlobServer from "./blob/BlobServer";
import { DEFAULT_ACCOUNT_MODEL_LOKI_DB_PATH } from "./blob/utils/constants";

import TableConfiguration from "./table/TableConfiguration";
import TableServer from "./table/TableServer";
Expand Down Expand Up @@ -97,8 +99,16 @@ async function main() {
await access(dirname(debugFilePath!));
}

// Create account model store
const accountModels = env.getAccountModels();
const accountModelStore = new LokiAccountModelStore(
join(location, DEFAULT_ACCOUNT_MODEL_LOKI_DB_PATH),
env.inMemoryPersistence(),
accountModels
);

const blobServerFactory = new BlobServerFactory();
const blobServer = await blobServerFactory.createServer(env);
const blobServer = await blobServerFactory.createServer(env, accountModelStore);
const blobConfig = blobServer.config;

// TODO: Align with blob DEFAULT_BLOB_PERSISTENCE_ARRAY
Expand Down
4 changes: 3 additions & 1 deletion src/blob/BlobConfiguration.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
import ConfigurationBase from "../common/ConfigurationBase";
import { StoreDestinationArray } from "../common/persistence/IExtentStore";
import { MemoryExtentChunkStore } from "../common/persistence/MemoryExtentStore";
import IAccountModelStore from "../common/account/IAccountModelStore";
import {
DEFAULT_BLOB_EXTENT_LOKI_DB_PATH,
DEFAULT_BLOB_LISTENING_PORT,
Expand Down Expand Up @@ -45,6 +46,7 @@ export default class BlobConfiguration extends ConfigurationBase {
disableProductStyleUrl: boolean = false,
public readonly isMemoryPersistence: boolean = false,
public readonly memoryStore?: MemoryExtentChunkStore,
public readonly accountModelStore?: IAccountModelStore,
) {
super(
host,
Expand All @@ -60,7 +62,7 @@ export default class BlobConfiguration extends ConfigurationBase {
key,
pwd,
oauth,
disableProductStyleUrl
disableProductStyleUrl,
);
}
}
16 changes: 15 additions & 1 deletion src/blob/BlobEnvironment.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@ import { access, ensureDir } from "fs-extra";
import { dirname } from "path";

import IBlobEnvironment from "./IBlobEnvironment";
import { parseAccountModelFlags } from "../common/EnvironmentFunctions";
import { AccountModel } from "../common/account/AccountModel";
import {
DEFAULT_BLOB_LISTENING_PORT,
DEFAULT_BLOB_SERVER_HOST_NAME,
Expand Down Expand Up @@ -70,6 +72,14 @@ if (!(args as any).config.name) {
.option(
["", "disableTelemetry"],
"Optional. Disable telemetry data collection of this Azurite execution. By default, Azurite will collect telemetry data to help improve the product."
)
.option(
["", "accountConfigFilePath"],
"Optional. Path to the account configuration file"
)
.option(
["", "accountConfigAsJson"],
"Optional. Account configuration in JSON format"
);

(args as any).config.name = "azurite-blob";
Expand Down Expand Up @@ -187,4 +197,8 @@ export default class BlobEnvironment implements IBlobEnvironment {

// By default disable debug log
}
}

public getAccountModels(): Map<string, AccountModel> | undefined {
return parseAccountModelFlags(this.flags);
}
}
Loading