Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
a1a6fe1
July demo code: refactored reader, updated dependencies, updated tests
cwlacewe Jul 2, 2026
3d1e983
[Docs] Update pipeline.md; [Fastapi] clean launch.json, handlers.py, …
cwlacewe Jul 16, 2026
75fcf4c
Code updates for AUG demo; Includes updated tests
cwlacewe Aug 31, 2026
fdcbc8f
Remove unused code (front end, vdms, udf, and cpu). The pipeline only…
cwlacewe Sep 9, 2026
0c1826f
July demo code: refactored reader, updated dependencies, updated tests
cwlacewe Jul 2, 2026
2e2729e
[Docs] Update pipeline.md; [Fastapi] clean launch.json, handlers.py, …
cwlacewe Jul 16, 2026
e656a9b
Code updates for AUG demo; Includes updated tests
cwlacewe Aug 31, 2026
12ca5ee
Remove unused code (front end, vdms, udf, and cpu). The pipeline only…
cwlacewe Sep 9, 2026
9567c1d
Merge branch 'cwl_eval_opt' of https://github.com/IntelLabs/Video-Cur…
cwlacewe Sep 9, 2026
cad912d
Remove codespell
cwlacewe Sep 9, 2026
ae9d282
Update dependencies
cwlacewe Sep 10, 2026
6931521
Remove unused files
cwlacewe Sep 10, 2026
0d0b50d
Update docs
cwlacewe Sep 10, 2026
9ae15af
Remove old note
cwlacewe Sep 10, 2026
6ff0325
update pipeline png image
cwlacewe Sep 10, 2026
8926713
Update pipeline and fine-tune docs
cwlacewe Sep 10, 2026
b27c6fa
Make GPU default device in Dockerfiles, and update finetune files
cwlacewe Sep 10, 2026
5363ef1
Update details regarding model and sample model location
cwlacewe Sep 14, 2026
2698287
Update file links
cwlacewe Sep 14, 2026
952bbe5
Update model links
cwlacewe Sep 15, 2026
4882075
Merge branch 'main' into model_updates
cwlacewe Sep 15, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 8 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,8 @@ Please see [High Resolution Object Detection Pipeline](./doc/pipeline.md) for mo
- FFmpeg is an open source project licensed under LGPL and GPL. See https://www.ffmpeg.org/legal.html. You are solely responsible for determining if your use of FFmpeg requires any additional licenses. Intel is not responsible for obtaining any such licenses, nor liable for any licensing fees due, in connection with your use of FFmpeg.


### Datasets & Attributions
- This project utilizes third-party open datasets. Please see our [Data Attributions](docs/DATASETS.md) for full licensing, copyright details, and citation parameters.
### Attributions
- This project utilizes third-party open datasets and a sample fine-tuned model. Please see our [Attributions](./doc/ATTRIBUTES.md) for full licensing, copyright details, and citation parameters.


## Install Prerequisites:
Expand All @@ -39,9 +39,14 @@ Please see [High Resolution Object Detection Pipeline](./doc/pipeline.md) for mo


## Deploy High Resolution Drone Detection using Smart Filtering
All components for this application are dockerize.
All components for this application are dockerized.
Scripts are provided to make deployment easier.


### Model Preparation
If using our sample fine-tuned model for Drone Detection, please download `drone_detection.pt` from [Video Curation Sample Models: Drone Detection](https://github.com/IntelLabs/video-curation-sample-models.git) and save it as `fastapi/resources/models/ultralytics/custom_models/drone_detection.pt`.


### Start
[Optional] To make sure there aren't any running containers for this application, run the following which stops the application and prunes containers:
```bash
Expand Down
26 changes: 17 additions & 9 deletions doc/DATASETS.md → doc/ATTRIBUTES.md
Original file line number Diff line number Diff line change
@@ -1,24 +1,32 @@
# Dataset and Third-Party Media Attributions
# Media Attributions

This repository utilizes external datasets and media for training/demo purposes, without storing the actual files.
This repository utilizes external datasets and media for training/demo purposes.
<br>

---

## 1. SynDroneVision Dataset
* **Source:** [Zenodo (Record 13360116)](https://zenodo.org/records/13360116)
* **License:** [CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/legalcode.en)
* **Citation:** See DOI [10.1109/WACV61041.2025.00742](https://doi.org/10.1109/WACV61041.2025.00742) (Lenhard et al., WACV 2025).
* **Usage:** External data for fine-tuning model for drone detection.
* **Usage:** External data for fine-tuning sample model for drone detection. Model generated using this data is available at [Video Curation Sample Models: Drone Detection](https://github.com/IntelLabs/video-curation-sample-models/tree/main/Drone_Detection).
<br>

---

## 2. Sample Drone Video (`anduril_swarm.mp4`)
## 2. Sample Drone Detection Model
* **Source:** [Video Curation Sample Models: Drone Detection](https://github.com/IntelLabs/video-curation-sample-models/tree/main/Drone_Detection)
* **License:** [AGPL 3.0](https://github.com/IntelLabs/video-curation-sample-models/tree/main/Drone_Detection/LICENSE)
* **Citation:** N/A
* **Usage:** Model fine-tuned using Ultralytics Yolo11n model and SynDroneVision dataset.
<br>


## 3. [Optional] Sample Drone Video (`anduril_swarm.mp4`)
* **Source:** [droneforge/yolov11-UAV-finetune](https://github.com/droneforge/yolov11-UAV-finetune/blob/main/anduril_swarm.mp4)
* **Usage:** External sample video for pipeline demonstration.
* **Usage:** Open sourced sample video for pipeline demonstration.
<br>

---

## 3. [Optional] DUT Anti-UAV Dataset
## 4. [Optional] DUT Anti-UAV Dataset
* **Source:** [wangdongdut/DUT-Anti-UAV](https://github.com/wangdongdut/DUT-Anti-UAV)
* **License:** [Apache License 2.0](https://github.com/wangdongdut/DUT-Anti-UAV/blob/master/LICENSE)
* **Citation:** See DOI [10.48550/arXiv.2205.10851](https://doi.org/10.48550/arXiv.2205.10851) (Zhao et al., IEEE T-ITS 2022).
Expand Down
11 changes: 6 additions & 5 deletions doc/finetune.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,15 +11,15 @@ If detecting only one class, be sure the dataset contains negative images (witho
For a well rounded dataset, be sure it contains train, validation, AND test sets to follow the expected [Ultralytics YOLO Format](https://docs.ultralytics.com/datasets/detect/).
The provided script checks if the original dataset contains `train`, `validation`, AND `test` directories.
If it does not, it proceeds with converting the dataset into this format assuming the original dataset has `images` and `labels` directory with sub-directories `train`, `validation`, AND `test`.
If your dataset does not follow this format, please modify `prepare_dataset` in `finetune.py`.
If your dataset does not follow this format, please modify `prepare_dataset` in [`finetune.py`](../finetune/app/finetune.py).

In this guide, the [SynDroneVision dataset](https://zenodo.org/records/13360116) is used.
Please see their [paper](https://ieeexplore.ieee.org/document/10943801) for more details.
The original dataset is saved in `SynDroneVision` directory and since it is not in the proper YOLO format, the script converts the data structure and saves new structure in directory `SynDroneVision_yolo`. ***NOTE:*** In other cases, if data structure is modified, the new directory prefixes the original directory name with `_yolo`.


## Training Configurations
The configurations used for training on 2x NVIDIA A100 80GB PCIe are specified in `include/train_args.py`.
The configurations used for training on 2x NVIDIA A100 80GB PCIe are specified in [`include/train_args.py`](../finetune/app/include/train_args.py).
Feel free to modify these parameters based on your hardware limitations such as VRAM of GPU.


Expand Down Expand Up @@ -49,7 +49,7 @@ THe following arguments are available:

## Deployment
Python 3.10.12 on Ubuntu 22 was used for testing.
To manually setup your environment, use the provided `requirements.txt` and `requirements.GPU.txt` files.
To manually setup your environment, use the provided [`requirements.txt`](../finetune/requirements.txt) file.
Be sure your system contains the required NVIDIA packages to use GPUs for training.

To avoid modifying your system for training, you can use the provided Dockerfile to deploy a container.
Expand Down Expand Up @@ -82,6 +82,7 @@ Please see below for instructions for deploying container via `docker` and `dock
```

Once all stages are completed, stop and/or remove running container.
<!-- docker compose down --remove-orphans -->
Keep note of the latest model, as this model will be copied to different location for inclusion in full application.

Keep note of the latest model, as this model will be copied to a different location for the detection application.

A sample model, generated using these instructions, is available in [Video Curation Sample Models: Drone Detection](https://github.com/IntelLabs/video-curation-sample-models/tree/main/Drone_Detection).
16 changes: 9 additions & 7 deletions doc/pipeline.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ The following are parameters used to fine-tune the Smart Filtering pipeline.
The current implementation of the Smart Filtering pipeline is optimized for the test use-case, drone detection.
Drones are typically small in the video frames so if your use-case of interest has different objects, it may be beneficial to test the pipeline results on an existing test video for your use-case.

For this case, we provide [`test_detections.py`](/fastapi/tests/test_detections.py) which annotates ROIs identified by the Smart Filtering pipeline onto each frame of the video for visual inspection.
For this case, we provide [`test_detections.py`](../fastapi/tests/test_detections.py) which annotates ROIs identified by the Smart Filtering pipeline onto each frame of the video for visual inspection.

For testing purposes, you can use VSCode DevContainer (easiest method) or manually deploy the fastapi dockerfile. Using VSCode is straight forward, so here, we will manually deploy the fastapi Dockerfile as it contains the same setup used in the application AND start the test script.
Here we will build the container, if not available. If behind proxy, be sure to set them using `--build-arg`.
Expand Down Expand Up @@ -186,6 +186,8 @@ Place your model in the appropriate directory for the application.
| `fastapi/resources/models/ultralytics/${MODEL_NAME}/FP16` | Ultralytics YOLO models are typically placed in this directory where MODEL_NAME is the short name for the model (i.e. `yolo11n`). The PT model (`${MODEL_NAME}.pt`) is exported to OpenVINO (`${MODEL_NAME}_openvino_model/`) or TensorRT (`${MODEL_NAME}.engine`), dependent on device used. |

Please note the model labels are retrieved from the model directly, so the model must contain these details.

A sample model is available in [Video Curation Sample Models: Drone Detection](https://github.com/IntelLabs/video-curation-sample-models/tree/main/Drone_Detection). If using this model, download and save model as `fastapi/resources/models/ultralytics/custom_models/drone_detection.pt`.
<br>


Expand Down Expand Up @@ -232,12 +234,12 @@ Here we provide details on each available test.

| Component | Test File | Description |
| --------- | --------- | ----------- |
| Model | [test_model.py](/fastapi/tests/test_model.py) | Test the model for GPU on provided RTSP URL or video file |
| Stream Readers | [test_readers.py](/fastapi/tests/test_readers.py) | Independently test the stream readers for GPU on provided RTSP URL or video file |
| Smart Filtering | [test_detections.py](/fastapi/tests/test_detections.py) | Independently test the detection pipeline (with and without Smart Filtering) only. Test does not include video clip generation or sending metadata to database for querying. |
| Stream Readers | [test_pipeline.py](/fastapi/tests/test_pipeline.py) | Scenario 1 tests the behavior of Readers when provided an invalid RTSP url.<br>Scenario 2 reads the RTSP url or video file for a specified duration or until it ends. |
| Video Clip Generation | [test_pipeline.py](/fastapi/tests/test_pipeline.py) | Scenario 3 mimics the clip generation within the pipeline. |
| Smart Filtering | [test_pipeline.py](/fastapi/tests/test_pipeline.py) | Scenario 4 tests the entire pipeline and saves output video of results. |
| Model | [test_model.py](../fastapi/tests/test_model.py) | Test the model for GPU on provided RTSP URL or video file |
| Stream Readers | [test_readers.py](../fastapi/tests/test_readers.py) | Independently test the stream readers for GPU on provided RTSP URL or video file |
| Smart Filtering | [test_detections.py](../fastapi/tests/test_detections.py) | Independently test the detection pipeline (with and without Smart Filtering) only. Test does not include video clip generation or sending metadata to database for querying. |
| Stream Readers | [test_pipeline.py](../fastapi/tests/test_pipeline.py) | Scenario 1 tests the behavior of Readers when provided an invalid RTSP url.<br>Scenario 2 reads the RTSP url or video file for a specified duration or until it ends. |
| Video Clip Generation | [test_pipeline.py](../fastapi/tests/test_pipeline.py) | Scenario 3 mimics the clip generation within the pipeline. |
| Smart Filtering | [test_pipeline.py](../fastapi/tests/test_pipeline.py) | Scenario 4 tests the entire pipeline and saves output video of results. |


### Test Model
Expand Down