Why
The current cua backend loads Cua-S1 Nano. Cua-S1 4B 0.2 uses a Qwen3.5-4B base with separate text and multimodal adapters and requires a dedicated integration.
Scope
Integrate both 0.2 adapters through the existing decision-model interface while keeping Nano available. Map the offered options and returned probabilities through the shared validation path. Verify that screenshots from the agent observation reach the multimodal adapter.
Compare Nano, 0.2 text, 0.2 multimodal, Jev, and the chat-model baseline on identical browser and desktop tasks. Improved accuracy and speed are hypotheses to test.
Acceptance criteria
- Installation and run instructions specify dependencies, model revisions, hardware, device settings, and a reproducible smoke test.
- Both adapters run through the agent. Behavioural tests cover option mapping, invalid outputs, and image delivery.
- Evaluation reports completion rate, total task time, decision latency, and failure reasons for every model on the same tasks and budgets.
- Model loading, warm inference, and peak memory use are measured separately. API charges and local compute assumptions are stated.
- Results identify where each adapter is useful and document failed or unsupported configurations.
Dependencies and references
Use the repeatable browser and desktop task suite linked below. Existing click tasks can establish an initial baseline before desktop operations expand.
Current CUA backend, decision-model interface, and Cua-S1 4B 0.2 model card.
Evaluation dependency: #13.
Why
The current cua backend loads Cua-S1 Nano. Cua-S1 4B 0.2 uses a Qwen3.5-4B base with separate text and multimodal adapters and requires a dedicated integration.
Scope
Integrate both 0.2 adapters through the existing decision-model interface while keeping Nano available. Map the offered options and returned probabilities through the shared validation path. Verify that screenshots from the agent observation reach the multimodal adapter.
Compare Nano, 0.2 text, 0.2 multimodal, Jev, and the chat-model baseline on identical browser and desktop tasks. Improved accuracy and speed are hypotheses to test.
Acceptance criteria
Dependencies and references
Use the repeatable browser and desktop task suite linked below. Existing click tasks can establish an initial baseline before desktop operations expand.
Current CUA backend, decision-model interface, and Cua-S1 4B 0.2 model card.
Evaluation dependency: #13.