Vast.ai: Support spot instances#4025
Draft
trashhalo wants to merge 1 commit into
Draft
Conversation
Vast.ai offers interruptible (spot) instances, but the backend filtered
out every spot offer with an `extra_filter` and never bid on one, so only
on-demand instances could be provisioned.
Vast's `PUT /asks/{id}/` endpoint creates an interruptible instance when
the payload includes a `price` (per-machine bid in $/hour); omitting it
creates an on-demand instance. gpuhunt already emits Vast spot offers
(price set to the offer's `min_bid`), so no catalog changes are needed.
* Drop the `extra_filter` that removed spot offers in
`get_offers_by_requirements`.
* In `run_job`, bid the spot offer's price and pass it to
`create_instance`; on-demand offers pass no bid.
* Add the `price` field to the `create_instance` payload.
Interruption detection and retry are backend-agnostic (the server treats
a lost spot instance as `INTERRUPTED_BY_NO_CAPACITY`), so no further
changes are required.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Vast.ai offers interruptible (spot) instances, but the
vastaibackend has never been able to provision them.Currently,
get_offers_by_requirements()strips every spot offer:and
create_instance()never bids, so only on-demand instances are provisioned. gpuhunt already emits Vast spot offers (each on-demand offer is duplicated withpriceset to the machine'smin_bidandspot=True), so these offers reach dstack today and are then discarded.Vast's create endpoint is the same for both instance types.
PUT /asks/{id}/creates an interruptible instance when the payload includes aprice(per-machine bid in $/hour), and an on-demand instance when it is omitted. This matches the officialvast-pythonclient, whosecreate instance --bid_pricesends the samepricefield to the same endpoint.The fix:
extra_filterthat removed spot offers inget_offers_by_requirements(). Spot vs. on-demand selection is already driven by the run'sspot_policythrough the catalog query, so no offers are surfaced that the user did not ask for.run_job(), bid the spot offer's price (which gpuhunt already set tomin_bid) and pass it tocreate_instance(); on-demand offers pass no bid.pricefield to thecreate_instance()payload (Nonefor on-demand preserves the existing behavior).Interruption handling requires no backend changes: the server infers a reclaimed spot instance from the lost runner connection and, because the offer is marked
spot, terminates the job withINTERRUPTED_BY_NO_CAPACITY, which maps to the genericinterruptionretry event. This is the same path RunPod, OCI, and Nebius spot instances already use.Tested against a live Vast.ai account: spot offers carry a
min_bidbelowdph_base(e.g. an RTX 4090 at $0.24/hr on-demand vs. $0.12/hrmin_bid), and the create endpoint accepts thepricefield. Unit tests cover thatrun_job()bids on spot offers and does not bid on on-demand offers.Vast.ai API reference: https://docs.vast.ai/api/create-instance
AI assistance: This PR was written primarily by Claude Code (investigation, implementation, and tests) and reviewed by me before submitting.