Before you rent

The supported setup uses a full Vast VM with systemd, not a regular Docker instance. The VM runs a single k3s node. Check that VM offers are available, and check for a working NVIDIA driver, a registered SSH key, the price and disk space.

On a trusted operator machine you need the release manifest and a clone of the public repository github.com/sleep3r/colloq that contains its exact sourceCommit. vast-up packs the tools from exactly that commit. The app never gets your Vast account key.

In the configuration, prepare VAST_TOKEN, an SSH key and the settings of an existing relay. For private images, set VAST_REGISTRY_CONFIG with separate pull-only credentials and VAST_REGISTRY_READ_ONLY=1. The file must contain only Docker auths, with no credential helpers. A syntax check doesn't prove the credentials work — the preflight checks the pull.

Launch and monitor

For a release whose images really are available anonymously:

VAST_PUBLIC_IMAGES=1 RELEASE=/absolute/path/release.json \
  NAME=hse HOST=hse.example.edu make vast-up
NAME=hse scripts/vast.sh status
NAME=hse SINCE=2h scripts/vast.sh logs

Replace the name, domain and path. VAST_PUBLIC_IMAGES=1 is an explicit statement that the images are available, not a way to make private images public. With mixed registries, it applies to the images you pass no credentials for.

The make vast-… targets choose a path by RELEASE: without it they call the old scripts/vast-legacy.sh, meant for older Docker installs without k3s. For a k3s setup, call scripts/vast.sh directly or pass RELEASE=… to every target. App and kernel logs are saved to logs/hse/<date>/.

The tool checks the manifest before it touches your account, offers a VM and asks you to confirm the price. Only the release tools and the allowed configuration are sent. On an update the remote configuration is kept: not every change to your local .env reaches the VM.

When the GPU counts as ready

An HTTP response from the app and the output of nvidia-smi are not enough. The installer checks the RuntimeClass and allocatable GPUs and runs a real PyTorch CUDA operation in a throwaway non-root Pod. vast-up also runs the cluster smoke test with independent rooms.

A GPU room requests one device; there is no automatic sharing and no fallback to a shared kernel. The GPU check needs a free card. Passing the repository's local tests doesn't prove that a particular rental works.

Tested configuration

On September 9, 2026 we tested a dedicated Vast VM with an RTX 3090 (24 GB), Ubuntu 22.04, NVIDIA driver 580.95.05 and k3s v1.36.4+k3s1. In a GPU class, two clients got the result of a matrix multiplication in PyTorch 2.14.0 / CUDA 13.0 through a shared notebook.

We also tested room separation, a GPU tensor surviving an app and broker restart, and updating and restoring a remote class from a consistent backup. We used NVIDIA Container Toolkit 1.20.0-1 and device plugin 0.19.3. The test VM was deleted afterwards.

These results apply to that configuration. Other GPUs, external registries, a public relay and restricted access to node services need their own checks. Release updates and backup restores stop kernels; Python memory is not guaranteed to survive them.

Copy data off the VM

# While running: the database and files are not one atomic snapshot
NAME=hse MODE=live scripts/vast.sh sync

# Stop all writers and get a consistent backup
NAME=hse MODE=consistent scripts/vast.sh sync

# Explicitly allow resuming after a successful backup
NAME=hse MODE=consistent RESUME=1 scripts/vast.sh sync

Archives are saved to backups/hse/ on the operator machine. Each archive is verified on the VM and again after transfer. It contains secrets, configuration, release/catalog files and the workspace; images stay in the registry.

Colloq data lives on the VM's disk, and deleting the rental can destroy it. Don't count on local PVs being kept automatically or on being able to attach a separate Vast Volume to the VM: check the limits of the product you chose with the provider.

Restore and update

Run these commands from the repository clone on the operator machine. To update, run make vast-up again with the same NAME and a new RELEASE: on a machine with a release installed it runs cluster.sh update — a consistent backup, stopping kernels, the new release and a smoke test. On a new empty VM, vast-up itself verifies and restores the newest backup from backups/hse/; if you want a clean setup, move the backups out of that directory first. For a manual restore, follow the portable restore procedure with the right name and a compatible release, then repeat prepare and the room checks. Change the k3s version as a separate operation.

Old root-service/Compose installs and unmanaged non-empty data are not migrated automatically. Keep the old machine, export its data with the tools made for it and check the move separately.

Before you delete the rental

Get a consistent backup off the VM, check it and make sure the images you need are available. Only then run NAME=hse scripts/vast.sh down and confirm deleting that rental. A stopped cluster is not a backup.

For diagnostics, use durations such as SINCE=30m or SINCE=2h. Logs of room Pods that were already deleted may be unavailable; app and cell output may contain class data.