SWAT+ calibration and validation on AWS EC2

Key takeaways

• Each job runs on its own dedicated EC2 machine, not a shared queue — the instance size is chosen automatically so the parallel swarm fits in cores and RAM.
• Spot vs on-demand is chosen automatically from estimated run length, and a reclaimed spot box auto-resumes on on-demand from the last checkpoint — an interruption costs a little time, never the run.
• You can watch the swarm converge live on your dashboard and stop the run whenever you judge the fit good enough — you keep the best parameters found so far, and the instance terminates the moment results sync.

How a calibration job reaches AWS

1. 1
   Bundle

   Your built model plus the pinned SWAT+ engine binary are packaged into a ~32 MB bundle on the SWATGenX server.

2. 2
   Select

   The instance size is picked from the model (cores plus the RAM needed to hold the parallel swarm), and the pricing model from the estimated run length — interruptible spot for short jobs, dedicated on-demand for jobs of roughly an hour or more so they can't be reclaimed mid-run.

3. 3
   Run & monitor

   The PSO swarm runs on the box — forward runs are evaluated in parallel across the cores, and each forward run can itself go multi-core with the opt-in parallel SWAT+ engine. Live progress streams back to your dashboard every ~20 seconds: objective metrics, the initial-stage hydrograph, and the global-best and per-particle convergence curves. You can watch the swarm converge and stop the run whenever you judge the fit is good enough — you keep the best result so far. If AWS reclaims a spot box mid-run, the job restarts automatically on an on-demand instance from the last checkpoint.

4. 4
   Fetch & terminate

   Calibrated parameters, metrics and hydrographs sync to your dashboard, then the instance terminates automatically (a shell trap guarantees it, even on error).

The optimizer default is role-aware MMPSO — mentor, mentee, and independent particles sharing multiple memories — with single-swarm PSO as the fallback. See calibration methods for the full method.

Instance options (c7i, us-east-1)

The instance is chosen automatically so the parallel swarm fits in memory and finishes fastest. RAM is the binding limit: each forward run needs roughly 0.5–4 GiB, so a 64 GiB box runs about 16 in parallel and a 128 GiB box about 32. Bigger swarms or big basins benefit from the larger instances — but the large sizes are subject to AWS capacity and account quotas, and may launch on-demand rather than spot.

Compute time by basin size (measured models)

Reference workload: 7-year calibration window, 32-particle swarm x 50 PSO iterations + 32 warm-up samples (1,632 forward runs).

Each row is a real SWATGenX model from the runtime sweep (engine 247e95b, NetCDF + print-filter), for a 7-year window and a 32-particle x 50-iteration PSO (1,632 forward runs) on one c7i.8xlarge. "Runs on" is the pricing model SWATGenX selects from the estimated wall time. Big basins finish faster on a larger instance or with a smaller swarm; swarm-level parallelism here is capped at the 32-particle pool, and the opt-in parallel SWAT+ engine can additionally put multiple cores on each forward run.

What a real calibration costs

One measured example, not a quote — cost scales with basin size, window, and swarm settings; the wizard estimates your run before launch.

Availability and accuracy

• Cloud calibration is in preview and offered to Pro users by request — it is not yet a self-serve checkout.
• Spot capacity is not guaranteed: a short job may wait briefly for a slot, and AWS can reclaim a spot box mid-run. SWATGenX surfaces the wait on your dashboard and, on a reclaim, automatically resumes the job on a non-reclaimable on-demand instance (warm-started from the latest checkpoint) — so spot interruptions never lose work. Long jobs run on on-demand capacity from the start.
• Runtime constants were fit (R²=0.996) to a six-model sweep and translated to c7i; your exact wall time depends on basin size, channel count, the calibration window, and swarm settings.

Engine 61.0.2.61-353-g247e95b · reference instances us-east-1.