SWAT+ performance research
SWAT+ performance profiling: two upstream fixes, before and after
We profiled the stock SWAT+ engine on a 94,303-HRU basin with Intel VTune: the two largest CPU costs are not hydrology but string name-matching and array zeroing — two small, results-identical fixes remove them and cut a 3-month run 1.58×.
Commit-pinned binaries that differ by nothing but the two fixes. Both are proposed upstream as independent pull requests (#219, #220); both already ship in the SWATGenX production engine.
- Top cost: name-matching 75s, not hydrology
- 462s → 292s (1.58×) on our server
- Byte-identical results
- Upstream PRs #219 + #220 (proposed)
Inside a single SWAT+ simulation, where does the CPU actually go? We profiled the stock distribution (swat-model/swatplus 5ccf6f0) on Peace River HUC-8 — 94,303 HRUs — with Intel VTune, and the answer is not what runtime advice usually assumes.
The two biggest costs are overhead, not science: string name-matching in the input reader and array zeroing in the daily reset. We trace each to a specific source pattern, fix it, and show the before/after on commit-pinned binaries that differ by nothing but the two fixes. Both are proposed upstream as independent pull requests, and both already ship in the SWATGenX production engine.
These two fixes are serial-CPU wins — they change no results and need no threads. On the engine-acceleration study’s full-year AWS benchmarks the same overhead-removal work, combined with the rest of the serial ladder, measures 1.67× on the c8a reference node and up to 2.47× on more memory-starved CPUs (machine-dependent); on top of that, an opt-in multi-core wavefront scales a single run further. This page isolates the two fixes on our server; the full serial ladder and multi-core scaling are on the SWAT+ parallel-engine page.
Motivation
Where does a SWAT+ simulation actually spend its time?
Runtime advice for SWAT+ usually assumes the per-HRU hydrology dominates the cost — so coarsening the model or parallelizing the HRU loop should be the big wins. We profiled the stock engine to test that, and the assumption does not hold on large basins. (Parallelization itself is a separate SWATGenX track: the open parallel SWAT+ engine does run the HRU land phase and channel routing multi-core — this page is about where the remaining serial CPU time actually goes.)
On a 94,303-HRU model, the two largest CPU consumers are not hydrology at all: they are string name-matching and memory zeroing — pure overhead. This page profiles stock SWAT+, identifies the two inefficiencies, fixes each one, and shows the before/after. Both fixes are small, leave results unchanged, and are offered upstream as two independent pull requests.
Methods
What we profiled, and against what
We profile the stock SWAT+ distribution and the same code with two small fixes applied. Both binaries are built identically (Intel ifx -O3 -ipo) and run the identical model and output configuration, so the only difference between them is the two fixes — the comparison isolates exactly their effect.
The model is Peace River HUC-8 (94,303 HRUs), a large real basin where O(N) vs O(N²) costs are visible. Output is the standard daily channel_sd table. CPU profiles are recorded with Intel VTune (hotspots, software sampling); end-to-end wall time is measured separately without the profiler attached.
| Role | Commit | What it is |
|---|---|---|
| Baseline | swat-model/swatplus@5ccf6f0 | stock SWAT+ (upstream/main) |
| Final | swat-model/swatplus@5ccf6f0 + #219 + #220 | stock + the two fixes |
Table 1. The two binaries compared, each pinned to a commit. They differ only by the two fixes; build flags, model, and output configuration are identical.
- Engine: stock SWAT+ swat-model/swatplus @ 5ccf6f0 (the original distribution).
- Fixes: two independent commits -> upstream PRs #219 (hru_read) and #220 (varinit).
- Build: Intel ifx -O3 -ipo, identical flags both sides.
- Model: Peace River HUC-8, 94,303 HRUs; daily channel_sd output.
- Profiler: Intel VTune hotspots (sw sampling, stacks on); 30-day window. Timing: 90-day, no profiler.
The two fixes are open as independent upstream pull requests: #219 and #220 on swat-model/swatplus.
Results and discussion
Before and after: where the CPU time goes
Peace River HUC-8 (03100101) — 94,303 HRUs, 8,181 channels · 30 simulated days.
Figure 1. VTune CPU self-time on Peace River (94,303 HRUs, 30-day run), stock SWAT+ vs the same code with the two fixes. The three overhead functions collapse; the two real-work functions (hydrology, routing) are unchanged.
| Function | Role | Baseline (s) | + fixes (s) | Change |
|---|---|---|---|---|
| for_cpstr_eq | overhead · hru_read (startup) | 75.4 | 6.9 | −91% |
| __intel_avx_rep_memset | overhead · varinit (daily) | 28.3 | 2.1 | −93% |
| proc_hru_ | overhead · proc_hru | 20.8 | 0.2 | −99% |
| hru_control_ | real work · real work | 24.6 | 22.6 | — |
| sd_channel_control3_ | real work · real work | 17.4 | 16.6 | — |
| command_ | overhead · orchestration | 8.4 | 7.0 | −17% |
Table 2. Per-function CPU self-time (seconds), stock baseline vs the two fixes, on a 30-day Peace River run.
In the stock engine the three biggest costs are overhead, not science: string name-matching (75 s), memory zeroing (28 s), and startup input handling (21 s). Together they outweigh the two functions that do real work — HRU hydrology (25 s) and channel routing (17 s).
After the two fixes, those overhead costs collapse — name-matching 75 s → 7 s, zeroing 28 s → 2 s, startup 21 s → 0.2 s — while the hydrology and routing functions are essentially unchanged. The profile flips from overhead-dominated to compute-dominated: the engine now spends its time on the actual simulation.
The two fixes
Startup name-matching → O(1) lookup PR #219
hru_read.f90 + name_index_module.f90 · hotspot: for_cpstr_eq, 75 s → 7 s
Problem. For every HRU, the input reader resolves the topography, hydrology, soil, and field database pointers by linearly scanning each database by name — an O(N_HRU × N_db) cost. On Peace River (94k HRUs; the topography table alone has ~104k entries) that is on the order of billions of character comparisons before day one, and it dominates startup.
Fix. Build a small name→index hash once per database, then look each HRU up in O(1). First-match semantics are preserved, so the resolved indices — and the results — are unchanged.
Scaling. One-time startup cost, quadratic in HRU count: negligible on small models, large on big basins. Most valuable for workflows that create and briefly run many models.
Daily array zeroing → reset only the active row PR #220
varinit.f90 · hotspot: __intel_avx_rep_memset, 28 s → 2 s
Problem. The per-HRU daily reset zeros a working array with an unindexed assignment (hhsedy = 0.), but that array is dimensioned over all HRUs. So every HRU, every day, zeros the entire all-HRU array — an O(N_HRU²)-per-day cost.
Fix. Zero only the active HRU's row (hhsedy(j,:) = 0.), matching how the array is used everywhere else. One indexed assignment; results unchanged.
Scaling. Per-simulated-day cost, so the saving scales with run length: ~0.83 s/day on Peace River — minutes per year, and it compounds across every forward run in a calibration.
End-to-end effect on a 3-month Peace River run
Peace River HUC-8 (03100101) — 94,303 HRUs, 8,181 channels · 90 simulated days, daily channel_sd output.
Correctness: Results are numerically identical: the data rows are bit-for-bit the same, and the only difference in the output file is the SWAT+ revision string in the header (which reflects the build commit). On NetCDF output the files are byte-identical.
What remains: the real work
With the two overhead costs removed, the profile is dominated by the functions that do real work — HRU hydrology (hru_control) and channel routing (sd_channel_control3). Those are the next frontier.
The HRU land phase is independent per HRU and is the natural target for shared-memory parallelism. Channel routing is the larger cost on dense networks but is dependency-constrained (downstream reaches need their upstream hydrographs first), so it parallelizes across independent tributaries rather than as a flat loop. Both are larger, structural changes — out of scope for these two bug-fix-class PRs.
Conclusion
- On a large basin, stock SWAT+ spends more CPU on avoidable overhead — string name-matching, array zeroing, and startup input handling — than on the HRU hydrology those costs are assumed to be about.
- Two small, results-identical fixes remove that overhead: a 3-month Peace River run drops from 462 s to 292 s (1.58×, 37%), and the CPU profile flips from overhead-dominated to compute-dominated.
- Each fix is one self-contained change with no dependency on any other feature, contributed upstream as an independent pull request: #219 (hru_read O(1) lookup) and #220 (varinit per-row reset).
- What remains is the real work — hydrology and routing — where the next gains require parallelism, a larger structural effort.
FAQ
Where does stock SWAT+ spend its CPU on a large basin?
On a 94,303-HRU model profiled with Intel VTune, the three biggest CPU costs are overhead, not science: string name-matching in the input reader (~75 s), array zeroing in the daily reset (~28 s), and startup input handling (~21 s) — together larger than the HRU hydrology (~25 s) and channel routing (~17 s) that do the actual work.
What are the two fixes?
One replaces an O(N_HRU x N_database) linear name scan in hru_read with an O(1) name-to-index hash (upstream PR #219). The other zeros only the active HRU’s working row in the daily reset instead of the whole all-HRU array (upstream PR #220). Each is a small, self-contained change with no dependency on any other feature.
Do the fixes change model results?
No. Results are numerically identical — the data rows are bit-for-bit the same. NetCDF output is byte-identical; text output differs only in the SWAT+ revision string embedded in the file header, which reflects the build commit.
How much faster, and on what?
On a 3-month Peace River run (94,303 HRUs), wall time dropped from 462 s to 292 s (1.58x, 37%) with identical output. By VTune the overhead collapsed: name-matching 75 s to 7 s, array zeroing 28 s to 2 s, startup 21 s to 0.2 s; the hydrology and routing functions were unchanged.
Related guides
Explore related
Last updated 2026-07-13.