How the calculations are built, part 1: foundations and fitted models
This is the developer's view of how EcoVal calculates. docs/calculations.md (and its water and city siblings)
explains what each number means and where its inputs come from. This document explains how the number is produced
in code: the formula, then the C# that implements it, then how the pieces fit together.
Part 1 covers two things:
- The foundations every calculation stands on: turning an outline into pixels, summing rasters, and the types that carry a result (ranges, tiers, availability).
- Every place where a model is fitted to data: what kind of fitting it is, how it is trained, where the result is stored, and how the API uses it.
Parts 2 to 6 (forest, water, city, the landing calculator) are listed in README.md.
All file paths are relative to backend/. Code excerpts are copied from the source and shortened; // … marks
what is left out.
Contents
- 1. The shape of every calculation
- 2. From outline to pixels
- 3. The types that carry a result
- 4. Which kind of regression does EcoVal use?
- 5. Fits made by the API at run time
- 6. Fits made offline by EcoVal.Calibration
- 7. Running a calibration again
1. The shape of every calculation
Every result follows the same path. The endpoint does the I/O (reads rasters and the database), then hands plain values to a calculator that is a pure function: the same inputs always give the same outputs, and nothing inside it touches a file or a database.
flowchart LR
A["Browser<br/>GeoJSON polygon, WGS84"] --> B["PolygonReader<br/>validate, reproject to SWEREF 99 TM"]
B --> C["Lookups and raster readers<br/>(I/O: PostGIS, GeoTIFF)"]
O["Options classes<br/>(parameters with sources)"] --> D
C --> D["Calculator<br/>(pure static methods)"]
D --> E["Contract records<br/>ValueRange, tiers, codes"]
E --> F["JSON response<br/>(OpenAPI contract)"]
The split matters for three reasons:
- Testing. A calculator can be tested with hand-written inputs and hand-checked expected values, with no raster files on disk.
- Reuse.
EcoVal.Calibrationcalls the same calculator methods the API calls. A fitted parameter is therefore fitted against exactly the code that will later use it. - Traceability. Every parameter lives in an options class with a comment naming its source, so the answer to "where does 1.5 km come from?" is one "go to definition" away.
A typical calculator signature shows the pattern. All data arrives as arguments, parameters arrive as options:
// EcoVal.Api/Features/Forest/Recreation/RecreationCalculator.cs
public static double[] EverydayShares(
double forestHectares,
IReadOnlyDictionary<(double X, double Y), double> polygonForest,
IReadOnlyList<ForestSquare> forest,
IReadOnlyList<PopulationNear> population,
double decayKm,
RecreationOptions options)
2. From outline to pixels
2.1 Reading and reprojecting the polygon
The browser sends coordinates as longitude and latitude (WGS84). Areas and distances cannot be computed in degrees, so the first step is to reproject to SWEREF 99 TM (EPSG:3006), where one unit is one metre.
PolygonReader.Read (EcoVal.Api/Spatial/PolygonReader.cs) checks the input in order and returns the first
problem as a stable error code, never as a sentence:
if (ring[0][0] != ring[^1][0] || ring[0][1] != ring[^1][1])
{
return PolygonReadResult.Invalid(PolygonErrors.RingNotClosed, "Each ring must end at its starting position.");
}
if (ring.Any(p => p[0] is < MinLongitude or > MaxLongitude || p[1] is < MinLatitude or > MaxLatitude))
{
return PolygonReadResult.Invalid(PolygonErrors.OutsideSweden, "The polygon must lie within Sweden.");
}
// …
var geometry = projection.ProjectPolygon(rings);
var validity = new IsValidOp(geometry).ValidationError;
The projection itself is a transverse Mercator transform done by the ProjNET library
(EcoVal.Api/Spatial/Sweref99Projection.cs); EcoVal only supplies the projection's definition (central meridian
15°, scale factor 0.9996, false easting 500,000 m). Geometry operations (validity, intersection, area) come from
NetTopologySuite.
2.2 Zonal statistics: summing a raster inside the polygon
Most forest numbers start as a raster: a grid of 10 m pixels where each pixel holds one value, for example the timber volume in m³sk per hectare. "Zonal statistics" means summarising those pixels inside a zone, here the polygon.
The formula. Let pixel have value , and let be the share of the pixel's area that lies inside the polygon. Pixels without data are left out of the sums and counted separately.
For a raster that holds a per-hectare quantity, the total over the polygon is the sum times the area of one pixel in hectares, where is the pixel size in metres:
With 10 m pixels each pixel is 0.01 ha, so a pixel showing 200 m³sk/ha that lies fully inside contributes 2 m³sk.
The implementation is ZonalStatistics.Compute (EcoVal.Api/Spatial/Rasters/ZonalStatistics.cs). It has
three steps:
var grid = PixelGrid.Of(tiles); // the pixel grid all tiles share
var window = grid.WindowCovering(polygon.EnvelopeInternal); // the pixel rectangle around the polygon
var pixels = Mosaic(tiles, grid, window, noData); // read that rectangle into one short[]
return Summarise(pixels, grid, window, polygon, noData); // weight and sum
Mosaic. A raster comes as many files (tiles).
ReadMosaicreads the window from each tile into one array. Where county files overlap, the first tile with data for a pixel wins, so nothing is counted twice.Coverage.
CoverageCellsworks out for every pixel. Computing a geometric intersection for every pixel would be slow, so it uses two tricks:- It first marks the pixels the boundary passes through (
BoundaryPixels). Only those get a real intersection:polygon.Intersection(box).Area / pixelArea. - Every other pixel is wholly inside or wholly outside. A scanline decides which: walking west to east along the row's centre line, each time the line crosses the boundary the state flips between outside and inside (the even–odd rule).
The cost therefore grows with the length of the boundary, not with the polygon's area.
- It first marks the pixels the boundary passes through (
Summarise. The loop is the formula above, almost symbol for symbol:
double sum = 0, valid = 0, noDataWeight = 0, zeroWeight = 0;
foreach (var cell in Coverage(grid, window, polygon))
{
var value = pixels[cell.Index];
if (value == noData)
{
noDataWeight += cell.Share;
continue;
}
// …
sum += value * cell.Share; // S = Σ s_p · v_p
valid += cell.Share; // N = Σ s_p
}
return new ZonalResult(sum, valid, noDataWeight, zeroWeight, grid.PixelSize);
ZonalResult then exposes the derived numbers as properties, so callers never repeat the arithmetic:
public double PixelAreaHectares => PixelSize * PixelSize / 10_000;
public double TotalOfPerHectareValue => Sum * PixelAreaHectares;
public double Mean => ValidPixels > 0 ? Sum / ValidPixels : 0;
public double NoDataShare => ValidPixels + NoDataPixels > 0 ? NoDataPixels / (ValidPixels + NoDataPixels) : 1;
Sharing reads within a request. One valuation reads the same window for several layers and several steps.
RasterReadScope (EcoVal.Api/Spatial/Rasters/RasterReadScope.cs) is an ambient cache held in an AsyncLocal:
inside using var reads = RasterReadScope.Begin(); each layer's window is decoded once and the coverage is
computed once. Outside a scope (tests, calibration) every read is fresh.
2.3 Class areas: counting by pixel centre, with sampling
Some rasters hold classes, not quantities (for example SLU's 2 m soil moisture map: dry, mesic, moist, wet). The
question is "how many hectares of each class?", and a 10,000 ha polygon holds 25 million 2 m pixels. ClassAreas
(EcoVal.Api/Spatial/Rasters/ClassAreas.cs) therefore does two things differently from ZonalStatistics: a pixel
counts whole if its centre is inside the polygon, and above a limit only every -th row and column is read.
The formula. With a window of pixels and a budget of samples, the step is
and each sample stands for pixels, so the area of class is
var pixels = (double)window.Width * window.Height;
var step = Math.Max(1, (int)Math.Ceiling(Math.Sqrt(pixels / maxSamples)));
var spacing = step * grid.PixelSize;
var sampleHectares = spacing * spacing / 10_000;
This is a systematic sample: the result is an estimate, and ClassCount.SpacingMetres tells the caller how coarse
it was.
2.4 The 1 km grid
Population (SCB) is published per 1 km square, so the models about people (visits, fishing days) work on a 1 km
grid aligned on SWEREF 99 TM's origin. SquareGrid.Split (EcoVal.Api/Spatial/SquareGrid.cs) cuts any geometry
into its parts per square, each with its hectares and the square's centre. Distances in those models are then
measured between square centres.
3. The types that carry a result
3.1 ValueRange: low, central, high
No result is a single number. Every computed value is three: a low, a central and a high case
(EcoVal.Api/Ranges.cs):
public sealed record ValueRange(double Low, double Central, double High)
{
public static ValueRange Spanning(double fromLow, double central, double fromHigh, int digits = 1) =>
new(
Math.Round(Math.Min(Math.Min(fromLow, fromHigh), central), digits),
Math.Round(central, digits),
Math.Round(Math.Max(Math.Max(fromLow, fromHigh), central), digits));
}
The range is not a statistical confidence interval. It is built by running the same formula three times, with the
low, central and high value of each uncertain parameter (ParameterRange). When a formula multiplies several
uncertain factors, the low case multiplies the lows and the high case the highs:
Spanning exists because a parameter can push the result in either direction; it takes the smallest and largest
of the three results as low and high, and leaves the central as computed.
3.2 Tier: how sure a number is
Each section of a result carries a confidence tier as a code (EcoVal.Api/Tiers.cs): measured (counted or
mapped at the place), modelled (modelled from data about the place) or borrowed (a factor or price from a
study elsewhere). A result is never more certain than its weakest input:
public static SectionTiers Priced(string quantity, string price) => new(quantity, Tier.Weakest(quantity, price));
3.3 Available<T>: a value, or the reason there is none
Missing data is not an error. A section either has a value or a code saying why not, modelled as a C# 15 union
(EcoVal.Api/Available.cs):
public union Available<T>(T, Unavailable)
where T : class
{
public T? ValueOrNull => Value as T;
public string? ReasonOrNull => (Value as Unavailable)?.Reason;
public void Deconstruct(out T? value, out string? reason) => (value, reason) = (ValueOrNull, ReasonOrNull);
}
A calculator writes return new Unavailable(UnavailableReasons.NoForestData); or return value;, and the
endpoint deconstructs it into the response's two fields: var (timber, timberUnavailable) = ….
3.4 Options: parameters are data
Model parameters are never constants in a formula. They are properties of an options class bound from
configuration (builder.Services.AddOptions<ValuationOptions>().BindConfiguration(ValuationOptions.Section)), and
the default value in the class is the documented value with its source in the XML comment:
// EcoVal.Api/Features/Forest/Recreation/RecreationOptions.cs
/// <summary>
/// Distance d₀ in w(d) = e^(−d/d₀), the weight a forest at distance d gets in where people go. In Friluftsliv 2018
/// (table 4.1) visits to the nearest nature area fall from about 180 a year within 100 m to about 55 at 0.5–1 km
/// and 33 at 1–3 km; an exponential through those points has d₀ ≈ 1.1–1.6 km. …
/// </summary>
public ParameterRange DecayKm { get; set; } = new(1, 1.5, 3);
This is also where fitted parameters end up, as section 6 shows.
4. Which kind of regression does EcoVal use?
Short answer: classical statistics and calibration, written by hand in C#. No machine-learning library, no neural network, no model file. The projects reference no ML.NET, Math.NET, ONNX or similar package; every fit is a few dozen lines of arithmetic you can read.
"Fitting a model" appears in four forms. They differ in how the best parameters are found:
| Form | How the parameters are found | Where in EcoVal |
|---|---|---|
| Statistical regression, closed form | A formula gives the best line directly (ordinary least squares) | Stream width from catchment area |
| Statistical regression, iterative | Repeated Newton steps until the estimate stops moving (logistic regression) | Chanterelle habitat model |
| Calibration by grid search | Try every combination on a grid of candidate values, keep the one with the smallest error | Excursions, tourism, fishing days |
| Empirical ratios and quantiles | Divide a counted total by a measured quantity; take medians and percentiles | Salmon smolts per km, sea trout densities, form height |
Is any of it machine learning? The chanterelle model is the closest: logistic regression is also a standard ML classifier, and it is evaluated the ML way, on data held out from training. But it is a 13-parameter statistical model with standard errors, not a learned black box, and the app does not use it (section 6.7). Everything else is calibration: a model whose form is decided by reasoning, with one to three free parameters tuned so the model matches counted data.
The fits also differ in when they run and where the result lives:
flowchart TB
subgraph offline ["Offline: EcoVal.Calibration, started by hand"]
C1["Counted data<br/>(visitor counts, surveys, GBIF finds)"] --> C2["Fit: grid search or regression"]
C2 --> C3["Report in data/calibration/*.md"]
C3 -->|"a developer copies the values"| C4["Options class defaults<br/>(source code, with a comment)"]
end
subgraph runtime ["Run time: EcoVal.Api, first request that needs it"]
R1["Database tables<br/>(electrofishing sites, river network)"] --> R2["Fit: least squares, quantiles, ratios"]
R2 --> R3["Singleton held in memory<br/>until the API restarts"]
end
C4 --> U["Calculators use the values on every request"]
R3 --> U
Two consequences worth knowing:
- Offline fits do not update themselves. New data changes nothing until someone runs the calibration again and copies the new values into the options class. The comment on the option records the date of the fit.
- Run-time fits follow the data. A new ingest of electrofishing surveys changes the stream width model the next
time the API starts. The current values are always visible at
/api/water/sea-trout-calibrationand/api/water/salmon-calibration.
| Fit | Kind | When | Stored in | Used by |
|---|---|---|---|---|
| Stream width | OLS on log–log | API, first use | SeaTroutModel singleton |
Dam: sea trout habitat |
| Sea trout density classes | Quantiles per group | API, first use | SeaTroutModel singleton |
Dam: sea trout habitat |
| Smolts per young trout | Ratios, median | API, first use | SeaTroutModel singleton |
Dam: sea trout smolts |
| Salmon smolts per km | Ratios, regional quantiles | API, first use | SalmonCalibrationSource singleton |
Dam: salmon smolts |
| Excursions: s, d₁, γ | Grid search | Offline | RecreationOptions.Excursion |
Forest: recreation |
| Tourism: α, d_w, weights | Grid search + 1-D minimiser | Offline | TourismOptions |
Forest: recreation |
| Tourism: q | Grid search | Offline | TourismOptions (kept as estimates) |
Forest: recreation |
| Fishing days: d₁, β | Grid search, weighted least squares | Offline | LakeOptions.FishingByPeople |
Water: lake |
| Form height by age | Binned ratio | Offline | CalculatorOptions…FormHeightByAge |
Landing calculator |
| Raw-water trend of a drinking-water source | Sen's slope with the Mann–Kendall test (a non-parametric trend estimate and test) | API, on each request | Not stored | Water: drinking water (part 5, section 5) |
| Growth model: production level, mortality, old-forest floor | Grid search, least squares | Offline, in a Python script (scripts/review/growth/fit-mortality.py) |
GrowthOptions |
Forest: growth and carbon (part 2, section 7.7) |
| Chanterelle habitat | Ridge logistic regression | Offline | Report only | Not used |
5. Fits made by the API at run time
All three live in EcoVal.Api/Features/Water/Dam/. They are built once, on the first request that needs them, by
a singleton "source" class that guards the build with a SemaphoreSlim so that two simultaneous first requests do
not both build it:
// SeaTroutModel.cs
public async Task<SeaTroutModel> GetAsync(CancellationToken cancellationToken)
{
if (_model is { } model)
{
return model; // the common path: already built
}
var network = await networks.GetAsync(cancellationToken);
await _lock.WaitAsync(cancellationToken);
try
{
if (_model is null) // checked again inside the lock
{
// … load the sites from the database …
_model = new SeaTroutModel(network, sites, options.Value);
}
return _model;
}
finally
{
_lock.Release();
}
}
5.1 Stream width: ordinary least squares on a log–log scale
What it is for. Young sea trout are counted per 100 m² of stream, so to know how many a stretch holds we need its width. Width is measured only at electrofishing sites; everywhere else it is predicted from how much land drains through the point.
Kind of fit: statistical regression (simple linear regression, ordinary least squares), trained by the API
itself from the electrofishing_surveys table.
The model. Width grows with catchment area as a power law, which is a straight line once both are log-transformed:
where is the wetted width in metres and the area draining through the point in km².
The fit. With and over the sites, least squares has a closed form:
is the share of the variation in that the line explains.
The implementation is the private method SeaTroutModel.Fit, a direct transcription:
private static WidthModel? Fit(List<(double X, double Y)> points)
{
if (points.Count < 10)
{
return null; // too few sites: no model
}
var meanX = points.Average(point => point.X);
var meanY = points.Average(point => point.Y);
var sxx = points.Sum(point => (point.X - meanX) * (point.X - meanX));
var sxy = points.Sum(point => (point.X - meanX) * (point.Y - meanY));
var syy = points.Sum(point => (point.Y - meanY) * (point.Y - meanY));
var slope = sxy / sxx;
return new WidthModel(
Math.Round(meanY - slope * meanX, 4), // a
Math.Round(slope, 4), // b
points.Count,
Math.Round(syy > 0 ? sxy * sxy / (sxx * syy) : 0, 3)); // R²
}
The training data is prepared in the constructor. Three choices are worth noticing:
Width = Fit(sites
.Where(site => site.Width is > 0 && site.Snap <= options.WidthFitMaxSnapMetres)
.Select(site => (Area: network.AreaAt(site.Position), Width: site.Width!.Value))
.Where(pair => pair.Area is > 0)
.Select(pair => (Math.Log10(pair.Area!.Value), Math.Log10(pair.Width)))
.ToList());
- A site surveyed several times enters once, with the median of its widths (computed just above this code), so heavily surveyed sites do not dominate.
- Only sites within 30 m of the network's line are used (
WidthFitMaxSnapMetres). A site on a small brook the network lacks would otherwise be attached to the big river nearby and teach the model that big rivers are narrow. - The logs are taken before fitting, so the regression minimises relative errors, not errors in metres.
Using the model. WidthModel.Width inverts the log, and SegmentValues clamps the result to a sensible range
and caches it per river segment:
public double Width(double areaKm2) => Math.Pow(10, Intercept + Slope * Math.Log10(Math.Max(areaKm2, 0.01)));
var width = Math.Clamp(
Width is { } model && network.AreaAtOutlet(id) is > 0 and var area ? model.Width(area) : options.MinWidthMetres,
options.MinWidthMetres, // 1 m
options.MaxWidthMetres); // 150 m
At the time of writing the fit is from 4,126 sites, with
(docs/calculations-water.md). An of 0.17 is weak: catchment area explains only about a sixth of the
variation in width. The model gives a reasonable typical width, not a good width for one particular stream. That
is acceptable here because widths are summed over many segments, where the errors partly cancel.
5.2 Sea trout densities and smolts per young trout: quantiles and ratios
Kind of fit: empirical statistics, not regression. Nothing is optimised; observed values are grouped and summarised.
Density classes. The sites that sea trout can reach freely today (no barrier on the way to the sea) are
grouped by water district and by width class (bounds at 2, 4, 8, 16 and 32 m). Each group's density is the 25th
percentile, median and 75th percentile of its sites, which become the low, central and high of a ValueRange.
The quantile used throughout interpolates linearly between neighbouring values. For sorted values and a quantile :
// SalmonCalibration.cs
public static double Quantile(IReadOnlyList<double> sorted, double q)
{
var position = q * (sorted.Count - 1);
var below = (int)Math.Floor(position);
var above = Math.Min(below + 1, sorted.Count - 1);
return sorted[below] + (sorted[above] - sorted[below]) * (position - below);
}
A group with fewer than MinSitesPerClass (10) sites is too thin to trust, so DensityFor falls back to the same
width class over all of Sweden and reports which source it used as a code (district or national).
Young trout in a stretch. For a stretch made of pieces with length (m), modelled width (m) and class density (young trout per 100 m²):
public (double YoungTrout, double AreaM2) Habitat(HabitatStretch stretch)
{
var (young, area) = (0.0, 0.0);
foreach (var piece in stretch.Pieces)
{
var length = (piece.To - piece.From) * (network.Segment(piece.SegmentId)?.LengthMetres ?? 0);
var (width, perMetre) = SegmentValues(piece.SegmentId); // perMetre = width × density / 100
young += length * perMetre;
area += length * width;
}
return (young, area);
}
Smolts per young trout. In three rivers the smolts leaving for the sea are counted in a trap. For each such river the model computes , the young trout its own method predicts above the trap, and divides each year's count by it:
The value used everywhere is the median over rivers of each river's median, with the lowest and highest single year as the range:
var ratios = young > 0 ? river.Smolts.Values.Select(smolts => smolts / young).Order().ToList() : [];
This is a calibration by ratio: the factor absorbs everything the density model gets wrong on average, so that the model reproduces the counted rivers by construction. It is honest about being three rivers only: the endpoint lists each river with its own ratio.
5.3 Salmon smolts per km: a ratio, with regional quantiles
Kind of fit: calibration by ratio. ICES publishes each wild salmon river's potential smolt production (PSPC). EcoVal walks the river in its own network, under the same habitat rule a dam's upstream stretch is walked with, and divides:
// SalmonCalibration.cs
var habitat = network.Habitat(mouth, new HashSet<int>(), options.MinUpstreamAreaKm2);
var km = habitat.LengthMetres / 1000;
var pspc = river.PspcThousands!.Value * 1000;
// …
new ValueRange(
Math.Round((river.PspcLowThousands ?? river.PspcThousands.Value) * 1000 / km),
Math.Round(pspc / km),
Math.Round((river.PspcHighThousands ?? river.PspcThousands.Value) * 1000 / km))
The point of using the same walk is that any systematic error in how EcoVal measures habitat length appears in both the calibration and the prediction, and cancels.
A dam's river then gets a density by a two-step rule in SalmonCalibration.Density: its own river's value if the
river is calibrated, otherwise the median of its region's rivers with the 10th and 90th percentiles as the range:
return (new ValueRange(
Math.Round(Quantile(region, 0.1)),
Math.Round(Quantile(region, 0.5)),
Math.Round(Quantile(region, 0.9))),
SalmonDensitySources.RegionMedian,
region.Count);
6. Fits made offline by EcoVal.Calibration
EcoVal.Calibration is a console program listed in the AppHost with explicit start. Program.cs registers the
API's own services (builder.AddForestry(), RecreationLookups, the options) and runs the models named in
Calibration__Models, writing one Markdown report per model to data/calibration/.
Four of the six fits tune the same family of model, so that model comes first.
6.1 The model being calibrated: sharing visits by distance
The idea. People in each 1 km population square make a fixed number of nature visits a year. Those visits are shared among all the forest within reach: more to forest that is near, less to forest that is far, and the shares always add up to one. A forest surrounded by people and little other forest gets many visits; a forest among endless forest gets few. Models of this form are known as gravity or spatial interaction models.
The distance weight. Forest at distance from a home gets the weight
is the distance at which the weight has fallen to . The floor (0.5 km) is there because people live spread over their square, so a distance is never taken as shorter than that.
The polygon's visits. With the adults in population square , the forest land in the polygon, the distance from the square's centre to the polygon, the forest land in every other 1 km square (the polygon's own forest taken out) and the visits a year per adult:
The fraction is the polygon's share of square 's visits: its own pull over the pull of everything within reach.
The implementation is RecreationCalculator.EverydayShares. It returns the per-square terms
share, so that callers (and the calibration) can sum, filter or group them:
double Weight(double distance) => distance > reach ? 0 : Math.Exp(-Math.Max(distance, floor) / decay);
// …
for (var i = 0; i < population.Count; i++)
{
var square = population[i];
var own = square.Adults > 0 ? forestHectares * Weight(square.DistanceMetres) : 0; // A_P · w(d_cP)
if (own <= 0)
{
continue;
}
var elsewhere = 0.0; // Σ_k F_k · w(d_ck)
foreach (var other in others)
{
var dx = other.CentreX - square.CentreX;
var dy = other.CentreY - square.CentreY;
if (Math.Abs(dx) > reach || Math.Abs(dy) > reach)
{
continue; // a cheap box test before the square root
}
elsewhere += other.Hectares * Weight(Math.Sqrt(dx * dx + dy * dy));
}
shares[i] = square.Adults * own / (own + elsewhere);
}
Excursions use the same form with two changes: a longer distance , and a size exponent that lets a large, well-known destination pull more per hectare than a small one. A destination of area hectares gets the pull , spread over its hectares as . A share of each adult's visits are excursions:
RecreationCalculator.ExcursionShares implements it. The denominator's "all destination land within reach of this
square" does not depend on the polygon, so DestinationGridData.Pull computes it once per population square and
parameter set and caches it in a ConcurrentDictionary.
So the free parameters are: (set from the literature, not fitted), and , and , which have no Swedish source and are fitted.
6.2 Excursions: grid search for s, d₁ and γ
Kind of fit: calibration by grid search. Not regression: there is no formula for the best parameters, and the
error surface is not smooth enough to trust a gradient method with so few data points. Instead every combination
on a grid is evaluated and the errors are compared. File: EcoVal.Calibration/ExcursionCalibration.cs.
private static readonly double[] Shares = [0, 0.10, 0.15, 0.20, 0.25, 0.30, 0.35];
private static readonly double[] DecaysKm = [5, 7.5, 10, 15, 20, 30, 50];
private static readonly double[] SizeExponents = [0.7, 1.0, 1.3, 1.6, 2.0, 2.5];
The data is three independent kinds of evidence, each of which pins down a different aspect:
- Levels. Seven nature reserves in Huddinge with visits counted from mobile-phone data (2021).
- Origins. The same study's split of where visitors came from, by municipality.
- A guard. Gotland's least visited reserves (estimated at 1–100 visits a year): adding excursions must not inflate these.
The objective to minimise is the sum of three errors. For parameters :
is a counted number, the model's, the reported share of visits from origin and the modelled one. The squared log ratio treats "twice too high" and "half as much" as equally wrong, which is right for counts that span several orders of magnitude. The divisor 1000 puts the origin error (in squared percentage points) on roughly the same scale as the other two, so each term is about 1 for a poor fit.
private static Fit Evaluate(/* … */)
{
var huddinge = areas.Where(area => area.Area.Region == "Huddinge").ToList();
var levelError = huddinge.Average(area =>
Math.Pow(Math.Log(Math.Max(ModelledKept(area, share, combo, perAdult), 1) / area.Area.Visits), 2));
var origins = OriginShares(huddinge, share, combo);
var originError = OriginSplit.Reported.Sum(row => Math.Pow(100 * (origins.GetValueOrDefault(row.Code) - row.Share), 2)) / 1_000;
var lowClass = areas.Where(area => area.Area.Region == "Gotland" && area.Area.VisitsHigh <= 100).ToList();
var lowRatio = Median(lowClass.Select(area => area.Area.Visits / Math.Max(Modelled(area, share, combo, perAdult), 1)));
// …
}
Comparing like with like. The phone data does not count everyone: it leaves out people living within 400 m and
stays under 20 minutes. MobileDataFilter.Kept(distance, …) returns the share of visits from a home at a given
distance that the phones would have counted, and the model's per-square terms are multiplied by it before
comparing (ModelledKept). Fitting the unfiltered model to filtered counts would bias every parameter.
Why the search is fast. The expensive part is the spatial sums, and enters the model linearly:
So ComputeAsync calculates (everyday shares) once per area and (excursion shares) once per
pair, and the loop over is plain arithmetic:
private static double Modelled(AreaShares area, double share, int combo, double perAdult) =>
perAdult * ((1 - share) * area.Everyday + share * area.Excursion[combo]);
flowchart LR
A["For each counted area:<br/>everyday shares E<br/>excursion shares X per (d₁, γ)"] --> B["For each s, d₁, γ on the grid:<br/>model = V·((1−s)·E + s·X)"]
B --> C["Objective J =<br/>level + origin + guard"]
C --> D["Sort by J,<br/>write the best 15 to excursions.md"]
D --> E["Developer chooses low, central, high<br/>and edits RecreationOptions.Excursion"]
The result and its use. The report's best set was , km, (), with several neighbours almost as good. The values stored are not simply the minimum; the options comment explains the judgement applied (the fit pushes to its bound, and would let the largest destinations dominate):
// RecreationOptions.cs
public List<ExcursionParameters> Excursion { get; set; } =
[
new(0.25, 10, 1.6), // low: s, d₁ km, γ
new(0.30, 10, 1.6), // central
new(0.35, 20, 1.6), // high
];
RecreationCalculator.Excursions runs the model once per case and builds the ValueRange from the three results.
6.3 Tourism, where guests go: grid search with a fitted scale factor
File: EcoVal.Calibration/AttractionCalibration.cs. Kind of fit: grid search, with a one-dimensional numerical
minimiser nested inside it.
The model. A municipality's guest nights are shared among its destination land. A share goes by area, and a share by attractiveness, where land near car parks, trailheads, viewpoints and beaches is more attractive:
is the attractiveness at a place : each access point adds its kind's weight , fading with distance at the rate . In code the blend is one line:
// RecreationContracts.cs
public double Combined(double attractionShare) => (1 - attractionShare) * Area + attractionShare * Attraction;
The grid covers (9 values), (4 values) and the weights of four groups of access point (each 0,
1, 3 or 10; only their ratios matter, so WeightSets() normalises and removes duplicates).
The nested fit. Gotland's 158 counted reserves are modelled as
with the residents' visits (from the excursion model, fixed) and the guest nights the candidate sends to reserve . The factor (visits per guest night) is unknown and of no interest here; what matters is whether the shape is right, that is, whether busy reserves come out busy. So for each candidate on the grid, the best is found first, and the candidate is scored at that :
The inner minimum over uses golden-section search, a method for finding the minimum of a function of one variable without derivatives. It keeps an interval that contains the minimum and shrinks it by the golden ratio each step:
double Error(double logK) => SquaredLogError(gotland, guestNights, Math.Exp(logK));
double low = Math.Log(1e-3), high = Math.Log(30);
var ratio = (Math.Sqrt(5) - 1) / 2;
for (var i = 0; i < 60; i++)
{
var a = high - ratio * (high - low);
var b = low + ratio * (high - low);
if (Error(a) < Error(b))
{
high = b; // the minimum lies in [low, b]
}
else
{
low = a; // the minimum lies in [a, high]
}
}
var k = Math.Exp((low + high) / 2);
The search runs on , so it covers from 0.001 to 30 evenly in relative terms. Sixty steps shrink the interval by , far more than needed.
The result is stored in TourismOptions: AttractionShare = new(0.8, 0.9, 1), AttractionDecayKm = 0.5 and
AccessPointWeights. The comments record what the fit could and could not tell: sits at the edge of its
grid, and several weight sets fit about equally well.
6.4 Tourism, visits per guest night: a grid search that could not decide
File: EcoVal.Calibration/TourismCalibration.cs. Kind of fit: grid search, over (nature visits per guest
night) for city municipalities and for other municipalities, with the same three-part objective as the excursion
fit (Huddinge's levels, Gotland's busy reserves, Gotland's quiet reserves as a guard):
private static readonly double[] OtherGrid = [0, 0.05, 0.1, 0.15, 0.2, 0.25, 0.3, 0.4, 0.5, 0.75, 1.0];
private static readonly double[] CityGrid = [0, 0.01, 0.02, 0.05, 0.1, 0.2];
// …
public double Objective => LevelError + TouristError + GuardError;
This is the instructive case of a calibration that ran but did not settle the parameter. The options comments say so plainly: the values are "a first estimate", because the counted data hardly moves with . The calibration is kept because it reports how the configured values score, and because it will decide the question once better counts exist. A fit is only as good as the data's power to tell candidate values apart.
6.5 Fishing days: weighted least squares on a grid
File: EcoVal.Calibration/FishingCalibration.cs. Kind of fit: grid search minimising a weighted sum of squares
(χ²). This is the cleanest example of the four.
The problem. A national survey gives the fishing days a year for a few large areas. The page needs days for one lake. Sharing a region's days by water area alone is badly wrong: it gives Vänern 1.2 million days where the survey found 233,000.
The model. A share of the days follow the people living near each water, and the rest follow hectares. For area , with the total days, the total hectares and the "pull" of area on people:
The pull uses the same distance-weighted sharing as section 6.1, now for water. Each population square splits its adults among all water squares within reach by hectares :
FishingGridData.Compute (EcoVal.Api/Features/Water/Lake/FishingGrid.cs) evaluates it for about 95,000
population squares against 160,000 water squares. It indexes the water by 10 km blocks so each home only looks at
nearby blocks, runs the homes in parallel with Parallel.For, and caches the result per distance setting:
Parallel.For(0, _homes.Length, i =>
{
var home = _homes[i];
var byArea = new double[_areas.Length];
// … for each 10 km block within reach of the home …
foreach (var (square, area) in block)
{
byArea[area] += square.Hectares * Weight(square.CentreX - home.X, square.CentreY - home.Y, distance);
}
total[i] = byArea.Sum(); // the denominator: this home's pull to all water
byHome[i] = byArea;
});
The objective. Each survey figure comes with a margin of error. A figure the survey is unsure of should count for less, so each squared error is divided by that figure's variance. With = margin of error ÷ 1.96 (the margin is a 95 % interval, 1.96 standard deviations wide):
foreach (var share in ShareGrid) // β = 0, 0.05, …, 1
{
var days = targets
.Select((_, i) => total * ((1 - share) * hectares[i] / allHectares + share * pull[i] / allPull))
.ToArray();
var chi = targets.Select((target, i) => Math.Pow((days[i] - target.Days) / target.Sigma, 2)).Sum();
fits.Add(new Fit(decay, reach, share, chi, days));
}
// …
var best = fits.MinBy(fit => fit.ChiSquare)!;
The pulls (expensive) are computed once per in the outer loop; the loop over inside is arithmetic, the same trick as in 6.2.
From the best fit to a range. χ² also gives a principled range. For a fit like this, parameter values whose χ² is within 1 of the minimum are the ones the data cannot reject at about one standard deviation. The report lists them, and they became the low, central and high cases:
// LakeOptions.cs
public List<FishingPeopleCase> FishingByPeople { get; set; } =
[
new() { DecayKm = 0.75, PeopleShare = 0.85 }, // low
new() { DecayKm = 1.5, PeopleShare = 0.90 }, // central: the best fit, χ² 0.10
new() { DecayKm = 3, PeopleShare = 0.90 }, // high
];
How the API uses it. The fit was made on four areas, but the page needs any lake in any region. So the fitted is first turned into a quantity that travels: , the fishing days a year an adult makes on water near home.
A lake in region (with the region's days from the survey) then gets days from the people near it, and its share by area of whatever the region has left:
// LakeCalculator.cs
private static double Rate(FishingPullCase pullCase, double fitDays, LakeOptions options) =>
options.FishingFitAreas.Sum(code => pullCase.AreaPull.GetValueOrDefault(code)) is var fitPull and > 0
? pullCase.PeopleShare * fitDays / fitPull
: 0;
// inside RegionShares:
peopleShare = Math.Min(1, rate * regionPull / regionDays);
var lakePull = pullCase.Lakes.GetValueOrDefault(lake.Id).Pull;
return regionDays * ((1 - peopleShare) * lake.Hectares / regionHectares + peopleShare * lakePull / regionPull);
Note what is and is not hard-coded: and are stored, but is recomputed from them and the current population and water data on every start, so it follows the data.
6.6 Form height by age: a binned ratio that becomes a lookup table
File: EcoVal.Calibration/FormHeightCalibration.cs. Kind of fit: an empirical table, not regression. No
functional form is assumed at all.
What it is for. The landing calculator knows a typical stand's volume but its berry models need basal area. Form height is the bridge: (volume ÷ basal area, in metres).
The method. Draw random 30 m squares over Sweden until 20,000 have forest and data in all four rasters, sort them into 10-year age classes , and take the ratio of sums in each class:
The ratio of sums (not the mean of each square's ratio) weights each square by its basal area, so sparse young squares with unstable ratios do not distort the class.
var random = new Random(20261030); // a fixed seed: the same squares on every run
// …
var age = (ageSpruce + agePine) / 2;
var key = Math.Min(LastClass, (int)(age / ClassYears) * ClassYears);
var entry = classes.GetValueOrDefault(key);
classes[key] = (entry.Volume + volume, entry.BasalArea + basalArea, entry.Count + 1);
Each square's values come from ZonalStatistics.Compute (section 2.2), the same code the API uses.
Using it. The table is stored in CalculatorOptions keyed by each class's middle age, and read with linear
interpolation: straight lines between the ages, flat beyond the ends.
// ForestStandCalculator.cs
private static double Interpolated(Dictionary<double, double> byAge, double age)
{
var table = byAge.OrderBy(entry => entry.Key).ToList();
if (age <= table[0].Key)
{
return table[0].Value;
}
for (var i = 1; i < table.Count; i++)
{
if (age <= table[i].Key)
{
var (from, to) = (table[i - 1], table[i]);
return from.Value + (age - from.Key) / (to.Key - from.Key) * (to.Value - from.Value);
}
}
return table[^1].Value;
}
6.7 Chanterelle habitat: ridge logistic regression, tried and left out
Files: EcoVal.Calibration/ChanterelleCalibration.cs and ChanterelleModel.cs. Kind of fit: statistical
regression (a generalised linear model), trained iteratively and validated on held-out data. This is the one
model in the repository that a data scientist would recognise as "trained", and the one closest to machine
learning. The API does not use it: it barely beat chance, so no score is shown. It stays in the repository so
the fit can be repeated on better data, and it is the best worked example of a full training pipeline.
flowchart TB
A["GBIF downloads:<br/>chanterelle finds (y = 1)<br/>21 other mushrooms (y = 0)"] --> B["Filter and thin<br/>one record per 100 m square"]
B --> C["Read the stand in a 60 m square<br/>8 inputs per record"]
R["12,000 random forest squares<br/>(reference sample)"] --> D
C --> D["Clamp to 1st–99th percentile<br/>add squared terms (12 terms)<br/>standardise on the reference"]
D --> E["Fit: weighted ridge logistic regression<br/>(Newton / IRLS)"]
D --> F["Validate: 5 folds of whole 100 km blocks<br/>AUC on the held-out fold"]
E --> G["chanterelle.md:<br/>coefficients, standard errors, AUC"]
F --> G
The model. The probability that a record is a chanterelle find, given its site's terms :
The twelve terms are eight inputs (age, basal area, pine share, spruce share, site index, dry share, wet share,
northing) plus the squares of four of them, so the response can peak in the middle of a range (ChanterelleModel.Raw).
Each term is standardised with the mean and standard deviation of the random forest sample, so
a coefficient reads as "per standard deviation of Swedish forest" and the ridge penalty treats all terms alike.
What is minimised. The penalised, weighted negative log-likelihood:
- is a class weight: 1 for finds and for background records, so both classes weigh the same in total even though there are many more background records than finds.
- The last term is the ridge penalty (). It pulls coefficients toward zero, which keeps correlated terms (age and age²) from taking huge opposite values. The intercept is not penalised.
How it is minimised. There is no closed form, so the fit uses Newton's method, which for logistic regression is known as iteratively reweighted least squares (IRLS). Each iteration computes the gradient and the Hessian at the current and takes a step:
for (; iterations < 50; iterations++)
{
hessian = new double[k, k];
var gradient = new double[k];
foreach (var (x, y, _) in rows)
{
var row = Row(x); // [1, x₁, …]
var p = 1 / (1 + Math.Exp(-row.Select((v, i) => v * beta[i]).Sum())); // p = 1 / (1 + e^(−η))
var w = weight(y);
for (var i = 0; i < k; i++)
{
gradient[i] += w * (y - p) * row[i];
for (var j = 0; j < k; j++)
{
hessian[i, j] += w * p * (1 - p) * row[i] * row[j];
}
}
}
for (var i = 1; i < k; i++) // the ridge, skipping the intercept
{
gradient[i] -= Ridge * beta[i];
hessian[i, i] += Ridge;
}
var step = Solve(hessian, gradient); // solves H · step = g
for (var i = 0; i < k; i++)
{
beta[i] += step[i];
}
if (step.Max(Math.Abs) < 1e-8)
{
break; // converged
}
}
Solve is Gaussian elimination with partial pivoting, about 30 lines, which is all the linear algebra a 13 × 13
system needs. The fit converged in 4 iterations. The standard error of each coefficient falls out of the same
matrix:
var errors = Enumerable.Range(0, k).Select(i => Math.Sqrt(Solve(hessian, Unit(k, i))[i])).ToArray();
Guarding against extrapolation. Squared terms explode outside the range the model has seen. So every input is
clamped to the 1st–99th percentile of the training records, both when fitting and when predicting
(ChanterelleModel.Clamped).
Validation: spatial block cross-validation. A model must be judged on data it was not fitted to. Ordinary cross-validation would cheat here, because finds from the same forest are near-duplicates: a model can score well by memorising places. So whole 100 km blocks are held out together:
var foldOf = blocks.OrderBy(_ => random.Next()).Select((block, i) => (block, fold: i % 5)).ToDictionary(/* … */);
for (var fold = 0; fold < 5; fold++)
{
var train = rows.Where(row => foldOf[row.Block] != fold).ToList();
var test = rows.Where(row => foldOf[row.Block] == fold).ToList();
var beta = Fit(train, columns, weight).Beta; // fitted without the held-out blocks
// … score the held-out rows with beta …
}
The score: AUC. The area under the ROC curve is the chance that a randomly chosen find scores higher than a randomly chosen background record. 0.5 is a coin toss, 1 is perfect. It is computed from ranks (the Mann–Whitney statistic): sort all scores, add up the ranks of the finds, and
return positives == 0 || negatives == 0
? double.NaN
: (rankSum - positives * (positives + 1) / 2.0) / ((double)positives * negatives);
Tied scores share the average of their ranks, which the loop above this line handles.
The outcome. The held-out AUC was 0.580, against 0.532 for a model that knows only how far north a place is. Leaving out any single group of terms changed the AUC by less than 0.01. In plain words: stand data tells chanterelle places from other mushroom places only slightly better than a coin, so the model was not put in the app. The calibration also shows why the choice of background matters: against all fungi the model mostly learned where conservation inventories go, which is why the background was narrowed to mushrooms that foragers report.
How it would be used. ChanterelleModel.LinearPredictor and Ratio are the prediction side, ready if a
better fit arrives. They leave out the intercept (it only reflects how many background points were drawn) and
report a ratio against average Swedish forest, not a probability:
public static double Ratio(ChanterelleSite site, ChanterelleOptions options) =>
Math.Exp(LinearPredictor(site, options)) / options.ReferenceMeanExp;
7. Running a calibration again
- Start the app with
aspire runso PostGIS is up and the ingest has run. - In the Aspire dashboard, start the
calibrationresource. By default it runsexcursions,attraction,tourismandfishing. SetCalibration__Models=form-heightorchanterellefor the two that are on request (they read the rasters at tens of thousands of points). - Read the report in
data/calibration/. Each report prints the configured values next to the best on the grid, so it shows at once whether the stored parameters still hold. - To adopt new values, edit the defaults in the options class, update the XML comment with the date and the
numbers, and update
docs/calculations.md. Nothing reads the report automatically.
The run-time fits of section 5 need no step: they are rebuilt from the database when the API starts.
A checklist for adding a fitted parameter
- Keep the model a pure function in the API project, and have the calibration call that function.
- Compare the model with the data on the data's own terms (the phone-data filter in 6.2 is the example).
- Hold something back as a guard or a check, so the fit cannot buy accuracy in one place by ruining another.
- Store low, central and high, not just the best value, and write down in the option's comment what the data could and could not determine.