1.2.4: Verification and validation#
Once the model has been implemented, two distinct checks are needed before it can be used to answer the research question. Nikolic, van Dam and Lukszo state the difference clearly: verification asks whether the implementation correctly represents the design, while validation asks whether the model is an adequate representation of the real system for the purpose at hand [1]. David, Fachada and Rosa, writing in the same handbook tradition, treat these as two complementary activities that together establish whether a model can be trusted [2]. The two look in opposite directions, verification inward at the code and validation outward at the world, and the sections below take each in turn before drawing out that contrast and showing how validation is done in practice.
1.2.4.1: Verification: did we build the thing right?#
Verification checks that the code does what the formal model says it should [1]. Computers do exactly what they are told, and what they are told is not always what the modeller intended. A practical way to organise verification is to work outward from one entity to the whole model: first check a single entity in isolation, then a minimal set of interacting entities, and finally the full model with all of them present [1]. Several standard practices support this, and a verified model usually has evidence from more than one of them:
What it checks. Test one function or method in isolation by giving it a known input and comparing its output with a result calculated in advance [1]. This localises faults: if the test fails, the problem lies in that small piece of code rather than somewhere in the complete model.
Traffic example. Give the speed-update method a vehicle with a specified speed and gap. Calculate the next speed by hand from the formal rule, then check that the method returns exactly that value.
Evidence to keep. A set of repeatable, automated tests that records the input, the expected output, and whether the code produced it. Rerunning the set after a code change reveals whether previously correct behaviour has been broken.
What it checks. Follow one entity through several time steps and compare every state change with the sequence prescribed by the formal rules [1]. A trace can expose a correct calculation being applied in the wrong order or at the wrong moment.
Traffic example. Start one vehicle at a chosen position and speed, record its gap, updated speed, and new position at each step, and calculate the same sequence by hand. The two traces should agree line by line.
Evidence to keep. A short table or log containing the initial state, intermediate calculations, and final state for each step. Keeping the trace makes it possible to identify the first step at which the implementation diverges from the design.
What it checks. Exercise unusual but permitted situations at the edges of the model’s operating range [1]. These cases often reveal assumptions that ordinary runs never challenge, such as dividing by a population size that can be zero.
Traffic example. Run an empty road, a road with one vehicle, the lowest allowed speed, the highest allowed speed, and a road close to maximum density. Each case should produce a defined outcome rather than a crash or an impossible vehicle state.
Evidence to keep. A list of the tested edge conditions and the behaviour expected in each one, including any condition the model is designed to reject with a clear error message.
What it checks. Confirm throughout a run that state variables stay within their allowed ranges and that quantities defined as invariant do not change [1]. Unlike an edge-case test, which starts the model in an extreme situation, a boundary check watches these constraints continuously.
Traffic example. Check after every step that speed remains between zero and the vehicle’s maximum speed, positions wrap correctly around the loop, and the number of vehicles on the closed road stays constant.
Evidence to keep. Automated assertions or diagnostic logs that stop or flag a run as soon as a range or conservation rule is violated, together with the step and agent responsible.
What it checks. Examine the decision branches in a rule and verify that every relevant combination of conditions leads to an allowed state [1]. This catches gaps, overlaps, and unreachable branches even when the arithmetic inside each branch is correct.
Traffic example. List the possible combinations of a safe or unsafe gap and a vehicle below or at its maximum speed. For each combination, confirm that exactly one intended response occurs, such as braking or maintaining speed, but never two contradictory actions at once.
Evidence to keep. A decision table that maps each condition combination to its expected action, supported by tests that exercise every row and show that no impossible state can be produced.
The purpose is to build up an evidence file that the model generates genuine results rather than artefacts of a coding mistake [1][2].
Verification comes down to a simple discipline: read the code line by line against the rule it is meant to implement, and ask at each step whether the line does what the rule says. The commonest mistakes are small ones, a comparison the wrong way round, a step carried out in the wrong order, a threshold that is off by one, and each of them makes the model do something other than what was intended. The activity below gives you four agent rules stated in plain English, each paired with a short list of pseudocode lines that are supposed to implement it. In every list exactly one line contains a bug.
1.2.4.2: Validation: did we build the right thing?#
A model that is verified to be a correct implementation of its design may still fail to represent the real system well enough to be useful. Validation compares the model’s behaviour with what is known about the real system, and it is useful to do this at two levels: micro validation asks whether individual entities’ behaviours match what real individuals do, while macro validation asks whether the system-level patterns match what the real system produces [1][2]. Common approaches, used in combination:
Does the behaviour look reasonable to someone who knows the system well? Often the first, cheapest check [2].
Do the model’s quantitative patterns align with empirical findings reported by others [2]?
Does the model reproduce the characteristic patterns, or “stylised facts”, that the real system is known to produce [2]?
Do domain experts find the assumptions and outcomes credible [2]?
Nikolic and colleagues emphasise that validation is best treated as a continuous process across the whole modelling cycle, not a single check carried out at the end [1]. It is also important to be honest about what validation can establish. Law writes that there is no such thing as absolute model validity: a model is valid for a specific purpose, and a model that is adequate for one question can be inadequate for another [3]. Validation does not produce a verdict of right or wrong; it produces a body of evidence that the model is, or is not, fit for the purpose for which it was built.
1.2.4.3: Verification versus validation: inward versus outward#
Having defined each check on its own, it is worth setting them side by side, because in practice the two are easy to confuse. Verification and validation are easy to confuse because both are about checking a model, yet they look in opposite directions. Verification looks inward and asks whether the code faithfully does what the formal model says, while validation looks outward and asks whether the model resembles the real system closely enough to be useful. Keeping the two apart matters, because a model can pass one and fail the other: flawless code can still describe the wrong world, and a plausible-looking model can still hide a coding mistake. In the activity below, decide for each check whether it is verification or validation, weighing deliberately which direction each one looks.
1.2.4.4: How validation is done in practice#
With the distinction settled, the question becomes concrete: how is validation actually carried out? Two complementary techniques dominate, and they work at different scales. The first fits the model’s output to data point by point; the second asks the model to reproduce an aggregate shape the real system is known to produce. We take each in turn.
Validation, at heart, is the act of laying a model’s output next to real data and asking whether the two agree. Real data almost never falls on a clean line, so it arrives as a scatter of points that trace out a shape, and the modeller’s job is to find the model curve that runs through that scatter. Adjusting the model’s parameters until its output matches the data is called calibration, the everyday work of tuning a model until it reproduces reality; validation is then the judgement of how good that match is. When the curve sits on top of the points, the model reproduces what the world did, and that overlay is the evidence validation asks for. In the activity below the scattered points follow a hidden growth curve, and your task is to find the curve that threads through them.
Fitting a curve point by point is one way to validate, but it is not always the right one: for many agent-based models the interesting behaviour is not any single data point but the overall shape the system settles into, so the second technique matches an aggregate pattern rather than every observation. Validation is what connects a model back to the world it claims to describe. A model earns trust not because its code runs without error, that is verification, but because the aggregate patterns it produces match patterns that independent real-world data actually shows. This is why validation so often works at the level of emergent behaviour: the model is asked to reproduce a stylised fact, a characteristic shape in the data that the real system is known to exhibit, and it counts as evidence only when the model’s output lines up with a real observation of that shape. In the matching activity below, each model output on the left names a pattern the model produces, and your job is to pick the real-world observation on the right that would validate it, the independent dataset the model must reproduce to be trusted.
1.2.4.5: The traffic example#
Fig. 8 Figure 1.8 Left: verification, a single vehicle on an empty loop should travel at its maximum speed without interruption. Right: validation, the model should reproduce the empirical stop-and-go wave, a band of congestion that propagates backwards against the direction of travel.#
The same model must clear two different evidence tests. Compare what each test asks and what would count as a successful result.
Guiding question: Does the implemented traffic model reproduce its formal design [1]?
Build the evidence from simple to demanding:
Check one rule: a vehicle accelerates when the gap ahead is large and decelerates when it is small.
Check an interaction: vehicles respond to one another without passing through or occupying the same position.
Trace the simplest case: one vehicle on an empty loop travels continuously at its maximum speed.
Stress the rules: at very high density, every small gap triggers the response specified by the formal model.
Pass signal: the observed state changes match the results predicted from the formal rules. This supports the claim that the code implements the intended model; it says nothing yet about whether the model resembles real traffic.
Guiding question: Does the model reproduce a pattern observed on real roads?
Use a known benchmark: beyond a critical vehicle density, real traffic flow can become unstable. Stop-and-go waves then emerge and propagate backwards, against the direction in which the vehicles travel [4].
Challenge the model: place enough vehicles on the loop for interactions to matter and look for spontaneous pockets of slowing rather than imposing a blockage yourself.
Pass signal: a compact region of slow traffic forms and drifts backwards while individual vehicles continue moving forwards through it. Reproducing this stylised fact is macro-level validation evidence [4]. If the pattern never appears under conditions where it should, the model or its assumptions need to be revisited [1][3].
The tabs above distinguish checking the implementation from checking its behaviour against the real world. In the interactive activity below, you can perform the macro-validation test yourself.
Reading the result
A backward-travelling wave appears: the model has reproduced one important qualitative pattern of real traffic. That strengthens its validation evidence, but does not prove that every part of the model is valid.
No wave appears: first confirm that the model was run at an appropriate density. If the pattern remains absent, revisit the conceptual assumptions, formal rules, or parameter choices rather than treating a successful run as guaranteed.
Verification now establishes that the code follows the design, while validation tests whether the resulting behaviour resembles the world closely enough for this purpose. The next section, 1.2.5: Simulation experiments, uses that checked model to generate results and answer the research question.
Key takeaways#
Key takeaways
Verification asks “did we build the thing right?” (code vs formal model); validation asks “did we build the right thing?” (model vs real system).
Verify by working from a single entity, to a minimal set, to the full model, using unit, trace, edge-case, boundary, and logic checks.
Validate at the micro and macro levels through face validity, comparison with data, pattern matching, and expert judgement.
There is no absolute validity: a model is only ever valid for a purpose, and validation builds evidence rather than a verdict.