5.4: Good practices for empirical ABM

5.4: Good practices for empirical ABM#

Integrating empirical data into an agent-based model (ABM) is not a “one-and-done” task; it is a continuous, iterative journey where the complexity of the data must match the maturity of the simulation logic. This section outlines how to move from raw datasets to a scientifically robust model that avoids common pitfalls in the study of complex Socio-Technical-Environmental (STE) systems. That journey runs through five practices, and this section takes them in order: build the model incrementally (5.4.1), calibrate its parameters to data (5.4.2), validate it against reality (5.4.3), generalise beyond the single case (5.4.4), and document it so it stays transparent (5.4.5).

5.4.1: Build incrementally: from v0 to v10#

The iterative journey has to start somewhere, and the first practice is about how you begin: not by pouring in all the data at once, but by growing the model one tested layer at a time. One of the most frequent traps in ABM development is attempting to incorporate every available dataset and behavioural rule into the initial script. This almost inevitably produces a model that is impossible to debug or verify. A rigorous workflow instead adopts an incremental approach, which can be described as moving from v0 to v10.

You should begin with a “v0” skeleton: a minimal Python script where agents move or interact on a basic grid with placeholder data. As you add layers of empirical complexity, such as GIS layers or real-world census distributions, you must perform Verification (checking that the code actually does what you intended, with no bugs) and Sensitivity Analysis (SA: changing each input a little to see how much the results shift, which reveals which inputs really matter) at every stage. Verification asks whether the code is right; validation, which asks whether the model is a faithful picture of reality, is a separate step treated in 5.4.3.

Ligmann-Zielinska and Sun (2010)[1] emphasise that a static snapshot of model sensitivity taken only at the final time step is inadequate as SA, because the influence of specific factors can vary considerably throughout model execution. They argue that the nonlinear processes inherent in complex systems necessitate time-dependent sensitivity analysis to investigate the evolving relationship between inputs and outputs [1]. This iterative evaluation should be complemented by the spatially explicit sensitivity analysis described by Ligmann-Zielinska (2013)[2], which allows you to map exactly where certain parameters contribute most to outcome uncertainty across the landscape. By using maps directly in these calculations, you can identify “sensitivity hot spots” and ensure that the spatial drivers of your model are correctly prioritised rather than lost in aggregate statistics [2]. Furthermore, performing joint uncertainty and sensitivity analyses at every stage is essential for building parsimonious models, because it lets you eliminate non-influential factors and focus data refinement on the parameters that truly drive model variance [3]. This systematic approach ensures that the model remains a credible tool for both scientific explanation and policy exploration throughout its development[3].

The incremental workflow just described has a definite shape, and it is worth fixing that shape in your mind before you rely on it. You begin from a skeleton so simple it barely does anything, add a single layer of empirical detail, and only once you have verified the code and checked its sensitivity do you allow yourself to add the next layer. Building in this order matters because it keeps each new effect traceable to the step that introduced it, which is what makes a growing model possible to debug at all, whereas pouring every dataset in at once produces something no one can untangle. In the activity below, the steps of this workflow appear in a jumbled order, from the first bare skeleton to a verified and validated model.

Learning Activity: Build it up, v0 to v10

see why models are best grown step by step, not built all at once.

The steps of building a model incrementally, from a bare v0 skeleton to a finished, validated model, presented shuffled and out of order.

Click the shuffled steps one at a time to drop them into the numbered slots in the order you would carry them out (click a placed step to send it back), then press Check order, or Reset to start again.

Your order

Steps (click to place)

Building up in small, tested steps is one of the most reliable habits in modelling, because every new effect can be traced back to the step that introduced it. Verifying and checking sensitivity after each addition, rather than at the very end, is what keeps that trace intact.

5.4.2: Calibrate, and beware equifinality#

Once the model has been built up layer by layer, its parameters still have to be tuned so that its output matches the data, and that tuning is calibration. Calibration is the process of adjusting model parameters to fit empirical data, and it can be enhanced using machine learning techniques such as Genetic Algorithms, as demonstrated by Stonedahl and Wilensky (2011)[4]. This method allows for an efficient search through the parameter space to find combinations that best replicate observed phenomena. In this paper, the authors use evolutionary search algorithms to explore the parameter space of two different flocking models, demonstrating how this approach can identify parameter settings that lead to specific qualitative behaviours such as convergence, non-convergence, volatility, and the formation of vee shapes [4].

Calibration sounds like a simple aim, that is, to find the parameter values that make the model match the data, but it hides a subtlety that every modeller has to reckon with. Very often more than one setting of the parameters fits the observations equally well, a situation known as equifinality, and when that happens the data alone cannot tell you which setting is the true one. This matters because a good fit is easily mistaken for a correct explanation, and a model tuned to reproduce the past can still rest on parameter values that do not reflect the real mechanism. The activity below makes equifinality concrete. Two parameters, A and B, can be fitted to the observed data, and a second, quite different pair turns out to fit just as well, because the fit depends only on the product of A and B. A good fit therefore cannot tell you which pair reflects the real mechanism.

Learning Activity: Calibration challenge

practise calibrating a model to match observed behaviour.

An interactive plot where grey dots are observed data and a blue model curve is controlled by two parameters, with a hidden twist: more than one pair of parameters fits the data equally well.

Drag the A and B sliders to tune the curve through the dots until the fit percentage climbs, then click Show a different fit (equifinality) to reveal another pair that matches the same data just as well.

A 1.0

B 1.0

fit 0%

The lesson to take from this is that a good fit is not proof that the parameters are right. Because different values can reproduce the same data, calibration needs to be backed by other evidence and by honesty about which parameters the data can and cannot pin down.

5.4.3: Validate at micro and macro levels#

Because a good fit is not proof that the parameters are right, calibration alone cannot tell you whether the model is a faithful picture of the real system; for that you must validate it against reality. Validation is crucial for ensuring that your model accurately represents the real-world system it is intended to simulate. This can be done at both micro and macro levels, and it often involves comparing model outputs to stylized facts or empirical data.

5.4.3.1: Micro-level validation#

On a micro level, you can validate individual agent behaviours against observed data or known decision-making processes. In an evacuation model, for example, you can validate the micro-level behaviours against observed evacuation data, such as exit times or the movement patterns of the agents. In this case, it would be illogical for an agent to walk through walls or past an exit without leaving. Grounding the micro-level behaviours of agents is essential, because you cannot expect to obtain the right macro-level patterns if the micro-level behaviours are unrealistic.

5.4.3.2: Macro-level validation#

On a macro level, you can compare aggregate model outputs to real-world patterns or trends. For example, you might validate a land-use change model by comparing its predictions to historical land-use data or by checking if it can reproduce known spatial patterns of urban growth. It is important to make sure that the patterns generated by the model are not just a result of the model’s structure but are actually reflective of the underlying processes in the real world.

Validating your model at multiple levels and against multiple types of data is important for ensuring that it is robust and credible. Such validation combines micro-level checks against observed behaviours with macro-level checks against aggregate patterns and stylised facts. The evidence you draw on can include the literature, empirical data, historical facts, expert opinion, and face validation.

Equifinality, met during calibration in 5.4.2, is exactly why this matters: just as different parameter values can produce the same fit, different socio-behavioural mechanisms can lead to the same outcome, making it difficult to determine which mechanism is actually at play in the real world. This is a common issue in complex systems modelling, and it is the reason validation must draw on multiple lines of evidence rather than a single match, alongside being transparent about the assumptions and limitations of your model. Troost et al. (2022)[5] discuss a protocol for ensuring the validity of ABMs.

Validation checks come in two kinds, and telling them apart is a useful habit because the two answer different questions. A micro-level check asks whether individual agents behave plausibly, for instance whether an agent’s exit time matches observed data or whether it ever walks through a wall, while a macro-level check asks whether the system as a whole reproduces real aggregate patterns, such as known urban growth or the stop-and-go of real traffic. The distinction matters because a model can pass at one level and still fail at the other, and a system-level match built on implausible individuals is not trustworthy, just as sensible individuals can still add up to the wrong whole. The activity below presents a set of validation checks, each of which is either a micro-level or a macro-level check.

Learning Activity: Micro or macro validation?

tell micro-level validation from macro-level.

Six ways of checking a model against reality, ranging from whether one agent behaves correctly to whether the whole system reproduces the right pattern.

For each row, click Micro validation (does an individual agent behave right?) or Macro validation (does the whole system reproduce the right pattern?), then press Check answers to see your score.

The point to remember is that credible validation works at both levels at once. Plausible individuals and realistic system-level patterns are each necessary, and a model that satisfies only one of them has not really been validated.

5.4.4: Generalise beyond the single case#

A model that is faithfully validated is still only useful to others if its lessons travel; a perfect fit to one case teaches nothing if nothing about it transfers. A central challenge in agent-based modelling is balancing the representation of complex, context-dependent systems with the need to generate insights that extend beyond a single case. This challenge is reflected in the notion of “YAAWN” (Yet Another Agent-Based Model), which highlights the risk that a proliferation of highly specific, case-driven models does not cumulatively contribute to generalisable knowledge (O’Sullivan et al., 2015)[6]. While such case studies are often necessary to capture the heterogeneity of social-ecological systems, their scientific value depends on whether they reveal underlying behavioural mechanisms that are transferable across contexts. In this regard, the modelling of agent decision-making plays a critical role. Despite widespread recognition of the complexity of human behaviour, many ABMs still rely on simplified assumptions such as perfectly rational agents, often without explicit justification of their fit to the decision context (Wijermans et al., 2023)[7]. This limits both the explanatory power and the generalisability of the model outcomes.

To address this, there is a growing emphasis on grounding agent behaviour in established social and behavioural theories. Integrating such theories allows modellers to move beyond ad hoc rule specification and instead formalise mechanisms that are conceptually robust and empirically informed (Constantino et al., 2021)[8]. At the same time, the diversity and fragmentation of behavioural theories pose challenges for their systematic integration into ABMs, requiring explicit reflection on theory selection and model design choices. Importantly, recent research suggests that theory-driven models can perform comparably to empirically calibrated models when behavioural mechanisms are carefully specified and justified, particularly in contexts involving incremental behavioural change (Taghikhah et al., 2021)[9]. This underscores that generalisability does not necessarily require fully data-driven microfoundations, but rather a transparent and theoretically consistent representation of decision processes.

Therefore, avoiding “drowning in complexity” in ABMs is not achieved by increasing model detail, but by selectively incorporating mechanisms that are both theoretically grounded and relevant to the research question. By prioritising clarity in behavioural assumptions and explicitly linking them to broader theoretical frameworks, ABMs can contribute to cumulative knowledge and avoid the pitfalls of isolated, case-specific modelling exercises.

Whether a model teaches anything beyond its own case comes down to the choices made while building it. Grounding agent behaviour in an established theory, or identifying a mechanism that recurs in other settings, produces insight that travels, whereas adding rules that fit one case with no wider justification, or assuming perfectly rational agents without checking the fit, ties the model to that case and no further. This distinction is the heart of the YAAWN concern, that is, the worry that yet another one-off model adds little to shared knowledge, and keeping it in view is what separates a case study that contributes from one that merely exists. The activity below makes this concrete. You can bend a model to fit the observed data ever more closely, but a model tuned to one dataset captures its noise rather than the underlying mechanism, and then fails on data it has not seen.

Learning Activity: Fit, then test on new data

see why a model must be tested on data it was not fitted to.

An interactive overfitting demonstration where an orange model curve can be bent towards blue observed dots, then judged against fresh green observations it has never seen.

Drag Model flexibility to bend the curve towards the observed dots, then press Test on new data to reveal fresh observations and compare how well the model does on those; press Reset to start again.

observed datanew datamodel

Model flexibility2

Fit to observed data

--

Fit to new data

--

The lesson is that generalisability is designed in, not added later. A case study earns wider value when it rests on mechanisms grounded in theory that carry over to other contexts, rather than on rules tuned to fit one situation.

5.4.5: Keep the model transparent, not a black box#

Beyond building, calibrating, validating, and generalising, a final set of practices governs how the finished model is run, reported, and shared, and together they serve one aim: to keep the model open to inspection rather than shut away as a black box. Alongside the careful selection of behavioural mechanisms, avoiding unnecessary complexity in agent-based models requires adherence to good modelling practices that enhance transparency, interpretability, and analytical rigour. Three of these practices deserve particular attention.

5.4.5.1: Design of experiments#

First, the design of experiments should be explicitly defined and justified. This involves clearly specifying which parameters or structural elements of the model are varied, what values they take, and how these variations relate to the research questions. A transparent design also states the baseline, number of replications and random seeds, duration or stopping rule, response variables, and analysis planned for the resulting outputs. In an empirical ABM, parameter ranges should be tied to observations or defensible scenarios rather than chosen solely because they produce interesting behaviour. Recording this design before inspecting the results separates hypothesis-driven comparison from post-hoc searching and lets another modeller reconstruct the same experiment.

5.4.5.2: Report multidimensional outputs#

Second, model outputs should not be reduced to a single aggregated key performance indicator. Given the micro-to-macro nature of ABMs, meaningful insights often emerge from analysing heterogeneous outcomes across agents and subgroups, as well as from tracing the processes that generate system-level patterns. Reporting multidimensional outputs, such as distributions, agent-level dynamics, and intermediate states, helps to uncover underlying mechanisms rather than obscuring them through aggregation.

An average can remain stable while inequality widens, a small vulnerable subgroup fares badly, or two spatially distinct patterns cancel one another out. A final value can likewise hide whether the system approached that value smoothly, oscillated, or crossed a critical threshold along the way. The reported outputs should therefore follow the claim: include distributions when heterogeneity matters, time series when pathways matter, maps or network measures when structure matters, and uncertainty across repeated runs when stochastic variation matters. These views complement rather than replace the system-level indicator.

5.4.5.3: Document and share (ODD+D, reusable blocks)#

Finally, rigorous model documentation is essential to ensure both transparency and cumulative knowledge building. Protocols such as ODD+D (Overview, Design concepts, Details + Decision-making) provide a structured approach to describing ABMs, with particular emphasis on explicitly documenting how agent decision-making is modelled and justified (Müller et al., 2013)[10]. By incorporating a dedicated section on the theoretical and empirical background of behavioural assumptions, ODD+D addresses a common shortcoming in ABM practice: the lack of clear rationale for chosen decision rules and the difficulty of comparing models across studies. Adopting such standards not only improves reproducibility, but also facilitates critical evaluation and reuse of models, thereby helping to prevent the development of opaque and overly complex “black-box” simulations.

Building on this, recent work has focused on improving the transparency and reusability of such complex representations. For instance, Filatova et al. (2025)[11] highlight how the community is now moving towards “reusable building blocks” to standardise these complex representations, making models of various STE systems more transparent and easier to validate.

The good practices set out above share a single purpose, which is to keep a model open to inspection rather than shut away as a black box. Stating exactly which parameters you varied and why, reporting distributions and agent-level outcomes instead of one aggregated number, documenting the model with a standard protocol, and sharing reusable parts all let others follow and check what the model does, whereas reporting a single figure, leaving the decision rules undocumented, or hiding the key assumptions in the code all close it off. Transparency matters because it is what turns a private program into a scientific result that others can scrutinise, trust, and build on, and it is the surest guard against the opaque, overly complex simulations these practices are meant to prevent. The activity below presents a set of modelling habits, some that keep a model transparent and some that turn it into a black box.

Learning Activity: Transparent or black box?

weigh a transparent model against a black-box one.

Seven working habits a modeller might have, some that keep a model open to inspection and some that hide its workings away.

For each row, click Transparent (others can understand and check it) or Black box (its workings are hidden), then press Check answers to see your score.

The habit to carry away is that transparency is not an optional extra at the end of a project. Explicit experiments, multidimensional outputs, standard documentation, and reusable parts are what let others inspect a model and treat its results as science rather than a black box.

These five practices are the path the opening of this section promised: they are how raw datasets become a scientifically robust model, and how the complexity of the data is kept matched to the maturity of the simulation logic. Building incrementally keeps that growth traceable, calibration tunes it to the data, validation tests it against reality, generalisation makes its lessons travel, and transparent documentation lets others check and reuse it. None of the five stands alone; taken together they turn empirical grounding from a one-off act of data-loading into the continuous, iterative discipline the section set out to describe. With this, the chapter’s account of empirical ABMs, from what grounding means, through what can be grounded and with what data, to how to build responsibly, is complete.

Key takeaways#

Key takeaways

Build incrementally (v0 → v10): start from a minimal skeleton and add empirical complexity in layers, verifying and running (time-dependent, spatially explicit) sensitivity analysis at every stage.
Calibrate parameters against observed data, using efficient search methods such as genetic algorithms to explore the parameter space.
Validate at both micro and macro levels against behaviours, aggregate patterns, and stylised facts, and use multiple lines of evidence to guard against equifinality.
Aim for generalisability by grounding agent behaviour in established theory and selectively including mechanisms relevant to the research question, rather than piling on detail.
Document rigorously using protocols such as ODD+D, report multidimensional (not single-KPI) outputs, and favour reusable building blocks to keep models transparent and reproducible.