The most disruptive OSM changes for a routing graph are rarely the ones that move a road. They are the ones that leave every geometry untouched and change what the geometry means: a maxspeed added across a corridor, an hgv value corrected on a bridge, a highway class revised after a survey. These edits produce no visible difference on a tile map and no obvious signal in a change file, but they shift edge weights and therefore ETAs. This page covers the comparison that makes them visible, and it sits under OSM data freshness and incremental updates within the broader OSM Graph Architecture & Network Modeling pipeline, immediately upstream of the weight recomputation described in configuring edge weights for freight logistics.

The technique is deliberately blunt: count every routing-relevant key-value pair in both snapshots and compare the distributions. It does not tell you which specific way changed — the change file already does that — it tells you whether the population of tags shifted in a way that will move costs.

When to use this approach

Run a tag-distribution diff:

  • Before every weight recomputation, whether the extract arrived by replication or by fresh download. This is the cheapest available guard against a mass retagging silently rewriting your cost surface.
  • After an import lands. Large imports — address data, forestry tracks, a national road-class harmonisation — are announced on OSM channels but rarely reach the team running the routing pipeline. The distribution diff catches them without anyone needing to be subscribed.
  • When ETAs drift without an obvious cause. If the regression suite reports corridor ETAs moving on unchanged geometry, the tag diff between the two extracts almost always names the cause in one line.
  • When adopting a new extract source. Two providers clipping the same region can differ in what they include. Diffing their tag distributions surfaces the difference far faster than routing against both and comparing outputs.

It is not a substitute for object-level inspection. Once a pair is flagged, you still need the change file to see which ways moved; the diff tells you where to look, not what happened.

Object counts hide what tag counts reveal Two intervals each contain about twelve thousand modified ways. In the first, the routing tag distribution is essentially unchanged. In the second, hgv equals no rises by four thousand instances while everything else holds steady — invisible in the object count, obvious in the tag count. Same volume of edits, very different consequences interval A — 12 400 ways modified highway=residential +0.2 % maxspeed=50 +0.4 % hgv=no −0.1 % distribution stable — absorb automatically interval B — 12 100 ways modified highway=residential +0.1 % maxspeed=50 −0.2 % hgv=no +4 010 instances, +38 % — review first Interval B is a freight-access harmonisation. Nothing moved on the map, and every heavy-vehicle route through the region is about to change.

Implementation

Counting is a single streaming pass per snapshot. Restrict the keys up front — counting every tag in OSM produces a distribution dominated by addresses and names that swamps the signal.

# requires: osmium (pip install pyosmium)
import osmium
from collections import Counter

# Keys whose values change what an edge costs or whether it is traversable
ROUTING_KEYS = (
    "highway", "oneway", "access", "motor_vehicle", "hgv", "bicycle", "foot",
    "maxspeed", "maxheight", "maxweight", "maxaxleload", "maxwidth",
    "surface", "tracktype", "smoothness", "junction", "barrier",
    "bridge", "tunnel", "toll", "lanes", "service",
)


class TagCounter(osmium.SimpleHandler):
    """Count routing-relevant key=value pairs across a snapshot's ways."""

    def __init__(self, keys: tuple[str, ...]) -> None:
        super().__init__()
        self.keys = frozenset(keys)
        self.pairs: Counter[str] = Counter()

    def way(self, w) -> None:
        # Ways carry the tags that matter for traversal cost
        for tag in w.tags:
            if tag.k in self.keys:
                self.pairs[f"{tag.k}={tag.v}"] += 1


def count_snapshot(path: str) -> Counter:
    counter = TagCounter(ROUTING_KEYS)
    counter.apply_file(path)
    return counter.pairs

Ways alone are enough for the common cases. Node-level tags such as barrier=gate also affect routing, and adding a node callback that filters on the same key set covers them at the cost of a slightly longer pass.

The comparison then needs a significance rule. A pure percentage misfires at both ends of the count scale, so combine a relative threshold with an absolute floor:

# requires: pandas (pip install pandas)
import pandas as pd

def diff_distributions(
    before: Counter,
    after: Counter,
    *,
    min_abs_delta: int = 250,
    min_rel_delta: float = 0.05,
) -> pd.DataFrame:
    """Return tag pairs whose count moved by both a meaningful count and share."""
    keys = sorted(set(before) | set(after))
    frame = pd.DataFrame(
        {
            "pair": keys,
            "before": [before.get(k, 0) for k in keys],
            "after": [after.get(k, 0) for k in keys],
        }
    )
    frame["delta"] = frame["after"] - frame["before"]
    # Guard the denominator so newly appearing pairs do not divide by zero
    base = frame["before"].clip(lower=1)
    frame["rel_delta"] = frame["delta"] / base

    flagged = frame[
        (frame["delta"].abs() >= min_abs_delta)
        & (frame["rel_delta"].abs() >= min_rel_delta)
    ]
    return flagged.reindex(
        flagged["delta"].abs().sort_values(ascending=False).index
    ).reset_index(drop=True)


before = count_snapshot("region-previous.osm.pbf")
after = count_snapshot("region-current.osm.pbf")
report = diff_distributions(before, after)

if not report.empty:
    print(report.to_string(index=False))
    raise SystemExit(
        f"{len(report)} tag distribution(s) moved significantly — "
        "review before recomputing edge weights."
    )

The two-condition rule is what makes this usable in an automated pipeline. Requiring both an absolute and a relative move means a rarely used key gaining a handful of instances stays quiet, and a very common key gaining a fraction of a percent stays quiet, while a genuine bulk edit trips both conditions comfortably.

Newly appearing pairs deserve a note. A pair absent from the earlier snapshot has a before of zero, which the clip turns into a relative delta equal to its count — always above threshold. That is usually the behaviour you want, because a value that did not previously exist in the region is exactly the kind of thing a weight-mapping table has no rule for.

Key parameters and tuning

Parameter Recommended value Notes
ROUTING_KEYS 20–25 keys Wider sets drown the signal in address and name churn; narrower ones miss access-rule flips
min_abs_delta 250 for a metro extract Scale roughly with extract size — a national file justifies 1 000–2 000
min_rel_delta 0.05 Below this, ordinary organic editing trips the rule most weeks
Object types counted ways, optionally nodes Ways carry traversal cost; nodes matter for barriers and crossings
Comparison cadence every extract update The pass is cheap relative to any rebuild and only useful if it runs every time
Baseline storage JSON alongside the extract Lets the next run compare against the accepted state rather than an arbitrary snapshot
Failure mode block weights, not topology Geometry changes are safe to absorb; cost-semantics changes are not

Integration points

Gating the weight recomputation. Wire the check between the extract update and the cost-assignment stage. A clean result lets the pipeline continue unattended; a flagged result stops before weights are recomputed, leaving the previous cost surface serving traffic while somebody looks. This is a different gate from the connectivity audit described in graph fragmentation prevention in OSM data, which catches topology damage rather than semantic drift.

Explaining a route regression. When the ETA regression suite fires, run the diff between the two extracts before opening the change file. In most cases one flagged pair explains the whole regression, and the object-level investigation becomes a targeted query rather than a search.

Feeding the incremental rebuild decision. A flagged pair is a strong signal that a rebuild is warranted even when the raw change count is below the threshold used in incremental graph rebuilds without full reprocessing. Volume and significance are different questions, and this check answers the second.

Maintaining the tag-mapping table. Newly appearing values are, in effect, a request for a mapping rule. A value the cost function has no branch for falls through to a default, and the diff is the only place that fall-through becomes visible before it reaches a route.

Why both an absolute and a relative threshold are needed Absolute delta is plotted against relative delta. A rare key gaining a few instances has a large relative move but a small absolute one. A very common key gaining a fraction of a percent has the opposite. Only pairs clearing both thresholds fall into the flagged region. Only pairs clearing both thresholds are flagged absolute delta relative delta min_abs_delta min_rel_delta flagged hgv=no · +4 010 · +38 % tracktype=grade4 · +40 · +31 % highway=residential · +900 · +0.2 % surface=asphalt · +60 · +0.1 % A relative-only rule flags the grade4 change every week; an absolute-only rule flags the residential churn every week. Neither is worth reviewing. Scale both thresholds with extract size — a national file needs a higher absolute floor to keep the flagged set readable. Where the gate sits, and what it blocks The extract update feeds both topology rebuild and weight recomputation. The tag-diff gate sits only on the weight branch, so a flagged change halts cost recomputation while topology continues unattended. Block the cost surface, not the topology extract updated topology rebuild runs unattended tag-distribution gate clean, or flagged recompute edge weights only once the gate is clean While the gate is held, the previous cost surface keeps serving traffic — stale by hours, not wrong. Blocking the whole pipeline on a tag flag would also stall geometry fixes, which are almost always safe to absorb immediately.

Validation checklist

  • A no-change comparison produces an empty report. Run the diff of a snapshot against itself; anything flagged means the counting pass is non-deterministic and needs fixing before the results mean anything.
  • A synthetic bulk edit is caught. Retag a few thousand ways in a copy of the extract and confirm the pair appears in the report with the expected delta. This calibrates the thresholds against your own extract size rather than against a guess.
  • Every flagged pair has a mapping rule. For each flagged key-value pair, confirm the cost function has an explicit branch. Pairs that fall through to a default are the ones that produce surprising weights.
  • Thresholds are stable across three ordinary intervals. If ordinary editing trips the rule most weeks, the thresholds are too tight and the report will be ignored — which is worse than not running it.
  • The accepted baseline is written after review. Persist the current counts once a report has been reviewed and accepted, so the next comparison measures from the reviewed state and not from an ever-older snapshot.