Replication diffs are the normal way to keep an extract current, but they only work when an unbroken chain of them exists between your snapshot and now. When a region has no replication directory, when the retention window has expired, or when a pipeline outage left a gap nobody noticed, the chain is unavailable and the usual approach in OSM data freshness and incremental updates cannot be applied. osmium derive-changes closes that gap by computing the difference between two snapshots directly, producing a change file you can apply exactly as if it had come from a replication server — a technique that fits into the broader OSM Graph Architecture & Network Modeling pipeline without disturbing anything downstream.

The specific case this page covers is narrow: you hold two full OSM files covering the same area at different times, and you need the change file between them. That is a different problem from fetching diffs, and it has one dominant failure mode — a mismatch in clip area between the two snapshots, which turns an ordinary update into a mass deletion.

When to use this approach

Reach for derive-changes when:

  • The region has no replication directory. Custom extracts, internally produced clips, and some third-party regional files ship without an accompanying -updates path. There is nothing to fetch, but successive downloads of the same file still differ.
  • The retention window has expired. Replication servers keep a finite history. An extract that has fallen months behind cannot be brought forward with diffs, and re-downloading a current snapshot plus deriving the change is often cheaper than a full reprocess, because the derived change file feeds the same incremental rebuild path.
  • You need to audit what changed between two known-good builds. Deriving the change between the extract that produced last month’s graph and the one that produced this month’s gives an exact, inspectable object list — much more useful for explaining a route regression than a timestamp comparison.
  • A replication gap is suspected. Deriving the change between your incrementally maintained extract and a freshly downloaded one should produce an empty change file. Anything else is proof that the diff chain lost something.

Do not reach for it as a routine substitute for replication. Deriving a change requires reading both full snapshots, so it costs roughly what a fresh extract download costs; replication diffs cost a fraction of that. It is a repair and audit tool, not a scheduling one.

Where a derived change file fits against the replication chain Along a timeline, replication diffs carry an extract forward hour by hour until a gap where the retention window expired. A derived change file computed between the stale extract and a freshly downloaded snapshot bridges that gap in one step. Two ways to move an extract forward in time seq 4120 replication diffs retention window expired — no diffs here today where diffs exist apply-changes per interval, cheap reads only the diff, not the snapshot across the gap derive-changes between old and new snapshots reads both files in full — costs a download The derived file plugs into the same apply-changes step, so everything downstream of it stays identical.

Implementation

The derivation itself is a single command. The care goes into what surrounds it — confirming the two inputs are comparable, and inspecting the output before it touches anything.

# Both inputs must cover the same area, cut the same way
osmium fileinfo -e region-2026-05.osm.pbf | grep -Ei 'box|timestamp'
osmium fileinfo -e region-2026-08.osm.pbf | grep -Ei 'box|timestamp'

# Derive the change between them
osmium derive-changes \
  region-2026-05.osm.pbf region-2026-08.osm.pbf \
  --output changes-may-to-aug.osc.gz \
  --update-timestamp \
  --overwrite

--update-timestamp matters more than its brevity suggests. Deletions in a derived change file have no natural timestamp — the object is simply absent from the newer file — so without the flag osmium reuses the timestamp the object carried in the older one. Consumers that compare timestamps then see a deletion apparently older than the data it is being applied to and may discard it, leaving the object alive in a graph that should have dropped it.

Before applying anything, look at the composition of what was derived. A change file that is overwhelmingly deletions is almost never a real update.

# requires: osmium (pip install pyosmium)
import osmium
from collections import Counter


class ChangeAudit(osmium.SimpleHandler):
    """Summarise a change file by object type and modification kind."""

    def __init__(self) -> None:
        super().__init__()
        self.counts: Counter[str] = Counter()

    def _record(self, obj, kind: str) -> None:
        # A visible=False object in a change file is a deletion
        action = "delete" if not obj.visible else ("create" if obj.version == 1 else "modify")
        self.counts[f"{kind}.{action}"] += 1

    def node(self, n) -> None:
        self._record(n, "node")

    def way(self, w) -> None:
        self._record(w, "way")

    def relation(self, r) -> None:
        self._record(r, "relation")


audit = ChangeAudit()
audit.apply_file("changes-may-to-aug.osc.gz")

total = sum(audit.counts.values())
deletes = sum(v for k, v in audit.counts.items() if k.endswith(".delete"))
delete_share = deletes / total if total else 0.0

for key in sorted(audit.counts):
    print(f"  {key:22s} {audit.counts[key]:>9,}")
print(f"\n  deletions are {delete_share:.1%} of {total:,} changes")

if delete_share > 0.25:
    raise SystemExit(
        "Deletion share above 25% — the two snapshots almost certainly differ in "
        "clip area rather than in content. Re-clip both to an identical boundary."
    )

The 25 % guard encodes the dominant failure mode. Real OSM editing is overwhelmingly additive: creates and modifies vastly outnumber deletions over any ordinary interval. A change file where a quarter of the entries are deletions is describing an area difference, not a time difference, and applying it will strip real roads out of the graph.

If the two snapshots genuinely were cut differently, normalise them before deriving:

# Cut both to one authoritative boundary, complete-ways so nothing dangles
for f in region-2026-05 region-2026-08; do
  osmium extract \
    --polygon service-area.geojson \
    --strategy complete_ways \
    "${f}.osm.pbf" --output "${f}.clipped.osm.pbf" --overwrite
done

With a clean change file in hand, applying it is the same operation the replication path uses:

osmium apply-changes \
  region-2026-05.clipped.osm.pbf changes-may-to-aug.osc.gz \
  --output region.updated.osm.pbf --overwrite

osmium fileinfo -e region.updated.osm.pbf
mv region.updated.osm.pbf region.osm.pbf

Key parameters and tuning

Parameter Recommended value Notes
--update-timestamp always set Without it, derived deletions carry stale timestamps and may be discarded by the consumer
Clip strategy for both inputs complete_ways Ways partially inside the boundary must arrive whole in both snapshots or they read as edits
Deletion-share guard 0.25 Above this, suspect an area mismatch rather than genuine deletions; tune down for very short intervals
Output compression .osc.gz Change files compress extremely well; uncompressed offers no meaningful speed gain
Snapshot age difference under 6 months Beyond that a full rebuild is usually cheaper than deriving and incrementally applying
--simplify off by default Collapses multiple versions into one; harmless for routing, destructive for history auditing
Working disk 3× the larger snapshot Both inputs are read and a third file is written

Integration points

Feeding the incremental rebuild. The derived change file is indistinguishable from a replication diff downstream, so the classification and partition-mapping steps described in incremental graph rebuilds without full reprocessing consume it unchanged. That is the main reason to derive a change file at all rather than simply swapping in the newer snapshot: swapping loses the information about what changed, and with it the ability to rebuild only the affected partitions.

Auditing a route regression. When a rebuild changes routes unexpectedly, deriving the change between the two extracts that produced the two graphs gives an exact object list to inspect. Filtering that list to routing-relevant tags, as covered in detecting breaking tag changes between PBF snapshots, usually identifies the culprit in minutes.

Verifying a replication pipeline. Derive the change between your incrementally maintained extract and a freshly downloaded snapshot of the same date. A correct pipeline yields an empty change file. Anything else quantifies exactly what the diff chain lost, and the object ids point straight at the interval where it happened.

Reading the composition of a derived change file A healthy three-month derivation is dominated by modifications and creations with a small deletion share. A derivation across mismatched clip areas is dominated by deletions, which is the signature of an area difference rather than a time difference. Deletion share is the tell same clip, three months apart modify 61 % create 34 % delete 5 % — normal apply it different clip areas delete 72 % modify 20 % create 8 % re-clip both inputs, derive again OSM editing is overwhelmingly additive. A deletion-heavy change file is describing a difference in area, and applying it removes real roads. The guard is worth keeping in the pipeline permanently — the mismatch usually arrives with a re-download nobody flagged.

Validation checklist

  • Both inputs report the same bounding box. osmium fileinfo -e on each; any difference means re-clipping before deriving.
  • Deletion share is below the guard. The audit above should exit cleanly. If it does not, the inputs are the problem, not the derivation.
  • Applying the change reproduces the newer snapshot. Derive the change from the applied result to the newer snapshot; it must be empty. This is the definitive end-to-end check and takes one extra command.
  • Routable way count moved in a plausible direction. Count highway=* ways before and after. A three-month interval in an active region typically adds a fraction of a percent; a drop is worth explaining before promoting the extract.
  • The result carries a usable timestamp. osmium fileinfo -e on the applied output should show the newer snapshot’s timestamp, so the next replication run can find its start sequence.
The round-trip check that proves the derivation was complete The old snapshot plus the derived change file produces an updated extract. Deriving a second change between that updated extract and the new snapshot must yield zero objects; anything else means the first derivation lost something. Derive, apply, then derive again — the second file must be empty old snapshot apply derived change updated extract new snapshot derive again expect 0 objects A non-empty second file lists exactly what the first derivation missed, with object ids you can inspect directly. The same round trip verifies a replication pipeline: derive between the maintained extract and a fresh download of the same date. It costs one extra read of two files, which is cheap next to discovering the gap through a routing incident.