Parallel Distance Matrix Requests with aiohttp

Computing a travel-time matrix for 500 origins against 500 destinations means either one request that no engine will accept, or thousands of tiny requests that saturate a connection pool and starve each other. This page covers the specific mechanics of parallelizing origin-destination matrix requests with aiohttp inside the broader async route matrix automation workflow, which also covers semaphore-gated concurrency and chunk tiling at a conceptual level. It assumes the fleet automation context described in routing API automation and fleet integration — geocoded stops that need an NxN or NxM cost matrix before a solver can run.

The problem is not raw request volume; aiohttp can issue thousands of requests per second. The problem is that OSRM’s /table endpoint and Valhalla’s /sources_to_targets endpoint both enforce hard coordinate-count ceilings per request, and both degrade badly under uncontrolled concurrency — OSRM’s single-threaded Lua-free table handler queues behind itself, and Valhalla’s thread pool thrashes when the number of in-flight requests exceeds its worker count by a wide margin. Getting this right means matching three independent limits: the engine’s per-request coordinate cap, the engine’s server-side worker concurrency, and the client’s connection pool size.

When to Use This Approach

Use parallel aiohttp fan-out when the origin-destination set exceeds a single engine request’s coordinate ceiling, or when a matrix that fits in one request still takes long enough (multiple seconds) that splitting it into concurrent sub-requests reduces wall-clock time. Typical triggers:

• The combined origin and destination coordinate count exceeds OSRM’s --max-table-size (default 8000) or Valhalla’s configured max_matrix_distance / max_locations for the costing model in use.
• A single-request matrix returns in acceptable time but you are computing matrices for many depots or many time-of-day snapshots back to back, and total batch throughput — not single-matrix latency — is the bottleneck.
• The origin set changes faster than the destination set (or vice versa), so caching destination-side sub-matrices and only re-requesting the changed rows saves real request volume.

Skip this pattern when the full matrix fits comfortably under the engine’s request cap and the query is one-off — a single synchronous /table call is simpler and easier to debug. It also does not replace chunking large origin-destination matrices, which handles how chunk boundaries are chosen; this page assumes chunks already exist and focuses on the transport layer that fetches and reassembles them.

Implementation

The snippet below issues chunk requests concurrently against an OSRM /table endpoint (the same shape applies to Valhalla with a POST body swap, noted inline), gates concurrency independently from the connection pool, and writes each response directly into a pre-allocated NumPy matrix at its known offset — no post-hoc merge step required.

# requires: aiohttp, numpy (pip install aiohttp numpy)
# Python 3.9+

import asyncio
import numpy as np
import aiohttp

OSRM_BASE = "http://localhost:5000"
POOL_LIMIT = 20          # total open connections in the TCPConnector
CONCURRENCY = 12         # in-flight requests; must be <= POOL_LIMIT
REQUEST_TIMEOUT = aiohttp.ClientTimeout(total=30, connect=5, sock_read=20)


async def fetch_chunk(
    session: aiohttp.ClientSession,
    sem: asyncio.Semaphore,
    origins: list[tuple[float, float]],
    dests: list[tuple[float, float]],
    row_offset: int,
    col_offset: int,
    out: np.ndarray,
    attempt: int = 0,
) -> None:
    """Fetch one OSRM /table chunk and write durations directly into `out`."""
    coords = origins + dests
    coord_str = ";".join(f"{lon},{lat}" for lat, lon in coords)
    sources = ";".join(str(i) for i in range(len(origins)))
    destinations = ";".join(str(len(origins) + j) for j in range(len(dests)))
    url = f"{OSRM_BASE}/table/v1/driving/{coord_str}"
    params = {"sources": sources, "destinations": destinations, "annotations": "duration"}

    async with sem:
        try:
            async with session.get(url, params=params, timeout=REQUEST_TIMEOUT) as resp:
                resp.raise_for_status()
                payload = await resp.json()
        except (aiohttp.ClientError, asyncio.TimeoutError) as exc:
            if attempt < 3:
                await asyncio.sleep(0.5 * (2 ** attempt))
                return await fetch_chunk(
                    session, sem, origins, dests, row_offset, col_offset, out, attempt + 1
                )
            raise RuntimeError(f"chunk [{row_offset}:{col_offset}] failed after retries") from exc

    block = np.array(payload["durations"], dtype=np.float64)
    out[row_offset:row_offset + block.shape[0], col_offset:col_offset + block.shape[1]] = block


async def build_matrix(
    origin_chunks: list[list[tuple[float, float]]],
    dest_chunks: list[list[tuple[float, float]]],
) -> np.ndarray:
    n_rows = sum(len(c) for c in origin_chunks)
    n_cols = sum(len(c) for c in dest_chunks)
    matrix = np.full((n_rows, n_cols), np.nan, dtype=np.float64)

    connector = aiohttp.TCPConnector(limit=POOL_LIMIT, limit_per_host=POOL_LIMIT)
    sem = asyncio.Semaphore(CONCURRENCY)

    async with aiohttp.ClientSession(connector=connector) as session:
        tasks = []
        row_offset = 0
        for origins in origin_chunks:
            col_offset = 0
            for dests in dest_chunks:
                tasks.append(
                    fetch_chunk(session, sem, origins, dests, row_offset, col_offset, matrix)
                )
                col_offset += len(dests)
            row_offset += len(origins)
        await asyncio.gather(*tasks)

    return matrix

Valhalla variant. Swap session.get for session.post(f"{VALHALLA_BASE}/sources_to_targets", json={...}) with a body of {"sources": [{"lat": ..., "lon": ...}, ...], "targets": [...], "costing": "auto"}, and parse payload["sources_to_targets"] — a list of lists of {"time": ..., "distance": ...} dicts — into the same block-write pattern. Valhalla’s response nests time and distance per cell rather than returning two parallel arrays like OSRM, so extract with np.array([[c["time"] for c in row] for row in payload["sources_to_targets"]]).

Key Parameters and Tuning

Connector vs. semaphore, not either/or. A common mistake is setting only the TCPConnector limit and assuming it caps concurrency. It caps the connection pool, not in-flight coroutines — with no semaphore, asyncio.gather will schedule every chunk task immediately, and excess tasks block silently inside the connector waiting for a socket rather than failing or logging anything. Always pair the two, with the semaphore value at or below the connector limit.

Engine-side worker count matters more than client concurrency. OSRM’s HTTP server is typically compiled with a fixed thread pool (-t at launch, or environment default). Setting client concurrency higher than the OSRM thread count does not parallelize computation — it just queues requests server-side, which is functionally equivalent to sending them sequentially but with added HTTP overhead per request. Match CONCURRENCY to the engine’s actual worker count, verified with osrm-routed --help or the container’s startup logs, as described in deploying OSRM with Docker for local routing.

Integration Points

Upstream: chunk boundaries. This page assumes origin_chunks and dest_chunks already exist as lists of coordinate lists sized under the engine’s per-request cap. See chunking large origin-destination matrices for how to compute chunk sizes from --max-table-size, exploit origin/destination symmetry, and reuse previously computed blocks.

Downstream: solver input. The assembled matrix array is a direct drop-in for OR-Tools’ RoutingIndexManager distance callback or any VRP solver expecting a dense cost matrix — index it as matrix[from_index][to_index] after converting to int (most solvers require integer costs; multiply by 100 and round if sub-second precision matters).

Engine choice affects chunk shape. If comparing engines for the same fleet workload, note that OSRM’s coordinate ceiling is a flat count across sources and destinations combined, while Valhalla enforces max_matrix_distance as a geographic bound in addition to a location count — a chunk sized correctly for OSRM may still be rejected by Valhalla if origins and destinations span a wide area. The trade-offs are covered in more depth for freight-specific routing in OSRM vs Valhalla vs GraphHopper for freight routing.

Caching partial results. Because each chunk writes to a known (row_offset, col_offset) block, a chunk-level cache keyed on the origin/destination coordinate hash lets you skip re-fetching unchanged blocks when only a subset of stops changes between runs — useful for daily depot-to-stop matrices where the depot side rarely moves.

Validation Checklist

1. No NaN cells remain. After build_matrix completes without raising, assert not np.isnan(matrix).any(). Any surviving NaN marks a chunk that silently failed all retries and was swallowed — check asyncio.gather(*tasks, return_exceptions=True) output during development to catch this before it reaches production.

2. Shape matches expected dimensions. matrix.shape == (n_origins, n_destinations) — a mismatch usually means a chunk’s coordinate order in the OSRM response did not match the order sources/destinations indices were built in.

3. Semaphore value never exceeds connector limit. Assert this at startup: CONCURRENCY <= POOL_LIMIT. Violating it does not raise an error — it degrades silently into queuing, which is the hardest version of this bug to diagnose in production.

4. Symmetric-route sanity check. For a handful of origin-destination pairs, compare matrix[i][j] against a direct single-pair /route request. Differences beyond a few percent usually indicate a coordinate order bug (lon,lat vs lat,lon — OSRM expects lon,lat in the URL, the opposite of most GeoJSON tooling).

5. Timeout coverage under load. Run the full batch against a rate-limited or intentionally slow local proxy and confirm no request hangs past REQUEST_TIMEOUT.total. A missing sock_read timeout is the most common cause of a batch that never completes.

6. Retry count stays bounded. Log retry attempts per chunk; if a large fraction of chunks hit the 3-attempt ceiling, CONCURRENCY is likely too high for the engine’s worker pool rather than the network being unreliable.

Related

• Async route matrix automation — the parent guide covering concurrency control, chunk tiling strategy, and matrix assembly at a conceptual level
• Chunking large origin-destination matrices — how to size and partition chunks before they reach the fan-out layer described here
• Deploying OSRM with Docker for local routing — configuring the OSRM thread pool and container resources that bound realistic client concurrency
• OSRM vs Valhalla vs GraphHopper for freight routing — how each engine’s matrix size limits differ when choosing a production routing backend