DNS Synchronization: Full Copies, rsync and Merkle Trees

The reliability of the Domain Name System (DNS), the internet's address book that turns human-friendly domain names into numeric IP addresses, depends on thousands of independent servers keeping their copies of data perfectly in step. In a 13 July 2026 APNIC blog post, Geoff Huston explores the tools used to synchronize replicated data, starting from a real-world incident: on the early morning of 8 July 2026, Australian network service provider Telstra suffered a large-scale outage that halted some urban and regional train services, caused point-of-sale terminal failures, and disrupted Telstra's mobile platform. A news report that day attributed the failure to a synchronization failure across data centres, which Huston notes is entirely plausible in a service provider platform hosted in many locations.

Huston first observes that today's internet runs on vast replicated data, and consistency across distributed points separates order from chaos. The Border Gateway Protocol (BGP), the routing protocol that lets internet providers exchange reachability information, keeps active routers in sync so each router can independently forward packets correctly. The DNS faces similar synchronization challenges. He then examines several approaches, beginning with brute force.

The simplest synchronization method is for a secondary server to pull a complete copy of the data from a primary source and overwrite its local copy. This full-copy approach requires only knowing where the original source is, and every other server retrieves the entire dataset whenever it needs to refresh. To decide whether a file has changed, systems typically compare the file's size and its modification date and time, which makes synchronized time across networked systems important. When such a uniform view of time is not available, the comparison must rely on a more expensive calculation such as an MD5 or SHA hash of the file contents. This is the model used by FTP-based file system mirror synchronization. It is simple but extremely inefficient: log files that are written in append mode are transferred in full each time, any altered file is downloaded again in its entirety, and very large collections require client and server to walk through the entire file space checking every file. Huston also points out that FTP in mirror mode is a poor choice because it transfers data in clear channel and exchanges credentials openly.

One improvement is rsync, designed by Andrew Tridgell in 1996. rsync treats a file as a sequence of blocks and updates only the blocks that differ, rather than the whole file. The receiver sends to the sender a list of file names and checksums of each block; where checksums differ, the sender sends a replacement block. A block is defined by rsync as the maximum of 700 bytes and the square root of the file length. The protocol also has mechanisms to cope with insertions and deletions, not just appending. This block-by-block approach reduces network traffic but increases processing load on both ends because of checksum calculation and block alignment. It works well when the number of files is modest, files are not massive, and changes are incremental. However, when the dataset contains a very large number of files, generating and sending checksums for each file becomes an intensive process. The sender must also work hard to check sequences and realign block boundaries, and that work is constant even if only one byte changed in one file. In a collection of a million files, Huston notes that this workload is a constant overhead, and it may sometimes be faster to forgo rsync checksums and perform a simple FTP mirror pass because file checks are far faster to compute.

A different approach treats the current state of a data collection as the result of editing transactions applied to an earlier state. This is best described by the DNS incremental zone transfer protocol, IXFR, defined in RFC 1995. When a DNS zone changes, the primary server updates its Start of Authority (SOA) serial number. A secondary server that wants to synchronize sends an IXFR request with its current SOA serial number. If the primary supports IXFR and holds a journal file of changes between that serial and the current one, it sends only the sequence of additions and deletions. This approach is also used in Near Real Time Mirroring (NRTM) defined in RFC 7682 for Route Registry synchronization and in the RPKI Repository Delta Protocol (RRDP) defined in RFC 8182 for the Resource Public Key Infrastructure. The server keeps a constant overhead by assigning version numbers and journaling changes between versions, but the advantage is that only the operations that transform one version to the next are transmitted. This can be applied to individual files or entire collections, so even very large file collections can be treated as one journalled artefact. In some scenarios it is inefficient: a client retrieving a current copy receives older changes that may be undone later in the sequence. A better understanding of the data can improve this. For example, BGP itself performs delta compression on routing updates. Each BGP peer is updated roughly every 30 seconds if Minimum Route Advertisement Interval (MRAI) timers are used. When a routing update is processed, what is queued is the prefix value, not the update itself. When the MRAI timer expires, the local BGP speaker dequeues each prefix and looks up its current state in the local Forwarding Information Base (FIB), then sends that state to the peer. If the local speaker keeps a record of what it has already sent to the peer, it sends no update when the state is unchanged. This kind of delta compression needs data items to have a unique primary key and a value; multi-valued keys cannot be readily supported.

Merkle trees take a hierarchical approach. Individual data items are sorted into a canonical sequence, each item is cryptographically hashed, hashes are paired in order and hashed again, repeating until a single root hash remains. If any data changes, the altered hash propagates to the root, so comparing two datasets can start by comparing just their root hashes. Differences can be found by descending the tree and ignoring branches whose superior hash values match. Merkle trees are used in the ZFS file system, Bitcoin, the Interplanetary File System and certificate transparency. Huston writes that the prerequisite is a canonical order of data items, and the synchronization function operates efficiently over both large and small datasets. It is currently proposed for the Resource Public Key Infrastructure (RPKI) and the ERIC protocol, where every client must maintain a synchronized local copy of the entire PKI. As RPKI use grows, the synchronization burden grows with the product of the number of clients and objects, but Merkle trees can break this growth by operating at a scale of the logarithm of the number of objects. A further improvement uses hash values a