NSD 4.14 cuts DNS server memory use with new RDATA storage and optional compile-time switches

DNS servers translate human friendly domain names into the numeric addresses computers use, and authoritative name servers hold the official records for a domain. NLnet Labs, the Dutch non profit behind the NSD authoritative server software, released NSD 4.14.0 on 4 December with a major change: resource record data (RDATA) is no longer stored as a list of pointers to individual fields, but as an opaque binary blob for most record types. This lazy approach saves memory and processing time because authoritative servers rarely need to inspect the individual fields inside a record unless they are writing out a zone file. The result is a smaller memory footprint for zones, especially those with many records that have a large number of fields.

Before version 4.14.0, NSD kept each field of a resource record separately in memory, along with pointers to those fields and wasted space from aligning data on 64 bit boundaries. The new approach stores most record types as a single opaque binary blob. Domain name fields that may be compressed in responses are still stored by reference so compression remains efficient. Record types with many fields benefit most, such as NSEC3 and RRSIG, which both have eight RDATA fields. The .nl zone is a prime example: it has more than 6 million delegations, more than 62 percent of which are secure (DNSSEC signed), and it uses NSEC3 opt-in, meaning there is an NSEC3 record for every name. NLnet Labs has a long standing agreement with SIDN, the registry for .nl, to use the zone for research and software improvement. With the new storage, NSD needs 14.9 percent less memory for the .nl zone.

Two compile time options can reduce memory further. The --enable-packed option drops the requirement that structure members be aligned on a typical 4 byte boundary, eliminating gaps between members. Not all CPU architectures support unaligned memory access, and it can carry a performance penalty on some, so it is disabled by default. In practice, the NLnet Labs team has only observed performance improvements, likely because related data sits closer together in the CPU cache. The --disable-radix-tree option switches from the default radix tree data structure to a red black tree, which uses less memory but at the cost of performance; for very large zones this may still be practical. Combined with the new RDATA storage, the packed option yields an additional 16.2 percent reduction on top of its own 14.2 percent saving for .nl. For the .com zone, which has relatively few secure delegations (4.2 percent) and uses NSEC3 opt-out, the new RDATA approach alone gives only 2.4 percent reduction, but combining all memory reducing options brings the memory needed to serve .com comfortably below the 64 gigabyte mark.

Table 1 in the original post lists reductions across six zones: .lol 18.3 percent, .se 17.7 percent, .nl 14.9 percent, .net 2.6 percent, .org 2.6 percent, .com 2.4 percent. With --enable-packed, reductions rise to 33.4 percent for .lol, 28.1 percent for .se, 30.4 percent for .nl, 19.6 percent for .net, 19.4 percent for .org, and 19.3 percent for .com. With --disable-radix-tree alone, the numbers are 41.4, 31.8, 46.5, 32.9, 33.1, and 31.3 percent respectively. Combining both options gives 51.0 percent for .lol, 39.5 percent for .se, 55.2 percent for .nl, 44.0 percent for .net, 43.9 percent for .org, and 42.6 percent for .com. The zones have different properties: .com is a large NSEC3 opt-out signed zone with 4.2 percent secure delegations, .net and .org are similar with around 5 percent secure delegations, .nl uses NSEC3 opt-in with 62.2 percent secure delegations, .se is somewhat larger NSEC signed with 61.3 percent secure delegations, and .lol is a small NSEC signed zone with 9.0 percent secure delegations.

After the 4.14.0 release, a user reported that memory consumption actually increased in their setup. The affected zones had a relatively small number of RRsets but a very large number of resource records per set, often thousands. NSD would grow those RRsets by adding each record individually, allocating new space and deallocating old space into a recycle bin that never reused the odd sizes. The fix in NSD 4.14.1 grows RRsets in batches rather than one record at a time. The reported memory usage with the fix is almost one sixth of what it was before 4.14.0 for that type of zone. This is important for operators who serve zones with many records in a single RRset, such as large DNS based blocklists or mail servers with many MX records of the same preference.

Loading and writing zones also became faster. Writing or printing zones is faster even without extra options, and with --enable-packed it is about 18 percent faster for .nl, 6.7 percent faster for .net and .org, and 25 percent faster for .se and .lol. Zone load times improve further when combined: 21.7 percent for .lol, 19.6 percent for .se, 12.4 percent for .nl, 7.9 percent for .com, 3.9 percent for .org, and 4.7 percent for .net. These improvements matter because DNS infrastructure powers every website visit and email, and efficient servers reduce operational costs and latency. For website owners and IT teams who run their own DNS or rely on hosting providers, this kind of memory efficiency at authoritative servers keeps the internet's address book fast and dependable; at the same time, using a privacy first encrypted DNS service like AEU DNS for your own queries adds a layer of protection against tracking and manipulation.