Performance Effects of DNSSEC Validation - July 2022
On July 30, 2022, Petr Špaček spoke at the DNS-OARC38 conference about the performance effects of DNSSEC validation in BIND 9.Read post
ISC is delighted to announce the release of Kea 1.6.3.
Kea 1.6.3 is is our current stable version. We normally avoid making significant changes in a stable version, but in this release we have backported three very useful features from our development branch. There is no need to update to this version unless you are interested in one of these new features. The new features in this version are:
status-getcall implemented, enabling monitoring with Stork
Kea 1.6.3 now supports a
status-get command that returns a detailed current status, giving insight into the current operation of any Kea servers. In particular, this command is heavily used by Stork, ISC’s new dashboard system. With the addition of that command, it is possible to use Kea 1.6.3 with Stork.
For a quick introduction to what Stork offers, look at the screen shot library. Stork is open source and can be downloaded from the ISC website where you download Kea, or there are packages available on Cloudsmith. We are currently doing monthly development releases of Stork, so the functionality is evolving rapidly.
During the process of assigning a dynamic address, Kea will cycle through all the addresses in the selected pool in order, checking each one to see if another client has already reserved it, and then checking to see if another client has already leased it. When it finds an address that is neither reserved nor already assigned, it can proceed to allocate it to the new client. With this change, the order of these two checks is reversed.
check address for HR
check address for existing lease
check address for existing lease
check address for HR
Prior to this change, every possible address was checked for an existing host reservation. Checking for host reservations is an expensive operation, particularly if host reservations are stored in a separate database. In a pool where most of the addresses are already assigned, it is more efficient to first check for an existing lease, and then only check for a host reservation if there is no existing lease.
The purpose of this change was to improve efficiency. In our internal testing at ISC, we did confirm that with a pool with high utilization this change lowered the CPU utilization and shortened the time to identify an available address for the client. This should also improve performance in cases where lease allocation is significantly faster than reservations, such as when using memfile for leases and a SQL database for reservations. This change was implemented for both DHCPv4 and DHCPv6. (Details on Gitlab #1124)
It is possible to specify per-client client classes within a host reservation. These classes, however, were previously evaluated only after an address had been selected for the client, so classification could only influence which DHCP options were given to the client. However, we found some use cases where it was helpful to assign client classes on a global host reservation and use those to influence subnet selection within a shared network and pool selection within a subnet.
Now, if there are client classes found in the global reservations or there are client classes specified within the non-global reservations and the subnet belongs to a shared network, we remove already evaluated classes (except for built-in classes and those set by the hooks libraries) and evaluate them again taking into account those that are found in the reservations.
To read an excellent issue report from user Mike Kazantsev see Gitlab #1324. Thank you Mike!
Client classes specified within host reservations can now be used to influence subnet selection within a shared network and pool selection within a subnet. (Details on Gitlab #1139)
Packages: We have produced and published on Cloudsmith packages for Kea 1.6.3 for Ubuntu 18.04, Debian 9, CentOS 7. Fedora 30 is now obsolete, so we will not be publishing a package for Fedora 30. There are newer versions of Ubuntu, Debian, Fedora and CentOS, that were not available when we started the Kea 1.6 branch. We have not created packages for these, although if we get significant user demand for them we could. We have made a number of improvements in our packaging scripts for the 1.7 branch, fixing issues with paths and permissions. We don’t plan to backport these changes to 1.6 because we expect this is the last 1.6 release we will be making, since 1.8 is only a month or two away from release. So, the upshot is, if you have been using the 1.6.x packages previously, this should be a very simple update. If you have not started using Kea packages, you might be better off building this from source as well, and adopting the Kea packages with a 1.7 or later version, because the package scripts will change after 1.6.
Kea 1.7.10 is our latest monthly development version. We are hoping this will be the last release on this development branch before we renumber it and produce our next stable version, Kea 1.8.0. If you are curious about the status of Kea 1.8.0, you can view the milestone and all the remaining work planned for version 1.8 in our Kea Gitlab project. Those users who are closely following the progress of our work on multithreading Kea might be interested in our latest performance test report, which now shows performance improvements in every scenario.
As with any other development release, use this with caution. Development releases are not recommended for production use.
The most significant changes introduced in this version are:
New Packages for Kea Migration Assistant (Keama). Keama is a tool that assists migration from ISC DHCP to Kea by using an ISC DHCP config file to generate a JSON config file in a format accepted by Kea. The configuration syntax and some core architectural designs are different, so the conversion does not cover 100% of cases, but at least it gives users a running start. Previously, to get the tool, a manual compilation was necessary, but now it is available as an RPM or DEB package (#1081). Packages are available in ISC’s Cloudsmith repository.
Removed obsolete configuration of top-level logging and other daemons. Very early Kea versions assumed that configuration for all daemons (DHCPv4, DHCPv6, DDNS, CA) were to be stored in the same file, sharing a top-level logging entry. This was convenient in the early days, but it was fundamentally incompatible when commands such as
config-write were introduced, as one daemon would overwrite other daemons' configurations. This was deprecated a long time ago, and since 1.6.0, Kea produced warnings when an old configuration was detected. This backward compatibility is now removed and Kea now prints an error if old configurations are detected (#916).
Improved performance of subnet selection. One of the crucial steps of packet processing in Kea is subnet selection, as Kea needs to determine which subnet to use to allocate leases. Prior to this change, Kea walked through subnets in the order configured, which was fine for a smaller number of subnets, but for large deployments that counted subnets in thousands this was inefficient. The data structure is now a multi-index container, which should help with large deployments. The side effect is that subnets are now evaluated in the order of their subnet-id. This generally should not affect any users, except in cases where people relied on the earlier evaluation order (as specified in the config file) (#553, #1265). This feature was also included in the Kea 1.6.3 version.
Multithreading tuning. Two new sections have been added in the ARM regarding DHCPv4 and DHCPv6 performance tuning for multithreading. We also added a link to our performance report that is updated weekly (#1285). Our Developer’s Guide has been updated as well (#996, #1264).
Better recovery after failed config. Kea’s configuration process is complex and can fail in various ways. Depending on the type of failure (e.g. syntax error, removal of a dynamic network interface, database connection failure, etc.), Kea may not be able to recover to its working state. Several improvements have been made: Kea now handles the interface misconfiguration better (#1330); when recovery is not possible, Kea clearly reports a critical message that it is unable to provide service; and message documentation now has a better explanation of the available steps needed to recover the service (#1166, #507).
DNS support for lease4-del and lease6-del commands. Kea previously provided
lease6-del commands. However, when removing leases, there was no way to instruct Kea to clean up the corresponding DNS entries. It is now possible (#544).
Cross-compilation for Raspberry Pi. We are working on a solution to be able to efficiently run Kea on Pis. This is a long process, but we’re getting there, step by step. In this release we improved the makefiles and added a section in the Developer’s Guide regarding cross-compilation (#1221). We also updated the parameters to the
./configure script to better support cross-compilation (#1223).
Better handling for a large number of interfaces. Believe it or not, there are users who need more than 64k interfaces in their system. Kea previously failed on interfaces with ifindex greater than 65535. It does not anymore (#1302).
LFC process now runs with lower priority. One user was concerned with high CPU utilization when Lease File Cleanup was running. The LFC process is now being started with lower priority, meaning the OS will deprioritize it. The amount of work needed has not changed, but the load should be spread over a longer period of time, making it less noticeable (#1311).
Negative statistics and getNextRow failed in MySQL The underlying problem causing some statistics to become negative has been fixed in the 1.7.9 release. However, there was no good way to fix the already-broken statistic after migration to 1.7.9. The
kea-admin tool now provides the
stats-recount command to correct the statistics (#1196).
Moved to standard chrono library in C++11. Previously Kea used the POSIX time library from boost. The code now uses the standard chrono library from C++11. In addition to the benefit of using a standard library, there’s a small related performance improvement as the standard lib does fewer conversions (#1174).
Assorted bugfixes and improvements GNU make-specific features were removed, which should improve compatibility with FreeBSD and possibly other BSD clones (#1248). We fixed a bug in leasequery that could cause some unit-tests to fail (#1344), and fixed a compilation problem with the musl library that affected Apline 3.10 builds (#1337). We fixed a test problem in the stats library that caused the tests to fail if run in a specific order (#1293). The config backend has better time granularity; previously, it could have missed configuration updates if multiple updates were done during the same second (#1247). And we improved the hook libraries unloading routine that previously could have caused a crash on shutdown in some rare cases (#1282, #1279).
Code of conduct. Kea now has a code of conduct adapted from the Django project. Please see the
code_of_conduct.md file or use this link.
Our internal testing is aware of only one outstanding issue, somewhat related to MT and HA: in some specific circumstances (notably, when Kea has received a packet and is still processing it before sending a response while the application is being shut down), the application may fail during shutdown. The underlying problem is related to improper hooks library de-initialization and has existed for a long time; it is not strictly related to MT. The issue is very rarely reproducible in our tests. However, due to the parallel nature of MT, the likelihood of triggering the problem, especially under heavy load, is greater. Details are in ticket #1279. This problem is expected to be fixed before the next stable (1.8.0) version is released.
The main source code repository for Stork is in ISC’s GitLab: https://gitlab.isc.org/isc-projects/kea.
The Kea source code tarballs are available on the ISC downloads page as well as on the ISC FTP server. ISC is also providing RPM and deb binary packages for Stork. They are available in our package repositories on Cloudsmith.
The Kea Administrator Reference Manual is posted on Readthedocs.io.
Kea has monthly development releases and approximately annually, we release a new stable branch. The stable version is then updated as needed, when we have significant maintenance updates.
Kea is released under the Mozilla Public License, version 2.0: https://www.mozilla.org/en-US/MPL/2.0
The easiest way to install the software is to use native deb or RPM packages. They can be downloaded from:
We support development of Kea, Stork and our other software through the sale of annual support subscriptions. Contact our sales team discuss your BIND and Kea support requirements. Contact us for more information.
What's New from ISC