Meet an ISC Engineer – Michal Nowak!
It’s time once again to get to know one of our engineers!Read post
We have been monitoring BIND 9 performance using the ISC Performance Lab, described in this 2016 blog post. We realized we really needed something like this when users reported to us a significant performance drop when upgrading from version 9.9.5 to either 9.9.6 or 9.10.0. We now review performance changes every month and when we see a negative change, we can identify the commit that caused the change and investigate.
These measurements are designed to identify changes in performance from one release to the next, not to measure typical or maximum performance in actual production use. Although we show the absolute number of queries per second on the charts below, we have not made any effort to tune the configuration for our hardware, and these results are not necessarily predictive of performance in another configuration or on other hardware. Many features can have a significant performance impact (e.g. query logging), and our tests don’t measure other important factors, such as the speed of adding and removing zones.
First, identify which of these ‘example applications’ is most like your deployment:
Then, find the result for the release you are running today, and compare with the release you are considering migrating to.
The data points below each represent the average of 3 test runs, with each test run consisting of 30 measurements. Changes of ~20KQPS in either direction are not significant given our test variability and sample size.
We introduced a change in 9.9.6 and 9.10.0 that was intended to improve performance, that caused a huge drop in KQPS. The drop in KQPS was due to reducing the number of UDP listeners to NUM_PROCESSORS / 2. We made this change because of a UDP locking bug in older versions of RedHat EPL, that led us to believe that this would improve performance.
3751. [tuning] The default setting for the -U option (setting the number of UDP listeners per interface) has been adjusted to improve performance. [RT #35417]
The change DID improve performance in several tests on our hardware, but the only 8-core machine we had was 5 years past EOL at the time, and in retrospect, not a good basis for making a performance adjustment.
After several other users reported a performance regression on their hardware, and a vigorous internal debate, we made a second change in 9.9.9 and 9.10.4 which brought performance back to the earlier level.
4236. [func] On machines with 4 or more processors (CPU), the default value for the number of UDP listeners has been changed to the number of detected processors minus one. [RT #40761]
We advise testing on your own hardware to get the optimal setting in this Knowledgebase article on ‘Choosing the right value for -U when starting named.’
Another, smaller drop in performance from 9.10 to 9.11 can be seen by comparing the 9.11 and 9.10 charts. While addressing a bug reported by a user, we introduced another performance regression in 9.11. BIND was incorrectly preserving case-sensitivity from the zone_name in a zone statement over what is defined in the zone itself. We(1) have made subsequent changes that mitigated the performance impact of case preservation, but some performance penalty is inherent in careful preservation of the case in the authority. (For those who are interested, the fix is one of the changes in commit# 4605, currently committed to the 9.12 branch, which we will be backporting to 9.11.3.)
Using our Performance Lab, we have been able to validate a number of changes which will come out in our next major branch, BIND 9.12, that will improve performance significantly. We will provide an update on that when we release the BIND 9.12.0 alpha.
(1) Both of the performance regressions were diagnosed and fixed by BIND core developer Mukund Sivaraman, who has had a special focus on BIND performance improvement.
What's New from ISC