DNS Benchmarking in Python (take 2)

A while ago I wrote a script to benchmark DNS servers by flooding them with queries and record the time to get an answer. That can be found here.

I've rewritten my DNS benchmarking script to compile better and easier to use results. The method of obtaining the results remains the same. Below is version 2.

You still need a list of domain names, 1 name per line and nothing else. List of domains can be obtained from Quantcast - 1 million most popular sites and then filtering out the extra data.

You can use something like this to filter out the rank.

bash# cat top-million.txt | gawk '{ print $2 }'

In this version the data is collected by doing multiple runs of DNS queries for the top n number of domains as it was before and this data is then saved in a csv file for each server. At the end, the data is re-read and then processed into meaningful numbers.

In this example I have done 1000 lookups 10 times against each server. That's 10,000 lookups per server and 100,000 lookups in total. I have included the output below the code.

#!/usr/bin/env python
#
# Requires python-adns
# http://linux-101.org
#
 
# List of DNS Servers
servers = ("8.8.8.8","8.8.4.4","4.2.2.1","4.2.2.2","4.2.2.3","4.2.2.4","4.2.2.5","4.2.2.6","208.67.222.222","208.67.220.220")
 
# Number of times to loop through
runs = 10
 
# Number of queries to perform on each run against each server. Must be an even number for ease of calculating the Median value in the resultant dataset.
queries = 1000
 
# Print a message every X number of queries
printCount = 100
 
 
import adns
import sys
import time
 
totalStart = time.time()
urlFile = open("top-million-sites.txt", "r")
urlList = urlFile.readlines()
 
run = 0
while run < runs:
    for server in servers:
        s = adns.init(adns.iflags.noautosys,sys.stderr, "nameserver " + server)
        if run == 0:
            resultsDataFile = open("DNS-Results-" + server + ".csv", "w")
        else:
            resultsDataFile = open("DNS-Results-" + server + ".csv", "a")
        count = 0
        quickestTime = 999999.9
        longestTime = 1.0
        averageTime = 999999.9
        totalTimeMS = 999999.9
        print "Server: " + server + ""
        print "Run: " + str(run + 1) + ""
        while count < queries:
            timeStart = time.time()
            resolve = s.synchronous(urlList[count].strip(), adns.rr.A)
            # For asynchronis queries use below line
            #resolve = s.submit(urlList[count].strip(), adns.rr.A)
            timeEnd = time.time()
            timeTaken = timeEnd - timeStart
            # Multiply by 1000 to get milliseconds from seconds. (Not sure windows has this kind of Granularity)
            timeTakenMS = timeTaken * 1000
            resultsDataFile.write(str(timeTakenMS) + "\n")
            count = count + 1
            if count % printCount == 0:
                print "Complete DNS Queries: " + str(count) + " "
        print "\n"
    run = run + 1
    resultsDataFile.close()
    print "\nRun " + str(run) + " complete\n"    
urlFile.close()
 
# At this point all data has been collected, now we just do the maths.
print "\n\nData recorded!\n"
print "\nProcessing... \n\n"
 
resultsFile = open("DNS-Results.txt", "w")
for server in servers:
    serverResultsFile = open("DNS-Results-" + server + ".csv", "r")
    serverResultsList = serverResultsFile.readlines()
    serverResultsList.sort()
    totalQueries = queries * runs
    totalQueryTime = 0
    for i in serverResultsList:
        totalQueryTime += float(i)
    medianPosition = totalQueries / 2
    median = serverResultsList[medianPosition].strip()
    meanQueryTime = totalQueryTime / totalQueries
    minQueryTime = serverResultsList[0].strip()
    maxQueryTime = serverResultsList[-1].strip()
    print ("Server: \t" + server)
    print ("Total Queries: \t" + str(totalQueries))
    print ("Total Time: \t" + str(totalQueryTime) + " ms")
    print ("Min Time: \t" + str(minQueryTime) + " ms")
    print ("Max Time: \t" + str(maxQueryTime) + " ms")
    print ("Mean Time: \t" + str(minQueryTime) + " ms (Average Time)")
    print ("Median Time: \t" + str(median) + " ms (Middle Value) \n\n")
    resultsFile.write("Server: \t" + server + "\n")
    resultsFile.write("Total Queries: \t" + str(totalQueries) + "\n")
    resultsFile.write("Total Time: \t" + str(totalQueryTime) + " ms" + "\n")
    resultsFile.write("Min Time: \t" + str(minQueryTime) + " ms" + "\n")
    resultsFile.write("Max Time: \t" + str(maxQueryTime) + " ms" + "\n")
    resultsFile.write("Mean Time: \t" + str(minQueryTime) + " ms (Average Time)" + "\n")
    resultsFile.write("Median Time: \t" + str(median) + " ms (Middle Value)" + "\n\n")
resultsFile.close()
 
totalEnd = time.time()
total = totalEnd - totalStart
print "Total time taken = " + str(total) + " seconds \n"

An example of the results produced.

Server: 	8.8.8.8
Total Queries: 	10000
Total Time: 	467052.149773 ms
Min Time: 	10.0200176239 ms
Max Time: 	99.995136261 ms
Mean Time: 	10.0200176239 ms (Average Time)
Median Time: 	65.3960704803 ms (Middle Value)
 
Server: 	8.8.4.4
Total Queries: 	10000
Total Time: 	170615.334511 ms
Min Time: 	10.0030899048 ms
Max Time: 	98.9630222321 ms
Mean Time: 	10.0030899048 ms (Average Time)
Median Time: 	7.47108459473 ms (Middle Value)
 
Server: 	4.2.2.1
Total Queries: 	10000
Total Time: 	4252006.11329 ms
Min Time: 	100.030899048 ms
Max Time: 	999.998092651 ms
Mean Time: 	100.030899048 ms (Average Time)
Median Time: 	302.79302597 ms (Middle Value)
 
Server: 	4.2.2.2
Total Queries: 	10000
Total Time: 	1666077.56567 ms
Min Time: 	10.0269317627 ms
Max Time: 	994.40908432 ms
Mean Time: 	10.0269317627 ms (Average Time)
Median Time: 	268.837928772 ms (Middle Value)
 
Server: 	4.2.2.3
Total Queries: 	10000
Total Time: 	952574.544668 ms
Min Time: 	100.006103516 ms
Max Time: 	995.326042175 ms
Mean Time: 	100.006103516 ms (Average Time)
Median Time: 	16.606092453 ms (Middle Value)
 
Server: 	4.2.2.4
Total Queries: 	10000
Total Time: 	800388.546705 ms
Min Time: 	10.0409984589 ms
Max Time: 	991.203784943 ms
Mean Time: 	10.0409984589 ms (Average Time)
Median Time: 	36.7720127106 ms (Middle Value)
 
Server: 	4.2.2.5
Total Queries: 	10000
Total Time: 	732073.317051 ms
Min Time: 	100.06403923 ms
Max Time: 	99.9100208282 ms
Mean Time: 	100.06403923 ms (Average Time)
Median Time: 	16.1869525909 ms (Middle Value)
 
Server: 	4.2.2.6
Total Queries: 	10000
Total Time: 	669226.772547 ms
Min Time: 	10.0059509277 ms
Max Time: 	99.9758243561 ms
Mean Time: 	10.0059509277 ms (Average Time)
Median Time: 	5.704164505 ms (Middle Value)
 
Server: 	208.67.222.222
Total Queries: 	10000
Total Time: 	710338.041544 ms
Min Time: 	100.03900528 ms
Max Time: 	99.9889373779 ms
Mean Time: 	100.03900528 ms (Average Time)
Median Time: 	12.7530097961 ms (Middle Value)
 
Server: 	208.67.220.220
Total Queries: 	10000
Total Time: 	706110.551596 ms
Min Time: 	100.018024445 ms
Max Time: 	99.9960899353 ms
Mean Time: 	100.018024445 ms (Average Time)
Median Time: 	12.7520561218 ms (Middle Value)
Burtronix Banner W3C Banner