Fixing Metasploit's AXFR support

DNS zone transfers (AXFR requests) are a great source of information about a network, allowing an "interested party" access to all the records on a DNS server. Robin Wood has offered a great resource to the community in the form of zonetransfer.me, a domain that allows zone transfers for testing purposes. Here's what it looks like:

me@here:~$ dig +short zonetransfer.me NS
ns12.zoneedit.com.
ns16.zoneedit.com.
me@here:~$ dig @ns12.zoneedit.com. zonetransfer.me AXFR

; <<>> DiG 9.8.1-P1 <<>> @ns12.zoneedit.com. zonetransfer.me AXFR
; (1 server found)
;; global options: +cmd
zonetransfer.me. 7200 IN SOA ns16.zoneedit.com. soacontact.zoneedit.com. 2012272996 2400 360 1209600 300
zonetransfer.me. 7200 IN NS ns16.zoneedit.com.
zonetransfer.me. 7200 IN NS ns12.zoneedit.com.
zonetransfer.me. 7200 IN A 217.147.180.162
zonetransfer.me. 7200 IN MX 0 ASPMX.L.GOOGLE.COM.
zonetransfer.me. 7200 IN MX 10 ALT1.ASPMX.L.GOOGLE.COM.
zonetransfer.me. 7200 IN MX 10 ALT2.ASPMX.L.GOOGLE.COM.
zonetransfer.me. 7200 IN MX 20 ASPMX2.GOOGLEMAIL.COM.
zonetransfer.me. 7200 IN MX 20 ASPMX3.GOOGLEMAIL.COM.
zonetransfer.me. 7200 IN MX 20 ASPMX4.GOOGLEMAIL.COM.
zonetransfer.me. 7200 IN MX 20 ASPMX5.GOOGLEMAIL.COM.
zonetransfer.me. 301 IN TXT "Remember to call or email Pippa on +44 123 4567890 or pippa@zonetransfer.me when making DNS changes"
zonetransfer.me. 301 IN TXT "google-site-verification=tyP28J7JAUHA9fw2sHXMgcCC0I6XBmmoVi04VlMewxA"
testing.zonetransfer.me. 301 IN CNAME www.zonetransfer.me.
164.180.147.217.in-addr.arpa.zonetransfer.me. 7200 IN PTR www.zonetransfer.me.
ipv6actnow.org.zonetransfer.me. 7200 IN AAAA 2001:67c:2e8:11::c100:1332
asfdbauthdns.zonetransfer.me. 7900 IN AFSDB 1 asfdbbox.zonetransfer.me.
office.zonetransfer.me. 7200 IN A 4.23.39.254
owa.zonetransfer.me. 7200 IN A 207.46.197.32
info.zonetransfer.me. 7200 IN TXT "ZoneTransfer.me service provided by Robin Wood - robin@digininja.org. See www.digininja.org/projects/zonetransferme.php for more information."
asfdbbox.zonetransfer.me. 7200 IN A 127.0.0.1
canberra_office.zonetransfer.me. 7200 IN A 202.14.81.230
asfdbvolume.zonetransfer.me. 7800 IN AFSDB 1 asfdbbox.zonetransfer.me.
email.zonetransfer.me. 2222 IN NAPTR 1 1 "" "E2U+email" "" email.zoneedit.com.zonetransfer.me.
dzc.zonetransfer.me. 7200 IN TXT "AbCdEfG"
rp.zonetransfer.me. 321 IN RP robin.zonetransfer.me.zonetransfer.me. robinwood.zonetransfer.me.
dr.zonetransfer.me. 300 IN LOC 53 20 56.558 N 1 38 33.526 W 0.00m 1m 10000m 10m
sip.zonetransfer.me. 3333 IN NAPTR 2 3 "au" "E2U+sip" "!^.*$!sip:customer-service@zonetransfer.me!" .
alltcpportsopen.firewall.test.zonetransfer.me. 301 IN A 127.0.0.1
www.zonetransfer.me. 7200 IN A 217.147.180.162
staging.zonetransfer.me. 7200 IN CNAME www.sydneyoperahouse.com.
deadbeef.zonetransfer.me. 7201 IN AAAA dead:beaf::
robinwood.zonetransfer.me. 302 IN TXT "Robin Wood"
vpn.zonetransfer.me. 4000 IN A 174.36.59.154
_sip._tcp.zonetransfer.me. 14000 IN SRV 0 0 5060 www.zonetransfer.me.
dc_office.zonetransfer.me. 7200 IN A 143.228.181.132
zonetransfer.me. 7200 IN SOA ns16.zoneedit.com. soacontact.zoneedit.com. 2012272996 2400 360 1209600 300
;; Query time: 61 msec
;; SERVER: 209.62.64.46#53(209.62.64.46)
;; WHEN: Wed Sep  5 17:12:42 2012
;; XFR size: 37 records (messages 37, bytes 2673)

I had previously used this service to update Nmap's dns-zone-transfer NSE script, so I thought similar updates would be a simple place to start with Metasploit development. Unfortunately (or fortunately!) I ran into a road block: Zone transfers were broken in Metasploit!

The module in question was auxiliary/gather/enum_dns, which has lots of other good DNS enumeration functions besides AXFR. The underlying problem, though was a problem with the Net::DNS library that Metasploit uses. First, a little about how AXFR transactions work:

DNS usually works over UDP, fitting an entire response into a single packet. For zone transfers, though, responses can be much larger. Since zone transfers are used for synchronizing DNS servers, ensuring that no data is lost is critical. For this reason, AXFR transactions are conducted over TCP.

Most DNS transactions involve a single request with a Questions section being answered by a single response with an Answers section (among other sections). AXFR is different; since it has to transfer an entire zone, the server responds with a series of DNS responses, each of which contains some number of Answers. The responses begin with the Start Of Authority (SOA) record for the zone, and end with the same record.

Now we come to the broken part. Net::DNS (and Metasploit by extension) was smart enough to switch to TCP when an AXFR request was passed, but it only read the first response. Modifying the method to loop until 2 SOA messages had been seen was a fun exercise in Ruby that I may write about another time. In the meantime, you could check out the commit diff. Here's how it looks now:

msf > use auxiliary/gather/enum_dns
msf  auxiliary(enum_dns) > show options

Module options (auxiliary/gather/enum_dns):

   Name         Current Setting     Required  Description
   ----         ---------------     --------  -----------
   DOMAIN                           yes       The target domain name
   ENUM_AXFR    true                yes       Initiate a zone transfer against each NS record
   ENUM_BRT     false               yes       Brute force subdomains and hostnames via the supplied wordlist
   ENUM_IP6     false               yes       Brute force hosts with IPv6 AAAA records
   ENUM_RVL     false               yes       Reverse lookup a range of IP addresses
   ENUM_SRV     true                yes       Enumerate the most common SRV records
   ENUM_STD     true                yes       Enumerate standard record types (A,MX,NS,TXT and SOA)
   ENUM_TLD     false               yes       Perform a TLD expansion by replacing the TLD with the IANA TLD list
   IPRANGE                          no        The target address range or CIDR identifier
   NS                               no        Specify the nameserver to use for queries (default is system DNS)
   STOP_WLDCRD  false               yes       Stops bruteforce enumeration if wildcard resolution is detected
   WORDLIST     /redacted/file.txt  no        Wordlist for domain name bruteforcing

msf  auxiliary(enum_dns) > set ENUM_SRV false
ENUM_SRV => false
msf  auxiliary(enum_dns) > set ENUM_STD false
ENUM_STD => false
msf  auxiliary(enum_dns) > set DOMAIN zonetransfer.me
DOMAIN => zonetransfer.me  
msf  auxiliary(enum_dns) > run

[*] Setting DNS Server to zonetransfer.me NS: 69.64.68.41
[*] Performing zone transfer against all nameservers in zonetransfer.me
[*] Testing nameserver: ns16.zoneedit.com.
AXFR query, switching to TCP
Error parsing axfr response: uninitialized constant Net::DNS::RR::AFSDB
Error parsing axfr response: uninitialized constant Net::DNS::RR::AFSDB
Error parsing axfr response: uninitialized constant Net::DNS::RR::NAPTR
Error parsing axfr response: uninitialized constant Net::DNS::RR::RP
Error parsing axfr response: uninitialized constant Net::DNS::RR::LOC
Error parsing axfr response: uninitialized constant Net::DNS::RR::NAPTR
[*] Zone transfer successful
[*] Name: ns16.zoneedit.com. Record: SOA
[*] Name: ns16.zoneedit.com. Record: NS
[*] Name: ns12.zoneedit.com. Record: NS
[*] Name: zonetransfer.me. IP address: 217.147.180.162 Record: A
[*] Name: ASPMX.L.GOOGLE.COM. Preference: 0 Record: MX
[*] Name: ALT1.ASPMX.L.GOOGLE.COM. Preference: 10 Record: MX
[*] Name: ALT2.ASPMX.L.GOOGLE.COM. Preference: 10 Record: MX
[*] Name: ASPMX2.GOOGLEMAIL.COM. Preference: 20 Record: MX
[*] Name: ASPMX3.GOOGLEMAIL.COM. Preference: 20 Record: MX
[*] Name: ASPMX4.GOOGLEMAIL.COM. Preference: 20 Record: MX
[*] Name: ASPMX5.GOOGLEMAIL.COM. Preference: 20 Record: MX
[*] Text: zonetransfer.me.        301     IN      TXT
[*] Text: zonetransfer.me.        301     IN      TXT
[*] Name: www.zonetransfer.me. Record: CNAME
[*] IPv6 Address: 2001:67c:2e8:11::c100:1332 Record: AAAA
[*] Name: office.zonetransfer.me. IP address: 4.23.39.254 Record: A
[*] Name: owa.zonetransfer.me. IP address: 207.46.197.32 Record: A
[*] Text: info.zonetransfer.me.   7200    IN      TXT
[*] Name: asfdbbox.zonetransfer.me. IP address: 127.0.0.1 Record: A
[*] Name: canberra_office.zonetransfer.me. IP address: 202.14.81.230 Record: A
[*] Text: dzc.zonetransfer.me.    7200    IN      TXT
[*] Name: alltcpportsopen.firewall.test.zonetransfer.me. IP address: 127.0.0.1 Record: A
[*] Name: www.zonetransfer.me. IP address: 217.147.180.162 Record: A
[*] Name: www.sydneyoperahouse.com. Record: CNAME
[*] IPv6 Address: dead:beaf:: Record: AAAA
[*] Text: robinwood.zonetransfer.me.   302   IN   TXT
[*] Name: vpn.zonetransfer.me. IP address: 174.36.59.154 Record: A
[*] Host: www.zonetransfer.me. Port: 5060 Priority: 0 Record: SRV
[*] Name: dc_office.zonetransfer.me. IP address: 143.228.181.132 Record: A
[*] Testing nameserver: ns12.zoneedit.com.
AXFR query, switching to TCP
Error parsing axfr response: uninitialized constant Net::DNS::RR::AFSDB
Error parsing axfr response: uninitialized constant Net::DNS::RR::AFSDB
Error parsing axfr response: uninitialized constant Net::DNS::RR::NAPTR
Error parsing axfr response: uninitialized constant Net::DNS::RR::RP
Error parsing axfr response: uninitialized constant Net::DNS::RR::LOC
Error parsing axfr response: uninitialized constant Net::DNS::RR::NAPTR
[*] Zone transfer successful
[*] Name: ns16.zoneedit.com. Record: SOA
[*] Name: ns16.zoneedit.com. Record: NS
[*] Name: ns12.zoneedit.com. Record: NS
[*] Name: zonetransfer.me. IP address: 217.147.180.162 Record: A
[*] Name: ASPMX.L.GOOGLE.COM. Preference: 0 Record: MX
[*] Name: ALT1.ASPMX.L.GOOGLE.COM. Preference: 10 Record: MX
[*] Name: ALT2.ASPMX.L.GOOGLE.COM. Preference: 10 Record: MX
[*] Name: ASPMX2.GOOGLEMAIL.COM. Preference: 20 Record: MX
[*] Name: ASPMX3.GOOGLEMAIL.COM. Preference: 20 Record: MX
[*] Name: ASPMX4.GOOGLEMAIL.COM. Preference: 20 Record: MX
[*] Name: ASPMX5.GOOGLEMAIL.COM. Preference: 20 Record: MX
[*] Text: zonetransfer.me.        301     IN      TXT
[*] Text: zonetransfer.me.        301     IN      TXT
[*] Name: www.zonetransfer.me. Record: CNAME
[*] IPv6 Address: 2001:67c:2e8:11::c100:1332 Record: AAAA
[*] Name: office.zonetransfer.me. IP address: 4.23.39.254 Record: A
[*] Name: owa.zonetransfer.me. IP address: 207.46.197.32 Record: A
[*] Text: info.zonetransfer.me.   7200    IN      TXT
[*] Name: asfdbbox.zonetransfer.me. IP address: 127.0.0.1 Record: A
[*] Name: canberra_office.zonetransfer.me. IP address: 202.14.81.230 Record: A
[*] Text: dzc.zonetransfer.me.    7200    IN      TXT
[*] Name: alltcpportsopen.firewall.test.zonetransfer.me. IP address: 127.0.0.1 Record: A
[*] Name: www.zonetransfer.me. IP address: 217.147.180.162 Record: A
[*] Name: www.sydneyoperahouse.com. Record: CNAME
[*] IPv6 Address: dead:beaf:: Record: AAAA
[*] Text: robinwood.zonetransfer.me.   302   IN   TXT
[*] Name: vpn.zonetransfer.me. IP address: 174.36.59.154 Record: A
[*] Host: www.zonetransfer.me. Port: 5060 Priority: 0 Record: SRV
[*] Name: dc_office.zonetransfer.me. IP address: 143.228.181.132 Record: A
[*] Auxiliary module execution completed
msf  auxiliary(enum_dns) >

See all the parsing errors? That's what I intended to fix when I started. Well, no time like the present!

XML output for Nmap's NSE scripts

The Nmap Scripting Engine (NSE) is a great feature of Nmap that turns it into a data-collecting, banner-grabbing, vulnerability-scanning, exploit-finding platform. It gathers information like SSL certs, enumerates users, brute-forces authentication, exploits back-doors, spiders websites, and more. But though Nmap's port scan, version detection, and OS fingerprinting output all have their own elements in Nmap's XML output, NSE script output has been stored in a blob of text, formatted in any arbitrary fashion, and shoved into the output attribute of the <script> element.

For the last year, I've been working on various prototypes of structured output for NSE scripts. The long and sordid history is mostly traced out on this SecWiki page, but after many rewrites, forks, and the closest thing I've seen to a flame war within the Nmap family, the functionality was finally merged into the trunk in r29570!

The basic idea is that NSE scripts can now return a data structure in addition to or instead of the usual string output, and that data structure will be recursively converted into XML. What follows is my attempt at pulling together into one place the way the system works and how it is supposed to be used.

What it does

The NSE engine converts Lua data structures into XML using a recursive algorithm. Lua really only has one data structure, the table, which can be used to represent arrays/lists and associative arrays/hashes/hashmaps. The only difference is whether the indices are integers (array) or any other value (hash). In keeping with this simplicity, the XML output consists of only 2 new elements: <table> and <elem>, each of which can have a key attribute. The string version of the output is still kept in the output attribute of the <script> element. A good explanation of this is given in the NSE API section of the Nmap documentation.

How to use it

There are basically 4 ways to return output under the new system:

1. Plain string output (ye Olde Way)

The new system is completely backwards compatible, so your old scripts will work just fine. The string output will continue to be put into the output attribute of the script tag, but no new XML will be generated.

 2. Automatically-stringified output

As your script executes, build a table of output elements, then return that. NSE will automatically create a string output with indentation, "key: value" pairs, and newline-separated lists. This is the easiest way to get XML output, but it can take up some extra screen real estate, since each element of output is displayed on its own line in the Normal output.

3. Structured AND string output

Your script can return 2 values: a table to be converted into XML, and a string to use for Normal output. It's best if these contain the same information, but the XML must never contain less information than the string output. This is a handy method for wordy output containing explanations of what's been discovered; this information isn't needed in machine-readable XML.

4. Self-stringifying output

Your script can return a table that contains its own instructions for turning it into a string. This is done by overriding the __tostring metamethod for the table or any of the nested tables within it. If this sounds confusing, don't worry. I'll be adding a formatting library that hides all this Lua meta-magic behind easy-to-use function calls. The underlying functionality will be the same, though. This method is useful for scripts that want to take advantage of the outline-flavored stringification of option (2) for much of their output, but have special formatting needs for subsections, like a file listing (tabular output) or a horizontal, comma-separated list. This also helps ensure that the XML and string outputs both contain the same information.

Tips for script authors

Changing the way things work can be confusing. Here are some tips to help you write scripts that work well with XML script output:

• Use stdnse.output_table() to create a table that remembers the order you assigned keys. Regular Lua tables order their keys in an unpredictable way.
• Break down your data into the smallest reasonable chunks. If your script returns a username/domain combination, return it as {username="joe", domain="EXAMPLECOM"} instead if simply {username="EXAMPLECOM\joe"}.
• Make sure your script returns nil when you don't want output. Returning false used to work, but now it will show up as "false".
• Use the automatic stringification (option 2 above) for testing. Choose one of the other options once you know what output you want, and only if the default doesn't look right.

Disable NFSv2 on Debian/Ubuntu

These instructions apply to the nfs-kernel-server package on Ubuntu 10.04 LTS and 12.04 LTS, and probably apply to various Debian versions as well.

The kernel NFS server is called rpc.nfsd, so its options are documented in the rpc.nfsd(8) man page. To turn off specific versions of NFS, pass the -N or --no-nfs-version argument to this daemon, as well as the rpc.mountd daemon. The accepted config file to do this is /etc/default/nfs-kernel-server.

For Ubuntu 12.04, there is a rather obviously-named variable, RPCNFSDARGS, that can be set to pass arguments to rpc.nfsd. Here's one for turning off NFSv2:

RPCNFSDARGS="-N 2"

On Ubuntu 10.04, there is no such variable, and creating it does nothing, since the init script (/etc/init.d/nfs-kernel-server) does not reference it. Instead, the RPCNFSDCOUNT variable can be used, since it is passed to the rpc.nfsd as well. Here's how to do the same thing on 10.04:

RPCNFSDCOUNT="-N 2 8"

Remember to keep the "8" (or whatever you choose to set it to).

For both versions, setting the RPCMOUNTDOPTS variable the same way is recommended, so that clients don't mount the wrong version of your exports:

RPCMOUNTDOPTS="-N 2 --manage-gids"

Do it faster Makes us stronger

What

Improvements to Nmap's ssl-enum-ciphers NSE script.

Who

Me! Original script by Mak Kolybabi and Gabriel Lawrence. Workaround for Microsoft servers discovered by Martyn Tovey.

Why

Secure Sockets Layer and Transport Layer Security (hereinafter called SSL) are protocols for using various hashing and encryption algorithms to provide a secure, mutually authenticated channel for communication between computer systems. By design, the particular algorithms used are pluggable; the client and the server decide between themselves which ones to use for any particular communication. But not all algorithms are created equal, and some are deliberately neutered for testing purposes, or to avoid export restrictions on cryptography.

If a client and a server both support some weak encryption algorithm, they may still never use it if they both favor a strong one they also share. Unfortunately, the decision of which algorithm to choose is neither encrypted nor authenticated, so an attacker in a man-in-the-middle scenario can perform a downgrade attack, reducing security to the least-strong combination of algorithms, to even include unauthenticated or do-nothing encryption. Server admins, security teams, and penetration testers need a way to determine which servers on a network may be vulnerable to this type of attack.

Enter Nmap. Since version 5.30BETA1 (2010-03-29), Nmap has included a script for enumerating the algorithms for each of the 4 modern versions of SSL (SSLv3, TLSv1.0, TLSv1.1, and TLSv1.2). The mechanism this script uses (also used by sslscan and other tools) is simple: for each of the 4 SSL versions, try each of the 213 cipher suites (combinations of hashing algorithm, encryption algorithm, key exchange algorithm, and cipher mode) and report back which ones succeeded. That's 4*213=852 exchanges between client and server, no matter how many SSL versions or cipher suites each uses. A run against 30 HTTPS servers on the Internet took me 9.5 minutes. That's slow, especially considering each bunch of 213 cipher suites is tested in parallel. I was sure there had to be a better way.

How

In the SSL protocol, the first message sent is from the client, the Client Hello message. It contains a list of cipher suites supported, as well as other information. The Server Hello message sent in reply contains one cipher suite (usually the strongest) that the server shares with the client. The protocol spec allows up to 2^16-1 cipher suites to be sent in the one Client Hello message.

Removing the multithreading code temporarily, I changed the script to follow this algorithm:

1. Send a Client Hello with all the cipher suites we wish to enumerate. 
2. When the server responds with one, end the connection. 
3. Remove and record the chosen suite, and send a new Client Hello with the remaining cipher suites. 
4. Repeat until the server sends a handshake failure message, indicating that it supports none of the remaining suites.

This worked great! Instead of 213 exchanges, only 1 plus the number of valid cipher suites were performed. Usually this number is between 3 and 16 (most commonly 6, 7, or 8). Moving the threading code to run each protocol in parallel, scan times reduced by two-thirds or more. I cleaned up the code and sent it in to the development mailing list for comment.

The idea is not new: the original author of the script, Mak Kolybabi, implemented a very similar idea before the final version was added, but scrapped it due to inconsistent results. Unfortunately, comments from the mailing list revealed that my version of the script was faring no better. Specifically, Windows systems running IIS tended to show only 1 or 2 ciphers supported, when previously they had shown 6 or more. Examining the traffic, I found that the server was terminating the underlying TCP connection with a RST, with no handshake failure to show what might be wrong.

At this point, Martyn sent me a message with the key to the problem: contrary to the protocol spec, some servers only look at the first 64 cipher suites, rejecting or ignoring any higher number. Rushing back to the script, I quickly broke up the scan into 64-suite chunks. Success! This added an extra 3 exchanges per valid protocol, but that's a small price to pay for the overall speedup.

When

The new version of the script was committed to Nmap's Subversion repository in r29206. It should be compatible with the latest release (version 6.01, available here), so you can download it directly without recompiling Nmap. Bonus in this update: 147 new cipher suites to check for! With the new script it still comes out faster than the old method. Happy hunting!

My format: Guardian

I come from the Net - through systems, peoples, and cities - to this place: MAINFRAME. My format: Guardian. To mend and defend - to defend my new found friends, their hopes and dreams, and to defend them from their enemies. They say The User lives outside the Net and inputs games for pleasure. No one knows for sure, but I intend to find out. ReBoot!

Hacking, Hollywood style

Dear Hollywood,

The following images and videos do not depict hacking, or even a plausibly realistic use of computers. I laughed, I cried, I wished I was watching something else.

"One keyboard four hands" is for piano, not computers.

"I'll spout a bunch of fancy words I don't understand, see if I can replicate a capability we should already have."

Edit: video removed due to copyright. "The Crack In The Code" episode of "Bones" featured a "fractal code" etched into a bone that when scanned caused the analyst's computer to burst into flames. There's so much wrong with this, I have to limit myself to just one: Computers don't catch fire from viruses. Period.

Watching a film or TV show shouldn't require me to ignore everything I know about computers.

 (Image credit Penny Arcade)

In closing,

(Image credit Saturday Morning Breakfast Cereal)

The Seven Circles of Linux Purgatory

For the purification of the geek-spirit:

1. panic: no init found. 
2. Obtaining network card drivers
3. fatal error: linux/sys.h: No such file or directory
4. vim
5. Xorg.conf
6. Compiling video card drivers
7. This site is best viewed using current versions of Netscape Navigator or Microsoft Internet Explorer at a screen resolution of 800 x 600 or higher.

A list of things to remember when debugging a C program

These go for C++ as well. Assuming gcc/g++ and gdb.

• Compile with -Wall and pay attention to warnings.
• Turn off compiler optimization. Remove -O from your CFLAGS.
• Turn on debugging. Add -g to your CFLAGS.
• Set appropriate breakpoints.
• There is nothing wrong with throwing in a printf statement to narrow down your search. Use fprintf to print to stderr.
• gdb --args ./program --program-args
• break filename:lineno
• Use valgrind to eliminate memory leaks and clean things up. If you can't trigger it while debugging, it's probably a memory bug. (valgrind --leak-check=full -v --log-file=vg.out ./program --program-args)
• Define the error condition, then determine how it got that way.
• When you find the bug, look for similar bugs in all your other code.