Monday, 16 March 2015

Pcap2XML/Sqlite - Convert 802.11 Packets to XML and SQLITE

This tool converts 802.11 packet traces (PCAP format) into an XML and SQLITE equivalent so you can now run XPATH/XQUERY/SQL queries on the packets.

Why do we need this?

Wireshark is great when it comes to capturing and filtering packet traces. However, it has no facility for macro level tasks. Here are some answers which Wireshark cannot give you out of the box:

  • Give me all device MAC addresses in the PCAP
  • Give me a unique list of all Access Point/Ad-Hoc networks in the PCAP
  • ... 
Of course, this is by design. Wireshark is a packet capture tool and not a data analysis platform. 

This is where Pcap2XMl/Sqlite comes in! We map every header field in an 802.11 packet to an XML and SQLITE Equivalent. Once we convert every packet into these formats, it is extremely easy to run analysis tools on them as you shall see in latter part of this post.

Where can this tool be used?

This tool can be used anywhere there is a need to analyze individual packets. However, we had the following purposes in mind:

  • Teaching Wi-Fi security using Packet Analysis
  • Deriving Macro-Stats on a PCAP file as discussed in the previous section
  • Writing a simple Wi-Fi IDS :) 
  • ...

How does the tool work?

Below is an example of the tool running on a 50MB 802.11 packet trace and creating an XML and SQLITE file:

Examples of Analysis

As you can see in the previous section, the tool has created sample.db and sample.xml.

Let us look at sample.db using Sqlite Browser:

Table containing all the packets:

Simple filter for Beacon Frames:

Macro Stats - Get All Distinct Device MAC Addresses in the PCAP

Macro Stats - Select All Distinct MACs sending Beacon Frames

Macro Stats - Select All Distinct MACs receiving Data Frames

Macro Stats - Get the Average Frame Length

Macro Stats - Get the inter-packet time delta! 

... there are tons of such examples possible as you can see. 

Can you outline some of the other interesting features?

  • Ability to select only a packet range or selected packets
  • Ability to monitor a PCAP file live for new packets
  • Automatic update checker for newer versions
  • ...
We highly recommend you check out the video tutorials we are making on this tool. 

How are the 802.11 headers mapped to XML and SQLITE?

In version 1.0, we are only mapping the 802.11 MAC Header for all frames (Management, Data and Control).

Here is the XML Schema:

Here is the SQLITE DB Schema:

Future versions will have all frames mapped apart from the Data frame payloads.

I need an output in other formats e.g. CSV, JSON, MySQL etc.


We chose XML and SQLITE because one could easily write simple tools in Python or other languages to convert this to other formats. At this time, we have no plans to support any other format.

What is the feature roadmap?

  • Parse and map all Management Packets (April 2015)
  • Parse and map all Control Packets (May 2015)
  • Parse and map some fields from Data Packet (May 2015)
  • Integrate with Graphs and Charting tools (June 2015)
  • Allow reverse conversion from XML, SQLITE to PCAP (July 2015) 
  • Integrate and test with PCAP to Air tools for Packet replay  (July 2015) 
  • ...

Is the tool open source? 

The tool will always be free to use but currently the source code is not public. We might change this later once we feel the tool is stable enough for collaborating. The tool is digitally signed with our certificate for authenticity.

Where can I Download this? are there any Tutorials/Documentation?

No comments:

Post a Comment