Extracting email attachements from pcap files

From time to time, I need to examine email attachements that may have been delivered to my users, usually to see exactly what type of malware was included. Now, I could call them up and ask them if they got the message, but there are several reasons I don't like to do that. For example, I hate playing telephone tag, and I don't really want to encourage them to open the message for any reason.

Since I use Sguil, I have another option. I can extract the attachement from the network data directly. Here's how.

1. Capture the session of interest as a pcap file. If you're also using Sguil, you can probably just do a SANCP search to find the network session that contained the message as it was delivered to your SMTP server. Open it up in Ethereal, which causes the Sguil client to copy the pcap file to your analysis workstation. Close Ethereal now, because you won't be using it for this.
   
2. Once you have the pcap file, use the tcpflow tool to extract an ASCII version of the conversation. This will create two files, one for the server side of the session, and one for the client side. Here's an example, where xxx.xxx.xxx.xxx is the sending host, and zzz.zzz.zzz.zzz is your mail server:
   

   
   % tcpflow -r xxx.xxx.xxx.xxx_yyyy_zzz.zzz.zzz.zzz_25-6.raw
   % ls
   xxx.xxx.xxx.xxx_yyyy_zzz.zzz.zzz.zzz_25-6.raw
   xxx.xxx.xxx.xxx.yyyy-zzz.zzz.zzz.zzz.00025
   zzz.zzz.zzz.zzz.0025-xxx.xxx.xxx.xxx.yyyy
   

   
   The file you need to concern yourself with now is the one that ends in .00025. That's the one that contains the email message sent by the client.
   
3. Edit the *.00025 file in your favorite text editor. As it is now, it's a complete record of all the SMTP protocol events, and what you really want is just the data. The easiest thing is just to delete everything up to (but not including) the first line that starts with "Received:".
   
4. Use the following perl command to read the mail file in via stdin and dump out the decoded MIME message. The script uses the MIME::Parser module (available from CPAN) to handle MIME decoding.
   

   
   %  perl -e 'use MIME::Parser; \
   $parser = new MIME::Parser; \
   $parser->output_under("/var/tmp"); \
   $entity = $parser->parse(\*STDIN); \
   $entity->dump_skeleton;' < *25
   
   Content-type: multipart/mixed
   Effective-type: multipart/mixed
   Body-file: NONE
   Subject: Message could not be delivered
   Num-parts: 2
   --
       Content-type: text/plain
       Effective-type: text/plain
       Body-file: /var/tmp/msg-1150990039-2841-0/msg-2841-1.txt
       --
       Content-type: application/octet-stream
       Effective-type: application/octet-stream
       Body-file: /var/tmp/msg-1150990039-2841-0/letter.zip
       Recommended-filename: letter.zip
       --
   

   
   As you can see, this command creates a directory /var/tmp/msg-XXXXXXX-XXXX-0/ which contains files for each piece of the MIME multipart message, including the attachement (in this case, /var/tmp/msg-1150990039-2841-0/letter.zip).
   

Now that you've got the attachement, you can use your favorite reverse engineering tools on it to figure out exactly what you've got on your hands.

Laptop encryption? I have a better idea.

I just read an AP article from last week, entitled Encryption can save data in laptop lapses. We hear about this problem in the news all the time now: someone loses a laptop with personally identifiable information about thousands of {customers|taxpayers|veterans|others}.

This is a real problem, make no mistake, but articles like this one are starting to piss me off. Instead of talking about encryption, let me propose a much more revolutionary idea: don't put the freakin' data on the laptop in the first place!

Laptops are designed to be mobile, and are easily lost, stolen or broken. Good encryption practices are rare (hint: it's not just a matter of software, but also of implementation, policy and user acceptance). Instead of worrying about who's laptop is going to be stolen next, let's try to address the real issue: does that data really need to be stored on the laptop in the first place?

If mobile users need access to data in the field, make them VPN back to the corporate network and work on it there. And don't tell me things like, "But we need to work on the plane" because you don't. You may want to work at 26,000 feet (it's better than watching Firewall on the tiny screen), but my privacy rights outweigh your productivity crisis anytime.

Let me end this rant by saying that if your company insists on placing your customers' personal information on laptops or other mobile devices, encrypted or not, I hope they hammer you in court.

Update 2006-06-09: Finally, some mainstream coverage for this apparently radical idea! In a story entitled Why Do Laptops Schlep Such Data?, the unnamed AP author raises the question about whether carrying such data around on mobile devices is appropriate. Short answer: "No, but some people will inevitably find a way to do it anyway." No doubt this is true, and that's why routine encryption of mobile data remains important. Just don't be confused about the proper use of mobile encryption: It's not the first line of defense, it's the last. It ideally only comes into play once someone has violated corporate policy by copying data onto the device.

Unfortunately, the article does miss (badly!) on an important point. The second-to-last paragraph describes software that "shreds" data on the hard drive if the laptop is stolen. The idea behind this type of so-called protection is absurd. The laptop's OS and security software has to be running in order for this to be effective, but any competent computer thief can swap out a hard drive and copy the data with only a few moments' work.

Tracking your most active network protocols with Sguil

All you Sguil users out there might find this interesting. I wanted to figure out a way to track the top most active protocols flowing over my perimeter, where "active" in this case means "most bytes transferred either in or out of my network". This is actually a fairly simple database query, but I had a special requirement.

You see, I wanted to know not only the total number of bytes transferred, but I wanted to see it broken down by whether the traffic was entering my network or leaving my network. SANCP (Sguil's session collector) tracks bytes by source and destination, but that's not good enough. The source of one connection could be an Internet host, while the source of another connection could be a system on my own LAN. Just because SANCP records how many bytes the source host sent doesn't mean it knows whether they were entering or leaving my LAN.

Here's a SQL query that really can tell the difference:

select dst_port, 
       sum(to_mynet) as bytes_to_mynet, 
       sum(from_mynet) as bytes_from_mynet,
       sum(to_mynet + from_mynet) as total_bytes
       from
       (
          select dst_port, 
          src_bytes as from_mynet,
   dst_bytes as to_mynet
   from sancp where
   (start_time between DATE_SUB(CURDATE(), INTERVAL 1 HOUR) 
                and CURDATE())
   and (src_ip between INET_ATON("192.168.0.0") and
        INET_ATON("192.168.255.255")) and
   not (dst_ip between INET_ATON("192.168.0.0") and
        INET_ATON("192.168.255.255"))
          UNION ALL
          select dst_port, 
   dst_bytes as from_mynet,
          src_bytes as to_mynet
   from sancp where
   (start_time between DATE_SUB(CURDATE(), INTERVAL 1 HOUR) 
                and CURDATE())
   and not (src_ip between INET_ATON("192.168.0.0") and
        INET_ATON("192.168.255.255")) and
       (dst_ip between INET_ATON("192.168.0.0") and
        INET_ATON("192.168.255.255"))
       ) as test_table
       group by dst_port
       order by total_bytes desc
       limit 5;

The trick here is to do two queries, each limiting itself to either sources on the Internet or sources on my LAN. I can then use the "SELECT" statements to order the columns to reflect the direction of the data flow. In this query, the second column will always be the number of bytes flowing out of my network, and the third column will always be the number of bytes coming into my network. Of course, the first column is the destination port number, which corresponds roughly to the network protocol used.

These two queries are joined by a "UNION ALL" statement to make them into one large result set. The SQL code above treats this as a temporary table, from which the outer SELECT statement grabs it's data and does the final work of summing both incoming and outgoing data flows and creating a neat table.

The output of this query will look something like the following table. Note that I ran it against some very odd sample data, so it doesn't show a full 10 rows. Also, I messed with the numbers because I was uneasy about posting my real data here. Still, you can see what the report looks like.

+----------+---------------+-----------------+-------------+
| dst_port | bytes_to_mynet| bytes_from_mynet| total_bytes |
+----------+---------------+-----------------+-------------+
|       80 | 1816          | 51101804400     | 51101806216 |
|       22 | 1816          | 51101412328     | 51101414144 |
|      443 | 1816          | 51096413496     | 51096415312 |
|       53 | 1804400       | 18093887        | 18274327    |
+----------+---------------+-----------------+-------------+

To use this in your own network, simply change the IP addresses above to represent the beginning and ending of your own address space. As written, the query will track only a single hour of data (from 23:00 to 00:00 of the current day), but you can play with the time specification to get other reporting periods.

HTTP PUT Defacement Attempts

I've seen the number of website defacement attempts on my network rise by about 3 orders of magnitude since yesterday, as evidenced by this report:

mysql> select date(timestamp) as dt, count(*) from event
where timestamp > "2006-05-30" and signature = "HTTP PUT
Defacement Attempt" group by dt order by dt ASC;
+------------+----------+
| dt         | count(*) |
+------------+----------+
| 2006-05-30 |        3 |
| 2006-05-31 |     2006 |
| 2006-06-01 |     1301 |
+------------+----------+
3 rows in set (0.55 sec)

The count for 2006-05-30 is pretty typical (an average day sees less than 10 defacement attempts). Other analysts I've talked with don't seem to be noticing anything unusual. Is it just me?

The reason for this, it turns out, is that I've forgotten to contribute the Snort rule I wrote to detect these attacks. So here it is, in case you're interested in tracking these yourself:

alert tcp $EXTERNAL_NET any -> $HTTP_SERVERS $HTTP_PORTS 
(msg:"HTTP PUT Defacement Attempt"; 
flow:to_server,established; content:"PUT "; depth:4; 
classtype:web-application-attack; sid:1000003; rev:1;)

The attack itself is quite simple. The attackers simple use a built-in HTTP method designed to allow an authorized user to upload a file directly into the web space. It's not even really an exploit, since they're using the PUT method the way it was designed to be used. They're just counting on a misconfiguration that allows anyone to do it.

Here's an example:

PUT /index.html HTTP/1.0
Accept-Language: en-us;q=0.5
Translate: f
Content-Length:153
User-Agent: Microsoft Data Access Internet Publishing Provider DAV 1.1
Host: education.jlab.org

spykids spykids spykids spykids spykids spykids spykids
spykids spykids spykids spykids spykids spykids spykids 
spykids spykids spykids spykids spykid\n\n

As you can see, they're trying to replace the site's default page ("/index.html") with a page consisting entirely of their own text ("spykids" repeated several times).

There seem to be a constant low-level of these attacks on the Internet, and if your servers are configured correctly, you have nothing to worry about. Still, if you're like me, you still want to track these defacement attempts. Using my rule, now you can. I'll be submitting it to the Sourcefire Community Ruleset, so hopefully it'll show up there soon.

Update 2006-06-01 14:37: I'm not the only one who has noticed this anymore. Apparently this group has been defacing a lot of sites lately. Fortunately, I didn't notice it in the quite the way this guy did. I guess they've decided to crank up the defacement machine recently.