Adam Sweet's Wiki

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
===== Create Your Own Anti-Virus Signatures with ClamAV =====

I use [[http://www.clamav.net/|ClamAV]] on my own mail servers, I've also used it at work alongside several commercial AV engines and every now and again there will be a viral attachment that none of the AV engines catch, especially when a new threat is released. As a Linux user, most virus and malware threats mean little to me, however if you are responsible for Windows users then you need to be on top of the game.

Even though viral email attachments aren't the major attack vector for Windows PCs that they were a few years ago, a few times recently I've found the need to block viral emails which the major AV engines weren't catching or they were sufficiently behind the curve that I've had to create my own signatures to block viral attachments while I waited for the AV vendors to catch up.

Enter ClamAV. ClamAV is an anti-virus toolkit for [[http://www.clamav.net/|Unix]] and [[http://www.clamwin.com/|Windows]]. Aside from being an on-demand virus scanner, ClamAV comes with a suite of tools for creating your own anti-virus signatures which can then be used as part of the regular AV definitions when running a scan.

The first thing you need is something which you want to detect. It might be a virus, some other piece of malware or maybe just a nuisance application installer. It helps if you're not running Windows so you don't infect yourself with whatever it is you are trying to detect and running the following commands will be easy for you. If you have an email with your attachment or file in, you need to save the attachment to your PC. If it's still on the mail server, either download the mail and save the file or if you have shell access to the server, copy the entire mail file itself to your PC which is easy if you're using maildirs. If you use mboxes you need to take a copy of the mail somehow so it's in a file of it's own (look at csplit for example).

If you have a file containing the email rather than having saved the attachment from within your mail client, you need to split the text and attachment parts out from each other. The following script does this for you. You need Perl and the MIME::Parser module from CPAN (sudo cpan install MIME::Parser for Ubuntu users).

  #!/usr/bin/perl
  use MIME::Parser;
  $file = $ARGV[0];
  my $parser = new MIME::Parser;
  mkdir("/tmp/$$");
  $parser-&gt;output_under("/tmp/$$/");
  $parser-&gt;output_prefix("msg");
  $entity = $parser-&gt;parse_open("$file");
  $entity-&gt;dump_skeleton;

Save it as strip-attach.pl or something and make it executable. Then run it with an argument of the file to strip such as:

  strip-attach.pl testfile

The output will give you the paths to the text portion and the attachment portion of the email. If you saved the email attachment to your PC from your mail client, you can start to pay attention now.

What you now have is the file you want to block. If it's zipped, compressed or in any other kind of container then unzip it or extract it as ClamAV can see inside these archives if you configured it to do so and you have the right tools installed (like unzip under Linux for example).

Next create a signature of the file using ClamAV's sigtool:

  cat testfile | sigtool --hex-dump | head -c 2048 > customsig.ndb

In this case, testfile is your undesirable file and we have taken a signature of the first 2KB, otherwise the signature would be huge and therefore scanning would be inefficient. We have saved the generated signature in customsig.ndb. In theory, you need to take a signature of a unique portion of the file. You can also take a signature from an off-set within the file, it doesn't have to be from the start of the file. See the ClamAV [[http://www.clamav.net/doc/latest/signatures.pdf|signature docs]] for more detail on how to create signatures.

You should edit customsig.ndb and prefix the content with the appropriate Name, Type and Offset in the following format:

Name:Type:Offset:malware hex output

Such as:

Trojan.Win32.Emold.A:1:*:4d5a80000100000004001000ffff000040010000000000004000000000000000000000000000000000000000

Name is the virus name. Type is one of the following:
  * 0 = any file
  * 1 = Portable Executable (ie Windows exe)
  * 2 = OLE2 component (e.g. a VBA script)
  * 3 = HTML (normalised)
  * 4 = Mail file
  * 5 = Graphics
  * 6 = ELF
  * 7 = ASCII text file (normalised)

Offset is either * or an offset in bytes from the beginning of the file to where the hex string occurs. This is best left as * unless you know your where in the file your hex string occurs. Read the Clamav documentation if this is the case.

For most purposes, a type of 0 (or 1 for a Windows exe), and an offset of * will suffice.

Either name the virus yourself if it's just a file you don't want on your network or it's a new virus, or take a look at what other AV engines call a virus by submitting your suspicious file to somewhere like [[http://www.virustotal.com/]]. ClamAV has it's own virus naming conventions as detailed in the docs.

My good friend and malware expert [[http://barbie.missbarbell.co.uk/|Barbie]] until recently of Message Labs and [[http://birmingham.pm.org/|Birmingham Perl Mongers]] gave a talk at [[http://www.lugradio.org/|LugRadio]] [[http://lugradio.org/live/UK2008/|Live UK 2008]] where he explained that the people that are first to identify a new virus are the people who name it, though different AV vendors often use the different names and the name which is popularised in the press is the one that sticks. If you detect a virus before anybody else, then name it as you like and then find a way of making sure everybody uses your chosen name. Fun and profit awaits you :-)

Now, test the signature against your suspect file:

  clamscan -d customsig.ndb testfile

It's pretty inefficient to store one virus signature per file, so if you're going to be doing this frequently or you want your signature to used as part of regular operations, you may as well start keeping your own virus db file as part of ClamAV itself. Simply copy your customsig.ndb to the directory used by ClamAV's own signatures. On most Linux boxes that's /var/lib/clamav/, though it might be something like /usr/local/share/clamav/ on FreeBSD or if you compiled ClamAV yourself. So restart ClamAV and run a regular scan without having to specify your custom sig:

  clamscan testfile

And that's it. Add each new signature line into the customsig.ndb file you put in ClamAV's signatures directory but be sure to test it first from a standalone sig file so you know it works as expected without affecting the operation of the main ClamAV installation.

Having created sigs for files which the commercial AV engines weren't catching, I submitted the suspicious file I was working on to the ClamAV team for detection within ClamAV. Now I guess you have to be a bit closer to the project and certainly more experienced than the novice I am to generate sigs and have them included in ClamAV, but there's nothing stopping you submitting the suspicious files to the project by uploading them at [[http://www.clamav.org/sendvirus/]].

I did exactly that and was quite pleased to get an email a few weeks later which said a signature for the file I submitted had been included in a ClamAV update, although the same file had been submitted by several other people.

Most people suggest advocacy or documentation as ways non-programmers can help a project, it just goes to show that there are many more ways to help a Free Software project than you might think if you're not a programmer.

So, why would you want to use ClamAV? If you run mail servers then you should be using it already, regardless of whether you run a proprietary AV engine. ClamAV is free and plugs easily into most Unix style mail servers, either directly or though something like Amavis. ClamAV is pretty good at catching phishing emails too, which is something I've not seen much of from the major AV vendors. Details on dealing with phishing sigs are [[http://www.clamav.net/doc/latest/phishsigs_howto.pdf|here]].

A few years ago I worked at a college where Windows permissions were sufficiently lax that the students were able to install MSN Messenger (now known as Windows Live Messenger) on the PCs which were supposed to be for educational purposes only, as certain applications they needed to run required access to write to parts of the registry so they couldn't be locked down any further without serious effort. We had a terrible time trying to keep up with removing it and stopping them downloading it. Had we known at the time, (ignoring the concept of actually trying to lock the machines down properly), we could have run ClamAV on a filtering proxy and created a signature which detected MSN Messenger or other unwanted installers, blocked them at the gate and run a scan across the user directories for saved copies brought in on memory sticks. While it's fighting fires instead of solving the bigger problem, you could apply a simple fix to the major threats and it would buy you enough breathing space to solve the real problems.

Note that ClamAV is not an in memory, on-access, real-time background virus scanner, it won't detect viruses in files as you open or execute them. You need to manually scan files to detect viruses, it's not intended as a replacement for a desktop AV, it's intended for gateway services like web and mail filtering or scheduled scanning.

Do I need to tell you any more? Go geddit tiger.
Adam Sweet's Wiki

User Tools

Site Tools

Page Tools
