Recently on an application security review project I ended up having analyze a lot of before- and after- statuses when installing several pieces of software on Linux operating systems. It made me realize that I really missed having something akin to Microsoft's Attack Surface Analyzer but then for Linux. Obviously there are a ton of monitoring tools out there already which log security information and do events management.
These tools are more commonly classified as SIEM software. A classic and obvious example of such SIEM software is Tripwire. And with the addition of modern VC-attention-grabbing-buzz-words such as machine learning and Big Data there are a ton of SIEM alternatives to choose from nowadays. Tripwire and its ilk and the promises they make can easily be brought up in conversation if you're inclined to win this week's edition of the bullshit bingo. Maybe an idea for when you're attending your daily agile-development-stand-up meeting. Get your post-its ready kids! But that's not really what I was looking for. Those tools definitely have their use cases but they're overkill for my purposes.
Yours truly wanted something nimble and simple. The tool should be easily installable on most modern Linux machines as well as being easily extensible with extra features. As such I relatively quickly settled on writing it in Python (3.x to be exact, it's 2017 after all) based on just standard Python libraries without any external dependencies (*cough*). Also no complicated daemon setups or installation procedures should be necessary. Simply dropping a tarball with some Python scripts on a machine should be sufficient to get the tool up and running. That way any automation can be easily achieved via CI/devops-like tools such as puppet or ansible if so desired.
Besides that I wanted to be able to easily extend it. So I made the decision to simply rely on shell commands. Please note that it's pretty much always sane advice to NOT rely on parsing output of command line tools such as GNU core-utils' ls. I opted to do it in this case anyway! This for the simple reason that I want a tool I can extend really quickly when doing projects. Reliability of results can be guaranteed for individual projects and quickly monitored and tested for anyway. There's an obvious difference in needs for reliability between having a tool which needs to work across hundreds of different Linux/UNIX-like machines and always yield reliable results without breaking down.
Some of the common pitfalls for example include the parsing of filenames when dealing with command line output (see here and here). However don't let this dissuade anyone from the fact that in realistic scenario's one totally gets away with parsing this output. Simply put there's already a ton of software out there which relies on the results of, for example, GNU core-utils' ls. As such there was a huge uproar when the developers of that command line utility changed the default output of ls to quote filenames as can be seen at stackexchange.
Having said all that the tool being introduced here has only been thoroughly tested on Ubuntu and Debian machines. There's no Operating System detection taking place just yet nor any adjustment of GNU specific parameters towards their BSD equivalents (e.g. ls --color=yes versus ls -G respectively). It should take a minimal amount of work to get it up and running on other systems though.
Without further ado. Here's some sample output of dawgmon -- dawg the hallway monitor. One first needs to run the tool to establish a baseline. This baseline is then used to compare the new results against. A sample run in which there's already a baseline and in which we open a port and then close it again might look as follows:
# ./dawgmon -gfA 0 changes detected (0 warnings) # nc -l -p 4455 & [1] 12489 # ./dawgmon -gfA 1 change detected (0 warnings) + port 4455 tcp opened # fg nc -l -p 4455 ^C # ./dawgmon -gfA 1 change detected (0 warnings) + port 4455 tcp closed #
As we can see in the output above the tool correctly detects that the port was closed. It does NOT do this by hooking into netlink libraries; it simply parses the output of netstat and compares it with the previously cached results.
In this example the baseline has already been cached and on the first run of dawgmon we discover zero changes. Then Virtual Box is being installed and another run of dawgmon yields a pretty useful list of detected attack surface.
# ./dawgmon -gfA 0 changes detected (0 warnings) # apt install virtualbox-5.1 [...] # ./dawgmon -gfA 33 changes detected (0 warnings) + size of file /etc/group changed from 937 to 954 + file /etc/group got modified on 2017-09-14 19:29:51.804811 +0200 + size of file /etc/group- changed from 934 to 937 + file /etc/group- got modified on 2017-09-14 19:29:14.000000 +0200 + file /etc/gshadow got modified on 2017-09-14 19:29:51.812811 +0200 + size of file /etc/gshadow- changed from 777 to 794 + size of file /etc/mailcap changed from 40777 to 41063 + file /etc/mailcap got modified on 2017-09-14 19:29:51.632812 +0200 + file /etc/systemd/system/multi-user.target.wants/vboxautostart-service.service got created (owner=root, group=root, perm=lrwxrwxrwx, size=49) + file /etc/systemd/system/multi-user.target.wants/vboxballoonctrl-service.service got created (owner=root, group=root, perm=lrwxrwxrwx, size=51) + file /etc/systemd/system/multi-user.target.wants/vboxdrv.service got created (owner=root, group=root, perm=lrwxrwxrwx, size=35) + file /etc/systemd/system/multi-user.target.wants/vboxweb-service.service got created (owner=root, group=root, perm=lrwxrwxrwx, size=43) + file /etc/udev/rules.d/60-vboxdrv.rules got created (owner=root, group=root, perm=-rw-r--r--, size=747) + group vboxusers added + package virtualbox-5.1 is to be installed + suid binary /usr/lib/virtualbox/VBoxHeadless got created (owner=root, group=root, perm=-r-s--x--x, size=158304) + suid binary /usr/lib/virtualbox/VBoxNetAdpCtl got created (owner=root, group=root, perm=-r-s--x--x, size=23144) + suid binary /usr/lib/virtualbox/VBoxNetDHCP got created (owner=root, group=root, perm=-r-s--x--x, size=158304) + suid binary /usr/lib/virtualbox/VBoxNetNAT got created (owner=root, group=root, perm=-r-s--x--x, size=158304) + suid binary /usr/lib/virtualbox/VBoxSDL got created (owner=root, group=root, perm=-r-s--x--x, size=158296) + suid binary /usr/lib/virtualbox/VBoxVolInfo got created (owner=root, group=root, perm=-r-s--x--x, size=10472) + suid binary /usr/lib/virtualbox/VirtualBox got created (owner=root, group=root, perm=-r-s--x--x, size=158304) + i-node for listening UNIX socket /run/systemd/private changed from 3428734 to 3452848 + systemd property NInstalledJobs changed from 8392199 to 3238035463 + systemd property NNames changed from 261 to 263 + systemd unit file vboxautostart-service.service added + systemd unit file vboxballoonctrl-service.service added + systemd unit file vboxdrv.service added + systemd unit file vboxweb-service.service added + systemd unit 'vboxautostart-service.service' added + systemd unit 'vboxballoonctrl-service.service' added + systemd unit 'vboxdrv.service' added + systemd unit 'vboxweb-service.service' added
Please note that this does not mean that there's anything wrong with any of these things per se; just that if I were so inclined to start examining Virtual Box for local privilege escalation bugs I know where to start (the suid root binaries seem like a good place to do so -- an implementation bug there might give one local root access).
The tool has already proved itself very useful for me in my own engagements. Besides that the moderate amount of infrastructure being used by Anvil Ventures is now being monitored too. Instead of having a fancy, complicated and expensive SIEM setup for just a handful of machines dawgmon is simply being run from a crontab with the results being emailed to me once every day. It gives me a bit of an indication what's going on and what changed on the machines as well as warnings such as that the machines need to be rebooted or that inadvertently some configuration files changed or whatever. A screen shot of a part of my inbox to illustrate:
The list of different analysis parts dawgmon implements right now is pretty diverse. It monitors SystemD changes, System V IPC, UNIX sockets, TCP/UDP ports, processes and changes to /etc and /boot and more. For caveats check the README included in the source code. Current output of the available commands in the current 1.0 release of dawgmon:
$ ./dawgmon -E NAME DESCRIPTION check_boot analyzes /boot directory check_etc analyzes /etc directory check_groups analyze UNIX group changes check_users analyze UNIX user changes env monitor changes in environment variables kernel_version analyze changes in kernel version list_blkdev analyze changes in available block devices list_ifaces analyze changes in network interfaces list_mount analyze changes in file system mounts list_msq analyze changes in System V message queues list_packages analyze changes in installed Debian packages list_pipes lists named pipes list_processes monitors changes in the running processes (mostly for debugging) list_sem analyze changes in System V sempahores list_shm analyze changes in System V shared memory segments list_suids lists setuid/setgid executables list_sysvinit_jobs analyze changes in available System V init jobs list_tcpudp_ports list changes in listening TCP/UDP ports for both IPv4/IPv6 list_unix_ports list changes in listening UNIX ports lsb_version analyze changes in Linux Standard Base release settings needs_restart checks whether a reboot is required (Ubuntu-only) systemd_props show all systemd properties systemd_sockets list systemd sockets systemd_timers list systemd timers systemd_unitfiles list all available systemd unit files systemd_units list all available systemd units uptime show uptime and check if reboot happened
I'm eager to hear suggestions for other commands and modules to add so definitely do not hesitate to reach out or simply send me a pull request for patches via github or a patch via email.
The code can be found under the Anvil Ventures github page in the dawgmon repository. Patches, comments, suggestions and criticism is more than welcome. Just contact me on github, twitter or email me directly.