In my ongoing desire to determine who/what/when/where/why my server had previously been compromised, I continue my analysis with the messages files and some other /var/log files.
messages.1
- This is the oldest messages file.
Aug. 10 - Aug. 17
- 3 reboots
- all manually intiated
- can see some logs indicating various USB devices being plugged in/unplugged
- otherwise, nothing obvious
messages
- This is the newest file with the newest logs
Aug. 17 - Aug. 20
- 1 reboot
- manually initiated
- a couple of USB devices plugged/unplugged
- otherwise, again, nothing obvious
Again, I didn't think there would be any obvious indications here, it's just a little TOO obvious. But the dates... They seem a little odd. Aug. 20th was the last logs, but I KNOW for sure the server was running past that date. Thinking of the final date of the logs, I am going back and taking a look at some of the other logs I already looked at, but this time noting the dates of the logs.
auth.log
- This one too ends on Aug. 20th
Seems like I may be narrowing down the date where the compromise first occurred. Let's take a look at some more files /var/log/* files...
daemon.log
- last log is Aug. 19 16:31 hrs
syslog
- last log is Aug. 20 10:30 hrs
user.log
- last log is Aug. 17 12:35 hrs
There are a bunch of other logs files, but none of them carry any kind of date/time stamps, so I cannot tell how (if at all) they correlate to the compromise. Continuing on to some of the /var/log/ subfolder...
/var/log/apt/*
history.log
- Well this is cool. It gives me a hgistory of what was installed via apt-get, and the date/time stamps. This file starts with:
Start-Date: 2014-12-05 19:29:48
Woo Yeah! I'm NOT crazy, FINALLY something to prove that the server was at least mostly functional after Aug. 20th. Let's take a look through these logs and see what may have happened on and around Aug. 20th. So I have to go to the first log file to get that far back... Let's see what was installed:
Aug. 17:
wine
dnsutils
Aug. 20:
scapy
purge python-scapy
scapy --install-suggests --fix-missing
(I very distinctly recall doing this, it caused me some pain)
Aug. 28:
usbutils
mutt
While certainly a cool set of logs, there is nothing to indicate any compromise. Maybe it had something to do with the failed scapy install...???
/vart/log/ajenti/*
So finally I find some more recent logs to hel pprove that it was indeed working, or at least the webservices were. I had been using Ajenti to remotely manage my server, and this as well as working just fine. The one thing it had going for it was a built-in web CLI console, and that wwas my primary reason for choosing it. The logs, however, only go as far back as Nov. 28th, so again, nothing to indicate any problem or compromise.
/var/log/apache2/*
From the Apache website: "The server access log records all requests processed by the server." And wow, is there ever a lot here. These files would require their own analysis on their own. I will be doing so later.
Again, there are certainly other folders and files, but they either contain no date/time stamps, or are for other things that are otherwise unrelated (proftpd for example... It was installed but never configured). So this is for this bit of analysis... The next panalysis will likely be for the /var/log/apache2/* files. Stay tuned!
