Analyzing APACHE logs in CLI (& GUI) using GoAccess

Analyzing logs is a major part of System Admin's profile. Whether they are system logs or application logs, its necessary to analyze them get various stats or troubleshoot errors/issues. In this tutorial, we are going to discuss about GoAccess, which we can use for analyzing Apache logs & Nginx logs.

GoAccess is an open source real time web-analyzer tool based for terminal/CLI. It also produces HTML reports that can be viewed through web browser. We can also store the anlayzed reports in JSON or CSV format. GoAccess is very light on resources, very fast in analyzing logs & we can also interact with logs in real time via a terminal.

We will learn to install GoAccess on CentOS & Ubuntu systems & also discuss how to use GoAccess for analyzing apache logs.

(Recommended Read: Checking website statistics using Webalizer)

Dependencies required

For Centos, install the following packages on the system,

$ sudo yum install GeoIP-devel

$ sudo yum install ncurses

For Ubuntu, install the following packages on the system,

$ sudo apt-get install libncursesw5-dev
$ sudo apt-get install libgeoip-dev

Installation on CentOS

GoAccess package is available with EPEL repository, so firstly install the EPEL repositories on your system with following command,

Centos 7
$ rpm -Uvh https://dl.fedoraproject.org/pub/epel/7/x86_64/Packages/e/epel-release-7-11.noarch.rpm

RHEL/CentOS 6 (64 Bit)
$ rpm -Uvh http://download.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm

RHEL/CentOS 6 (32 Bit)
$ rpm -Uvh http://dl.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm

Use one of the mentioned commands based on your system OS & Architecture. Once the EPEL repository has been enabled, install GoAccess with the following command,

$ sudo yum install goaccess

Installation on Ubuntu

GoAccess package is available with the default Ubuntu repository, install GoAccess on Ubuntu using the following command,

$ sudo apt-get install goaccess

But you might not always get the latest package with default repository. Another way is to use the Official GoAccess reporitory. Use the following commands to add the official GoAccess repository & install the package from there,

$ echo "deb http://deb.goaccess.io/ $(lsb_release -cs) main" | sudo tee -a /etc/apt/sources.list.d/goaccess.list
$ wget -O - https://deb.goaccess.io/gnugpg.key | sudo apt-key add -
$ sudo apt-get update
$ sudo apt-get install goaccess

Installation from Source

We can also use the source packages to install the latest GoAccess packages on the system. To install GoAccess, execute the following commands mentioned below as root,

$ wget http://tar.goaccess.io/goaccess-1.2.tar.gz
$ tar -xzvf goaccess-1.2.tar.gz
$ cd goaccess-1.2/
$ ./configure --enable-utf8 --enable-geoip=legacy
$ make
$ make install

That's it, we now have GoAccess installed on our sytems. We will now discuss how to use it to analyze the apache logs.

Analyzing Apache logs

To start analyzing the web server logs, open the terminal & run the following command,

$ goaccess -f /var/log/apache2/access.log

where '/var/log/apache2/access.log' is the location of the web server logs, it may be different for you. Once the above mentioned command has been executed, you will presented with the following screen. Here you have to select the Log format with date & time format. We are using 'Comman Log format',

Alternatively, we can also uncomment the following parameters from '/usr/local/etc/goaccess.conf' file to set a time & date format for all the logs,

$ vi vi /usr/local/etc/goaccess.conf
time-format %H:%M:%S (uncomment it)
date-format %d/%b/%Y (uncomment it)
log-format %h %^[%d:%t %^] "%r" %s %b (Uncomment it as well)

Save file & exit. Now when you execute the following command, you will not be asked to select a log format,

$ goaccess -f /var/log/apache2/access.log

We can now see the dashboard for GoAccess, you can check all the analyzed web server logs using this screen. Once done, press 'Q' key to exit.

We can also analyze the archived logs using the following commands,

$ cat /var/log/httpd/access* | goaccess (For CentOS)
$ zcat -f /var/log/apache2/access.log* | goaccess (For Ubuntu)

Exporting Analyzed logs to html file

We can also export the analyzed logs in html format for accessing them offline directly or through a web server. To create the html file of analyzed logs, execute the following command,

$ zcat -f /var/log/apache2/access.log* | goaccess > access.html , or,

$ goaccess -f /usr/local/apache2/logs/access_log > access.html

This will create a html file which we can open using a web browser or we can also copy the file to document root of web server to access it using a URL. The html reports created, shows detailed graphical view of the analyzed logs,

This completes our tutorial on how to analyzing apache logs using GoAccess. Please feel free to send in your queries/suggestions to us using the comment box below.