fbpx

Offline Squid Proxy log analysis

Squid Access Log analysis with simple python command

What is SquidMagic Master?

squidmagic is a tool which is designed to analyze a web-based network traffic to detect central command and control (C&C) servers and Malicious site, using Squid proxy server and Spamhaus.

 

Who is  Spamhaus?

The Spamhaus Project is an international nonprofit organization that tracks spam and related cyber threats such as phishing, malware and botnets, provides realtime actionable and highly accurate threat intelligence to the Internet’s major networks, corporations and security vendors, and works with law enforcement agencies to identify and pursue spam and malware sources worldwid

 

 

 

You Could get many post regarding squid log analyse in a sort way which are not in depth, However, while you are doing in practice scenario, i’m sure you would get many error to run a tool.

Requirement:

Linux Machine (I’m Using Ubuntu 16.04)
Squidmagic Master tool
 
Dependent packages:
Python3

 

sh, termcolor, configparser, pyzmq.

So here is the step by step configuration.

Install Python 3.6 from PPA

You can also install Python 3.6 from J Fernyhough’s Personal Package Archive
Install the following requirements.
apt-get install software-properties-common python-software-properties
Run the Following command to add the ubuntu repository.
sudo add-apt-repository ppa:jonathonf/python-3.6
Update the repository
sudo apt-get update
Install Python version 3.6 on Ubuntu 16.04
and finally install Python version 3.6
sudo apt-get install python3.6
However, it can shows you the older python version which is 3.5
# python3 -V 
Python 3.5.2
On my Ubuntu machine, there are two versions of python available, python3.5 as default python version and python3.6. In this step, we will change the default python version to python 3.6
sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.5 1
 
sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.6 2
sudo update-alternatives --config python3
 
 
 
Selection    Path                Priority   Status
————————————————————
* 0            /usr/bin/python3.6   2         auto mode
  1            /usr/bin/python3.5   1         manual mode
  2            /usr/bin/python3.6   2         manual mode
 
After the installation of python and selected a best version, we need to install python module so here is the following command to get install.
 
Python Packages name : python-zmq
sudo apt-get install python-zmq
Install Python Packages: sh
 
sh is a full-fledged subprocess replacement for Python 2.6 – 3.6, PyPy and PyPy3 that allows you to call any program as if it were a function:
pip install sh
Install Python Package : python-pip
 
 
Pip is a package management system used to install and manage software packages written in Python which can be found in the Python Package Index (PyPI).
sudo apt-get install python-pip
install python package: termcolor
termcolor is Color formatting for output in terminal
pip install termcolor
Install python package: configparser
 
 

 

Note
The ConfigParser module has been renamed to configparser in Python 3. The 2to3 tool will automatically adapt imports when converting your sources to Python 3.
 
 
sudo pip install configparser
Go to your squidmagic dirctory which you had downloaded, if not you can download the tool https://github.com/ch3k1/squidmagic
 
Now we can run the script to following command.
python squidmagic.py /path of your access.log
                 _     _                       _      
                (_)   | |                     (_)     
 ___  __ _ _   _ _  __| |_ __ ___   __ _  __ _ _  ___ 
/ __|/ _` | | | | |/ _` | '_ ` _ \ / _` |/ _` | |/ __|
\__ \ (_| | |_| | | (_| | | | | | | (_| | (_| | | (__ 
|___/\__, |\__,_|_|\__,_|_| |_| |_|\__,_|\__, |_|\___|
        | |                               __/ |       
        |_|                              |___/        
     Analyzing...

Analyzing by SBL Advisory...
 Spam server detected, ip is 65.182.101.221
Analyzing by SBL_CSS Advisory...
 safe server detected, host or ip is 65.182.101.221
Analyzing by PBL Advisory...
 safe server detected, host or ip is 65.182.101.221

Leave a Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.