IR 382: Cabby (40 pts extra)

What You Need for This Project

Purpose

To use Cabby, a Python-based tool to efficiently use the TAXII exchange protocol and collect open-source threat intelligence.

Installing Cabby

In your Linux machine, execute the following commands. These commands prepare a Python virtual environment and install Cabby in it. 
sudo apt install python3-venv -y
python3 -m venv ./cabby-env
source ./cabby-env/bin/activate
python -m pip install libtaxii
python -m pip install cabby

taxii-discovery --path http://hailataxii.com/taxii-discovery-service
The response is nicely formatted list of available services, as shown below.

Finding Available Collections

In your Linux machine, execute this command: 
taxii-collections --path http://hailataxii.com/taxii-data
The response is longer list of available feeds, as shown below.

Filtering with Grep

Let's clean that list up. All we need is the names.

In your Linux machine, execute this command: 

taxii-collections --path http://hailataxii.com/taxii-data | grep Name
You see only the lines including "Name", as shown below.

Polling Phishtank

In your Linux machine, execute this command, to poll some data from Phishtank: 
taxii-poll --path http://hailataxii.com/taxii-data \
           --collection guest.phishtank_com \
           --begin 2018-05-25T02:00:00 \
           --end 2018-05-25T02:10:00
You get a long list of data, ending with "136 blocks polled", as shown below.

Redirecting Output

In your Linux machine, execute these commands, to poll the same data and place it in a file named "phish.txt", and view the first ten lines of that file: 
taxii-poll --path http://hailataxii.com//taxii-data \
           --collection guest.phishtank_com \
           --begin 2018-05-25T02:00:00 \
           --end 2018-05-25T02:10:00 > phish.txt

head phish.txt
The first ten lines show a lot of data, but none of it is simple or obviously useful, as shown below.

Using "less"

To view more of the file, execute this command: 
less phish.txt
Use the down-arrow on your keyboard to scroll down a few lines to find the "indicator:Type" line, highlighted in the image below.

This line shows what type of data is given in the object. We'll start with that information.

Press q to exit from "less".

Counting Lines

Execute these commands to count various types of lines: 
wc -l phish.txt
grep indicator:Type phish.txt | wc -l
As shown below, there are 4687 lines in the phish.txt file, but only 62 of them contain "indicator:Type".

Counting Unique Values

Execute this command to count the unique values of the lines containing "indicator:Type": 
grep indicator:Type phish.txt | sort | uniq -c
As shown below, all 62 lines are identical. The only type of data in this file is "URL Watchlist".

Using "less" Again

To see the layout of the data, execute this command: 
less phish.txt
Use the down-arrow on your keyboard to scroll down a few lines to find the "indicator:Type" line we saw before.

Notice that the next line contains a URL, highlighted in the image below.

All we need is a list of those URLs, without all the other information.

Press q to exit from "less".

Extracting the URL

First, let's get the lines containing "indicator:Description" with grep.

In your Linux machine, execute the following command: 

grep indicator:Description phish.txt | head
The first ten description lines appear. They are rather long, as shown below.

Using "cut"

The URL ends with a "]" character, and we don't care about anything past that.

Execute this command to cut the lines at the delimiter "]" and keep only the first field, the text before that delimiter: 

grep indicator:Description phish.txt | cut -d "]" -f 1 | head
The first ten description lines appear. They are much cleaner, as shown below.

Using "cut" Again

The URL begins with a "{" character, and we don't care about anything before that.

Execute this command to cut the lines at the delimiter "{" and keep only the second field, the text after that delimiter: 

grep indicator:Description phish.txt | cut -d "]" -f 1 | cut -d "[" -f 2 | head
Now we have just the URLs, as shown below.

Putting the URLs into a File

Execute these commands to put the URLS into a file, see how many lines are in that file, and print the first ten lines: 
grep indicator:Description phish.txt | cut -d "]" -f 1 | cut -d "[" -f 2 > bad_urls.txt
wc -l bad_urls.txt
head bad_urls.txt
The file contains 62 URLs, as shown below.

IR 382.1: Folder (10 pts)

One of the bad URLs begins with "organicplus". Find that URL.

The flag is covered by a green rectangle in the image below.

IR 382.2: Chill (10 pts)

Poll the phishank for the entire date: Oct 12, 2021.

One of the bad URLs contains "chill". The flag is that URL.

IR 382.3: Anomali Feeds (10 pts)

This command finds the services available on the "limo.anomali.com" threat intelligence server.

Notice that this server requires you to include the username and password. 

taxii-discovery \
  --path https://limo.anomali.com/api/v1/taxii/taxii-discovery-service/ \
 --username guest --password guest
Get a list of the collections available from that server. Be careful to use the correct Service Address.

Find the name of the feed containing "TOR". That name is the flag.

IR 382.4: C&C (10 pts)

Find the feed on the "limo.anomali.com" server that lists Command & Control servers (abbreviated as "C_C_Server").

Find all the servers reported on Oct 12, 2021.

Find the IP address with the lowest number in the first octet. That IP address is the flag.

Exiting your Venv

To exit the Python virtual environment, execute this command. 
deactivate

Posted 10-14-21