Apache Reverse Proxy on Multiple ports

Problem

I needed to share logs with a colleague on one url that is used to access an application. In the backend, apache proxies requests to one application and the logs are reserved by a different application

Solution

  • Open the apache site host configuration file and then add the other application in the ProxyPass directive.
  • Note the order in which you add the directive because if for example the application is served in the / directory and the logs are served in the /log directory then make sure /log comes before / as shown below:
root@whiscardz:# vim /etc/apache2/sites-available/app.whiscardz.org.conf
<VirtualHost *:80>
        ServerName app.whiscardz.org
        ProxyPass        /logs http://localhost:1223/
        ProxyPassReverse /logs http://localhost:1223/

        ProxyPass        / http://localhost:1221/
        ProxyPassReverse / http://localhost:1221/
        ProxyPassReverseCookieDomain localhost app.whiscardz.org
        ProxyPreserveHost On
</VirtualHost>

References

reverse proxy on multiple ports

Breakdown disk usage on Linux server

Problem

Have an Ubuntu server whose disk space keeps growing over time with no clear visible culprits causing the same.

Need a way to be able to sort the directories by usage.

Solution

  • Login as the root user and then run the following command:
root@whiscardz:/# du -h -a / | sort -h | tail -n 15
28G     /home/whiscardz/idempiere/do/database/dump
29G     /home/teamcity
38G     /home/vagrant/VMs/default_1587717642629_75966
38G     /home/vagrant/VMs/default_1587717642629_75966/ubuntu-bionic-18.04-cloudimg.vdi
43G     /opt
46G     /home/whiscardz
57G     /var/lib/docker/aufs
57G     /var/lib/docker/aufs/diff
94G     /var/lib/docker
98G     /var/lib
102G    /var
111G    /home/vagrant/VMs
114G    /home/vagrant
212G    /home
371G    /
  • The above estimates space usage of all files in the root(/) directory in human readable form then sorts them and lists the last 15 of the summary
  • After getting the above summary if you want to drill down further into a directory, then run the following:
root@whiscardz:~# sudo du -h /home/teamcity/ | sort -rh | head -5                                                                                                        
29G     /home/teamcity/
15G     /home/teamcity/.BServer
8.7G    /home/teamcity/bAgent
8.6G    /home/teamcity/.BServer/backup
6.7G    /home/teamcity/bAgent/system

References

13 du (Disk Usage) Command Examples In Linux

How to Check Disk Space Usage in Linux

3 Simple Ways to Get the Size of Directories in Linux

How to Get the Size of a Directory in Linux

Install Fail2Ban using UFW and Custom ssh port Ubuntu

Problem

Noticed automated ssh brute-force login attempts in the auth logfile on a server

Jun 22 12:49:55 Whiscardz sshd[29114]: Received disconnect from 18.2.17.3 port 47636:11: Normal Shutdown, Thank you for playing [preauth]

Solution

Fail2ban attempts to alleviate these issues by providing an automated way of not only identifying possible break-in attempts, but acting upon them quickly and easily in a user-definable manner.

Fail2ban scans log files and detects patterns which correspond to possible break-in attempts and then performs actions such as adding a new rule in a firewall chain and sending an e-mail notification to the system administrator.

Installation and Configuration

  • Install fail2ban
imela@whiscardz:~$ sudo apt update
imela@whiscardz:~$ sudo apt install fail2ban
  • Enable the ufw firewall that comes with most ubuntu distros. This is what fail2ban will use to block ips that fail2ban finds:
imela@whiscardz:~$ sudo ufw default deny incoming
imela@whiscardz:~$ sudo ufw default allow outgoing
imela@whiscardz:~$ sudo ufw allow ssh
imela@whiscardz:~$ sudo ufw allow 3322
imela@whiscardz:~$ sudo ufw enable
Firewall is active and enabled on system startup
imela@whiscardz:~$ sudo ufw status verbose
Status: active
  • If you use ssh on a custom port, then create ufw app profile:
imela@whiscardz:~$ sudo vim /etc/ufw/applications.d/openssh-server

  1 [OpenSSH]
  2 title=Secure shell server, an rshd replacement
  3 description=OpenSSH is a free implementation of the Secure Shell protocol.
  4 ports=22/tcp
  5 
  6 [OpenSSH-3322]
  7 title=Secure shell server, an rshd replacement
  8 description=OpenSSH is a free implementation of the Secure Shell protocol.
  9 ports=3322/tcp

imela@whiscardz:~$ sudo ufw app list 
[sudo] password for imela: 
Available applications:
  OpenSSH
  OpenSSH-3322
  • Customize the sshd jail:
imela@whiscardz:~$ sudo vim /etc/fail2ban/fail2ban.local

[sshd]
port = 3322
action = ufw[application="OpenSSH-3322", blocktype=reject]
logpath = %(sshd_log)s
backend = %(sshd_backend)s
  • Note that “action” pass the “application” parameter that corresponds to the app profile that we have created earlier. Reload fail2ban so that it recognizes the new jail configuration.
sudo fail2ban-client reload
  • Now you can test the jail. Try logging in to the box with invalid credentials a couple of times, and check the jail status
imela@whiscardz:~$ sudo fail2ban-client status sshd
    Status for the jail: sshd
    |- Filter
    | |- Currently failed: 0
    | |- Total failed: 10
    | `- File list: /var/log/auth.log
    `- Actions
    |- Currently banned: 1
    |- Total banned: 2
    `- Banned IP list: 14.24.xxx.yyy
  • Check whether the ufw filter has been added:
    To                         Action      From
    --                         ------      ----
    3222/tcp (OpenSSH-3222)    REJECT IN   14.24.xxx.yyy
    3222                       ALLOW IN    Anywhere
    8443                       ALLOW IN    Anywhere
    3222 (v6)                  ALLOW IN    Anywhere (v6)
    8443 (v6)                  ALLOW IN    Anywhere (v6)
  • The setup will result in failban inserting an ufw filter that block both ports configured on that particular profile.

References

fail2ban wiki

fail2ban, ufw, and sshd with custom port on Ubuntu

Securing Ubuntu 18.04 ssh server with ufw and fail2ban

potential ufw and fail2ban conflicts

A Tutorial for Using Fail2ban to Secure Your Server

Limiting failed ssh login attempts with fail2ban

WARN: Duplicate profile ‘Apache’, using last found (ufw)

Problem

When I run any ufw command on the terminal, I get the following error:

imela@whiscardz:~$ sudo ufw status verbose
WARN: Duplicate profile 'Apache', using last found
WARN: Duplicate profile 'Apache Secure', using last found
WARN: Duplicate profile 'Apache Full', using last found
Status: inactive

Solution

Check the following location, there are 2 duplicate apache configuration files with the same content

root@whiscardz:~/ufw/2020-07-17# less /etc/ufw/applications.d/apache2.2-common 
[Apache]
title=Web Server
description=Apache v2 is the next generation of the omnipresent Apache web server.
ports=80/tcp

[Apache Secure]
title=Web Server (HTTPS)
description=Apache v2 is the next generation of the omnipresent Apache web server.
ports=443/tcp

[Apache Full]
title=Web Server (HTTP,HTTPS)
description=Apache v2 is the next generation of the omnipresent Apache web server.
ports=80,443/tcp
root@whiscardz:~/ufw/2020-07-17# less /etc/ufw/applications.d/apache2-utils.ufw.profile 
[Apache]
title=Web Server
description=Apache v2 is the next generation of the omnipresent Apache web server.
ports=80/tcp

[Apache Secure]
title=Web Server (HTTPS)
description=Apache v2 is the next generation of the omnipresent Apache web server.
ports=443/tcp

[Apache Full]
title=Web Server (HTTP,HTTPS)
description=Apache v2 is the next generation of the omnipresent Apache web server.
ports=80,443/tcp

Now move one of the files to a specific location and now check the status of any ufw command:

root@whiscardz:~/ufw/2020-07-17# mv /etc/ufw/applications.d/apache2.2-common .
root@whiscardz:~/ufw/2020-07-17# ufw status verbose
Status: inactive

References

Unable to add firewall rule “Duplicate Profile”

vim search for ) replace with text then newline then )

Problem

I have a huge sql query with 574 lines and I needed to replace a closing bracket ) with text and then a new line with a closing bracket

Before:


) AS

After:


GROUP BY v.visit_id
) AS

Solution

Using vim’s search and replace feature


:%s/^)/GROUP BY v.visit_id\r)/gc

The above searches for the lines beginning with the closing bracket ) and then replaces it with text and then new line (\r) followed by a closing bracket )

References

How to replace a character by a newline in Vim

Add a newline after given patterns

Enable Windows Defender on Windows 7

Problem

Windows defender was not running and needed to activate it.

Solution

There is this nice youtube video explaining this process. Below is a summary:

  • Open Control Panel
  • Then Go to View by Categories and select Large icons:

Selection_999(1261).png

  • Then you should see windows defender at the bottom of the page:

Selection_999(1262).png

  • Once you click on it, it will ask you if you want to start it, click ok and you are good to go.

This program is blocked by group policy

Problem

I got this error when trying to enable windows defender from the control panel.

This program is blocked by group policy error message
This program is blocked by group policy

On doing some research it appears this may have been caused by a malware and a virus scan upon activating windows defender validated this suspicion.

Solution

The solution as shown in this youtube video was to delete policies from the registry. I did not use the batch file. Deleting the registry entries worked.

  • Reboot the machine and then press F8 as it is booting to go into safe mode.
  • Once its started from the start menu type Regedit to open the windows registry.
  • Then First create a backup of the registry by clicking File Followed by Export and save to a location of your choice.
  • Then Next delete the following registry entries. You can review this video in-case the screenshots below are not clear.

First regristry key to be deleted
First registry key to be deleted

Second regristry key to be deleted
Second registry key to be deleted

Third regristry key to be deleted
Third registry key to be deleted

Last registry key to be deleted
Last registry key to be deleted

References

How To Fix This Program is Blocked by Group Policy in Windows 7/8/10

Installing virtualbox plus GUI ubuntu 16.04

Problem

Had virtual box installed and was using it from the command line. The process of creating a new virtual machine using this approach proved difficult. Tools like vagrant make this process easier but I needed to share this image with a non vagrant user.

Solution

Installed the default version of virtualbox that comes with ubuntu 16.04.

When I tried installing the latest version of virtual box, I would get the following error when starting a newly defined image:


Failed to open a session for the virtual machine bandago.

The virtual machine 'vmImage' has terminated unexpectedly during startup with exit code 1 (0x1).

Result Code: NS_ERROR_FAILURE (0x80004005)
Component: MachineWrap
Interface: IMachine {85cd948e-a71f-4289-281e-0ca7ad48cd89}

The solution maybe to after installing virtualbox, additionally installing the following packages:

(I never got to test this though)

vde2

virtualbox-guest-additions-iso

virtualbox-qt

Solution

Install virtualbox and the above packages:


imela@whiscardz ~ $ sudo apt-get install virtualbox virtualbox-guest-additions-iso virtualbox-qt
Reading package lists... Done
Building dependency tree
Reading state information... Done
virtualbox is already the newest version (5.1.38-dfsg-0ubuntu1.16.04.2).
The following NEW packages will be installed:
virtualbox-guest-additions-iso virtualbox-qt
0 upgraded, 2 newly installed, 0 to remove and 2 not upgraded.
Need to get 47.4 MB of archives.
After this operation, 87.4 MB of additional disk space will be used.
Do you want to continue? [Y/n]

Note: After defining a new vm, when it’s time to link the ISO, double click on the vm definition and then select the iso.

References

Creating a Virtual Machine

Installing Oracle VM VirtualBox and Extension Packs

Oracle Linux_Downloads

How to Install VirtualBox 5.2 on Ubuntu 16.04 LTS

Exporting and Importing Virtual Box Images

Problem

Needed to save a virtual box image to be shared with someone else

Solution

Using vboxmanage command line to export the image and then test importing the image.

Steps

  • First list the virtual box images available

imela@whiscardz$ vboxmanage list vms
"test_default_1549514247520_18667" {6e29d56a-6531-4274-9d91-c6719e9fdaf3}
  • Then export the image you want to a file

imela@whiscardz$ vboxmanage export test_default_1549514247520_18667 -o test_default_1549514247520_18667.ova

  • Then copy it to the machine that you want to import the image.
  • Finally import the image.

imela@whiscardz2$ vboxmanage import test_default_1549514247520_18667.ova
0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%

Potential Issues

When I tried exporting to a file name other than the name of the image, it failed and threw the error below (it could be the extention):


imela@whiscardz2$ vboxmanage export test_default_1549514247520_18667 --output otherFileName.ovf
0%...
Progress state: VBOX_E_IPRT_ERROR
VBoxManage: error: Appliance write failed
VBoxManage: error: RTVfsFsStrmAdd failed for 'otherFileName.ovf' (VERR_ALREADY_EXISTS)
VBoxManage: error: Details: code VBOX_E_IPRT_ERROR (0x80bb0005), component ApplianceWrap, interface IAppliance
VBoxManage: error: Context: "RTEXITCODE handleExportAppliance(HandlerArg*)" at line 1263 of file VBoxManageAppliance.cpp

References

How to export and import VirtualBox VM images?

VBoxManage registervm/unregistervm

Export to OVF

VBoxManage export

VBoxManage import