Review Tomcat7 Ubuntu Package Upgrade Before Updating

Problem

I needed to see what had changed in the new ubuntu update of tomcat 7 before installing the update for apt-get upgrade.

Solution

First check the version of ubuntu installed. Also check the version of ubuntu you are running:

 # lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 14.04.5 LTS
Release: 14.04
Codename: trusty

Then go to the tomcat7 package in Ubuntu and depending on you Ubuntu release check what has changed :

Ubuntu Tomcat 7 Package information (https://launchpad.net/ubuntu/+source/tomcat7)
Ubuntu’s Tomcat7 Package Information (https://launchpad.net/ubuntu/+source/tomcat7)
Ubuntu Tomcat 7 Package information (https://launchpad.net/ubuntu/+source/tomcat7)
Ubuntu’s Tomcat7 Package Information (https://launchpad.net/ubuntu/+source/tomcat7)

Sources

tomcat7 package in Ubuntu

Why don’t the Ubuntu repositories have the latest versions of software?

 

Check Tomcat Version

Problem

Needed to check the version of tomcat running before I could decide whether of not to update it.

Solution

From the command line look for where the version.sh file is:

$ sudo find / -name "version.sh"

Once you find it run this bash script as follows:

# /usr/share/tomcat7/bin/version.sh
Using CATALINA_BASE: /usr/share/tomcat7
Using CATALINA_HOME: /usr/share/tomcat7
Using CATALINA_TMPDIR: /usr/share/tomcat7/temp
Using JRE_HOME: /usr/lib/jvm/java-7-oracle
Using CLASSPATH: /usr/share/tomcat7/bin/bootstrap.jar:/usr/share/tomcat7/bin/tomcat-juli.jar
Server version: Apache Tomcat/7.0.52 (Ubuntu)
Server built: Jun 30 2016 01:59:37
Server number: 7.0.52.0
OS Name: Linux
OS Version: 3.13.0-53-generic
Architecture: amd64
JVM Version: 1.7.0_80-b15
JVM Vendor: Oracle Corporation

Sources

How to check Tomcat version Installed

Mysql 5.7 Customizations

Problem

Needed to remove the ONLY GROUP BY sql mode and also change the default Character Set

Solution

Login to the MySQL shell and the select the available SQL Modes excluding the ONLY GROUP BY mode:

mysql> select @@sql_mode;
+-------------------------------------------------------------------------------------------------------------------------------------------+
| @@sql_mode |
+-------------------------------------------------------------------------------------------------------------------------------------------+
| ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION |
+-------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)

Add the SQL modes to the mysql conf file on Ubuntu 16_04, located in /etc/mysql/mysql.conf.d/mysqld.cnf

Also add the utf8 character set under mysqld in the above conf file so you will have the following:

[mysqld]
character-set-server=utf8
collation-server=utf8_general_ci
sql-mode="STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION"

Then restart MySQL:

root@whiscardz ~ # service mysql stop
root@whiscardz ~ # ps aux | grep mysql
root 9789 0.0 0.0 15444 1028 pts/1 S+ 09:27 0:00 grep --color=auto mysql
root@whiscardz ~ # service mysql start
root@whiscardz ~ # ps aux | grep mysql
mysql 9828 12.0 0.8 1229212 146004 ? Ssl 09:27 0:00 /usr/sbin/mysqld
root 9865 0.0 0.0 15444 980 pts/1 S+ 09:27 0:00 grep --color=auto mysql

Check the Character SET and sql mode:

mysql> select @@sql_mode;
+------------------------------------------------------------------------------------------------------------------------+
| @@sql_mode |
+------------------------------------------------------------------------------------------------------------------------+
| STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION |
+------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)

mysql> show variables where variable_name like '%char%';
+--------------------------+----------------------------+
| Variable_name | Value |
+--------------------------+----------------------------+
| character_set_client | utf8 |
| character_set_connection | utf8 |
| character_set_database | utf8 |
| character_set_filesystem | binary |
| character_set_results | utf8 |
| character_set_server | utf8 |
| character_set_system | utf8 |
| character_sets_dir | /usr/share/mysql/charsets/ |
+--------------------------+----------------------------+
8 rows in set (0.00 sec)

mysql> show variables where variable_name like '%collation%';
+----------------------+-----------------+
| Variable_name | Value |
+----------------------+-----------------+
| collation_connection | utf8_general_ci |
| collation_database | utf8_general_ci |
| collation_server | utf8_general_ci |
+----------------------+-----------------+
3 rows in set (0.00 sec)

References

disable-only-full-group-by

mysql-5-7-12-disabling-only-full-group-by-sql-mode-is-not-fully-successful

cant-change-charset-in-mysql-5-7-from-latin-to-utf8

mysql-utf8-character-set

Java Heap Size Exception

Problem

I was getting this error after setting up a new instance of Tomcat7, using Java 1.7 and running OpenMRS2:

Exception in thread “ActiveMQ Journal Checkpoint Worker” java.lang.OutOfMemoryError: Java heap space
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
at java.lang.Class.getDeclaredMethod(Class.java:2128)
at java.io.ObjectStreamClass.getInheritableMethod(ObjectStreamClass.java:1442)
Exception in thread “ActiveMQ Broker[localhost] Scheduler” at java.io.ObjectStreamClass.access$2200(ObjectStreamClass.java:72)
at java.io.ObjectStreamClass$2.run(ObjectStreamClass.java:510)
at java.io.ObjectStreamClass$2.run(ObjectStreamClass.java:472)
at java.security.AccessController.doPrivileged(Native Method)
at java.io.ObjectStreamClass.<init>(ObjectStreamClass.java:472)
at java.io.ObjectStreamClass.lookup(ObjectStreamClass.java:369)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1134)
at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1548)
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1509)
at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178)
at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348)
at org.apache.activemq.store.kahadb.MessageDatabase.checkpointProducerAudit(MessageDatabase.java:1250)
at org.apache.activemq.store.kahadb.MessageDatabase.checkpointUpdate(MessageDatabase.java:1134)
at org.apache.activemq.store.kahadb.MessageDatabase$10.execute(MessageDatabase.java:698)
at org.apache.kahadb.page.Transaction.execute(Transaction.java:728)
at org.apache.activemq.store.kahadb.MessageDatabase.checkpointCleanup(MessageDatabase.java:696)
at org.apache.activemq.store.kahadb.MessageDatabase$3.run(MessageDatabase.java:293)
java.lang.OutOfMemoryError: Java heap space
at java.util.HashMap.newNode(HashMap.java:1734)
at java.util.HashMap.putVal(HashMap.java:630)
at java.util.HashMap.putMapEntries(HashMap.java:514)
at java.util.HashMap.<init>(HashMap.java:489)
at org.apache.activemq.broker.region.AbstractRegion.getDestinationMap(AbstractRegion.java:223)
at org.apache.activemq.broker.region.RegionBroker.getDestinationMap(RegionBroker.java:134)
at org.apache.activemq.broker.region.RegionBroker.purgeInactiveDestinations(RegionBroker.java:922)
at org.apache.activemq.broker.region.RegionBroker$1.run(RegionBroker.java:110)
at org.apache.activemq.thread.SchedulerTimerTask.run(SchedulerTimerTask.java:33)
at java.util.TimerThread.mainLoop(Timer.java:555)
at java.util.TimerThread.run(Timer.java:505)

A Java Heap Size error occurs because of certain memory intensive actions (like Data Exporting). The new Spring managed API uses a lot of memory to cache the xml beans. You may need to increase the default memory allocation.

Solution

According to this wiki page, the solution was to adjust the memory settings. Depending on your OS, I was using Ubuntu 16.04.1 LTS.

vim /etc/default/tomcat7

Then change the JAVA_OPTS variable to:

JAVA_OPTS=”-Djava.awt.headless=true -Xmx1024M -Xms1024M -XX:PermSize=256m -XX:MaxPermSize=256m -XX:NewSize=128m”

You can adjust the memory as per your needs, in my case this was multiplying everything by 2:

JAVA_OPTS=”-Djava.awt.headless=true -Xmx2048M -Xms2048M -XX:PermSize=512m -XX:MaxPermSize=512m -XX:NewSize=256m”

Resources

Troubleshooting Memory Errors

Java heap space errors when starting a new installation

Configure Priority of Process Startup Linux

Problem

I had an application that is dependent on another, starting up at the same time as the other application. One potential solution was to add a wait/sleep timer to the startup script but this would slow up booting or add unnecessary wait time when the application needed to be restarted without server reboot.

Solution

Change the boot sequence of the application that is dependent on the other, making it start up much later than the rest. This is well explained here.

Ubuntu still uses init scripts and that are symlinked to different runlevels. Therefore:

  • Check the current boot sequence. This will give you a long list, you can filter it further to see the actual startup script you are looking for:
root@server:/home/user# ls /etc/rc*/
  • Remove the current scripts:
root@server:/home/user# update-rc.d -f source remove
Removing any system startup links for /etc/init.d/source ...
/etc/rc0.d/K20source
/etc/rc1.d/K20source
/etc/rc2.d/S20source
/etc/rc3.d/S20source
/etc/rc4.d/S20source
/etc/rc5.d/S20source
/etc/rc6.d/K20source
root@server:/home/user# update-rc.d source start 95 2 3 4 5 . stop 05 S 0 1 6 .
Adding system startup for /etc/init.d/source ...
/etc/rc0.d/K05source -> ../init.d/source
/etc/rc1.d/K05source -> ../init.d/source
/etc/rc6.d/K05source -> ../init.d/source
/etc/rcS.d/K05source -> ../init.d/source
/etc/rc2.d/S95source -> ../init.d/source
/etc/rc3.d/S95source -> ../init.d/source
/etc/rc4.d/S95source -> ../init.d/source
/etc/rc5.d/S95source -> ../init.d/source

Finally restart your server and see if this was successful.

Resources

What’s the connection between “/etc/init.d” and “/etc/rcX.d” directories in Linux?

How to Delay A Startup Service?

update-rc.d command warning

UbuntuBootupHowto

upstart Getting Started

Ubuntu’s Upstart event-based init daemon

custom service start at boot

UpstartHowtoUpstart Intro, Cookbook and Best Practises

update-rc.d ignoring my init script’s runlevels

MySQL Strip Time from Timestamp Columns

Problem

I needed a way to be able to only use the date part of a timestamp column in the WHERE clause without having to introduce another element such as formatting.

Solution

Solution was to use BETWEEN and then CAST the start and end dates as date as explained in this post.

For instance:

SELECT * FROM testdata WHERE current_date BETWEEN CAST(start_date as DATE) and CAST(end_date as DATE);

Side notes,

CURDATE() Returns the current date as a value in 'YYYY-MM-DD' or YYYYMMDD format, depending on whether the function is used in a string or numeric context.

Use IFNULL() to check if a returns a null and replace the null with something e.g blank:

IFNULL(personname.given_name,”) as GivenName

Update

For some reason the above did not work for me on MySQL 5.7.13. When I used the method above, data in the end date which was the last day of the month 2016-08-31 was not being included in the result. According to this explanation, it could be that BETWEEN isnt incuding the last / end date.

Solution 2

Solution was remove the CASTing and to instead concatenate the time i.e midnight 00:00:00 to the begin date and to the end date, add 23:59:59 so the query now becomes:

SELECT * FROM testdata WHERE current_date BETWEEN CONCAT(start_date' ' , '00:00:00') and CONCAT(end_date' ' , '00:00:00');

Resource:

SQL between dates including start and end dates

How to check if field is null or empty mysql?

How do I check if a column is empty or null in mysql

Working with NULL Values

Tomcat URL Redirect

Problem

Needed to re-direct a user to the default Tomcat web app when they select the base URL

Solution

Simple solution was to add the following line:

<% response.sendRedirect("/nameofyourwebapp"); %>

to the index.jsp page in /usr/local/tomcat/webapps/ROOT.

Therefore your top section of the default Tomcat index.jsp will now probably look like this:

<!DOCTYPE html>
<% response.sendRedirect("/nameofyourwebapp"); %>
<%@ page session="false" %>
<%
java.text.SimpleDateFormat sdf = new java.text.SimpleDateFormat("yyyy");
request.setAttribute("year", sdf.format(new java.util.Date()));
request.setAttribute("tomcatUrl", "http://tomcat.apache.org/");
request.setAttribute("tomcatDocUrl", "/docs/");
request.setAttribute("tomcatExamplesUrl", "/examples/");
%>

The best solution would be to use UrlRewriteFilter I havent tested this out yet though. The above was simple enough for my needs.

Resources

Tomcat base URL redirection

UrlRewriteFilter

UrlRewriteFilter 4.0.3

301 permanent redirect with Tomcat HOWTO

How do I override the default home page loaded by Tomcat?

Ubuntu Install Only Certain Updates

Problem

Needed a way to choose which updates to install.

Solution

apt-get update — To check what’s new

apt-get upgrade — To update installed packages without kernel modules

apt-get dist-upgrade — To install all updates for installed packages with kernel modules

apt-get -s dist-upgrade — To simulate the above without making any changes.

apt-get install –only-upgrade [package_name] e.g

apt-get install –only-upgrade liboxideqt-qmlplugin — To only upgrade one specific package and not all (This is useful if you first run the simulate dist upgrade command above)

Resources

How can I install just security updates from the command line?

How to install updates via command line?

How to upgrade a single package using apt-get?