Copy Multiple lines from a logfile

Problem

I needed to capture an error / stack trace (list of the method calls that the application was in the middle of when an Exception was thrown – explanation from stackoverflow) from a log file to another file for attaching to a bug tracker

Solution

Used this answer from unix.stackexchange that proposed using a shell function that used sed that you could enter the from,to line numbers, log file and output file as shown

In the user’s .bashrc file add:

$ tail .bashrc 
#Printing section of a Line From A logFile
printLine () 
{ 
 sed -n -e "$1p" "$2"
}

Useage:

$ printLine 2000,2250 /var/log/tomcat/catalina.out > java_error.txt

Sources

Easy way to copy lines from one file to another

What is a stack trace, and how can I use it to debug my application errors?

Install Multiple Java versions and Setup Maven to Use one – Java 1.7

Problem

I was running Java 1.8 by default but I needed to build some modules that require Java 1.7

Solution

Check the default version of Java installed as well as the version of Java that maven is using:


implementer@whiscardz ~ $ java -version
java version "1.8.0_131"
Java(TM) SE Runtime Environment (build 1.8.0_131-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode)

implementer@whiscardz ~ $ mvn -v
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512m; support was removed in 8.0
Apache Maven 3.3.9
Maven home: /usr/share/maven
Java version: 1.8.0_131, vendor: Oracle Corporation
Java home: /usr/lib/jvm/java-8-oracle/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "4.4.0-78-generic", arch: "amd64", family: "unix" 
  • Install Java 1.7 as instructed in this link.
  • Create a .mavenrc file in your home directory and then add the JAVA_HOME of Java 1.7 In my case this would be JAVA_HOME=/usr/lib/jvm/java-7-oracle/jre
implementer@whiscardz ~ $ vim .mavenrc

JAVA_HOME=/usr/lib/jvm/java-7-oracle/jre
  • Optional, Run the maven file to assign the JAVA variable on the current terminal
implementer@whiscardz ~ $ source .mavenrc
  • Check version of Java Maven is now using:
implementer@whiscardz ~ $ mvn -version
Apache Maven 3.3.9
Maven home: /usr/share/maven
Java version: 1.7.0_80, vendor: Oracle Corporation
Java home: /usr/lib/jvm/java-7-oracle/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "4.4.0-78-generic", arch: "amd64", family: "unix"

References

Create .mavenrc file

How to change maven java home

Manually Install Tomcat7 on Ubuntu 16.04

Problem

Tomcat7 in the Ubuntu 16.04 repository was compiled using Java8 and this caused a problem with OpenMRS that needed Java7.

This issue has been mentioned also on stackoverflow and also apache bugzilla.

Solution

The solution was to download Apache manually. I used Apache v7.0.70. You can download and extract the folder in the tmp folder:

root@whiscardz # cd /tmp/
root@whiscardz /tmp # curl -O http://archive.apache.org/dist/tomcat/tomcat-7/v7.0.70/bin/apache-tomcat-7.0.70.tar.gz 
 % Total % Received % Xferd Average Speed Time Time Time Current
 Dload Upload Total Spent Left Speed
100 8715k 100 8715k 0 0 2155k 0 0:00:04 0:00:04 --:--:-- 2155k

root@whiscardz /tmp # ll apache-tomcat-7.0.70.tar.gz 
-rw-r--r-- 1 root root 8924465 Jan 13 13:00 apache-tomcat-7.0.70.tar.gz
root@whiscardz /tmp # tar xzvf apache-tomcat-7*tar.gz -C /opt/tomcat --strip-components=1

root@whiscardz /tmp # ll /opt/tomcat/
total 124
drwxr-xr-x 9 root root 4096 Jan 13 13:01 ./
drwxr-xr-x 11 root root 4096 Jan 13 12:55 ../
drwxr-xr-x 2 root root 4096 Jan 13 13:01 bin/
drwxr-xr-x 2 root root 4096 Jun 15 2016 conf/
drwxr-xr-x 2 root root 4096 Jan 13 13:01 lib/
-rw-r--r-- 1 root root 56846 Jun 15 2016 LICENSE
drwxr-xr-x 2 root root 4096 Jun 15 2016 logs/
-rw-r--r-- 1 root root 1239 Jun 15 2016 NOTICE
-rw-r--r-- 1 root root 8965 Jun 15 2016 RELEASE-NOTES
-rw-r--r-- 1 root root 16195 Jun 15 2016 RUNNING.txt
drwxr-xr-x 2 root root 4096 Jan 13 13:01 temp/
drwxr-xr-x 7 root root 4096 Jun 15 2016 webapps/
drwxr-xr-x 2 root root 4096 Jun 15 2016 work/
root@whiscardz /tmp # 

Create a tomcat user and group:

root@whiscardz # groupadd tomcat
root@whiscardz # useradd -s /bin/false -g tomcat -d /opt/tomcat tomcat

cd to the directory and then change the owner and permissions and group of the files and directories:

root@whiscardz # cd /opt/tomcat/

root@whiscardz /opt/tomcat # chgrp -R tomcat conf
root@whiscardz /opt/tomcat # chmod g+rwx conf
root@whiscardz /opt/tomcat # chmod g+r conf/*
root@whiscardz /opt/tomcat # chown -R tomcat work/ temp/ logs/ 

Then check the path of Java. This will be used in the JAVA_HOME env variable only appending /jre to the end of the result.

See JAVA_HOME Environment variable in the systemd service file below:

root@whiscardz ~ # update-java-alternatives -l
java-7-oracle 1 /usr/lib/jvm/java-7-oracle 

Create a systemd service file below. Make sure to edit the JAVA_HOME;CATALINA_PID;  CATALINA_HOME; CATALINA_BASE; CATALINA_OPTS; ExecStart and ExecStop Variables to suite your environment

root@whiscardz ~ # vim /etc/systemd/system/tomcat.service

[Unit]
Description=Apache Tomcat Web Application Container
After=network.target

[Service]
Type=forking

Environment=JAVA_HOME=/usr/lib/jvm/java-7-oracle/jre
Environment=CATALINA_PID=/opt/tomcat/temp/tomcat.pid
Environment=CATALINA_HOME=/opt/tomcat
Environment=CATALINA_BASE=/opt/tomcat
Environment='CATALINA_OPTS=-Xmx2048M -Xms2048M -XX:PermSize=512m -XX:MaxPermSize=512m -XX:NewSize=256m -server -XX:+UseParallelGC'
Environment='JAVA_OPTS=-Djava.awt.headless=true -Djava.security.egd=file:/dev/./urandom'

ExecStart=/opt/tomcat/bin/startup.sh
ExecStop=/opt/tomcat/bin/shutdown.sh

User=tomcat
Group=tomcat
UMask=0007
RestartSec=10
Restart=always

[Install]
WantedBy=multi-user.target

Reload the systemd daemon so that it picks up your tomcat.service file above. Then you can start tomcat. You can also stop tomcat and check if it stops.

Then you can enable the service so that it is started when the machine is rebooted :

root@whiscardz ~ # systemctl daemon-reload
root@whiscardz ~ # systemctl start tomcat

root@whiscardz ~ # systemctl stop tomcat
root@whiscardz ~ # systemctl enable tomcat 

Incase you should down tomcat using the shutdown.sh script alone and it doesnt shutdown, check out this post.

Resources

How To Install Apache Tomcat 8 on Ubuntu 16.04

Install Apache Tomcat 9 on CentOS 7 / RHEL 7 / Ubuntu 16.04

How to Install Tomcat 7.0.68 Server on Ubuntu, Debian & LinuxMint

Tomcat shutdown does not kill process

Tomcat doesn’t stop. How can I debug this?

Tomcat 7 doesn’t shut down, process keeps running?

Apache Tomcat 8 Start stop script init.d script

Starting, Stopping, and Restarting Tomcat

Check Tomcat Version

Problem

Needed to check the version of tomcat running before I could decide whether of not to update it.

Solution

From the command line look for where the version.sh file is:

$ sudo find / -name "version.sh"

Once you find it run this bash script as follows:

# /usr/share/tomcat7/bin/version.sh
Using CATALINA_BASE: /usr/share/tomcat7
Using CATALINA_HOME: /usr/share/tomcat7
Using CATALINA_TMPDIR: /usr/share/tomcat7/temp
Using JRE_HOME: /usr/lib/jvm/java-7-oracle
Using CLASSPATH: /usr/share/tomcat7/bin/bootstrap.jar:/usr/share/tomcat7/bin/tomcat-juli.jar
Server version: Apache Tomcat/7.0.52 (Ubuntu)
Server built: Jun 30 2016 01:59:37
Server number: 7.0.52.0
OS Name: Linux
OS Version: 3.13.0-53-generic
Architecture: amd64
JVM Version: 1.7.0_80-b15
JVM Vendor: Oracle Corporation

Sources

How to check Tomcat version Installed

Java Heap Size Exception

Problem

I was getting this error after setting up a new instance of Tomcat7, using Java 1.7 and running OpenMRS2:

Exception in thread “ActiveMQ Journal Checkpoint Worker” java.lang.OutOfMemoryError: Java heap space
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
at java.lang.Class.getDeclaredMethod(Class.java:2128)
at java.io.ObjectStreamClass.getInheritableMethod(ObjectStreamClass.java:1442)
Exception in thread “ActiveMQ Broker[localhost] Scheduler” at java.io.ObjectStreamClass.access$2200(ObjectStreamClass.java:72)
at java.io.ObjectStreamClass$2.run(ObjectStreamClass.java:510)
at java.io.ObjectStreamClass$2.run(ObjectStreamClass.java:472)
at java.security.AccessController.doPrivileged(Native Method)
at java.io.ObjectStreamClass.<init>(ObjectStreamClass.java:472)
at java.io.ObjectStreamClass.lookup(ObjectStreamClass.java:369)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1134)
at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1548)
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1509)
at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178)
at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348)
at org.apache.activemq.store.kahadb.MessageDatabase.checkpointProducerAudit(MessageDatabase.java:1250)
at org.apache.activemq.store.kahadb.MessageDatabase.checkpointUpdate(MessageDatabase.java:1134)
at org.apache.activemq.store.kahadb.MessageDatabase$10.execute(MessageDatabase.java:698)
at org.apache.kahadb.page.Transaction.execute(Transaction.java:728)
at org.apache.activemq.store.kahadb.MessageDatabase.checkpointCleanup(MessageDatabase.java:696)
at org.apache.activemq.store.kahadb.MessageDatabase$3.run(MessageDatabase.java:293)
java.lang.OutOfMemoryError: Java heap space
at java.util.HashMap.newNode(HashMap.java:1734)
at java.util.HashMap.putVal(HashMap.java:630)
at java.util.HashMap.putMapEntries(HashMap.java:514)
at java.util.HashMap.<init>(HashMap.java:489)
at org.apache.activemq.broker.region.AbstractRegion.getDestinationMap(AbstractRegion.java:223)
at org.apache.activemq.broker.region.RegionBroker.getDestinationMap(RegionBroker.java:134)
at org.apache.activemq.broker.region.RegionBroker.purgeInactiveDestinations(RegionBroker.java:922)
at org.apache.activemq.broker.region.RegionBroker$1.run(RegionBroker.java:110)
at org.apache.activemq.thread.SchedulerTimerTask.run(SchedulerTimerTask.java:33)
at java.util.TimerThread.mainLoop(Timer.java:555)
at java.util.TimerThread.run(Timer.java:505)

A Java Heap Size error occurs because of certain memory intensive actions (like Data Exporting). The new Spring managed API uses a lot of memory to cache the xml beans. You may need to increase the default memory allocation.

Solution

According to this wiki page, the solution was to adjust the memory settings. Depending on your OS, I was using Ubuntu 16.04.1 LTS.

vim /etc/default/tomcat7

Then change the JAVA_OPTS variable to:

JAVA_OPTS=”-Djava.awt.headless=true -Xmx1024M -Xms1024M -XX:PermSize=256m -XX:MaxPermSize=256m -XX:NewSize=128m”

You can adjust the memory as per your needs, in my case this was multiplying everything by 2:

JAVA_OPTS=”-Djava.awt.headless=true -Xmx2048M -Xms2048M -XX:PermSize=512m -XX:MaxPermSize=512m -XX:NewSize=256m”

Resources

Troubleshooting Memory Errors

Java heap space errors when starting a new installation

Installing solr 4.10 on Tomcat 7

Problem

Beginning with Solr 5.0, Support for deploying Solr as a WAR in servlet containers like Tomcat is no longer supported. And I needed a way to deploy solr as a WAR. So I downloaded solr.4.10.4 from apache lucene

Solution

Assuming you’ve installed Oracle Java/openjdk and Tomcat7 on Ubuntu 14.04 then:

Install Solr:

# wget ftp://&amp;amp;lt;download-mirror&amp;amp;gt;/mirrors/ftp.apache.org/lucene/solr/4.10.4/solr-4.10.4.tgz
# tar xzvf solr-4.10.4.tgz
# mv solr-4.10.4 /usr/share/solr
# mkdir -p /usr/share/solr/live/solr
# cp /usr/share/solr/example/webapps/solr.war /usr/share/solr/live/solr/solr.war
# chown -R tomcat7:tomcat7 /usr/share/solr

Configure logging:

# cp /usr/share/solr/example/lib/ext/* /usr/share/tomcat7/lib
# cp /usr/share/solr/example/resources/log4j.properties /usr/share/tomcat7/lib

Edit /usr/share/tomcat7/lib/log4j.properties and set solr.log=logs/ to solr.log=/var/log/solr. Next create the log directory and set the proper permissions:

# mkdir /var/log/solr
# chown tomcat7:tomcat7 /var/log/solr

Add solr to logrotate.Create a file “/etc/logrotate.d/solr” with this content:

/var/log/solr/solr.log {
 copytruncate
 daily
 rotate 5
 compress
 missingok
 create 640 tomcat7 tomcat7
}

Add Solr to Tomcat7

To deploy Solr to Tomcat, create a file named “/etc/tomcat7/Catalina/localhost/solr.xml” with the folowing content:

<Context docBase="/usr/share/solr/live/solr/solr.war" debug="0" crossContext="true">
     <Environment name="solr/home" 
     type="java.lang.String" 
     value="/usr/share/solr/live/solr"
     override="true" /> 
</Context> 

Copy over the Solr configuration file from the examples.

cp /usr/share/solr/example/solr/solr.xml /usr/share/solr/live/solr/solr.xml

After restarting Tomcat you should already see a running Solr instance in the Tomcat application manager e.g http://localhost:8080/manager then click link to /solar to access the solr dashboard.

References
Solr 4.10 and Tomcat 7 on Debian 7
Installing Solr
Running Solr
How To Install Solr 5.2.1 on Ubuntu 14.04
Solr with Apache Tomcat
How to Install Apache Solr with Tomcat on Linux
Sol 5.0 and Above Why No War
Taking Solr to Production
Solr with Apache Tomcat