Copy Multiple lines from a logfile

Problem

I needed to capture an error / stack trace (list of the method calls that the application was in the middle of when an Exception was thrown – explanation from stackoverflow) from a log file to another file for attaching to a bug tracker

Solution

Used this answer from unix.stackexchange that proposed using a shell function that used sed that you could enter the from,to line numbers, log file and output file as shown

In the user’s .bashrc file add:

$ tail .bashrc 
#Printing section of a Line From A logFile
printLine () 
{ 
 sed -n -e "$1p" "$2"
}

Useage:

$ printLine 2000,2250 /var/log/tomcat/catalina.out > java_error.txt

Sources

Easy way to copy lines from one file to another

What is a stack trace, and how can I use it to debug my application errors?

Installing OpenMRS Platform 1.x and 2.x with OpenMRS Reference Application 2.x Modules on Ubuntu 14.04 LTS

Requirements

  1. MySQL Server – Tested using 5.5 and 5.6 (MySQL 5.7 needs to be tuned for it to work with OpenMRS. For instance, check this link and this one as well this one )
  2. Oracle Java JDK. Most version of OpenMRS support  version 1.6 and version 1.7. Here’s how to install Oracle Java.
  3. Tomcat –  Tested using Tomcat 6 and 7 that are available in the Ubuntu 14.04 repository. Tomcat 7 on Ubuntu 16.04 seemed to have issues and this was documented here.
  4. OpenMRS Platform – Tested using WAR for Enterprise version 1.11.4 with 2.3 Modules. New versions of OpenMRS Platform and Modules keep being released. You can find these on this OpenMRS download page.
  • Download Platform 1.11.6 WAR (for Enterprise) Version 1.11.6 should be the last of the 1.11.x version. The new Platform 2.0.0 is significantly different from the 1.11.x platform as can be seen from the documentation here. The rest this documentation assumes you are installing at most Platform version 1.11.6

Installation Process

Start by Installing MySQL then Java and then Tomcat7.

  • From the command line to install MySQL server. You will be asked to enter a password for the root user at some point during the installation:
root@whiscardz ~ # apt-get install mysql-server 
root@whiscardz ~ # apt-get install tomcat7 
  • Open a Linux Command Prompt
  • Check if Tomcat7 is running, then stop it:
whiscardz ~ # ps aux | grep tomcat
root 7786 194 3.2 4169316 528144 pts/0 Sl 08:18 0:17 /usr/bin/java -Djava.util.logging.config.file=/opt/tomcat7/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.awt.headless=true -Xmx2048M -Xms2048M -XX:PermSize=512m -XX:MaxPermSize=512m -XX:NewSize=256m -XX:+UseConcMarkSweepGC -XX:+HeapDumpOnOutOfMemoryError -Djdk.tls.ephemeralDHKeySize=2048 -Djava.endorsed.dirs=/opt/tomcat7/endorsed -classpath /opt/tomcat7/bin/bootstrap.jar:/opt/tomcat7/bin/tomcat-juli.jar -Dcatalina.base=/opt/tomcat7 -Dcatalina.home=/opt/tomcat7 -Djava.io.tmpdir=/opt/tomcat7/temp org.apache.catalina.startup.Bootstrap start
root 7808 0.0 0.0 15444 928 pts/0 S+ 08:18 0:00 grep --color=auto tomcat
whiscardz ~ # service tomcat7 stop
whiscardz ~ # ps aux | grep tomcat
root 7773 0.0 0.0 15444 1084 pts/0 S+ 08:18 0:00 grep --color=auto tomcatChange directory (cd) to where you download the openmrs.war file
  • Copy the openmrs.war file to the webapps directory (/var/lib/tomcat7/webapps)
 # root@whiscardz ~# cd /ToThe/LocationOfThe/OpenMRSWarFile/
root@whiscardz # ls
openmrs.war
root@whiscardz # cp openmrs.war /var/lib/tomcat7/webapps/ 

==> Change the owner and group of the openmrs.war file to the tomcat7 user and group otherwise it wont be deployed:

root@whiscardz # cd /var/lib/tomcat7/webapps
webapps # chown tomcat7:tomcat7 openmrs.war 
  • Create a directory called .OpenMRS in /usr/share/tomcat and then change the owner of the directory to the tomcat7 user and group as shown:
root@whiscardz # mkdir /usr/share/tomcat7/.OpenMRS
root@whiscardz # chown -R tomcat7:tomcat7 /usr/share/tomcat7/.OpenMRS 

NB: The above was to fix an issue that was documented in Start Using OpenMRS

  • Then Create a database and database user. You will need this during the install process.
 root@whiscardz ~ # mysql -uroot -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 6

Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> create database openmrs_demo;
Query OK, 1 row affected (0.01 sec)

mysql> grant all privileges on openmrs_demo.* to 'openmrs_user'@'localhost' identified by 'openmrsuserpassword'; 

From the above, the openmrs username  will be openmrs_user and password will be openmrsuserpassword

You should change the username and password above to whatever would suite you

  • Now you need to change the assign more RAM to tomcat.
  • On Ubuntu 14.04, if you have installed tomcat from Ubuntu repository i.e using the command (apt-get install tomcat7), its recommended to change configurations for tomcat config file:/etc/default/tomcat7 using a command line file editor like vi so, you’ll open the file as follows:
  • vi /etc/default/tomcat7
    And then changing the JAVA_OPTS variable to the below:
    JAVA_OPTS=”-Djava.awt.headless=true -Xmx1024M -Xms1024M -XX:PermSize=256m -XX:MaxPermSize=256m -XX:NewSize=128m”
  • Then save and exit the file.
  • The Reference on the above issue can be found in this post Troubleshooting Memory Errors Especially Steve’s answer in the comments section
  • Now you are ready to start Tomcat
whiscardz ~ # service tomcat7 start
  • Then Go to your Browser (Use Google Chrome or Mozilla Firefox) and enter the URL http://localhost:8080/openmrs/
  • You will be presented with a wizard to setup OpenMRS. The Tutorial below contains the steps you’ll need to complete the installation of OpenMRS.
  • installing-openmrs-wizards
  • Once you are done. Then you will be directed to your local OpenMRS login page.
  • To add modules to your local instance, download the OpenMRS 2.3 Modules & Data
  • Then unzip the download and in the appdata folder pick the modules i.e. the *.omod files and copy them to /var/lib/OpenMRS/modules/ Remember to change the owner and group of the *.omod file to tomcat otherwise tomcat will be unable to load the modules due to permissions errors which you will see in the catalina.out log file.
  • NOTE: Tomcat looks for the openmrs-runtime.properties in /var/lib/tomcat7/ yet the openmrs wizard created it in /usr/share/tomcat7/.OpenMRS/. Therefore I created a symlink:
# cd /var/lib/tomcat7/
# ln -s /usr/share/tomcat7/.OpenMRS/openmrs-runtime.properties

Note that the reason for the above issue can be explained by Additional Notes at the end of this article:

  • Upon restarting Tomcat you may notice errors like the below, meaning additional modules are needed.

ERROR – ModuleFactory.startModules(236) |2015-11-03 16:02:36,352| Module Form Entry App Module cannot be started because it requires the following module(s): org.openmrs.module.appframework 2.2.1 Please install and start these modules first.

10. For my case, I Downloaded the following modules from OpenMRS Modules and copied them to /var/lib/OpenMRS/modules.

Remember to change the owner of the above to tomcat and restart tomcat.

To Re-deploy OpenMRS afresh after a failed trial then:

  • Remember to delete the folder created in webapps and also the openmrs folder in /var/lib/OpenMRS/ and finally (/var/lib/tomcat7/work/Catalina/localhost/)

TO SORT Memory Issue

Incase you see the following error in the catalina logs, then proceed with the instructions below:

Java HotSpot(TM) 64-Bit Server VM warning: Exception java.lang.OutOfMemoryError occurred dispatching signal SIGTERM to handler- the VM may need to be forcibly terminate
d

  • In-case you notice the error/warnings in /var/log/tomcat7/catalina.out:

WARNING: Problem with directory [/usr/share/tomcat8/common/classes],
exists: [false], isDirectory: [false], canRead: [false]

  • The issue is Tomcat is looking for the folders below in the CATALINA_HOME(usr/share/tomcat7/) yet they are in the CATALINA_BASE(/var/lib/tomcat7)
  • Solution was to create a sym link to the folders:
sim-specialist tomcat7 # cd /usr/share/tomcat7
sim-specialist tomcat7 # ln -s /var/lib/tomcat7/common
sim-specialist tomcat7 # ln -s /var/lib/tomcat7/server
sim-specialist tomcat7 # ln -s /var/lib/tomcat7/shared 

NB: References on the above issue is documented here

Additional Notes

Tomcat defines the Java Runtime Environment – JAVA HOME Directory in the /etc/init.d/tomcat file. Apparently it uses the Catalina Base directory that’s set there in.

Snippet Below:

# Directory where the Tomcat 6 binary distribution resides
CATALINA_HOME=/usr/share/$NAME

# Directory for per-instance configuration files and webapps
CATALINA_BASE=/var/lib/$NAME

In my case it was /var/lib/tomcat7, therefore when starting up Tomcat it looks for the OpenMRS openmrs-runtime.properties file in the directorty specifically to connect to the DB.

The Install Wizard creates openmrs files like openmrs-runtime.properties and folders like modules in /var/lib/OpenMRS (As per the above, I assume this is /var/lib/$NAME where the wizard specifies the name as OpenMRS).

Since Tomcat looks for the properties in /var/lib/tomcat7, I created a sym link in /var/lib/tomcat7 for the openmrs-properties-runtime file.

Update 2017-04-24

Installed the New Platform 2.x:

Requirements:

Oracle Java 8 or openjdk 8

Instructions:

  • Uninstall oracle java 7:
implementer@whiscardz ~ $ java -version
java version "1.7.0_80"
Java(TM) SE Runtime Environment (build 1.7.0_80-b15)
Java HotSpot(TM) 64-Bit Server VM (build 24.80-b11, mixed mode)
implementer@whiscardz ~ $ sudo apt-get purge jre
Reading package lists... Done
Building dependency tree
Reading state information... Done
Package 'jre' is not installed, so not removed
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
implementer@whiscardz ~ $ sudo apt-get purge oracle-java7-installer
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
ca-certificates-java default-jre-headless openjdk-8-jre-headless
Suggested packages:
default-jre openjdk-8-jre-jamvm fonts-ipafont-gothic fonts-ipafont-mincho
The following packages will be REMOVED:
oracle-java7-installer*
The following NEW packages will be installed:
ca-certificates-java default-jre-headless openjdk-8-jre-headless
  • Uninstalling oracle java 7, openjdk 8 will be installed. You can remove it before installing oracle java 8:
implementer@whiscardz ~ $ java -version
openjdk version "1.8.0_121"
OpenJDK Runtime Environment (build 1.8.0_121-8u121-b13-0ubuntu1.16.04.2-b13)
OpenJDK 64-Bit Server VM (build 25.121-b13, mixed mode)
implementer@whiscardz ~ $ sudo apt-get purge openjdk-\*
Reading package lists... Done
Building dependency tree
Reading state information... Done
Note, selecting 'openjdk-9-jre-headless' for glob 'openjdk-*'
Note, selecting 'openjdk-8-jdk' for glob 'openjdk-*'
implementer@whiscardz ~ $ sudo apt-get install oracle-java8-installer
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages were automatically installed and are no longer required:
junit libaopalliance-java libapache-pom-java libasm4-java libatinject-jsr330-api-java libbsh-java libcdi-api-java libcglib3-java libclassworlds-java
libcommons-cli-java libcommons-codec-java libcommons-httpclient-java libcommons-io-java libcommons-lang-java libcommons-lang3-java libcommons-logging-java
libcommons-net-java libcommons-net2-java libcommons-parent-java libdoxia-core-java libeclipse-aether-java libgeronimo-interceptor-3.0-spec-java libguava-java
libguice-java libhamcrest-java libhttpclient-java libhttpcore-java libjaxen-java libjaxp1.3-java libjdom1-java libjetty-java libjsch-java libjsoup-java
libjsr305-java liblog4j1.2-java libmaven-parent-java libmaven2-core-java libobjenesis-java libplexus-ant-factory-java libplexus-archiver-java
libplexus-bsh-factory-java libplexus-cipher-java libplexus-classworlds-java libplexus-classworlds2-java libplexus-cli-java libplexus-component-annotations-java
libplexus-component-metadata-java libplexus-container-default-java libplexus-container-default1.5-java libplexus-containers-java libplexus-containers1.5-java
libplexus-interactivity-api-java libplexus-interpolation-java libplexus-io-java libplexus-sec-dispatcher-java libplexus-utils-java libplexus-utils2-java
libqdox2-java libservlet2.5-java libsisu-inject-java libsisu-plexus-java libslf4j-java libwagon-java libxalan2-java libxbean-java libxerces2-java
libxml-commons-external-java libxml-commons-resolver1.1-java
Use 'sudo apt autoremove' to remove them.
Suggested packages:
visualvm ttf-baekmuk | ttf-unfonts | ttf-unfonts-core ttf-kochi-gothic | ttf-sazanami-gothic ttf-kochi-mincho | ttf-sazanami-mincho ttf-arphic-uming
Recommended packages:
gsfonts-x11 oracle-java8-set-default
The following NEW packages will be installed:
oracle-java8-installer
  • Install oracle-java8-set-default
update-alternatives: using /usr/lib/jvm/java-8-oracle/bin/xjc to provide /usr/bin/xjc (xjc) in auto mode
update-binfmts: warning: current package is oracle-java8, but binary format already installed by openjdk-8
update-alternatives: using /usr/lib/jvm/java-8-oracle/jre/lib/amd64/libnpjp2.so to provide /usr/lib/mozilla/plugins/libjavaplugin.so (mozilla-javaplugin.so) in auto mod
e
Oracle JRE 8 browser plugin installed
Oracle JDK 8 installed

#####Important########
To set Oracle JDK8 as default, install the "oracle-java8-set-default" package.
E.g.: sudo apt install oracle-java8-set-default
On Ubuntu systems, oracle-java8-set-default is most probably installed
automatically with this package.
######################

implementer@whiscardz ~ $ sudo apt-get install oracle-java8-set-default
  • Then remove all the packages that are no longer needed:
implementer@whiscardz ~ $ sudo apt autoremove
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages will be REMOVED:
junit libaopalliance-java libapache-pom-java libasm4-java libatinject-jsr330-api-java libbsh-java libcdi-api-java libcglib3-java libclassworlds-java
libcommons-cli-java libcommons-codec-java libcommons-httpclient-java libcommons-io-java libcommons-lang-java libcommons-lang3-java libcommons-logging-java
libcommons-net-java libcommons-net2-java libcommons-parent-java libdoxia-core-java libeclipse-aether-java libgeronimo-interceptor-3.0-spec-java libguava-java
libguice-java libhamcrest-java libhttpclient-java libhttpcore-java libjaxen-java libjaxp1.3-java libjdom1-java libjetty-java libjsch-java libjsoup-java
libjsr305-java liblog4j1.2-java libmaven-parent-java libmaven2-core-java libobjenesis-java libplexus-ant-factory-java libplexus-archiver-java
libplexus-bsh-factory-java libplexus-cipher-java libplexus-classworlds-java libplexus-classworlds2-java libplexus-cli-java libplexus-component-annotations-java
libplexus-component-metadata-java libplexus-container-default-java libplexus-container-default1.5-java libplexus-containers-java libplexus-containers1.5-java
libplexus-interactivity-api-java libplexus-interpolation-java libplexus-io-java libplexus-sec-dispatcher-java libplexus-utils-java libplexus-utils2-java
libqdox2-java libservlet2.5-java libsisu-inject-java libsisu-plexus-java libslf4j-java libwagon-java libxalan2-java libxbean-java libxerces2-java
libxml-commons-external-java libxml-commons-resolver1.1-java
0 upgraded, 0 newly installed, 68 to remove and 0 not upgraded.
After this operation, 28.7 MB disk space will be freed.
Do you want to continue? [Y/n] Y
(Reading database ... 799808 files and directories currently installed.)...............................
  • Next install tomcat8:
implementer@whiscardz ~ $ sudo apt-get install tomcat8
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
tomcat8-common
Suggested packages:
libtcnative-1 tomcat8-admin tomcat8-docs tomcat8-examples tomcat8-user
Recommended packages:
authbind
The following NEW packages will be installed:
tomcat8 tomcat8-common
0 upgraded, 2 newly installed, 0 to remove and 0 not upgraded.
implementer@whiscardz /etc/default $ sudo cp tomcat8 tomcat8.bak
implementer@whiscardz /etc/default $ sudo vi tomcat8
implementer@whiscardz /etc/default $ sudo diff tomcat8 tomcat8.bak
21c21
< JAVA_OPTS="-Djava.awt.headless=true -Xmx1024M -Xms1024M -XX:NewSize=128m -XX:+UseConcMarkSweepGC"
---
> JAVA_OPTS="-Djava.awt.headless=true -Xmx128m -XX:+UseConcMarkSweepGC"
  • Copy the openmrs.war file to the webapps folder and then change the owner and group to tomcat8:
implementer@whiscardz /var/lib/tomcat8/webapps $ sudo cp ~/openmrs/platform2.x/2.0.5/openmrs.war .
implementer@whiscardz /var/lib/tomcat8/webapps $ ll
total 69M
drwxrwxr-x 3 tomcat8 tomcat8 4.0K Apr 24 09:35 ./
drwxr-xr-x 4 root root 4.0K Apr 24 09:04 ../
-rw-r--r-- 1 root root 69M Apr 24 09:35 openmrs.war
drwxr-xr-x 3 root root 4.0K Apr 24 09:04 ROOT/
implementer@whiscardz /var/lib/tomcat8/webapps $ sudo chown -R tomcat8:tomcat8 openmrs.war
  • Next cd to /usr/share/tomcat8 and create the .OpenMRS directory. Also create the server, commons and shared directories. Then change the owner and group of this folder /usr/share/tomcat8.
implementer@whiscardz /usr/share/tomcat8 $ sudo mkdir .OpenMRS/
implementer@whiscardz /usr/share/tomcat8 $ sudo mkdir -p common/classes
implementer@whiscardz /usr/share/tomcat8 $ sudo mkdir -p server/classes
implementer@whiscardz /usr/share/tomcat8 $ sudo mkdir -p shared/classes
implementer@whiscardz /usr/share/tomcat8 $ ls
bin common defaults.md5sum defaults.template lib logrotate.md5sum logrotate.template server shared
implementer@whiscardz /usr/share/tomcat8 $ sudo chown -R tomcat8:tomcat8 ../tomcat8/
implementer@whiscardz /usr/share/tomcat8 $ sudo systemctl start tomcat8.service
  • Go to your browser, localhost:8080 and then install openmrs. You should see this page after going through all the wizard options:
Final OpenMRS Installation Wizard Screen
Final OpenMRS Installation Wizard Screen
  • Then after installation you should see the following page before installation of the modules:
Platform 2.x successful installation screen
Platform 2.x successful installation screen

Additional References:

TroubleshootingTomcat-Generaltroubleshooting 

tomcat-on-ubuntu-14.04-install

Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize