Using Local YUM repository on CentOS 7

Problem

I have the CentOS 7 complete DVD ISO and need to install packages from this ISO instead of via the internet.

Solutions

Only allow installations via DVD Rom:

  • Mount the DVD ROM as described here.
  • Create a Local Repository File called local.repo as below:
# vi /etc/yum.repos.d/local.repo

Add the following lines to the file:

Local Repository File Details
Local Repo File Details

NB:// gpgcheck is optional but you are advised to use it. To use it set value to 1 and add as shown below:

gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7

Also baseurl depends on where your cd/dvd is mounted in my case its as above.

  • Remove (# rm *.repo) the other .repo file(s) (all except local.repo) in the location /etc/yum.repos.d/
  • Then clear the repository cache and update using:
#yum clean all
#yum update
  • Test installing a package:
yum install screen
Testing Local Repository Installation

Disabling Other Repos and Only Using LocalRepo

    • Add the local.repo file to /etc/yum.repos.d/ with the details described above or as below:
Local Repository File
Local Repository File
    • Then clear the repository cache and then update using
#yum clean all
#yum update
#yum install --disablerepo="*" --enablerepo="LocalRepo" vim

NB:// The Name LocalRepo is case sensitive so it should be the same name as the name in the header [] tag in your local.repo file in /etc/yum.repos.d

yum install --disablerepo=
Installing Package from Local Repository

Resources

Setup Local Yum Repository On CentOS 7

Create Local YUM repository on CentOS 7 / RHEL 7 using DVD

Creating a Local Yum Repository Using an ISO Image

How To Enable EPEL Repository in RHEL/CentOS 7/6/5?

Installing Guest Additions on Linux Host

Setup

  • Download the Guest Additions ISO from here. You will need to select the Version you want from the list as suggested here.
  • On the Linux host, in my case an ubuntu based distro:
On Host OS install dkms
Virtual Box Host OS
  • On Guest OS, in my case CentOS 7 minimal install. You will need to install bzip2, gcc, kernel-devel:
# yum update
# yum install bzip2
# yum install gcc
# yum install kernel-devel
cd to /media/cdrom --> sh ./VBoxLinuxAdditions.run
Installing Guest Additions
Guest Additions Installation Progress
Guest Additions Installation Progress

Resources

Guest Additions for Linux

VirtualBox Guest Additions ISO Download?

Index of /virtualbox

Installing the VirtualBox Guest Additions on Windows Host

Switching Between Virtual Box ISO on Linux Host

Problem

Needed to switch between mounting 2 ISOs on a Centos Guest in Virtual Box without having to restart.

Solution

  • Umount drive from guest OS:
    #umount /media/cdrom
    
    
  • From the booted CentOS guest select “Devices” then:

Devices->CD/DVD Devices -> Remove Disk From Virtual Drives

  • Incase of Error Below, force unmount:
Click Force Umount Button
Umount Virtual Drive
  • Double Check that Drive has been unmounted:
Confirm Virtual Drive Umounted
Unmounted Virtual Drive
  • Mount New Image, From the Guest OS, select Devices, then:
Devices-->CD/DVD Devices-->
Mounting New Virtual Drive
  • From the terminal window on the guest OS, mount the New ISO in this case Guest Additions as described here and also shown below:
mount cd/dvd drive on guest OS
Run the above on Guest OS

Resources

How to Mount and Unmount an ISO Image in RHEL/CentOS/Fedora and Ubuntu

How to Mount CDROM Media on CentOS 5/CentOS 6/CentOS 7

mount DVD ROM on CentOS guest on VM

How TO

  1. Click on the VM and select Devices —> CD/DVD Devices —> ISO image file
  2. Log in to the centos box and check if cd/dvd rom is mounted
  3. #df -h
  4. If not mounted then check the device in /dev using:
  5. #ls /dev -l | grep dvd
    #ls /dev -l | grep cd
  6. /dev/sr0 is what you are looking for, now you can mount the dvd/cdrom. Create the mount point:
  7. #mkdir /media/cdrom/
  8. Mount the drive:
  9. #mount -r -t iso9660 /dev/sr0 /media/cdrom
  10. check if its mounted, df -h

Resources

Mount DVD on CentOS

ifconfig on CentOS 7

Problem

Need to use ifconfig on CentOS 7. Apparently this has been depricated and one if advised to use ip addr sh:

#ip addr sh

Solution

Incase you still need to use ifconfig then:

Use the “yum provides” command to figure out what packages provide certain commands or files you’re used to using.

#yum provides ifconfig

Install nettools and you’ll get the ifconfig package.

#yum -y install net-tools

Resources
No ifconfig in CentOS 7 Minimal Install
How to Install ifconfig and netstat on RHEL 7.0/CentOS 7.0/Oracle Linux 7

MySQL Replication Best Practise

Master accepts read/write connections from applications to change data in the DB
Slave connects to the master and replicates all the changes to itself

Master bin logs contain the changes done in the master DB.
Relay Logs on the slave are a copy of the master’s bin file.

Bin Log Types or bin log format (variable in config file is binlog_format and can be binlog_format = STATEMENT/MIXED/ROW.
Statement format has all the queries executed on the master
Row format contains each event ran on the master
Bin Logs can be read by MySQLbinlog application.
When binlog_format=MIXED then for instance when a non-deterministic statement (i.e. a statement that may give different results when ran in the master and on the slave) is ran on the master, MySQL defaults to the ROW edit in the bin log.

Threads
Master Thread. It has Bin log dumps reads content bin log and pulls it to the master

IO Thread connects to master and gets data from bin logs and creates relay logs and maintains the coordinates (Copy data from Master To Slave)
SQL Thread Reads contents on the slave relay logs and applies then in the DB ( Applies changes from bin logs to Slave DB)

Replication Variable in my.cnf file
Master

log_bin=basename --- Use this to set the name of the bin logs. This is useful if you have many servers. Default is host_name.bin.XXX
binlog_rows_query_log_events -- only used in version >= 5.6 Makes bin logs more verbose if format = ROW

Slave

log_bin --- its recommended to have bin logs on slave for backups/additional slaves
log-slave-updates --- Tells MySQL all events done on the slave by the SQL thread mentioned above. If disabled, MySQL only writes events where a connection is made to the slave and write operation done. Consequently bin logs will be small if this is not enabled on slave.
relay_log is same as log bin.
expire_log_days -- How long do you want to retain bin logs. Best practise is setup depending on length of days of backups e.g if you have weekly backups, expire logs after 7 days.

Filtering binlogs

replicate-do-db=dbname -- filter dbs that you dont want replicated on the slave and not on the master.

Create Crash-safe slaves Only available in >= MySQL 5.6
This puts the data in the master.info and relay.info files into innodb tables. This helps us survive server crashes i.e. file system issues coz innodb tables are quite crash resilient. To do this on the slave:

master-info-repository=TABLE -- stores info about master binlog files and position into a table ( replaces master.info ) file
relay-log-info-repository=TABLE -- stores info about relay log files and position into a table (replaces relay.info) log file
relay-log-recovery=1 -- enable automatic recovery of relay log files after crash, Checks events that were ran before crash and if need be redownload master binlogs

read-only -- Set to read only so that there are no writes on the slave ( you can still ran reporting e.g selects)
skip-slave-start --- Avoid automatic start of slave incase slave crash and server restart

Creating a slave from a busy server
Use xtrabackup coz it wont lock tables when dumping
Restore data on slave then change master command to tell the slave the credential, ip, binlog and position of binlog to start with. That is it tells the slave how to start replicating.

Replication Monitoring
You can use:
1. show slave status and check for
Slave_IO_Running: Yes
2. Slave_SQL_Runnning: Yes
3. Seconds_Behind_Master: 13 — Replication works with a single thread so events are done serially therefore slave will lag behind master that is concurrent

2. Using pt-hearbeat
Creates and stores records in the master using timestamp based values and check these values in the slave to check how long it takes to reach slave

pt-heartbeat -D test --update -h master-server -- daemonize
pt-heartbeat -D test --monitor -h slave-server (Check replication)

You can also use Nagios, Cacti, Munin

Check Data consistency
Run checksums regularly e.g

pt-table-checksum --replicate=test.cksums --create-replicate-table
Checks checksums on the table on master and slave if there is a difference it'll alert you.

Syncing Data
pt-table-sync
Uses info provided by pt-table-checksum. It reruns the above to check which row has difference.
Use –print and –verbose to check the data it may change before executing the changes using –execute

GTID Overview
Its Global Transaction Identifier. Its a good way to identify a single transaction on a cluster or replication environment, GTIDs are stored in the binlog. It ensures transactions belongs to only 1 server and ensures transactions are applied once in a replication stream e.g having master master setup so you avoid transactions being re-replicated. GTID contains UUID is the server id and transaction id.
Allows slaves to be promoted to masters

Deployment
1. STOP SLAVE;
2. stop both servers i.e master and slave
3. in my.cnf on master and slave, add the below where applicable

skip-slave-start
gtid_mode=ON
enforce_gtid_consistency
log-bin
log-slave-updates

4. Reset slave to use GTID based positions:

CHANGE MASTER TO MASTER_HOST='master_ip',MASTER_USER='repl',MASTER_PASSWORD='password',MASTER_AUTO_POSITION=1;

NB: MASTER_AUTO_POSITION allows us to stop using binary log position. Slave autodiscovers where to start replicating
5. START SLAVE;

Check out GTID Online Implementation

References

MySQL Replication Best Practices

Percona Nagios check replication delay quick setup

Installing Percona Server on Debian and Ubuntu

MySql Replication – slave lagging behind master

Benchmarking MySQL Replication with Multi-Threaded Slaves

MySQL Replication

Problem

Need to setup simple MySQL replication with 1 Master and 1 Slave.

Prerequisites

MySQL Server installed on both Master and Slave (This was done with Version 5.5.44)

Linux command line (This was done with Ubuntu Based Distro)

Access to root user on server as well as MySQL

Make Sure on the Server on Slave Machines, the Following Lines are either commented out or have an IP:

#bind-address = 127.0.0.1

OR

bind-address = 192.168.0.2

Setup

1. Edit the my.cnf config file on the master. Under the [mysqld] section, Uncomment the following:

server-id = 1
log_bin = /var/log/mysql/mysql-bin.log
expire_logs_days = 2
max_binlog_size = 100M

NOTE. The server-id should be unique. In some ways it can be likened to an IP address. Omitting it defaults the id to 0 and the master will consequently refuse all slave connections.

After this restart MySQL.

service mysql stop
service mysql start

2. Edit the my.cnf config file on the slave. Under the [mysqld] section, Uncomment the following:

server-id = 2
relay-log               = /var/log/mysql/mysql-relay-bin.log

Note that the relay log is supposed to prevent replication from failing due to a hostname change.

After this restart MySQL.

service mysql stop
service mysql start

3. On the Master Create a User that the slave will use to connect to the master for replication.

mysql> CREATE USER 'replication'@'%' IDENTIFIED BY 'replicaiton';
mysql> GRANT REPLICATION SLAVE ON *.* TO 'replication'@'%.';

4. Dump the database(s) on the master and copy them over to the slave. I’m assuming this is not in a busy environment so probably MySQL server has been stopped. If not you’d probably need to lock the tables ( to stop write operations )

mysql> FLUSH TABLES WITH READ LOCK;
mysqldump -uroot -p support > testDB.sql (On the Master) 
mysql-uroot -p < testDB.sql (On the Slave)

Note, depending on your environment you may need to include mysqldump options such as –master-data etc.

5. Connect to the MySQL prompt on the Master Machine and do the following:

mysql> show master status;
+------------------+----------+--------------+------------------+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB |
+------------------+----------+--------------+------------------+
| mysql-bin.000001 | 107 | | |
+------------------+----------+--------------+------------------+

The above shows the bin log and position which the slave will start reading from the master’s bin log.

NOTEThe File column shows the name of the log file and Position shows the position within the file. In this example, the binary log file is mysql-bin.000003 and the position is 73. Record these values. You need them later when you are setting up the slave. They represent the replication coordinates at which the slave should begin processing new updates from the master.

NOTEIf the master has been running previously without binary logging enabled, the log file name and position values displayed by SHOW MASTER STATUS or mysqldump –master-data will be empty. In that case, the values that you need to use later when specifying the slave’s log file and position are the empty string ('') and 4.

5. Connect to the MySQL prompt on the slave machine. For the slave to communicate with the master, you need to supply the connection information by issuing the commands:

mysql> stop slave;
Query OK, 0 rows affected, 1 warning (0.00 sec)

mysql> CHANGE MASTER TO MASTER_HOST='192.168.0.2', MASTER_USER='replication', MASTER_PASSWORD='replication', MASTER_PORT=3306,MASTER_LOG_FILE='mysql-bin.000001', MASTER_LOG_POS=338, MASTER_CONNECT_RETRY=10;
Query OK, 0 rows affected (0.27 sec)

mysql> start slave;
Query OK, 0 rows affected (0.00 sec)

6. Unlock the table on the master if you had locked then. MySQL replication should start on the slave. Test by probably inserting a new record on one of the databases on the master and check the slave to see if the same was replicated.

NOTE: Incase you experience a error especially in step 5 and need to start afresh then you will need to:

mysql> stop slave;
Query OK, 0 rows affected (0.09 sec)

mysql> reset slave all

Query OK, 0 rows affected (0.14 sec)

RESET SLAVE makes the slave forget its replication position in the master’s binary log. This statement is meant to be used for a clean start: It deletes the master.info and relay-log.info files, all the relay log files, and starts a new relay log file. This can also be used if you want to change the slave to a master.

Incase you are tempted to filter databases to replicate or not replicate then read this Why MySQL’s binlog-do-db option is dangerous.

Also check out How Servers Evaluate Replication Filtering Rules and Replication Slave Options and Variables

Also before you start, check out Evaluation of Database-Level Replication and Binary Logging Options

References

MySQL MASTER and SLAVE CONFIGURATION ON LINUX (Video)

How to Set Up Replication (PLEASE START HERE)

Setting Up Replication with Existing Data

Replication Slave Options and Variables

Evaluation of Database-Level Replication and Binary Logging Options

How To Set Up Database Replication In MySQL

How To Set Up Master Slave Replication in MySQL

How to add a new MySQL slave

MySQL master-slave and master-master replication. Step by step configuration instructions

Master Slave, Master Master Replication

How to create Replication on Mysql: Step by step

How do I completely disable MySQL replicationHow to change a MySQL previous slave to be a master and remove slave status information?

RESET SLAVE Syntax

Why MySQL’s binlog-do-db option is dangerous

How Servers Evaluate Replication Filtering Rules

Linux Change Default Route – Machine with Multiple Internet connections

Problem

I have an ubuntu based distro with 2 internet connections, wireless connections and 3G Dongle. The wireless connection is both LAN and Internet while 3G dongle is for backup internet.

So problem is the wireless LAN is the default route for out-going traffic / internet traffic. I don’t want to disconnect the wireless connection because I need the LAN resources and internet connection from the 3G dongle at the same time.

Solution

Switch the default route from the Wireless LAN to the 3G dongle for outgoing connections.

Implementation

As the route user:

Check the current route:

# ip route list
default via 192.168.0.1 dev wlan0 proto static
50.94.94.94 dev ppp0 proto kernel scope link src 305.531.67.991
192.168.0.0/24 dev wlan0 proto kernel scope link src 192.168.0.7 metric 9

Change the default route to the wireless lan:

# ip route replace default via 50.94.94.94 proto static

Check the default route:

# ip route list
default via 50.94.94.94 dev ppp0 proto static
50.94.94.94 dev ppp0 proto kernel scope link src 305.531.67.991
192.168.0.0/24 dev wlan0 proto kernel scope link src 192.168.0.7 metric 9

Go to the browser and confirm connectivity. Check also location connections.

References

How can I change the default gateway?

Routing issue on Linux with Mobile Broadband

10 Useful “IP” Commands to Configure Network Interfaces

Linux Set Up Routing with ip Command