Rsync and details on what has changed

January 13, 2017 Leave a comment

Setup 

# create a directoy and a couple of files 
mkdir /tmp/mydir1;
mkdir /tmp/mydir1/data;
echo "hello" > /tmp/mydir1/hello.txt;
echo "name is john" > /tmp/mydir1/name.txt;
mkdir -p /tmp/mydir1/dir1/dir2/dir3/dir4/dir5;

# copy the directory with all permissions etc to a new directory 
# -a : copy exactly with owner, group, permissions etc
# -r : recursive
cp -ar /tmp/mydir1 /tmp/mydir2

Some rsync options 

  • –dry-run : only do a simulation – don not perform the actual action
  • -v : verbose
  • -a : archive mode – this is equal to specifying all these options ( -r -l -p -t -g -o – D )
  • -r : recursive
  • -l : copy symlinks as symlinks
  • -p : preserve permissions
  • -t : preserve file timestamps
  • -g: preserve group
  • -o: preserve owner
  • -D: preserve device files and special files
  • -c : compare file checksum instead of timestamp and filesize
  • -i : format the output
  • –delete: delete any files in destination that are not in the source

First try – simple dry run 

# rsync --dry-run -avc --delete /tmp/mydir1/ /tmp/mydir2/
sending incremental file list

 

Nothing has changed as we have exactly copied the two directories.
Now lets make some changes to mydir2

echo "hello again" >> /tmp/mydir2/hello.txt;
touch /tmp/mydir2/newfile.txt;
chmod o+rwx /tmp/mydir2/dir1;
mkdir /tmp/mydir2/newdir;
chgrp nobody /tmp/mydir2/dir1/dir2;
chown nobody /tmp/mydir2/dir1/dir2/dir3;

Try again 

# rsync --dry-run -avc --delete /tmp/mydir1/ /tmp/mydir2/
sending incremental file list
./
deleting newdir/
deleting newfile.txt
hello.txt
dir1/
dir1/dir2/
dir1/dir2/dir3/

You will see what changes will be done in this list

To see more details use the  –itemize-changes( -i ) option.
This will tell in detail what attribute has changed

Second try – lets format the output 

# rsync --dry-run -avci --delete /tmp/mydir1/ /tmp/mydir2/
sending incremental file list
.d..t...... ./
*deleting newdir/
*deleting newfile.txt
>fcst...... hello.txt
.d...p..... dir1/
.d.....g... dir1/dir2/
.d....o.... dir1/dir2/dir3/

We can now see that some things have been appended to the list of files – explanation are ( see man rsync for more details on –itemize-changes )

  • (>) means file is being transferred
  • (c) means a change is happening or file is being created
  • (*deleting) means file will be deleted on destination
  • (p) means permission changed
  • (g) means group changed
  • (o) means owner changed

 

 

 

Categories: Linux Tags:

PHP Remote CLI Script Debugging with PHPStorm

February 9, 2016 Leave a comment

PHPStorm is one of the best IDE’s to develop in PHP. I recently came across tons complex php cli scripts and needed a way to debug them.

Follow this PHPStorm Docs post first to set up php storm and the server.

Most important things to configure are

  • Set up xdebug correctly on the remote server. Dont use xdebug.remote_connect_back. Instead use xdebug.remote_host
  • Xdebug must be set up for php cli – check with the command php -i | grep xdebug and you should see many entries
  • Set up deployment path mapping correctly in your project settings – a lot of people miss this and wonder why debug is not working
  • Check your firewalls on both machines, make sure required ports are open, especially port 9000
  • Check debugger settings in phpstorm and make sure you have break at first line set up

In the PHPStorm tutorial, it asks you to set up SSH tunnel. If you are not doing tunneling, you should set these environment variables on the remote server

Eg, if my remote server is centos, i will set these 2 variables

export XDEBUG_CONFIG="idekey=PHPSTORM";
export PHP_IDE_CONFIG="serverName=myDeploymentServerName";

serverName is the name of the server you set up for deployment in phpstorm deployment settings

these variables are valid for the session, if you logout and log in, you have to set them again.

You can add these to your .bashrc file in your home folder to make them permanent

In case you want to use xdebug.remote_connect_back, you might have to run your php scripts on the command line with additional args like this

php -dxdebug.remote_enable=1  -dxdebug.remote_host=10.0.1.2 -dxdebug.remote_connect_back=0 /path-to-php-script

 

Installing VirtualBox Guest Addition on CentOS 7 server – no GUI

January 19, 2016 3 comments

I am doing this on

  • VirtualBox 5.0.12
  • Windows 8.1 64 bit Host
  • CentOS 7 server 64 bit guest up to date

Steps

  • Start CentOS 7 guest
  • From the Devices Menu,go to Optical Drives and remove previous CD/DVD using the Remove Disk from VirtualDrive
  • Then click on Devices and select Insert Guest AdditionsCD Image.
  • This will put the Guest addition cd into /dev/cdrom in CentOS
  • SSH into CentOS and mount the cdrom with the command
mount /dev/cdrom /mnt
  • Install required libraries
sudo yum install bzip2 gcc kernel-devel dkms
  • install the guest addition, nox11 is to indicate that we dont have a GUI
bash /mnt/VBoxLinuxAdditions.run --nox11
  • It will install and finally give some messages like below
Verifying archive integrity... All good.
Uncompressing VirtualBox 5.0.12 Guest Additions for Linux............
VirtualBox Guest Additions installer
Removing installed version 5.0.12 of VirtualBox Guest Additions...
Removing existing VirtualBox DKMS kernel modules[ OK ]
Removing existing VirtualBox non-DKMS kernel modules[ OK ]
Copying additional installer modules ...
Installing additional modules ...
Removing existing VirtualBox DKMS kernel modules[ OK ]
Removing existing VirtualBox non-DKMS kernel modules[ OK ]
Building the VirtualBox Guest Additions kernel modules[ OK ]
Doing non-kernel setup of the Guest Additions[ OK ]
You should restart your guest to make sure the new modules are actually used
Installing the Window System drivers
Could not find the X.Org or XFree86 Window System, skipping.

  • Shutdown the CentOS VM and add shared folders and select Auto Mount
  • Start the CentOS VM and the shared folder should be available at /media on CentOS

 

Setting the default editor to nano Linux

January 14, 2016 Leave a comment

In CentOS, the default system editor is VI
If you want to edit the crontab with crontab -e command, the text editor that opens up is VI

In order to change the system wide default text editor to nano, edit /etc/bashrc and put the below line in it at the bottom

export EDITOR="nano"

Exit and login again for the changes to take effect

Categories: Linux Tags: , ,

Disabling SELinux on CentOS 7

January 14, 2016 Leave a comment

Security-Enhanced Linux (SE Linux) is a Linux kernel security module that provides a mechanism for supporting access control security policies. It controls which applications can access what directories in the system

For example, the default rules for apache is to only be allowed to access /var/www and /var/logs/httpd and some other configuration directories. If apache tries to access any other directory, then SELinux will not permit it if it is enabled.

Example, default web root for apache is /var/www, if you change it to /home/code, then SELinux will not allow apache to access files in /home/code and the application will fail to load on the web page

You have 2 options,

  • manually add the new location to SELinux apache rules by giving appropriate groups ( recommended )
  • disable SELinux permanently

Similary if you change the data directory for mysql, you will come across this issue

Sometimes you need a quick fix and might need to disable SELinux

This is not recommended on production systems. Do it on your own risk.

Command to check if SELinux is active is sestatus

[root@ip-172-30-0-220:/]$ sestatus
SELinux status: enabled
SELinuxfs mount: /sys/fs/selinux
SELinux root directory: /etc/selinux
Loaded policy name: targeted
Current mode: enforcing
Mode from config file: enforcing
Policy MLS status: enabled
Policy deny_unknown status: allowed
Max kernel policy version: 28

Current Mode is set to enforcing, which means SELinux is active

Temporarily Disabling SELinux

To temporarily disable SELinux, use the command

sudo setenforce 0

Then check with sestatus and Current Mode should be permissive. This will revert back on boot to enforcing

To enable SELinux again, use

setenforce 1

 

Permanently Disabling SELinux

edit /etc/selinux/config

change SELINUX=enforcing to SELINUX=disabled

restart the server and check with sestatus command

Categories: Linux Tags:

Changing MySQL data directory CentOS 7

January 14, 2016 Leave a comment

Doing this on CentOS 7 64 bit and MySQL 5.6 community edition

Sometimes it is better to put mysql in a separate partition than its regular location

Typically the mysql database are located in /var/lib/mysql

I want to change it to /var/data/mysql 

Modify the paths as required in the below commands

stop mysql

systemctl stop mysqld.service

create new mysql data directory

mkdir /var/data/mysql

modify /etc/my.cnf and point to new data directory – add the client section to the top

[client]
port=3306
socket=/var/data/mysql/mysql.sock

[mysqld]
datadir=/var/data/mysql
socket=/var/data/mysql/mysql.sock

copy all files from /var/lib/mysql to the new directory /var/data/mysql

cp -r /var/lib/mysql/* /var/data/mysql

permissions for the new directory

chown -R mysql /var/data/mysql;
chgrp -R mysql /var/data/mysql;
chmod -R g+rw /var/data/mysql;

also modify SELINUX settings to allow mysql to use the different path

# add context and make it permanent 
semanage fcontext -a -s system_u -t mysqld_db_t "/var/data/mysql(/.*)?"
restorecon -Rv /var/data/mysql

start mysql

systemctl start mysqld.service

 

MySQL should start cleanly.
You can verify the change by creating a test database.
Then go to /var/data/mysql and you should be able to see the new database there

Categories: MySQL Tags: ,

Adding a Self Signed Certificate to Trusted Certificate on Linux

January 14, 2016 Leave a comment

Some times, when we generate self signed certificates, some libraries need it to be a part of the operating systems trusted certificates

I am doing this on CentOS 7 and for openSSL

This will only work for apps/libraries that use OpenSSLs trusted certificate list

I already have my self signed certificate in /etc/pki/tls/certs/my-self-signed-cert.crt

cp /etc/pki/tls/certs/devinviteright.crt /etc/pki/ca-trust/source/anchors

sudo update-ca-trust