Parse Command Line arguments

Bash

#!/bin/bash
OPTIND=1           # reset in case getopts has been used previously
# default settings
output=""
verbose=0
while getopts "h?vf:" opt; do
    case "$opt" in
    h|\?)  show_help; exit 1;;
    v) verbose=1;;
    f) output=$OPTARG;;
    esac
done

shift $((OPTIND-1))
[ "$1" = "--" ] && shift;
echo "verbose=$verbose, output=$output, leftover: $@"

Reference:

Perl

#! /usr/bin/env perl

use strict;
use warnings;
use Getopt::Std;

sub usage {
    print "usage:\n".
          "$0 -hvf other_list\n".
          "  -h     help\n".
          "  -v     verbose\".
          "  -f file output file\n".
          "\n";
    exit 1;
}

my %opts;
my $rc = getopts('hvf:', \%opts);
usage() if ($rc || $opts{h});
print "options:\n";
print "$_: $opts{$_}\n" foreach (sort keys %opts);
print "leftovers: @ARGV\n";

Reference:

Python

#! /usr/bin/env python

import argparse

parser = argparse.ArgumentParser()
parser.ad_argument('-f', '--output_file', type=str,
                     default='my_output', help='output file name')
parser.add_argument('-v', '--verbose', action='store_true', help='verbose mode')
args = parser.parse_args()
if args.verbose:
    print 'verbose mode'
with open(args.output_file, 'w') as f:
    # do something
    pass

Reference:

C/C++

under construction

Set Hostname on Linux

Types of hostnames

The hostname can be configured as follows

  1. Static host name assigned by sysadmin. For example, “server1”, “wwwbox2”, or “server42.cyberciti.biz”.
  2. Transient/dynamic host name assigned by DHCP or mDNS server at run time.
  3. Pretty host name assigned by sysadmin/end-users and it is a free-form UTF8 host name for presentation to the user. For example, “Vivek’s netbook”.

hostnamectl command

Let us see how to use the hostnamectl command.

How do I see the host names?

$ hostnamectl
## OR ##
$ hostnamectl status

Sample outputs:

   Static hostname: centos-7-rc
         Icon name: computer
           Chassis: n/a
        Machine ID: b5470b10ccfd49ed8e4a3b0e953a53c3
           Boot ID: f79de79e2dac4670bddfe528e826b61f
    Virtualization: oracle
  Operating System: CentOS Linux 7 (Core)
       CPE OS Name: cpe:/o:centos:centos:7
            Kernel: Linux 3.10.0-229.1.2.el7.x86_64
      Architecture: x86_64

How do I set the host name?

The syntax is:

# hostnamectl set-hostname Your-New-Host-Name-Here
# hostnamectl set-hostname "Your New Host Name Here" --pretty
# hostnamectl set-hostname Your-New-Host-Name-Here --static
# hostnamectl set-hostname Your-New-Host-Name-Here --transient

To set host name to “R2-D2”, enter:

# hostnamectl set-hostname R2-D2

To set static host name to “server1.cyberciti.biz”, enter:

# hostnamectl set-hostname server1.cyberciti.biz --static

To set pretty host name to “Senator Padmé Amidala’s Laptop”, enter:

# hostnamectl set-hostname "Senator Padmé Amidala's Laptop" --pretty

To verify new settings, enter:

# hostnamectl status

Sample outputs:

   Static hostname: server1.cyberciti.biz
   Pretty hostname: Senator Padmé Amidala's Laptop
Transient hostname: r2-d2
         Icon name: computer
           Chassis: n/a
        Machine ID: b5470b10ccfd49ed8e4a3b0e953a53c3
           Boot ID: f79de79e2dac4670bddfe528e826b61f
    Virtualization: oracle
  Operating System: CentOS Linux 7 (Core)
       CPE OS Name: cpe:/o:centos:centos:7
            Kernel: Linux 3.10.0-229.1.2.el7.x86_64
      Architecture: x86_64

How do I delete a particular host name?

The syntax is:

# hostnamectl set-hostname ""
# hostnamectl set-hostname "" --static
# hostnamectl set-hostname "" --pretty

How do I change host name remotely?

Use any one of the following syntax:

# ssh root@server-ip-here hostnamectl set-hostname server1

OR set server1 as host name on a remote server called 192.168.1.42 using ssh:

# hostnamectl set-hostname server1 -H root@192.168.1.42

 

 

 

Reference

 

Note:

  • hostnamectl exists on RHEL/Centos7, Debian8 (validated)

 

Some topics in SSH

Login SSH without typing password every time

  1. create public and private keys on local-host (debian)
    jason@debian$ ssh-keygen
    Generating public/private rsa key pair.
    Enter file in which to save the key (/home/jason/.ssh/id_rsa):[Enter key]
    Enter passphrase (empty for no passphrase): [Press enter key]
    Enter same passphrase again: [Pess enter key]
    Your identification has been saved in /home/jason/.ssh/id_rsa.
    Your public key has been saved in /home/jason/.ssh/id_rsa.pub.
    The key fingerprint is:
    33:b3:fe:af:95:95:18:11:31:d5:de:96:2f:f2:35:f9 jason@debian
    

    This command creates two RSA key files, id_rsa for private key, and id_rsa.pub for public key, in the directory $HOME/.ssh.

  2. copy the public key file to remote-host (fedora) using ssh-copy-id
    jason@debian$ ssh-copy-id -i ~/.ssh/id_rsa.pub fedora
    jason@fedora's password:
    jason@fedora$
    

    Now try logging into the remote machine (fedora)

    jason@debina$ ssh fedora
    Last login: Thu Feb 16 11:45:20 2017 from debian
    jason@fedora$
    

    Notice that ssh does not ask for password. Check the authorized_key to make sure we haven’t added extra keys that you weren’t expecting.

    jason@fedora$ cat .ssh/authorized_keys
    

ref – 3 Steps to Perform SSH Login Without Password Using ssh-keygen & ssh-copy-id
In case ssh-copy-id is not available (for example, on mingw), as an alternated solution, you can manually copy the public key file (id_rsa.pub) to the remote host, and add the content as a record into $HOME/.ssh/authorized_keys. The file is a text file, so if it doesn’t exist, you can create it with a text editor. Don’t forget to change the mode to 644, otherwise sshd won’t use it.

Define host aliases in SSH config

Following lines in ~/.ssh/config define two aliases for the host staging.example.com

host foo bar # aliases separated by whites
  hostname staging.example.com

ref – Defining host aliases in your SSH config

Setup PostgreSQL on Linux

Install from source

The current latest release version is 9.5.4.

# get and install postgres from source
$ cd $YOUR_WORKING_PATH
$ curl https://ftp.postgresql.org/pub/source/v9.5.4/postgresql-9.5.4.tar.gz > postgresql-9.5.4.tar.gz
$ tar xzf postgresql-9.5.4.tar.gz
$ cd postgresql-9.5.4
$ ./configure # default --prefix=/usr/local/pgsql
$ make

$ sudo make install
$ sudo adduser postgres
$ sudo passwd postgres
$ sudo mkdir /usr/local/pgsql/data
$ sudo chown postgres /usr/local/pgsql/data

$ su - postgres
$ /usr/local/pgsql/bin/initdb -D /usr/local/pgsql/data # initialize database cluster

Configure Access from network

By default, postgres allows connection only from localhost. To allow clients connect over network, host entries need to be added in /usr/local/pgsql/data/pg_hba.conf, such as

host all all 192.168.0.0/16 md5

allows all users from 192.168.x.x to access all database, using md5 to hash their password. See more details in the manual 18.3 Connection and Authentication

Auto start when system boots

Although 17.3. Starting the Database Server in the manual gives the solution for old-style /etc/rc.d, modern linux distributions use systemd, here is the solution

add postgresql.service

[Unit]
Description=PostgreSQL database server
After=network.target
[Service]
Type=forking
User=postgres
Group=postgres

# Where to send early-startup messages from the server (before the logging
# options of postgresql.conf take effect)
# This is normally controlled by the global default set by systemd
# StandardOutput=syslog
# Disable OOM kill on the postmaster
OOMScoreAdjust=-1000

# ... but allow it still to be effective for child processes
# (note that these settings are ignored by Postgres releases before 9.5)
Environment=PG_OOM_ADJUST_FILE=/proc/self/oom_score_adj
Environment=PG_OOM_ADJUST_VALUE=0

# Maximum number of seconds pg_ctl will wait for postgres to start.  Note that
# PGSTARTTIMEOUT should be less than TimeoutSec value.
Environment=PGSTARTTIMEOUT=270
Environment=PGDATA=/usr/local/pgsql/data

ExecStart=/usr/local/pgsql/bin/pg_ctl start -o "-h '*'" -D ${PGDATA} -s -w -t ${PGSTARTTIMEOUT}
ExecStop=/usr/local/pgsql/bin/pg_ctl stop -D ${PGDATA} -s -m fast
ExecReload=/usr/local/pgsql/bin/pg_ctl reload -D ${PGDATA} -s

# Give a reasonable amount of time for the server to start up/shut down.
# Ideally, the timeout for starting PostgreSQL server should be handled more
# nicely by pg_ctl in ExecStart, so keep its timeout smaller than this value.
TimeoutSec=300

[Install]
WantedBy=multi-user.target

as /usr/lib/systemd/system/postgresql.service, then run following commands

sudo systemctl daemon-reload
sudo systemctl enable postgresql
sudo systemctl start postgresql

After that you can connect the postgresql cluster as

/usr/local/pgsql/bin/psql -U postgres -h hostname
postgres=# create role myaccount login password='my passowrd';
postgres=# create database mydb;
postgres=# alter database mydb owner to myaccount;

 

Reference

 

Perl Tricks – each %hash

Perl has three ways to iterate a hash:

  • keys %hash iterates keys
  • values %hash iterates values
  • each %hash iterates the pair (key, value)

Perl contains an internal iterator for such kind of iteration. So far so good, as long as you finish the iteration every time. What if you break the iteration before it finishes, and start a new iteration later? According to perl doc, the iterator is reset only when

  • end of iteration
  • keys or values is called

so the script might not work as you expect in the following pattern:

while (my ($k, $v) = each %hash) {
      # do some work
      last if ($k == $somekey || $v == $somevalue);
}
# sometime later
while (my ($k, $v) = each %hash) {  # iterator is not reset if the previous while not finish
         # some work
}

The conclusion – don’t use each if you plan to break the iteration, use keys instead.

 

Capture signals in perl

Here’s a sample perl script

#! /usr/bin/perl
=pod

This program demonstrate the usage of %SIG to capture signals in perl.

An alternative solution is using pragma sigtrap.
    use sigtrap qw(handler hdl_int INT QUIT);
The disadvantage of sigtrap is that you can't save and restore
the original handlers.

=cut

use strict;
use warnings;
use Time::HiRes qw( usleep) ;

package ST;             # scope tracer
sub new {
    my $class = shift;
    my $name = shift;
    my $self = {
        name => $name,
    };
    print "++++++++++++++++enter $self->{name}\n";
    bless $self, $class;
    return $self;
}
sub DESTROY {
    my $self = shift;
    print "----------------leave $self->{name}\n";
}



package main;

my %orgSig = ();        # save original handlers if restoring is wished

sub listAllSignals {
    print "supported signals\n";
    foreach (sort keys %SIG) {
        unless (/NUM\d+/) {
            my $v = $SIG{$_} || '';
            print "  $_=$v\n";
        }
    }
}

sub longtask {
    # emulating a task that runs for a while.
    my $title = shift || 'some long-time task';
    my $loop = shift || 5;
    my $interval = shift || 1;
    my $v = new ST($title);
    foreach (1..$loop) {
        print "      $title: working ...$_\n";
        usleep($interval * 1000000);
    }
}

sub foo {
    my $s = ST->new('foo');
    my $greet = shift;

    print "$greet from foo\n";
    longtask('foo body', 10, 0.5);
}

sub hdl_int {
# signal is blocked and appended in the queue
# DURING the signal handler is running,
# so restoring original handler inside the handler
# IS GENERARLLY NOT a good idea unless that is expected behavior
    my $s = ST->new('hdl_int');
    longtask('hdl_int body', 5, 0.2);
#   $SIG{'INT'} = $orgSig{'INT'};
}

$orgSig{$_} = $SIG{$_} foreach (keys %SIG);
listAllSignals();

# these two signals are used for "normal" exit
$SIG{'INT'} = \&hdl_int;
$SIG{TERM} = sub { print "I captured TERM\n"; };

# There two can't be captured
#   For sure KILL (9) can't be captured
#   According to doc, QUIT is not able to be captured either,
#   however the interesting fact is that it's captured on debian 8
$SIG{QUIT} = sub { print "I captured QUIT\n"; };
$SIG{KILL} = sub { print "I captured KILL\n"; };


print("hello world, my pid=$$\n");
sleep(3);
foo("Greeting");
print("The end\n");

Here is the output when Control-C is pressed twice during the running

jasonz@jzdebian$ perl sig.pl
supported signals
  ABRT=
  ALRM=
  BUS=
  CHLD=
  CLD=
  CONT=
  FPE=IGNORE
  HUP=
  ILL=
  INT=
  IO=
  IOT=
  KILL=
  PIPE=
  POLL=
  PROF=
  PWR=
  QUIT=
  RTMAX=
  RTMIN=
  SEGV=
  STKFLT=
  STOP=
  SYS=
  TERM=
  TRAP=
  TSTP=
  TTIN=
  TTOU=
  UNUSED=
  URG=
  USR1=
  USR2=
  VTALRM=
  WINCH=
  XCPU=
  XFSZ=
hello world, my pid=47074
^C++++++++++++++++enter hdl_int
++++++++++++++++enter hdl_int body
      hdl_int body: working ...1
      hdl_int body: working ...2
      hdl_int body: working ...3
^C      hdl_int body: working ...4
      hdl_int body: working ...5
----------------leave hdl_int body
----------------leave hdl_int
++++++++++++++++enter hdl_int
++++++++++++++++enter hdl_int body
      hdl_int body: working ...1
      hdl_int body: working ...2
      hdl_int body: working ...3
      hdl_int body: working ...4
      hdl_int body: working ...5
----------------leave hdl_int body
----------------leave hdl_int
++++++++++++++++enter foo
Greeting from foo
++++++++++++++++enter foo body
      foo body: working ...1
      foo body: working ...2
      foo body: working ...3
      foo body: working ...4
      foo body: working ...5
      foo body: working ...6
      foo body: working ...7
      foo body: working ...8
      foo body: working ...9
      foo body: working ...10
----------------leave foo body
----------------leave foo
The end
jasonz@jzdebian$ 

The interesting fact is that QUIT is captured although the linux document declares that it is not capture-able. Here is the snippet of the output

  XCPU=
  XFSZ=
hello world, my pid=47344
++++++++++++++++enter foo
Greeting from foo
++++++++++++++++enter foo body
      foo body: working ...1
      foo body: working ...2
      foo body: working ...3
I captured QUIT
      foo body: working ...4
      foo body: working ...5
      foo body: working ...6
      foo body: working ...7
      foo body: working ...8
      foo body: working ...9
      foo body: working ...10
----------------leave foo body
----------------leave foo
The end
jasonz@jzdebian$ 

when a QUIT signal is sent

jasonz@jzdebian$ kill -QUIT 47344
jasonz@jzdebian$ 

Extended Usage – capture warn() and dir()

This is extremely convenient to log all those information

sub WARN_handler {
    my ($signal) = @_;
    log("WARN: $signal");
}
sub DIE_handler {
    my ($signal) = @_;
    log("DIE: $signal");
}
sub log {
    my (@array) = @_;
    open(LOGFILE, ">>my.log");
    print LOGFILE (@array);
    close(LOGFILE);
}

$SIG{__WARN__} = 'WARN_handler';
$SIG{__DIE__} = 'DIE_handler';
chdir('/printer') or warn($!);
chdir('/printer') or die($!);

p4 – perforce common commands for batch operations

p4 help
get online help
p4 dirs //Drivers/*
list direct directories under given path (//Drivers)
p4 files //Drivers/…
list files (recursively) under given path (//Drivers)
p4 labels -m 5 //Drivers/…
list latest 5 labels of a path
p4 changes -m 1 //Drivers/…
list  the latest change list
p4 user jason
list user jason’s information
p4 client
show current client
p4 opened
list opened files (that are in the pending change lists)
p4 sync
get files to the workspace

p4 sync file#rev
p4 sync @label
p4 sync //depot/proj/...@rev
p4 sync @2011/06/24
p4 sync file#none           # delete from workspace
p4 unshelve -s changelist [ file_pattern … ]
unshelve a file/change list

perforce file name format

file#n              the n-th revision of file
file#m,n            revision range m,n
file#none           nonexistent revision (delete from workspace)
file#0
file#head           head (latest) revision
file#have           the revision on the current client
file@=n             change number
file@n
file@label          file in label
file@clientname     revision of file last taken into client workspace
file@datespec       datespec    yyyy/mm/dd([ :]hh:mm:ss)?
file@now