Searching for a sub string in a String

It is usual to check for an item in a list, tuple, or dict using 'in' operator.

List=[ 'l' , 'i' , 's' , 't']

if 's' in List:

    print "success"

for string we usually go with 'find' function. Its quite easier to go with 'in' operator for string too.

Read More

Fancy Formatting using Python

Its quite easy in python to pretty print the output...
Here are two ways to write a table of squares and cubes:
for x in range(1, 6):
...     print repr(x).rjust(2), repr(x*x).rjust(3),repr(x*x*x).rjust(4)
...
 1   1    1
 2   4    8
 3   9   27
 4  16   64
 5  25  125

>>> for x in range(1,6):
...     print '{0:2d} {1:3d} {2:4d}'.format(x, x*x, x*x*x)
...
 1   1    1
 2   4    8
 3   9   27
 4  16   64
 5  25  125
_____________________________________________________________________________________________

similar to rjust function, there are ljsut and center, which justifies the string with spaces (if filler is not specified)

>>> print 'PYTHON'.center(15)
     PYTHON  
>>>
>>> print 'PYTHON'.center(15, '*' )
*****PYTHON****
_____________________________________________________________________________________________

We can use the brackets ( {} ) to format the data.
The brackets and characters within them (called format fields) are replaced  with the objects passed
into the str.format() method.
A number in the brackets refers to the position of the object passed into the str.format() method.

>>> print 'The story of {0}, {1}, and {other}' .format('Bill', 'Manfred', other='Georg')
The story of Bill, Manfred, and Georg
_____________________________________________________________________________________________

An optional ':' and format specifier can follow the field name. This allows greater control over how the value is formatted. The following example rounds Pi to three places after the decimal.

>>> import math
>>> print 'The value of PI is approximately {0:.3f}.'.format(math.pi)
The value of PI is approximately 3.142.


Passing an integer after the ':' will cause that field to be a minimum number of characters wide. This is useful for making tables pretty.

>>> table = {'Sjoerd': 4127, 'Jack': 4098, 'Dcab': 7678}
>>> for name, phone in table.items():
...             print '{0:10} ==> {1:10d}'.format(name, phone)
...
Jack        ==>       4098
Dcab       ==>       7678
Sjoerd     ==>       4127

___________________________________________________________________________

The % operator can also be used for string formatting in place of str.format() . but it is old fashion. this old style of formatting will eventually be removed from the language. 

>>> import math

>>> print 'The value of PI is approximately %.3f' % math.pi

The value of PI is approximately 3.142

Ref: Python v2.7 Documentation

Read More

Some Optimization Tips/Tricks in Python

Looping:
Use xrange for looping across long ranges; it uses much less memory than range, and may save time as well. Both versions are likely to be faster than a while loop:


xrange is a generator. The performance improvement from the use of generators is the result of the lazy generation of values, means values are generated on demand. Furthermore, we do not need to wait until all the elements have been generated before we start to use them.

You can often eliminate a loop by calling map instead.

Strings:

Building up strings with the concatenation operator + can be slow, because it often involves copying strings several times. Formatting using the % operator is generally faster, and uses less memory.

For example:


If you are building up a string with an unknown number of components, consider using string.join to combine them all, instead of concatenating them as you go:


Sample code :



Here are the results for above codes:

	range	xrange	list_range	list_xrange
real	0m4.136s	0m2.863s	0m1.867s	0m1.569s
user	0m3.960s	0m2.804s	0m1.732s	0m1.516s
system	0m0.160s	0m0.048s	0m0.124s	0m0.048s

File Operation: (Ext to my previous post)

# Each call to a file’s readline method is quite slow:


# It is much faster to read the entire file into memory by calling readlines; however, this uses up a lot of RAM.


# Another approach is to read blocks of lines.


#Best of all is to use the xreadlines method of a file:


Sample Codes:



Here are the results for above codes:

	readline	readlines	readblock	xreadlines
real	0m9.948s	0m13.037s	0m9.880s	0m9.574s
user	0m7.144s	0m4.316s	0m2.716s	0m2.372s
system	0m0.332s	0m1.092s	0m0.404s	0m0.328s

Read More

Best way to open files in Python

C like syntax is as follows,

f = open("file", 'r')
line = f.readline( )
print line
close(f)

some one may forget to close f. this leads to leak, which is considerable in large scale programs. in order to avoid that we  can make use of 'with' or 'for' keywords.


with open("file", 'r') as f:
        line = f.readline()
        print line
or

for line in open("file", 'r'):
       print line


in these scenarios, file will get closed when f is out of scope.

Read More

Timeout - timed wait in shell scripting

TMOUT is the shell built in variable to set the timeout .

TMOUT is used in three different ways: by the read builtin command, by the select builtin, and by the interactive bash shell. If it is unset, or equal to zero, then it is ignored. If it has any positive value, then these three commands which make use of it will timeout after $TMOUT seconds.

Example:
$> cat timeout.sh

1 2 3 4 5 6 7 8 9 10

#!/bin/bash TMOUT=5 read -p "Enter password : " pass if [ $? -eq 0 ] then echo "Password is $pass" else echo "Timed out...." fi

same effect can be achieved using read with -t option.
read -t 5 -p "Enter password : " pass

Read More

PIPESTATUS and its Alternative

$> cat /etc/hosts | grep 000.000
$> $?
1


$> cat /etc/hosts | grep 000.000 | uniq
$> $?
0

Why is it returning success (0) though it fails during grep ??
the return code of a pipeline will be that of the return status of the rightmost command .
How to resolve this ??
use inbuilt PIPESTATUS variable.

PIPESTATUS is a array variable which contain the exit status of each command in piped commands. 

$> cat /etc/hosts | grep 000.000 | uniq
$> echo ${PIPESTATUS[*]}
0 1 0

here $PIPESTATUS[0] contains the exit status of cat /etc/hosts command
$PIPESTATUS[1] contains the exit status of grep 000.000
$PIPESTATUS[2] contains the exit status of uniq


The other way around for this is setting pipefail.

$> set -o pipefail
$> cat /etc/hosts | grep 000.000 | uniq
$> $?
1

Read More

Auto Correct Directory Names / Shell Options

"shopt" can be used to correct typos in cd command.
you can use "cdspell" which will auto correct the directory name you have entered.

Usage:   shopt -s cdspell  
( -s : set
 -u : unset)

$ cd /usr/local/sben     will results in bash: cd: /usr/local/sbEN: No such file or directory
but once you have enabled cdspell option,

$ cd /usr/local/sben    
/usr/local/sbin
$pwd
 /usr/local/sbin

some other useful options are,
histappend : this will append to the history file, instead of overwriting it
checkwinsize: check the window size after each command and, if necessary, update the values of LINES and COLUMNS.

lot of other options are available with shopt.
to list all these options used shopt -p

Or you can refer the man page  http://ss64.com/bash/shopt.html

Read More

vim/gvim-Tips and Tricks

Tip 1 :

Use markers to set a place where you want to go back quickly or to delete some block of code

ma   - mark current position as 'a' [ can use a-z in same file, or can use A-Z between files]
'a     - go to mark a
d'a   - delete current position to mark a [ can use 'p' after this to paste that block ]

 

Tip 2:

 Do you want to reverse the entire file or a block ?? its easy. Try the following command
     :g/^/m0                          [ Reverse file ]
    :'a, 'bg/^/m'b                   [ Reverse a section a to b ]

 

Tip 3:

Want to delete all empty lines from a file, here you go

    :g/^\s*$/d 

Tip 4: 

Display all lines which matches the pattern
    :g/<pattern>/
    :g/<pattern>/#                     [ display all line with line numbers (in vi) ]

Tip 5: 

Finding some successive lines

    :/^\n\{3}                                     [ find 3 successive empty lines ]
    :/\(^str.*\n\)\{2}                          [ find 2 successive line starting with str ]  
    :%s/^\n\{3}//                              [ delete block of 3 empty line ]    

    :%s/^\(.*\)\n\1$/\1/                    [ delete duplicate lines ] 

    :%s/^\(.*\)\(\n\1\)\+$/\1/            [ delete multiple duplicate line ]

    :%s/\v(.*\n){5}/&\r                     [ insert a blank line every 5 lines (can write a string also after & )]

 

Tip 6:

changing the case
    guu or Vu              [ lower case line ]
    gUU or VU           [ uppercase line ]
    vEu                        [ lower case word ] 
    vEU                       [ upper case word ]
    g~~                        [ flip case ]
    guG                       [ lower case entire file (go to line 1 and execute the command) ]
    gUG                      [ upper case entire file (go to line 1 and execute the command) ]

 Tip 7: 

    gf         [ Go to  file name under curser ]
    gd        [ Go to declaration of local variable under cursor ]
    gD       [ Go to declaration of global variable under cursor ]

 Tip 8:

*In ESC mode

    vib         [ Select a block - selecting function arguments ( ) ]
    viB         [ Select a block - selecting entire function definition { } ]  - useful when indenting a function(viB and = )
    ci"          [ Edit content within " " ] - useful when replacing string

    ci'          [ Edit content within ' ' ]

Read More

Creating A List Using vim

Some time you may need to insert a list of ascending numbers in a file. We can do this using vim very easily.

Method 1 :

:put =range(11,15)

This will create a series of increasing numbers after the current line including 11 and 15.
output will be-     

11
12
13
14
15

 
Method 2 :
Some times if you want to add a constant predefined string before the number series, then you you can go using loop to create a list.
       

for i in range(1,6) | put ='10.168.0.'.i | endfor

Executing the command will insert the following lines after the current line

10.168.0.1
10.168.0.2
10.168.0.3
10.168.0.4
10.168.0.5
10.168.0.6

   
Method 3 : 
Another simple way is to use Ctrl+A in a macro.
for example, some times you may need to initialize a array of size 100.
array[0] = 0
array[1] = 0
array[2] = 0
... so on

type array[0] = 0; then start recording macro.
Type the following commands without pressing enter.

qa          [ record to buffer 'a' ]
Y           [ copy the current line ] 
p           [ paste the line ]
Ctrl-A      [ Ctrl+A increment the number or use Ctrl+X to decrement]
q           [ stop recording macro ] 

now type 100@a to perform the macro 100 times.

array[0] = 0;
array[1] = 0;
array[2] = 0;
array[3] = 0;
array[4] = 0;
... and so on

if you want to increase the index and assigned value also then type Ctrl+A by moving the courser to the value.

Read More

`Tar`- An Ultimate Archive Utility

How to use tar

tar  [options] [name of tar file to be created] [list of files and directories to be included]

1. Creating an archive using tar
    
      $ tar -cvf target.tar file1 file2 dir1 
      $ tar -cvzf target.tar.gz dirname/             [ creates gzipped tar archive file ]
      $ tar -cvjf target.tar.bz2 dirname/           [ creates bzipped tar archive file ]

    Note: this -cvf options alone does not provide any compressions

2 Extracting an archive
 
      $ tar -xvf target.tar
      $ tar -xvzf target.tar.gz                            [ extracts gzipped tar archive file ]
      $ tar -xvjf target.tar.bz2                          [ extracts bzipped tar archive file ]

3. Listing an archive

      $ tar -tvf target.tar
      $ tar -tvzf target.tar.gz                             [ lists gzipped tar archive file ]
      $ tar -tvjf target.tar.bz2                           [ lists bzipped tar archive file ]

4. Extract a file/directory

    $ tar -xvf target.tar /mydir/myfile
    $ tar -xvzf target.tar.gz /mydir/myfile
    $ tar -xvjf target.tar.bz2 /mydir/myfile 

    if you want to extract a group of files from the archive use -wildcard option
    $ tar -xvf target.tar --wildcards '*.cc'
 
5. Adding a file or directory to an existing archive
 
    You can add additional files to an existing tar archive by specifying -u or -r options.
    use -u option if u do not want duplicates in the tar archived file. -r option will add the file to the existed archive if it exist also.

    $ tar -rvf target.tar newfile     or $ tar -uvf target.tar newfile

    Note: this can't be done for gz or bz2 archives.
 

Read More

'cat' can be harmful..

'cat' can be harmful at times. some people might have lost the data because of improper usage of cat command like cat > filename.txt, or it can do over write some existing files data without prompting the user.
to get rid out of these situations

1. open the terminal

2. execute " set -o noclobber "

that's it. now next time when cat tries to over write the existing file it will prompt the user.

Read More