Educating yourself does not mean that you were stupid in the first place; it means that you are intelligent enough to know that there is plenty left to 'learn'. -Melanie Joy

Sunday 23 December 2012

Searching for a sub string in a String

December 23, 2012 Posted by Dinesh No comments


It is usual to check for an item in a list, tuple, or dict using 'in' operator.

List=[ 'l' , 'i' , 's' , 't']
if 's' in List:
    print "success"

for string we usually go with 'find' function. Its quite easier to go with 'in' operator for string too.


Saturday 22 December 2012

Fancy Formatting using Python

December 22, 2012 Posted by Dinesh 1 comment

Its quite easy in python to pretty print the output...
Here are two ways to write a table of squares and cubes:
for x in range(1, 6):
...     print repr(x).rjust(2), repr(x*x).rjust(3),repr(x*x*x).rjust(4)
...
 1   1    1
 2   4    8
 3   9   27
 4  16   64
 5  25  125

>>> for x in range(1,6):
...     print '{0:2d} {1:3d} {2:4d}'.format(x, x*x, x*x*x)
...
 1   1    1
 2   4    8
 3   9   27
 4  16   64
 5  25  125
_____________________________________________________________________________________________

similar to rjust function, there are ljsut and center, which justifies the string with spaces (if filler is not specified)

>>> print 'PYTHON'.center(15)
     PYTHON  
>>>
>>> print 'PYTHON'.center(15, '*' )
*****PYTHON****
_____________________________________________________________________________________________

We can use the brackets ( {} ) to format the data.
The brackets and characters within them (called format fields) are replaced  with the objects passed
into the str.format() method.
A number in the brackets refers to the position of the object passed into the str.format() method.

>>> print 'The story of {0}, {1}, and {other}' .format('Bill', 'Manfred', other='Georg')
The story of Bill, Manfred, and Georg
_____________________________________________________________________________________________

An optional ':' and format specifier can follow the field name. This allows greater control over how the value is formatted. The following example rounds Pi to three places after the decimal.

>>> import math
>>> print 'The value of PI is approximately {0:.3f}.'.format(math.pi)
The value of PI is approximately 3.142.


Passing an integer after the ':' will cause that field to be a minimum number of characters wide. This is useful for making tables pretty.

>>> table = {'Sjoerd': 4127, 'Jack': 4098, 'Dcab': 7678}
>>> for name, phone in table.items():
...             print '{0:10} ==> {1:10d}'.format(name, phone)
...
Jack        ==>       4098
Dcab       ==>       7678
Sjoerd     ==>       4127

___________________________________________________________________________

The % operator can also be used for string formatting in place of str.format() . but it is old fashion. this old style of formatting will eventually be removed from the language. 

>>> import math
>>> print 'The value of PI is approximately %.3f' % math.pi
The value of PI is approximately 3.142



Ref: Python v2.7 Documentation

Some Optimization Tips/Tricks in Python

December 22, 2012 Posted by Dinesh

Looping:
Use xrange for looping across long ranges; it uses much less memory than range, and may save time as well. Both versions are likely to be faster than a while loop:


xrange is a generator. The performance improvement from the use of generators is the result of the lazy generation of values, means values are generated on demand. Furthermore, we do not need to wait until all the elements have been generated before we start to use them.

You can often eliminate a loop by calling map instead.

Strings:

Building up strings with the concatenation operator + can be slow, because it often involves copying strings several times. Formatting using the % operator is generally faster, and uses less memory.

For example:


If you are building up a string with an unknown number of components, consider using string.join to combine them all, instead of concatenating them as you go:


Sample code :



Here are the results for above codes:


range xrange list_range list_xrange
real 0m4.136s 0m2.863s0m1.867s 0m1.569s
user 0m3.960s 0m2.804s0m1.732s 0m1.516s
system 0m0.160s 0m0.048s 0m0.124s0m0.048s




File Operation: (Ext to my previous post)

# Each call to a file’s readline method is quite slow:


# It is much faster to read the entire file into memory by calling readlines; however, this uses up a lot of RAM.


# Another approach is to read blocks of lines.


#Best of all is to use the xreadlines method of a file:


Sample Codes:



Here are the results for above codes:

readlinereadlinesreadblockxreadlines
real0m9.948s0m13.037s0m9.880s0m9.574s
user0m7.144s0m4.316s0m2.716s0m2.372s
system0m0.332s0m1.092s0m0.404s0m0.328s

Friday 21 December 2012

Best way to open files in Python

December 21, 2012 Posted by Dinesh No comments
C like syntax is as follows,

f = open("file", 'r')
line = f.readline( )
print line
close(f)

some one may forget to close f. this leads to leak, which is considerable in large scale programs. in order to avoid that we  can make use of 'with' or 'for' keywords.


with open("file", 'r') as f:
        line = f.readline()
        print line
or

for line in open("file", 'r'):
       print line


in these scenarios, file will get closed when f is out of scope.

Thursday 4 October 2012

Timeout - timed wait in shell scripting

October 04, 2012 Posted by Dinesh , , No comments
TMOUT is the shell built in variable to set the timeout .

TMOUT is used in three different ways: by the read builtin command, by the select builtin, and by the interactive bash shell. If it is unset, or equal to zero, then it is ignored. If it has any positive value, then these three commands which make use of it will timeout after $TMOUT seconds.

Example:
$> cat timeout.sh


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
#!/bin/bash

TMOUT=5
read -p "Enter password : " pass
if [ $? -eq 0 ]
then
   echo "Password is $pass"
  else
    echo "Timed out...."
fi



same effect can be achieved using read with -t option.
read -t 5 -p "Enter password : " pass


PIPESTATUS and its Alternative

October 04, 2012 Posted by Dinesh , , No comments
$> cat /etc/hosts | grep 000.000
$> $?
1


$> cat /etc/hosts | grep 000.000 | uniq
$> $?
0

Why is it returning success (0) though it fails during grep ??
the return code of a pipeline will be that of the return status of the rightmost command .
How to resolve this ??
use inbuilt PIPESTATUS variable.

PIPESTATUS is a array variable which contain the exit status of each command in piped commands. 

$> cat /etc/hosts | grep 000.000 | uniq
$> echo ${PIPESTATUS[*]}
0 1 0

here $PIPESTATUS[0] contains the exit status of cat /etc/hosts command
$PIPESTATUS[1] contains the exit status of grep 000.000
$PIPESTATUS[2] contains the exit status of uniq


The other way around for this is setting pipefail.

$> set -o pipefail
$> cat /etc/hosts | grep 000.000 | uniq
$> $?
1


Saturday 18 August 2012

Auto Correct Directory Names / Shell Options

August 18, 2012 Posted by Dinesh , , No comments

"shopt" can be used to correct typos in cd command.
you can use "cdspell" which will auto correct the directory name you have entered.

Usage:   shopt -s cdspell  
( -s : set
 -u : unset)

$ cd /usr/local/sben     will results in bash: cd: /usr/local/sbEN: No such file or directory
but once you have enabled cdspell option,

$ cd /usr/local/sben    
/usr/local/sbin
$pwd
 /usr/local/sbin

some other useful options are,
histappend : this will append to the history file, instead of overwriting it
checkwinsize: check the window size after each command and, if necessary, update the values of LINES and COLUMNS.

lot of other options are available with shopt.
to list all these options used shopt -p

Or you can refer the man page  http://ss64.com/bash/shopt.html





Tuesday 5 June 2012

vim/gvim-Tips and Tricks

June 05, 2012 Posted by Dinesh No comments

Tip 1 :

Use markers to set a place where you want to go back quickly or to delete some block of code

ma   - mark current position as 'a' [ can use a-z in same file, or can use A-Z between files]
'a     - go to mark a
d'a   - delete current position to mark a [ can use 'p' after this to paste that block ]

 

Tip 2:

 Do you want to reverse the entire file or a block ?? its easy. Try the following command
     :g/^/m0                          [ Reverse file ]
    :'a, 'bg/^/m'b                   [ Reverse a section a to b ]

 

Tip 3:

Want to delete all empty lines from a file, here you go

    :g/^\s*$/d 

Tip 4: 

Display all lines which matches the pattern
    :g/<pattern>/
    :g/<pattern>/#                     [ display all line with line numbers (in vi) ]


Tip 5: 

Finding some successive lines
    :/^\n\{3}                                     [ find 3 successive empty lines ]
    :/\(^str.*\n\)\{2}                          [ find 2 successive line starting with str ]  
    :%s/^\n\{3}//                              [ delete block of 3 empty line ]    
    :%s/^\(.*\)\n\1$/\1/                    [ delete duplicate lines ] 
    :%s/^\(.*\)\(\n\1\)\+$/\1/            [ delete multiple duplicate line ]
    :%s/\v(.*\n){5}/&\r                     [ insert a blank line every 5 lines (can write a string also after & )]

 

Tip 6:

changing the case
    guu or Vu              [ lower case line ]
    gUU or VU           [ uppercase line ]
    vEu                        [ lower case word ] 
    vEU                       [ upper case word ]
    g~~                        [ flip case ]
    guG                       [ lower case entire file (go to line 1 and execute the command) ]
    gUG                      [ upper case entire file (go to line 1 and execute the command) ]


 Tip 7: 

    gf         [ Go to  file name under curser ]
    gd        [ Go to declaration of local variable under cursor ]
    gD       [ Go to declaration of global variable under cursor ]

 Tip 8:

*In ESC mode

    vib         [ Select a block - selecting function arguments ( ) ]
    viB         [ Select a block - selecting entire function definition { } ]  - useful when indenting a function(viB and = )
    ci"          [ Edit content within " " ] - useful when replacing string
    ci'          [ Edit content within ' ' ]

Monday 4 June 2012

Creating A List Using vim

June 04, 2012 Posted by Dinesh No comments
Some time you may need to insert a list of ascending numbers in a file. We can do this using vim very easily.

Method 1 :

:put =range(11,15)


This will create a series of increasing numbers after the current line including 11 and 15.
output will be-     

11
12
13
14
15
 
Method 2 :
Some times if you want to add a constant predefined string before the number series, then you you can go using loop to create a list.
       
for i in range(1,6) | put ='10.168.0.'.i | endfor

Executing the command will insert the following lines after the current line

10.168.0.1
10.168.0.2
10.168.0.3
10.168.0.4
10.168.0.5
10.168.0.6
   
Method 3 : 
Another simple way is to use Ctrl+A in a macro.
for example, some times you may need to initialize a array of size 100.
array[0] = 0
array[1] = 0
array[2] = 0
... so on

type array[0] = 0; then start recording macro.
Type the following commands without pressing enter.

qa          [ record to buffer 'a' ]
Y           [ copy the current line ] 
p           [ paste the line ]
Ctrl-A      [ Ctrl+A increment the number or use Ctrl+X to decrement]
q           [ stop recording macro ] 

now type 100@a to perform the macro 100 times.

array[0] = 0;
array[1] = 0;
array[2] = 0;
array[3] = 0;
array[4] = 0;
... and so on

if you want to increase the index and assigned value also then type Ctrl+A by moving the courser to the value.

`Tar`- An Ultimate Archive Utility

June 04, 2012 Posted by Dinesh , , No comments
How to use tar
tar  [options] [name of tar file to be created] [list of files and directories to be included]

1. Creating an archive using tar
    
      $ tar -cvf target.tar file1 file2 dir1 
      $ tar -cvzf target.tar.gz dirname/             [ creates gzipped tar archive file ]
      $ tar -cvjf target.tar.bz2 dirname/           [ creates bzipped tar archive file ]

    Note: this -cvf options alone does not provide any compressions

2 Extracting an archive
 
      $ tar -xvf target.tar
      $ tar -xvzf target.tar.gz                            [ extracts gzipped tar archive file ]
      $ tar -xvjf target.tar.bz2                          [ extracts bzipped tar archive file ]

3. Listing an archive

      $ tar -tvf target.tar
      $ tar -tvzf target.tar.gz                             [ lists gzipped tar archive file ]
      $ tar -tvjf target.tar.bz2                           [ lists bzipped tar archive file ]

4. Extract a file/directory

    $ tar -xvf target.tar /mydir/myfile
    $ tar -xvzf target.tar.gz /mydir/myfile
    $ tar -xvjf target.tar.bz2 /mydir/myfile 

    if you want to extract a group of files from the archive use -wildcard option
    $ tar -xvf target.tar --wildcards '*.cc'
 
5. Adding a file or directory to an existing archive
 
    You can add additional files to an existing tar archive by specifying -u or -r options.
    use -u option if u do not want duplicates in the tar archived file. -r option will add the file to the existed archive if it exist also.

    $ tar -rvf target.tar newfile     or $ tar -uvf target.tar newfile

    Note: this can't be done for gz or bz2 archives.
 





'cat' can be harmful..

June 04, 2012 Posted by Dinesh , , No comments
'cat' can be harmful at times. some people might have lost the data because of improper usage of cat command like cat > filename.txt, or it can do over write some existing files data without prompting the user.
to get rid out of these situations
1. open the terminal
2. execute " set -o noclobber "
that's it. now next time when cat tries to over write the existing file it will prompt the user.