Python v/s Shell

I was going through the "xarg" documentation, why it is required. Man page doesn’t reveal much info so I did Wiki and got basic idea. The reason it is in place is because of the limitation of the max number of arguments passed to any application. Prior to kernel version 2.6.23, there was a hard code limit to the max number of args passed to application. This limitation was eliminated by this patch :

http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=b6a2fea39318e43fee84fa7b0b90d68bed92d2ba

Now I wanted to find out the exact number of argument that can be passed to an app, so I wrote a shell script:

#!/bin/bash
i=1
j=0;
str="test"
while true
do
    i=`expr $i \* 2`;
    j=0;
#    echo $i
    while [ $j -le $i ]
    do
        str=$str" "$j
        j=`expr $j + 1`
    done
    echo $str > /tmp/test
    if [ `echo $?` -gt 0 ]
    then
        echo $j
    fi

done

And I didn’t know that shell script is so slow. Most of the time was being spent in the above script was the "strcat" kind of operation. I never figured out the exact amount of args from shell, as it was slow. So I turned back to python, which is blazing fast as compared to shell. And here is the script.

#!/usr/bin/env python
import sys
import os

arg=23680
while True:
    arg=arg+1;
    string="";
    temp=0;
    while temp < arg:
        string=string+" "+str(temp);
        temp = temp + 1;
    f=os.system("echo "+string+">/dev/null");
    if (f > 0):
        print "error"
        print string
        f = os.system("echo "+string);
        break;
 

The number 23680 was figured out by doing binary search initially. It happened in just 5-6 seconds :) Python rocks!! 

The following link shows the value  of max args on various operating systems: http://www.in-ulm.de/~mascheck/various/argmax/

Edit:

In previous python example, I have taken the initial value by doing some binary search by hand. Last night I have made a program which does the binary search, so it will initially start with 1, and eventually reveal the number (of arguments) where the command fails. Here is the code:

#!/usr/bin/python
import os
import time
low = 1;
high = 2;
arg="1 ";
factor = 0;
itr = 0;
while True:
arg="1 ";
itr = itr + 1;
for i in range(0,high):
arg=arg+"1 ";
ret = os.system("echo "+arg+">/dev/null");
if (ret > 0):
print "Error for arg = ",high
if (factor == 0):
factor = low/2;
else:
factor = factor/2;
high = high – factor;
print "reducing high to ", high, factor
time.sleep(1)
continue;

if (factor == 0):
low = high;
high = low + high;
else:
high = high + factor/2;
if (factor == 1):
print "high found, ", high, "in ", itr, "iteration";
break;
print i , low, high
time.sleep(0.5);

Advertisements

2 thoughts on “Python v/s Shell

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s