Notorious Devil

My new address: http://freethreads.wordpress.com/

2011-06-10T01:12:00.002+05:30

This blog has been moved to: http://freethreads.wordpress.com/

NDMP: An open protocol to backup enterprise data

2011-05-29T21:01:00.000+05:30

NDMP is an open protocol that enables backup/ restore of data in a heterogeneous environment. A real world case: You have two file servers(filer) from NetApp and EMC. And you want to backup your data without bothering about which filer you'd use.

Here NDMP comes into play. If both the filers are running an NDMP server, you can seamlessly carry out backup and restore. You just concentrate on your backup operation and leave out the details of :
- How data is stored ( you need to know data format though)
- What operating system is running on the filer
- Add/ remove a filer with no problem

Extensive information: www.ndmp.org/

VMWare player vs VirtualBox: which is better?

2011-04-08T23:36:00.000+05:30

I have a Windows XP host machine with guest Ubuntu OS. I started with the latest VMWare player and benefits I noticed:

1. Hassle-free installation of Ubuntu
2. Seamless integration between guest and host
o) You can copy-paste files across host and guest!
o) Clipboard is shared bi-directionally

3. Very good performance: applications, network, and devices (DVD)
4. Display scales well on bigger screen with VMWare tools.
5. Alas! you can have at-max one processor simulated

VirtualBox

1. Easy Ubuntu installation
2. Most horrible and pathetic clipboard sharing. Contrary to claimed, it provided one-way clipboard from guest to host.
3. XP clipboard stopped working and never worked till I stopped VirtualBox.
4. Supports VMDK files but it's crappy, buggy and leaves VMDK in an un-usable state. I could never run my VMDK with VMWare later.
5. Can simulate up to 4 cores.
6. Supports shared folder between host and guest. I think it's a generation behind what VMWare provides.
7. Very slow performance: application, system start-up/ shutdown or stand-by. This is the biggest letdown.

My verdict: VirtualBox has miles to go and I am using VMWare still :)

How to Upgrade Ubuntu guest in VmWare?

2011-03-25T08:31:00.000+05:30

o) Get the ALTERNATE Ubuntu image from UBUNTU website. (I prefer using Torrent to get this file)
A DESKTOP ISO will _NOT_ work!
o) Now, it's a blatant lie that you would not need an Internet connection. You got to have it; if not, go and do something else.
o) Okay, once you got you alternate ISO image ready, do the following:

sudo mkdir -p /media/cdrom
sudo mount -o loop ~/Desktop/ubuntu-10.10-alternate-i386.iso /media/cdrom

```
gksu "sh /media/cdrom/cdromupgrade"
```

o) Yup! that's pretty much you need to do.

Corrupted VMWare VMDK file? How to fix?

2011-03-23T20:42:00.000+05:30

o) Create a new VMWare machine of the same guest OS.
o) Assign the corrupted VMDK file as additional HDD to the new machine. VMWare machine config allows you to add an existing HDD (VMDK file) to a machine.
o) Browse your corrupted virtual HDD. Enjoy!

Understanding quirks of C: Structures

2011-02-15T23:03:00.000+05:30

Targeted audience

=============

C programmers

System information

==============

Linux ubuntu 2.6.32-25-generic #44-Ubuntu SMP i686 GNU/Linux

gcc version 4.4.3 (Ubuntu 4.4.3-4ubuntu5)

Structure are the most popular user-defined data type in C. It allows an user to create new data type by packing different data types and use them with a single name. In this article we will discuss fundamental operations provided by C structures and their internal behavior. We'd use following structure for understanding the concepts.

----CODE----

struct test{

char c;

int a;

char d;

};

----CODE----

A variable of type “structure test” would have following memory layout in the process stack. We are assuming that compiler padding is done for a 4-byte boundary.

Structure layout in memory

Let's assign values to this structure variable.

----CODE----

struct test var = {'a', 10, 'b'};

struct test *ptr;

ptr = &var;

printf(“%c”, var.d);

printf(“%c”, *(ptr->d));

----CODE----

How do the member access expression “var.d” and “ptr->d” work?

“var.i”: This expression is converted by the compiler into two components:

a) Base address of symbol 'var' and,

b) Offset of symbol 'd' in the structure memory layout

Base address of the structure variable is “0x0000” in our example. Next, compiler finds the distance of 'd' from the base. The distance is 8 bytes (1+3+4).

Similarly, “ptr->d” is also factored in mentioned manner, and address of member 'd' is returned.

How to calculate a member's byte offset in a given structure

In a trivial manner, we can calculate offset of member 'd' by getting the address of first element of the structure i.e. 'c' and then finding the difference with address of 'd'.

----CODE----

offset_of_d = (&var.d - &var.c);

----CODE----

The difference of these two addresses would fetch us the offset of 'd'. “&(var.c)” fetches us the base address of the structure.

The statement &(var.d) would be treated as = (Base address of variable 'var' + distance of 'd' from the base).

Efficient method to find the offset of a structure member

If we could set the base address to zero in above equation, we would directly get the offset of 'd' i.e. 8.

We can accomplish this as follows:

----CODE----

#define POINTER ((struct test*)(0))

int main()

{

unsigned int x;

x = &(POINTER->d);

printf("\nOffset of d = %d\n", x);

}

----CODE----

The macro “POINTER” expands to “&(((struct test*) (0))->d)”.

We are typecasting '0' as an address, pointing to type of data 'struct test'. Now when you do “&(((struct test*)(0))->d)”, it is fetching the address of 'd' as:

(Base address + offset of d in the structure).

Since base address is set to zero, we are left with the offset of 'd'. We are tricking the compiler by giving base address as zero. This trick will work irrespective of compiler's padding scheme.

Both the discussed approaches lack portability though, and hence may suffer with incorrect output on different hardware. Another portable way to accomplish the same is with the help of “offsetof” macro, defined in header file stddef.h. The macro 'offsetof' accepts two arguments:

– The structure definition, so you don't need to create a structure variable

– The member element, for which the offset is to be calculated.

This macro too takes care of structure padding of elements, performed by the compiler.

------CODE------

#include "stddef.h"

int main()

{

unsigned int x = offsetof(struct test, d);

printf("\nOffset of d = %d\n", x);

}

------CODE--------

How “sizeof” behave in a quaint manner, where syntactically you find a NULL pointer dereference?

Consider the following code:

----CODE-----

struct test{

char c;

int a;

char d;

};

int main()

{

// Define a dangling pointer of type struct test

struct test *p;

// Getting the size of the structure with an uninitialized pointer

printf("%u", sizeof(*p));

printf("%u", sizeof(p));

// Try it with one of implicit types like 'char'

char *i= 'a';

printf("%u", sizeof(*i)); // Displays 1, as the size needed to store a character

}

----CODE----

If you dereference a pointer inside “sizeof” operator, it fetches the size of data-type, the pointer is pointing to, i.e. 12 bytes for “struct test”.

It is because sizeof operator works on “data type”, not data. And interestingly, it works for both internal types and language defined types. So, when you dereference a pointer inside sizeof; you are asking for the number of bytes pointee would need in the memory.

----CODE----

char *c;

// It would fetch you result as 1 byte.

sizeof(*c);

----CODE----

Remember that it is illegal to dereference a dangling pointer otherwise. In the mentioned code, you do not dereference the address carried by the pointer, and hence never get a classic segmentation fault error.

Hope you enjoyed these interesting facts of C. Please share your comments and suggestions.

----References----

[1] http://en.wikipedia.org/wiki/Offsetof

[2] http://en.wikipedia.org/wiki/Struct_(C_programming_language)

How to debug with GDB: Philosophy

2011-02-01T22:50:00.000+05:30

Debugging is an art.

In general, debugging is the most admired tool in the developer's arsenal. My points in this article are related to GNU GDB debugger, for Linux operating system.

What is your first reaction when you see a core-dump?

Let's debug it!

Hold-on. Wait, and first ruminate over the problem. I'd suggest performing a few rituals before treading on to debugging would be helpful in saving time and effort.

o) Try to reproduce the problem. Ensure problem is consistent.

o) Check for the logs emitted by the application. If not available, see if you could get the logs.

o) Note down the application configuration, including the input, system configuration and every tiny detail that makes sense to you.

o) Okay, check the stack-frames with "backtrace".

o) You may inquire about frame locals and dig-in parent frames as well.

o) Also, it's useful to have a look at the code and a 2-3 level dry-run of code to reach at the point of failure. So, say you have a stack-trace as:
foo()
bar()
pop()
top() --> dumps core
hop()

Check how did we reach till top from say bar(). Look out for any ancillary function call in meanwhile and report its side-effect.

Remember, debugging is more than digging-deep; it's understanding the code tree and it's branches. Always approach to the problem on a high level, think what might have caused this failure.

Then, just jump in.

Publication in Linux For You, January, 2011: Python threading and its Caveats

2011-01-29T01:44:00.000+05:30

My friend and I had got the article "Python threading and its Caveats" published in prestigious Linux for You magazine. The article is part of Jan, 2011 edition.

What a great treat of new year!

Best C coding practices: Saving many hours of efforts

2011-01-29T01:40:00.000+05:30

C code has potential to be very compact, often at a price of losing comprehensibility and maintainability.

Let's discuss a few tips that might put smile on your face in a distress time.

o) Always save return value from a function, if available

if(!myfun()) {...}

This is very poor coding style. If myfun() can return one out of hundreds of non-zero error codes, it makes sense to save the return value.

if( (ret = myfun()) != 0 )

This value would help you in
o) zero down the point of return
o) debugging as "ret" is now part of the caller's frame and hence available for examination.

o) An extra log message would cause less harm than being savvy and avoiding it. It is tantamount to have enough log messages embedded in your code at critical points. Trust me, it makes your life so easy. Logging in itself is a vast topic and, I plan to take that separately.

o) Comments should be added generously. Explain what a file, function is doing. Explain the algorithm, input/output parameters of a function and return types.
If your function allocates dynamic memory, who bears the burden of freeing that memory, be explicit and mention that.

o) Keep a function "static" unless you need it in other file of the project. This saves linkers effort.

goto vs return statement: C programming

2010-12-01T23:25:00.000+05:30

It is a classic conflict of opinion when comes to choosing between goto and return.

I feel that goto is treated unfairly, biased, and unjustly. Why people are so paranoid about avoiding goto, is beyond my comprehension.

Let me explain my point with following example:

void foo()
{
if{...}
if{...}
if{...}
if{...}
if{...}
if{...}
if{...}
if{...}
}

Now if I populate the body of "if" with "return"; it is perfectly okay. It creates problems in later point of time.

Now, say you allocate a resource in foo(), where would you free it?

Going with "return" makes you free the resource in all "if" conditions that follow the place where you have allocated the reosurce.

Now, solve this problem with goto. All "if" should goto a single label, which would be placed at the end of the foo().

So, the clean-up work can now be placed under this label. This is much better than "return" statements:

- Code is more exensible. You may add more "if" conditions, without any fear of resource leaks.

- Avoids code duplication

- Even if the current function do not need "goto", I propose to use "goto" for future extensibility.

The best virtualization software: VirtualBox v/s VMPlayer

2010-11-13T21:41:00.000+05:30

I was an avid user of VirtualBox and was pretty happy with it. It gave me freedom to experiment with different OS in a rather safe, transparent, and convenient way. So I used to keep Linux as host and Windows as guest. Performance was good and bugs were non-existent.

Then I found VMPlayer. It surprised me a lot with following reasons:

a) It supported ISO images for Ubuntu, my fav Linux disto.

b) Set up was easier/comparable to VirtualBox.

c) Unity mode: What a feature! A seemless integration of guest OS with host application. You don't have to click the mouse, you never lose window focus. I can now browse through guest OS with Alt+Tab.

d) Very stable, good performance, and I am a happy customer.

How tail -f work?

2010-10-20T23:43:00.000+05:30

"tail -f" is a special command in a way that it polls the specified file for any change and prints the new stuff on the fly. It is very helpful in observing logs and any event based data.

Ever wondered how tail achieves this?

"tail" opens the given file and obtains the file-descriptor. It opens it with xfreopen() -> freopen() -> fopen() call. It does its first round of fstat() on the file as well.

Once it has got the fd, it loops infinitely and do the following:

It does fstat() of the file and observes the mtime value. If the mtime value is changes from the last time..it dumps the data. To print the latest data, it lseek() the file to the last reported file size.

Source: http://git.savannah.gnu.org/cgit/coreutils.git/tree/src/tail.c

How touchpad of a laptop work?

2010-09-19T21:56:00.000+05:30

It was a curiosity to know the internal details of one of the most used 6x6 cm2 part of a laptop. My curiosity increased with diffrent behavior of the touchpad with different objects. For example, it works well with fingers but a pen, pin, or paper are not entertained.

So how does it work?

It is based on concept of capacitive difference on two parallel plates. Each of these plates have a grid of conductors. When we put our finger on the surface, it creates a charge difference on this surface and the difference of capacitance is sensed by the hardware.

This difference is mapped to mouse motion on our screen.

MeeGo: Marriage of Nokia and Intel

2010-09-16T00:29:00.000+05:30

Nokia and Intel are merging their Linux based intiatives: Meamo and Moblin = Meego

* Open source platform to serve mobiles, notebook, TV, and tablets.
* Hosted under Linux Foundation
* Support x86 and ARM
* Visit http://meego.com for more information

But why would people use this new OS? What's wrong with Andriod? How is it better than Symbion, Android, Windows Mobile or BADA?

We already have Linux based mobile OS, so it'd be hard for this new guy to find a decent place.

Uninterrupted Linux session : screen command

2010-09-06T22:37:00.000+05:30

Have you ever faced losing connections to a remote machine(e.g. from a putty) and you happened to be in middle of a script that took ten hours to complete. So what would you do??

Restart the script after re-connecting.

Not anymore...

Linux screen solves this problem with providing a terminal that runs on server and just exported to your putty/Terminal client. In simple terms, you are running your putty on remote machine and watching the output on your local client.

Now, if you client goes down, just chill!

Screen is running your script on remote machine.

To start a screen session:
- Login to remote machine with as you may wish.
- Run $screen
- Do you stuff

Power cut and, no net connection and your client is down.

Once power is back, just re-login to remote machine.
and issue:
$screen -r

It'll list all screen sessions running on the remote machine. Get attached to one of the session with:

$session -r

Now,

Expect Pexpect: Python module to help you automate interactive tests

2010-08-21T21:11:00.001+05:30

Many a times we encounter writing a code that requires user input to proceed. Testing and validating such an application can become tedious and frustating. Python implements an elegant solution to cut down the effort needed to "talk to application".

This is called pexpect module, borrowed from Tcl "expect". Pexpect is a tool for controlling and automoating programs. It simple "fools" the application with an user input. It runs the program and monitors the output. When output matches a given pattern, it respond to application mimicking the human intervention.

Pexpect can be extensively used in testing and automation. I found it particularly useful for interacting with application like ssh, ftp, passwd, telnet etc.

It comes packaged with standard Ubuntu 9.10. The version of Python used by me is:

$python --version
Python 2.6.4rc2

Half century of technical know-how sharing

2010-07-31T20:22:00.000+05:30

It's 50th post on this blog. I am happy with my contribution. Learning is a continuous process and I am an active participant. Today's post will talk of something that I had never imagined to do in my development career: Testing.

Testing of the product is an interesting job, especially when you are a new user. Past two weeks, I am testing the product I'd be developing.

I'm sharing my experience here:

- Don't test just for sake of doing it; Understand the ecosystem in which your product would be working.
- Take generous logs of your testing.
- Keep an eye on what you can improve: Be it a process or a feature.
- Try to automate processes if possible.
- Try feeding weird input to your product.
- Testing is learning. You become a very good user of your product.

Bash Command fo you

2010-06-18T00:58:00.001+05:30

Once again, I am sharing a few good commands with you.

a) Run command in a subshell
$(cd /tmp && ls)

# You will see the contents of /tmp but will stay in PWD only.

b) Reusing cscope database
$cscope -d

c) If you default shell is csh, and you want to move to bash as soon as you login
- Create a file .cshrc in your home directory.
- Edit and add "bash"
- Save and exit.

d) vi can help you see a split screen with two or more files. Try :vsplit.

Ubuntu 9.10 on Windows XP: Virtual Box

2010-05-19T22:27:00.000+05:30

My office laptop has Windows XP installed and I need Windows for sake of Outlook and Communicator. So I started to look out for alternative for having Linux on my laptop. There were three options suggested by different people:

- Get Ubuntu installed and run Windows XP as a guest on VirtualBox.
Now my workplace's IT guys do not install Windows on a VM, so they asked me to create an image of the Windows installed on my laptop. I searched for software to accomplish this and tried a few in vain. I tried a software called WinDD to copy Windows image to my portable HDD and since I did not checked thoroughly where it was copying, I got my portable HDD corrupted. It caused me a lot of pain, my important docs and many pics just got shredded.
Well that was it. I dropped this option.

- Get the Ubuntu on a pen-drive(PD) and make your PD boot-able.
It had a problem that Ubuntu installation was not persistent. Many of my setting could not be saved. Though I got a way to create a persistent image, still it can't save a few things like system time. Verdict: Dropped

- Get the Ubuntu running as a guest on host Windows.
After installing the guest additions, I am very happy with the screen resolution. I got few glitches with network setting and shared folders help. I got over them and will post the details in the next post.

So I am a happy man now, seeing my Ubuntu running on XP. All happy and gay!

PS: I am a fan of VirtualBox and so far have not used any other VM software.

Torrent: Demystified

2010-04-26T21:36:00.000+05:30

What is a torrent?

A torrent is like a broker between a customer and a seller. It works in a peer-to-peer setup that means a true democracy; all are equal, no master-no slave.

So you need to download a classic movie, which may be on a computer(s), located anywhere across the globe.

How would you know where is it?

Torrent comes to help you here. You google for a torrent of this movie. Download the torrent and voila!.

Torrent has three primary components:

a) Seeders: People that have a copy of the movie you want
b) Leechers: That's you!
c) Torrent file : It has information about seeders and leechers.

While you are downloading your movie, torrent share the downloaded part with anyone else interested in downloading the same movie. So effectively every leecher becomes a seeder. Now you are even! Hail Democracy!!

Ubuntu 9.10: Wireless network problem with Broadcom Corporation BCM4312

2010-04-11T22:17:00.000+05:30

Many laptops with Broadcom Corporation BCM4312 802.11b/g hardware shows no response to wireless network with Ubuntu 9.10 Casper.

I could solve this problem with the following method:

a) Go to System-> Administration->Hardware Drivers
Keep your system on Internet(wired ethernet) while doing this.

Install Broadcom 43 and Broadcom STA driver, activate them then.

b) Reboot and see if it work for you.

c) If not, then do the following:

sudo apt-get --purge remove bcmwl-kernel-source
sudo apt-get install patch
sudo apt-get install bcmwl-kernel-source

This should solve the matter. At least it worked for me :)

Ref: Ubuntu Forum
http://ubuntuforums.org/showthread.php?t=1347483
http://ubuntuforums.org/showthread.php?t=1043168

Running Ubuntu 9.10 Casper On your PC

2010-04-07T12:15:00.000+05:30

A complete solution: Image Writing to Installation

Ubuntu is a free open source Linux distribution, primarily for desktop users. There are many reasons to use Ubuntu instead of other Linux distro(Just Google for it).

I am assuming that reader is an Ubuntu user on a x86 system and knows some stuff about Linux.

o) First thing is get the latest Ubuntu bundle(an .iso file) from Ubuntu website.
o) Now get a pen drive.
o) From "Application" menu, run USB creator.
Give the path of iso file and click create USB drive.
+ If pendrive is more than 4GB, You would get an error saying "partition not found":
Just do the following:
* Install gparted utility with Synopsis Bundle Manger
* Create a partition table with "gparted" on your pendrive
* Now create a FAT32 partition of size 1000MB on this pendrive
o) Run USB startup disk creator again.

o) Voila! You are done

Benefits:
o) You save money spent on CD/DVD to write Ubuntu image.
o) This method is very fast and reliable, compared to use of optical media.

Thread stack in heap (pthreads)

2010-03-25T19:37:00.000+05:30

I had known that a thread has a stack. But never thought where is this stack lie in the memory?

Well, it should be process stack. But you can set the thread stack to a heap area.

How to do that?

#include "pthread.h"
#include "stdio.h"
#include "unistd.h"

pthread_mutex_t mutex1;

void* foo(void *arg)
{
int arr[1024*8];

int x = 0;
while(x++ < (8*1024) )
{ arr[x] = x; }

pthread_mutex_lock(&mutex1);
puts("Thread created");
pthread_mutex_unlock(&mutex1);
}

int main()
{
int *p;
void *ret;

p = malloc(1024);

free(p);

pthread_attr_t attr;

pthread_attr_setstack(&attr, p, 2);

pthread_t thread1;

pthread_create(&thread1, NULL, foo, NULL);

pthread_join(thread1, &ret);

free(p);

return 0;
}

But still I am not very sure of this concept.

Stay tuned for more!

Some thoughts on Linux malloc()

2010-03-22T16:39:00.002+05:30

* Always check for equal number of malloc()/free() calls.
• Use calloc() instead of malloc() + memset()
• Instead of allocation memory in a piece-meal manner, try to allocate a bigger chunk with malloc() and keep on resizing it with realloc() if needed.
• malloc() over-commits the memory i.e. it gives you an address which actually does not exist!
• malloc() grouping & reallocing with "more than you want"

Cut short your web-addresses: TinyURL

2010-03-11T12:32:00.000+05:30

An utility called "tinyurl" makes your life easier with shorter URL without creating an all new website. Many times it become cumbersome(i.e. for a long string) and unsafe(if you miss to copy contents of the link) to send a link to your friend.

TinyURl helps you with providing an alias for a long link. Concept is very simple but effective.
e.g. a link is: "http://http://www.blogger.com/post-create.g?blogID=4233584295994918105"

You may get an easier name for this link as: http://tinyurl.com/abc

Here is more information this:
http://tinyurl.com

Exceptional exception handling in C++

2010-02-20T11:21:00.000+05:30

#include

using namespace std;

//Global
int a = 20;

void foo()
try {
throw a;
}
catch(int &x)
{
// You can't modify value of "a"
x = 10;
}

int main()
{
foo();
cout<<"A="<}

A=20 !!!!
When you write "throw a", an invisible, phantom exception object gets created. You pass reference to that object and hence any change made to original object have no effect.

Mutex v/s Semaphore v/s Spinlock

2010-02-19T15:05:00.000+05:30

Similarity

- All are used for synchronization

Difference

Mutex provides one person to access a resource at a time, others must wait in a queue. Once person is done, the guy next in the queue acquire the resource.
So access is serial, one guy after other.

Semaphore provides more than one(up to N) guys to access N resources. So it gives you parallel access to a resource.

Spinlock is an aggressive mutex. In mutex, if you find that resource is locked by someone else, you (the thread/process) switch the context and start to wait(non-blocking).
Whereas spinlocks do not switch context and as soon as resource is free, they go and grab it. In this process of spinning, they consume many CPU cycles. Also, on a uni-processor machine they are useless and perform very badly (do I need to explain that?).

GL: Question

2010-02-04T16:18:00.000+05:30

- fun(int, int);
fun(char, char)
Are they overloaded properly?

- Using static variable in a member function?

- Finding the common node of two link lists?

- Implement BFS, DFS?

- Inorder, preorder is DFS or BFS?

- What is a heap?

- Deleting a node in BST?

- Difference of memcpy() and strcpy()

C Structures: Calculating offset of an element

2010-01-06T19:38:00.001+05:30

Let's say we have the following structure:

struct
{
int a;
char c;
} example;

To calculate offset of 'c', we have following ways:

1. Use the macro offsetof() defined in header file stddef.h.
unsigned int x = offsetof(struct example, c);
This macro takes care of padding of elements done by the compiler.

2. Other way to get the offset f c is:
// C snippet
example var, example *ptr;
ptr = &var;
offset = &(ptr->c)- &(ptr->a)
//

------|base_addr = 0x0000
int a |
------|addr = (0x0000 + 0x0004)
int c |
------|addr = (0x0004 + 0x0004)

We are getting the distance of 'c' from base of the structure. So, address &(ptr->c) should be calcuated by compile as: base_address of var + distance of 'c' from base
If we somehow set base_addr as zero, we will get the offset i.e. 0x0004.

A hack to accomplish this result is as follows:
#define POINTER = ((struct example*)(0))
unsigned int x = (unsigned int) & (POINTER->c);

We are typecasting an address '0' pointing to a type of data 'struct example'.
Now when you do (x)->c, it is equivalent to: BASE_ADDESS(x) + size_of_data elements preceding 'c'. We are tricking the compiler by giving BASE_ADDRESS as zero.

This hack may be incorrect if element padding is done by the compiler.

Signals in Linux

2009-12-15T15:34:00.001+05:30

- List of signals: $kill -l
- You can't handle SIGSTOP and SIGKILL. You can't priorities which signal to handle.
- Signals are generated by setting the appropriate bit in the task_struct's signal field. If the process has not blocked the signal and is waiting but interruptible (in state Interruptible) then it is woken up by changing its state to Running and making sure that it is in the run queue. (linuxhq.com)
- Signals that are sent to a process if an illegal flow of execution happens, are synchronous. They are also called trap e.g. illegal memory access.
- Asynchronous signals are also called interrupts and are sent from a process to another process or thread-to-thread.

Some useful questions

2009-12-14T18:15:00.002+05:30

- Access privileges for private functions in C++?
- Controlling exclusive access to a variable with two processor and two threads?
- Pattern substitution with sed?
- What is md5?
- When do you allocate memory for static variables?
- Phases of compiler, AST?
- When do we allocate storage for static?
- Data hiding and encapsulation?
- Is Vtable per class/object?
- Difference between COFF and ELF?
- Compiler phase of function in-lining?
- Difference between macro and function in-lining?
- Types of parsers?
- YACC is which parser?
- How lex and yacc work?
- How does linker gets info about static variables?
- How protected specifier works in C++?
- What is memory leak?
- Accessing static outside its scope?
- How assembler works?
- When do we do semantic analysis?
- You have a file with write-only access, and you have N threads. How would you ensure sharing of this file among these threads considering performance a prime concern?
- Singleton class? A real world example.
- How would you classify a file based on contents of the class? How many search keyword in the file are necessary?

Bouquet of questions-3

2009-12-01T09:24:00.001+05:30

o How can you add attributes in gcc, e.g., changing function call way from cdecl to stdcall?

GCC allows you to attribute your functions with __attribute__ macro.
This macro allows you to write more readable, clean code. I liked,

- __attribute__((destructor))
- __attribute__((constructor))
- __attribute__((warning("Function definition is not found)))

There are plenty of them, check them out at: http://gcc.gnu.org/onlinedocs/gcc/Function-Attributes.html

o What does following code do?

int flag = 2;
int (*fp)(void) __attribute__((cdecl));
void fun() __attribute__((warning("No definition")));
int main(void)
{
//fun();
fp = main;
puts("called");
while(flag--) {
int x = (*fp)();
}
}

Bouquet of questions-2

2009-11-29T18:41:00.001+05:30

o What is a bus error?

Bus refers to address bus and an error means passing an illegal address to the address bus. There are two signals that can be sent by kernel for an illegal address:
- SIGBUS
- SEGSEGV

A SIGBUS is issued for an obvious wrong virtual address, there is no address translation and CPU can outright say that address is bogus.

SIGSEGV is issued when after translating the address, CPU realizes that address is bogus.

SIGBUS is always better since it avoids address translation. But which signal to issue, depends on CPU.

o What will be the output?

main()
{
int a[10];
printf("%d", sizeof(a);
}

Bouquet of questions

2009-11-06T16:31:00.004+05:30

o What is a cache line?

- Smallest unit of data transfer between cache and main memory.
- Finest level of granularity

o How can I get IPC information over processes?

- Use $ipcs

o Tell the value of enum's elements.

enum e_tag{ a, b, c, d=20, e, f, g=20, h
}var

- a=0, b=1, c=3, d=20, e=21, f=22, g=20, h=21

o Why use volatile?

- To avoid compiler optimization on variables involved in two cases:
+ Shared library
+ Value updated implicitly by hardware

Avoid optimization of Load/Store instructions by the compiler.
e.g., If you have a variable(X) that reads from a shared lib variable(Y), while Y is being updated by other process. Compiler would have no idea of this dependency of external value updates and it may remove these LOAD/STORE instructions during compilation.

Digging Intel Itanium: RSE, Register Stack Engine

2009-10-22T18:13:00.004+05:30

Itanium processors heralded a new era or processing. Though not very successful commercially, IPF has added many wonderful techniques to computer science. I'll discuss the one I liked the most.

It is called RSE (Register Stack Engine), a technique to avoid using main memory during function calls and do the stuff in processor register itself. When a function call is made, calling function passes the arguments that are saved in main memory. After this, the return address is saved. Main memory access is slower when compared to processor speed. Itanium has 128 GPR and out of those 96 are available for RSE. These 96 registers take care of function call mechanism, appearing as a register stack frame to the application. This bypasses memory access till all 96 registers are occupied. Interestingly, processor itself is responsible for running this show and also it's transparent to the application.

More here: http://software.intel.com/en-us/articles/itaniumr-processor-family-performance-advantages-register-stack-architecture

Fast hard drives: How?

2009-10-13T16:25:00.002+05:30

A simple hard drive today is capable of things that sound like some outlandish technology. Just try to do some file I/O in your application and do it with many threads.
Say you have 4 threads, A,B,C, and D. And request to do I/O comes in A then B and so on. If you check the return status of these threads, the ordering might be surprising. Thread D may return before A. How?

Disk have a technology called Native Command Ordering. So they take your request in and process them on a single, simple logic:
-> Serve the one which you can do fastest.
This depends on the head position of the disk. The request that can be served with minimal movement of head, is served first.

A few 'why' answered

2009-10-04T15:50:00.003+05:30

There are certain 'why' that we may have missed, so I am trying to attend them...one by one.

o Why do we need hash table?
Of course, for better, faster search. It could even get us an element in O(1).

But we need to use hash because if given data is in a form which can't be ordered; we need hashing. Example can be images. How will you order a set of images and search any.
Hashing comes to help here. It generates a unique(ideally) hash key for given such data. We save these keys in a hash table. So searching an image is now searching a key in hash table. Keys are generated with a hash function. More about that later.

To be continued with more whys.

HP Caliper : A Profiler

2009-09-22T14:22:00.003+05:30

Developer rarely concern for efficiency of their programming logic. It's not a hard-and-fast observation but that's how most of amateur developer write the code. But performance is becoming critical day by day and commercial applications vie for as much performance gain as possible.

Performance can be hit for motley of reasons but application logic is what we are going to stress here. You may write a code that performs badly with cache system of your hardware or you schedule your threads inappropriately. There may be many reasons that one may not become aware unless someone find it out.

HP Caliper is one of my favorite tool that can help you with finding many causes of application slowdown. It is an Intel Itanium based tool and runs on HPUX & Linux.
Talk about its feature and I may run out of space. It can make basic profiles like sampled call-graph, flat function profile, CPU events profile. Besides it can provide call-stack profile (critical for I/O bound applications), data cache profile (to help you re-layout the data structures).

Best part is that it does not need a recompile of application or any library. Just give it a binary or attach it to a process. Run it and there you are with third party insight in to your logic. It comes with a command line interface and GUI.

Only downside is that it runs on Itanium binaries only so other users have to wait till it become available for them too.

Happy profiling!!

NUI: Mobile GUI development

2009-08-28T19:38:00.003+05:30

I found "nui", a C++ based GUI development framework. You can develop iPhone apps as well. It runs on Linux, Windows, Mac.
http://www.libnui.net/

I also came to know about "enthought" (http://www.enthought.com/). Another GUI development framework. This guy automatically develop the GUI if you provide the class definition.

I've not tried both of these. But they both sound interesting, so thought of sharing :-)

Python questions for interview

2009-08-17T18:47:00.001+05:30

Python is a bliss for quick development. It's a mix of C & C++ features and comes in flavor of a script. For a C/C++ developer who happen to work with Python, here are few interview questions that are frequently asked:

1. what all Python can do for you in OOP?
Python fairly supports OOP principles. You can declare and define classes that follow same philosophy of OOP(similar to C++, Java). Besides features like inheritance, polymorphism are also there.

2. Does it support operator overloading?
You can have operator overloading also.

3. What is pickling?
It's an object serialization technique. Very similar to marshaling/un-marshaling that packet data in networks.

4. Does it support function overloading?
No.

5. How do you pass variable number of arguments to a function?
def foo(*pass_many):
...
In the function you should retrieve the arguments as a tuple.

6. Difference between a tuple and list?
You know it. Tuples are immutable objects while list are mutable.

Highly recommended: http://www.learningpython.com/2008/06/21/operator-overload-learn-how-to-change-the-behavior-of-equality-operators/

Careful with both hands while using fork!

2009-07-21T20:48:00.003+05:30

fork() is one of the most useful feature of C/Linux/UNIX. But it's like a double edged sword, so be careful with fork :-)
Of late, I got stuck in a weird problem with one of the client application(A) that interacts with another application(B). Application A was hanging when used application B; otherwise alone A runs just fine.
Now what to do? We did a thorough examination of both the applications and found that A is waiting on a pipe P. P has its write end with B and A has got the read end. But why this wait? There is no need to keep this pipe open in first place.
So here fork() comes in to picture. Actually A forks B and then B interacts with A. When A fork() B, B gets a copy of all open file descriptors(FD) of A as well. There you go!
After getting these FDs, B does not take care to close them. But A checks if any of its FD is still open. Since the file is open with B, kernel will tell A that some of your files are being accessed. So just wait :( And this wait never ends...
This was it. A simple close() call in B for all FDs worked for us. And B happily got away with A.

A word of advice: Always call exit() from child. exit() does basic clean up and calls _exit() which more work including closing all files open with child.

Just to verify, you can use this test program:

#include "fcntl.h"
#include "stdlib.h"

int main()
{
int fd = -1;
int status;
char buf[512];

fd = open("abc.txt", O_CREAT);

int pid = fork();

if(pid == 0) { // Child
puts("Child says bye");
exit(status);
} else { // Parent
sleep(1);
int ch = read(fd, buf, 16);
printf("\nRead returns %d\n", ch);
exit(status);
}
}

Lichen: Our first invention

2009-07-02T10:34:00.002+05:30

We, a group of three friends have got our first ever invention disclosure published today. It's quite a happy moment for all of us. We started with a small idea that eventually got transformed into a serious paper :)

Title : Lichen: A Framework For Characterizing Sporadic Performance Constrictions In Non-deterministic Applications"

The idea deals with better profiling of application with a special way of sample collection.
Our paper is published in worldwide publication Research Journal. (http://www.researchdisclosure.com)

Terminal problem with csh/ksh

2009-07-01T19:31:00.003+05:30

If you are a bash addict and you have to work in csh/ksh; you may face terminal problem with csh/ksh while using vi/more or any such command. vi expects complete knowledge of the terminal where it's going to display its contents. There is a simple way to fix it.

1. First get the shell type by: $echo $SHELL
This is just to confirm that you are in csh/ksh.

2. Figure out what is the terminal type. If you do not know then ask your system administrator.

3. If you can't, that's also okay. :-) We'll try to use default terminal emulator type called "vt100".

4. Now you need to export the terminal type to your csh/ksh.
$setenv TERM vt100
$tset

5. There you go. Try to open a vi session.

Let me know if you face any issue.

Installation: Fedora core 9

2009-06-03T15:14:00.001+05:30

- Installing printer on FC9
Use 'system-config-printer' command. For more details, refer following link.
http://foo2zjs.rkkda.com/fedora/hp1020.html

How to install Flashplayer in Firefox on Linux?

2009-06-03T12:25:00.004+05:30

Firefox on Linux needs manual installation of Flash player. Surprisingly it's bit messy, so here are a few steps to accomplish this task:

- Get the rpm/deb package downloaded on your system(typically titled adobe-release-i386-1.0-1.noarch.rpm/ flash-plugin-10.0.22.87-release.i386.rpm).

- Install with YUM/aptget like follows:
$yum install adobe-release-i386-1.0-1.noarch.rpm

- Open Firefox browser, and in the address bar type "about:config".
If you see flash player entry in there, you are fine. Else, go to next step.

- Run $updatedb
It'll update the database for "locate" command.

- Run $locate libflashplayer
Most probably it will tell you the directory, "/usr/lib/flash-plugin" directory. Here you will find libflashplayer.so.

- Copy "libflashplayer.so" to Firefox installation directory (e.g. "/usr/local/bin/firefox/plugins/"). Also copy it to "/usr/lib". This will make this library available to all users.

- Again check your Firefox browser with "about:config". You should see a couple of flash player entries there.

That's all. Try running Youtube for verification:-)

Ref: http://www.linuxquestions.org/questions/linux-software-2/how-to-install-flashplayer-in-firefox-487406/

Windows flavor in Linux, YUM

2009-05-29T11:48:00.003+05:30

Linux is bad for naive users especially while installing new software. You always see some dependency missing and all. That's what I used to think. But no more. Linux has given a break from messy-dirty way of installing a software to a very very clean and simple way.

Yes, it's by Fedora 9(New name for good old Redhat), and it's called YUM.
Yum makes installing new RPM(Redhat Packet Manager) butter smooth and a slightly simpler than Windows.
All Yum demands is an Internet connection and lo, you are done.

Just download your desired RPM and specify it to yum like this:

$yum install my.rpm

After asking you for a 'YES', it will take care of everything.

Simple right!

Idiotic scanf() : Scanning whitespaces in a string

2009-05-21T19:42:00.004+05:30

In C, a few functions are real nasty and scanf() is one of them. It's specially bad for inputting strings.
What will happen if you give a string:

-> scanf("%s", string)

-> Input: "C is stupid".

Well, you will have only "C" in the "string".

Reason: It happens because scanf() considers white-spaces as delimiter. So "C" being followed by a white space is considered as the only input. Simple, isn't it?

Remedy: We have something like follows:

int main()
{
char arr[10];
scanf("%*[ \t\n]%s", arr);
puts(arr);
}

IT DOESN'T WORK!!!

Only way is this:

#define SIZE 128

int main() {
char arr[SIZE];
char ch;
int index = 0;

while(ch != '\n' && index < SIZE)
{
ch = getchar();
arr[index] = ch;
index++;
}
arr[index] = '\0';
}

Done!

GNU gdb: The toolbox of commands

2009-04-15T20:11:00.002+05:30

Many a times application throws signals during its execution. By default gdb has some setting for all UNIX defined signals. If this default is "stop" the application, it becomes quite irritating.
To handle a signal: (gdb) handle SIGUSR1 nostop
All available options are:
nostop
GDB should not stop your program when this signal happens. It may still print a message telling you that the signal has come in.

stop
GDB should stop your program when this signal happens. This implies the print keyword as well.

pass
noignore
GDB should allow your program to see this signal; your program can handle the signal, or else it may terminate if the signal is fatal and not handled. pass and noignore are synonyms.

nopass
ignore
GDB should not allow your program to see this signal. nopass and ignore are synonyms.

(Ref: http://sources.redhat.com/gdb/current/onlinedocs/gdb_6.html#SEC44)

Ruby on Rails

2009-04-15T15:01:00.006+05:30

Bored of J2EE? I came across a new web app development framework called Ruby on Rails. Based on Agile development philosophy, it's one of the most popular web framework today.

As name suggests, it let's you develop your web application faster and conveniently. All you need to know Ruby before getting started.

Check out more at: http://rubyonrails.org/

Linux Scheduling: A Few Facts

2009-03-17T13:57:00.002+05:30

- Linux scheduler favors I/O bound processes. It uses dynamic priorities to schedule processes. So a process that has not got CPU for a long time, would get its priority increased and vice verse.

- Processes are moved to different queues and al processes on ready queue are assigned an 'epoch'. The epoch is relevant for processes in ready queue only.

- Now each process is assigned a quantum which is the CPU time allotted to a process. If a process is blocked, it does not use its quantum and unused quantum is carry forward to next epoch. An epoch completes as soon as all processes in ready queue complete their quantum.

- Dynamic priority("goodness") of a process is calculated by base priority and quantum. Hence a I/O bound process which is blocked for a long time, gets its priority improved every time it saves its quantum while it was blocked.

Linux Tips

2009-02-17T14:23:00.003+05:30

- Changing Linux root password
http://www.paulspoerry.com/2008/09/06/replace-linux-root-password/

- Ways of saying 'Hello! World' : http://www.fitzrovian.com/hello.html
- sizeof (type) requires parenthese, while sizeof expression
does not.

- How to easily read a declaration from left to right:
transform function argument types from inside out first
move the base type to the end
add outer parentheses if there's an initial *
change every (*...) to ... ->
one -> for each *
move qualifiers, so * const becomes const ->

Example: const int *(**const x [])()

*(**const x [])() const int base type to end
(*(**const x [])()) const int add outer parens
(**const x [])() -> const int remove outer ()
x [] const -> -> () -> const int remove inner ()

array of constant pointers to pointers to functions
returning pointers to constant ints

- p + 1 == &p [1]

-

Unfolding 2-D array in C

2009-02-10T14:03:00.003+05:30

Last weekend my friends and I had a discussion on 2-D array in C. Surprisingly it went long as everyone had some points and knowledge to share :) Finally we could gather some important information and understood this pearl of C better.

I'll clear this idea with C code snippets.

// Declare an array
int arr[2][3]= {1,2,3,4,5,6};

It's size is 2*3*sizeof(int) = 24 bytes (Intel x86)

okay, now what will get printed for following statements:
// Say starting address is 1024

o) print &a + 1
o) print a + 1
o) print *a + 1

To answer these questions better, I'll explain what exactly is a 2-D array. When we declare and define a 2-D array, it's like a new user defined type to C. It's a pointer to a memory location that contiguously contains storage of 6 integers. Logically it's like holding many 1-D array in a followed by one-another.

Ans 1: When you say '&a' , it implies address of user defined type and it's 1024. Now incrementing it by 1 means moving the pointer by the size of this type which is 24 bytes. Thus it'll print 1048.

Ans 2: 'a' fetch us address of 1-D array or 0th row of our matrix. a+1 would get us address of next row i.e. 1024 + (3*4) = 1036

Ans 3: '*a' would get you the 0th element of 0th row which is 1. So *a + 1 would get us 2.

More Linux Tips

2009-01-15T19:25:00.002+05:30

- Detailed info about system: Actually these are kernel messages and are stored as logs in /bin/dmesg.
$dmesg

- Determining runlevel {Shows previous & current runlevel}
$runlevel
0->Halt 1->Single User Mode
2-> Multiuser w/o NFS 3-> Full Multiuser 6-> Reboot

To modify init level-> /etc/inittab

- To switch runlevels, use init command.
$init 3

- Difference between Paging and Swapping
Paging refers to movement of pages to frames on the disk. It's a considerably cheaper operation. Swapping means to move entire address space of a process to disk. This is very expensive operation and happens when a process sleeps or when thrashing occurs.

Linux Linker Unveiled: The Secret Services

2009-01-12T19:16:00.002+05:30

Linux linker does not seem to be a lead actor in play of application. But it work like the script which is essential to success of the play. Linker(ldd) is a piece of software that is primarily responsible for address patching a.k.a. relocation. Object files bear absolute and relative addresses.

// test.c
int main()
{
foo();
}

Disassembly of section .text:

00000000

:
0: 55 push %ebp
1: 89 e5 mov %esp,%ebp
3: 83 ec 08 sub $0x8,%esp
6: 83 e4 f0 and $0xfffffff0,%esp
9: b8 00 00 00 00 mov $0x0,%eax
e: 29 c4 sub %eax,%esp
10: e8 fc ff ff ff call 11
15: c9 leave
16: c3 ret

# An object file always starts on address 0x0000.

You can see that addresses are relative to the module and called function is completely unknown to foo. Linker take these object files, relocate their addresses to loading addresses(virtual addresses) and resolve calls to function by a function offset table.

To get a function offset table:
$objdump -x ./test.o

RELOCATION RECORDS FOR [.text]:
OFFSET TYPE VALUE
00000011 R_386_PC32 foo

All functions called from a file are listed in this table and here foo() has got 0x11 offset. Once all object files are there linker patches the offset with virtual address. Though call to foo() has been patched with correct virtual address, offset of the instruction is still the same(main+0x11).

08048344

:
8048344: 55 push %ebp
8048345: 89 e5 mov %esp,%ebp
8048347: 83 ec 08 sub $0x8,%esp
804834a: 83 e4 f0 and $0xfffffff0,%esp
804834d: b8 00 00 00 00 mov $0x0,%eax
8048352: 29 c4 sub %eax,%esp
8048354: e8 03 00 00 00 call 804835c
8048359: c9 leave
804835a: c3 ret
804835b: 90 nop

0804835c :
804835c: 55 push %ebp
804835d: 89 e5 mov %esp,%ebp
804835f: 5d pop %ebp
8048360: c3 ret

Linux Secrets!

2009-01-09T17:37:00.004+05:30

1. Linux uses COW scheme with virtual memory management.
2. Threads in Linux can be LinuxThreads, NPTL(Redhat). NPTL is more efficient and from kernel 2.6 onwards it'll be used. Using env variable LD_ASSUME_KERNEL you can decide which thread library to choose.
3. Linux kernel do not discriminate between threads and processes while making scheduling decision.
4. Memory allocation of a process can be seen with 'pmap' command.
5. Linux CPU scheduler is O(1) scheduler i.e. regardless of number of processes, it always take a constant time to select a process.
6. Linux memory management for IA-32 can address only 1GB of physical memory. Beyond this memory has to mapped to the 1GB range hence allocation a page beyond 1GB degrades performance.
7. IA64 can address 64-128GB of memory.
8. Linux allocates most part of the disk's free space to swap. This improves the performance of VMM.
9. It follows Buddy System for page allocation which try to keep memory address contiguous. Have a loot at /proc/buddyinfo.
10. If no free page is available, kswapd kernel thread reclaims free pages. This thread is used by buddy system and follows LRU. Kernel pages are never swapped out of memory.

Stack Unwinding

2009-01-08T19:09:00.003+05:30

A very nice explanation of what stack unwinding is and why is it needed.
http://docs.hp.com/en/B9106-90012/unwind.5.html

Linux Memory Management Secrets!

2009-01-07T19:53:00.004+05:30

Tips to Improve Dynamic Memory Performance

- Instead of using memset() to initialize malloc()'ed memory, use calloc(). Because when you call memset(), VM system has to map the pages in to memory in order to zero initialize them. It's very expensive and wasteful if you don't intend to use the pages right away.
calloc() reserves the needed address space but does not zero initialize them unless memory is used. Hence it postpones the need to load pages in to memory. It also lets the system initialize pages as they’re used, as opposed to all at once.

- Lazy allocation: A global(normal variable or a buffer) can be replaced with a static and a couple of functions to allow its access.

- memcpy() & memmove() needs both blocks to be memory resident. Use them if size of blocks is small(>16KB), you would be using the blocks right away, s/d blocks are not page aligned, blocks overlap.
But if you intend to postpone the use, you would increasing the working set of the application. For small amount of data, use memcpy().

- To check heap dysfunctional behavior: $ MALLOC_CHECK_=1 ./a.out
It'll give an address related to each violation of dynamic memory routines.

- Electric fence : Works very well with gdb

- Libsafe: for libc routines

Rational Clearcase: Some Useful Commands

2008-10-15T15:47:00.005+05:30

Hi,
Rational Clearcase is perhaps the most popular version management software across various organizations. Here is my humble attempt to capture most useful commands that can save a lot of your time.

a) Creating an element
$cleartool mkelem -eltype{binary, directory etc.} elem_name

b) Creating many elements at once or say you want to convert your non-clearcase element as versioned.
- first copy & paste all such files/dir in a destination directory.
$find -type f/d |xargs cleartool mkelem eltype elem_name

b) Browsing files in your branch
$ct find ./ -branch 'brtype(_view_name)' -print

c) Soft Links
$cleartool ln -s target link_name

To update a link, remove it first($ cleartool rmname link_name), create a new link with same/different in a checked-out directory.

d) Merging changes in main branch
You have two options:
1. Use cleartool merge
2. Use diff/patch OR vimdiff

cleartool merge
---------------
- Checkout file with your changes.
- $cleartool merge -to file_with_your_changes -version main_branch_version

vimdiff
-------
Amazingly simple and useful.

- Checkout file with your changes.
- $vimdiff file_with_your_changes file_on_main_branch
- You can make out the changes which are maked with different colors.
- Look the file on right side pane, a red color signifies some change in the line
- Merge changes with :,+1 diffget
- Moreover you can use any vi command.

Hope these inputs would help people.

Thanks a lot!