My new address: http://freethreads.wordpress.com/

This blog has been moved to: http://freethreads.wordpress.com/

NDMP: An open protocol to backup enterprise data

NDMP is an open protocol that enables backup/ restore of data in a heterogeneous environment. A real world case: You have two file servers(filer) from NetApp and EMC. And you want to backup your data without bothering about which filer you'd use.

Here NDMP comes into play. If both the filers are running an NDMP server, you can seamlessly carry out backup and restore. You just concentrate on your backup operation and leave out the details of :
- How data is stored ( you need to know data format though)
- What operating system is running on the filer
- Add/ remove a filer with no problem

Extensive information: www.ndmp.org/

VMWare player vs VirtualBox: which is better?

I have a Windows XP host machine with guest Ubuntu OS. I started with the latest VMWare player and benefits I noticed:

1. Hassle-free installation of Ubuntu
2. Seamless integration between guest and host
     o) You can copy-paste files across host and guest!
     o) Clipboard is shared bi-directionally

3. Very good performance: applications, network, and devices (DVD)
4. Display scales well on bigger screen with VMWare tools.
5. Alas! you can have at-max one processor simulated

VirtualBox

1. Easy Ubuntu installation
2. Most horrible and pathetic clipboard sharing. Contrary to claimed, it provided one-way clipboard from guest to host.
3. XP clipboard stopped working and never worked till I stopped VirtualBox.
4. Supports VMDK files but it's crappy, buggy and leaves VMDK in an un-usable state. I could never run my VMDK with VMWare later.
5. Can simulate up to 4 cores.
6. Supports shared folder between host and guest. I think it's a generation behind what VMWare provides.
7. Very slow performance: application, system start-up/ shutdown or stand-by. This is the biggest letdown.

My verdict: VirtualBox has miles to go and I am using VMWare still :)

How to Upgrade Ubuntu guest in VmWare?

o) Get the ALTERNATE Ubuntu image from UBUNTU website. (I prefer using Torrent to get this file)
A DESKTOP ISO will _NOT_ work!
o) Now, it's a blatant lie that you would not need an Internet connection. You got to have it; if not, go and do something else.
o) Okay, once you got you alternate ISO image ready, do the following:

• sudo mkdir -p /media/cdrom
  sudo mount -o loop ~/Desktop/ubuntu-10.10-alternate-i386.iso /media/cdrom

• gksu "sh /media/cdrom/cdromupgrade"

o) Yup! that's pretty much you need to do.

Corrupted VMWare VMDK file? How to fix?

o) Create a new VMWare machine of the same guest OS.
o) Assign the corrupted VMDK file as additional HDD to the new machine. VMWare machine config allows you to add an existing HDD (VMDK file) to a machine.
o) Browse your corrupted virtual HDD. Enjoy!

Understanding quirks of C: Structures

Targeted audience

=============

C programmers

System information

==============

Linux ubuntu 2.6.32-25-generic #44-Ubuntu SMP i686 GNU/Linux

gcc version 4.4.3 (Ubuntu 4.4.3-4ubuntu5)

Structure are the most popular user-defined data type in C. It allows an user to create new data type by packing different data types and use them with a single name. In this article we will discuss fundamental operations provided by C structures and their internal behavior. We'd use following structure for understanding the concepts.

----CODE----

struct test{

  char c;

  int a;

  char d;

};

----CODE----

A variable of type “structure test” would have following memory layout in the process stack. We are assuming that compiler padding is done for a 4-byte boundary.

 Structure layout in memory

Let's assign values to this structure variable.

----CODE----

struct test var = {'a', 10, 'b'};

struct test *ptr;

ptr = &var;

printf(“%c”, var.d);

printf(“%c”, *(ptr->d));

----CODE----

How do the member access expression “var.d” and “ptr->d” work?

“var.i”: This expression is converted by the compiler into two components:

          a) Base address of symbol 'var' and,

          b) Offset of symbol 'd' in the structure memory layout

Base address of the structure variable is “0x0000” in our example. Next, compiler finds the distance of 'd' from the base. The distance is 8 bytes (1+3+4).

Similarly, “ptr->d” is also factored in mentioned manner, and address of member 'd' is returned.

How to calculate a member's byte offset in a given structure

In a trivial manner, we can calculate offset of member 'd' by getting the address of first element of the structure i.e. 'c' and then finding the difference with address of 'd'.

----CODE----

offset_of_d = (&var.d - &var.c);

----CODE----

The difference of these two addresses would fetch us the offset of 'd'. “&(var.c)” fetches us the base address of the structure.

The statement &(var.d) would be treated as = (Base address of variable 'var' + distance of 'd' from the base).

Efficient method to find the offset of a structure member

If we could set the base address to zero in above equation, we would directly get the offset of 'd' i.e. 8.

We can accomplish this as follows:

----CODE----

#define POINTER ((struct test*)(0))

int main()

{

  unsigned int x;

  x = &(POINTER->d);

  printf("\nOffset of d = %d\n", x);

}

----CODE----

The macro “POINTER” expands to “&(((struct test*) (0))->d)”.

We are typecasting '0' as an address, pointing to type of data 'struct test'. Now when you do “&(((struct test*)(0))->d)”, it is fetching the address of 'd' as:

                      (Base address + offset of d in the structure).

Since base address is set to zero, we are left with the offset of 'd'. We are tricking the compiler by giving base address as zero. This trick will work irrespective of compiler's padding scheme.

Both the discussed approaches lack portability though, and hence may suffer with incorrect output on different hardware. Another portable way to accomplish the same is with the help of “offsetof” macro, defined in header file stddef.h. The macro 'offsetof' accepts two arguments:

– The structure definition, so you don't need to create a structure variable

– The member element, for which the offset is to be calculated.

This macro too takes care of structure padding of elements, performed by the compiler.

------CODE------

#include "stddef.h"

int main()

{

  unsigned int x = offsetof(struct test, d);

  printf("\nOffset of d = %d\n", x);

}

------CODE--------

How “sizeof” behave in a quaint manner, where syntactically you find a NULL pointer dereference?

Consider the following code:

----CODE-----

struct test{

  char c;

  int a;

  char d;

};

int main()

{

  // Define a dangling pointer of type struct test

  struct test *p;

  // Getting the size of the structure with an uninitialized pointer

  printf("%u", sizeof(*p));

  printf("%u", sizeof(p));

  // Try it with one of implicit types like 'char'

  char *i= 'a';

  printf("%u", sizeof(*i)); // Displays 1, as the size needed to store a character

}

----CODE----

If you dereference a pointer inside “sizeof” operator, it fetches the size of data-type, the pointer is pointing to, i.e. 12 bytes for “struct test”.

 It is because sizeof operator works on “data type”, not data. And interestingly, it works for both internal types and language defined types. So, when you dereference a pointer inside sizeof; you are asking for the number of bytes pointee would need in the memory.

----CODE----

char *c;

// It would fetch you result as 1 byte.

sizeof(*c);

----CODE----

Remember that it is illegal to dereference a dangling pointer otherwise. In the mentioned code, you do not dereference the address carried by the pointer, and hence never get a classic segmentation fault error.

Hope you enjoyed these interesting facts of C. Please share your comments and suggestions.

----References----

[1] http://en.wikipedia.org/wiki/Offsetof

[2] http://en.wikipedia.org/wiki/Struct_(C_programming_language)

How to debug with GDB: Philosophy

Debugging is an art.

In general, debugging is the most admired tool in the developer's arsenal. My points in this article are related to GNU GDB debugger, for Linux operating system.

What is your first reaction when you see a core-dump?

Let's debug it!

Hold-on. Wait, and first ruminate over the problem. I'd suggest performing a few rituals before treading on to debugging would be helpful in saving time and effort.

o) Try to reproduce the problem. Ensure problem is consistent.

o) Check for the logs emitted by the application. If not available, see if you could get the logs.

o) Note down the application configuration, including the input, system configuration and every tiny detail that makes sense to you.

o) Okay, check the stack-frames with "backtrace".

o) You may inquire about frame locals and dig-in parent frames as well.

o) Also, it's useful to have a look at the code and a 2-3 level dry-run of code to reach at the point of failure. So, say you have a stack-trace as:
foo()
bar()
pop()
top() --> dumps core
hop()

Check how did we reach till top from say bar(). Look out for any ancillary function call in meanwhile and report its side-effect.

Remember, debugging is more than digging-deep; it's understanding the code tree and it's branches. Always approach to the problem on a high level, think what might have caused this failure.

Then, just jump in.

Publication in Linux For You, January, 2011: Python threading and its Caveats

My friend and I had got the article "Python threading and its Caveats" published in prestigious Linux for You magazine. The article is part of Jan, 2011 edition.

What a great treat of new year!

Best C coding practices: Saving many hours of efforts

C code has potential to be very compact, often at a price of losing comprehensibility and maintainability.

Let's discuss a few tips that might put smile on your face in a distress time.

o) Always save return value from a function, if available

if(!myfun()) {...}

This is very poor coding style. If myfun() can return one out of hundreds of non-zero error codes, it makes sense to save the return value.

if( (ret = myfun()) != 0 )

This value would help you in
o) zero down the point of return
o) debugging as "ret" is now part of the caller's frame and hence available for examination.

o) An extra log message would cause less harm than being savvy and avoiding it. It is tantamount to have enough log messages embedded in your code at critical points. Trust me, it makes your life so easy. Logging in itself is a vast topic and, I plan to take that separately.

o) Comments should be added generously. Explain what a file, function is doing. Explain the algorithm, input/output parameters of a function and return types.
If your function allocates dynamic memory, who bears the burden of freeing that memory, be explicit and mention that.

o) Keep a function "static" unless you need it in other file of the project. This saves linkers effort.