syntax highlight

Tuesday 29 June 2010

Vim tips: make things work again

Are you using :make within vim? If you are, then you probably noticed that getting lots of compiler errors is not uncommon, and that vim has a tendency of jumping to the first error found... but what happens if you want to see subsequent errors too, because you're a really crappy programmer and need to find the 10 or 20 errors you just introduced with your single line edit?

Luckily Vim is always the anwser, just use cnext and cprev until you reach the error you are looking for.

Friday 25 June 2010

There's still hope

A log time ago, in a galaxy far away, I posted a list of stuff I apt-get'd as soon as my Buguntu install was ready. Well, fast forward to KK and this is the new apt-get script:

sudo apt-get install opera mocp unclutter build-essential gimp \
        mmv gnome-do knetworkmanager xdiskusage iptraf sshfs \
        shntool cuetools flac wavpack unrar vlc vim gvim vim-doc \ 
        vim-full vim-gnome ddd devhelp doxygen doxygen-gui \
        exuberant-ctags ctags  subversion ifstat antiword

Quite a progress I'd say, as these are mostly programs I use which don't come in the default install (even though they should). You know, I even have hopes for the sound system in my notebook this time...

Wednesday 23 June 2010

Nooooooooooo

(gvim:13664): Gdk-CRITICAL **: gdk_window_get_user_data: assertion `GDK_IS_WINDOW (window)' failed
Vim: capté una señal de muerte SEGV
Vim: finalizado.

Tuesday 22 June 2010

C++ pretty functions

There are two well known macros from the preprocessor which every macro-sorcer must know. They are __FILE__ and __LINE__. You probably already know about them but anyway, __FILE__ will give you the current file and __LINE__ the current line. Easy, huh?

int main() {
   printf("%s : %i", __FILE__, __LINE__);
   return 0;
}

The program above would give you "main.cpp : 3" as a result. There is nothing going on at execution time, it's all preprocesor wizardy. In fact with "g{++/cc} -E" you can even check what the "real" output is (-E means to return the preprocessor output. Keep in mind a lot of stuff will be included from the headers you use).

int main() {
   printf("%s : %i", "main.cpp", 3);
   return 0;
}

Well that's nice and all, but g++ can top this easily:

int main() {
   std::cout << __PRETTY_FUNCTION__ << "n";
   return 0;
}

There are a couple of notable things about this new "pretty function" thing:

  • 1. It will demangle a function's name
  • 2. This time it isn't a preprocessor secret thing but a real variable g++ will create.

You can easily use this for better logging functions now (with some macro wizardy, obviously).

Thursday 17 June 2010

Template metaprogramming X: Zero Minus Ten

So far we've learned the basic constructs of template metaprogramming (loops, branching, return values) and some basic list operations (getting the length of a list, appending and prepending elements, checking if an element is included in a list). Let's put it all together by creating an operation to return the position of an element. It'll be very useful later on too.

If we go back to the Includes operation we can get some help to define the Position operation: the position of an element in a list is one plus the position of the element we're searching for in the tail, or zero if the head equals said element. The operation is not defined if the element is not in the list.

Translating to pseudo-code:

Position (lst.head, lst) <- 0
Position (e, lst) <- 1 + Position(e, lst.tail)

The translation to C++ is not so trivial this time. Try it, I'll wait... ready? OK, let's start

template <class Elm, class Lst> struct Position {
	typedef typename Lst::head Head;
	typedef typename Lst::tail Tail;
	static const bool found = (Head == Elm);
	static const int result = found? 0 : 1 + next;
	static const int next = Position<Elm, Tail>::result;
};

Looks easy... but doesn't work. First problem, we can't compare two types, remember? We need to use Eq again. Second problem, although we said the operation is undefined if the element is not included on the list, it would be nice if we could force the compiler to fail if (or when) that happens. Let's rewrite the operation using a façade again, but adding an Assert:

template <typename Elm, typename LST> struct _Position {
	typedef typename LST::head Head;
	typedef typename LST::tail Tail;

	static const bool found = Eq<Elm, Head>::result;
	static const int result = (found)? 0 : 1 + _Position<Elm, Tail>::result;
};

template <typename Elm, typename LST> struct Position {
	typedef typename Assert<Includes< Elm, LST >::result>::check include;
	static const int result = _Position<Elm, LST>::result;
};

Oh, we haven't defined assert yet! There's another problem, too: even if it won't compile, the compiler will try to expand _Position< ..., NIL > indefinitely, causing an error after too many nested template calls. Not nice. We need to add a case to make the compiler stop:

/******************************************************/

// Helper: Will fail to compile if the assert is false
class Assertion{};
template <bool cond, class T=Assertion> struct Assert {
	typedef typename T::fail check;
};
template <> struct Assert<true> {
	typedef void check;
};

/******************************************************/

template <typename Elm, typename LST> struct _Position {
	typedef typename LST::head Head;
	typedef typename LST::tail Tail;

	static const bool found = Eq<Elm, Head>::result;
	static const int result = (found)? 0 : 1 + _Position<Elm, Tail>::result;
};

// The compiler will try to expand the position check
// after NIL has been reached if this isn't here
template <typename Elm> struct _Position<Elm, NIL> {
	static const int result = 0;
};

template <typename Elm, typename LST> struct Position {
	typedef typename Assert<Includes< Elm, LST >::result>::check include;
	static const int result = _Position<Elm, LST>::result;
};

All that code for such a simple operation, man. Also, see what we did with Assert<>? It seems making a compile fail is actually quite easy. That's what I have most experience with.

We've been through quite a lot, and our toolboox should be quite big already. Next time we'll start steering towards some sort of applicability, trying to use some of all these stuff to implement a real, useful and working program... assuming that's even possible.

Tuesday 15 June 2010

Binary portability in Linux

An interesting topic for a change: is Linux binary portable? That is, can we take a binary file and be sure it'll run in any other Linux system? What happens if we broaden that to any POSIX system, will it blend? Eh, I mean, will it run?

Doing some research on the subject I wrote down a list of the thought process which led my to an (inconclusive) answer:

  1. First we should define what a binary is for us: When we talk about a binary we are usually thinking about a compiled binary file, not an interpreted script file like Ruby or Python. Those are for people who like things to actually work, so let's focus on a compiled executable file, like a C/C++ application.
  2. Defining compiled file: What could it be other than a sequence of bytes the microprocessor can understand? Yes, that's right, it's sort of interpreted code, only there's electronics behind, not more code. This brings us to the first interesting conclusion: the executable must be (leaving emulators aside) compatible with the architecture you're on. Running Sparc? Well then, the binary better be compiled for Sparc because otherwise to the uP will not make any sense.
  3. Format: as any other thing, a binary file must have a format. That is a standard which defines the structure the file will follow. ELF is the binary format for Linux and it's quite standard. Of course, if the binary format is a standard then we should get perfect portability between different platforms running on equal architecture. Unfortunately that's not the case.
  4. (Cont'd) Why don't we? The binary depends not only on compile time "stuff" but a loading time linking occurs: the executable binary will get linked with the system files like glibc, or any other dependency on a shared library it may have.

So, what are the keypoints for Linux binary portability? Architecture, binary format and system libraries.

Of course, making the executable run is only part of the equation, as running and segfaulting on the spot is not so nice either. For this last part you'll have to closely follow the standards defined by POSIX for paths and stuff like that.

Epilogue

As an epilogue, we could add that Windows binary compatibility tends to be great. Running binaries from 12 years back is no small feat, yet this leads to a whole lot of other problems: an incredible complex loader, security bugs, backwards compatibility headaches, et al. The old new thing is a great source of information for this topics, I'm quite illiterate about Windows binaries nowdays :)

Followup links

Friday 11 June 2010

Vim tips: make things work

So, you are an uber console geek, using only vim and the command line to compile all your projects, execute the tests, blah blah blah... if only you could squeeze that microsecond lost whenever you switch from vim to compile you'd be 1e-4 seconds more productive... oh, wait, you can!

Whenever you think you're project is good enough to compile just hit :make to be proven wrong. Type :make test to run your tests (because you are using TDD, aren't you?) and watch all those red flags fly by.Also, add the following mapping to your ~/.vimrc for an extra happy coding session:

map <F5> :make
map <F6> :make test

Thursday 10 June 2010

Template metaprogramming IX: Absolute Zero

By now we should have learned how to perform loops, branching and returns using templates. Let's add a couple of useful operations to our library: append and prepend.

Prepending an element to a list is very easy: the result is a list (oh surprise) consisting of a head (the element we want to add) and a tail (the old list). In the pseudocode I've been using so far:

Prepend(e, lst) <- LST(e, lst)

And in C++ (trivial, this time):

template <typename Elm, typename Lst=NIL> struct Prepend {
	typedef LST<Elm, Lst> result;
};

Appending is a little bit more difficult, as we need to first find the end of the list. Think for a second how would you define it... back? Ok, I'd define it this way: appending an element to the list yields a list, consisting of the same head and the result of appending said element to the tail. The null case, as usual, is appending an element to a NIL list; in this case the result is a list with the element itself. So:

Append(e, NIL) <- LST(e)
Append(e, lst) <- LST(lst.head, Append(e, lst.tail))

Looks complicated but it follows the same structure as the rest of the basic-ops:

template <class Elm, class Lst> struct Append {
	typedef typename Lst::head Head;
	typedef typename Lst::tail Tail;

typedef typename Append<Elm, Tail>::result Next; typedef typename LST<Head, Next>::result result; };

template <class Elm> struct Append<Elm, NIL> { typedef LST<Elm> result; };

Easy. Now, what happens if we want to add a default value for Lst, so we can use Append to create lists? Easy too, but we need a façade this time; just rename Append to _Append, then

// This is here just because I wanted a default param :D
template <typename Elm, typename Lst=NIL> struct Append {
	typedef typename _Append<Elm, Lst>::result result;
};

I promised to add one more operation to our toolbox, returning the position of an element, but this post is getting quite long and I'm afraid it may be too much for the average attention span of a programmer... we'll leave it for next time.

Tuesday 8 June 2010

Just WTF

Who the hell can create a monster like this one?
CREATE PROC shutdown10
AS
EXEC xp_cmdshell 'net send /domain:SQL_USERS ''SQL Server
shutting down in 10 minutes. No more connections
allowed.', no_output
EXEC xp_cmdshell 'net pause sqlserver'
WAITFOR DELAY '00:05:00'
EXEC xp_cmdshell 'net send /domain: SQL_USERS ''SQL Server
shutting down in 5 minutes.', no_output
WAITFOR DELAY '00:04:00'
EXEC xp_cmdshell 'net send /domain:SQL_USERS ''SQL Server
shutting down in 1 minute. Log off now.', no_output
WAITFOR DELAY '00:01:00'
EXEC xp_cmdshell 'net stop sqlserver', no_output

Thursday 3 June 2010

Template metaprogramming VIII: A Rough Whimper of Insanity

Remember last time? We learned how to get the lenght of a list. This time I'll introduce some more of these basic ops. Let's begin with "Nth": getting the Nth element of a list; which, remember, in this case is a type, not a concrete element. This means the Nth element will be something like int, char, const char*, not 1, 2 or 3. We introduced a trick to get around this limitation before using a template , go there to refresh your memory if needed.

So, what would the coloquial definition of "Nth" be? I'd put it like "The operation Nth for a list equals the head of the list for N = 0 and Nth (minus one) of the tail otherwise". A little bit more formally:

Nth(0, lst) <- lst.head
Nth(n, lst) <- Nth(n-1, lst.tail)

Translating this to C++ should be a breeze to you now. Try it, I'll wait. Read? OK, this is MY answer:

template <typename LST, int N> struct Nth {
	typedef typename LST::Tail Tail;
	typedef typename Nth<Tail, N-1>::result result;
};

template <typename LST> struct Nth<LST, 0> { typedef typename LST::head result; };

Though the structure is very similar to the previous "basic operation", getting the length of a list, the concept is quite different. This time we're defining a return type recursively. Anyway, it was too easy indeed, let's try a more complex operation now.

How can we check if an element exists on a list? Seems easy enough, an element is included in a list if the head equals the element itself or if the element is included in the tail. In the pseudo language I just invented:

Includes(lst.head, lst) <- true
Includes(e, lst) <- Includes(e, lst.tail)

Looks easy, right? Well, there's a bug there, can you spot it? Yeah, we're missing the false condition. We should add a third specialization:

Includes(lst.head, lst) <- true
Includes(e, NIL) <- false
Includes(e, lst) <- Includes(e, lst.tail)

Again, let's translate the pseudocode to C++. Try it, I'll wait. Read? OK, this is MY answer:

template <class Elm, class Lst>
struct Includes {
	typedef typename LST::head Head;
	typedef typename LST::tail Tail;

	static const bool found = (Elm == Head);
	static const bool found_tail = Includes<Elm, Tail>::result;
	static const bool result = found || found_tail;
};

template <class Elm> struct Includes <Elm, NIL> {
	static const bool result = false;
};

Looks nice, doesn't it? Too bad it won't work, you can't compare two types. What would (int == char) mean in C++? We need a helper there, some kind of trick to compare two types. We can use partial template specialization again:

template <class X, class Y>
struct Eq { static const bool result = false; }

template <class X>
struct Eq<X, X> { static const bool result = true; }

With this little struct now we can write our include operation this way:

template <class Elm, class Lst>
struct Includes {
	static const bool result = Eq<Elm, typename LST::head>::result
				   || Includes<Elm, typename LST::tail>::result;
};

template <class Elm> struct Includes<Elm, NIL> {
	static const bool result = false;
};

Very esoteric looking, the right mix of Haskel, C++ and booze to ensure job security for life. Next time we'll find a way to search for the position of an element, a somewhat more complicated task.

Tuesday 1 June 2010

Oh shit, the stack

Post from the wayback machine. I wrote this a long time ago but it got way down the posts queue, don't know why

I liked my vacations very much, thank you. Some people enjoyed vacations from me too. At work they even decided to keep this gem for my return. Upon my arrival a nice coredump was waiting at my desk, so to speak. Check it out, isn't it beautiful?

0 0xff05d070 in inflate_fast () from /usr/lib/libz.so
1 0xff05a13c in inflate () from /usr/lib/libz.so
2 0x00146224 in ZDecompress::decompress (this=0xfbc7b300, sauce=@0xfbe7b740, dest=@0x27c910) at Compressor.h:134
3 0x00145e80 in HandleClient::get_client_data (this=0x27c810, output_stream=0x27c910) at IPC/DataReceiver.cpp:54

Yeah, that's getting killed inside zlib. Nice way to start the year, a bug in zlib. What led me to that conclusion? Easy, the same compressed file worked in Ubuntu. Must be a bug in zlib then!

The next step was getting zlib's code and adding enough printf's to know the problem was in the middle of the file, not at the beginning nor the end; indeed, most of the file could be correctly decoded, but then it just died. This looked more and more like a bug in zlib.

I began to scramble things around, trying to isolate the problem. Things just got weirder, the same code worked fine if instead of being inside a thread I was on the main thread. If you have psychic powers you now have enough information to know what the problem was. Although I should have known too (this wasn't even the first time I saw a problem like this one!) I was mindset on finding a bug in zlib, which now, it seems, only appears while interacting with ACE (in my defence, I did see these kind of bugs too).

Fiddling around with the code some more, even stranger backtraces began to appear. First this one:

Program received signal SIGSEGV, Segmentation fault.
[Switching to LWP 10]
0xfd6b88fc in _pollsys () from /usr/lib/libc.so.1
(gdb) bt
#0  0xfd6b88fc in _pollsys () from /usr/lib/libc.so.1
#1  0x696e7661 in ?? ()
#2  0x696e7661 in ?? ()

And then this other one, which led me into the right direction:

Program received signal SIGSEGV, Segmentation fault.
[Switching to LWP 9]
0x000b6784 in std::operator| (__a=Cannot access memory at address 0xfbb7b094
)
    at /usr/local/lib/gcc/sparc-sun-solaris2.10/3.4.6/../../../../include/c++/3.4.6/bits/ios_base.h:124
124       { return _Ios_Openmode(static_cast(__a) | static_cast(__b)); }
(gdb) bt
#0  0x000b6784 in std::operator| (__a=Cannot access memory at address 0xfbb7b094
)
    at /usr/local/lib/gcc/sparc-sun-solaris2.10/3.4.6/../../../../include/c++/3.4.6/bits/ios_base.h:124
#1  0x00152d5c in HandleClient::get_client_data (this=Cannot access memory at address 0xfbb7b088
) at IPC/DataReceiver.cpp:46

That last stack trace got me to think how could it be possible for an otherwise working program to coredump while creating an stdlib object. I mean, stdlib is quite well tested, isn't it? Then it struck me: the keyword isn't stdlib but creating. It was allocating memory from the stack, upon entering the function.

Some more research later I found out that Solaris default thread size is about 1 mb, while in Ubuntu this thread is of about 8 mb. And I also noticed the buffer I was allocating for zlib was taking up space in... the stack.

If there's something to learn from this story is that you should always know what goes in the stack: only small objects should live there, and you should always know the max stack depth a function could reach. Otherwise it may come back and bite you in the ass when you're back from your vacations.