syntax highlight

Tuesday, 31 August 2010

Template Metaprogramming XIII: Heart of Darkness

Last time we had a virtual template dispatch problem... we got to the point of knowing which was the index of the cache we were searching for, now we need to actually retrieve an instance of that cache. That's a problem. Why? To begin with, there are no instances, only types!

The next logical step would be to create a Map device, to map a list of types to a list of instances... let's see how can we do that, in pseudocode

instances( H|T ) <- [ create_instance(H), instances(T) ]
instances( NIL ) <- NIL

Looks easy. How can we map that to c++?

template <class Lst> struct Instance {
    typedef typename Lst::head Elm;
    Elm instance;
    Instance< typename Lst::tail > next;
};
template <> struct Instance<NIL> {};

#include <iostream>
using std::cout;

int main() {
    typedef LST<int, LST<char, LST<float> > > Lst;
    Instance<Lst> lst;
    lst.instance = 1;
    lst.next.instance = 'a';
    lst.next.next.instance = 3.1;
    std::cout << lst.next.instance << "n";
    return 0;
}

All those next.next.next.instance look ugly. Let's use some more meta-magic to get the Nth instance (why not a [] operator? several reasons, you can't mix non-const ints with templates nicely, there would be problems to define the return type... all those options are workable but it's easier if we do this in another device.

template <typename LST> struct Nth {
	typedef typename LST::tail Tail;
	typedef typename Nth::result result;
};
template <typename LST> struct Nth {
	typedef typename LST::head result;
};
Remember that one from the toolbox? Now we know how to get a specific index position, yet getting the instance is a different problem (the Nth device returns a type, not an instance). We should do something different, the problem is knowing the return type. What's the return type for the Nth instance of the Instances list?
   type <- Nth(TypesLst, Type)
   type var <- NthInstance(InstancesLst, N)
Not so easy, right? This is the translated C++:

template <int N, typename TypeLst> struct NthInstance {
    // This one isnt easy...
    
    // This is the next type in the list
    typedef typename TypeLst::tail TypeNext;

    //  * Nth::result is the Nth type in Lst (i.e. char, int, ...)
    typedef typename NthInstance<N-1, TypeLst>::NthInstanceType NthInstanceType;

    //  * typename Nth::result & is a reference to said type and the ret type
    template <InstancesLst>
    static NthInstanceType& get(InstancesLst &instances_lst) {
        return NthInstance::get(instances_lst.next);
    }
};

// Remember, just for fun we choose a 1-based system (wtf..)
template <typename TypeLst> struct NthInstance<1, TypeLst> {
    typedef typename TypeLst::head NthInstanceType;

    template <InstancesLst>
    static NthInstanceType& get(InstancesLst &instances_lst) {
        return instances_lst.instance;
    }
};
And the code from fetching the instance itself is even more difficult, so I'll leave that for next time.

Thursday, 26 August 2010

Date time WTF

Another one to add to my growing list of bad things about Ubuntu. For some reason my clock froze. I only noticed it when it started to be dark outside, for me the time had frozen at about 17pm.

Tuesday, 24 August 2010

Template Metaprogramming XII: You Really Got a Hold on Me

Remember our virtual template method problem, from the other time? (I know, I said the answer was scheduled for a week after that post, but then I just forgot about it). May be we could avoid the virtual part by keeping a list of all our caches... how would we know which one should we dispatch the message to? Easy, using templates.

Instead of a list let's keep two, for twice the fun. One for the rows cache, another for the PKs. We can use PK to know which ROW Cache should we choose. Let's try to write a pseudo code for it:

ROW get_row(PK id) {
    pos <- Position of PK in pks_lst
    return cache[ pos ].get_row( id )
}

Doesn't look too hard. Building on our previous toolbox, let's use Eq, Position and the definition of a list:

struct NIL {
    typedef NIL head;
    typedef NIL tail;
};

template < class H, class T=NIL> struct LST {
    typedef H head;
    typedef T tail;
};

template <class X, class Y> struct Eq { static const bool result = false; };
template <class X> struct Eq<X, X> { static const bool result = true; };

template <class Elm, class LST> struct Position {
    private:
    typedef typename LST::head H;
    typedef typename LST::tail T;
    static const bool found = Eq<H, Elm>::result;
    public:
    static const int result = found? 1 : 1 + Position<Elm, T>::result;
};

template <class Elm> struct Position<Elm, NIL> {
    static const int result = 0;
};

class Facade {
    typedef LST<int, LST<char, LST<float> > > Lst;

    public:
    template <class PK> int find(PK) {
        return Position< PK, Lst >::result;
    }
};

#include <iostream>
using std::cout;

int main() {
    Facade f;
    std::cout << f.find(1.0) << "\n";
    return 0;
}

Great, now we can find an element on a list of types. The real virtual dispatch for the next entry :D

Thursday, 19 August 2010

MySQL upsert, Oracle merge

How many times have you seen this "pattern"?

unsigned int row_count = foo->update();
if (row_count == 0) {
   foo->insert();
}

Wouldn't it be nice if you could write all that in a single line? Say, something like

foo->update_or_insert_if_it_doesnt_exists();
Well, good news, you can! Obviously it's not standard SQL, nothing useful ever is, but even so I think using an upsert (who comes up with those names?) can be quite good for your health.

So, how do you use it? It's easy;

INSERT INTO Table ( col1, col2 )
SELECT 'a', 'b'
ON DUPLICATE KEY UPDATE col1 = 'a', col2 = 'b';
Go on, try it, I'll wait. What? It didn't work? Oh, I forgot, you need to create a unique key so the engine can recognize when there is a duplicate key (say, 'create index unique on col1'). Try it now.

Nice, isn't it? Oracle has its own version of upsert, called merge (at least the name is better) but it itches a little bit when I write about Oracle, so go and check this page instead.

Tuesday, 3 August 2010

C++ Namespaces and g++

Have you ever tried to leave open a C++ namespace after EOF (that is, openning a namespace in a headerfile but forgetting to close it). It's a little bit like getting your balls caught by the door. The compiler will throw at you an incredible number of seemingly unrelated errors, all of which occur in a different file than the offending header.

Reaching EOF on a C++ file without closing all its namespaces should be ilegal; or at least you should have better error reporting, because right now it's almost impossible to know what's the source of the error (for g++, that is).

Thursday, 29 July 2010

Oh my god

An old good Unix console joke goes like this:

% ar m God
ar: God does not exist

Obviously thats a very old and used joke, nowdays we're much more advanced than that:

$ ar m God
$ ar: creating God

Tuesday, 27 July 2010

Design Patterns: C++ Idiom RAII

Ohh design patterns. I love buzzwords. Who doesn't? To increase my buzzword count I will be writing about a topic most people programming C++ should already know: RAII, resource acquisition is initialization.

Patterns everywhere

Knowing that Gamma et al didn't list all the known patterns in the universe does come as a surprise to some, not sure why though. The twenty some patterns they write about in their now famous book are (arguably, perhaps) some of the most common design patterns, but the list hardly finishes there.

Some patterns only have meaning in a very specific context; a reactor is a nice pattern for handling asynchronous events yet most applications would never need it. Sometimes "the context" means a specific domain in which the application must work, like a web application, a real time application or a distributed application, sometimes the context is the language itself. RAII falls in this last domain, it only makes sense for C++ applications (actually there are others, but thinking what kind of languages would support this idiom is left as an exercise for the reader).

No, really. What is RAII?

If you made it past that long intro you are probably really interested in knowing what RAII is, and don't know how to search it in Wikipedia. OK, I'll explain it the best I can then: RAII means that a resource acquisition is the initialization.

Seriously. That is all the secret there is to RAII. Talking in code:

template
class RAII_Wrapper {
   T *resource;

   public:
      RAII_Wrapper (T *resource)
            : resource(resource)
      {
         resource->open();
      }

      ~RAII_Wrapper ()
      {
         resource->close();
      }
};

An example

Compare that to a visitor pattern. It's just too simple to be of any use, isn't it? Well, contrary to what Java fanboys tend to believe you can do lots of nice things without writing a bazillion lines of code.

int foo() {
	Expensive_Resource x;
	x.open();

	try {
		if (not bar()) {
			x.close();
			return -1;
		}
	} catch (...) {
		x.close();
		throw "up";
	}

	int ret;
	try {
		ret = baz();
	}catch(...) {
		// We don't care, we're closing x anyway
	}

	x.close();
	return ret;
}

Wow... a whole lot of code just for calling bar and baz. And as I wrote that without even compiling I'm sure there are too many hidden bugs, lots of code-paths my simple programmer's mind can't even begin to imagine which will cause my Expensive_Resource to be leaked!

Let's rewrite that using RAII:

void foo() {
	Expensive_Resource x;
	RAII_Wrapper release(&x);

	if (not bar()) return -1;
	return baz();
}

A lot nicer, isn't it? But, what happened to all the try/catch if's and closes there?

Where's the magic?

The magic of RAII lies in how C++ handles exceptions. When we have a built object (can an object be in an unbuilt state?) it means it's constructor has correctly ran. It also means it's destructor will run when it goes out of scope, doesn't mater HOW it goes out of scope.

See how brilant that is? Doesn't matter if a function we're calling throws, or if we need to return before reaching the end of the function: the destructor will be called and thus our Expensive_Resource will be free!

Why is this C++ specific?

This is an easy one: think how would you implement this in Java: right, you can't. Not knowing when is your object going to be destroyed means you can't do anything useful in its destructor, therefore RAII is deeply rooted within the memory management in C++ and it's pretty much a language specific pattern (or is it?). But, is that so good?

Not everything is so great...

You are probably thinking this is the best discovery since ice cream was invented. Well, not so fast, RAII has it's detractors too.

The first problem in RAII, it doesn't have a graceful way of handling resource acquisition failure. If Expensive_Resource is a database, and it's connection fails, we have a throwing constructor.

Even if throwing constructors are acceptable, throwing destructors may even be a worst idea: throwing while an exception is already active is a cause for concern (tip: it'll crash your application, doesn't matter how many try/catch blocks you use, due to stack unwinding issues).

And then, even if we don't care about throwing destructors you have the issue of a release failure: how do you notify the user that a release failure has occurred? And what do you do, should it happen?

So, RAII is a great idea indeed, and it has it's uses, but you should be careful when choosing this C++ specific pattern.

Friday, 23 July 2010

Fuuuuuuuuuuuuuu (Opera)

I hate ubuntu, but I hate windows the most. I hate firefox, but internet explorer makes me want to vomit. I don't like gnome, but kde is uglier. But I love Opera. Well, loved it, I guess version 10.60 will be my last version.

It's been a loyal application which I've been using since its 6.x version. Always fast, with all the functionality I needed (some more too) and none of the bloated BS needed to make FF be a usable (though ugly) browser. It had it's ussual screw-ups, everyone does, but after I updated to version 10.60 it has become unusable.

Its new features include random crashe, making gmail work really slow (or not at all: the fucking scrollbar won't work anymore, and keyboard shortcuts in google reader (j & k, next & prev) are foobar'd. Scrolling on a large webpage eats 100% cpu, the upgrade fucked up my nice dark theme (and changed it back to a hellish abomination which seems to be a time-traveller from 1998), and it has random stupid bugs. And I mean STUPID, like, double click a word and the popup menu won't go away.

Fuck. I always liked you Opera, but it seems it may be time to give chrome a chance.

Tuesday, 20 July 2010

Template metaprogramming XI: Hidden Agenda

Wow, number eleven already. We're getting more chapters here than Final Fantasy games. I didn't even imagine there was so much to write about such an esoteric language features like templates. I do wonder if anyone will actually read it, but that's a completely different problem.

Enough meta-meta talk: what can we do with all the things we have learned? We can calculate pi and e, we already showed that as an example on one of the first chapters. This chapter I'm going to write about what motivated me to explore the bizarre underworld of template metaprogramming. Some time ago I had to work with a Berkeley DB researching the feasibility of developing a magic cache for (real) DB table. Leaving aside the discussion of whether this is a good idea (the project did have a good reason to be researched) I hit a major roadblock when trying to provide a façade for every table; something like this:

See the problem? To do something like that we'd need a virtual template method, and you can't have that. After seeing that I thought to myself "Hey, I'll use templates!". Then I had two problems, but the damage was done, I couldn't go back. What kind of contorted device could we implement to make such a devious requirement work? I'll leave you to think it, the answers I came up with next week.

Thursday, 15 July 2010

Thanks for flying vim

Have you ever used Vim through ssh and saw your xterm title changes to "Thanks for flying vim"? It happens a lot to me, and I usualy notice about an hour later. I'm not sure what's the use of this, I guess it's related to Vim airlines (no, really, check vim-avia.com), but it can be turned off:
When using vim in an xterm it renames the title of that window to "Thanks for flying vim" on exit. Q: How to turn off the message "Thanks for flying vim"? A: :set notitle
-- http://www.vmunix.com/vim/answ.html

Tuesday, 13 July 2010

The truth about SNMP

Seen @ http://wiki.wireshark.org/SNMP:
After years thinking and reading RFCs and various other documents, today, I finally understood. "Simple" refers to "Network" not to "Management Protocol"! So it is a Management Protocol for Simple Networks not a Simple Protocol for Management of Networks... That explains why it's called "Simple". It was that Simple but it took me years to understand it! -- LuisOntanon

Thank you Luis. That explains a lot.

Thursday, 8 July 2010

Opera borks gmail

Ubuntu sucks, but less than windows. Gmail sucks, but less than hotmail. Opera rocks, but it tends to fuck up gmail every once in a while. After a lot of research and having found no help on the interweb I traced the problem to having a lot of tabs open for a lot of time (weeks, not hours).

In Firefox this shouldn't be a problem as having a FF browser open for a week should hog all the memory on its host computer, forcing you to reboot. In Opera, being a little bit better behaved browser, this may actually be a problem.

Luckly the fix is simple: open a Vim editor or take out pencil and paper, make a list of all your open tabs, close opera and using your favourite console type "rm -rf ~/.opera/sessions" (i.e. delete the sessions folder in your .opera dir). Restart Opera and restore your tabs from your backup list. Problem should be gone.

Tuesday, 6 July 2010

Elvis is alive!

Unix trivia day: in the olden days of the 90's there were a lot of Unix boxes out there named "elvis". Nowdays it's not uncommon to find one, either. Have you ever wondered why are there so many boxes called elvis?

This is related to Solaris' ping command. When you ping $HOST it prints "$HOST is alive" (if it's responding the pings), thus elvis is alive!

Tuesday, 29 June 2010

Vim tips: make things work again

Are you using :make within vim? If you are, then you probably noticed that getting lots of compiler errors is not uncommon, and that vim has a tendency of jumping to the first error found... but what happens if you want to see subsequent errors too, because you're a really crappy programmer and need to find the 10 or 20 errors you just introduced with your single line edit?

Luckily Vim is always the anwser, just use cnext and cprev until you reach the error you are looking for.

Friday, 25 June 2010

There's still hope

A log time ago, in a galaxy far away, I posted a list of stuff I apt-get'd as soon as my Buguntu install was ready. Well, fast forward to KK and this is the new apt-get script:

sudo apt-get install opera mocp unclutter build-essential gimp \
        mmv gnome-do knetworkmanager xdiskusage iptraf sshfs \
        shntool cuetools flac wavpack unrar vlc vim gvim vim-doc \ 
        vim-full vim-gnome ddd devhelp doxygen doxygen-gui \
        exuberant-ctags ctags  subversion ifstat antiword

Quite a progress I'd say, as these are mostly programs I use which don't come in the default install (even though they should). You know, I even have hopes for the sound system in my notebook this time...

Wednesday, 23 June 2010

Nooooooooooo

(gvim:13664): Gdk-CRITICAL **: gdk_window_get_user_data: assertion `GDK_IS_WINDOW (window)' failed
Vim: capté una señal de muerte SEGV
Vim: finalizado.

Tuesday, 22 June 2010

C++ pretty functions

There are two well known macros from the preprocessor which every macro-sorcer must know. They are __FILE__ and __LINE__. You probably already know about them but anyway, __FILE__ will give you the current file and __LINE__ the current line. Easy, huh?

int main() {
   printf("%s : %i", __FILE__, __LINE__);
   return 0;
}

The program above would give you "main.cpp : 3" as a result. There is nothing going on at execution time, it's all preprocesor wizardy. In fact with "g{++/cc} -E" you can even check what the "real" output is (-E means to return the preprocessor output. Keep in mind a lot of stuff will be included from the headers you use).

int main() {
   printf("%s : %i", "main.cpp", 3);
   return 0;
}

Well that's nice and all, but g++ can top this easily:

int main() {
   std::cout << __PRETTY_FUNCTION__ << "n";
   return 0;
}

There are a couple of notable things about this new "pretty function" thing:

  • 1. It will demangle a function's name
  • 2. This time it isn't a preprocessor secret thing but a real variable g++ will create.

You can easily use this for better logging functions now (with some macro wizardy, obviously).

Thursday, 17 June 2010

Template metaprogramming X: Zero Minus Ten

So far we've learned the basic constructs of template metaprogramming (loops, branching, return values) and some basic list operations (getting the length of a list, appending and prepending elements, checking if an element is included in a list). Let's put it all together by creating an operation to return the position of an element. It'll be very useful later on too.

If we go back to the Includes operation we can get some help to define the Position operation: the position of an element in a list is one plus the position of the element we're searching for in the tail, or zero if the head equals said element. The operation is not defined if the element is not in the list.

Translating to pseudo-code:

Position (lst.head, lst) <- 0
Position (e, lst) <- 1 + Position(e, lst.tail)

The translation to C++ is not so trivial this time. Try it, I'll wait... ready? OK, let's start

template <class Elm, class Lst> struct Position {
	typedef typename Lst::head Head;
	typedef typename Lst::tail Tail;
	static const bool found = (Head == Elm);
	static const int result = found? 0 : 1 + next;
	static const int next = Position<Elm, Tail>::result;
};

Looks easy... but doesn't work. First problem, we can't compare two types, remember? We need to use Eq again. Second problem, although we said the operation is undefined if the element is not included on the list, it would be nice if we could force the compiler to fail if (or when) that happens. Let's rewrite the operation using a façade again, but adding an Assert:

template <typename Elm, typename LST> struct _Position {
	typedef typename LST::head Head;
	typedef typename LST::tail Tail;

	static const bool found = Eq<Elm, Head>::result;
	static const int result = (found)? 0 : 1 + _Position<Elm, Tail>::result;
};

template <typename Elm, typename LST> struct Position {
	typedef typename Assert<Includes< Elm, LST >::result>::check include;
	static const int result = _Position<Elm, LST>::result;
};

Oh, we haven't defined assert yet! There's another problem, too: even if it won't compile, the compiler will try to expand _Position< ..., NIL > indefinitely, causing an error after too many nested template calls. Not nice. We need to add a case to make the compiler stop:

/******************************************************/

// Helper: Will fail to compile if the assert is false
class Assertion{};
template <bool cond, class T=Assertion> struct Assert {
	typedef typename T::fail check;
};
template <> struct Assert<true> {
	typedef void check;
};

/******************************************************/

template <typename Elm, typename LST> struct _Position {
	typedef typename LST::head Head;
	typedef typename LST::tail Tail;

	static const bool found = Eq<Elm, Head>::result;
	static const int result = (found)? 0 : 1 + _Position<Elm, Tail>::result;
};

// The compiler will try to expand the position check
// after NIL has been reached if this isn't here
template <typename Elm> struct _Position<Elm, NIL> {
	static const int result = 0;
};

template <typename Elm, typename LST> struct Position {
	typedef typename Assert<Includes< Elm, LST >::result>::check include;
	static const int result = _Position<Elm, LST>::result;
};

All that code for such a simple operation, man. Also, see what we did with Assert<>? It seems making a compile fail is actually quite easy. That's what I have most experience with.

We've been through quite a lot, and our toolboox should be quite big already. Next time we'll start steering towards some sort of applicability, trying to use some of all these stuff to implement a real, useful and working program... assuming that's even possible.

Tuesday, 15 June 2010

Binary portability in Linux

An interesting topic for a change: is Linux binary portable? That is, can we take a binary file and be sure it'll run in any other Linux system? What happens if we broaden that to any POSIX system, will it blend? Eh, I mean, will it run?

Doing some research on the subject I wrote down a list of the thought process which led my to an (inconclusive) answer:

  1. First we should define what a binary is for us: When we talk about a binary we are usually thinking about a compiled binary file, not an interpreted script file like Ruby or Python. Those are for people who like things to actually work, so let's focus on a compiled executable file, like a C/C++ application.
  2. Defining compiled file: What could it be other than a sequence of bytes the microprocessor can understand? Yes, that's right, it's sort of interpreted code, only there's electronics behind, not more code. This brings us to the first interesting conclusion: the executable must be (leaving emulators aside) compatible with the architecture you're on. Running Sparc? Well then, the binary better be compiled for Sparc because otherwise to the uP will not make any sense.
  3. Format: as any other thing, a binary file must have a format. That is a standard which defines the structure the file will follow. ELF is the binary format for Linux and it's quite standard. Of course, if the binary format is a standard then we should get perfect portability between different platforms running on equal architecture. Unfortunately that's not the case.
  4. (Cont'd) Why don't we? The binary depends not only on compile time "stuff" but a loading time linking occurs: the executable binary will get linked with the system files like glibc, or any other dependency on a shared library it may have.

So, what are the keypoints for Linux binary portability? Architecture, binary format and system libraries.

Of course, making the executable run is only part of the equation, as running and segfaulting on the spot is not so nice either. For this last part you'll have to closely follow the standards defined by POSIX for paths and stuff like that.

Epilogue

As an epilogue, we could add that Windows binary compatibility tends to be great. Running binaries from 12 years back is no small feat, yet this leads to a whole lot of other problems: an incredible complex loader, security bugs, backwards compatibility headaches, et al. The old new thing is a great source of information for this topics, I'm quite illiterate about Windows binaries nowdays :)

Followup links

Friday, 11 June 2010

Vim tips: make things work

So, you are an uber console geek, using only vim and the command line to compile all your projects, execute the tests, blah blah blah... if only you could squeeze that microsecond lost whenever you switch from vim to compile you'd be 1e-4 seconds more productive... oh, wait, you can!

Whenever you think you're project is good enough to compile just hit :make to be proven wrong. Type :make test to run your tests (because you are using TDD, aren't you?) and watch all those red flags fly by.Also, add the following mapping to your ~/.vimrc for an extra happy coding session:

map <F5> :make
map <F6> :make test