gdb-tui and the previous-command problem

Raise your hand if you have run gdb in tui (graphical) mode, only to find you can't refer to the previous command when pressing "up". I can't see you but I know this is true for pretty much everyone reading this blog. All three of you.

In the gdb-TUI mode, the arrow keys are used by the active window for scrolling. This means they are not available for readline, the component that takes care of the magic invocations needed to bring back the previous command from the land of the dead. Luckily there are alternative readline keybindings: just try C-p, C-n, C-b and C-f. Takes a while getting used to it but you can finally use gdb-TUI and forget about copy-pasting every gdb command.

Bonus tip: if pressing "up" (or C-p) in gdb doesn't bring back the previous command, it probably means you don't have the readline package installed. Go ahead an install it. It'll change your life.

Android studio and ndk-gdb to debug a native app

I don't know how good Android Studio support for native apps is nowadays (it changes from week to week!). Up to a few months ago, Gradle, the build system used by AS, had poor support for native development. If you're having problems, you may find it easier to workaround it completely when it comes to build and debug C/C++ applications.

To debug a native Android application, a binary called gdbserver and its associated gdb.setup must be included in the generated APK file. Including this into the APK can be very painful in Gradle, so here's a workaround I found:

1. Build your stuff the way you normally would (I'm assuming you know already how to build a native app, and if you don't there are guides online that explain it much better than I could).
2. Deploy your application the way you normally would.
3. Discover ndk-gdb won't run. Bang forehead against keyboard a few times.
4. After losing some hours looking at logs, figure out there's no gdbserver included in your apk.
5. Lose some more hours trying to figure out how to include it in your apk using Gradle.
6. Give up. Bang forehead against keyboard some more.
7. find the gdbserver and gdb.setup in your build directory.
8. adb push each of these files to the device.
9. Using adb shell, move the files you copied to /data/app-lib/com.yourapp/ - you may need to root your device for this.
10. Profit! ndk-gdb now works.

Edit: remember you may need to chmod +777 your gdbserver.

Useful predefined variables in make

I always forget about two very useful make variables, so I'll leave this here: $(COMPILE.cpp), $(LINK.cpp). It's easy, instead of writing a rule as

foo.o: foo.cpp
  g++ -c foo.coo

you should instead write this:

foo.o: foo.cpp
  $(COMPILE.cpp) foo.coo

COMPILE.cpp will have the default compiler you are supposed to use, and probably some helpful parameters as well. Likewise, LINK.cpp will have the linker you are supposed to use.

There are many useful predefined variables in make. Be sure to check them all by running "make -p" in a console.

Vim tip: reload your vimrc

If you're changing your vimrc, it can get boring to close and restart it only to see the changes applied. Want something quicker? You can ":so %". So stands for source, so you'll just be telling vim "include this file". % happens to be the path to the current file. If you're not editing your .vimrc but for some reason you still want to reload it, just use "so ~/.vimrc" instead.

ndk-gdb life tip: use --verbose

Crosscompiling is always fun. No matter how ready-to-use it's packaged, and Android does a pretty decent job at that, you're still bound to find problems that leak through the abstraction layers. If something says it's dummy-proof, I always find the way to perfect myself and be even dumber. For people like me; do yourselves a favour and start launching ndk-gdb this way:

ndk-gdb --start --verbose

Using the --verbose parameter will probably reveal some hidden errors. For example, when I forgot to chmod 777 my gdbserver binary:

## COMMAND: adb_cmd pull /system/bin/app_process ./obj/local/armeabi-v7a/app_process
run-as: exec failed for /data/data/com.nico.trippingsdcardphotomanager/lib/gdbserver Error:Permission denied
117 KB/s (9560 bytes in 0.079s)
Pulled app_process from device/emulator.

I now write Android apps: presenting Tripping Photo Manager

I have been working in an awesome Android app recently, so I'd like to throw in a shameless self-plug here. Oh, wait, it's my blog. It's all basically a big shameless self-plug, isn't it? Anyway, I've been working on https://github.com/nicolasbrailo/TrippingSdCardPhotoManager , an open source SD card photo manager for Android.

Why an SD card photo manager? Whenever I go on holidays I never have enough SD cards to store all the snaps I take. Luckily I'm a crappy photographer, so I end up deleting half of the pictures I took during a day. This makes it easy to somewhat re-use the same SD card. In my experience, Android is not always great when it comes to managing files from an external SD card mounted through an USB adapter, hence this little app was born: it'll let you select the directory you want to use to manage pictures, from anywhere in the filesystem. It's also somewhat faster than the native gallery app, which is a plus when working with a slow-ish SD card.

This app also supports some stuff I find useful in my workflow, like renaming the current directory to something more meaningful than "YourCamera4242", backing stuff up in the device and batch deleting files. It also packs a version of ImageMagick I ported for Android, so in theory you can use your Android device to do anything you can do with a regular "mogrify" command in Linux.

The app is not available in Playstore, mostly because I'm a cheap bastard and don't want to pay the 10 bucks Google charges you to create an account, only to publish an open source application.

You can still install the APK from this link: https://github.com/nicolasbrailo/TrippingSdCardPhotoManager/releases - or you can contribute to the open source world (?) and buy me a Play Store account.

Debugging multiple processes with gdb

If your buggy program generates lots of child processes, gdb will keep attached to the parent program and let all the children run loose. If you're having problems to find what causes  your crash this is probably not what you want: for those occasions gdb has a very helpful flag called detach on fork.

With detach on fork you can tell gdb to keep debugging the parent, follow the children, or keep track of all processes. Must be nice to troubleshoot forkbombs with this option.

cerr vs cout

In C++, is there a difference between cerr and cout, other than using a different file descriptor? Turns out you can learn something new every day.

Whenever I just want to add a quick print statement because I'm too lazy to debug something, I used to use cout. All along I thought cout and cerr would be exactly the same for my use case, but turns out there's a slight difference: cout is buffered, cerr is not. This very small difference can have a huge impact, because you shouldn't need to flush cerr after a write to make sure the changes are visible, it should happen automagicaly. In turn cerr might be slightly slower, but you probably don't care about that when writing cerr << "I'm here!".

Vim tip: "polymorphic" key bindings

If you use vim daily, you probably have a bunch of maps for your most common tasks. You should also remember you can assign the same key to do different things according to which mode you are in. For example, let's say you have a mapping to open a new tab:

map <leader>t :tabnew<cr>

You can also map t to open a new tab using the selected text as a filename. You just need to define two mappings:

nmap <leader>t :tabnew<cr>
vmap <leader>t :call Foo()<cr>

nmap stands for normal (mode) map, vmap for visual. How to get the text under the cursor is a bit more complex and out of scope for this vim tip, but you might want to check http://vim.wikia.com/wiki/Mapping_keys_in_Vim_-_Tutorial_%28Part_1%29.

Remember to check ":help map" for a list of all mode mappings.

PSA: Use nautilus (GTK) in Kubuntu if Dolphin crashes

So, for some reason my (brand new) Kubuntu is not very stable. KDE tends to randomly crash whenever I send a large-ish file to the recycling bin. Dunno why, looks like some threading + caching issue in dolphin, and I couldn't find a solution anywhere on the web. Well, there are two workarounds:

1. Don't use the recycling bin. This sort of breaks my workflow, so I prefer to: 2. Use nautilus.

You'll have to install gtk packages, but that's a small price to pay to have KDE not crash every couple of minutes.

Vim tip: Stop escaping slashes

If you do a lot of search & replace in Vim, eventually you'll end up escaping a lot of slashes. Whenever you have to replace a path, for example. Isn't that annoying? After a couple of levels you end up with a horrible "\/path\/to\/foo\/bar" pattern. And if you miss an escape slash, good luck. It's better to scrap the whole thing and start over.

Luckily, when you are using the 's'earch command you can pick a different separator. Instead of typing "%s/\/foo\/bar\/baz\//foo\/bar\//g", you can simply type "%s#/foo/bar/baz/#foo/bar/#g". Vim will automagically detect you want to use '#' as a delimiter, and you'll end up with a much more readable pattern.

Extra tip: this also works in sed

C++: A jump table with a template device

A few articles ago we saw how gcc might need some help when mixing template instanciation (pure compile time data) with function calls (deducible compile time information, but not available to the template expander). Now we'll go one step further and combine all three types: pure compile time data, deducible compile time data and pure run time data (*). Just to annoy the compiler, and to see how gcc is able to optimize the results.

Let's build a simple example, similar to what we used last time: an object that will determine the range of an integer and then invoke a callback with the closest range. Something like this could be used, for example, to allocate a buffer.

void boring(int x, func f) {
    if (x < 2) {
        f(2);
    } else if (x < 4) {
        f(4);
    } else if (x < 8) {
        f(8);
    } else if (x < 16) {
        // You get the idea...
    }
}

Can we build a prettier template version of this code, without any overhead? Let's try:

typedef void (*func)(int);

template <int My_Size>
struct Foo {
    void bar(size_t size, func callback) {
        if (size > My_Size) {
            callback(My_Size);
        } else {
            next_foo.bar(size, callback);
        }
    }

    Foo<My_Size/2> next_foo;
};

// Stop condition
template<> struct Foo<0> {
    void bar(size_t, func) { }
};

void wrapper(int x, func f) {
    Foo<512> jump_table;
    jump_table.bar(x, f);
}

And now, let's compile like as "g++ -fverbose-asm -S -O0 -c foo.cpp -o /dev/stdout | c++filt". You'll see something like this:

wrapper(int, void (*)(int)):
    call    Foo<512>::bar(unsigned long, void (*)(int))

Foo<512>::bar(unsigned long, void (*)(int)):
    cmpq    $512, %rsi    #, size
    jbe    .L4
    call    *%rdx    # callback
    jmp    .L3
.L4:
    call    Foo<256>::bar(unsigned long, void (*)(int))    #
.L3:
    leave

Foo<256>::bar(unsigned long, void (*)(int)):
    cmpq    $256, %rsi    #, size
    jbe    .L4
    call    *%rdx    # callback
    jmp    .L3
.L4:
    call    Foo<128>::bar(unsigned long, void (*)(int))    #
.L3:
    leave

# You get the idea, right?

Foo<0>::bar(unsigned long, void (*)(int)):
    # Stop condition, do nothing

That doesn't look too good, does it? We don't need to worry: we already learned that gcc needs help from the optimizer to handle template expansion and non static function calls. Let's move to O1:

rapper(int, void (*)(int)):
.LFB14:
    cmpq    $512, %rdi    #, D.2974
    jbe    .L2    #,
    movl    $512, %edi    #,
    call    *%rsi    # f
    jmp    .L1    #
.L2:
    cmpq    $256, %rdi    #, D.2974
    jbe    .L4    #,
    movl    $256, %edi    #,
    call    *%rsi    # f
    jmp    .L1    #

# Again, it should be clear what's going on...

.L11:
    cmpq    $1, %rdi    #, D.2974
    .p2align 4,,2
    jbe    .L1    #,
    movl    $1, %edi    #,
    .p2align 4,,2
    call    *%rsi    # f
.L1:

It's better than last time, but it doesn't look great either: gcc managed to inline all calls, but it stopped there. Let's move to O2 and see what happens:

wrapper(int, void (*)(int)):
    movslq    %edi, %rdi    # x, D.2987
    cmpq    $512, %rdi    #, D.2987
    ja    .L13    #,
    cmpq    $256, %rdi    #, D.2987
    ja    .L14    #,
    [ .... ]
    cmpq    $2, %rdi    #, D.2987
    ja    .L21    #,

.L13:
    movl    $512, %edi    #,
    jmp    *%rsi    # f

.L14:
    movl    $256, %edi    #,
    jmp    *%rsi    # f

[ .... ]

.L21:
    movl    $2, %edi    #,
    jmp    *%rsi    # f

.L1:
    rep
    ret
    .p2align 4,,10
    .p2align 3

Now, that looks much better. And we can now see that gcc generates the same code at -O2 for both versions of our code.

(*) Just for the sake of completion:

• Pure compile time data is information directly available during compilation time, like a constant.
• Deducible compile time data means something that can easily be deduced, like a function call to a non virtual method.
• Run-time only data means something that a compiler could never deduce, like a volatile variable or the parameter of a function called from outside the current translation unit.

Globing in bash

There is a pretty common and unnecessary pattern used by bash scripts: whenever you need to loop through a list of file names in a path, you might tempted to write something like this.

for fname in $(ls | grep foo); do echo $fname; done

You can save some typing by using bash-globbing:

for fname in *foo*; do echo $fname; done

Not only the script should be cleaner and faster, bash will take care of properly expanding the file names and you won't have to worry about things like filenames with spaces. This should also be portable to other shells too. Want to know more about bash globbibg? Check out http://www.linuxjournal.com/content/bash-extended-globbing

gdb: Print very long strings

gdb defaults are usually quite sensible and just "let you work". Some times, though, your project is not very sensible and you have to do weird things in gdb. An example: printing huge strings or vectors to try and reproduce a heisenbug. A lot of people get surprised at first because gdb will refuse to print very long strings, printing only the first few chars. Same for vectors. And if you have many repeating elements (eg "f000000000000000000000000000000bar") you might see something like "fooo0bar".
Just type these magic commands to see the whole string:

> set print repeats 0
> set print elements 0

gcc: Optimization levels and templates

Analyzing the assembly output for template devices can be a bit discouragging at times, specially when we spend hours trying to tune a mean looking template class only to find out the compiler is not able to reduce it's value like we expected. But hold on, before throwing all your templates away you might want to figure out why they are not optimized.

Let's start with a simple example: a template device to return the next power of 2:

template <int n, long curr_pow, bool stop>
struct Impl_Next_POW2 {
    static const bool is_smaller = n < curr_pow;
    static const long next_pow = _Next_POW2<n, curr_pow*2, is_smaller>::pow;
    static const long pow = is_smaller? curr_pow : next_pow;
};

template <int n, long curr_pow>
struct Impl_Next_POW2<n, curr_pow, true> {
    // This specializtion is important to stop the expansion
    static const long pow = curr_pow;
};

template <int n>
struct Next_POW2 {
    // Just a wrapper for _Next_POW2, to hide away some
    // implementation details
    static const long pow = _Next_POW2<n, 1, false>::pow;
};

Gcc can easily optimize that away, if you compile with "g++ foo.cpp -c -S -o /dev/stdout" you'll just see the whole thing is replaced by a compile time constant. Let's make gcc's life a bit more complicated now:

template <int n, long curr_pow, bool stop>
struct Impl_Next_POW2 {
    static long get_pow() {
        static const bool is_smaller = n < curr_pow;
        return is_smaller?
                    curr_pow :
                    _Next_POW2<n, curr_pow*2, is_smaller>::get_pow();
    }
};

template <int n, long curr_pow>
struct Impl_Next_POW2<n, curr_pow, true> {
    static long get_pow() {
        return curr_pow;
    }
};

template <int n>
struct Next_POW2 {
    static long get_pow() {
        return _Next_POW2<n, 1, false>::get_pow();
    }
};

Same code but instead of using plain static values we wrap everything in a method. Compile with "g++ foo.cpp -c -S -fverbose-asm -o /dev/stdout | c++filt" and you'll see something like this now:

main:
    call    Next_POW2<17>::get_pow()

Next_POW2<17>::get_pow():
    call    _Next_POW2<17, 1l, false>::get_pow()

_Next_POW2<17, 1l, false>::get_pow():
    call    _Next_POW2<17, 2l, false>::get_pow()

_Next_POW2<17, 2l, false>::get_pow():
    call    _Next_POW2<17, 4l, false>::get_pow()

_Next_POW2<17, 4l, false>::get_pow():
    call    _Next_POW2<17, 8l, false>::get_pow()

_Next_POW2<17, 8l, false>::get_pow():
    call    _Next_POW2<17, 16l, false>::get_pow()

_Next_POW2<17, 16l, false>::get_pow():
    call    _Next_POW2<17, 32l, false>::get_pow()

_Next_POW2<17, 32l, false>::get_pow():
    movl    $32, %eax    #, D.2171

What went wrong? It's very clear for us the whole thing is just a chain of calls which could be replaced by the last one, however that information is now only available if you "inspect" the body of each function, and this is something the template instanciator (at least in gcc) can't do. Luckily you just need to enable optimizations, -O1 is enough, to have gcc output the reduced version again.

Keep it in mind for the next time you're optimizing your code with template metaprogramming: some times the template expander needs some help from the optimizer too.

Bash traps: almost like RAII for bash

Everywhere, but specially in bash, cleaning up is annoying and error prone. Resource leaks can be common if your bash script is interrupted half-way. Do you need to execute something always, even if your script fails or gets killed? Try using traps:

#!/bin/bash
foobar() {
    echo "See ya!"
}
trap "foobar" EXIT

It doesn't mater how you end this script, "foobar" will always be executed. Want to read more about bash traps? Check http://linuxcommand.org/wss0160.php

C++: Invalidating references to elements in a vector

Is this valid C++?

void do_something(const int&);
#include <vector>

void foo() {
    std::vector<int> v = {1,2,3,4,5};
    const int &num = v.at(1);
    v.push_back(42);
    do_something(num);
}

Doesn't seem quite right, does it? push_back will most likely trigger a resize for the vector, and that will invalidate references to elements in the vector. num will end up pointing anywhere and so using it to call do_something is not valid C++. Or is it? What happens if we reserve some space for v?

void do_something(const int&);
#include <vector>

void foo() {
    std::vector<int> v = {1,2,3,4,5};
    v.reserve(40);
    const int &num = v.at(1);
    v.push_back(6);
    do_something(num);
}

It again might seem wrong, but this in fact is valid C++ code. Common sense might tell us that a call to push_back automatically invalidates references to elements in the vector, and it only works because most implementations will do the reasonable thing (ie not to invalidate references unless they must). Turns out the standard makes a special prevision for this case in section 23.3.6.5: a resize for a vector is guaranteed to be triggerd if, and only if, the capacity of the vector is not enough, and references to elements in the vector are guaranteed to be valid unless resize is triggered.

A bit of language laweyering shows that what seems like an error is in fact allowed by the standard, but even if this is valid C++ code you should always keep in mind that assuming that the capacity of a vector will be enough is a VERY big assumption, it's very easy to break and you won't get any warning when it happens (maybe a core dump, if you're lucky).

Code natural selection

A funny thought just came to me: if you write nice clean code, it's easy to replace it. Just plug out an object somewhere, replace it with another one implementing the same interface, run the tests. Tada! On the other hand, if you write crappy code it might be nigh impossible to replace it. It will probably be worked around whenever a change is needed, simply adding layers of crust. Maybe that's why legacy code sucks: it's simply code natural selection - and the fittest to survive is simply the crappiest one. I think I'm depressed now.

Ruby-style digit separator for C-98?

A silly, and not very useful tip: I love how ruby allows you to write numbers like 1000000 as 1_000_000. Very useful to write benchmarking tests.

Until we get to C++14 we don't have a nice alternative, but we have an ugly hack we can use: instead of writing 1000000 write "1 ## 000 ## 000".

It works, '##' is the preprocessor's token pasting operator, and it will paste two tokens together. Looks ugly, breaks the GUI highlighting, but at least you can count how many zeros you've got.

Nitpicker's corner: multiplying by 10 is easier, but there is no job-safety involved in that.

Nitpicker's corner II: The evaluation order of a chain of '##' is not defined, but I don't expect this to cause any problems; any order of evaluation should result in the same result for this case.

Vim Tip: I want more menus!

As a uber vim geek, you shouldn't be using a lot of gui menus. Scratch that, you shouldn't be using any menus at all, period. Still, I'll admit it's a bit hard to remember every single shortcut for actions you rarely use.

Say, for example, you like to encrypt your text. Not always, but every once in a while. Enough to make a shortcut for it but not enough to remember what the shortcut is. You can try to grep your ~/.vimrc. You might find something like:

" Encrypt my stuff
map <leader>e ggg?G<CR>

(Yes, that command will actually encrypt your text in Vim. Try it!)

Wouldn't it be nice if you had a simpler way, though?

Turns out you can add your "encrypt" command to your gui. Then "menu" commands work just like the "map" family, but they create a GUI menu instead. Change your vimrc to something like this:

" Encrypt my stuff
map <leader>e ggg?G<CR>
menu Project.Encrypt ggg?G<CR>

Now if you reload your vimrc you'll find a new GUI menu created, from which you can easily encrypt your text. Decrypting is left as an exercise to the reader.

Extra tip: Want to try to learn the actual shortcut, like a real vim'er? Then try this:

menu Project.Encrypt<TAB>ggg?G ggg?G<CR>

Everything after the TAB will be right-aligned: you can use that space to annotate the key-combo you should use next time.

As usual, for more info check :help menu