Gimple

Lately I've been toying around with gcc to learn a bit better how its optimization phases work. Understanding Gimple, the intermediate representation used by gcc, is a useful skill for this. Of course actually *understanding* it is quite an ambitious and daunting task, so it may be a bit more useful to skim through it.

Turns out that using -fdump-tree-all and -fdump-rtl-all its possible to get a lot of interesting information on the phases the compiler follows to get your code optimized, but the sheer amount of information produced makes it rather hard to make sense out of it. During the next few posts (weeks? months? probably until I satisfy my curiosity about gcc) I will be investigating a little bit the output of the -fdump options in gcc, to see what can be learned from it.

Vim tip: Replacing builtin commands

If you spent a day writting a cool new version of the "tabnew" Vim command, you'll probably want it to be the default command used to open new tabs. Right?

Luckily, there's an easy way to replace built in commands with custom ones: cabbrev. cabbrev will do a textual replacement, so if you add "cabbrev tabnew TabNew" to your vimrc, eachtime you type "tabnew" it will be translated to TabNew.

Bonus tip: The command is actually, "abbrev", not "cabbrev". The "c" stands for command: it's telling Vim that you want the abbrev command applied in command mode. You can also use it as "nabbrev" to have it applied in normal mode. "abbrev" it's a nice way to correct common typos!

Public service announcement: searching your terminal's output

Short tip today, but a life-changer one: you don't need to copy&paste your terminal's scrollback to search on it, you can do it in place. At least in terminator that's possible (and I hear it's also doable in Gnome's default terminal application). Just press Ctrl+Shift+F. No more copy and pasting to vim!

Vim tip: autocommands

Whenever you find yourself thinking "I wish Vim could do this automagically for me", you probably are thinking about autocommands. With autocommands, autocmd for short or au for lazy people, will let you tell Vim, "Hey, use this callback when an event occurs".

The basic structure is pretty simple: "autocmd Event FileType Action". So, for example, "autocmd BufEnter *.txt call Rot13()" would tell vim to set a callback on BufEnter, that is whenever you change buffers, for all *.txt files, which will rot13 your text. Feel free to use this for actually useful things, like spell checking or auto indenting.

Bash tip: idiom to get the first error code from a pipe

When writing a bash script, often times you'll end up with something like this:

real_command | filter_stuff | prettify | do_something_else

The problem arises when you try to figure out if your command succeeded or not. If you `echo $?` you'll get the return code for the last chain in the pipe. You don't really care about the output value of do_something_else, do you?

I haven't found a solution I really like to this problem, but this idiom is handy:

out=`real_command` && echo $out | filter_stuff | prettify | do_something_else
echo $?

Now $? will hold the value of real_command, and you can actually use it to diagnose the real problem.

smaps: A quick memory analysis

Many times you see your process' memory consumption skyrocketing even though you're quite certain you have no memory leaks. This usually marks for the beginning of a very lengthy debugging process with valgrind or a similar tool, but even so some times you might get stuck trying to debug some third party library.

There's a quick tip in Linux that can help you track down a lib gone haywire:

cat /proc/<pid>/smaps

smaps will report every mapped section of memory for a certain process, how big the memory allocation is and which binary created the allocation.

Some new set operations in C++11 stl

The std header has a few cool additions that make life easier in C++11:

void f() {
  vector<int> v = {1, 2, 3, 4, 5, 60, 70, 80, 90};

  auto less_than_10 = [](int x){ return x < 10; };
  if (all_of(v.begin(), v.end(), less_than_10)) {
    cout << "Yay!";
  }
}

Besides all_of, in you can also find any_of and none_of.

Bonus: do you find that initializer list hideous? Just use std::iota, from stl too:

vector<int> v(100, 0);
iota(v.begin(), v.end(), 0);

Goto hell

Small and funny easter egg I found by accident today: stop using "adb shell" to access your device. You're wasting precious keystrokes. Instead, you should be using "adb hell". Yes, 'adb hell' works just as well as 'adb shell' - but it's even more awesome. Go and try it!

PSA: OEM unlocks may result in wiped filesystems

o, I bricked my tablet. Turns out the bootloader couldn't mount /data: after doing an oem unlock thingy, /data gets wiped and (this is the part the manual I was following didn't warn me about) no filesys is created.

If this happens, go back to recovery mode, then adb shell and run 'mount /data'. This will give you an error like "Can't mount /dev/block/mmcblk0p23". Write down the /dev/block id and run 'mke2fs -t ext4 /dev/block/mmcblk0p30'. That should fix it.

In some systems you may be missing libext2_quota.so. If this happens, just look for libext2_quota.so in the interwebs, then adb push this file into /sbin.

KDE: Lock screen from CLI

For some reason, one of my (seriously outdated) Kubuntu installations has the nasty habit of not locking the screen when pressing Ctrl+Alt+L. Not always, though. It seems to do this only when I'm in a hurry and need to quickly lock my PC before walking away. This happens often enough to be annoying, but not so frequently as to bother me enough to look for a proper solution. Instead of looking for a proper solution, trying to determine what's stealing the focus of the Ctrl+Alt+L key command, I just settled for an easier workaround: lock the screen from the command line. I use the terminal most of the time anyway, so why not just use it to lock the screen as well? The magic incantation is easy, if a bit cryptic at first:

qdbus org.freedesktop.ScreenSaver /ScreenSaver Lock

"qdbus" is a broadcasting service for KDE (Qt, actually). This command basically tells the screen saver service to lock the screen. Works every time, and with an alias in my bashrc, I don't need to remember that horribly long command. Now I only need to determine if my computer detecting when I'm in a hurry is a sign of sentience, and whether this is a threat to mankind. Will report soon.

VLCFreemote: no need to leave the couch

I've been quite prolific in my Github account recently, if I may say so myself. The latest of my open-source projects which I think is more or less ready to be "released to the world" is VlcFreemote, a remote control for VLC in Android. From its README file:

How many times have you been comfortably watching a movie from the couch only to find out you forgot the subtitles? How about being snugly tucked under a blanket, only to find out you need to brave the cold of winter just to add a new episode of your latest binge-watching series? Yeah, that can easily ruin your day.

Worry no more: with VlcFreemote you can now install a tiny Android app to control your VLC server.

FAQ:

• Another VLC remote? Why?There are a few VLC remote controls out there. I think this is the only once that's open source (not 100% sure). It has some nice extra features I haven't found in other remote controls too: bookmarks, automagic movie-skip (jump forward by a percentage of the file length, much more useful than it sounds!) a compact layout and other small things probably not even worth mentioning. In the future, whenever I get some free time, I'd like to add the ability to start VLC automatically from SSH, a feature I would use a lot and I have seen nowhere else.
• Why isn't this in Google play?Mostly cause I'm lazy and cheap. Getting a Google Play account costs 10 dollars or so, and I'm too lazy and too cheap to get one. Want me to upload it to Google Play? Feel free to buy me a beer. If not you can just get the APK from Github, or download the source code and build it yourself.

Don't exit gdb just to run make!

f you're more or less successful in your debugging session, it's quite likely that you'll have to modify some source code so you can actually fix a bug. And if you're more or less careful, you might want to validate your changes actually work. We saw some time ago that you don't need to restart gdb after a recompile because gdb is already smart enough to know that the binary changed.

Turns out you don't even need to drop from gdb to a shell: just type make (using the parameters you'd usually call make with) and watch gdb take care of building your binary again.

Rebuilding your project like this is not only useful to save time: you can also keep your breakpoints and they should still make sense, assuming you didn't refactor your code too much.

On the poor state of geotagging applications for Linux

tl;dr

I hacked together IMGeotagger, a Geotagger for Linux (though it should work in Windows too) that uses Google maps.

Not-so-tl;dr-version

Recently, after coming back from a trip, I tried to geotag (*) my pictures. I don't have a fancy GPS device for my camera but I do have a fairly good memory. There are applications that will let you drop your pics to a map, then get the GPS coordinates out of that. Unfortunately, the current state of geotagging applications in Linux is just sad.

Gotten Geography mostly works. I don't find it very nice to use, though. Pictag doesn't even work anymore, although I have hacked a version which at least manages to start in modern Ubuntu setups. That puts it more or less at the same level as Gotten Geography. Both suffer from a fatal problem: there is no latin charset maps for places that don't use a latin alfabet. Now, this is clearly not a fault in either program.

Both Gotten and Pictag use the (incredibly awesome) Open Street Map project. OSM provides some default map rendered tiles, and those are in the "local" language. Transliterating the local written name of a street is not an easy task. The rules in each place are completely different, some places have a completely different name in latin chars than they do in the local alphabet and a million other problems that can't be solved as a general case.

What to do when there are no readily available OSM tiles with latin chars? There is a fairly good product that does provide latin names for most (all?) places in the world: Google maps. Now, there is a reason neither Gotten nor Pictag use Google maps for geotagging: Google maps has no (free?) API to get tiles which you can embed in an application. It does let users view their maps in a browser, though. And the map URI is a nice and easy way to translate from map-tiles to map-coordinates.

I'm not 100% sure if IMGeotagger falls under Google maps terms of use, so I may have to take it down in the future. What it does is pretty simple: you use a browser to select a place, then IMGeotagger retrieves the location from the URI of the browser. This will break when Google maps changes their URI structure; until then, IMGeotagger works pretty well and it uses a really nice map (sorry OSM, G. Maps are pretty good).

You can grab the IMGeotagger (and its source code, as it's open source) from https://github.com/nicolasbrailo/IMGeotagger.

(*) What's geotagging? That's adding GPS coordinates on the exif metadata of your pictures. This is only useful to nerds and very pedantic people who enjoy analyzing photo albums to get GPS plots and other nerdy things like that.

gdb-tui and the previous-command problem

Raise your hand if you have run gdb in tui (graphical) mode, only to find you can't refer to the previous command when pressing "up". I can't see you but I know this is true for pretty much everyone reading this blog. All three of you.

In the gdb-TUI mode, the arrow keys are used by the active window for scrolling. This means they are not available for readline, the component that takes care of the magic invocations needed to bring back the previous command from the land of the dead. Luckily there are alternative readline keybindings: just try C-p, C-n, C-b and C-f. Takes a while getting used to it but you can finally use gdb-TUI and forget about copy-pasting every gdb command.

Bonus tip: if pressing "up" (or C-p) in gdb doesn't bring back the previous command, it probably means you don't have the readline package installed. Go ahead an install it. It'll change your life.

Android studio and ndk-gdb to debug a native app

I don't know how good Android Studio support for native apps is nowadays (it changes from week to week!). Up to a few months ago, Gradle, the build system used by AS, had poor support for native development. If you're having problems, you may find it easier to workaround it completely when it comes to build and debug C/C++ applications.

To debug a native Android application, a binary called gdbserver and its associated gdb.setup must be included in the generated APK file. Including this into the APK can be very painful in Gradle, so here's a workaround I found:

1. Build your stuff the way you normally would (I'm assuming you know already how to build a native app, and if you don't there are guides online that explain it much better than I could).
2. Deploy your application the way you normally would.
3. Discover ndk-gdb won't run. Bang forehead against keyboard a few times.
4. After losing some hours looking at logs, figure out there's no gdbserver included in your apk.
5. Lose some more hours trying to figure out how to include it in your apk using Gradle.
6. Give up. Bang forehead against keyboard some more.
7. find the gdbserver and gdb.setup in your build directory.
8. adb push each of these files to the device.
9. Using adb shell, move the files you copied to /data/app-lib/com.yourapp/ - you may need to root your device for this.
10. Profit! ndk-gdb now works.

Edit: remember you may need to chmod +777 your gdbserver.

Useful predefined variables in make

I always forget about two very useful make variables, so I'll leave this here: $(COMPILE.cpp), $(LINK.cpp). It's easy, instead of writing a rule as

foo.o: foo.cpp
  g++ -c foo.coo

you should instead write this:

foo.o: foo.cpp
  $(COMPILE.cpp) foo.coo

COMPILE.cpp will have the default compiler you are supposed to use, and probably some helpful parameters as well. Likewise, LINK.cpp will have the linker you are supposed to use.

There are many useful predefined variables in make. Be sure to check them all by running "make -p" in a console.

Vim tip: reload your vimrc

If you're changing your vimrc, it can get boring to close and restart it only to see the changes applied. Want something quicker? You can ":so %". So stands for source, so you'll just be telling vim "include this file". % happens to be the path to the current file. If you're not editing your .vimrc but for some reason you still want to reload it, just use "so ~/.vimrc" instead.

ndk-gdb life tip: use --verbose

Crosscompiling is always fun. No matter how ready-to-use it's packaged, and Android does a pretty decent job at that, you're still bound to find problems that leak through the abstraction layers. If something says it's dummy-proof, I always find the way to perfect myself and be even dumber. For people like me; do yourselves a favour and start launching ndk-gdb this way:

ndk-gdb --start --verbose

Using the --verbose parameter will probably reveal some hidden errors. For example, when I forgot to chmod 777 my gdbserver binary:

## COMMAND: adb_cmd pull /system/bin/app_process ./obj/local/armeabi-v7a/app_process
run-as: exec failed for /data/data/com.nico.trippingsdcardphotomanager/lib/gdbserver Error:Permission denied
117 KB/s (9560 bytes in 0.079s)
Pulled app_process from device/emulator.

I now write Android apps: presenting Tripping Photo Manager

I have been working in an awesome Android app recently, so I'd like to throw in a shameless self-plug here. Oh, wait, it's my blog. It's all basically a big shameless self-plug, isn't it? Anyway, I've been working on https://github.com/nicolasbrailo/TrippingSdCardPhotoManager , an open source SD card photo manager for Android.

Why an SD card photo manager? Whenever I go on holidays I never have enough SD cards to store all the snaps I take. Luckily I'm a crappy photographer, so I end up deleting half of the pictures I took during a day. This makes it easy to somewhat re-use the same SD card. In my experience, Android is not always great when it comes to managing files from an external SD card mounted through an USB adapter, hence this little app was born: it'll let you select the directory you want to use to manage pictures, from anywhere in the filesystem. It's also somewhat faster than the native gallery app, which is a plus when working with a slow-ish SD card.

This app also supports some stuff I find useful in my workflow, like renaming the current directory to something more meaningful than "YourCamera4242", backing stuff up in the device and batch deleting files. It also packs a version of ImageMagick I ported for Android, so in theory you can use your Android device to do anything you can do with a regular "mogrify" command in Linux.

The app is not available in Playstore, mostly because I'm a cheap bastard and don't want to pay the 10 bucks Google charges you to create an account, only to publish an open source application.

You can still install the APK from this link: https://github.com/nicolasbrailo/TrippingSdCardPhotoManager/releases - or you can contribute to the open source world (?) and buy me a Play Store account.

Debugging multiple processes with gdb

If your buggy program generates lots of child processes, gdb will keep attached to the parent program and let all the children run loose. If you're having problems to find what causes  your crash this is probably not what you want: for those occasions gdb has a very helpful flag called detach on fork.

With detach on fork you can tell gdb to keep debugging the parent, follow the children, or keep track of all processes. Must be nice to troubleshoot forkbombs with this option.