Google Test: Quarantine for tests?

Google Test: Putting a test under quarantine

GTest works wonders for c++ testing, even more so when combined with GMock. I've been using these frameworks for a few side projects. I've seen them used in large scale projects too. In all cases, there is a very common problem for which (I think) there is no elegant solution: managing temporarily disabled tests.

It may be because you found a flaky piece of code or a test that exposes a heisenbug. Maybe the test itself is just unstable, or perhaps you are using TDD and want to submit a test to your CI before its implementation is ready. In these cases, you can choose to disable the offending test or let it run, possible halting your CI because of it. When that happens, you maybe masking other, real, problems.

Most people would stick a "DISABLED_" before the test name, to let GTest know not to run it. Maybe even stick a "// TODO: reenable" in there too. When run, GTest will generate a message to let you know there is a disabled test. Even so, I find that people -myself included- tend to forget to re-enable the disabled tests.

For one of my side projects, I hacked GTest to quarantine tests up to a certain date:

TEST(Foo, Bar) {
    QUARANTINE_UNTIL("16/8/22");
    EXPECT_EQ(1,2);
}

In my CI setup, that test will be showing a happy green (and a warning, which I will probably ignore) until the 22nd of August. By the 23rd the test will run again and fail if I haven't fixed the code. If I have indeed fixed it, it will print a warning to remind me that it's safe to delete the quarantine statement.

Is there any advantage in this approach over the usual _DISABLE strategy? In my opinion, there is: if you ignore warnings in your test, for whatever reason, a _DISABLE might go unnoticed and it may hide a real problem. In the same scenario, for a quarantined test, nothing bad happens: the warning just says "you should delete this line" but the quarantined test is again part of your safety net.

How does it work? The first caveat in my article mentions it: hackishly. There are a few facilities missing in GTest to make this implementation production-ready but, ugly as it looks, it should work as intended:

#include <ctime>
#include <string>
#include <sstream>
std::string now() {
    time_t t = time(0);
    struct tm *now = localtime(&t);

    std::stringstream formatted_date;
    formatted_date << (now->tm_year+1900) << '/'
                   << (now->tm_mon+1) << '/'
                   << now->tm_mday;

    return formatted_date.str();
}

#define QUARANTINE_UNTIL(date_limit)                                     \
        if (now() < date_limit) {                                        \
            GTEST_LOG_(WARNING) << "Test under quarantine!";             \
            return;                                                      \
        } else {                                                         \
            GTEST_LOG_(WARNING) << "Quarantine expired on " date_limit;  \
        }

If I find there is interest in this approach for real world applications, I may try to come up with a nicer interface for it.

Vimcrypt

Have you ever been working on your plans for world domination but got scared someone else might find them? It happens to me all the time. Or maybe you are so paranoid that you need to encrypt your grocery list. Perhaps you are sharing a semi-private text file through a public service? Good news, Vim has you covered. Just type ":X". Vim will ask you for a password. Save your file again and voila, your file is now encrypted. Open the same file with Vim to decrypt it. Your plans for world domination are now safe!

Things you should never do

I think I may start a new series in my "Rants" category: things you shouldn't do. First one: Never ever use "strange" characters on your wifi's AP name, where strange is defined as non-ascii. I made the huge mistake of choosing a name with an ñ on it, then had to spend an entire evening hacking on a printer driver with no unicode support. No, I couldn't have changed the AP's name. That would have required me to physically connect a computer to my router, and I was too lazy to get up from the couch.

Self reminder: setting the default boot option in UEFI

Bought a new laptop (*) and I'm 100% sure I'll forget this if I don't put it here:

From http://askubuntu.com/questions/291905/how-can-i-make-ubuntu-the-default-boot-option-on-a-newer-laptop-uefi

To set Ubuntu as the default boot OS in a multi-OS setup (ie, dual boot Windows) with UEFI, goto Windows and exec (as admin) bcdedit /set {bootmgr} path \EFI\ubuntu\grubx64.efi

Why am I using Windows, you may ask? I'm still in the process of discovering which features will be broken and which hardware will work out of the box. So far I'm actually quite surprised, with only the video card and the touchpad not working. Luckily bash doesn't use either of those. Who needs a mouse anyway?

Simple vim plugin IV: project greping

I recently wrote about some of the utilities I created for my Vim setup. Using someone else's Vim scripts is not nearly as fun as writing your own, so I decided to also write a short summary on what it takes to get started writting Vim plugins. For this task, I decided to start with greping.

Greping can be improved a bit: if you do it a lot in a project, you might find it's useful to also grep the results themselves, to further refine your search. If you have your grep results in vim itself, that is trivial.

Let's start hacking something in our .vimrc file. Try this:

function! FG_Search()
    let needle = input("Search for: ")
    tabnew
    setlocal buftype=nofile bufhidden=wipe nobuflisted noswapfile nowrap
    let grepbin = 'grep -nri '
    let cmd = grepbin . ' "' . needle . '" *'
    execute '$read !' . cmd
    setlocal nomodifiable
endfunction

map <leader>s :call FG_Search()<CR>

This function should be pretty clear: it will map <leader>s (in my case, ",s") to FG_Search(). FG_Search will prompt the user for a term to grep, then search for it executing the command. In the end the results are written to a new tab, which is declared as a temp non-modifiable buffer.

Just paste that in your .vimrc and you're good to grep.

Extra tip: integrate this with my fast grep cache and you have a nice and quick project search integration for vim that works even for very large projects with tools available in most default Linux installs.

Simple vim plugin III: a polymorphic project greper

We've recently seen a very basic function to integrate grep to vim. We can improve it a little bit with very simple changes. Using this tip to have different key binding for different modes we can do something a bit smarter . Let's create two functions, one for normal mode that should prompt the user what to search for, and another function to automagically pick whatever is selected:

function! FG_DoSearch(needle)
    let grepbin = 'grep -nri '

    tabnew
    setlocal buftype=nofile bufhidden=wipe nobuflisted noswapfile nowrap
    let cmd = grepbin . ' "' . a:needle . '" *'
    execute '$read !' . cmd
    setlocal nomodifiable
endfunction

function! FG_Search()
    let needle = input("Search for: ")
    call FG_DoSearch(needle)
endfunction

function! FG_Visual_Search()
    " Copy whatever is selected in visual mode
    try
        silent! let a_save = @a
        silent! normal! gv"ay
        silent! let needle = @a
    finally
        silent! let @a = a_save
    endtry

    call FG_DoSearch(needle)
endfunction

nmap <leader>s :call FG_Search()<CR>
vmap <leader>s :call FG_Visual_Search()<CR>

The magic here happens in the mapping: nmap will create a mapping that's only enabled when on "normal" mode, vmap when you're in visual mode. As usual, check :help map for more details.

Simple vim plugin II: a psychic project greper

We have been working on a quick grep integration for Vim, and it's looking decent enough for a quick plugin. There's one more easy thing we can improve, though: let's make it psychic! So far we had to tell grep what to look for, either by selecting the text in visual mode or by actually typing the search terms. Typing! That's so old fashioned. Let's make grep guess what to look for.

In vim you have a psychic function, expand(""). If you call expand(""), it will return whatever word is under the cursor. No need to visually select it. If you're still using the same vimrc definitions, you can do something like

nmap s :call FG_DoSearch(expand(""))

Let's clean things up a little bit:

" Wrap a grep command: search for needle, show results in a new window
function! FG_DoSearch(needle)
    let grepbin = 'grep -nri '

    tabnew
    setlocal buftype=nofile bufhidden=wipe nobuflisted noswapfile nowrap
    let cmd = grepbin . ' "' . a:needle . '" *'
    echom cmd
    execute '$read !' . cmd
    setlocal nomodifiable
endfunction

" Wrap a normal action: ask the user for input, then call func with it
function! FG_RequestInputAction(msg, func)
    let needle = input(a:msg)
    if strlen(needle) > 0
        execute 'call' a:func .'("'. needle . '")'
    endif
endfunction

" Wrap a visual action: call func with whatever is selected under the cursor
function! FG_VAction(func)
    " Copy whatever is selected in visual mode
    try
        silent! let a_save = @a
        silent! normal! gv"ay
        silent! let needle = @a
    finally
        silent! let @a = a_save
    endtry

    " Remove whitespaces
    let needle = substitute(needle, "\\n\\+","","g") 
    let needle = substitute(needle, "\\r\\+","","g") 
    let needle = substitute(needle, "^\\s\\+\\|\\s\\+$","","g") 

    if strlen(needle) > 0
        execute 'call' a:func .'("'. needle . '")'
    endif
endfunction

" Wrap a normal action: call func with whatever is under the cursor
function! FG_NAction(func)
    let needle = expand("<cword>")
    if strlen(needle) > 0
        execute 'call' a:func .'("'. needle . '")'
    endif
endfunction

nmap <leader>s :call FG_NAction("FG_DoSearchText")<CR>
vmap <leader>s :call FG_VAction("FG_DoSearchText")<CR>
map  <leader>S :call FG_RequestInputAction("Text search: ", "FG_DoSearchText")<CR>

Just copy paste that in your vimrc, now you can grep your project in three different ways: press s (,s) to look for the word currently under the cursor, S to type in a search term or select something in visual mode, then S to grep it.

Simple vim plugin I: integrating new commands

TL;DR: Here's some code to integrate system commands into vim. You can just drop it in your vimrc, create a small wrapper function in your vimrc and configure a few key binding to make it work.

Longer version:
We can extend our quick grep integration to other commands, quite easily. Since we defined a few wrappers to request input, get it from visual mode or just guess it, we can also have a helper function to create a scratch buffer and read a system command into it:

" Find&Grep command wrapper: execute cmd, shows the results in a scratch buffer
function! FG_EvalSysCmdInNewBuff(cmd)
    tabnew
    setlocal buftype=nofile bufhidden=wipe nobuflisted noswapfile nowrap
    execute '$read !' . a:cmd
    setlocal nomodifiable
endfunction

" Wrap a normal action: ask the user for input, then call func with it
function! FG_RequestInputAction(msg, func)
    let needle = input(a:msg)
    if strlen(needle) > 0
        execute 'call' a:func .'("'. needle . '")'
    endif
endfunction

" Wrap a visual action: call func with whatever is selected under the cursor
function! FG_VAction(func)
    " Copy whatever is selected in visual mode
    try
        silent! let a_save = @a
        silent! normal! gv"ay
        silent! let needle = @a
    finally
        silent! let @a = a_save
    endtry

    " Remove whitespaces
    let needle = substitute(needle, "\\n\\+","","g")
    let needle = substitute(needle, "\\r\\+","","g")
    let needle = substitute(needle, "^\\s\\+\\|\\s\\+$","","g") 

    if strlen(needle) > 0
        execute 'call' a:func .'("'. needle . '")'
    endif
endfunction

" Wrap a normal action: call func with whatever is under the cursor
function! FG_NAction(func)
    let needle = expand("<cword>")
    if strlen(needle) > 0
        execute 'call' a:func .'("'. needle . '")'
    endif
endfunction

Integrating any new command into our plugin is now trivial. Let's do it for grep and for find:

" Wrap a find command: search for file "needle", show results in a new window
function! FG_DoFindFile(needle)
    let cmd = 'find -type f -iname "*' . a:needle . '*"'
    call FG_EvalSysCmdInNewBuff(cmd)
endfunction

" Wrap a grep command: search for needle, show results in a new window
function! FG_DoSearchText(needle)
    let cmd = 'grep -nri "' . a:needle . '" *'
    call FG_EvalSysCmdInNewBuff(cmd)
endfunction

Then just add a few key bindings and you're good to go:

gt;f :call FG_NAction("FG_DoFindFile")<CR>
vmap <leader>f :call FG_VAction("FG_DoFindFile")<CR>
map  <leader>S :call FG_RequestInputAction("Text search: ", "FG_DoSearchText")<CR>

nmap <leader>s :call FG_NAction("FG_DoSearchText")<CR>
vmap <leader>s :call FG_VAction("FG_DoSearchText")<CR>
map  <leader>F :call FG_RequestInputAction("Find file: ", "FG_DoFindFile")<CR>

This is an actual plugin I use in my Vim setup. You can grab the latest version from my Github repo.

Extra tip: add these too if you want to have a GUI menu for your new commands as well.

menu Project.Find\ File :call FG_RequestInputAction("FG_DoFindFile")
menu Project.Text\ Search :call FG_RequestInputAction("FG_DoSearchText")

The best hack you should never use

Please don't do this. But if you do: leave a comment here!

#define private public;
#include "something";
#define private private;

Found in some random project.

Vim tip: custom commands

If you have a function that you use a lot, you may find it interesting to use a custom command for it. Try this:

:command Foo echo('Hola!')

Now invoke the command with ':Foo' and Vim should say hello. Neat, huh? This is especially useful (and dangerous) when combined with cabbrev, like this:

:command! Foobar echo('Nope!')
:cabbrev close Foobar

If you try to :close a document, Vim will now say "Nope!". Other than using this to mess with someone's Vim session, you can replace builtin commands with your own tweaked functions. I tend to use that quite frequently in my own .vimrc.

Ageing Stack overflow?

Hacking away on a little side project of mine, I found myself checking Stack Overflow for implementation tips about things I don't usually work with. Android UI stuff, mostly, which apparently is a very dynamic and ever changing ecosystem. After more than a few wasted hours, I noticed a worrying trend: in SO, answers tend to age horribly. If you are looking for "How to do X in platform Y", you may find a 4 year old answer that solves the problem, but only for platform Y, version "ancient".

Information ageing is quite a problem on its own. The answer is still valid, and, for people working on that specific platform, probably relevant. This will make it the first answer, leaving a lot of people (like myself) frustrated because the solution won't work in newer platforms. Is there a solution? Implement some kind of ageing time-window for information? Make the date a more prominent search parameter? Explicitly specify your platform and environment's version when asking a question? I have no idea.

While Stack Overflow seems to exacerbate the issue, this is a problem even for products with a company actively maintaining their documentation. A very annoying example; looking for ways to manage the media key I ended up in a page which (as of August 2016) points to a very outdated API (registerMediaButtonEventReceiver, in case you are wondering). If even Google encounters problems when managing documentation ageing for their own products, what can we expect of people like us, who only have a tiny fraction of that budget?

/Rant

Awesome quote stolen from "The Embedded Muse"

While I try to keep this blog to (more or less) original content I write, I liked this quote from the Embedded Muse so much that I had to steal it:

Norm Augustine, retired chairman and CEO of Lockheed Martin, wrote a book called Augustine's Laws that documents observations he made as a government contractor. One of those was that by the year 2054 the entire US defense budget would buy just one aircraft, due to their spiraling costs. In the May 9-22, 2016 issue of Aviation Week and Space Technology (often called Aviation Leak and Secret Technology due to their outing of many programs) he wrote this revealing statement:

"Another law from the 1960s used historical data, fearlessly extrapolated, to show than an ever-increasing fraction of an aircraft's weight was being devoured by electronics and in 2015 would leave no room for pilots. Thus, we have today's dilemma wherein pilots must remain in Nevada while their aircraft burn holes in the sky all around the planet. The Wright Brother surely are doing barrel rolls in their graves.

"Of course, some observers have questioned how, as these laws seem to imply, it would be possible to endlessly add cost to an aircraft when nothing more can be added to an aircraft's weight, not even a pilot. To do this would require a colorless, odorless, weightless substance whose cost - like entropy - would continue to increase forever. Fortunately, the industry's engineers were up to the task: the elusive substance is software."

Vim utilities: Findgrep & Fastgrep

I spent some time writing utility scripts for my Vim setup. I figured I can share them here, someone may even find them useful or at least get a laugh out of it. Last time I presented "Impl switcher", and "Better Tab New" before that. Today it's Findgrep & Fastgrep's turn.

I wrote about Fastgrep a long time ago. The idea behind it is to speed up the slowest part in a grep command, the disk seek time, by creating a huge blob file with all the files in a project concatenated.

Fastgrep works great. But it requires a context switch, going from your IDE to your console just to grep. Findgrep fills the gap between the IDE and the command line: this utility provides a few key bindings to let you quickly run some common commands, like searching for a selected string or finding a file in the project directory.

You can get Fastgrep here. Findgrep is available in Github, and you can easily replace Fastgrep with normal grep if you need to.

Vim utilities: Impl switcher

I spent some time writing utility scripts for my Vim setup. I figured I can share them here, someone may even find them useful or at least get a laugh out of it. Last time I presented "BTN: Better Tab New". Today it's the turn for "Impl switcher".

With its very imaginative name, "Impl switcher" has a very obvious purpose: it will just switch from a header file to an implementation file. So, between .h and .cpp. Surely there are lots of Vim plugins to do just that, why write another one?

Most impl-switcher plugins in the wild tend to, in my experience, require either a lot of configuration, a lot of dependencies, or a very specific project layout (or a combination of all three). They also seem to be huge and very complicated projects.

Impl switcher will recursively go up your directory hierarchy and use "find" to locate any files named like your base file but with a different extension. That makes it very simple and it requires minimal (if any) configuration: just drop it in your .vimrc file and you're good to go. OK, not exactly: it requires a Linux-like system with utilities like "find". Still, a good trade-off to keep the project's dependencies as small as possible.

Get "impl swicher" here:

• Git hub repo
• Impl switcher's Vim page

Vim utilities: Better Tab New

I spent some time writing utility scripts for my Vim setup. I figured I can share them here, someone may even find them useful or at least get a laugh out of it. The first one is called "Better Tab New" and you can get it from my Github repo or from the script's Vim page.

Why would you want a better tab new, and what's wrong with the default one? Simple: tabnew, in Vim, will just open a file. For that to happen you need to specify the exact path of a file. That's usually perfectly fine, but sometimes you need tabnew to be a bit smarter: maybe you just grep'ed something and ended up with a path that looks like "/foo/bar/baz:my_text:42". Or maybe you want to open a file and go to a specific line. Those are things for which the default tabnew implementation isn't very good. BTN fills that niche and lets you create a simpler workflow when using grep.

Get BTN: Better Tab New here:

• Gihub repo
• BTN's Vim page

Shared pointers: don't

Ahh, shared pointers! A nice, magical pointer type that will make all of your memory problems go away. Sprinkle some shared_ptrs here and there and, voilà, Valgrind is now happy. Sounds like a silver bullet, doesn't it? That should be your first clue that something's up with the promise of getting Java-like memory management in your c++ application.

Java and C(++) have a very different concept of memory management. My Java-foo, obviously enough to anyone reading this blog, is not that great, but, from experience, memory management is seen as a chore better left to the bowels of your system, something you don't need (nor want) to care about. Sure, you can tweak and somewhat manage memory allocations if you really want to; the default mindset, however, is to ignore those concerns. The garbage collector will, eventually, find any unused resource and deal with it accordingly.

C++ programs, at least those that have been more or less successfully designed as opposed to organically grown, tend to have a radically different approach. Memory management is an integral part of a program's design and it can't be left to some automagic memory manager. This leads, again, for those more or less successful designs, to programs with a tree-like hierarchy in which a child or dependent object must live at least as long as its parent scope. This hierarchy leads to easier to understand programs. Some idioms (RAII!) depend on this hierarchy. Some tools (like scoped and unique pointers) make its implementation much simpler. I've seen that Rust really builds on this idea (and seems to take it to 11! I'm still trying to figure out if that's a good or a bad thing, but so far I quite like it).

The tree-like structure of the scopes in C++ also implies single ownership (again something Rust seems to be very idiosyncratic about). While you may "use" someone else's objects (for example, via a reference) there is always one single owner. If this owner goes away while someone holds a reference to one of its children... well, you get a bug. But sure enough this bug is clear as long as you can visualize the tree scope structure of your program. Shared pointers completely obliterate this tree.

A shared pointer means an object can have multiple owners. Whoever goes out of scope last needs to clean it. Why is that bad? In my (highly subjective but experience based) opinion:

• It becomes harder to reason about your program. You never know if all the "pieces" you need are in scope. Is an object already initialized? Who is responsible for building it? If you call a method with side effects, will any of the other owners be affected by it?
• It becomes harder to predict whether going out of scope is trivial, or an operation that can take a minute. If you're the last one holding a reference to an object through a shared pointer, you may be stuck running its destructor for a long time. That's not necessarily a bad thing, but not being able to predict it can lead to all sort of obscure bugs.

There are also many ways to avoid shared pointer usage:

• Try restructuring your code. This will usually yield the biggest benefits, you'll end up with a clearer structure and less clutter.
• Try to use scoped pointers from boost or unique pointers if you can. Way too often shared pointers are used when a scoped pointer would be enough.
• Don't be scared of copies! Sometimes you can just copy your object and end up with cleaner (and maybe even faster) code. Are you really sure you need to share state?

Does that mean you should never ever use shared pointers? Of course not. In some cases it's unavoidable. Some algorithms are probably impossible to implement without them (or even impossible without GC). A shared pointer is just one more tool. Just like gotos. And, just like gotos - although not quite as dangerous - they have some appropriate use cases. Just try not to make it your default goto (lol) tool for memory management.

// TODO: There is a very good reason I found to use shared pointers: to create weak_ptr's. Is there a good solution without actually using shared_ptr's? I don't know!

C++: Why is undefinedness important

Let's start with an example:

int *x = (int*) NULL;
x[0] = 42;

Luckily so far I've never seen anyone argue about this one: we all know we're dealing with undefined behavior and that it's bad. Things get a bit more tricky when the example is not so trivial.

C's abstract machine

In a way, C and C++ describe a "virtual machine". This is what the standard defines: what kind of operations are valid in this VM. This VM resembles an old single-thread mono-processor architecture. Most often, the code will run in a modern architecture that will resemble very little the design of C's VM. "New" features (like caching, vectorization, atomics, pipelining, etc) implemented by the target architecture make the process of mapping our code (in the VM that C defines) much more difficult. The compiler needs to map instructions in C's simple architecture to a much (*MUCH*) more complex design. To do that, it needs to analyze the code to guarantee certain constrains are met.

Let's see how these constrains and undefined behavior relate to each other with this snippet:

template <typename T>
bool always_true(T x) {
return (x < x+1);
}

From a math perspective, and assuming that T is a numeric type, always_true should always return true. Is that the case for C's virtual machine?

If we call always_true with a type like "unsigned int", then x+1 may overflow and wrap around. This is fine because unsigned int's are allowed to wrap around. What happens if instead we use a signed type? Things get more interesting.

Signed types are not allowed to overflow. If they do, the standard says the behavior is undefined. And the standard also says that our program can not invoke undefined behavior. This is a very important phrase: the standard says undefined behavior can NOT occur. There is no "if it does": it just can't, so the compiler will assume that UB will never happen. What if it does happen? Nasal demons, that's what!

Knowing that UB can't happen, and in our example above, the compiler can assume that x+1 will never overflow. If it will never overflow, (x<x+1) will always be true.

The compiler, by analyzing our program, can detect what conditions might trigger undefined behavior. By knowing that undefined behavior is not allowed, it can assume those conditions will never happen. That's why, for the sample above, any optimizing-compiler will just produce code similar to "return true", at least for -O2.

Undefined behavior is not (only) to make programmer's lives miserable, it actually is needed to create optimizing compilers.

Deobfuscate your bash

Who hasn't written some read-only magical Bash voodoo, only to find you need to decrypt your own creation later on? Luckily, Explain Shell can help with that. Here's an example from my .bash_history file:

for fn in *; do echo cat $fn | sed "s|' '$URL'||g" | sed "s|curl -X POST -d '||g" ; done

And Explain Shell's explanation: it's no substitute for knowing Bash but it sure helps.

Bonus: while reading my bash history file, I realized I accidentally copy&paste a lot of code to my terminals. There are way too many "template <FOO>" entries in there...

Bonus II: It's a good thing they wrote "shell" in a different color. I was wondering why I had "explains hell" in my bookmarks.

Awesome (and useless) trivia: Ubuntu's first bug

I'm still not over the disappointment from my latest Ubuntu install but recently I found a bug which is quite remarkable: https://bugs.launchpad.net/ubuntu/+bug/1

Yes, Ubuntu's #1 bug, reported by Shuttleworth himself, is "Microsoft too big". I'm not too sure I agree with the bug's resolution.

KSnapshot is getting smarter

I just noticed KSnapshot is smart. Too smart. If you save a snapshot to a folder with a bunch of files like "Snapshot_N_foo" it'll name the next one "Snapshot_N+1_foo". That already makes my computer smarter than some humans!

I can't believe this works!

Are you bored? Try pasting this, as is, in a cpp file:

// What is going on here??/
Is this even legal C++??/
Yes, it is!

NB: You may have to use -trigraphs to compile this. Try it out! You can use this command:

echo -e "// What is going on here??/Is this legal C++?" | g++ -E -c -trigraphs -

With some luck, this won't be legal C++ anymore after C++ 17 deprecates trigraphs.

Some sane advice from Firefox's console

Tinkering with Firefox' JS console I had to copypaste some stuff. I got a really nice surprise when it wouldn't let me:

Scam Warning: Take care when pasting things you don't understand. This could allow attackers to steal your identity or take control of your computer.

That's some really good advice coming from your browser's console.

Quickly sharing files in Linux via HTTP

Isn't it awful when you have to share a file too big for email and don't know how? You'd think by 2016 we'd have that figured out. Actually we do, many times over. Just pick a standard that works for you!

If you don't want to read many pages on file transfer standards (Samba? What's that?) you can try this little snippet:

python -m SimpleHTTPServer $PORT

This will create an http server sharing the current directory. HTTP, luckily, is one of those things that tend to work everywhere, always.

Bonus: some other ways of doing the same thing at https://gist.github.com/willurd/5720255

Initialization oddities: Aggregate initialization

Do you know the quickest way to create a constructor that initializes the elements in this struct?

#include <string>
struct MyStruct {
    int x;
    std::string y;
    const char *z;
};

If you answered "by typing really fast", you may be interested in knowing that the fastest way to create this constructor is to not write it at all!

MyStruct a = {42, "Hello", "World"};

Yes, the line above works and it's perfectly legal C++. It's event C++ 98! This language feature is called aggregate initialization and it says the compiler should be smart enough to initialize MyStruct using each value successively. Of course C++11 has made this syntax somewhat simpler and a lot more uniform:

MyStruct a{42, "Hello", "World"};

There are some caveats when using this initialization, namely that the initialized type must be an aggregate. An aggregate, in standard lingo, is a type that has some restrictions. No virtuals, no privates, etc. You can say it's a POD and in most cases you'd be right.

Now, is this also legal?

MyStruct a = {42, "Hello"};

You'd be tempted to say that's a syntax error. It's not, now z will just be default-initialized. What about this, then?

MyStruct a = {42, "Hello", "World", "Extra!"};

According to the standard, that's an error. Or... is it? Let's try out this example:

struct A {
    int x;
};

struct B {
    A a;
    std::string y;
};

struct C {
    B b;
    const char *z;
};

C o = {42, "Hello", "World"};

Yes. Believe it or not, the object o will now contain three members: o.b.a.x, o.b.y and o.z. All three will be properly initialized with their respective value.

Aggregate initializations should, according to the standard, be smart enough to initialize aggregate objects and use any "spill over" to continue initializing other values/aggregate objects recursively.

Bonus I:

Aggregate initialization is also what makes this idiom valid:

char x[] = {1, 2, 3}

In this case, x will be of length 3 because that's the length of its aggregate initializer.

Bonus II:

I'm sure anyone trying to get up to date with C++11 will have played around with variadic templates. One of the first exercises I'd recommend for this would be a compile-time list of different types. Knowing about aggregate initializations now, how would you write a constructor for this type?

template <typename H, typename... T>
struct Multilist<H, T...> {
    H x;
    Multilist<T...> next;
};

Multilist<int, string, float> foo{42, "XXX", 1.23};

Code and Google translate: awesomeness

Some time ago I found out one of my articles was translated to another language (yay for that, woo for not letting me know about it). To understand what my own article said, I had to use Google translate on the site. Guess what? c++ and Google translate can produce hilarious results:

# The include "throw.h" the extern "the C" {void seppuku () {throw statement the Exception () ; }}

Another one I liked:

the struct the Exception {};
#ifdef __cplusplus
  the extern "the C" {
#endif

void seppuku ();
# Ifdef __cplusplus
}
# endif

Now you know it. Next time you're looking at some incomprehensible c++ code, run it through Google translate. It may improve it.

Ubuntu 15.10: Ubuntu ME

Warning: semi-useless rant ahead. TL;DR: Avoid Ubuntu 15.10 - it's the closest Linux has ever been to Windows ME.

I have been using Ubuntu for a while now. From the time when Canonical actually mailed real, physical CDs of the distro. So get of my lawn.

In all of my Linux years I have never, ever, had such a horrible installation experience as I did this weekend with Ubuntu 15.10. I may go as far as saying not even Windows ME was this horrible to install. I hit dozens of critical show-stopper bugs, from poor UEFI support to an installer that crashed when clicking "Go back". And that's only the installation, don't get me started on the new KDE Plasma 5 desktop... (hint: my big desktop screen is NOT a phone. Swiping to login? Bad idea for a mouse).

A few hints for any other poor souls that made the fatal mistake of installing Ubuntu ME:

• UEFI? Say no. Get a different computer if you can. Try to set it in legacy mode if you can not.
• Try not to repartition and install Ubuntu on the same go. Even more so if you have UEFI. First install, then rearrange partitions with a live cd.
• If you get a few (or a few dozen) "system crash notifications" when starting up your GUI, check /var/crash. Delete everything from there.
• If you want Kubuntu, don't install Ubuntu and then apt-get install kubuntu-desktop. That's broken. If you want Kubuntu just get its install image.
• Don't install Kubuntu. Really, it's horrible and it crashes. (You though I called 15.10 Ubuntu ME for no reason?)
• Don't like Gnome? XFCE is usable and can be configured to look more or less like a sane version of KDE. It still crashes but at least it's quick to boot.
• If you get a disk check on every startup just disable it on fstab. No, it's not nice. I haven't found any other workarounds yet.

I have no idea when has Ubuntu gone so horribly bad, but I'm not looking forward to installing any Ubuntu distro anytime soon. I wonder what Slackware looks like these days.

2016? Still not the year of the Linux desktop.

Update: XFCE is great... except it doesn't really support moving the mouse. Seems Ubuntu is having a nostalgic release and decided to introduce old bugs from 2012!

Gimple

Lately I've been toying around with gcc to learn a bit better how its optimization phases work. Understanding Gimple, the intermediate representation used by gcc, is a useful skill for this. Of course actually *understanding* it is quite an ambitious and daunting task, so it may be a bit more useful to skim through it.

Turns out that using -fdump-tree-all and -fdump-rtl-all its possible to get a lot of interesting information on the phases the compiler follows to get your code optimized, but the sheer amount of information produced makes it rather hard to make sense out of it. During the next few posts (weeks? months? probably until I satisfy my curiosity about gcc) I will be investigating a little bit the output of the -fdump options in gcc, to see what can be learned from it.

Vim tip: Replacing builtin commands

If you spent a day writting a cool new version of the "tabnew" Vim command, you'll probably want it to be the default command used to open new tabs. Right?

Luckily, there's an easy way to replace built in commands with custom ones: cabbrev. cabbrev will do a textual replacement, so if you add "cabbrev tabnew TabNew" to your vimrc, eachtime you type "tabnew" it will be translated to TabNew.

Bonus tip: The command is actually, "abbrev", not "cabbrev". The "c" stands for command: it's telling Vim that you want the abbrev command applied in command mode. You can also use it as "nabbrev" to have it applied in normal mode. "abbrev" it's a nice way to correct common typos!

Public service announcement: searching your terminal's output

Short tip today, but a life-changer one: you don't need to copy&paste your terminal's scrollback to search on it, you can do it in place. At least in terminator that's possible (and I hear it's also doable in Gnome's default terminal application). Just press Ctrl+Shift+F. No more copy and pasting to vim!

Vim tip: autocommands

Whenever you find yourself thinking "I wish Vim could do this automagically for me", you probably are thinking about autocommands. With autocommands, autocmd for short or au for lazy people, will let you tell Vim, "Hey, use this callback when an event occurs".

The basic structure is pretty simple: "autocmd Event FileType Action". So, for example, "autocmd BufEnter *.txt call Rot13()" would tell vim to set a callback on BufEnter, that is whenever you change buffers, for all *.txt files, which will rot13 your text. Feel free to use this for actually useful things, like spell checking or auto indenting.

Bash tip: idiom to get the first error code from a pipe

When writing a bash script, often times you'll end up with something like this:

real_command | filter_stuff | prettify | do_something_else

The problem arises when you try to figure out if your command succeeded or not. If you `echo $?` you'll get the return code for the last chain in the pipe. You don't really care about the output value of do_something_else, do you?

I haven't found a solution I really like to this problem, but this idiom is handy:

out=`real_command` && echo $out | filter_stuff | prettify | do_something_else
echo $?

Now $? will hold the value of real_command, and you can actually use it to diagnose the real problem.

smaps: A quick memory analysis

Many times you see your process' memory consumption skyrocketing even though you're quite certain you have no memory leaks. This usually marks for the beginning of a very lengthy debugging process with valgrind or a similar tool, but even so some times you might get stuck trying to debug some third party library.

There's a quick tip in Linux that can help you track down a lib gone haywire:

cat /proc/<pid>/smaps

smaps will report every mapped section of memory for a certain process, how big the memory allocation is and which binary created the allocation.

Some new set operations in C++11 stl

The std header has a few cool additions that make life easier in C++11:

void f() {
  vector<int> v = {1, 2, 3, 4, 5, 60, 70, 80, 90};

  auto less_than_10 = [](int x){ return x < 10; };
  if (all_of(v.begin(), v.end(), less_than_10)) {
    cout << "Yay!";
  }
}

Besides all_of, in you can also find any_of and none_of.

Bonus: do you find that initializer list hideous? Just use std::iota, from stl too:

vector<int> v(100, 0);
iota(v.begin(), v.end(), 0);

Goto hell

Small and funny easter egg I found by accident today: stop using "adb shell" to access your device. You're wasting precious keystrokes. Instead, you should be using "adb hell". Yes, 'adb hell' works just as well as 'adb shell' - but it's even more awesome. Go and try it!

PSA: OEM unlocks may result in wiped filesystems

o, I bricked my tablet. Turns out the bootloader couldn't mount /data: after doing an oem unlock thingy, /data gets wiped and (this is the part the manual I was following didn't warn me about) no filesys is created.

If this happens, go back to recovery mode, then adb shell and run 'mount /data'. This will give you an error like "Can't mount /dev/block/mmcblk0p23". Write down the /dev/block id and run 'mke2fs -t ext4 /dev/block/mmcblk0p30'. That should fix it.

In some systems you may be missing libext2_quota.so. If this happens, just look for libext2_quota.so in the interwebs, then adb push this file into /sbin.

KDE: Lock screen from CLI

For some reason, one of my (seriously outdated) Kubuntu installations has the nasty habit of not locking the screen when pressing Ctrl+Alt+L. Not always, though. It seems to do this only when I'm in a hurry and need to quickly lock my PC before walking away. This happens often enough to be annoying, but not so frequently as to bother me enough to look for a proper solution. Instead of looking for a proper solution, trying to determine what's stealing the focus of the Ctrl+Alt+L key command, I just settled for an easier workaround: lock the screen from the command line. I use the terminal most of the time anyway, so why not just use it to lock the screen as well? The magic incantation is easy, if a bit cryptic at first:

qdbus org.freedesktop.ScreenSaver /ScreenSaver Lock

"qdbus" is a broadcasting service for KDE (Qt, actually). This command basically tells the screen saver service to lock the screen. Works every time, and with an alias in my bashrc, I don't need to remember that horribly long command. Now I only need to determine if my computer detecting when I'm in a hurry is a sign of sentience, and whether this is a threat to mankind. Will report soon.

VLCFreemote: no need to leave the couch

I've been quite prolific in my Github account recently, if I may say so myself. The latest of my open-source projects which I think is more or less ready to be "released to the world" is VlcFreemote, a remote control for VLC in Android. From its README file:

How many times have you been comfortably watching a movie from the couch only to find out you forgot the subtitles? How about being snugly tucked under a blanket, only to find out you need to brave the cold of winter just to add a new episode of your latest binge-watching series? Yeah, that can easily ruin your day.

Worry no more: with VlcFreemote you can now install a tiny Android app to control your VLC server.

FAQ:

• Another VLC remote? Why?There are a few VLC remote controls out there. I think this is the only once that's open source (not 100% sure). It has some nice extra features I haven't found in other remote controls too: bookmarks, automagic movie-skip (jump forward by a percentage of the file length, much more useful than it sounds!) a compact layout and other small things probably not even worth mentioning. In the future, whenever I get some free time, I'd like to add the ability to start VLC automatically from SSH, a feature I would use a lot and I have seen nowhere else.
• Why isn't this in Google play?Mostly cause I'm lazy and cheap. Getting a Google Play account costs 10 dollars or so, and I'm too lazy and too cheap to get one. Want me to upload it to Google Play? Feel free to buy me a beer. If not you can just get the APK from Github, or download the source code and build it yourself.

Don't exit gdb just to run make!

f you're more or less successful in your debugging session, it's quite likely that you'll have to modify some source code so you can actually fix a bug. And if you're more or less careful, you might want to validate your changes actually work. We saw some time ago that you don't need to restart gdb after a recompile because gdb is already smart enough to know that the binary changed.

Turns out you don't even need to drop from gdb to a shell: just type make (using the parameters you'd usually call make with) and watch gdb take care of building your binary again.

Rebuilding your project like this is not only useful to save time: you can also keep your breakpoints and they should still make sense, assuming you didn't refactor your code too much.

On the poor state of geotagging applications for Linux

tl;dr

I hacked together IMGeotagger, a Geotagger for Linux (though it should work in Windows too) that uses Google maps.

Not-so-tl;dr-version

Recently, after coming back from a trip, I tried to geotag (*) my pictures. I don't have a fancy GPS device for my camera but I do have a fairly good memory. There are applications that will let you drop your pics to a map, then get the GPS coordinates out of that. Unfortunately, the current state of geotagging applications in Linux is just sad.

Gotten Geography mostly works. I don't find it very nice to use, though. Pictag doesn't even work anymore, although I have hacked a version which at least manages to start in modern Ubuntu setups. That puts it more or less at the same level as Gotten Geography. Both suffer from a fatal problem: there is no latin charset maps for places that don't use a latin alfabet. Now, this is clearly not a fault in either program.

Both Gotten and Pictag use the (incredibly awesome) Open Street Map project. OSM provides some default map rendered tiles, and those are in the "local" language. Transliterating the local written name of a street is not an easy task. The rules in each place are completely different, some places have a completely different name in latin chars than they do in the local alphabet and a million other problems that can't be solved as a general case.

What to do when there are no readily available OSM tiles with latin chars? There is a fairly good product that does provide latin names for most (all?) places in the world: Google maps. Now, there is a reason neither Gotten nor Pictag use Google maps for geotagging: Google maps has no (free?) API to get tiles which you can embed in an application. It does let users view their maps in a browser, though. And the map URI is a nice and easy way to translate from map-tiles to map-coordinates.

I'm not 100% sure if IMGeotagger falls under Google maps terms of use, so I may have to take it down in the future. What it does is pretty simple: you use a browser to select a place, then IMGeotagger retrieves the location from the URI of the browser. This will break when Google maps changes their URI structure; until then, IMGeotagger works pretty well and it uses a really nice map (sorry OSM, G. Maps are pretty good).

You can grab the IMGeotagger (and its source code, as it's open source) from https://github.com/nicolasbrailo/IMGeotagger.

(*) What's geotagging? That's adding GPS coordinates on the exif metadata of your pictures. This is only useful to nerds and very pedantic people who enjoy analyzing photo albums to get GPS plots and other nerdy things like that.

gdb-tui and the previous-command problem

Raise your hand if you have run gdb in tui (graphical) mode, only to find you can't refer to the previous command when pressing "up". I can't see you but I know this is true for pretty much everyone reading this blog. All three of you.

In the gdb-TUI mode, the arrow keys are used by the active window for scrolling. This means they are not available for readline, the component that takes care of the magic invocations needed to bring back the previous command from the land of the dead. Luckily there are alternative readline keybindings: just try C-p, C-n, C-b and C-f. Takes a while getting used to it but you can finally use gdb-TUI and forget about copy-pasting every gdb command.

Bonus tip: if pressing "up" (or C-p) in gdb doesn't bring back the previous command, it probably means you don't have the readline package installed. Go ahead an install it. It'll change your life.