syntax highlight

Thursday, 1 December 2016

Simple vim plugin IV: project greping

I recently wrote about some of the utilities I created for my Vim setup. Using someone else's Vim scripts is not nearly as fun as writing your own, so I decided to also write a short summary on what it takes to get started writting Vim plugins. For this task, I decided to start with greping.

Greping can be improved a bit: if you do it a lot in a project, you might find it's useful to also grep the results themselves, to further refine your search. If you have your grep results in vim itself, that is trivial.

Let's start hacking something in our .vimrc file. Try this:

function! FG_Search()
    let needle = input("Search for: ")
    tabnew
    setlocal buftype=nofile bufhidden=wipe nobuflisted noswapfile nowrap
    let grepbin = 'grep -nri '
    let cmd = grepbin . ' "' . needle . '" *'
    execute '$read !' . cmd
    setlocal nomodifiable
endfunction

map <leader>s :call FG_Search()<CR>

This function should be pretty clear: it will map <leader>s (in my case, ",s") to FG_Search(). FG_Search will prompt the user for a term to grep, then search for it executing the command. In the end the results are written to a new tab, which is declared as a temp non-modifiable buffer.

Just paste that in your .vimrc and you're good to grep.

Extra tip: integrate this with my fast grep cache and you have a nice and quick project search integration for vim that works even for very large projects with tools available in most default Linux installs.

Wednesday, 30 November 2016

Simple vim plugin III: a polymorphic project greper

We've recently seen a very basic function to integrate grep to vim. We can improve it a little bit with very simple changes. Using this tip to have different key binding for different modes we can do something a bit smarter . Let's create two functions, one for normal mode that should prompt the user what to search for, and another function to automagically pick whatever is selected:

function! FG_DoSearch(needle)
    let grepbin = 'grep -nri '

    tabnew
    setlocal buftype=nofile bufhidden=wipe nobuflisted noswapfile nowrap
    let cmd = grepbin . ' "' . a:needle . '" *'
    execute '$read !' . cmd
    setlocal nomodifiable
endfunction

function! FG_Search()
    let needle = input("Search for: ")
    call FG_DoSearch(needle)
endfunction

function! FG_Visual_Search()
    " Copy whatever is selected in visual mode
    try
        silent! let a_save = @a
        silent! normal! gv"ay
        silent! let needle = @a
    finally
        silent! let @a = a_save
    endtry

    call FG_DoSearch(needle)
endfunction

nmap <leader>s :call FG_Search()<CR>
vmap <leader>s :call FG_Visual_Search()<CR>

The magic here happens in the mapping: nmap will create a mapping that's only enabled when on "normal" mode, vmap when you're in visual mode. As usual, check :help map for more details.

Tuesday, 29 November 2016

Simple vim plugin II: a psychic project greper

We have been working on a quick grep integration for Vim, and it's looking decent enough for a quick plugin. There's one more easy thing we can improve, though: let's make it psychic! So far we had to tell grep what to look for, either by selecting the text in visual mode or by actually typing the search terms. Typing! That's so old fashioned. Let's make grep guess what to look for.

In vim you have a psychic function, expand(""). If you call expand(""), it will return whatever word is under the cursor. No need to visually select it. If you're still using the same vimrc definitions, you can do something like

nmap s :call FG_DoSearch(expand(""))

Let's clean things up a little bit:

" Wrap a grep command: search for needle, show results in a new window
function! FG_DoSearch(needle)
    let grepbin = 'grep -nri '

    tabnew
    setlocal buftype=nofile bufhidden=wipe nobuflisted noswapfile nowrap
    let cmd = grepbin . ' "' . a:needle . '" *'
    echom cmd
    execute '$read !' . cmd
    setlocal nomodifiable
endfunction

" Wrap a normal action: ask the user for input, then call func with it
function! FG_RequestInputAction(msg, func)
    let needle = input(a:msg)
    if strlen(needle) > 0
        execute 'call' a:func .'("'. needle . '")'
    endif
endfunction

" Wrap a visual action: call func with whatever is selected under the cursor
function! FG_VAction(func)
    " Copy whatever is selected in visual mode
    try
        silent! let a_save = @a
        silent! normal! gv"ay
        silent! let needle = @a
    finally
        silent! let @a = a_save
    endtry

    " Remove whitespaces
    let needle = substitute(needle, "\\n\\+","","g") 
    let needle = substitute(needle, "\\r\\+","","g") 
    let needle = substitute(needle, "^\\s\\+\\|\\s\\+$","","g") 

    if strlen(needle) > 0
        execute 'call' a:func .'("'. needle . '")'
    endif
endfunction

" Wrap a normal action: call func with whatever is under the cursor
function! FG_NAction(func)
    let needle = expand("<cword>")
    if strlen(needle) > 0
        execute 'call' a:func .'("'. needle . '")'
    endif
endfunction

nmap <leader>s :call FG_NAction("FG_DoSearchText")<CR>
vmap <leader>s :call FG_VAction("FG_DoSearchText")<CR>
map  <leader>S :call FG_RequestInputAction("Text search: ", "FG_DoSearchText")<CR>

Just copy paste that in your vimrc, now you can grep your project in three different ways: press s (,s) to look for the word currently under the cursor, S to type in a search term or select something in visual mode, then S to grep it.

Thursday, 24 November 2016

Simple vim plugin I: integrating new commands

TL;DR: Here's some code to integrate system commands into vim. You can just drop it in your vimrc, create a small wrapper function in your vimrc and configure a few key binding to make it work.

Longer version:
We can extend our quick grep integration to other commands, quite easily. Since we defined a few wrappers to request input, get it from visual mode or just guess it, we can also have a helper function to create a scratch buffer and read a system command into it:

" Find&amp;Grep command wrapper: execute cmd, shows the results in a scratch buffer
function! FG_EvalSysCmdInNewBuff(cmd)
    tabnew
    setlocal buftype=nofile bufhidden=wipe nobuflisted noswapfile nowrap
    execute '$read !' . a:cmd
    setlocal nomodifiable
endfunction

" Wrap a normal action: ask the user for input, then call func with it
function! FG_RequestInputAction(msg, func)
    let needle = input(a:msg)
    if strlen(needle) &gt; 0
        execute 'call' a:func .'("'. needle . '")'
    endif
endfunction

" Wrap a visual action: call func with whatever is selected under the cursor
function! FG_VAction(func)
    " Copy whatever is selected in visual mode
    try
        silent! let a_save = @a
        silent! normal! gv"ay
        silent! let needle = @a
    finally
        silent! let @a = a_save
    endtry

    " Remove whitespaces
    let needle = substitute(needle, "\\n\\+","","g")
    let needle = substitute(needle, "\\r\\+","","g")
    let needle = substitute(needle, "^\\s\\+\\|\\s\\+$","","g") 

    if strlen(needle) &gt; 0
        execute 'call' a:func .'("'. needle . '")'
    endif
endfunction

" Wrap a normal action: call func with whatever is under the cursor
function! FG_NAction(func)
    let needle = expand("&lt;cword&gt;")
    if strlen(needle) &gt; 0
        execute 'call' a:func .'("'. needle . '")'
    endif
endfunction

Integrating any new command into our plugin is now trivial. Let's do it for grep and for find:

" Wrap a find command: search for file "needle", show results in a new window
function! FG_DoFindFile(needle)
    let cmd = 'find -type f -iname "*' . a:needle . '*"'
    call FG_EvalSysCmdInNewBuff(cmd)
endfunction

" Wrap a grep command: search for needle, show results in a new window
function! FG_DoSearchText(needle)
    let cmd = 'grep -nri "' . a:needle . '" *'
    call FG_EvalSysCmdInNewBuff(cmd)
endfunction
Then just add a few key bindings and you're good to go:
gt;f :call FG_NAction("FG_DoFindFile")&lt;CR&gt;
vmap &lt;leader&gt;f :call FG_VAction("FG_DoFindFile")&lt;CR&gt;
map  &lt;leader&gt;S :call FG_RequestInputAction("Text search: ", "FG_DoSearchText")&lt;CR&gt;

nmap &lt;leader&gt;s :call FG_NAction("FG_DoSearchText")&lt;CR&gt;
vmap &lt;leader&gt;s :call FG_VAction("FG_DoSearchText")&lt;CR&gt;
map  &lt;leader&gt;F :call FG_RequestInputAction("Find file: ", "FG_DoFindFile")&lt;CR&gt;

This is an actual plugin I use in my Vim setup. You can grab the latest version from my Github repo.

Extra tip: add these too if you want to have a GUI menu for your new commands as well.

menu Project.Find\ File :call FG_RequestInputAction("FG_DoFindFile")
menu Project.Text\ Search :call FG_RequestInputAction("FG_DoSearchText")

Tuesday, 22 November 2016

The best hack you should never use

Please don't do this. But if you do: leave a comment here!


#define private public;
#include "something";
#define private private;


Found in some random project.

Thursday, 17 November 2016

Vim tip: custom commands

If you have a function that you use a lot, you may find it interesting to use a custom command for it. Try this:

:command Foo echo('Hola!')

Now invoke the command with ':Foo' and Vim should say hello. Neat, huh? This is especially useful (and dangerous) when combined with cabbrev, like this:

:command! Foobar echo('Nope!')
:cabbrev close Foobar

If you try to :close a document, Vim will now say "Nope!". Other than using this to mess with someone's Vim session, you can replace builtin commands with your own tweaked functions. I tend to use that quite frequently in my own .vimrc.

Tuesday, 15 November 2016

Ageing Stack overflow?

Hacking away on a little side project of mine, I found myself checking Stack Overflow for implementation tips about things I don't usually work with. Android UI stuff, mostly, which apparently is a very dynamic and ever changing ecosystem. After more than a few wasted hours, I noticed a worrying trend: in SO, answers tend to age horribly. If you are looking for "How to do X in platform Y", you may find a 4 year old answer that solves the problem, but only for platform Y, version "ancient".

Information ageing is quite a problem on its own. The answer is still valid, and, for people working on that specific platform, probably relevant. This will make it the first answer, leaving a lot of people (like myself) frustrated because the solution won't work in newer platforms. Is there a solution? Implement some kind of ageing time-window for information? Make the date a more prominent search parameter? Explicitly specify your platform and environment's version when asking a question? I have no idea.

While Stack Overflow seems to exacerbate the issue, this is a problem even for products with a company actively maintaining their documentation. A very annoying example; looking for ways to manage the media key I ended up in a page which (as of August 2016) points to a very outdated API (registerMediaButtonEventReceiver, in case you are wondering). If even Google encounters problems when managing documentation ageing for their own products, what can we expect of people like us, who only have a tiny fraction of that budget?

/Rant

Thursday, 4 August 2016

Awesome quote stolen from "The Embedded Muse"

While I try to keep this blog to (more or less) original content I write, I liked this quote from the Embedded Muse so much that I had to steal it:
Norm Augustine, retired chairman and CEO of Lockheed Martin, wrote a book called Augustine's Laws that documents observations he made as a government contractor. One of those was that by the year 2054 the entire US defense budget would buy just one aircraft, due to their spiraling costs. In the May 9-22, 2016 issue of Aviation Week and Space Technology (often called Aviation Leak and Secret Technology due to their outing of many programs) he wrote this revealing statement:

"Another law from the 1960s used historical data, fearlessly extrapolated, to show than an ever-increasing fraction of an aircraft's weight was being devoured by electronics and in 2015 would leave no room for pilots. Thus, we have today's dilemma wherein pilots must remain in Nevada while their aircraft burn holes in the sky all around the planet. The Wright Brother surely are doing barrel rolls in their graves.

"Of course, some observers have questioned how, as these laws seem to imply, it would be possible to endlessly add cost to an aircraft when nothing more can be added to an aircraft's weight, not even a pilot. To do this would require a colorless, odorless, weightless substance whose cost - like entropy - would continue to increase forever. Fortunately, the industry's engineers were up to the task: the elusive substance is software."

Wednesday, 6 July 2016

Vim utilities: Findgrep & Fastgrep

I spent some time writing utility scripts for my Vim setup. I figured I can share them here, someone may even find them useful or at least get a laugh out of it. Last time I presented "Impl switcher", and "Better Tab New" before that. Today it's Findgrep & Fastgrep's turn.

I wrote about Fastgrep a long time ago. The idea behind it is to speed up the slowest part in a grep command, the disk seek time, by creating a huge blob file with all the files in a project concatenated.

Fastgrep works great. But it requires a context switch, going from your IDE to your console just to grep. Findgrep fills the gap between the IDE and the command line: this utility provides a few key bindings to let you quickly run some common commands, like searching for a selected string or finding a file in the project directory.

You can get Fastgrep here. Findgrep is available in Github, and you can easily replace Fastgrep with normal grep if you need to.

Monday, 4 July 2016

Vim utilities: Impl switcher

I spent some time writing utility scripts for my Vim setup. I figured I can share them here, someone may even find them useful or at least get a laugh out of it. Last time I presented "BTN: Better Tab New". Today it's the turn for "Impl switcher".

With its very imaginative name, "Impl switcher" has a very obvious purpose: it will just switch from a header file to an implementation file. So, between .h and .cpp. Surely there are lots of Vim plugins to do just that, why write another one?

Most impl-switcher plugins in the wild tend to, in my experience, require either a lot of configuration, a lot of dependencies, or a very specific project layout (or a combination of all three). They also seem to be huge and very complicated projects.

Impl switcher will recursively go up your directory hierarchy and use "find" to locate any files named like your base file but with a different extension. That makes it very simple and it requires minimal (if any) configuration: just drop it in your .vimrc file and you're good to go. OK, not exactly: it requires a Linux-like system with utilities like "find". Still, a good trade-off to keep the project's dependencies as small as possible.

Get "impl swicher" here:

Friday, 1 July 2016

Vim utilities: Better Tab New

I spent some time writing utility scripts for my Vim setup. I figured I can share them here, someone may even find them useful or at least get a laugh out of it. The first one is called "Better Tab New" and you can get it from my Github repo or from the script's Vim page.

Why would you want a better tab new, and what's wrong with the default one? Simple: tabnew, in Vim, will just open a file. For that to happen you need to specify the exact path of a file. That's usually perfectly fine, but sometimes you need tabnew to be a bit smarter: maybe you just grep'ed something and ended up with a path that looks like "/foo/bar/baz:my_text:42". Or maybe you want to open a file and go to a specific line. Those are things for which the default tabnew implementation isn't very good. BTN fills that niche and lets you create a simpler workflow when using grep.

Get BTN: Better Tab New here:

Wednesday, 15 June 2016

Shared pointers: don't

Ahh, shared pointers! A nice, magical pointer type that will make all of your memory problems go away. Sprinkle some shared_ptrs here and there and, voilà, Valgrind is now happy. Sounds like a silver bullet, doesn't it? That should be your first clue that something's up with the promise of getting Java-like memory management in your c++ application.

Java and C(++) have a very different concept of memory management. My Java-foo, obviously enough to anyone reading this blog, is not that great, but, from experience, memory management is seen as a chore better left to the bowels of your system, something you don't need (nor want) to care about. Sure, you can tweak and somewhat manage memory allocations if you really want to; the default mindset, however, is to ignore those concerns. The garbage collector will, eventually, find any unused resource and deal with it accordingly.

C++ programs, at least those that have been more or less successfully designed as opposed to organically grown, tend to have a radically different approach. Memory management is an integral part of a program's design and it can't be left to some automagic memory manager. This leads, again, for those more or less successful designs, to programs with a tree-like hierarchy in which a child or dependent object must live at least as long as its parent scope. This hierarchy leads to easier to understand programs. Some idioms (RAII!) depend on this hierarchy. Some tools (like scoped and unique pointers) make its implementation much simpler. I've seen that Rust really builds on this idea (and seems to take it to 11! I'm still trying to figure out if that's a good or a bad thing, but so far I quite like it).

The tree-like structure of the scopes in C++ also implies single ownership (again something Rust seems to be very idiosyncratic about). While you may "use" someone else's objects (for example, via a reference) there is always one single owner. If this owner goes away while someone holds a reference to one of its children... well, you get a bug. But sure enough this bug is clear as long as you can visualize the tree scope structure of your program. Shared pointers completely obliterate this tree.

A shared pointer means an object can have multiple owners. Whoever goes out of scope last needs to clean it. Why is that bad? In my (highly subjective but experience based) opinion:

  • It becomes harder to reason about your program. You never know if all the "pieces" you need are in scope. Is an object already initialized? Who is responsible for building it? If you call a method with side effects, will any of the other owners be affected by it?
  • It becomes harder to predict whether going out of scope is trivial, or an operation that can take a minute. If you're the last one holding a reference to an object through a shared pointer, you may be stuck running its destructor for a long time. That's not necessarily a bad thing, but not being able to predict it can lead to all sort of obscure bugs.

There are also many ways to avoid shared pointer usage:

  • Try restructuring your code. This will usually yield the biggest benefits, you'll end up with a clearer structure and less clutter.
  • Try to use scoped pointers from boost or unique pointers if you can. Way too often shared pointers are used when a scoped pointer would be enough.
  • Don't be scared of copies! Sometimes you can just copy your object and end up with cleaner (and maybe even faster) code. Are you really sure you need to share state?

Does that mean you should never ever use shared pointers? Of course not. In some cases it's unavoidable. Some algorithms are probably impossible to implement without them (or even impossible without GC). A shared pointer is just one more tool. Just like gotos. And, just like gotos - although not quite as dangerous - they have some appropriate use cases. Just try not to make it your default goto (lol) tool for memory management.

// TODO: There is a very good reason I found to use shared pointers: to create weak_ptr's. Is there a good solution without actually using shared_ptr's? I don't know!

Thursday, 2 June 2016

C++: Why is undefinedness important

Let's start with an example:

int *x = (int*) NULL;
x[0] = 42;

Luckily so far I've never seen anyone argue about this one: we all know we're dealing with undefined behavior and that it's bad. Things get a bit more tricky when the example is not so trivial.

C's abstract machine

In a way, C and C++ describe a "virtual machine". This is what the standard defines: what kind of operations are valid in this VM. This VM resembles an old single-thread mono-processor architecture. Most often, the code will run in a modern architecture that will resemble very little the design of C's VM. "New" features (like caching, vectorization, atomics, pipelining, etc) implemented by the target architecture make the process of mapping our code (in the VM that C defines) much more difficult. The compiler needs to map instructions in C's simple architecture to a much (*MUCH*) more complex design. To do that, it needs to analyze the code to guarantee certain constrains are met.

Let's see how these constrains and undefined behavior relate to each other with this snippet:

template <typename T>
bool always_true(T x) {
return (x < x+1);
}

From a math perspective, and assuming that T is a numeric type, always_true should always return true. Is that the case for C's virtual machine?

If we call always_true with a type like "unsigned int", then x+1 may overflow and wrap around. This is fine because unsigned int's are allowed to wrap around. What happens if instead we use a signed type? Things get more interesting.

Signed types are not allowed to overflow. If they do, the standard says the behavior is undefined. And the standard also says that our program can not invoke undefined behavior. This is a very important phrase: the standard says undefined behavior can NOT occur. There is no "if it does": it just can't, so the compiler will assume that UB will never happen. What if it does happen? Nasal demons, that's what!

Knowing that UB can't happen, and in our example above, the compiler can assume that x+1 will never overflow. If it will never overflow, (x<x+1) will always be true.

The compiler, by analyzing our program, can detect what conditions might trigger undefined behavior. By knowing that undefined behavior is not allowed, it can assume those conditions will never happen. That's why, for the sample above, any optimizing-compiler will just produce code similar to "return true", at least for -O2.

Undefined behavior is not (only) to make programmer's lives miserable, it actually is needed to create optimizing compilers.

Tuesday, 31 May 2016

Deobfuscate your bash

Who hasn't written some read-only magical Bash voodoo, only to find you need to decrypt your own creation later on? Luckily, Explain Shell can help with that. Here's an example from my .bash_history file:

for fn in *; do echo cat $fn | sed "s|' '$URL'||g" | sed "s|curl -X POST -d '||g" ; done

And Explain Shell's explanation: it's no substitute for knowing Bash but it sure helps.

Bonus: while reading my bash history file, I realized I accidentally copy&paste a lot of code to my terminals. There are way too many "template <FOO>" entries in there...

Bonus II: It's a good thing they wrote "shell" in a different color. I was wondering why I had "explains hell" in my bookmarks.

Thursday, 26 May 2016

Awesome (and useless) trivia: Ubuntu's first bug

I'm still not over the disappointment from my latest Ubuntu install but recently I found a bug which is quite remarkable: https://bugs.launchpad.net/ubuntu/+bug/1

Yes, Ubuntu's #1 bug, reported by Shuttleworth himself, is "Microsoft too big". I'm not too sure I agree with the bug's resolution.

Tuesday, 24 May 2016

KSnapshot is getting smarter

I just noticed KSnapshot is smart. Too smart. If you save a snapshot to a folder with a bunch of files like "Snapshot_N_foo" it'll name the next one "Snapshot_N+1_foo". That already makes my computer smarter than some humans!

Thursday, 19 May 2016

I can't believe this works!

Are you bored? Try pasting this, as is, in a cpp file:

// What is going on here??/
Is this even legal C++??/
Yes, it is!

NB: You may have to use -trigraphs to compile this. Try it out! You can use this command:

echo -e "// What is going on here??/Is this legal C++?" | g++ -E -c -trigraphs -

With some luck, this won't be legal C++ anymore after C++ 17 deprecates trigraphs.

Tuesday, 17 May 2016

Some sane advice from Firefox's console

Tinkering with Firefox' JS console I had to copypaste some stuff. I got a really nice surprise when it wouldn't let me:

Scam Warning: Take care when pasting things you don't understand. This could allow attackers to steal your identity or take control of your computer.


That's some really good advice coming from your browser's console.

Thursday, 12 May 2016

Quickly sharing files in Linux via HTTP

Isn't it awful when you have to share a file too big for email and don't know how? You'd think by 2016 we'd have that figured out. Actually we do, many times over. Just pick a standard that works for you!

If you don't want to read many pages on file transfer standards (Samba? What's that?) you can try this little snippet:

python -m SimpleHTTPServer $PORT

This will create an http server sharing the current directory. HTTP, luckily, is one of those things that tend to work everywhere, always.

Bonus: some other ways of doing the same thing at https://gist.github.com/willurd/5720255

Tuesday, 10 May 2016

Initialization oddities: Aggregate initialization

Do you know the quickest way to create a constructor that initializes the elements in this struct?

#include <string>
struct MyStruct {
    int x;
    std::string y;
    const char *z;
};

If you answered "by typing really fast", you may be interested in knowing that the fastest way to create this constructor is to not write it at all!

MyStruct a = {42, "Hello", "World"};

Yes, the line above works and it's perfectly legal C++. It's event C++ 98! This language feature is called aggregate initialization and it says the compiler should be smart enough to initialize MyStruct using each value successively. Of course C++11 has made this syntax somewhat simpler and a lot more uniform:

MyStruct a{42, "Hello", "World"};

There are some caveats when using this initialization, namely that the initialized type must be an aggregate. An aggregate, in standard lingo, is a type that has some restrictions. No virtuals, no privates, etc. You can say it's a POD and in most cases you'd be right.

Now, is this also legal?

MyStruct a = {42, "Hello"};

You'd be tempted to say that's a syntax error. It's not, now z will just be default-initialized. What about this, then?

MyStruct a = {42, "Hello", "World", "Extra!"};

According to the standard, that's an error. Or... is it? Let's try out this example:

struct A {
    int x;
};

struct B {
    A a;
    std::string y;
};

struct C {
    B b;
    const char *z;
};

C o = {42, "Hello", "World"};

Yes. Believe it or not, the object o will now contain three members: o.b.a.x, o.b.y and o.z. All three will be properly initialized with their respective value.

Aggregate initializations should, according to the standard, be smart enough to initialize aggregate objects and use any "spill over" to continue initializing other values/aggregate objects recursively.

Bonus I:

Aggregate initialization is also what makes this idiom valid:

char x[] = {1, 2, 3}

In this case, x will be of length 3 because that's the length of its aggregate initializer.

Bonus II:

I'm sure anyone trying to get up to date with C++11 will have played around with variadic templates. One of the first exercises I'd recommend for this would be a compile-time list of different types. Knowing about aggregate initializations now, how would you write a constructor for this type?

template <typename H, typename... T>
struct Multilist<H, T...> {
    H x;
    Multilist<T...> next;
};

Multilist<int, string, float> foo{42, "XXX", 1.23};