Vim tim: quickly switch from header to impl

Switching from header to implementation in vim takes up precious milliseconds of typing and thinking, so we'd better delegate that to a computer. Instead of typing :tabnew FOO.cpp, just download A (for alternate) from this url: http://www.vim.org/scripts/script.php?script_id=31

Add it to your bundles in vim and, for extra magic, just map some key to :AT in your vimrc. I have added this one:

map <F4> :AT<CR>

I don't know how I lived without this for such a long time.

Setting up a Linux GW VIII: Proxy and content filtering

Now that we have a basic gateway we can do crazy stuff, like installing a proxy. You may want to manually configure a proxy for each client, but you can also choose to install a transparent proxy for all your users. This can be done with squid, let's see how.

Start by installing squid on your gateway. You can choose a different machine, but you'll have to do some magic with iptales. It's easier to just use the same machine.

Once squid is installed head to /etc/squid/ to vim squid.conf. Yes, it's very scary to see such a long config file, but it's mostly just comments. Luckily squid has reasonable defaults, so you can just ignore most of this file. Just to test if your squid installation was successful, before changing anything, you can tail -f /var/log/squid/access.log and set your browser's proxy to your gateway's IP, port 3128 (squid's default port). If everything works you should be able to browse and also see the access logs scrolling by.

If you are getting a 'denied' page on every request, you may have to configure squid to allow http access. Search for the 'http_access deny all' and comment it. You may also have to search for the local networks definitions and set it up correctly (something like 'acl localnet src 192.168.0.0/24').

Once you have verified that your proxy is working, you can configure it to run on transparent mode. Search for the http_port directive, and change it to something like 'http_port 8213 transparent' (noticed I changed the default port). It is also a good practice to specify IP and port, so squid can bind only to the local interface (you are probably not interested in serving as a proxy for the outside world, unless you plan to run a reverse proxy).

Changing squid to run on transparent mode is not enough, though. You will also need to tell your router to redirect every incoming packet from port 80 to squid. Assuming your LAN is on the 192.168.10.0/24 address and squid is listenning on port 1234, you can use this magic command to setup your iptables rule:

iptables -t nat -A PREROUTING -s 192.168.10.0/24 -p tcp --dport 80 -j DNAT --to :1234

If this doesn't work for you, or you want a more detailed explanation, you can check my post about this iptables rule.

Everything ready, you should be able to unconfigure the proxy from your browser and start using squid right away, no configuration needed. tail -f /var/log/squid/access.log for hours of (thought-policing) fun.

Adding a content filter to squid

Now that you have a gateway and a transparent proxy, it's time to install a content filter too. It's not hard, just go to your squid's config file and search for the acl section. Over there, add the following two lines:

acl blocksites url_regex "/home/router/blocked_sites.acl"
http_access deny blocksites

This will include the blocked_sites.acl file and deny access to every URL on it. There are many blacklist services out there, from which you can download a nice filter to suit your needs.

Of course, you probably don't want to restart squid each time a new site is added to your blocklist. For this you can use "squid -k reconfigure" to make squid reload its configuration.

Some random tips for squid:

• If you think your squid is responding too slowly, you can manually setup your DNS servers. Considering squid will most likely be installed on the gateway, it might be easier to just use the gateway's gateway for the DNS, instead of the bind service running on the box. You can set this option with the directive "dns_nameservers xxx.xxx.xxx.xxx yyy.yyy.yyy.yyy" on squid.conf.
• The TCP_MISS on the access.log means that a request was successfully served, but the content was not cached. You can review your cache limits if you get this message too much, may be you can increase the caching limit.
• You don't need to restart squid each time you change the configuration. That would be ackward if you have a lot of users. Try "squid -k reconfigure" instead.

Human friendly c declarations

An appropriate use of typedef's can transform 99% of c's gruesome type declarations into a mostly maintainable and maybe even readable piece of code. For that remaining 1%, or if you got a legacy application from someone with a very twisted mind, you'll probably need a way decode what "int (*(Foo::*foo)(void**))[3]" means.

To decipher weird c declarations go to http://cdecl.org/ and type your type. It works for most cases... good luck trying to figure out templates, though, for template metaprogramming you are on your own.

Setting up a Linux GW VII: Fun with iptables, setting up port forwardings

In any LAN you'll probably want to expose some services to the outer world, be it for a bittorrent connection or because you have internal servers you need to access from outside your internal LAN. To do this, you'll have to tell your router to forward some external port to an internal one, like this:

iptables -t nat -A PREROUTING -i eth0 -p tcp 
	--dport PORT -j DNAT --to INTERNAL_IP:INTERNAL_PORT

# This rule may not be needed, depending on other chain confings
iptables -A INPUT -i eth0 -p tcp -m state --state NEW 
	--dport PORT -j DNAT --to INTERNAL_IP:INTERNAL_PORT

This is enough to expose a private server to the world, but it will not be very useful when your dynamic IP changes, so you'll need to set INTERNAL_IP to be a static IP.

Of course, this commands are little less than black magic. iptables are rather complex and quite difficult to master, but as a short description we can say they are a way of applying a set of rules to incoming network packets. In iptables you have different tables of rules (in this case we use -t[able] nat) and specify that we want our rule to be applied in the PREROUTING phase. -i specifies that this rule should be applied only to packets incoming from eth0, and --dport means this rule applies only to packets incoming from a certain port. Of course, if you are going to specify a port then you need to specify the protocol (in this case, tcp).

Now we have replicated in our setup almost all the functionalities a small COTS router has. Next time we'll see how to improve that by adding a proxy.

Automagically setup breakpoints with gdb

When you are trying to debug a project you don't know you'll probably end up recompiling a few times, then restarting your debugging session. This can be quite frustrating, when you have gdb workset full of breakpoints, watch expressions and all that stuff.

Luckily you can easily restore your state if you just write all the gdb commands you need into a file, then start gdb with "--command=state.gdb". Magic! All your breakpoints are there.

Alternatively, an even better solution: just don't exit gdb after recompiling, simply "kill" your currently under-debug process (ie type "kill" inside gdb, do not kill gdb itself!) and gdb will be smart enough to reload your binary if it changed.

Setting up a Linux GW VI: Configuring a console friendly router and setting up static DHCP IPs

We have so far setup a device capable of working as a router for a medium sized LAN, providing NAT, DHCP and DNS services. This is great if you have a dedicated network admin, but you may prefer something easier for casual console users. We'll see how to "refactor" your server configuration now to make it more console friendly.

Moving DHCP config files

Since I want to keep everything together for easy administration, I will move the configuration files for DHCP to /home/router/dhcp. Changing the dhcpd.conf file itself is easy, just move the subnets declarations and add this line:

include "/home/router/dhcp/subnets.conf";
include "/home/router/dhcp/static_hosts.conf";

Like we did before with bind, we need to configure apparmor. vim /etc/apparmor.d/usr.sbin.dhcpd and add this two lines:

/home/router/dhcp/ rw,
/home/router/dhcp/** rw,

Restart apparmor service, then restart dhcpd. Everything should work just fine.

Setting up static IPs

Remember the static_hosts file we created before? We can use that to define a static IP. Add the following to set a static IP host:

host HostName {
	hardware ethernet 00:00:00:00:00:00
	fixed-address 192.168.10.50;
}

After that, just restart the DHCP service and renew your client's IP. Done, it's static now!

Wait a minute: how do you find the MAC for your host? I'm to lazy to copy and type it, so I did the following:

cd /home/router/dhcp
ln -s /var/lib/dhcp leases

Then you can check the hardware address in the leases/dhcpd.leases file. I created a symlink to keep this directory at hand, since it gives you a status of the current leases.

Have you checked your stack?

While getting bitten by running out of stack space is not a common thing, it sure is painful to debug. Unless it's caused by a (very obvious) stack overflow you will usually just get an unrelated segmentation fault in a seemingly random place, and not much help to troubleshoot the problem.

Luckily gcc seems to have an option to verify that your functions do not use an unbounded amount of stack space: just compile with the option "-fstack-usage" and a file .su will be generated with stack information for each function.

You probably want to see only static or bounded stack usages; an unbounded stack usage might be a sign that you should be storing that object on the stack instead.

Setting up a Linux GW V: DCHP

In our custom Linux router we now have DNS and NAT so far, but the client configuration has been absolutely manual. We can't have many clients with this sort of setup, so let's automate the client config with a DHCP server. Begin by installing isc-dhcp-server.

Edit /etc/dhcp/dhcpd.conf, set the domain-name and the domain-name-servers, like this:

option domain-name "lan";
option domain-name-servers 192.168.10.1 192.168.0.1;

default-lease-time 86400;
max-lease-time 172800;

authoritative;

I'm not sure if the first line is needed. The other two will set the DNS servers for your clients. Also, increasing your lease time is recommended, I used one day for default leases. I set this DHCP server as the authoritative server. If this is your router, that's probably what you want.

Now we need to define the network topology:

# This is the WAN network, and we won't provide a service here
subnet 192.168.0.0 netmask 255.255.255.0 {
}

# Define the service we provide for the LAN
subnet 192.168.10.1 netmask 255.255.255.0 {
	range 192.168.10.100 192.168.10.200;
	option routers 192.168.10.1;
}

Now we need to restart ISC:

sudo /./etc/init.d/isc-dhcp-server restart

And now we need to check if everything worked in the client. It's easy this time, we just ask for an IP:

sudo dhclient
ifconfig

If everything went fine, we should now have an IP in the 100-200 range, as well as the DNS server in /etc/resolv.conf. We have now setup a very basic router and should be able to server several clients for basic browsing capabilities.

Next time we'll see how to tidy up everything, for easier administration.

Some gratuitous MSVC bashing

Recently I found out Microsoft's Visual Studio doesn't support alternative tokens (ie "and" instead of "&&"). Even worse than that, apparently they don't think it's even necessary. And by the looks of this thread, the people working on MSVC need to take some time to actually READ the cpp standard. You know... it's kind of like a spec for your product. It's always good to take some time to understand the specs for your product...

I can only imagine how incredibly ugly their lexer must be to say it's not a fixable problem.

Setting up a Linux GW IV: Setting up apparmor

Apparmor is a service that runs in the background, checking what other binaries can and can't do. For example, it will allow bind9 to open a listening socket on port 53 (DNS), but it will deny an attempt to open a listening socket on port 64. This is a security measure to limit the damage a compromised bind9 binary running as root might do. And since we are going to use a non standard configuration, we need to tell apparmor that it's OK.

After installing bind9 we should get a new file in /etc/apparmor.d/usr.sbin.named. Add the following lines at the bottom:

  /home/router/named/** rw,
  /home/router/named/ rw,

And restart apparmor service:

/./etc/init.d/apparmor restart

Since we were modifying apparmor to allow a non-standard bind installation, now restart bind too. This time it will start without any errors, and you should be able to tail -f /home/router/named/dns.log to see the DNS queries on real time. If it doesn't, check that /home/router/named is writable to the bind user (I did a chgrp -R bind named).

A C++ template device to obtain an underlying type

What happens when you need to get the underlying data type of a pointer or reference? You can write some crazy metaprogram to do it for you. Like this:

template <typename T> struct get_real_type      { typedef T type; };
template <typename T> struct get_real_type<T*>  { typedef T type; };
template <typename T> struct get_real_type<T&>  { typedef T type; };

template <class T>
int foo() {
    return get_real_type<T>::type::N;
}

struct Bar {
    static const int N=24;
};

#include <iostream>
using namespace std;
int main() {
    cout << foo<Bar*>() << endl;
    cout << foo<Bar&>() << endl;
    cout << foo<Bar>() << endl;
}

Incidentally, this is also the basis for the implementation of std::remove_reference. Actually you'd be better of using std::remove_reference, for your own sanity.

Setting up a Linux GW III: Setting up DNS with bind9

If you have been following my series on how to install a Linux based router, you should now have a setup where a client is able to see the outside world via a router. We can try something more complex now, like pinging a domain instead of an IP. Something like this:

ping google.com

You should get a message saying the host is unknown. Can you guess why? Right, there's no DNS.

Setting up DNS

DNS will be necessary to resolve domains to IPs. bind9 is the default option for Debian based servers (are there others? no idea).

sudo apt-get install bind9

This will get your DNS server up and running, but you will still need to add this server manually to your client (again, because there's no DHCP running):

sudo echo "nameserver 192.168.10.1" > /etc/resolv.conf

And now:

ping google.com

Magic again, it (may) work. If it doesn't, you may need to open /etc/bind/named.conf and setup your router (192.168.0.1) as a forwarder, then restart the bind server.

Of course this is rather boring. If you are going to install a DNS you might as well create a custom TLD for your LAN.

Setting up a custom TLD with bind9 for your LAN

So far on the series about how to install a Linux based router, we set up a Linux router with NAT and a basic DNS. Now we'll setup a custom TLD, so you can have custom domains for your LAN. For example, if you want your router to have a nice user friendly name, instead of just an IP.

Let's start by adding a local zone to /etc/bind/named.conf.local, for a domain we'll call "lan":

zone "lan" {
        type master;
        file "/home/router/named/lan.db";
};

Now we need to add a reverse zone. Note how the name is the IP reversed:

zone "10.168.192.in-addr.arpa" {
	type master;
        file "/home/router/named/rev.10.168.192.in-addr.arpa";
};

We still need to create both files (lan.db and rev.10.168.192.in-addr.arpa), but will do that later. Lets setup a place to log all the DNS queries (optional):

logging {
    channel query.log {
        file "/home/router/named/dns.log";
        severity debug 3;
		  print-time yes;
    };

    category queries { query.log; };
};

For the log entry I have chosen /home/router/named as the log directory, just because for this project I'm keeping everything together (config and logs) so it's easy for people not used to administer a Linux box, but of course this means that apparmor must be configured to allow reads and writes for bind in this directory. We'll get to that in a second, first let's create the needed zone files for our new TLD.

Remember our two zone files? I put them on /home/router/named, but usually they are on /etc/bind. Again, I did this so I can have all the config files together. These are my two files:

For lan.db

lan.      IN      SOA     ns1.lan. admin.lan. (
                                                        2006081401
                                                        28800
                                                        3600
                                                        604800
                                                        38400
 )

lan.      IN      NS              ns1.lan.

wiki             IN      A       192.168.0.66
ns1              IN      A       192.168.0.1
router           IN      A       192.168.0.1

For rev.10.168.192.in-addr.arpa

@ IN SOA ns1.lan. admin.example.com. (
                        2006081401;
                        28800; 
                        604800;
                        604800;
                        86400
)

                     IN    NS     ns1.lan.
1                    IN    PTR    lan

Most of these lines are black magic, and since an explanation of both DNS and Bind is out of scope (feel free to read the RFC if you need more info) let's just say you can add new DNS entries by adding lines like this:

NICE_NAME           IN      A       REAL_IP

This will make bind translate NICE_NAME.lan to REAL_IP. Of course, this will depend on the TLD you defined. Now restart bind to get a crapton of errors. It will complain about not being able to load a master file in /home/router/named. Remember that apparmor thing I mentioned?

Gcc tip: better disassembly

Few things are more awesome than compiling with "g++ -S" and inspecting gcc's dissasembly and learn how the compiler optimizes things you wouldn't even think about. Unfortunately, the assembly might not be the most human friendly format for a program (though I've seen worse).

While you won't escape the need to learn some assembly to get any meaningful information out of gcc's assembly listing, there are some tips which might make your life much easier:

C++ filt

c++filt is part of the build essentials package, and will turn mangled names into proper C++ names. You won't need to remember that _Znwm is the mangled version of "operator new", just run "g++ -E foo.cpp -o /dev/stdout | c++filt" and you'll get an assembly with unmangled names.

fverbose-asm

Some people have the ability to read assembly and automatically understand how the data flows between registers and variables very quickly. For the mere mortals like us, gcc has a very helpful flag called "-fverbose-asm" which will add a comment to each line where a variable is referenced. This will let you keep track of the data flow inside a function.

Extra, unrelated, tip:

As far as I know, gcc has no option to write to stdout; just use "-o /dev/stdout" to let it write to a fake file which Linux will helpfully create for you, then you can pipe the hell out of gcc's output.

Five years

It's been a while since I've used the meta-post category. I think it's a good opportunity to do so: exactly five years ago I typed "/./etc/init.d/blog start", on this same blog. Quite a long time.

I'm really surprised I've managed to keep on writing more or less regularly for five years on this blog. There are now about 360 articles on this site, which gives an average of about one per week. That's a nice metric, even if not very accurate. I spent almost a year without writing, while moving to a different country. Maybe I should start a blog about that too.

Here's for five years more!

Setting up a Linux GW II: NATting and forwarding

For our Linux GW, services like DNS and DHCP are nice-to-have, but having real connectivity is way more important. Let's set up the NAT and connection forwarding features of the new router, then we can test if our setup is working properly by pinging an IP of one LAN from the other.

We'll do this by setting up NAT with iptables. We'll also have to configure the OS to forward connections from one network card to the other:

echo 1 > /proc/sys/net/ipv4/ip_forward
iptables --table nat --append POSTROUTING --out-interface eth0 -j MASQUERADE
# Add a line like this for each eth* LAN
iptables --append FORWARD --in-interface eth1 -j ACCEPT

We will also need to setup the IP for eth0, since there won't be a DHCP server (we ARE the server!). Open /etc/network/interfaces and add something like this:

# Configure the WAN port to get IP via DHCP
auto eth0
iface eth0 inet dhcp
# Configure the LAN port
auto eth1
iface eth1 inet static
	address 192.168.10.1	# (Or whatever IP you want)
	netmask	255.255.255.0	# Netmasks explanations not included

Once that's checked, restart networking services like this:

sudo /./etc/init.d/networking restart

Everything ready, now just plug your PC to the new router and test it. Remember to manually set your IP in the same network range as your router, since there's no DHCP at the moment. This may be useful to debug a problem.

In your client PC, set your IP address:

ifconfig eth0 192.168.10.10

Test if you IP is set:

ping 192.168.10.10

If you get a reply, your new IP is OK, if not there's a problem with your client. Second step, see if you can reach the router:

ping 192.168.10.1

Note that you may need to renew everything (i.e. restart networking and manually assign your IP) after you connect the cable.

Again, if you get a reply then you have connectivity with the router. So far we haven't tested the iptables rules nor the forwarding, so any issue at this point should be of IP configuration. If everything went well, it's time to test the NAT rules and the forwarding.

ping 192.168.1.1

That should give you an error. Of course, since there's no DHCP there's no route set. Let's manually set a route in the client:

sudo route add default gateway 192.168.10.1

Then again:

ping 192.168.0.1

Magic! It works! If it doesn't, you have a problem either in the NAT configuration or the IP Forwarding of the router. You can check this with wireshark, if the pings reach the server but they never get a reply back then it's the NAT, i.e. it can forward the IP packages on eth1 to eth0 but the router has no NAT, and it doesn't know how to route the answer back. If the pings never even reach eth0, then you have an ip forwarding problem.

Persisting the forwarding rules

In order to have the forwarding rules persisting after a reboot, we need first to change /etc/sysctl.conf to allow IP forwarding. It's just a mater of uncommenting this line:

net.ipv4.ip_forward = 1

We will also have a lot of iptables rules we need to setup during boot time. I have created a script at /home/router/set_forwarding.sh, which I also linked into /etc/init.d/rc.local so it's run whenever the system boots.

Next time we'll move on to something more complex: installing a DNS server and using domains instead of IPs.

C preprocessor VII: Recursive expansion on function macros

The last time we talked about recursive expansion rules on C's preprocessor: to sum it up, each expansion creates a scope, that contains a list of all macros which have already been expanded in said scope, or in a parent scope. That gives us a very nice and easy to understand tree of already-expanded rules.

Clearly that's too easy for C. We need more complexity: we need to make the expansion rules interact with the argument substitution process and the preprocessor operators too!

How exactly? The whole process is specified by a very tiny paragraph, 16.3.1, on the standard, which despite being tiny contains a lot of information. Actually, it contains all the expansion and precedence rules for the preprocessor. And it's more or less like this:

1. Argument scanning: the perprocessor binds a set of tokens to each argument name. If there are extra arguments and the token "..." is part of the macro's signature, a __VA_ARGS__ argument is created. (to put it simply: it will bind a set of tokens like "(a,b)" to an identifier like "ARG1").
2. Stringify and token pasting is applied ONLY to the arguments, not to the body function.
3. Each argument is recursively scanned for macro expansion, as if each argument was on a file on its own (imagine a new file is created with only preprocessor directives and the argument, then apply the expansion algorithm recursively to that file).
4. After the arguments have been fully expanded, they are substituted on the macro's body.
5. The resulting definition is then rescanned for macro expansions or token pasting operators.
6. A side effect of this multi-phase macro expansion is that the nice expansion tree we used to have no longer works.

Let's take this example:

#define str(...) #__VA_ARGS__
#define foo(a, b) foo a bar str(b)
#define bar foo bar 1
foo(bar, (1, 2, 3))

How can we expand this macro call? Like this:

expand{ foo(bar) }
        Match foo with definition of macro: foo(a)
            Bind a to bar
            Macro expand argument a -> expand{ bar }
                    bar takes no arguments, no binding is done
                    Apply rule bar -> foo bar 1
                    Scan the result for new expanions
                            foo was already expanded, no further expansion

            Bind b to (1, 2, 3)
            Macro expand argument b -> nothing to expand

        Replace macro expanded arguments in body definition:
            -> foo foo bar 1 bar str((1, 2, 3))

        Rescan the body for further expansion:
                foo: Already expanded on current scope
                foo: Already expanded on current scope
                bar: Already expanded (The compiler will have too keep a map of expanded macros for each identifier in a definition!)
                bar: Needs expansion
                        Apply rule bar -> foo bar 1
                        Rescan for further expansion
                                foo: Already expanded on parent scope
                                bar: Already expanded on current scope
                str((1, 2, 3)): Expand macro call
                        Bind (1, 2, 3) to __VA_ARGS__
                            Analyze (1, 2, 3) for further expansion
                            Apply operator '#' to (1, 2, 3) -> "(1, 2, 3)"
                        Replace #__VA_ARGS__
                Replace the result of str((1,2,3)) -> "(1, 2, 3)"

        Replace the original call "foo(bar, (1, 2, 3))" for the result
            -> foo foo bar 1 foo bar 1 "(1, 2, 3)"

This last example should be a good representative of the complexities involved in a macro expansion; hopefully now you know more than you ever wanted to know about macros.

Setting up a Linux gateway/router, a guide for non network admins

Setting up a Linux GW or router is not as hard as it may seem, as long as you are reading a friendly enough guide. Yes, there are a lot of guides for this, but since I needed to document how I did it, I might as well write a post about it here. My addition to the usual "setting up a linux gw guide": I'll do it using Virtualbox first, so I can test my setup before actually deploying it.

I'm going to write about how can you setup a regular Linux distro to be your border router/gateway for your LAN, but for easy of use I'll base my examples on Ubuntu.

As expected, if we are going to replace a device, say, a router, we need to replace it with something that can provide the same functionality. In this case, we have chosen a Linux server, so we need to figure out which services are provided by the router and then emulate them someway:

• DHCP to manage leases
• DNS to translate domains to IPs
• NAT, to multiplex a single connection
• Service forwarding, to expose internal services to an external network

Luckily Linux supports all of these:

• ISC for DHCP
• bind9 for DNS
• iptables for NAT
• iptables again, for service forwarding

We'll be setting up each of these services in the next posts, for now:

Preliminary work, the hardware setup

Before you setup any services, you are going to need two things: first two network cards, one for the outgoing connection and another one for the (switched) LAN, and a way of telling your server that you want all traffic from network 1 forwarded to network 2. You may want to install more than two cards, in case you need to route several LANs. We'll see that later.

You will also need an OS. I have chosen Ubuntu because it's very simple to install, and has all the software we need available in the repositories, but you can use any other distribution if it suits your needs.

Also, throughout this guide I will assume a setup like this:

• WAN access through eth0, DHCP address
• LAN routing in eth1, network 192.168.10.1/24

If you don't have all this hardware...

Not everyone may have two spare desktops with three NICs ready for testing. Even if you do, you may be too lazy to setup the physical part of your network. If this is your case, you can also setup a virtual machine to emulate your setup, and Virtualbox is great for the task:

1. Begin by creating what will be your router VM.
2. Enable the first network adapter. This one should be able to see your physycal router (i.e. connect to a WAN).
3. Enable a second network adapter. Use the 'Internal network' option in the 'Attached to' field. This will be your LAN interface.
4. Create a second VM. This one will be your client.
5. Enable a single network adapter, attached to an internal network as well. The name for this network should match that of the other VM.

You are all set now, with this virtual setup you can begin setting up your router. We'll see how next time.

C preprocessor VI: Recursive macro expansion rules

What happens if you define a recursive macro? This might seem like a silly question, but by asking it we can gain some insight on the inner working of the preprocessor.

Let's start with a simple example:

#define foo bar 1
#define bar foo 2
foo

Luckily the preprocessor is smart enough not to trip up on this simple piece of code. When expanding foo on line three it will do something like this:

#define foo bar
#define bar foo
foo
// Applies foo -> bar 1
bar 1
// Applies bar -> foo 2
foo 2 1
// Scans foo again... but doesn't expand it

The second time the preprocessor scans foo it won't expand it: it "knows" foo was already expanded, so it won't do it again. But how does it know that foo was already expanded? Let's try something a bit more complicated:

#define foo bar a baz b
#define bar foo 1
#define baz bar 2
foo

And then let's see how foo is expanded, step by step:

#define foo foo a bar b baz c
#define bar foo 1
#define baz bar 2
foo

First the rule "foo -> foo a bar b baz c" will be applied and the results rescanned: let's call this scope 1. We'll end up with:

foo a bar b baz c

Now the results of this expansion will be scanned, in a new scope. Let's call it scope 2. The first token the preporcessor will see is "foo", which was already expanded on scope 1: it will be ignored and it will continue to the next expandable token, "bar", and it will expand it like this:

foo a foo 1 b bar 2 c

On the scope that baz's expansion creates (scope 4), the parent's scope expansion rules are "inherited", so for scope 4 "foo" was already expanded but "bar" was not, because bar's expansion happened on scope 3 and scope 3 is not scope's 4 parent. Not following me? Try following this diagram:

foo -> foo a bar b baz c
    foo -> already expanded, ignore
    a   -> not a macro, ignore
    bar -> expand to "foo 1"
        foo -> expanded at parent scope, ignore
        1   -> not a macro, ignore
    b   -> not a macro, ignore
    baz -> expand to "bar 2"
        bar -> expand to "foo 1"
            foo -> already expanded at parent scope, ignore
            1   -> not a macro, ignore
        2   -> not a macro, ignore
    c   -> not a macro, ignore

Hopefully the preprocessor expansion rules should be a bit more clear now: each expansion creates a scope, each scope inherits from parent's scopes whether a rule was applied or not and if it was then said rule is ignored in the current scope.

Of course these rules get more complicated when dealing with token pasting and stringifying operators, because each phase (stringifying, token pasting, rescanning and expansion) will happen in a specific order. Things get even more complicated when you realize (by reading the standard) that said order is not the same when you deal with argument replacement.

Then again, it's probably a good idea if your macros don't rely on the recursive expansion rules of the preprocessor.

Vim tip: Jump to a tag definition

I admit it, hitting control enter to jump to a definition is a very useful feature of fancy GUIs. Fortunately, it's not exclusive to fancy GUIs, apparently it's been available using ctags too for a little while (if you consider the last 20 or 30 years to be little, that is).

Just add this magic spell to your .vimrc; next time you open a file in a project with a tags file generated just press ctrl-enter with your cursor over the definition you wish to find:

map <C-CR> :tab split<CR>:exec("tag ".expand("<cword>"))<CR>

C preprocessor V: Conditionals

While walking around the c preprocessor we came to know the stringify operator, the crazy token pasting operator and a __VA_ARGS__ macro. All very weird, but at least the #if's work in a sane way... or do they? They do, but there's some room for unexpected behavior if you don't know some implementation details. Take this code for example:

#if 0
#  if 0
#  else
#  elif true
#  endif
#endif

Clearly the inner if is wrong because the else clause comes before the elseif, however you might think it doesn't matter because it's surrounded by an #if 0. Surprise: it does matter, that's not valid preprocessor input. Even if the outer #if is not "taken", whatever preprocessing directives are inside it should still be valid (though anything that's not a preprocessing directive will indeed be ignored).

Even though at first it might seem weird for things inside an #if 0 to be important, it makes sense if you think that should an internal #if not respect the proper structure then the preprocessor wouldn't know when to end the first #if 0. Then again, if you find any real-world utility to this bit of preprocessor implementation trivia, you are doing something horribly wrong!