via iRights: Buchanan International highlighted by a FT.com | People | Special:

In an achievement that is almost the equivalent of the Human Genome project for the internet, a new Scottish software company has not only succeeded in plotting a map of the world wide web but has also uncovered its Dark Side.

...

The technical difficulties in creating the map were considerable. Handling the data alone became a gigantic headache. Consuming about 80 gigabytes of data an hour, Buchanan and Actis have created the second largest database of any organisation in Europe - governments included - according to Oracle.