The Problem

So, I set out to find a URL shortener to integrate into my WordPress.
But was disappointed as all of them seem to use MySQL and make 4-5 character identifiers
for URLs. I knew that I would only be shortening URLs for myself and hence didn’t need such overheads(extra time and characters).

So, I set out to build a really fast URL shortener to suit my needs. I soon came to the realization that a URL shortener is nothing but a key-value store. The key being the URL fragment in the shortened URL and the value being the URL being redirected to. This led me to start my more ambitious project of making the fastest key-value store that I could imagine specifically for the purpose of a URL shortener(I don’t want to make another Memcached or Kyoto Cabinet).

The Solution

The main and perhaps the only deciding factor for speed of a key-value store is its lookup time and any decent CS grad(whether you believe it or not I am) knows that the data structure offering the best speed for that is hash table(O(1) lookup).

Then, I set out to search for the fastest hash table implementation in C++ known to mankind and these benchmarks led me to choose Dense Hash Map by Google.

I later came across this and this(this guy has already done what I am doing now) but I have had no time to port RDESTL to Linux and integrate it to my implementation but its definitely on my TODO list.

But a Hash Table is only as fast as the hash function it uses and as such finding the fastest hash function was not that hard with the most prominent ones listed here.

I finally settled on MurmurHash 3 due to the benchmark here.

So, what was the end result?

The end result is the following code. The code should be pretty self-explanatory. If its not you know where to find me.

TODO

• Replace Dense Hash Map with RDESTL DONE!
• Embed a GET-only HTTP daemon in the code that can listen on 80 and thus serve requests directly.
• Make a API for CRUD of URLs on the fly.
• Make a persistence layer on top of the code to insure the state can be restored if the above API is implemented.
• Add logging of requests so that the logs can be used for passive data collection.

I think it sums its up for now. Contributions, criticisms are welcome.

But, writing the first post it gives me a feeling akin to the one which I got when I first wrote my “Hello World!”  in VB6.

Private Sub cmdButton_Click()
    MsgBox "Hello, World!"
End Sub

A bad start I guess(and as many of you know), but truly this blog is one of the few creative works that I have done recently and creating it was as exciting as seeing that message box with “Hello World!”.

Why Now?

• I have lot of ideas. I read a lot and in general share a lot too(if you follow me on Twitter or Buzz you will know)
• I have lots of free time and peace of mind to write things due to my awesome employers
• Everybody seems to have a blog these days and I don’t seem to like Tumblr that much.

Thanks

• First and foremost my awesome employers i.e Webfaction who also host this site.
• Abinash Pant – My awesome friend who bought this domain for me(of course I am going to pay him back sometime in the near future).
• Rodrigo Galindez – The author of this Awe-Inspiring Theme

Well then, I am off. Hoping to return with more interesting stuff to write about soon.