Thoughts about technology, business and all that's life.

This blog has moved to http://shal.in.

Sunday, June 18, 2006

The Bird's eye view

Life's a drag, many people say but then it must be the only drag force known to cause incessant motion. Funny, isn't it?

Most people go about life in a very routine way, seldom pausing to think differently. In most cases this has to do with the incessant flow of events and our reactions towards them. I feel that our reactions and not the events themselves, determine the course of our lives. Agreed that life is ironic and strange. But equally strange is the fact that with each event, God gave each one of us the power to change ourselves and thus, our lives at any moment of time. We may have little control over the events that were destined by God. But for each such event we get some choices. A choice to fret about this change or to take corrective action or to adapt.

It's always fascinated me that most people are so scared of change. Maybe this is the bane of self-awareness. If you remove this aspect from the equations, life becomes so much easier. For each event, our instinct guides us to a response to be followed without deliberation and the world becomes free of criticism. The conflict still remains though. Since you don't stop to think, you don't try to estimate the outcomes and the fear of change vanishes from the mind. Animals follow this exact procedure. But then, we don't call our human race an intelligent one for nothing. The ability to be aware about our surroundings, to take deliberate action, to ponder upon the reasons for our existence, to acquire knowledge and to apply it to areas which, at first, do not seem logically related, is what intelligence is all about. This definition is by no means complete and may not even be entirely correct. It's just what I feel.

Unfortunately, the need to conform to social standards, to be accepted and appreciated holds back many from accomplishing great things. To break free, you need to see things from a higher level. This is what the bird's eye view is all about. Rising above the crowd and it's noise is no easy task. But it's a task that one must undertake to realise the locked potential in each one of us. I've been trying and I'll continue to try so that one day I may also experience the euphoria that comes along with the higher level. And I believe there's no end to this journey.

I have found that the Bird's eye view is a very effective technique to understand newer aspects and plausible reactions that one could never have seen with the perspective one follows in the normal course. It gives more than just a clear view, it enhances our understanding and thus our ability to think differently. The next time you get stuck, take a moment out of the normal and see things through the Bird's eye view.

Sunday, June 11, 2006

The power of the community

What do BitTorrent, Orkut, TopCoder, del.icio.us, Wikipedia and hundreds of products/services have in common? Seasoned webbies may recognize the pattern and say that all of them are examples of Web 2.0 (which...ahem...yes they are). But apart from this fact, the one underlying theme behind the diverse fabrics of all these products/services is that they are highly community driven.

The new poster-boys of today's internet have utilized the power of the community channeling the huge cluster of the internet users themselves to improve the services offered. The whole idea is pretty simple and has been in use since a long time in the hospitality industry. It's called Self-Service. Their are no waiters and the customer services himself. He goes to the ordering counter, places the order, pre-pays the money and then fetches his food from the service counter to his table himself. The cost of the waitors and other such staff are eliminated and this arrangement makes the customer pay more attention to the clean-up. You always place the tray at the designated spot while going out, don't you?

Some clever people applied the same concept with some modifications to the web and Wiki was born. People collaborate online and add, delete, modify, re-organize content to the website. No one person does it, no content writer is hired and the websites evolves instead of being made. Can you imagine what would happen if we apply this model back to our McDonalds? It's akin to McDonalds allowing you to use their tools to make your own burger for yourself and others. Ofcourse one can change the ingredients list and include poison in it. But the underlying idea is that atleast one good samaritan will take notice and change it back to the original copy. The idea says that there are more good people in the world than bad people hence the overall progress will be positive. What Britannica took so many decades to collect and classify, Wikipedia did in a few years. You don't believe me? Check out Wikipedia, Algorithmist and our newest entrant in this field, the MSDN Wiki. By the way, did you know that Wikipedia is one of the Top 100 most popular websites in the world?


As software became complex, new delivery platforms had to be evolved. Suppose you have a product which you give away for free download. It gets slashdotted (or digg'ed) and soon swarms of users come your way and your site goes down. What can you do about that? Buy new servers, take up dedicated hosting, larger bandwidths and scale up. And all that only for the initial excitement about your product. What happens when it dies? You're stuck with your new servers, your expensive hosting plan and all the bandwidth in the world that you'll never need. Enter the community...BitTorrent & Jidgo. The idea...give out torrents for the file that you intend to distribute. The users connect to each other to send and recieve your software and a central tracking server manages these peer-to-peer interactions. Send and recieve is simultaneous to utilize network bandwidth and the protocol is designed to perform better as the number of users interested in the software increases, unlike traditional methods.

Ever seen Rent-A-Coder or e-Lance? They used the community to bring together the people who needed work and the people who wanted some work done. It was successful but the model had an inherent deficiency. It was hard to scale up. It was unsuitable for larger projects as there was no control over the developer and no fixed model of interaction. TopCoder took this further. They developed a model of interaction between the company and the individual. They presented the interaction in the form of a competition to build out the best possible effort from the individual. To attract the best possible talent, they gave away huge amounts of prize money for algorithmic programming contests. So they gave this proposition to the companies needing software...At TopCoder, the individuals from a world-class community compete to design, develop and review software for you. Other revenue streams for TopCoder were sponsorships from companies for their tournaments and employment services.


Traditional search methods used algorithms to identify the relevance of documents for certain keywords. They can't be accurate all the time so the ideal would be to employ people to classify the information. But this is limited because the web is too big. Enter Tagging, the community joins hands to tag their favorite websites and the model evolves to include all relevant website. If a hundred people say that this particular site is great for cricket information, then it has to be so. Folksonomy was born. Examples are del.icio.us, ma.gnolia for general purpose searching and Flickr for photos.




So the conclusion is that for all these websites, the service automatically improves as more and more people use it. Orkut, TopCoder, Bittorrent, Wikipedia wouldn't exist if the community didn't exist. They use the community to harness collective intelligence (Wikipedia, del.icio.us, Flickr, Cloudmark), collective bandwidth (BitTorrent, Jidgo) and collective development (TopCoder, Sourceforge) to grow themselves.

So what will your website do? Does it stay the same way if five people use it or if five thousand people use it everyday? If yes, then it should not. That's the way to survive in the Web 2.0, just think about it.

About Me

My photo
Committer on Apache Solr. Principal Software Engineer at AOL.

Twitter Updates

    follow me on Twitter

    Recently shared stories

    Recent questions on Apache Solr

    Recent development in Apache Solr