Thoughts about technology, business and all that's life.

This blog has moved to

Tuesday, October 27, 2009

Why you should contribute to Open Source

Note: The following material and presentation was prepared for students of the Indian Institute of Information Technology (IIIT), Allahabad. The aim was to get them excited about contributing to open source projects and in particular about Apache Lucene, Solr and Hadoop. The first talk was titled "Why you should contribute to Open Source" and was aimed at freshmen and has no technical content. The second was titled "Get involved with the Apache Software Foundation" and was given to sophomore, junior and senior students and it goes into some basic technical information on Apache Lucene, Solr and Hadoop projects. The following post comprises of some notes that I put together for the talks.

Work on what you like, when you like

Everybody wants to work on "cool" products. However, the reality is that most of you will get stuck in a job which although may pay well, it will hardly be about the things you wanted to work on. In your course, you will learn about algorithms, distributed systems, natural language processing, information retrieval, bio-informatics and other areas of computer science and its applications but in real life, the majority of the work done in software companies will have little direct application of things you will learn in your course.

Most of the times you will be using things built by others and writing glue code to build things needed by your company's business. This is not to say that all that knowledge will go waste; it will definitely help you become a better programmer and you should learn it but there's a fair chance that it may not be used directly in your job.

Open Source projects offer you a chance to work on something that you want rather than something that others want you to work on. It is a great opportunity to work on something that is both cool and useful as well as to associate with a well known brand and all the publicity and goodwill it brings. You are free to pick and choose between the thousands of open source projects out there. Moreover, you are free to decide on how much you want to contribute. You won't have a boss and you won't have the pressure of deadlines and schedules.

Development in the "real" world

Academic projects are insufficient to impart many of the skills that you'd need once you start developing software full-time. Many of these skills are "social" rather than technical in nature but are at least as important.

Most academic projects are "toy" projects. By that, I mean that their whole life cycle revolves around you. You are the designer, developer, tester and also the user. As a result, there are few key things missing in those projects.
  • No build system - Makefiles? Ant? Maven? Very few students are familiar with using them. Don't even ask about creating a build from scratch. "Hey! Just open those files in a text editor or an IDE and hack away" is not an unusual thing to be heard
  • No source control - CVS? SVN? Git? A single person writing all the code or >80% of the code is very common
  • No bug tracker - "It is never going to be used after we demo it to the professors"
  • No user documentation - maybe you will write a research paper detailing your findings but there is little or no documentation written for "other" people
  • No mailing lists or forums for support - Nobody but you is going to use it
Moreover, under these circumstances, you never learn how to:
  • Discuss technical design or issues in writing
  • Resolve conflicts in matters of design, architecture and a project's road map.
  • Build usable interfaces (whether command line options or a GUI or an API)
  • Write proper error handling and logging code
  • Identify hooks for monitoring systems in production
  • Think about backup and recovery
  • Identify components which can be extended or replaced to add or modify functionality of the system
Open source projects are the real deal. If you are involved for long enough, you will either see or be a part of many such discussions and conflicts. All of the above skills are things you will need when you get around to software development in the real world.

Learn from the best

How many great developers do you know about? How many of them work or have worked on an open source project? I bet there are many names common to both the lists.

Open Source development will help you observe how experienced developers work and their various ways of designing, coding and discussing solutions. You will learn new ideas and new ways of solving problems. The second and probably more important part is that many smart programmers will be looking over your code and will provide review comments which will help you improve yourself. You will learn more efficient or shorter (or both) ways to solve the same problem. That kind of feedback is invaluable to a budding programmer.

I know that I've learned a great deal since I got involved in Apache Solr.

Build a publicly verifiable resume

What you tell in your resume are things like contact information, performance in academia, programming languages you know, projects you've worked on and other such stuff. There is very little in this document which can be verified easily. This is a problem for you as well as for the prospective employer because:
  1. It may not represent you, your skills and your hard work sufficiently enough
  2. It makes hiring a game of chance for the prospective employer and prevents them from making more informed decisions
The best thing about contributing to an open source project is that everything you do is public. So you can say things like the following:
  • I have worked on this project for the last two years
  • I wrote features X, Y and Z on Project P
  • I have over two hundred posts on the user forum or mailing list
  • I have commit access to the project
  • I am the expert because "I wrote it"
And your prospective employer can search and verify such things easily. Congratulations, you have just landed on top of the stack of resumes!

Companies will find you

When a company evaluates that an open source Project X can save them a lot of money, it is likely that they will hire a few people who have experience on project X and can support its use internally. Many such companies also allow their developers to work on the project either part-time or full-time. And who else is more qualified to work on the project but you - an existing contributor!

More and more companies are starting up around providing training, consulting and support for open source projects. Many such companies exclusively hire existing contributors.

Even if an open source project is not used directly inside the company, many tech companies hire open source contributors because:
  1. Hiring popular open source developers makes them more cool in the eyes of other developers
  2. Developers who contribute to open source projects are good programmers

I'm sure there are many more reasons other than the ones I've given here. In the end, contributing to an open source project is a good investment of your time and it may well be your big ticket to finding that great job. Good Luck!


Nikhil Verma said...

An excellent post ! :)
Will distribute it to people in my college

Shalin Shekhar Mangar said...

Thanks Nikhil :)

Eric Wendelin said...

Well said. Coincidentally, I was just writing a post yesterday on the very same subject. Guess I'll scrap it, you said pretty much everything I would have myself :)

Anonymous said...

Well written and intelligent.

Abhinav.Singh said...

Excellent post...

As I was reading through I was able to link most part of it with my college life.

Vaib|-|av said...

thanks for your guidance sir..

Shalin Shekhar Mangar said...

Thanks for the kind words guys.

@Eric - Please don't let this stop you :)

Aniruddha said...

Awesome post buddy :) Totally agree with what you've said.

Admin said...
This comment has been removed by the author.
raghib said...

Very Very True :)

Mavsprak said...

Nicely done...

Girish the great said...

Wow never looked at it that way.

About Me

My photo
Committer on Apache Solr. Principal Software Engineer at AOL.

Twitter Updates

    follow me on Twitter

    Recently shared stories

    Recent questions on Apache Solr

    Recent development in Apache Solr