Helpful hints on Large Solr Indexes and Schema Design

Solr user Lance Norskog has been kind enough to contribute documentation on:

• Common Solr Schema Design problems and solutions
• Design considerations for Unique Keys for common use-cases
• Common problems and solutions for Large Solr Indexes

Very useful documentation which, no doubt, will be made more comprehensive with time.

Update - Mark Miller has written a very nice article on Scaling Lucene and Solr at the Lucid Imagination website.

Google Summer of Code 2009 at Apache

Google Summer of Code program is back again this year and Apache is looking for students interested in contributing and making money with the program.

Apache Software Foundation received quite a few students with excellent proposals who did a lot of great work last year. Take a look at the last year's proposals to get a feel of the level of competition. I'm sure there would be quite a few this year as well. A wiki page has been put up which will list all the proposals.

You can come up with their own proposals as well and add it to the wiki. However, the ASF being a community driven eco-system, it is highly recommended that you drop a line to the project mailing lists and get feedback on your proposal. That way, you will have time to convince one or more committers to sign up as mentors for your proposal. They will help you develop your proposal as well as guide you along the project with regular reviews and feedback. If your proposal attracts no mentors, it cannot be accepted for the program.

Open Source is a different ball game than academic projects and the code itself is a small part. One needs to write unit tests to inspire confidence in the code before it can be incorporated in a project. If other developers are interested in your project, they'll want to collaborate with you. With each patch, you'll get review comments which you may need to incorporate. There are very few places, if any, where you can get such great feedback on your work and that too, absolutely free.

Users will need documentation and tutorials about your code before they can start using it. Sometimes, one also needs to create working examples to demonstrate usage and features. Users will ask questions on your features, post bug reports and suggest enhancements. It is the open source way to courteously answer them and guide them to solutions. As the feature matures, the community also benefits from best practices, FAQs and guidelines on performance optimization. Ultimately, it is well worth the effort to learn the open source way of developing software.

I've been thinking about a few features which can help Solr but more on that later. For now, see the announcement on solr-dev mailing list on GSOC 2009 and reply with your ideas if you are interested.

Grant has also written a useful post with advice to aspring GSOC participants on his blog.

Announcing my return to blogging

Yes, it has been a long long time since my last post. I guess I lost interest in writing about the myriad of things out there. But, I sure did not lose interest in reading and learning about them.

I work at AOL Bangalore Development Center as a Software Engineer on a variety of cool projects. Life is great, work is fun and I'm having a good time. They pay me to work on such interesting things that I'd probably do them for free anyways :)

A lot of my work revolves around open source projects chief among them being Apache Solr. I started using Apache Solr (more on this in another post) as part of my job. Through a stroke of luck, my colleague Noble Paul and I, built DataImportHandler for Solr and contributed it back. Between then and now, both of us have been actively involved in working on Solr as a small part of our day job and as a big part of our spare time. These days, I'm a committer on the project and spend a large amount of time on adding new features, fixing bugs and answering user queries on the user mailing list.

Expect a lot of posts related to search in general and Apache Solr in particular. From time to time, I'll keep writing posts on the myriad of things I keep doing and reading.