The 4th annual UK conference for MongoDB developers was hosted by 10gen at the Mermaid Conference Centre in London's Blackfriars. With four tracks running throughout the day - both technical and non-technical, in addition to an "ask the experts" drop-in track, choosing which talks to visit was always going to be a challenge!
Eliot Horowitz, Co-Founder and CTO 10gen
The Keynote from Horowitz was excellent and pitched just right. There were around 530 delegates at the conferences this year, and with MongoDB still being a relatively new tool, many will have been attending for the first time (myself included). The talk covered the history and rapid growth of MongoDB as a database as well as 10gen as a company. The company was set up back in 2007 by Eliot and Dwight, and for the first 2 years it was just these two guys. Four years later 10gen now has 140 staff - 17 in Europe alone (most of whom appeared to be at the conference!)
Eliot also covered some of the key design considerations behind MongoDB. These included being easy to setup, easy to scale, and doing the right thing right out of the box. Having used MongoDB in a production environment I can testify that these claims are true.
Schema Design by Example
Christian Kvalheim, 10gen
Christian gave a useful account of the importance of schema design with reference to performance and scalability, the detail of which is beyond the scope of this article. For me, however, two key points came out of this talk.
The first of these is that although it's important to have a good design for performance reasons, it is possible (and in principle straight-forward) to change the design over time. The speaker gave some very good examples of this.
The second interesting point is that noSQL Design Patterns are emerging, which can only be a good thing. With new technology such as this, it's often tempting to wade in and start playing around with the fun stuff. However, as professionals we should always be looking for reusable solutions. I have no doubt that the blogosphere will see articles about noSQL Design Patterns in the months ahead.
MongoDB on Innovation Projects / Products
Noberto Leite, Telefonica R&D
It is refreshing to know that not all the talks at this conference were technical. As much as I love geekery, it's also great to hear how other companies are using MongoDB and what they are using it for. Although Telefonica have only been using MongoDB for little over a year, Noberto was able to give plenty of examples of the kinds of projects they had undertaken. Of particular interest was a project which had originally been developed using Oracle at the back end, and had taken two years to complete. The system was slow and deemed not fit for purpose so it had to be rebuilt from scratch, this time using MongoDB. The new system was around 15 times faster and only took 3 months to build. Definitely a win for MongoDB!
MongoDB for Online Advertising at AOL
Jonathan Reed, AOL
Back in the main auditorium, Jonathan Reed from AOL explained how they built an analytics engine for their online advertising arm Advertising.com. They store huge amounts of global user-tracking data across multiple regions. Interestingly they employed a clever sharding tactic whereby the data was sharded by region, but replicated to the neighbouring regions. This approach allowed for very fast local writes, but still gave the failover if the regional node was unavailable. Overall this was a very interesting talk that demonstrated the power and flexibility of MongoDB.
Ask the Experts
During the lunch break I spoke to a couple of the exhibitors including Cloud Foundry and Equal Experts. 10gen were also offering an "Ask the Experts" drop-in session where you can....well...ask the experts! I grabbed the opportunity to ask a few high-level questions about MongoDB. I was interested to know what the typical use cases are for MongoDB. I got the usual response of archiving, logging, high volume, high availability but interestingly they also came back with "any application where you may need to quickly change the type of data you store". Hmmmm - isn't that anything built using Agile techniques?
This led to my next question – what percentage of new apps being built today should be considering MongoDB for some or all of their data persistence? In my head I was expecting to hear a figure like 10%, but the response surprised me. Although not committing to a figure, when I suggested 50% they didn't flinch. As the day went on I became less and less surprised by that figure.
The New Aggregation Framework
Ross Lawley, 10gen
One of the most exciting additions for developers in the upcoming 2.2 release is the new Aggregation Framework. Once you've seen it you'll understand why. Not only is it around 25 times faster than the equivalent map-reduce functions, but you can run multiple aggregations simultaneously. If you've ever experienced the pain of trying to craft a map-reduce function, you'll be pleased to know that the aggregation framework is very simple to use. Ross ran an excellent talk taking us from the simplest
$sum functions, right through to document-specific functions such as
$unwind. The real beauty of the framework is how functions can be chained together in what are known as a pipelines, similar to the way you might perform sub-selects in SQL.
I spoke to Ross briefly afterwards and his belief was that the aggregation framework would eventually replace map-reduce altogether due to its flexibility, speed and ease of use.
Six Months of MongoDB at Nokia Bristol
Tom Coupland, Nokia
Another non-technical talk (I told you there were a few) catalogued the experience of one team's adoption of MongoDB. The background to this story was that the team were spending too many of their days dealing with ORM-related issues and had come to the conclusion that far too much of their time was spent doing things other than delivering features. They assembled a crack team of "devops" people to get buy-in and build knowledge across departments from the very beginning. Furthermore, they threw all of these interested parties into the same room, hooked one computer up to a projector and designed the architecture and schemas as a team! What an amazing way to get everyone up-to-speed and invested at the same time.
Why the Schema Diagram is Ruining Enterprise Computing
Sean Reilly, Equal Experts
Having spoken to Sean earlier in the day, I decide to go along to this talk to see what it was all about. Sean is a very entertaining speaker and knows how to convey his message. The gist of the talk was that if you have a schema diagram there are 3 things you should not do with it.
- Mistake a schema diagram (which is a model of the persistence layer) for a model of the application.
- Share the schema diagram with anyone outside your team.
- Put it on a wiki where it can be seen by everyone.
Why not? Because as soon as you do that, someone will login to your database and read your data. This all sounds reasonable ... until one day you decide to make a tiny change to your data model. It's at this point you will be told that you can't because someone else is reading from it and it will break everything! Sounds like a nightmare, but the solution is thankfully rather simple. The answer is force everyone to use an API to access your data. At one point Sean got the whole room to chant "The schema diagram is an implementation detail which you do not need to know". Brilliant!
Sean also produced two marvellous text-to-speech videos that had the whole room in stitches. They depicted the "without an API" and "with an API" scenarios. As soon as these videos are available online I'll post a link in the comments below.
Mobilize Your MongoDB! Developing iPhone and Android Apps in the Cloud
Marek Jelen, OpenShift
These days it seems as if every man and his dog are offering cloud hosting and/or PaaS - and Red Hat are no exception. Marek bravely did a live demo of building an Android app powered by a Ruby back-end all hosted on the OpenShift platform. Barring a couple of minor DNS gremlins, Marek got the whole platform up and running during the talk. OpenShift is a pretty sweet platform and with the backing of Red Hat, its bound to make a big splash in the PaaS pool.
Sharding Best Practices
Daniel Roberts, 10gen
What was perhaps surprising about this talk is that Daniel opened with a list of alternatives to sharding. The logic being that although sharding is amazing and offers huge potential for linear scaling, it also adds complexity to your architecture, and to some extent your code. If you can solve your performance issues by either a redesign of your schema, by adding an index or through buying better hardware, you can at the very least postpone the need to shard your collection.
Following on from that, the speaker looked at some of the best practices in sharding (the clue was in the title I guess), including the idea of choosing a sharding key based on region to give local writes, which can greatly improve performance. If you were just on the brink of sharding some or all of your database, then this talk would have been perfect for you. With a talk of this nature it could have been tricky to hold the audiences' attention for a full 40 minutes, but Daniel's deep understanding of the topic coupled with his accessible presentation style made this talk whiz by.
Thank You and Road Map
Eliot Horowitz, Co-Founder and CTO 10gen
Eliot wrapped up the day whilst holding a pint in his hand. He set the tone for the evening ahead (a five bar pub crawl). As Eliot put it "I'm having a drink now so you can too". Indeed many delegates did listen to the wrap-up whilst sipping beer .
The CTO of 10gen reminded us that although they are all still hard at work fixing bugs and adding features, we should still continue to hassle them with request for the stuff we wanted. The next release, due any day now, will include the aggregation framework, time-to-live collections (where documents can be told when to expire - great for caches et al), in addition to some pretty clever internal stuff that will improve concurrency, reduce fragmentation and generally make everything faster – wow!
On the shortlist for future releases include the long-awaited full text search, more concurrency, online compaction, internal compression and some more security options such as LDAP auditing.
This was my first MongoDB conference – in fact it was my first database conference full stop. To some extent I did not know what to expect. Actually that's a complete lie! I totally expected back-to-back technical talks and hands on sessions. And if that's what you go to a MongoDb conference for then you wouldn't have been disappointed. However, there was much more besides.
Something else that was mentioned by many of the speakers (no, not just the ones from 10gen) was how great the documentation and support were - both the paid-for support from 10gen and the active community blogs and forums.
The opening and closing talks by Eliot Horowitz showed us how far we've come and where we're heading. Although MongoDB has been around for a few years now, the first release that was considered production-ready only came out in April 2011. Looking at the “Who's Who” of MongoDB users; AOL, Nokia, MTV, Foursquare – the list goes on, on thing is for sure - it's is not a fad. MongoDB is here to stay!