The hype cycle is at a high for cloud computing, distributed “NoSQL” data storage, and high availability map-reducing eventually consistent distributed data processing frameworks everywhere. Back in the real world we know that these technologies aren’t a cure-all. But they’re not worthless, either. We’ll take a look behind the curtains and share some of our experiences working with these systems in production at SimpleGeo.
Our stack consists of Cassandra, HBase, Hadoop, Flume, node.js, rabbitmq, and Puppet. All running on Amazon EC2. Tying these technologies together has been a challenge, but the result really is worth the work. The rotten truth is that our ops guys still wake up in the middle of the night sometimes, and our engineers face new and novel challenges. Let us share what’s keeping us busy— the folks working in the wee hours of the morning— in the hopes that you won’t have to do so yourself.
I joined NewRelic along with the rest of the Opsmatic. I helped start that ops-focused company after spending time building, breaking and evolving various systems at Urban Airship, SimpleGeo, and Flickr. When not wrasslin’ with computers, I’m likely involved in some combination of surfing, exploring new places with my special lady friend, enjoying the company of friends, watching sports, reading and playing guitar poorly. But my real passion is growing beards.
Derek Smith is a recent graduate from Loyola Marymount Univiersity. With early guidance from a few CS professors, he began his Senior year with the iPhone SDK. As a student developer he earned a WWDC Student Scholarship. After college, Derek began working as an iPhone Dev consultant which lead him to his current position at SimpleGeo. At SimpleGeo, Derek was given the opportunity to explore different implementations for providing an augmented reality environment for the iPhone.
Paul is the Operations team lead at SimpleGeo where he works to build and maintain fully automated and fault-tolerant systems to support the company’s location platform. Before joining SimpleGeo, Paul brought systems automation to Digg, building out configuration management for 500+ servers. Since joining SimpleGeo, Paul has been focused on systems automation and integration while trying to maintain stability in the face of rapid growth and obscure Linux kernel bugs. In his off time, Paul enjoys tinkering with new technologies, playing tabletop RPGs, and chasing his toddler around the house.
Mike Malone is lead architect at SimpleGeo, where he works on building and integrating scalable systems that power the company’s location platform. Since joining SimpleGeo, Mike has been working to ensure operational continuity in the face of rapid growth, partial system failures, and traffic bursts. Before joining SimpleGeo, Mike helped build the microblogging web site Pownce, where he learned a lot about the technical and social difficulties of scaling an online community. After Pownce’s acquisition by Six Apart in 2008, Mike worked on the TypePad platform team, where he gained a great deal of experience building RESTful web services. In his spare time Mike enjoys tinkering with new technologies. When he’s not on the computer, you can probably find him hanging out with his girlfriend, Katie, and their friends at a good bar.
Comments on this page are now closed.