Out on a limb with data persistence
This past weekend I was searching for open source implementations of a web application that has recently become popular. I came across several, but one in particular seemed to have really good community energy and direction, so I explored it further.
The app in question is built with Ruby-on-Rails, with which I have enough experience with to be reasonably confident. Rails seemed like a good choice, so I continued to dig deeper. I learned that the application is relatively new and has no deployments running at serious scale. Since I anticipate my little site could possibly blossom into a million pages per month or so, I was somewhat concerned that no existing sites approached anything like that level of traffic.
Still, a million pages is not that much compared to the big sites, and I didn’t really expect scalability to be a big problem ( at least not unless the site grew much larger.)
Then I hit the issue that stopped me dead in my tracks, and I realized I had truly become more conservative in the choices I am willing to make. This Rails app uses MongoDB for persistence, which was a step too far off the beaten path for me. Don’t get me wrong, as I think MongoDB is genuinely cool and probably works great. It may even be the best possible tool for the job.
But I will probably never know. I simply don’t have the stomach for the double-risk of choosing a fairly new open source web application AND ALSO the non-traditional data persistence system it mandates. If the web app supported multiple implementations of a persistence interface, then I would probably try the MongoDB with confident knowledge I could switch to a more familiar SQL environment if necessary. Lacking that, I just couldn’t go out on a limb with the persistence component. I just couldn’t.
I feel a little bit cowardly, but I have paid the price for being at the bleeding edge before and don’t tread into such territory casually. I’ve got a lot of SQL experience and have the tools and the resources to manage, backup, and maintain high-performance SQL databases. If I took the bet on MongoDB, I’d have to start all over, and I just couldn’t balance the risk-reward equation.
Maybe if I was still in my 20’s, or even still in my 30’s…
Rick Ross
> Maybe if I was still in my 20’s, or even still in my 30’s…
Hmm, why do we older guys give fuel to the fire? Perhaps a better way of saying it is perhaps that your previous experience of getting successful sites running on a large-ish scale shows you that it pays to be cautious?
@Andrew McVeigh
You’re undoubtedly right that it’s experience which proves “once burned, twice shy” and not just age. A younger, less risk-averse edition of me might have gone forward with it anyway just out of curiousity regarding the MongoDB. Data persistence, however, is just not one of the areas where I am comfortable with taking too many risks.
So what your saying is… now that you are in your 40’s (+?) you are unable to use tools with which you are unfamiliar. Don’t blame it on cutting edge, or scale, or any other such nonsense. Call it for what it is. *You* have decided to become irrelevant and to stop learning and expanding your horizons. That is *your* choice.
As a developer and architect I am excited by the opportunities and challenges being offered by the new generation of (not-so-stodgy) ‘NoSQL’ engines. Oh, and BTW, the largest sites in the world are using key/value stores as their primary storage (Google would not run without it). Tools like MongoDB and CouchDB offer far more in terms of their ability to horizontally scale than any monolithic ‘traditional’ choice. Either of those engines could serve a million page a *day* site without breaking a sweat with a few app instances in front of them.
What it sounds like is that you have chosen to retire, without admitting it even to yourself.
(PS – I am in my 40’s too, so don’t tar all of us with that old saw of ‘experience’ forbids you from trying new things. Some of us continue to grow.)
That’s really not at all what I am saying. In fact, what I said is that I am not willing to swallow the COMBINED risk of two separate, unfamiliar pieces.
Understanding why not is as simple as compound interest. Imagine there’s a 70% chance that using the unproven web app would be a success (apart from the persistence engine) and a 70% chance that the unproven persistence system would also be successful and reliable. My combined risk would then be 0.7 * 0.7 = 0.49. This suggests a greater than 50% risk I would end up in problem territory. That’s more risk than is comfortable, especially for unnecessary risk.
Also, it is easy to presume that newer equals better, but that’s not always the case.
I know where you’re coming from regarding the risk. There are two kinds of risks here, it seems: (1) risk of bugs or other unanticipated problems in MongoDB itself; and (2) risk of not being able to maintain it because it’s effectively a different approach and requires a different way of thinking. It sounds like it’s (2) that concerns you the most, and you’ll need to evaluate that yourself. Regarding (1), if you have a conservative customer, then sticking with MySQL is the safe choice. I avoided upgrading from MySQL 5.1 to 5.4 last year simply as a CYA move in a critical project.
I’m currently in the process of setting up a new VirtualBox VM on my laptop so I can experiment with MongoDB. I’m planning a new version of my company’s flagship product for next year and I want to start playing around with a MongoDB implementation because once I stop thinking about relational databases, a key-hash store really makes a lot of sense for my app. What I’m going to do is start building a simple, toy version of the app using MongoDB and see how far I go before I hit a problem. If all goes well, this will be version 0.01 of the new system. If not, then I’ll have learned something at least.
By the way, I turned 53 yesterday. My first experience programming was in high school using Fortran II on a DEC PDP-12. Look it up
Being aware of risks and making informed choices is not a bad side-effect of being older. It’s the reason why at 45 you’re far less likely to wrap your car around a tree than at 25.. and the same reason why you’re unlikely to lose a day trying to rebuild your development environment after installing the latest version of a critical component (just because it was the latest).
I don’t think you’re being cowardly Rick. It’s the same logic that stops you from twiddling multiple variables when you’re debugging – the complexity rises exponentially with each new factor you add to the mix.
I get where GR is coming from – none of us “oldies” can afford to stand still and not try new things. But similarly we can’t discount the experience that comes from being in the game for a long time. Not every shiny new thing turns out to be gold.
I’m with ferrisoxide and Rick on this one. It’s fun to experiment on small test apps with new technology, but having to rely on untried stuff with your reputation on the line takes some serious consideration.