Rails (Im)maturity Model?

2009 February 17, 03:21 h - tags: ruby obsolete

As usual in the Ruby on Rails community, another drama took place :-) It all started by a very well intended Obie Fernandez in the Rails-business Google Group. He jotted down a proposal for a Rails Maturity Model (RMM), which would somewhat resemble the CMMi in its core but geared towards helping companies and clients to get higher quality Rails projects.

I can totally understand where it went bad: the association with “CMMi” and the assumption that it is all about “certifications”. Good developers hate certifications, this is a given. We all know that you can totally suck in programming and still get certified at something fancy. When you’re ‘certified’ you’re only asserting that you can memorize random stuff out of a book, not that you can really apply the theories in real life. Hence, real developers are always terrified of working close to ‘certified’ developers with no real working hours.

The bottom line about certification is that good developers will be good without any certification process, period. It is also a given that certification has the sole purpose of being a marketing stunt for people with mediocre real past success cases in real life. HR people – who are total noobies about technology – rely on certifications to filter out candidates for an interview. They would be lost without it. I know, I am being too general, but you get the idea. Don’t argue on this point just yet and keep on reading.

So, this is a vicious market, sorry. The rules of engagement are almost craved in stone and it is very difficult to change them. Agilists know what it is like to fight old habits in companies. Convincing an old corporate geezer to take a look at the Agile philosophy hurts, badly.

Since the inicial discussion, Obie wrote another article trying to be more specific on his ideas of a maturity model for Rails. And just yesterday he was really pissed off and wrote yet again that RMM has nothing to do with certifications. Understandably, people can’t spell CMM without thinking of certifications right away. Please, take your time to read Obie’s articles before continue reading my points here. But rest assured: if you don’t agree with Obie, chances are that you have no idea of what CMMi stands for.

People don’t understand CMMi

Ok, for starters, I am not AT ALL a fan of CMMi. But let me explain about CMMi so I can go back to Obie’s point on the RMM. And also note that Obie himself stated that he doesn’t fully understand the CMMi either.

I am not an expert in CMMi by far. I’ve researched a lot about it and for a short period of time I lived and breathed CMMi. Actually it was SE-CMM, prior to it becoming CMMi. I think it was around 2002. I’ve studied the texts, participated in conferences, talked to consultancies and so on. I was very much into it – and believed it, go figure! 3 years later, I also certified myself as a PMP (a Project Management Institute certificate for Professional Managers) and I studied a lot of the OPM3 model. Before all that I’ve been into the Unified Process and RUP stuff. So, yes, I’ve been into lots of ‘enterprisey’ stuff in the early days. I am out of the drugs by now, thanks for asking. None of it matters because I am very rusty in all this.

So, first and foremost: the CMMi is not a “standard” and, therefore, you can’t be “certified” in CMMi – at least not in the legal definition of “certification”. There is no certification for CMMi. It is a body of knowledge that outlines a model of process improvement, thus ‘maturity’. Well, at least that’s the theory.

You can go through an appraisal though. The SEI, the lead organization that manages CMMi, is still an academic research center in Carnegie Mellon University. People can go through a lengthy process to become a Lead Appraiser and work for Partners to appraisal a company. I think prior to 2007, when there was an update to the appraisal policy, it was a little bit easier to get appraised.

Now, don’t quote me on that because I will be saying just what I have heard in the industry some 6~7 years ago. Back then we were very interested in the outstanding growth of the indian IT market, specially the big guns such as Tata, Infosys, Wipro, Satyam, etc. There were unfounded rumors that they were all appraised CMM Level 5 because they had their own people certified as Lead Appraisers. If you just look at the SEI partner’s list, you will see them listed there, though it is not evidence for anything and this is not an accusation. Rumors, just rumors. The appraisal model has been criticized before, and if I am not mistaken, one of the reasons being that once you’re asserted to be in one Level of maturity you don’t go down because there was not a post-appraisal follow up.

CMM was first created as a kind of an insurance for the military software projects to assert that the winning companies could demonstrate at least a shred of evidence that at the very least they had some best practices and the processes in place to back them up. Anyone that worked for Tata will tell you how bureacratic they are. The appraisal process itself (SCAMPI) is based on evidence collection – though this can be very subjective at times and it is not an investigative process.

Well, to be fair, I hope that big companies do at least have that. Really big government projects have to have some level of manageability. A maturity model is by no means a hard evidence for guaranteed success, but it helps for big guns to have that. Companies such as Lockheed-Martin, for instance. If you’ve never worked for really really massive enterprise projects, you have no idea of the nightmare.

But again, I have the personal feeling that the SEI’s original goals were corrupted long ago. The way the market works is pretty straight forward: big companies and governments “requires” you to be CMMi at some Level. So, ‘serious’ companies will invest some time (a couple of years) and money (a lot) to put their processes in place and call in an appraiser. This is very expensive and traumatic to do, so only companies that can take the punch will do it. This is very elitist indeed, but it works. This is outside of the market that I personally want to be in, that’s why I walked away.

If you have no idea of what CMMi stands for, it is a model divided in 5 levels of maturity, from 1 to 5, and can be summarized this way:

  • Level 1: Initial (Ad hoc, chaotic, no processes, success out of luck, where most companies really are)
  • Level 2: Repeatable (some project management, some discipline, a rough process that can be used repeatedly)
  • Level 3: Defined (the organization has institutionalized the processes)
  • Level 4: Managed (management means that is can be quantified, there are metrics in place to actually measure a project)
  • Level 5: Optimizing (with the proper processes in place, the process can give feedback to itself, helping to refine the process)

To assert that you’re in a certain level, you must implement (in a verifiable manner) a series of Key Process Areas (KPAs). Each level has its own set of KPAs which define a Goal, Commitments, Abilities, Activities, Methods of monitoring, Methods for verification. As I said, this is not a small thing.

“Implementing” the CMMi up until Level 5 can really take some good years. Implement a KPA properly means really to understand the gut of it, sometimes knowing how to consciously bend it over and still have it validated. It is a real struggle to go through the whole process, specially when you already have lots of projets going on in an ad hoc manner (Level 1)

There is a whole lot more to be said about the CMMi and I recommend you read this small FAQ about it that can be quite enlightening. Also read some of the criticisms, specially to its appraisal system.

This is a very very short summary of what CMMi is. There is a vast literature about this subject and if you’re interested I suggest you go research about it. You will learn a lot, but don’t take it too seriously. This is the kind of thing that every good developer/manager should’ve read at least once in their lives.

Some people think that it is possible to blend Agile with CMMi. I wouldn’t say it is impossible, but I would certainly ask “What is the point?” I really think it is kind of pointless to attempt such a thing, but I digress.


Now, going back to Obie’s idea, the way I understood it, it is just an idea of maybe implementing something akin of CMMi’s 5 Levels of maturity, where in each level you comply to a set of KPAs, such as “we do pair-programming 100% of the time” or “we have extensive and almost total test coverage”. Not going into the merit of the KPAs themselves, one could assert a certain level of maturity with some well known criteria, if they can be clearly defined.

The idea in itself is not bad. Having a reliable, updated body of knowledge is a good idea. The CMMi is based on past history and past data of successful projects. A possible “RMM” would be similar, with the best practices and techniques used by the most successful companies of the time, such as Hashrocket or Pivotal.

There are some problems though: this is an appraisal for the company, not for an individual, so the problem of lack of good quality developers would still be an issue as this doesn’t attempt to solve this problem. The second problem is just that there is still not that many Rails projects, consultancies and a true record of past projects to even start thinking of a maturity model. We are, by definition, not mature enough yet. And don’t take this as a criticism to anyone. Thinking about it, Rails is just 4 ~ 5 years old now. This is just too short.

Finally, Rails is a technology. Even though the idea may sound compelling for us, fans, Rails is a technology, not an specific professional area such as “Software Engineering”. You can certainly apply Rails into CMMi companies, but having Rails implemented properly doesn’t lend anyone a Level 5 stamp.

I think the CMMi is not a very good starting point as it is just too monumental, with a very very big overhead, and very very bureaucratic processes. It is just like RUP vs Agile again. The core idea is good if we keep the CMMi analogies out of the question.

Now, I do think the idea of a body of knowledge is appealing. This could be an organic repository where people would be able to organize their metrics, their own discovered best practices and outlines for successful projects. Over time new material would stack up until we could finally have enough data to actually start thinking of maturity: when the whole community goes into the next level.

Neither a possible RMM nor CMMi actually answers the questions most clients have. None of them can guarantee a software life-cycle with real healthy, without sloppy programming. None of them will make it easier for customers to find good freelance Railers. And if you, as a savvy customer, ever need a Railer you would never trust a consultancy with an “RMM badge”, I bet.

In the end of the day, processes are important, but never to become too overwhelming. There is a gray area between monumental models such as CMMi and agile methodologies (I know, I know, a capability model is not an engineering methodology, I am just comparing complexity). I think the community is still too young to make decisions on maturity. But, I am open to ideas. If someone come up with a detailed RMM specification proposal, I would be very interested in reading it.

But again, I do think the Agile philosophy, the techniques from XP, DDD are enough by now. Before we start debating on bigger processes, some people don’t even know the basics yet, such as algorithms, data structures, proper OO programming, etc. Heck, there are web developers that don’t even understand the HTTP protocol or the TCP stack yet. This article is by no means an attempt to answer the question and it is still open for debate, but it is very valid to debate it. Bashing and complaining will not bring good ideas to the table.


comentários deste blog disponibilizados por Disqus