In a recent conversation I was asked about PMBOK (Project Management Body of Knowledge) and its applicability to measuring an Agile project manager's effectiveness. While I did not know much about PMBOK and its measurement approaches I was curious to understand the problem. As it was explained to me, the problem was along these lines... When following a PMBOK approach to measuring a project manager's effectiveness, one of the things you would look at is on-time delivery.
If the project was delivered exactly on time the manager would get a rating of one. If the project was delivered ahead of plan then the manager would get a rating of less than one (based on how much ahead) and vice versa if delivered late. However, when following the OutSystems approach to Agile application delivery we define a project time-box and even if we deliver all features early, we will then fit any additional backlog items into the plan and keep working until the time-box is complete. Thus, you never finish early - you always finish on time and in most cases you just exceed your customer's expectations on how much you deliver.
So, off I went and did some research on PMBOK. What I learned is that the OutSystems' Agile methodology, while based on SCRUM, incorporates lots of extra management roles and responsibilities that align with PMBOK. I will leave the convergence of agile and PMBOK for a later discussion, but in my opinion, there are lots of PMBOK practices that are applicable to running Agile projects in Enterprise IT shops.
What I am interested in is the following: how can we measure the success of an Agile project manager's success?
In the conversations on Agile and PMBOK I've had over the last couple of weeks I have come to the conclusion that the best measure for an Agile project manager's effectiveness is not based on on-time delivery, staying within budget, etc. But rather a measure of a new application's adoption by the business. Everyone I have talked to on the topic agreed with this notion of adoption but none have really offered a concrete technique for measuring adoption. In most of the discussions the notion of return on investment came up as a solution. However, in drilling into ROI we always came to the conclusion that while ROI is important it is not necessarily a good measure of your project manager's effectiveness.
So my quest for a good measure of an Agile project manger's success continues. Everyone agrees that application adoption could be it but I have yet to find anyone with a good definition on how to measure it.