In a recent conversation I was asked about PMBOK (Project Management Body of Knowledge) and its applicability to measuring an Agile project manager's effectiveness.  While I did not know much about PMBOK and its measurement approaches I was curious to understand the problem.  As it was explained to me, the problem was along these lines... When following a PMBOK approach to measuring a project manager's effectiveness, one of the things you would look at is on-time delivery.

If the project was delivered exactly on time the manager would get a rating of one.  If the project was delivered ahead of plan then the manager would get a rating of less than one (based on how much ahead) and vice versa if delivered late.  However, when following the OutSystems approach to Agile application delivery we define a project time-box and even if we deliver all features early, we will then fit any additional backlog items into the plan and keep working until the time-box is complete.  Thus, you never finish early - you always finish on time and in most cases you just exceed your customer's expectations on how much you deliver.

An OutSystems Agile project in sprints

So, off I went and did some research on PMBOK.  What I learned is that the OutSystems' Agile methodology, while based on SCRUM, incorporates lots of extra management roles and responsibilities that align with PMBOK.  I will leave the convergence of agile and PMBOK for a later discussion, but in my opinion, there are lots of PMBOK practices that are applicable to running Agile projects in Enterprise IT shops.

What I am interested in is the following: how can we measure the success of an Agile project manager's success?

In the conversations on Agile and PMBOK I've had over the last couple of weeks I have come to the conclusion that the best measure for an Agile project manager's effectiveness is not based on on-time delivery, staying within budget, etc.  But rather a measure of a new application's adoption by the business.  Everyone I have talked to on the topic agreed with this notion of adoption but none have really offered a concrete technique for measuring adoption.  In most of the discussions the notion of return on investment came up as a solution.  However, in drilling into ROI we always came to the conclusion that while ROI is important it is not necessarily a good measure of your project manager's effectiveness.

So my quest for a good measure of an Agile project manger's success continues.  Everyone agrees that application adoption could be it but I have yet to find anyone with a good definition on how to measure it.

Your thoughts?