Agile Earned Value Management

Posted on June 11, 2013 by ThePMJournal

Earned Value Management (EVM), described in PMBoK by PMI, is a technique for measuring project performance and progress in an objective manner. Although EVM involves three areas of knowledge in project management: scope, schedule and cost, when we talk about EVM, in general, it is about budget management. It is well known that EVM offers one of the most efficient method of control and forecasting during the project. Because of its complexity, EVM technique is usually used on large projects, managed “by the book”, in traditional way as in PMBoK. The question is: Can Agile framework offer the same methods and techniques of control and predictability as EVM from traditional project management?

“Agile” people don’t bother with budget management and complicated formulas of forecasting, because they say: “Instead of making plans, estimations and predictions, we better do some work!“. That’s perfectly true! But managers want the business they manage to be very predictable, estimable and to look great on paper (in figures). Agile does not offer too many figures, especially in money, because in an “Agile” company nobody asks: “How much money will we spend on this project?” but: “How many Points per Sprint will burn the team?“.

Although many organizations have at least on Agile methodology implemented (usually Scrum) , in real life, few are “100% Agile”. And this is due not only to resistance to adoption of Agile from the teams, but also because Agile cannot be implemented at all levels in the organization.

The implementation of Agile at all levels is sometime impossible because the discrepancy between what top management wants (an absolute control of all the costs during the projects) and what can Agile teams offer (a relative control of budget). Most of top managers don’t have a clue about what is Agile, but they badly want Agile implemented in their companies because is the new trend in the industry and because it has been said that “increases the efficiency of the teams“. Managers prefer Agile environment also because a part from their responsibility (related to monitoring and controlling activities) is transferred to the team.

So, almost all managers in software industry implemented an Agile framework in their companies, but actually very few run their business in Agile manner. On top management level remains the same traditional management,  while on low/middle levels exists a “new age” Agile culture, with a lot of freedom and non-bureaucratic. With this kind of half-Agile organization, managers decided to split the responsibilities in this way:  “Hey team! Be Agile, deliver fast, auto organize yourself! Related to projects budget, you will develop projects considering Agile Points (or whatever you call it), but let the project finances in our hands!”. The news has been received warmly: Employees were happy, because they enjoyed the freedom: no board meetings, no complicated budget reports and no elaborated plans. Also, management were happy because they had more time for them and the teams needed less supervision.

Everything sounds perfect so far, with one exception: from time to time, top manager need to make some budget forecasting for company owners and this means the Agile teams need to provide some figures about “burning” cost and budget forecasting of the projects.

Here is what figures can Agile team offer

In Agile, the most used method of predictability and control is Burndown Chart. This graphic can offer the following information: “what has been done”, “how much work need to be done”, “ideally, how much work need to be done”. For a small project, the estimations “worked hours” or “days remaining for work” are based on relative measurements, called “Agile Points” and, even they are rough, are enough satisfactory for management. However, when the Burndown chart looks “ugly” (showing a delay of deliverable), a top manager will ask for a detailed report of the project with absolute estimation units (i.e. money). Moreover he/she will probably ask also other details as: What will be the absolute cost (in money) at the end of the project, knowing the actual graph? If you are about to offer an answer strictly based on the Burndown Chart, we can draw a line between current coordinates and using the current or average “burning slope”, we can estimate the most probably date the project will end. Knowing the time remaining to finish the project and the average cost of burning of the Team in Points (a relative value based on team history), it can be obtained a relative cost how much the project will cost at the end. Besides this method of calculation is rough, it will be difficult also to convince a Financial Officer how we have obtained this results.

Another method of budget forecasting

Budget forecasting based on EVM is used on large scale both in government organization but also in corporations. Because of EVM’s notoriety, but also because of holistic approach on budget (EVM includes all operational costs, while Agile BurnDown chart includes only a small part of operational costs), EVM is more suitable for budget report.

How can we can be estimated budget at completion (Estimated at Completion or EAC from EVM) in Agile? There are several methods of calculations. For example: If we know how much money have been allocated for entire project at the beginning of the project (Budget at Completition or BAC parameter in EVM) and considering a burning cost approximately constant (as usually happens, after several iterations), we can find out instantly how much money we will spend until the end of the project: EAC=BAC/CPI (I will show below how can we obtain Cost Performance Index or so called CPI from EVM).

In Agile, the project is not measured in money, but in Points (a relative unit), but if we have the number of points “burned”, by converting Points to ManDays (or ManHours) and then from ManDays in money, it can be quantified the project budget also in money (EVM parameters depends only on money). So we need to know how to convert Points in ManDays (or ManHours). Conversion could be obtain based on previous projects history, or based on the past iterations. Of course the budget of a project is not measured only in money the team is burning, but covers also other costs: operational, administrative, etc. And for an Agile project, which is usually involves a small team, the budget can be expressed only in ManDays because the rest of costs (equipment, electricity, transport, risks, etc) can be included easily in ManDays cost of a resource.

Conversion between Points and ManDays

The following table conversion between Points and ManDays is based on the history of the team:

Please note conversion between Points and ManDays is not linear. Value of the Points usually are discrete values from Fibonacci series (we use Fibonacci, because in Agile, the estimations from present are always taking into account the estimations from the past: project history, complexity of past user stories, etc; similar Fibonacci numbers are calculated based on the last two past values: F(t)=F(t-1) + F(t-2)).

The advantage using a scale between Points and ManDays is because team can offer estimations either in ManDays, either in Points, depending on situation. In my case, I prefer estimations in ManDays. To calculate EVM parameters, I need to collect for each deliverable, at the end of each iteration, the following information: how much % is completed (see the table below: column % Work Completed). To obtain “% completed”, I use exclusively the feedback from the client (or key stakeholders) according with acceptance criteria defined for each deliverable. Please note, Product Owner cannot decide himself “% completed”, unless he has the delegated authority from client (external clients or Sponsor) or Product Owner is the client himself.

So if I want to obtain “% completed” I just asked my clients: is this deliverable complete or incomplete? (To simplify the decision, we can use the “0/100” rule: a deliverable is considered complete, only and only if it matches 100% the acceptance criteria, and incomplete (0% completed), if the deliverable does not match all acceptance criteria).

Note: The rule “0/100” is rule of “all or nothing”. If deliverable is not fully completed, even the deliverable is 99% done, the deliverable is still considered 0% completed. We could also use the rule of “20/80” (a deliverable it is considered 20% completed, if the work is in progress) or “50/50” rule or a combination of rules if necessary. The rules can be established by contract (with the client) or at organization level. I personally do not use any  predefined rule, but I prefer to collect the exact “% completed” for each deliverable, even if this requires more time (in order to obtain an accurate %, we should have multiple acceptance criteria defined for each deliverable and each deliverable should be: independent, measurable, feasible, etc.).

EVM parameters

EVM and Agile Burndown Chart techniques use almost the same principle to calculate the current cost and future cost of the project. You will probably ask “What can EVM offer more than BurnDown Chart?”. Lets find out an answer.

Knowing what is complete and what is not complete (“%Work Completed” in ManDays), we can calculate the Earned Value (EV) of each deliverable (EV is a parameter from EVM and represents how much money values your work).

In table above, the column “Original Estimates” is estimated in Sprint planning, before work actually begins and is equivalent with Planned Value (PV) parameter from EVM.

Example. The Sprint called “Sprint Alpha 20 june” has PV = 2 MD (ManDays), so 2 ManDays were initially estimated, but only a small part of the deliverable was considered done (100% completed), according to Acceptance Criteria. In this case, Earned Value = EV =1 MD. Knowing EV and PV, we can calculate other EVM parameters: Schedule Variante (SV)=EV – PV=-1 and Schedule Perfomance Index (SPI)=EV / PV=0.5.

Why SPI and SV are important?

SPI answers to the following question: What is the speed of development? Our progress is  [SPI] % from planned. If SPI<1, then we are behind of schedule and we will not hit the deadline if we continue in the same way, if SPI>1 then we are ahead of schedule and we could finish the project before deadline. SPI = 0.5, means: We progress at 50% of our potential.

The Actual Cost (Real Cost of the Deliverable)

Planned Value and Earned Value, can be obtained also from Agile metrics (BurnDown chart or using other Agile project management tools). Now, suppose during an Agile sprint, the team has to work overtime or a new equipment has been bought or a new colleague needed to be trained and all this activities have an extra cost not originally included in plan. The extra costs of the project (costs other than user stories costs) are also part of project costs (Actual Cost of the project).

Unfortunately, by default, the Agile framework doesn’t keep the track of Actual Cost parameter, because the forecasting in Agile are only based on remaining tasks (the cost of the Backlog, not the real cost of the project). Since AC is one of the most important parameter in the budget forecasting and since AC is not calculated in Agile Burndown Chart, Burndown chart might not be the suitable method to forecast the budget of a project.

When extra cost occurs during the Sprint we can’t even quantify it in the Sprint Backlog (sprint backlog contains only user stories and tasks and should remain that way). Of course, in Agile we can calculate AC anytime by summing the team’s effort for tasks completed and extra-costs, but realistically speaking, in Agile no one does it, by default.

In the example above, “Sprint Alpha 20 june” has a Actual Cost (AC) of 4 MD (because team made overtime). Knowing AC, we could find Cost Performance Index (CPI) from EVM. CPI = EV/AC = 1/4 = 0.25. Also, another important parameter can be calculated: Cost Variance (CV) = EV – AC = 1 – 4= -3.

Why CV and CPI are so important?

CV and CPI can answer to the following questions: How much money really value our project now? Will we fit into the budget? At the first question we could answer in this way: For each [CPI] $ earn, we spend 1$ (in the example above for each 25 centi (0.25$) earned, we actually spent 1$). We could answer to the second question in this way: If [CPI] 1 (or CV > 0), means the resources of the project are very efficient and we are below of budget. In our case, the resources have a low outturn, because CPI=0.25=25%.

If we know AC parameter, we could also calculate parameter Estimate at Completition (EAC) and Estimate to Complete (ETC) from EVM:

EAC can be calculated in multiple ways, but in most of the cases we can use the formulas: EAC = BAC/CPI (if you know CPI will remain constant) or EAC = BAC + AC – EV (when CPI is unlikely to remain constant or does not reflect how the project will evolve).

ETC = EAC – AC. 

EAC means: How much money the project will really cost? and ETC means: “How much money we will spend from this moment until the end of the project?”

Agile EVM applied in real life

I have tried to apply Agile EVM on 3 companies. Unfortunately, the forecasting and reporting based on Agile EVM had limited success. For example, in F-Startup, EVM report didn’t help to much the top management or clients because they don’t care too much about forecasting and reports… In another company I have worked, the report based on EVM has not accepted as a useful tool of reporting, because the key stakeholders didn’t understand the theory behind Earned Value Management. However I have success on Agile EVM in a startup, where I optimized the operations cost of entire company (SPI and CPI were not used only as reporting tools, but were also used as KPI for staff evaluation. EVM also helped the company to become more aware about the importance of clear definition of projects acceptance criteria and sign-off documents at delivery, pre-sales estimations for clients become more accurate (because we used those estimates at forecast) and we saved thousands of manhours by including a transparent reporting procedures to/from client and internal operations of how the money are spent in the projects/in organization).

Here is a simple Agile project. Contract with client was signed for 1400 manhours, aprox. 7 sprints (2-week sprints). The initial planning was not 100% Agile based, since the budget was fixed, scope was fixed, only the deadline is negotiable. Estimations included the original contract were too optimistic and also the project planning was unrealistic. When the project has been handover to me, the planning has already approved. But after the first demo review, I figure out there might be a problem with the deadline, the budget and possible with the scope itself.

At the end of first iterations, I collected the following information: Planned Value (per Sprint was 160 manhours), Earned Value (per sprint) = 128 manhours, Actual Cost = 116 manhours, a project has a Scheduled Performance Index = 80% (speed of team), Cost Performance Index = 110% (how much do we get back for each dollar spent). So, as you can see in report, only 80% of work was considered “DONE” according to acceptance criteria and to client.

Here were my conclusions:

• project is behind the schedule (SPI=80%)
• we earned more value for deliverable than we spent (CPI=110%, under the budget; funds are spent efficiently), but we are behind of schedule (SPI=80%). So this means we could allocate extra resources in the next sprints (for the same money) in order to produce more Earned Value per sprint and to increase also the SPI.
• if we keep the same cost performance until the end of the project, Estimated at Completion = BAC/CPI = 1400 manhours/1.1 = ~1270 manhours (project will save the cost of equivalent work for 130 working hours).
• if we keep the same speed, the remaining work will be BAC – EV = 1400-160 = 1240 manhours. So based on actual speed SPI=0.8,  the number of remaining sprints will be: BAC-EV/EV = 1240 manhours /128 manhour/sprint= 9.7 sprints. The deadline should be extended with 2.7 sprints or we must plan 206 working hours/sprint (instead of 160 manhours/sprint, as now).
• Report sent to client with the CPI information was irrelevant, because the client want to know only if we can hit the deadline (the project price was fixed), so we decide to remove the CPI from client’s report. But CPI, showing the under budget/over budget, was extremely useful for the Sponsor because he could allocate more resources, for the same money, in order to create more earned value for this project).

Note: this is one example when Cost Performance is irrelevant for client. Another reason why CPI is not included in project report for clients is: the companies selling “Agile services” they are generally “selling” availability, not fix-price projects. Because you offer only the “availability”, the scope and the real cost of the project it is unknown, so the budget forecast might be useless. In a 100% Agile environment, you can forecast the budget roughly only on certain level (based on the lowest and the highest team speed). If you have recently implemented Agile in your company, pay attention on terms of contracts, because the contracts should be based on availability and not cost predictability. Implementing Agile with clients do not understand the Agile philosophy, might be dangerous, because, on long term, might impact your credibility, customers satisfaction and could even change the entire business model of the company.

CPI and SPI, KPIs to evaluate company performance

In three of the startups I have worked, one of my responsabilities was to define performance KPIs for the employees. As an experiment, I tried to see in what degree the EVM (SPI and CPI) parameters can evaluate performance of the employees and projects. I started with a pilot project where I have collected a series of informations for each team/resources/feature. Example:

To see more details about how can Agile EVM be used to evaluate people, projects and operations in general, read EVM, KPI for evaluation of staff, projects and company.

Agile EVM, reporting tool to top management

If most of the Agile projects do not use CPI parameter, here is an example of Agile project, where CPI parameter is important (project with variable scope, variable cost, fix deadline, 3-week iteration, the client and the sponsor are directly interested on real cost of the project). Agile EVM parameters are:

Note: The table can be easily calculated in Excel, with few clicks at the end of each Sprint Review. Please also observe, the BAC parameter (total budget allocated) is not constant, because in our case, after each Sprint, the scope of the project was changed and the cost baseline has also been changed.

Having all this information, we could draw, to the delight of top management, a series of very useful graphs:

Based on information above, here is a variant of Burndown chart “customized”, called suggestively “Estimated to be Completed” made on EVM calculations and Agile estimations:

“Estimated to be completed” is a graph is looking good on board presentation. The graph “says”: “until 23th of August 80-90% of the product will be completed” (a very useful information knowing in the second part of August, when everyone is on holiday, almost entire project is completed one month before deadline). This graph give to organization a sens of “tranquility” and completeness. Also the visual impact is strong than in a BurnDown chart (where the trend is always descending: Work Remaining vs. Time).

“Estimated to be completed” has a “natural” ASCENDING trend (because accumulates the days worked and not remaining worked, as in Burndown chart), and suggest the idea of “PROGRESS”, “EVOLUTION”, although in reality is just a “Inverse BurnDown Chart”.

Finally, here is another version of “customized” BurnDown chart, called “Estimated to be Completed” (vs Total Work) obtained by summing EV values until present (to estimate the future EV in next sprints, we can consider having the same Earn Value as in present or extrapolate a value based on our experience/project history). The graph highlights when change of scope occurred (new change request, but also cut scope).

Conclusions

After I have applies Agile EVM in three companies (2 startups and 1 middle-level company), I have learned that implementing Agile EVM and well-defined KPIs for EVM at all levels of operations, it has the following benefits:

Follows: Reactive and Chaotic Management.

Recommendation: EVM, KPI for evaluation of staff, projects and company, Agile under PMI umbrella.