EVM, KPIs for evaluation of staff, projects and company

Posted on September 15, 2013 by ThePMJournal

In three of the startups I have worked, one of my responsabilities was to define performance KPIs (Key Performance Indicators) for the employees. As an experiment, I tried to see in what degree the Earned Value Management (SPI and CPI) parameters can be used to evaluate the performance of the employees and the projects. I started with a pilot project where I have collected a series of informations for each team/resources/feature. Example:

Resources	PV	AC	EV	SPI	CPI
Backend	59	64	45	0.76	0.70
UX&Design	6	24	5.25	0.95	0.22
WebFrontEnd	34.5	65	30.8	0.89	0.47
QA	37	40.3	26.2	0.72	0.65
PM	28	42.5	28	1.00	0.66

It is known that ideal parameters are SPI=1 (when project is on schedule) and CPI=1 (on budget), but in practice, very rarely, the development teams reach or exceed these ideal values. At the begining, when I begin to collect statistics, I was worried about the fairness of the evaluation method, because even the best people in company had subunitary indexes. Despite this facts, nobody complained about lack of performance or about cost or delays. Then I realized the projects success did not depend on matching the ideal performance indexes (for  cost and budget), but depend on how customers perceive performance (customer satisfaction). So, the success of the project depend on the extent of which a Project Manager convice his clients the success of the project is relative to measurements criteria he can control (depend on how Project Manager presents the performance indexes). If the clients are happy with the PM’s explanations and justifications, then, even the project is under schedule or above the budget, the clients will be available to renegociate the scope, the deadline and the cost of the project (I will explain later the reason why companies are willing to accept it).

In an “Agile” project, the negociation is the key to success. Each company has its own problems (lack of personal, over allocated resources, distributed teams, etc), all these making impossible, in most of the cases, achieving the ideal performance indexes, but this don’t stop the companies to make profit, to satisfy the clients, to deliver services and products at high exigency demands. That’s why, a Project Manager must estabilish his own performance criteria depending on clients (projects) and to organization. Here is an example, from our organization:

As we can see, all SPI and CPI parameters are subunitary, but with all these facts, the company accepts ideal parameters cannot be achieved in present conditions (economical and organizational reasons). That’s why, company imposed KPI performance parameters for SPI and CPI: KPI-SPI=0.75 (calculated on average speed of on employee in company and assuming a person, no matter how good is, can’t deliver more than 6 hours/day from 8 hours total allocated) and a KPI-CPI =0.5 (I will explain below why has this value). In the image above, with green, are parameters considered to be good (above target KPIs), with red tones are parameters considered to be poor (under target KPIs) and the recomandations will be made to each employee.

A method to define KPIs for projects, staff and company

1. Start with Agile EVM definitions:

• SPI = EV/PV (represents development speed or % of work “done” according to client, acceptance criteria and planned activities)
• CPI = EV/AC (represents how much company earns for each 1$ spent)

            where:

• AC = how much really work took (manhours, Points, money). This value is ussually take from time report application (which each company is supposed to have it).
• EV = how much really work values (manhours, Points, money). This value represents how much client considers DONE according to plan and acceptance criteria.
• PV = how much was planned (manhours, Points, money)

Interpretation:

• If SPI <1, project is behind the schedule, if SPI=1 (on schedule), SPI>1 (ahead of schedule)
• If CPI <1, above budget, CPI=1 (on budget), CPI>1 (below budget)

2. Based on SPI, CPI, we can define KPI at operations level (per company)

• KPI-SPI = EV/PV = SPI
• KPI-CPI = (EV * [billable hour per resource])/(AC * [average wage per resource]) = CPI*[revenue per hour]/[cost per hour].

Notes: Let’s say, for an employee, wage is 10 EUR/h, the company can earn 50EUR/h. In our case, the company is aware that it cannot obtain a Revenue of 50EUR for each worked hour, so would be happy if would obtain 25EUR/h revenue (even it means 50% less than ideal revenue). So:

• Wage per resource = cost_per_resource_hour = 10 EUR/hour.
• Billable hour = ideal revenue_per_resource_hour = 50 EUR/hour;
• target_revenue_per_resource_hour= 25 EUR/hour.

Ideal KPIs per company are:

• Ideal KPI-CPI_per_company = ideal CPI *  billable_hour/wage = 50/10 = 5
• Ideal KPI-SPI_per_company = ideal SPI = 1 (here we can include in formula also wage and billable hour, but SPI ussually reflects the burning speed vs planned, so we can simplify the formula to be relative only to manhours (points) as in Agile).

Company knows the ideal values for KPI parameters are hard to obtain in present conditions, so it has been decided to define the following target KPIs:

• target KPI-CPI_per_company = ideal CPI * [target_revenue_per_resource_hour]/[cost per hour] = 1* 25/10 = 2.5
• target KPI-SPI_per_company = average SPI = 0.75 (based on average history speed per employee)

3) Define KPI per resources and per projects:

Notes: The formulas for KPIs per company are ussually not public to internal staff, because involve salaries and revenues. So we must define KPI for resources and for projects in a transparent way:

• Target KPI-SPI_per_resource = Target KPI-SPI_per_project = KPI-SPI_per_company = 0.75 (can be equal to KPI-SPI_per_company = 0.75 because its value can be achievable, according to project history)
• Target KPI-CPI_per_resource = Target KPI-CPI_per_project (cannot be equal KPI-CPI_per_company=2.5 because it involves revealing cost and revenue information to staff).

So, we can define:

• Target KPI-CPI_per_resource = [target_KPI_CPI_per_company]/[ideal_KPI-CPI_per_company] = 25/50 = 0.5 Notes: This parameter KPI-CPI=0.5 is achievable based on projects history and can be choosen as a reference (target). Even the parameter is much below ideal (1.00), the company realize target Revenue_per_resource_hour /Cost_per_resource_hour = 25 / 10 =2.5 (250% revenue vs cost, so 150% gross profit for each working hour, which is not quite bad) and this is why the company accepts a subunitary CPI = 0.5.

4) How EVM-KPIs impact business operations:

For the moment, despite the resource performance technically looks bad, the profitability figures look great. But on long term, each company wants more profitability, so it is wise to have a plan in order to increase staff performance:

On short term:

1. Set a standard for KPI-CPI = 0.5 and SPI=0.75 (per resources and projects).

On long term:

1. Once at 6 months, increase standards KPI-CPI per resource at 0.6, 0.7, etc. and SPI_per_resource at 0.85, 0.9, 0.95, etc.
2. Define KPI depend on level of seniority, the resource type, the resource wage/hour.
3. Define different KPIs for projects and for resources.
4. Use CPI and SPI as standard reporting for performance.

See how EVM are applied in Agile and what are the benefits: Agile Earned Value Management.