Can Function Points Be Counted/Estimated From User Stories?

Trusted Advisor

Introduction

Since the invention of function points (FPs) any time new development methods, techniques, or technologies are introduced the following questions always arise:

  • Can we still use FPs?
  • Do FPs apply?
  • How do we approach FP counting?

These questions came up around middleware, real-time systems, web applications, component-based
development, and object-oriented development, to name a few. With the increased use of Agile methodologies, and therefore the increased use of User Stories, these questions are being asked again.  It is good to ask these questions and have conversations to ensure that the use and application of FPs is consistent throughout the industry in all situations.  The short answers to the questions are:

  • Can we still use FPs? YES. 
  • Do FPs apply? YES. 
  • How do we approach FP counting? The answer to this last question is what this article will address.

Getting Started – Determine the Purpose and Scope

As with any FP count, it is important to identify the purpose of the count and to fully understand how the resulting data will be used.  This will ensure that the correct timing, scope, and approach is used for the FP count.  The following are examples of situations where FPs can be useful.

Purpose: High-level estimate to determine feasibility

If the purpose is to determine the feasibility of moving forward with the project or to complete a proposal, then typically a high-level estimate in a range is adequate. For this count the timing would be "now," and the scope would be whatever functionality is going to be developed. At this point in the life cycle not all information may be available, so some assumptions may need to be made. It is important to document these assumptions so that if the project progresses differently than planned it can be explained. For example, a User Story may state that "As a User I want to have a Dashboard showing application statistics."  It may be too soon to know the exact details, so an assumption of five average complexity External Outputs (EOs) may need to be made.  

Purpose: Estimate for Project Planning

Once a detailed plan is required, then more detailed estimates for effort and cost are necessary. For this purpose, the FP sizing should be completed at the start of the life cycle and updated at each major development stage. For Waterfall it could be at Requirements, Logical Design, and Physical Design phases. For Agile, the timing could be at Program Increment (PI) planning, or Sprint planning or both.  This purpose will require more accurate and thorough data, which requires a more detailed FP count, so more detailed User Stories are typically available. For example, the above Dashboard User Story may be broken down into 5 separate User Stories each describing a specific report: "As a User I want to be able to see a pie chart showing customer complaints by type."  In this case, each report can be examined to determine uniqueness and counted accordingly.

Purpose: Manage Change of Scope

Once a project is underway it is a good idea to track changes in scope to determine if the effort, cost, and schedule are going to be impacted by the change. These types of counts can be completed at different phases or at the time the scope change is identified. Once a change is sized using FPs, estimates can be developed to determine if the change should be incorporated into the current project and/or Sprint, or moved to another project and/or Sprint. A new User Story could be "As a User I want to be able to search customer complaints by type." In this case, a new report would be identified. If the User Story was "As a User, I want to be able to choose the color of customer complaint types in the pie chart;" this would be a change to the initial report we counted.

Purpose: Measure Quality and Productivity

If the purpose of the FP count is to support measuring the actual quality and productivity achieved for a project, PI, or Sprint, then typically User Stories wouldn’t be the source document of choice. This type of count is completed once functionality has been delivered, so ideally one would want to use the "live" system or user manuals to identify the actual functionality delivered to obtain the most accurate measurement. However, if access to the system isn’t available, User Stories may be the only source documentation available. Often documentation isn’t updated after the fact to show changes of what was and what wasn’t implemented so for this purpose it is important to confirm with development staff and/or users what was actually delivered along with referencing the User Stories. 

Utilizing User Stories for FP Counting – Overall Approach

Once the purpose and scope have been determined, the actual FP counting can begin. Applying the International Function Point User Group (IFPUG) rules is the same regardless of the purpose; however, the level of detail and the inclusion of functionality may be different depending on how the data will be used.

Conducting FP counts from User Stories is a bit easier than from other documentation since the majority of User Stories focus on the User perspective of "what" functionality is desired and not on "how" the functionality will be developed and delivered. Sometimes this perspective is difficult to find when looking at Designs or even the flow of physical screens. User Stories by their nature keep the FP analyst seeing things from the User perspective.

The IFPUG counting process starts with defining scope and boundaries and then moves on to identifying data functions and transaction functions. With a list of User Stories, it is more likely that all of this will be decided together as the count develops.

When counting from User Stories, the best approach is to just start walking through them one by one.  Oftentimes User Stories are grouped by categories (e.g. Order Entry, Validations, Reporting, Financials, etc.). If that is the case, it is best to focus on one category at a time. If it is early in the life cycle and the application boundaries are uncertain, it is best to take a first cut at counting the functions. Once the full scope of functionality is known boundaries can be determined and the FP count can be adjusted as necessary.

In following the IFPUG rules, it is important to count the logical functions. This can be difficult depending on the level of User Stories. It would be wonderful if everyone followed the same format and wrote User Stories the same way, but unfortunately that is not the case. One organization may have one high-level User Story for a project, while another organization may write multiple User Stories for the same functionality. One of the benefits of using FPs for sizing is that the method is consistent across all methodologies and isn’t impacted by how the documentation is completed. For example:

High Level – One User Story

  • As a User, I want to be able to enter, update, delete and view orders in the system to avoid manual paperwork.

Lower Level – Multiple User Stories

  • As a User, I want to be able to enter new orders in the system to stop paperwork.
  • As a User, I want to be able to edit orders previously entered in the system to stop paperwork.
  • As a User, I want to be able to delete orders previously entered to avoid incorrect orders be processed.
  • As a User, I want to be able to enter selection criteria to view orders previously entered in the system to stop searching paperwork.
  • As a User, I want the system to use entered selection criteria to display the correct orders to stop searching paperwork.
  • As a User, I want the system to validate the data entered into the fields when an order is added or updated to ensure accurate data is entered.
  • As a User, I want the system to validate the ordered product is "on hand" before accepting the order.

In the above examples, the FP count would be the same. When a User Story seems to be at a high level, it is important to break it down into all of the Elementary Processes (EP). When User Stories are written at a lower level, it is important to look at all of the similar stories together to potentially combine them into the EPs.

The result of the example above is as follows:

User Stories and Function Points

User Stories typically equate to the Transactional Functions (EIs, EOs, EQs); however, it is important for the FP Analyst to also keep Data Functions in mind while analyzing the User Stories. There may not be a list of tables or a data model available, so the FP Analyst may have to assume the ILFs based on the transaction functions.

If early in the life cycle assumptions may need to be made as documented above. Since the User Stories imply the project is automating a manual system then all functions would be new. That would mean that in order to edit or display orders previously entered, they would need to be stored somewhere; hence counting the ILF. 

If at all possible, the FP Analyst should meet with Subject Matter Experts (SMEs) who understand the User Stories to get a full understanding and/or answer any questions. In addition, the FP Analyst should reference existing systems that may be comparable or past counts that may be relevant. FP Analysts usually have knowledge of many types of systems. It is okay to bring that knowledge and experience to the FP count to help identify potential functionality. Of course, everything still should be validated by the SMEs. 

If SME involvement is not possible, or if things are still not clear, then any assumptions that are made need to be documented fully. This will ensure that the FP count can be explained and updated correctly as the project progresses. In the example above, the assumptions document how the complexity was determined (e.g. Product file used for validation on Create and Edit EIs; Data Element Type (DET) assumptions). In addition, any further questions are documented (e.g. Need to check for multiple order Types that could impact the Record Element Types (RETs) and thus functional complexity of the ILF – this may also impact the number of Transactional functions).

Agile Development - Additional FP Counting Considerations

Since User Stories are typically associated with Agile development, it is worth mentioning a few items to consider for the FP counting in terms of timing and inclusion.

FP counting can be completed at the Program Increment (PI) level and/or the Sprint level. The PI usually encompasses the final delivered functionality, so the FP counting is completed normally. For an estimate, the count can be completed at PI planning. For quality and productivity measures, the counting occurs at delivery of the PI. Counting Sprints is handled a little differently.

Sprints can also be counted for estimation/planning and at the end of the Sprint for productivity and quality measures.  However, the sum of the Sprints is often greater than the PI count. The level
at which the User Stories are written can be impacted by the time boxing of the Sprints. For example, an initial User Story may be, "As a User, I want to be able to create a new order." During Sprint planning, it may be determined that the entire function cannot be completed in one Sprint, so it may be changed to two User Stories:

  • As a User, I want to be able to enter general information when creating an order.
  • As a User, I want to be able to enter order details when creating an order.

In this case, the FP count of the Transaction would be as follows:

FP count of user stories

The Sprints cannot be added together to obtain the total FP count for the project. Counting at the Sprint level is usually for internal measures to ensure the PI goals will be attained. It can also point out inefficiencies in the development process. If too much "rework" is occurring, perhaps changes need to made in how the project is being planned and managed. The ultimate goal would be to complete an entire EP in one Sprint and only have to revisit it in a later Sprint if new requirements are discovered.

Conclusion

FPs are the best measure for "size" and can be used for all methodologies and technologies. FPs can be counted from any documentation or from just interviewing SMEs. The most efficient and accurate FP counting uses both supporting documentation and information from SMEs. User Stories are an excellent source of information for FP counting. User Stories represent the User perspective and are typically written in a way that describes the functionality required. So, “Can function points be counted/estimated from user stories?” Absolutely. “What level of granularity is required?” Any level can be used; however, as with any documentation used, the more detailed the User Story the more accurate the FP count.

Written by Default at 05:00
Categories :

Are Function Points Still Relevant?

Let's start with a quick overview of Function Point Analysis:

Function Point Analysis is a technique for measuring the functionality that is meaningful to a user, independent of technology.  It was invented by Allan Albrecht of IBM in 1979. Several standards exist in the industry, but the International Function Point Users Group (IFPUG) is the most widely used.  IFPUG produces the Function Point Counting Practices Manual, used by Certified Function Point Specialists (CFPS) to conduct function point counts.  IFPUG is one of the ISO standards for software sizing (ISO/IEC 28926:2009). 

Function Point Analysis considers five major components of an application or project: External Inputs, External Outputs, External Inquiries, Internal Logical Files and External Interface Files. The analyst evaluates the functional complexity of each component and assigns an unadjusted function point value. The analyst can also analyze the application against 14 general system characteristics to further refine the sizing and determine a final adjusted function point count.

Function Point Analysis

“The effective use of function points centers around three primary functions: estimation, benchmarking and identifying service-level measures.” [i] 

More and more organizations are adopting some form of Agile framework for application development and enhancement.  The most recent VisionOne State of Agile Survey reveals that 94% of organizations practice Agile.[ii]   Hot technologies such as big data, analytics, cloud computing, portlets and APIs are becoming ever more popular in the industry.

Let’s explore each of the three primary functions of function points and their relevance in today’s Agile dominated IT world and with new technologies.

Estimation:

Whether it is a move from traditional waterfall to Agile or from mainstream technologies to new innovations, project teams still have a responsibility to the business to deliver on time and within budget.  Estimates of the overall project spend and duration are critical for financial and business planning.

Parametric estimation is the use of statistical models, along with parameters that describe a project to derive cost and duration estimates.  These models use historical data to make predictions.  The key parameters necessary to describe a project are size, complexity and team experience.   Many other parameters can be used to further calibrate the estimate and increase its accuracy, including whether the project is using an Agile framework.  Several tools can be used to perform parametric estimation, including SEER, SLIM and COCOMO. 

Project size can be described in several ways, with software lines of code (SLOC) and function points being the most common.  SLOC has some inherent problems, one being that inefficient coding produces more lines of code, another being the fact that determining the SLOC size of a project before it is coded is itself an estimate.  That’s where function point analysis provides real value as a sizing tool.  Even in software developed using the latest innovations in technology, the five components of function point analysis still exist so function point counting remains a valuable tool for measuring software size.  Because a function point count can be done based on a requirements document or user stories, and the expected variance in function point counts between two certified function point analysts is between 5% and 10%, an accurate and consistent measure of the project size can be derived.  And because function point analysis is based on the users view and independent of technology it works just as well as technology evolves.

The function point size, along with the other parameters described above are then used by the parametric estimation tool to provide a range of cost and duration estimates for the entire project within a cone of uncertainty.  This information can be used for financial budgeting and business planning.  

Projects in an Agile framework can create estimates for the individual user stories with techniques like planning poker, t-shirt size or relative mass valuation.  These estimates are used for sprint planning and are refined through the backlog grooming process.  As the team measures and refines its velocity the estimates are further updated.   Ultimately all of these estimates should converge on the overall projected estimate created using parametric estimation.

Regardless of the technologies used for development, in this way estimates of the overall project through parametric estimation and Agile estimation techniques can coexist and complement each other in support of the business’s need for financial and business planning.

Benchmarking:

Whatever technology or development framework is being used, constant improvement is essential to an organizations ability to survive and thrive in a competitive environment.  Baselining an organization’s performance relative to productivity, quality and timeliness is the starting point for benchmarking and the first step toward an IT organization’s delivery improvement. 

Function points are a common currency for metrics equations.  They provide a consistent measure of the functionality delivered, allowing benchmark comparison of performance over time, of one technology against another, internally across various departments or vendors, and externally against the industry in which a company competes.  Benchmarking is also used in outsourcing governance models as a way to ensure a vendor is providing value with respect to contractual commitments and competitors in the marketplace.

A large amount of function point based industry benchmark data is available from many suppliers.  Some of the suppliers include: The Gartner Group, Rubin Systems Inc. META Group, Software Productivity Research, International Software Benchmarking Standards Group (ISBSG) and DCG Software Value. 

To execute a benchmark, data is collected for the target projects, including function point size, effort and duration.  The data is analyzed and functional metrics are created and baselined for the target projects.  Quantitative comparison of these baselines is done against suitable industry benchmarks.  Qualitative assessment is done to further analyze the target projects and determine contributing factors to performance differences with the benchmark.

Regardless of the development framework or technology used, function points is the basis for baselining and benchmarking an organization to determine their performance relative to the industry and allowing for improvements to move toward best-in-class performance.

Service-Level Measures:

Service-level metrics are most commonly used in outsourcing governance to measure the performance of the outsourcer to ensure contract compliance.  With IT’s increased alignment with the business, service-level metrics are increasingly used internally as well.  Delivery framework and technology don’t change the need for this kind of oversight. 

Outsourcing is typically done at the individual project or application level, for application maintenance, or the entire ADM environment.  Let’s examine each of these outsourcing models and how function point based service-level metrics can be used to monitor them.

Individual project or application:

In the case of individual project or application outsourcing service-level definition is based on the provider’s responsibility, the standards required by the customer and how success is defined.  Function point analysis has a role in all three of these areas. 

Definition of the outsourcer’s responsibilities helps identify the hand-off points.  Function point sizing at requirements hand-off provides an initial baseline of the project size for all metrics to be built upon.  As requirements change throughout the project the baseline can be updated through change control. 

The standards and development practices lead to establishment of compliance measures and targets for the outsourcer to meet.  Function point sizing can be used here as the basis of measures like productivity.

Success can be measured with function point based measures of delivery rate, duration and quality against contractual requirements or internal standards.

Application maintenance:

Measurement of maintenance in an outsourcing includes customer expectations, response time, defect repair, portfolio size, application expertise and others.  Let’s explore those that involve function point analysis.

Customer expectations can be thought of as the size of the portfolio being maintained, as well as the cost of maintaining it.  The portfolio size can be measured with function points to establish the maintenance baseline and its growth over time can be monitored. 

Support efficiency can be measured as the size of the support staff needed to maintain the maintenance baseline.  This can also be measured over time to show trends.

Entire ADM environment:

The measurement needs for ADM outsourcing are different from those of the previous two scenarios.  A multi-year outsourcing requires more complex measures to ensure the services provided by the outsourcer meet contractual commitments.  To do this more complex metrics dashboards are often built to allow a wide range of measurements to be analyzed. 

To build a metrics dashboard that provides the level of monitoring required, many factors must be considered including contractual requirements, end customer expectations and organizational standards and goals. 

The table below describes metrics derived from performance considerations and business drivers. [iii]

Function Points

Many of these metrics are based on functional size so function point analysis can be used to build the measurements.

For outsourcing and internal IT, effective measurement is critical to monitor performance and improvements and should be linked to the organizations goals and objectives.  Metrics based on functional size are key to a service-level measurement program without regard to the delivery framework or technology used.

Conclusion:

We have seen above that function point analysis is versatile and adaptable with changing technology and processes.  All technologies still have the five basis components of function point analysis and organizations are still asking “when it will be done?”, “how much will it cost?” and “what will I get?”.  It is for these reasons that function point analysis remains relevant in today’s IT world.

References

  1. Garmus, D. Herron, D., Function Point Analysis, Measurement Practices for Successful Projects, Addison-Wesley, 2001
  2. IFPUG Metrics View, February 2016, International Function Point Users Group
  3. 9th Annual State of Agile Survey, VersionOne Inc., 2015
Written by Default at 05:00

Fourth Volume of Trusted Advisor Anthology Now Available

Trusted Advisor

We're excited to announce the release of the fourth Trusted Advisor anthology!

As you know, every month we publish a Trusted Advisor report. We research and draft this report based on IT-related questions that are submitted by members of Trusted Advisor. This helps us to keep up with IT trends and issues plaguing those in the field - and it means you can spend your time working instead of looking for the answers to your problems. In essence, we do the research for you!

At the end of the year we package the reports, 12 in total, into an anthology, making it easy to have the research available at your fingertips.

The fourth edition of the book features reports written throughout 2015, such as:

  • Why Should I Have More Than One Technique for Retrospectives?
  • Our Software is Full of Bugs. What Can We Do About It?
  • Story Points or Function Points or Both?

Buy the Book

All the reports are individually available to download from our website. But, if you're interested in the full anthology of reports, it can be purchased on Amazon.

Join Trusted Advisor

Do you have a question you'd like to submit to our research team? Membership to Trusted Advisor is open to all IT professionals at no cost. Registration details, and more information about Trusted Advisor, is available here.

Written by Default at 05:00
Categories :

The Mathematical Value of Function Points

"Anything you need to quantify can be measured in some way this is superior to not measuring it
all." – Gilb’s Law(1).

To assess the value of function points (any variety), it is important to step back and address
two questions.  The first is “What are function points (in a macro sense)” and secondly “Why do we measure?”

Function points are a measure of the functional size of software. What are IFPUG Function
Points? IFPUG Function Points (there are several non-IFPUG variants) are a measure of the functionality delivered by the project or application. The measure is generated by counting features and functions of the project or application based on a set of rules. In this case, the rules for counting IFPUG Function Points are documented in the IFPUG Counting Practices Manual. Using the published rules, the measure of IFPUG Function Points is a consistent and repeatable proxy for size. Consistency and repeatability increase the usefulness of estimating and measurement. An analogy for the function point size of a project is the number of square feet of a house when building or renovating. Knowing the number of square feet provides one view of the house, but not the other attributes, such as the number of bedrooms. A project function point count is a measure of the function size a project while an application count is a measure of the functional size of the application. 

The question of why do we measure is more esoteric. The stated reasons for measuring often
include:

  • To measure performance,
  • To ensure our processes are efficient,
  • To provide input for managing,
  • To estimate,
  • To pass a CMMI appraisal,
  • To control specific behaviour, and
  • To predict the future.

Douglas Hubbard (2) summarizes the myriad reasons for measuring into three basic categories.  
1. Measure to satisfy a curiosity.
2. Measure to collect data that has external economic value (selling of data).
3.Measure in order to make a decision.

The final reason, to make a decision, is the crux of why measurement has value in most organizations. The decision is the driver to the value of counting function points. The requirements for making a decision are uncertainty (lack of complete knowledge), risk (a consequence of making the wrong decision) and a decision maker (someone to make the decision).  

The attribute of uncertainty is the direct reflection that there exists more than one possible outcome for a decision. Represent the measurement of uncertainty as a set of probabilities assigned to the possible outcomes. For example, there are two possibilities for the weather tomorrow, precipitation or no precipitation. The measurement of uncertainty might be expressed as a 60% chance of rain (from the statement we can infer a 40% chance of no rain). Define risk as the uncertainty that a loss or some other “bad thing” will occur.  In this case, the risk might be that we intend to go picnic if it does not rain and must spend $30 for food the day before that will perish if we can't go on the picnic.  
Measurement of risk is the quantification of the set of possibilities that combines the probability of occurrence with the quantified impact of an outcome.  We would express the risk as a 60% chance of rain tomorrow with a potential loss of $30 for the picnic lunch that won't be eaten. In simplest terms, we measure so we can reduce the risk of a negative outcome. In our picnic example, a measure would have value if it allows us to reduce the chance that we spend $30 for a picnic on a rainy day.

A simple framework hybridized from Hubbard’s How to Measure Anything or determining the value of counting function points to support decision making is:

  • Define the decision.
  • Determine what you already know (it may be sufficient).
  • Determine if knowing functional size will reduce uncertainty.
  • Compute the value of knowing functional size (or other additional information).
  • Count the function points if they have economic value.
  • Make the decision!

The Process and an Example:

1. Define the decision.

Function points provide useful information when making some types of decisions. Knowing the size of the software delivered or maintained would address the following questions:

  • How much effort will be required to deliver a set of functionality?
  • Given a potential staffing level, is a date or budget possible?
  • Given a required level of support, is staffing sufficient?

Summarizing the myriad uses of function points into four primary areas is useful for understanding where knowing size reduces uncertainty.

a) Estimation: Size is a partial predictor of effort or duration. Estimating projects is an important use of software size. Mathematically, the effort is a function of size, behaviour, and technical complexity. All parametric estimation tools, home-grown or commercial, require project size as one of the primary inputs. The simple parametric model that equates effort to size, behavior and complexity are an example of how knowing size reduces uncertainty.
b)Denominator: Size is a descriptor that is generally used to add interpretive information
to other attributes or as a tool to normalize other attributes. When used to normalize other measures or attributes, size is usually used as a denominator. Effort per function point is an example of using function points as the denominator. Using size as a denominator helps organizations make performance comparisons between projects of differing sizes. For example, if two projects discovered ten defects after implementation, which had better quality? The size of the delivered functionality would have to be factored into the discussion of quality.  
c) Reporting: Collect the measures needed to paint a picture of project performance,
progress or success. Leverage measurement data for Organizational report cards and performance comparisons. Use functional metrics as a denominator to synchronize many disparate measures to allow comparison and reporting.
d) Control: Understanding performance allows project managers, team leaders, and project team members to understand where they are in an overall project or piece of work and, therefore, take action to change the trajectory of the work. Knowledge allows the organization to control the flow of work in order to influence the delivery of functionality and value in a predictable and controlled manner.

2. Determine what you already know (it may be enough).

Based on the decision needs, the organization may have sufficient information to reduce
uncertainty and make the decision. For example, if a table update is made every month, takes 10 hours to build and test, then no additional information is needed to predict how much effort is needed to make the change next month. However, when asked to predict a release of a fixed but un-sized backlog, collect more data.

3. Determine if knowing functional size will reduce uncertainty.

Not all software development decisions will be improved by counting function points (at
least in their purest form). Function point counting for work that is technical in nature (hardware and platform related), non-functional in nature (changing the color of a screen) or an effort to correct defects rarely provides significant economic value.

4. Compute the value of knowing the functional size (or other additional information).

One approach to determining whether measurement will provide economic value is to calculate the expected opportunity loss. As a simple example assume a high profile $10M project, estimated to have a 50% chance of being on a budget (or below) and a 50% probability of being 20% over budget.  
In table form:

Mathematical Value of Function Points

The expected opportunity loss is $1M (50% * 2M, very similar to the concept of Weighted Shortest Job First used in SAFe®). In this simple example, if we had perfect information we could make a decision to avoid a $2M over budget scenario.  The expected value of perfect information is $2M. If counting function points and modeling the estimate improves the probability of meeting the budget to 75% then the expected opportunity loss is $500K (a 50% reduction).

5. Count the function points if there is economic value.

Assuming that the cost of the function point count and the estimate is less than the improvement in the opportunity loss, there is value to counting function points.  The same basic thought process is valid to determine whether to make any measure.

6. Make the decision!

Using the reduction of uncertainty make the decision. For example, if the function point count and estimate based on that count reduce our uncertainty that we can meet the estimate by 50% we would be more apt to decide to do the project and to worry less about the potential ramifications to our career. 

Conclusion

While the scenario used to illustrate the process is simple, the basic process can be used to evaluate the value of any measurement process. The difference in the expected gain and the expected value or the percentage not spent on measurement is the value of the function point count. Modeling techniques
such as Monte Carlo Analysis and calibrated estimates are useful to address more robust scenarios in addition to the use of historical data. Counting function points reduces the amount of uncertainty so that we can make better decisions. If this simple statement is true, we can measure the economic value of counting function points.

Sources

1. Demarco, Tom and Lister, Tim. Peopleware: Productive Projects and Teams (3rd Edition). 2013.
2. Hubbard, Douglas. How to Measure Anything. (Third Edition). 2014. Wiley. 

Download

The report can downloaded here.

Written by Default at 05:00

How Do I Calculate Estimates for Budget Deliverables on Agile Projects this Year?

Scope of this Report

This report discusses the tension between organizational need of budgetary data for planned Agile deliverables vs traditional project cost accounting. Agile project lean-budgeting best practices at the portfolio level are highlighted to illuminate the importance of estimating and budgeting as Agile scales in an organization. The Scaled Agile Framework (SAFe) portfolio and value steam levels, as presented in SAFe 4.0, provide the backdrop for this discussion.

Budgetary Needs of the Organization at Scale

Small to medium sized businesses with 100 or less developers organized to develop, enhance or maintain a small group of software products can account for the labor and material needed with straightforward software cost accounting methods. This is true whether they are using traditional waterfall methods or Agile methods such as Scrum because, typically, such businesses are small enough to ignore or work around the differences between the project perspective of waterfall and the product or perspective of Agile.

Larger organizations with hundreds to thousands of developers, organized to develop and maintain a portfolio of software products, are more challenged in software cost accounting and budgetary planning or estimating early in the planning cycle for a given year. Estimating and budgeting early presents challenges to Agile credibility as well as governance.

Software leaders in the trenches deal with day to day change while higher-up executive or management seek predictability and certainty. This contributes to preliminary rough order of magnitude estimates becoming memorialized as commitments instead of a work-in-process numbers to drive conversations and decision-making. This is driven by the financial reporting needs in the executive suite rather than the workflow optimizing needs of the development activity.

Financial Reporting Tensions Rise as Agile Scales

Public accounting standards guide CPAs and CFOs shaping financial information to quantify business-created value. That value can be in the form of a hard good, such as a car, or a soft good, such as software or music. You can consult the Financial Account Standards Board (FASB) repository of standards1 and statements on software cost accounting for the specifics but in general, you can either expense or capitalize your costs when developing software. Choosing when to do so is left up to the organization (and their reporting needs) as long as they can defend the decision per the standards and they are below a certain size. Consequently, initial budgetary estimates are important inputs into the process because they allow for the reporting professionals to partition or plan the expense vs capitalize decision based on schedule

In the traditional waterfall world of software development, it is conceptually easier to decide when to expense and when to capitalize as the diagram below illustrates.

Figure 1: Waterfall Project Stages

Waterfall Project Stages

A waterfall software project has a distinct beginning and end with clear ‘phases’ identifying activity versus value created. Code, integration and test produce actual ‘capital’ value. Requirements, design and maintenance are ‘expenses’ required to produce ‘capital’ value. Also, substantial waterfall projects are longer in term and length and involve more resources, both labor and material. Longer term this makes it seem “easier” to estimate and schedule reporting events.

Agile projects, with most using the Scrum method, are inherently different in structure and execution employing an incremental value creation model as illustrated in Figures 2 and 3 below. This is a cyclical model where short bursts of activity create immediately available value (shippable product increment). Multiple cycles, iterations or sprints are crafted together over time to create releases of software product. Estimates can be made as to how much time will be spent designing, coding, testing, etc, in Agile and the work of people like Even Leybourne
suggests that it is not difficult to deal with the CAPEX/OPEX distinction but auditors like to see timesheet records to support estimates of the amount of time spent, say, designing versus coding. Splitting time recording like this for Agile teams within a sprint is hard to the point of being impossible. DCG Clients use approaches similar to those described by Leybourne to deal with this issue with little friction but only after they have convinced their auditors.

Figure 2: Agile (Scrum) Sprint Cycle

Agile Sprint Cycle

Figure 3: Multiple Sprints to Produce a Release

Multiple Sprints to Produce a Release

Estimating within an Agile cycle is a real-time small-bore activity focused on small pieces of potential value that is more often called “planning” in the Agile context. These are not estimates that cover the beginning, middle and end of an Agile initiative (or project if you prefer that term). These real-time estimates, done with relative methods and unanchored to any financial or engineering reality are unsuitable for aggregation and turning into a budget. You cannot aggregate bottom-up estimates to a budget in Agile.

However top-down needs persist for Agile budgets and early estimates to support not only prioritization but also the expense vs capitalize allocation process. As Agile scales, so do the reporting demands. Furthermore these early estimates or budget must be as reliable and consistent as possible, be based on a repeatable, verifiable process to increase confidence and accurately represent the value creation activity. If you cannot bottom-up aggregate Agile estimates, what about top-down?

Epics, Story Points and Releases

Epics are the large, high-level stories that can usually be broken down into many smaller stories. Epics drive Agile development, from the top, to create business value in the form of shippable product. Epics form the contents of portfolio and product backlogs rather than sprint backlogs and as such they are the level of stories most familiar to executive and technical leadership. Despite being high-level, they are brief, concise and unelaborated just like normal stories and, hence, almost impervious to early estimation due to the relative lack of information.
The Agile method, at the team level, is designed to decompose Epics into smaller units such as user stories. These user stories are estimated using relative methods such as story point, t-shirt sizing and other techniques. The Agile team continuously elaborate these user stories, in each sprint or iteration, leveraging end-user involvement and team dynamics to create incremental value. Discovery of requirements is an intimate process conducted by the team and not the organization.

At some endpoint in the Agile development a Release is designated ready and complete for shipment to the marketplace. This all costs money so how does the organization decide to fund one or more Agile teams when initial Epics are hard to estimate?

Portfolio Level Value Creation and Reporting

The term portfolio is common in financial and investment conversation while programs is a term common in planning and organization. For example, Federal contractors, building large military systems, use the term programs to cover the planned series of events, projects and other activities, such as in the F-22 Raptor Stealth Fighter Program. When the methodologists at Scaled Agile, Inc. (SAI) constructed their Scaled Agile Framework (SAFe) approach, they recognized that “Program” is a valid term to organize multiple subordinate activities (within multiple Agile teams) and applied it to their lexicon2. While there are several different established approaches to scaling Agile, the SAI methodologists seem to have paid the most attention to high-level budget challenges so we will spend some time on their approach in the report.

Using “Portfolios” to group multiple Programs seemed a common sense next step because of the financial implications rising from larger scale activity in an Enterprise. Agile is driven at the team level to produce software that has value but the Enterprise is driven by financial considerations to fund, extract that value and report accurately along the way.

In the SAFe 4.0 Big Picture3, which illustrates the SAFe framework, the Portfolio level is well articulated and includes Strategic Themes and the constructs of Value Streams that are budgeted or funded. Epics are represented as children of Strategic Themes that guide Value Streams toward the larger aims of the Portfolio.
Consequently early budget estimates, good ones at that, are important to the Enterprise in order to decide on funding priorities and trigger strategic and tactical initiatives. But if Epics at the top are not suitable for early estimation and you cannot bottom-up aggregate, how do you estimate or budget at all?

Estimating Early Means Uncertainty

Let’s assume an existing organization, say a large healthcare insurer, has recently acquired a smaller, complementary company. Both company’s systems have to work together in the first few years to give the combined company time to either consolidate, merge or sunset the systems. Both companies employ Agile methods so it is decided to launch a strategic Agile initiative to create application program interfaces (APIs) that will make the two systems function as one. This has great value for the organization.

Let’s assume for our example that an integration working group made up of representatives of the two companies presents a high-level design outlining the, say, thirteen API’s presumed needed. Executive managements then asks for budget estimates (hardware and software) and a more specific schedule to begin the approval process.

If the organization, through their annual planning process, has already allocated or budgeted a total overall IT spend with some software component then the question is how much to set aside for this particular initiative with little information and lots of uncertainty?

Funding Value Streams Not Projects

The 13 APIs, when delivered by the Agile teams, will provide real value to the organization. The total spend to cover all of the teams for the time period needed will be governed by the organization’s fiduciary authorities. Depending on the size and scope of the functionality needed, you could conceivable have 13 separate Agile teams each working on an API. Traditional project cost accounting is challenged by this model.

In the SAFe® 4.0 framework, strategies of Lean-Agile Budgeting5 are described to address these challenges and the tensions discussed above. The takeaway from the strategies are simple, continue the fiduciary authorities’ traditional role of overall budgeting and spend reporting while empowering the Agile Teams to own content creation using the organizing construct of one or more Value Streams.

The Agile Teams, and implicitly the Agile method, are trusted to build the right things on a day-to-day, week-to-week basis within an overall approved budget. The traditional project cost-accounting methods, that seek command and control assurance, are replaced by a dynamic budgeting5 approach within the Value Stream.

Going back to our example, the 13 APIs could have a natural affinity or grouping and drive the association of 3 separate Value Streams as illustrated in Figure 4.

Figure 4: Value Stream Example

Value Stream Example

Each of these Value Streams would be funded from the annual allocated software spend, by the fiduciary authorities but how big do you make the allocation for API Group 2?

Anchoring Reality to Functionality

At this point the Integration workgroup has only two choices: Estimate by experience or estimate by functionality.

Estimating by experience can work if the right set of circumstances occur. The estimators involved are experts in the proposed work, there is a rich history of prior work where analogy analysis could be made, re-estimation is done by the same group and the technology is familiar and stable. When these factors do not exist, risk rises as to the quality and goodness of the estimate and future estimates.

Estimating by functionality means quantifying the proposed work using industry standard sizing methods and leveraging parametric or model based estimating4. This approach leverages historical repositories of industry-similar analogous projects to create estimates that include success vs failure probabilities (risk) profiles. The organization is borrowing the past experience of others to help them predict their future. When this method is used it increases confidence because of the statistical and mathematical nature of the process. The process is suitable for internal adoption as a repeatable and tunable method. Management overhead costs do increase when this method is adopted.

Whatever method an organization chooses, it should always be considered a starting point and a work-in process number and not memorialized.

Budgeting for Value

The answer to the question, “How do I calculate estimates for Agile budget deliverables this year?” is to define the value (stream) desired and estimate using your method of choice to define the portion of the overall software spend required.

As the budget is spent, within this API Group 2 Value Stream’s allocation, multiple deliverables (Releases) would be created and their individual allocations dynamically adjusted, as needed, by the team as content (Epics and derived user stories) are elaborated, understood and converted into working software.

Figure 5: Fund Value Streams not Projects or Releases

Fund Value Streams

One or more Releases will be created and shipped supporting the expense or capitalize decision and the dynamic budget changes within each release activity updates the overall budget, governed by the fiduciary authorities.

Conclusion

This lean-Agile budgeting approach relieves the organization from using traditional, command and control project cost accounting methods, which are challenged by the Agile method. This approach allows the fiduciary authorities to own what they should, the overall software spend, divided by value streams at the same time the content authorities, the Agile teams, own the day-to-day, week-to-week spending. This is a big step away from the past for many organizations but a good step forward to a more Agile organization.

Sources:
1. FASB http://www.fasb.org/summary/stsum86.shtml ; http://www.gasb.org/cs/ContentServer?c=Document_C&pagename=FASB%2FDocument_C%2FDocumentPage&cid=1176156442651;
2. Scaled Agile Framework, http://scaledAgileframework.com/glossary/#P
3. SAFe 4.0 Big Picture, http://scaledAgileframework.com/
4. International Society of Parametric Analysts, Parametric Estimating Handbook, http://www.galorath.com/images/uploads/ISPA_PEH_4th_ed_Final.pdf
5. SAFe® 4.0 Lean-Agile Budgeting, http://scaledAgileframework.com/budgets/

 

Download this report here.

Written by Default at 05:00

"It's frustrating that there are so many failed software projects when I know from personal experience that it's possible to do so much better - and we can help." 
- Mike Harris, DCG Owner

Subscribe to Our Newsletter
Join over 30,000 other subscribers. Subscribe to our newsletter today!