Tom Cagley the Only TMMi Accredited Assessor in the U.S.

TMMi Accredited Assessor

We're happy to announce that we officially have the only Test Maturity Model integration (TMMi) Accredited Assessor in the United States, Tom Cagley, our Vice President of Consulting.

In case you haven't noticed, we're passionate about the TMMi. There is no doubt that it is one of the most effective ways to improve software testing processes, leading to improved software quality and reduced risk. What this news means for you is that we're here and available to help you with a TMMi assessment - or even just to answer your questions.

Only an accredited assessor can perform a Test Maturity Model integration assessment. Assessments include a Gap Analysis to help evaluate how an organization functions against the model, identifying strengths, weaknesses and areas for improvement. To become a TMMi Accredited Assessor, a person must be a certified tester, take the TMMi Foundation training course and pass the examination. 

Tom has actually been an assessor for quite some time, and he's helped a number of companies here in the U.S. utilize the TMMi framework (you can read about one engagement here).

The TMMi framework was developed by the TMMi Foundation, a non-profit organization dedicated to improving test processes and practices. It is the de facto international standard to assess and improve test maturity, featuring independent best practices from more than 14 quality and test models.

DCG Software Value is a TMMi Accredited Supplier. More information about our TMMi services is available here.

Written by Default at 05:00
Categories :

Why Everyone Should Care About Software Value?

Michael D. HarrisIn the majority of organizations today, it would seem that those most involved in a project (of any kind) would be the ones that care the most about its value. In the case of software development initiatives, the development teams and development managers are entrenched in the day-to-day. However, they are so focused on meeting deadlines, remaining within budget and maximizing utilization, software value is often not even on their radar. Yes, they should care about it, but the other factors tend to shift their priorities.

So, if they aren’t focusing on software value, who is? Should it be the CIO’s responsibility? Should the heads of the business units who are driving the requirements for projects be overseeing them to ensure maximum flow of the software’s value? In a perfect world, it should be both IT management and the business units working together to determine the business value of the software, develop goals, communicate those goals, and to measure against those goals to maximize value throughout the development effort.

It doesn’t stop with IT and the business units. Executives, and, yes, even the board, should care about maximizing software asset value and the flow of software value. If the software asset pool is not continually enhanced with new software, its value will decrease. If any aspect of an organization is declining in value, the upper echelon should care!

If the business value of software is realized and communicated from the top down in an organization to those who can impact the flow of the software value on a daily basis, more educated tactical decisions can be made to maximize the value. Therefore, everyone can play an important part and should care about the business value of software.

What do you think? Should software value be on the minds of the IT team, business units, as well as the c-level?

Mike Harris
DCG President & CEO


Written by Michael D. Harris at 05:00
Categories :

How Do I Calculate Estimates for Budget Deliverables on Agile Projects this Year?

Scope of this Report

This report discusses the tension between organizational need of budgetary data for planned Agile deliverables vs traditional project cost accounting. Agile project lean-budgeting best practices at the portfolio level are highlighted to illuminate the importance of estimating and budgeting as Agile scales in an organization. The Scaled Agile Framework (SAFe) portfolio and value steam levels, as presented in SAFe 4.0, provide the backdrop for this discussion.

Budgetary Needs of the Organization at Scale

Small to medium sized businesses with 100 or less developers organized to develop, enhance or maintain a small group of software products can account for the labor and material needed with straightforward software cost accounting methods. This is true whether they are using traditional waterfall methods or Agile methods such as Scrum because, typically, such businesses are small enough to ignore or work around the differences between the project perspective of waterfall and the product or perspective of Agile.

Larger organizations with hundreds to thousands of developers, organized to develop and maintain a portfolio of software products, are more challenged in software cost accounting and budgetary planning or estimating early in the planning cycle for a given year. Estimating and budgeting early presents challenges to Agile credibility as well as governance.

Software leaders in the trenches deal with day to day change while higher-up executive or management seek predictability and certainty. This contributes to preliminary rough order of magnitude estimates becoming memorialized as commitments instead of a work-in-process numbers to drive conversations and decision-making. This is driven by the financial reporting needs in the executive suite rather than the workflow optimizing needs of the development activity.

Financial Reporting Tensions Rise as Agile Scales

Public accounting standards guide CPAs and CFOs shaping financial information to quantify business-created value. That value can be in the form of a hard good, such as a car, or a soft good, such as software or music. You can consult the Financial Account Standards Board (FASB) repository of standards1 and statements on software cost accounting for the specifics but in general, you can either expense or capitalize your costs when developing software. Choosing when to do so is left up to the organization (and their reporting needs) as long as they can defend the decision per the standards and they are below a certain size. Consequently, initial budgetary estimates are important inputs into the process because they allow for the reporting professionals to partition or plan the expense vs capitalize decision based on schedule

In the traditional waterfall world of software development, it is conceptually easier to decide when to expense and when to capitalize as the diagram below illustrates.

Figure 1: Waterfall Project Stages

Waterfall Project Stages

A waterfall software project has a distinct beginning and end with clear ‘phases’ identifying activity versus value created. Code, integration and test produce actual ‘capital’ value. Requirements, design and maintenance are ‘expenses’ required to produce ‘capital’ value. Also, substantial waterfall projects are longer in term and length and involve more resources, both labor and material. Longer term this makes it seem “easier” to estimate and schedule reporting events.

Agile projects, with most using the Scrum method, are inherently different in structure and execution employing an incremental value creation model as illustrated in Figures 2 and 3 below. This is a cyclical model where short bursts of activity create immediately available value (shippable product increment). Multiple cycles, iterations or sprints are crafted together over time to create releases of software product. Estimates can be made as to how much time will be spent designing, coding, testing, etc, in Agile and the work of people like Even Leybourne
suggests that it is not difficult to deal with the CAPEX/OPEX distinction but auditors like to see timesheet records to support estimates of the amount of time spent, say, designing versus coding. Splitting time recording like this for Agile teams within a sprint is hard to the point of being impossible. DCG Clients use approaches similar to those described by Leybourne to deal with this issue with little friction but only after they have convinced their auditors.

Figure 2: Agile (Scrum) Sprint Cycle

Agile Sprint Cycle

Figure 3: Multiple Sprints to Produce a Release

Multiple Sprints to Produce a Release

Estimating within an Agile cycle is a real-time small-bore activity focused on small pieces of potential value that is more often called “planning” in the Agile context. These are not estimates that cover the beginning, middle and end of an Agile initiative (or project if you prefer that term). These real-time estimates, done with relative methods and unanchored to any financial or engineering reality are unsuitable for aggregation and turning into a budget. You cannot aggregate bottom-up estimates to a budget in Agile.

However top-down needs persist for Agile budgets and early estimates to support not only prioritization but also the expense vs capitalize allocation process. As Agile scales, so do the reporting demands. Furthermore these early estimates or budget must be as reliable and consistent as possible, be based on a repeatable, verifiable process to increase confidence and accurately represent the value creation activity. If you cannot bottom-up aggregate Agile estimates, what about top-down?

Epics, Story Points and Releases

Epics are the large, high-level stories that can usually be broken down into many smaller stories. Epics drive Agile development, from the top, to create business value in the form of shippable product. Epics form the contents of portfolio and product backlogs rather than sprint backlogs and as such they are the level of stories most familiar to executive and technical leadership. Despite being high-level, they are brief, concise and unelaborated just like normal stories and, hence, almost impervious to early estimation due to the relative lack of information.
The Agile method, at the team level, is designed to decompose Epics into smaller units such as user stories. These user stories are estimated using relative methods such as story point, t-shirt sizing and other techniques. The Agile team continuously elaborate these user stories, in each sprint or iteration, leveraging end-user involvement and team dynamics to create incremental value. Discovery of requirements is an intimate process conducted by the team and not the organization.

At some endpoint in the Agile development a Release is designated ready and complete for shipment to the marketplace. This all costs money so how does the organization decide to fund one or more Agile teams when initial Epics are hard to estimate?

Portfolio Level Value Creation and Reporting

The term portfolio is common in financial and investment conversation while programs is a term common in planning and organization. For example, Federal contractors, building large military systems, use the term programs to cover the planned series of events, projects and other activities, such as in the F-22 Raptor Stealth Fighter Program. When the methodologists at Scaled Agile, Inc. (SAI) constructed their Scaled Agile Framework (SAFe) approach, they recognized that “Program” is a valid term to organize multiple subordinate activities (within multiple Agile teams) and applied it to their lexicon2. While there are several different established approaches to scaling Agile, the SAI methodologists seem to have paid the most attention to high-level budget challenges so we will spend some time on their approach in the report.

Using “Portfolios” to group multiple Programs seemed a common sense next step because of the financial implications rising from larger scale activity in an Enterprise. Agile is driven at the team level to produce software that has value but the Enterprise is driven by financial considerations to fund, extract that value and report accurately along the way.

In the SAFe 4.0 Big Picture3, which illustrates the SAFe framework, the Portfolio level is well articulated and includes Strategic Themes and the constructs of Value Streams that are budgeted or funded. Epics are represented as children of Strategic Themes that guide Value Streams toward the larger aims of the Portfolio.
Consequently early budget estimates, good ones at that, are important to the Enterprise in order to decide on funding priorities and trigger strategic and tactical initiatives. But if Epics at the top are not suitable for early estimation and you cannot bottom-up aggregate, how do you estimate or budget at all?

Estimating Early Means Uncertainty

Let’s assume an existing organization, say a large healthcare insurer, has recently acquired a smaller, complementary company. Both company’s systems have to work together in the first few years to give the combined company time to either consolidate, merge or sunset the systems. Both companies employ Agile methods so it is decided to launch a strategic Agile initiative to create application program interfaces (APIs) that will make the two systems function as one. This has great value for the organization.

Let’s assume for our example that an integration working group made up of representatives of the two companies presents a high-level design outlining the, say, thirteen API’s presumed needed. Executive managements then asks for budget estimates (hardware and software) and a more specific schedule to begin the approval process.

If the organization, through their annual planning process, has already allocated or budgeted a total overall IT spend with some software component then the question is how much to set aside for this particular initiative with little information and lots of uncertainty?

Funding Value Streams Not Projects

The 13 APIs, when delivered by the Agile teams, will provide real value to the organization. The total spend to cover all of the teams for the time period needed will be governed by the organization’s fiduciary authorities. Depending on the size and scope of the functionality needed, you could conceivable have 13 separate Agile teams each working on an API. Traditional project cost accounting is challenged by this model.

In the SAFe® 4.0 framework, strategies of Lean-Agile Budgeting5 are described to address these challenges and the tensions discussed above. The takeaway from the strategies are simple, continue the fiduciary authorities’ traditional role of overall budgeting and spend reporting while empowering the Agile Teams to own content creation using the organizing construct of one or more Value Streams.

The Agile Teams, and implicitly the Agile method, are trusted to build the right things on a day-to-day, week-to-week basis within an overall approved budget. The traditional project cost-accounting methods, that seek command and control assurance, are replaced by a dynamic budgeting5 approach within the Value Stream.

Going back to our example, the 13 APIs could have a natural affinity or grouping and drive the association of 3 separate Value Streams as illustrated in Figure 4.

Figure 4: Value Stream Example

Value Stream Example

Each of these Value Streams would be funded from the annual allocated software spend, by the fiduciary authorities but how big do you make the allocation for API Group 2?

Anchoring Reality to Functionality

At this point the Integration workgroup has only two choices: Estimate by experience or estimate by functionality.

Estimating by experience can work if the right set of circumstances occur. The estimators involved are experts in the proposed work, there is a rich history of prior work where analogy analysis could be made, re-estimation is done by the same group and the technology is familiar and stable. When these factors do not exist, risk rises as to the quality and goodness of the estimate and future estimates.

Estimating by functionality means quantifying the proposed work using industry standard sizing methods and leveraging parametric or model based estimating4. This approach leverages historical repositories of industry-similar analogous projects to create estimates that include success vs failure probabilities (risk) profiles. The organization is borrowing the past experience of others to help them predict their future. When this method is used it increases confidence because of the statistical and mathematical nature of the process. The process is suitable for internal adoption as a repeatable and tunable method. Management overhead costs do increase when this method is adopted.

Whatever method an organization chooses, it should always be considered a starting point and a work-in process number and not memorialized.

Budgeting for Value

The answer to the question, “How do I calculate estimates for Agile budget deliverables this year?” is to define the value (stream) desired and estimate using your method of choice to define the portion of the overall software spend required.

As the budget is spent, within this API Group 2 Value Stream’s allocation, multiple deliverables (Releases) would be created and their individual allocations dynamically adjusted, as needed, by the team as content (Epics and derived user stories) are elaborated, understood and converted into working software.

Figure 5: Fund Value Streams not Projects or Releases

Fund Value Streams

One or more Releases will be created and shipped supporting the expense or capitalize decision and the dynamic budget changes within each release activity updates the overall budget, governed by the fiduciary authorities.


This lean-Agile budgeting approach relieves the organization from using traditional, command and control project cost accounting methods, which are challenged by the Agile method. This approach allows the fiduciary authorities to own what they should, the overall software spend, divided by value streams at the same time the content authorities, the Agile teams, own the day-to-day, week-to-week spending. This is a big step away from the past for many organizations but a good step forward to a more Agile organization.

1. FASB ;;
2. Scaled Agile Framework,
3. SAFe 4.0 Big Picture,
4. International Society of Parametric Analysts, Parametric Estimating Handbook,
5. SAFe® 4.0 Lean-Agile Budgeting,


Download this report here.

Written by Default at 05:00

Test Maturity Model integration (TMMi): Definition and History

Tom Cagley“All models are wrong, but some are useful.” – George E. P. Box

Testing is a mechanism for affecting product quality. The definition is of quality is varied, ranging from precise (Crosby – “Conformance to requirements”) to meta-physical (Juran – “Quality is an attitude or state of mind”). Without a standard model of testing that codifies a definition, it is difficult to determine whether testing is affecting quality in a positive manner. The Test Maturity Model integration (TMMi®) is an independent test maturity model. A model provides a framework of the activities and processes that need to be addressed, rather than merely laying out a set of milestones or events that need to be followed explicitly.

The TMMi is a reference model representing an abstract framework of interlinked concepts based on expert opinions. The Wikipedia definition suggests that a reference model can be used as a communication vehicle for ideas and concepts among the members of the model’s community. The use of a model as a tool to define the boundaries of a community also amplifies its usefulness as a communication tool, as it defines the language the community uses to describe itself. Thus, the TMMi is a testing reference model, for the testing community, defining the boundaries of testing, the language of testing and a path for process improvement and assessment.

Many developers (and development managers) think of testing as a group of activities that occur at the end of coding. This flies in the face of software engineering practices that have been in use since the 1980s and the Agile tenant of integrating testing into the entire development process. The TMMi model explicitly details a framework in which testing is not an event or gate that has to be hurdled, but rather a set of activities that stretch across the development lifecycle (waterfall, iterative or Agile). The TMMi model extends the boundary of testing to the entire development process.

The model lays out a set five maturity levels and sixteen process areas, ranging from test environment to defect prevention. The model has a similar feel to the classic CMMI model. The TMMi, through its framework of maturity levels, process areas, practices and sub-practices, lays out best practices for testing that should be considered when developing testing practices. Like other reference models, the TMMi provides a framework but does not prescribe how any project or organization should do any of the practices or sub-practices. By not prescribing how practices are to be implemented, the TMMi can be used in any organization that includes testing. A framework that is neutral to lean, Agile or waterfall practices is a tool that can be molded by managers and practitioners to make testing more efficient and effective in almost any organization.

DCG is a TMMi Accredited Supplier, which means that we can walk you through the model and address all of your questions and concerns, as well as assist with TMMi assessments and appraisals. If you're interested in learning more about how the TMMi works, read this case study on how DCG helped one organization to apply the TMMi and improve its testing processes.

Tom Cagley
VP of Consulting, TMMi Accredited Assessor

Written by Tom Cagley at 05:00

Managing Business Value of Software as an Asset

Michael D. HarrisManaging software as an asset is critical for organizations that want to maximize the value of that software. There are a number of different ways that organizations (and individuals within those organizations) look at asset value when it comes to their software. These different viewpoints include: financial, software maintenance and portfolio management.

Those that look at software from the financial side, often the CFO and the accounting team, consider all software purchased from vendors or software that is developed internally to be a financial capital asset that is accrued and depreciated on the organization’s balance sheet. In today’s world, where more and more software is “rented,” the subscription fees are posted as expenses on the balance sheet. How a software asset is treated in the financial realm may significantly impact an organization’s decision to purchase software outright or license it from a vendor.

The CIO often looks at software from the software maintenance angle. The more software assets an organization has under one roof, the more there is to maintain. And, the more complex each software package is, the more likely there will be more problems that need to be fixed, significantly driving up maintenance costs. Much like the financial perspective, the software maintenance view comes down to dollars and cents. The more maintenance a software asset requires, the less value it will have. We know that every software solution is going to require some maintenance, but it is important to ensure that a software asset isn’t going to require so much maintenance that it becomes a negative value.

Portfolio management is the third perspective where software value is looked at on an individual basis and also collectively with the organization’s other software applications. Software is purchased (or rented) because it is going to provide benefits for the organization (i.e. productivity improvements, enhance customer service); however, in some situations, when that software is integrated with other applications, the value can increase exponentially. For example, when data can easily flow from one software solution to the next (i.e. an HR solution to a payroll application), it saves valuable time for staff and reduces the potential for errors. Therefore, when assessing the value of the various software assets, it is likely that the sum of the whole exceeds the sum of the parts.

All three viewpoints are valid and need to be considered when making software investment decisions and determining the actual value of a software asset for an organization.

Mike Harris
DCG President & CEO

Written by Michael D. Harris at 05:00
Categories :

"It's frustrating that there are so many failed software projects when I know from personal experience that it's possible to do so much better - and we can help." 
- Mike Harris, DCG Owner

Subscribe to Our Newsletter
Join over 30,000 other subscribers. Subscribe to our newsletter today!