Quantitative approaches to prioritize and evaluate product features
By Anna Shorina (aashorina@edu.hse.ru)
Introduction
Today's software companies are not often faced with a lack of good ideas. Rather, the failures of IT companies are related to the wrong choice of ideas to implement: they may be irrelevant to the market or cost the developers too much. The process of evaluating and prioritising features therefore plays a significant role in development.
However, after this conclusion, the question of how to do the evaluation and what to consider when prioritizing comes up. In Microsoft's book “Software Requirements” [1], the authors refer to Alan Davies, who advises to identify the following points when determining the value of features:
• The needs of the customers;
• The relative importance of requirements to the customers;
• The timing at which capabilities need to be delivered;
• Requirements that serve as predecessors for other requirements and other relationships among requirements;
• Which requirements must be implemented as a group;
• The cost to satisfy each requirement.
Main Problems in assessing and prioritizing features
The task of evaluating and prioritizing features is by no means an easy one: it can pose some problems. First of all, it is not always possible to obtain the data needed for analysis, and the data is often limited to the sample in which the study was conducted (this does not always give a complete picture of what is happening). Secondly, feature prioritization can be an opinions-based process. Secondly, feature prioritization can be an opinion-based process. Managers or developers decide on the importance of the features by making their own judgements. A third problem is assessing the value of features in their future work, especially for radically new developments. In order to remain competitive and develop a really marketable software product, these problems must be overcome as successfully as possible. Qualitative methods of evaluation do not seem to be the most effective in this regard, as they are inherently subjective and based on 'guesswork'. An alternative approach to evaluation would be to use quantitative methods (introducing mock-ups to collect what users find interesting).
Main quantitative approaches
According to the web's famous “periodic table of methods for evaluating and prioritizing features” [2], the most quantitative method is financial analysis based on calculating NPV, IRR and other financial indicators. This method identifies 4 main financial targets (New Revenue, Retained Revenue, Incremental Revenue, Cost Savings) and evaluates them in terms of costs and benefits over time. This approach is good because it does not really contain many evaluative concepts and values, but is more based on formal calculations. It is also clear and accessible for use in projects at different levels.
One of the most popular approaches is the Kano model, which performs a discrete analysis of respondents' answers, classifying all the features into 4 main categories: Must-Be, Performance, Attractive, Indifferent.
Two other well-known approaches, RICE and ICE, use simple formulas to quantify features. In the first method, the letters R I C E stand for Reach, Impact, Confidence and Effort respectively. And the features are prioritised by the value of the expression (R*I*C)/E. In the second method, the letters I C E stand for Impact, Confidence and Easy, respectively. And the features are prioritised by the value of the expression I*C*E. Although these approaches can nominally be considered quantitative, there is still the unresolved problem of a great deal of subjectivity in them. However, because they are not complex, they can be used in almost any level of company or project.
Another quantitative approach is to select the determinants, determine their importance and calculate the values for the features on the basis of these factor values. In this case, perhaps the most important task is to determine the weight of each factor in the overall system for evaluating features. At present, sophisticated machine learning is used to solve this problem, models are built on samples of these indicators, and classification and regression problems are solved. An example of the use of such an approach is described in “A feature selection approach based on sensitivity of RBFNs” [3], where the authors also prove the efficiency of using such an approach. However, the use of such methods raises the question of feasibility. Whether the costs of such an analysis are justified in comparison to the results. The answer is probably simple - it depends on the level of the company, the project, the number of features to be assessed and the information that can be used for the study. Undoubtedly, when you are developing a complex product such approaches would be preferable, but in the opposite case it is easier to use perhaps a qualitative assessment.
Conclusion
The issue of prioritisation of features is indeed one of the most important during the development of a software product. The use of both qualitative and quantitative methods to evaluate features is often hindered by the subjectivity of the process. However, in today's reality there are options to circumvent this problem quite well. The use of sophisticated methods is a solution to this problem, but there is still the question of their feasibility. Before choosing such methods, it is clearly worth assessing the time and cost benefits of using them.
1. Software Requirements, Microsoft, 3td edition.
2. https://www.career.pm/briefings/product-prioritization-techniques
3. “A feature selection approach based on sensitivity of RBFNNs” Xiaoqin Zenga,Zhilong Zhena,Jiasheng Hea, Lixin Han
5. “A quantitative approach for assessing the priorities of software quality requirements”, Xiaoqing Frank Liu
6. “Kano qualitative vs quantitative approaches: An assessment framework for products attributes analysis”, Maria Grazia Violante