Simple method for calculating decision criteria weights

Improving human decision intelligence with Julia code

This is a technical posting using the Julia language.¹ The posting starts here on Substack. To finish reading this posting visit Team Public Health Technical Notes which is a Jupyterlab notebook hosted on GitHub.com.

1 Introduction

Human decision intelligence (HDI) is applying ethics, science, and technology to support team and individual decisions to solve problems, achieve objectives, and improve and innovate in the face of time constraints, uncertainty, and trade-offs. Think of human decision intelligence in the same way you might think about human emotional intelligence. A foundational component of HDI is decision quality — the six requirements of good decision making. At a minimum, a decision quality checklist (DQ) (Table 1) improves the quality of decisions at any stage of problem solving. A good decision is only as strong as its weakest link.

What should I read to improve my decision making?

2 Decision making

Hiring an employee, selecting a contractor, or ranking a set of proposals uses a common team approach. Team members usually rate the alternatives using pre-determined criteria that have been weighted based on importance. Alternatives with high scores on the most important criteria (ie, higher weight) will be ranked at or near the top.

To summarize:

1. develop criteria

2. weight criteria

3. rate alternatives using the weighted criteria.

4. rank the alternatives

Ideally, the criteria should be weighted without any knowledge of the alternatives. This is to prevent evaluators from biasing the criteria weights in favor of their favorite alternative.

In this blog posting I show how to weight criteria using a simple ranking method. In a future blog post, I will show how to apply weighted criteria to rank and select alternatives. This first step, weighting criteria, is very powerful and practical. We will use a trivial example to nail down the concepts.

Now, suppose we wish to buy a car and our choices are a Honda Civic, and Subaru Impreza, or Toyota Corolla. We have data on the following attributes: safety (S), mileage (M), reliability (R), color (C), price (P), and resale value (V). Table 2 summarizes the DQ requirements for buying our car.

3 Calculating criteria weights — the easy way

Group deliberative decision-making is cognitively exhausting. So, anything you can do to make the process easier will keep team members engaged. Do not let “perfection become the enemy of the good.” The easiest way to generate criteria weights from a team of evaluators is to use a rank ordinal method.²

Give evaluators small pieces of paper with one criterion printed on on each. If we have five criteria, they get five small pieces of paper. Have them rank them from top to bottom. Once they have ranked them, tape their ranking onto an 8.5in x 11in paper and hand to the facilitator. This ranking is entered into the computer for analysis (see below).

3.1 Ratio ordinal method in Julia

I will demonstrate this method using the Julia language. This method can also be implemented using R, Python, or Microsoft Excel.

For rating the cars we have six criteria (attributes) for which we need to calculate weights:

1. Color (C)

2. Mileage (M)

3. Price (P)

4. Reliability (R)

5. Safety (S)

6. Value, resale (V)

We have five evaluators that will rank the criteria based on their knowledge, experience, expertise, and wisdom. It is much better for them to rank the criteria independently and without thinking about specific cars, otherwise they may game (bias) the weighting.

Here are the steps:

1. Select a ranking method to calculate weights for a specific number of criteria, in this case we have six criteria. We will write and use a Julia function that implements the SR method from Danielson and Ekenberg, 2017.³

2. Have each evaluator independently rank the criteria.

3. Use Julia to calculate the final criteria weights. We will use the split-apply-combine workflow that I introduced in a previous blog post and book review.⁴

3.1.1 Step 1: The SR method for generating criteria weights

Here is the formula⁵ where 𝑁 is the number of criteria, and 𝑤_𝑖^𝑆𝑅 is the weight for the 𝑖𝑡ℎ criterion.

\(w_i^{SR} = \frac{1/i + \frac{N+1-i}{N}} {\sum_{j=1}^N \left(\frac{N+1-i}{N}\right)}\)

For this calculation I use the Julia Language. Julia is as simple to program as Python but with the speed of C++. These calculations can also be completed in R.

This is a technical posting using the Julia language. The posting starts here on Substack. To finish reading this posting visit Team Public Health Technical Notes which is a Jupyterlab notebook hosted on GitHub.com.

Footnotes

1. For scientific computing I love the Julia language (see my “My Journey from R to Julia“). To learn more visit https://julialang.org/ .

2. Danielson M, Ekenberg L. Trade-offs for ordinal ranking methods in multi-criteria decisions. In: Bajwa D, Koeszegi ST, Vetschera R, editors. Group decision and negotiation Theory, empirical evidence, and application [Internet]. Cham: Springer International Publishing; 2017. p. 16–27. Available from: https://doi.org/10.1007/978-3-319-52624-9_2 .

3. Danielson M, Ekenberg L. Trade-offs for ordinal ranking methods in multi-criteria decisions. 2017. p. 16–27. Available from: https://doi.org/10.1007/978-3-319-52624-9_2 .

4. Tomás Aragón. Julia for Data Analysis: A book review for population health data scientists. Team Public Health Technical Notes. Feb 20, 2023. Available from https://drtomasaragon.github.io/posts/2023-02-20-julia-for-data-analysis/.

5. The SR method was selected because it was the best performing.