Simple method for calculating decision criteria weights
Improving human decision intelligence with Julia code
This is a technical posting using the Julia language.1 The posting starts here on Substack. To finish reading this posting visit Team Public Health Technical Notes which is rendered with Quarto2 and hosted on GitHub Pages.3
1 Introduction
Human decision intelligence (HDI) is applying ethics, science, and technology to support team and individual decisions to solve problems, achieve objectives, and improve and innovate in the face of time constraints, uncertainty, and trade-offs. Think of human decision intelligence in the same way you might think about human emotional intelligence. A foundational component of HDI is decision quality — the six requirements of good decision making. At a minimum, a decision quality checklist (DQ) (Table 1) improves the quality of decisions at any stage of problem solving. A good decision is only as strong as its weakest link.
2 Decision making
Hiring an employee, selecting a contractor, or ranking a set of proposals uses a common team approach. Team members usually rate the alternatives using pre-determined criteria that have been weighted based on importance. Alternatives with high scores on the most important criteria (ie, higher weight) will be ranked at or near the top.
To summarize:
develop criteria
weight criteria
rate alternatives using the weighted criteria.
rank the alternatives
Ideally, the criteria should be weighted without any knowledge of the alternatives. This is to prevent evaluators from biasing the criteria weights in favor of their favorite alternative.
In this blog posting I show how to weight criteria using a simple ranking method. In a future blog post, I will show how to apply weighted criteria to rank and select alternatives. This first step, weighting criteria, is very powerful and practical. We will use a trivial example to nail down the concepts.
Now, suppose we wish to buy a car and our choices are a Honda Civic, and Subaru Impreza, or Toyota Corolla. We have data on the following attributes: safety (S), mileage (M), reliability (R), color (C), price (P), and resale value (V). Table 2 summarizes the DQ requirements for buying our car.
3 Calculating criteria weights — the easy way
Group deliberative decision-making is cognitively exhausting. So, anything you can do to make the process easier will keep team members engaged. Do not let “perfection become the enemy of the good.” The easiest way to generate criteria weights from a team of evaluators is to use a rank ordinal method.4
Give evaluators small pieces of paper with one criterion printed on on each. If we have five criteria, they get five small pieces of paper. Have them rank them from top to bottom. Once they have ranked them, tape their ranking onto an 8.5in x 11in paper and hand to the facilitator. This ranking is entered into the computer for analysis (see below).
3.1 Ratio ordinal method in Julia
I will demonstrate this method using the Julia language. This method can also be implemented using R, Python, or Microsoft Excel.
For rating the cars we have six criteria (attributes) for which we need to calculate weights:
Color (C)
Mileage (M)
Price (P)
Reliability (R)
Safety (S)
Value, resale (V)
We have five evaluators that will rank the criteria based on their knowledge, experience, expertise, and wisdom. It is much better for them to rank the criteria independently and without thinking about specific cars, otherwise they may game (bias) the weighting.
Here are the steps:
Select a ranking method to calculate weights for a specific number of criteria, in this case we have six criteria. We will write and use a Julia function that implements the SR method from Danielson and Ekenberg, 2017.5
Have each evaluator independently rank the criteria.
Use Julia to calculate the final criteria weights. We will use the split-apply-combine workflow that I introduced in a previous blog post and book review.6
3.1.1 Step 1: The SR method for generating criteria weights
Here is the formula7 where 𝑁 is the number of criteria, and 𝑤_𝑖^𝑆𝑅 is the weight for the 𝑖𝑡ℎ criterion.
For this calculation I use the Julia Language. Julia is as simple to program as Python but with the speed of C++. These calculations can also be completed in R.
This is a technical posting using the Julia language. The posting starts here on Substack. To finish reading this posting visit Team Public Health Technical Notes which is rendered with Quarto and hosted on GitHub Pages.
Footnotes
For scientific computing I love the Julia language (see my “My Journey from R to Julia“). To learn more visit https://julialang.org/ .
Quarto.org is an open-source scientific and technical publishing system. You can run Julia, Python, or R and create documents, articles, books, blogs, or websites.
GitHub Pages hosts static websites such as drtomasaragon.github.io which was rendered using Quarto. To learn more visit https://pages.github.com/ .
Danielson M, Ekenberg L. Trade-offs for ordinal ranking methods in multi-criteria decisions. In: Bajwa D, Koeszegi ST, Vetschera R, editors. Group decision and negotiation Theory, empirical evidence, and application [Internet]. Cham: Springer International Publishing; 2017. p. 16–27. Available from: https://doi.org/10.1007/978-3-319-52624-9_2 .
Danielson M, Ekenberg L. Trade-offs for ordinal ranking methods in multi-criteria decisions. 2017. p. 16–27. Available from: https://doi.org/10.1007/978-3-319-52624-9_2 .
Tomás Aragón. Julia for Data Analysis: A book review for population health data scientists. Team Public Health Technical Notes. Feb 20, 2023. Available from https://drtomasaragon.github.io/posts/2023-02-20-julia-for-data-analysis/.
The SR method was selected because it was the best performing.
Dr. Aragon - thanks for bringing awareness of the science of assuring the public's health with the Julia method.
Vivian Thomas RHIA CHDA CHPS CPHQ CDIP CPHIMS