Abstract
Research exploring how to support decision-making has often used machine learning to automate or assist human decisions. We take an alternative approach for improving decision-making, using machine learning to help stakeholders surface ways to improve and make fairer decision-making processes. We created “Deliberating with AI”, a web tool that enables people to create and evaluate ML models in order to examine strengths and shortcomings of past decision-making and deliberate on how to improve future decisions. We apply this tool to a context of people selection, having stakeholders---decision makers (faculty) and decision subjects (students)---use the tool to improve graduate school admission decisions. Through our case study, we demonstrate how the stakeholders used the web tool to create ML models that they used as boundary objects to deliberate over organization decision-making practices. We share insights from our study to inform future research on stakeholder-centered participatory AI design and technology for organizational decision-making.
Create and evaluate ML models to examine strengths and shortcomings of past decision-making and deliberate on how to improve future decisions.
- Reflection: help people realize behaviors to sustain or change
- Deliberation: help people share perspectives and/or reach a common understanding
2.2 is all on stakeholder involvement in AI/ML design.
zhuValueSensitiveAlgorithmDesign2018
This paper clearly states, “intended to be used by an organization that has to reach consensus on criteria for a selection or allocation problem and has data about past decisions.” (Zhang et al., 2023, p. 5) This, I think, will be essential to our work as well. The problem space must not be too broad.
Questions for deliberation included in their figure:
- Is it desirable for your model to have a high accuracy, since it is trained on past decisions?
- Are there any feature weights that the model came up with that you did not expect or disagree with?
- Since the model is created using past decisions, what might this indicate about patterns of past decisions?