Workshop: Mapping Human Intervention at Design Time

Background

Contestability of AI systems can happen during use-time and design-time. In this workshop, we focus on the latter. How can we allow for adversarial participation from citizens in the design and development process of your responsible urban AI system? Design-time contestability is essential because it is proactive and forward-looking. Use-time contestability can only respond to harms once they have already happened.

Learning Objectives

After completing this activity, you will be able to create a map of an ML pipeline specific to a particular design challenge, and you will be able to include a contestability loop in this map that enables contestability at design time.

Instructions

You can use the attached diagram as a scaffold for this exercise by placing it on your board and working on top of it.

On your board, create a new area where you can map out your system’s ML pipeline in the form of a flowchart consisting of the following steps:

  1. Problem definition: What specific task should the model accomplish?
  2. Data wrangling: What data is collected? Where does it come from? What metrics are used to evaluate data quality?
  3. Modeling: What type of model is used? What metrics are used to assess model quality?
  4. Deployment: Where does the model go? What more extensive system does it become a part of? In other words, How is it used in the real world?
  5. Feedback: What signals determine the model’s real-world performance? What data is collected to improve its performance

Next, add a section to the flowchart that captures what efforts are made to make things transparent:

  1. Technical transparency: Which critical elements of the dataset and model must be made public to enable contestability?
  2. Procedural transparency: Which critical decisions (made by system designers & and developers) need to be made public to enable contestability?
  3. Connect these elements with the steps of your pipeline.

Finally, add a section that captures the participatory methods put in place to enable contestability:

  1. Stakeholders: Which parties should be part of this participatory process? Do citizens need to participate directly, or will you invite civil society organizations to represent citizens? How will you select citizens or representative bodies?
  2. Problem: Which aspect(s) of the ML pipeline should these stakeholders participate in? Why?
  3. Process: What rules and structures should govern the participatory process?
  4. Evaluation: How will you determine whether the participatory method achieves its goals?

Product

Upon completing this exercise, you will have a map of your system’s machine-learning pipeline on the board, including a design-time contestability loop.

Follow-up

In the plenary discussion, we will discuss your findings and experiences with this workshop.

Downloads