Workshop: Prototyping Models

Background

The performance evaluation of machine learning models has historically been based on relatively narrow technical criteria. Arguments have been made to expand the scope of such assessments to include a broader range of metrics, including ethical ones (e.g., Mitchell et al., 2019).

Learning Objectives

By completing this activity, you will have learned how to analyze the risks related to a hypothetical machine-learning model for an envisioned public AI system.

Instructions

  1. Use the model you imagine would be at the core of your envisioned design.
  2. Take the attached canvas and put it on your board.
  3. Discuss the various sections and potential answers to the questions.
  4. For each answer, add at least one post-it note.
  5. If you can’t answer the question right now, answer it as a hypothesis instead, and make a note of what you need to do to get the answer you need.

Product

The result of this activity is a filled-out canvas on your board.

Follow-up

We will use the questions and issues you have identified by doing this exercise as the starting point for the plenary discussion later in the day.

Downloads