Data Project Scoping

Intended audience: Social Impact Professionals

Identifying & Scoping an Equitable Data Project

At DataKind, we feel “a problem well-stated is half-solved” (Charles Kettering), which is why our teams spend a significant amount of time working with social impact organizations to understand their pain points, co-design an ideal end state, and identify the resources needed and available to create a data-driven solution. We call this process “scoping,” and find that scoping a project well is the most important element of a “data science for good” project.

Our scoping process is split into two primary stages: Discovery and Design. We go through each stage with a project partner in order to ensure that each element of the project is co-created. Scoping a project well means that our partners and DataKind teams have a clear understanding of the problem and the pathway to solving it using data science.

Discovery

Each potential project starts with the Discovery Stage – where DataKind and a prospective partner work together to identify potential ways we might collaborate. We understand data science might not be the only – or best – solution to a prospective partner’s needs, so this stage is all about mutual fact-finding, brainstorming, and mission alignment. You can use this article to understand what to expect if you are scoping a project with DataKind, or as a resource to scope data projects within your own organization.

At DataKind, we believe there are six components of a successful AI and data science for social impact project, and we look for all six during the Discovery Stage.

The six components of a successful data science for good project

What Do We Look for in a Partner?

While it’s important to note that not everything about what makes a great partner can be put into a rubric, we look for clear signs that indicate a partnership would be set up for mutual success. When discovering with a project partner, we ask ourselves a number of questions, including:

Is there a mutual desire to work together and is there buy-in for a partnership from both organizations?
Is the organization bought in to implement the results of a data science solution that has the potential to create meaningful impact?
Are the prospective partner’s mission and values aligned with DataKind’s goal of serving humanity?
Is the organization data mature enough to sustain a project output?
Does the prospective partner have staff with bandwidth to support the project?

Reflecting on these questions, teams sometimes find that it isn’t the right time to partner with DataKind. Continue on to find the resources in this Playbook and other DataKind capacity building programs, which exist to support you in completing an internal data project independently without necessarily working directly with a technical partner like DataKind. We hope to build the social sector’s capacity to use data science and AI through transparently sharing our best practices and lessons learned. Additionally, we will happily make an introduction to other data science support providers if you’re looking for partnership and we think they’re a better fit.

After Discovery, We Complete an Impact Map

One of our favorite tools for discovering a possible project’s mission alignment is what we call an Impact Map – a logic model in which we map potential project ideas to the organization’s processes and theory of change. These can be summarized in project statements that look something like this:

For example: “Riders for Health wants to optimize routes and schedules of medical samples collection using route information of its fleet of sample transport couriers so that all communities regardless of the remoteness of their location, are equitably served so that patients are offered timely care and treatment for HIV and TB and remote communities are safeguarded from being underserved.”

In the Discovery Stage, we often draft multiple project statements based on possible data science projects. The project scope will be refined and defined in the Design Stage.

Check out our discovery stage question bank for some questions to ask yourself as you begin to identify your project scope.

Prove Us Wrong – Critiquing Our Own Ideas!

Before moving to the Design Stage of a project, which requires several weeks to deep dive into the data and fully scope the project, we make sure to engage in significant internal review and reflection to ensure we’re setting our projects up to be ethical, useful, and intelligent. In this step of the Discovery Stage, we:

Consider the possible ethical implications of the project, asking questions about risks and the worst possible project outcomes
Assess that there’s no alternative or existing transferable solution with research on the landscape of the potential projects
Ensure the partner has data available now or knows of public data that will meet their requirements

If DataKind and our potential partner decide to move forward to a deeper shared understanding of a potential project, we move to the Design Stage of scoping. Data license and non-disclosure agreements provide a mutual understanding around sharing information in this transition.

Design

Our Design Stage often includes sharing data, ideas, programs, and outcomes. To ensure our partners are protected during this stage, and comfortable with sharing materials with us, the Design Stage kicks off with a mutual non-disclosure agreement and a data-sharing agreement. Once we’ve signed contracts and shared data, the deep design starts. In this stage, we dive into the data and incorporate human-centered design principles to pull together a coherent and detailed project design.

What’s in the Data? Completing a Detailed Data Audit

The Data Audit is how we determine what solution is possible. With data security requirements and a data management plan in place, we dive into the data to explore all the different end-state possibilities. With a deeper understanding of what’s realistic with the data we have, DataKind is able to:

Determine if the proposed project ideas are feasible
Refine the project statement
Define technical methodology

Ensuring Ethical and Responsible Project Design

There are many essential ethical elements to incorporate into our project design. We’re constantly learning and improving our processes around ethical data science for good project design, but some of the steps we always take to ensure ethical design include:

Conduct project risk and ethical assessments for each project idea
Evaluate possible data inclusion risks and create associated mitigation strategies
Define accountability to the communities affected by the work
Establish project-specific responsible data science ethical standards for analysis
Create a pathway for evaluating the ethical implications of any end products
Identify who should weigh in on project ideas as advisors and subject matter experts and incorporate their feedback

Please note this list is just a starting point and far from exhaustive.

Identifying a Clear Pathway to Success

In order to move forward, we go back to our Impact Map and the updated project statement that was summarized above. We work with our partner to develop associated success metrics for the project and agree on a plan for how we will gather these metrics and measure impact. We draft a pathway to adoption and sustainability plan for the organization post-project, checking that the project is realistically sustainable. We agree on any software and tools needed for the project to be successful and a path for implementation.

With these defined, we’re ready for all parties to agree on clearly defined deliverables and decide whether to pursue the project. A final services agreement confirms that both parties are on board with the action plan and ready to execute on the project!

What Do You Think?

All this said, we’re always refining and improving our processes, and this is what DataKind’s Playbook is all about. Share your feedback and ideas for how we can improve our scoping process to support Social Impact Organizations in the comments below.

Contributer(s): Rachel Wells, with help from Caitlin Augustin, Shanna Lee, Caroline Charrow, Sarah Lenet, Fotis Zapantis