DataKind logo logo Return to homepage

Assessing Data Maturity

Intended audience: DataKind Volunteers

Assessing an organization’s data maturity - broadly, its ability to collect and use data, internally and externally - is an important step when determining both the fit of a potential project and the organization’s capacity to use and sustain a data science solution.

There is no specific data maturity “level” that an organization needs to be at to work with DataKind. They simply must have sufficient familiarity with data to understand and maintain the project output, make continuous improvements, or even build a data pipeline to incrementally update the data to sustain a solution. When scoping, evaluate each of these elements to assess whether the organization can (1) actually use whatever DataKind builds, and (2) leverage existing resources (e.g., technical, operational, financial, human, etc.) to manage the continuation of the DataKind solution.

Step one: Provide an online data maturity assessment to the partner organization

To save time in conversations, we’ve created a brief survey template for you to share with the organization before setting up a discovery call - be sure to copy the template into your own folder and share the link to your version, so you can access the responses. This survey should take under 10 minutes to complete, and provide valuable information that will complement the qualitative assessment of the organization’s data maturity that you would typically conduct on your first call. We recommend that the Project Champion should complete the survey. If certain questions are outside the scope of this person’s work (e.g., technical questions), we encourage them to seek confirmation from another team member who may have more familiarity with the organization’s data and infrastructure.

Step two: Conduct a qualitative assessment on the organization’s data maturity during the Discovery Call

Qualitatively assessing the organization’s data maturity during a discovery call gives you an opportunity to ask specific questions to the Project Champion and other team members, as well validate any responses uncovered during the survey. Feel free to embed the relevant data maturity questions below into the existing calls.

While many choose to assess data maturity by asking followup questions during a discovery call, others have found it more helpful to use structured tools during intentional conversations with a variety of stakeholders. This set of questions that DataKind UK uses is one such example. However you decide to gather the qualitative data, be conversational and don’t be afraid to change topics with the natural flow of the discussion! There’s no right or wrong way, and it is important you make the organization feel you are seeking to understand their current orientation to data, rather than to interview and judge them.

Questions on data maturity could include, but are not limited to, the organization’s:

Knowledge and skills:

  • Do staff demonstrate skills in data literacy and general analytics-related tasks?
  • Are there specific skilled analytics roles (e.g., reporting, evaluation, data engineering, data science, etc.) within the organization?
  • Do staff have a general understanding of data and the ways it can be used to make decisions across the organization?

Data and infrastructure:

  • Does the organization have access to a central data repository (e.g., relational database)?
  • Do they understand how the data was acquired and is currently being stored? Is there documentation or a data dictionary?
  • Does the organization monitor data for its quality? How?

Breadth of data use cases:

  • How does the organization currently use data? Is it for descriptive purposes (e.g., reporting, evaluation, etc.), or more advanced applications using predictive and inferential methodologies?
  • Does the organization learn from data to improve processes?

Leadership and culture:

  • Is there evidence that leadership promotes a data-driven culture?
  • Does the organization and leadership team engage in efforts that demonstrate the value of data?
  • Do they use data to make decisions, and make appropriate investments in resources to do so?

Additionally, you can provide the partner organization with a link to the external version of DataKind’s and Data Orchards Data Maturity Framework in advance, so they know the kinds of questions and areas that will be covered during the call.

What should you do with the results?

Once you have the results from the online assessment and notes from your Discovery Calls, reflect on the partner’s answers and the project idea they are exploring to decide if it’s a good fit. Here are three scenarios you may find yourself in.

1. The organization does not demonstrate a sufficient level of data maturity to build or sustain a DataKind project solution.

Unfortunately, DataKind is only able to impactfully engage with projects that can be supported before, during, and after their execution. Data may not be available, data infrastructure may not exist, the organization may not have the right skillset to sustain the solution, and much more. In these situations, it is best to provide recommendations to the organization on potential next steps that might help them increase their data maturity, so that they might someday be ready to do a project with DataKind. Give the organization an understanding of a long term goal they could work towards and what might be possible with data science, as well as next steps that might help them on that data journey.

Example: An education nonprofit working with US high school students is interested in applying data science and machine learning to help evaluate their program effectiveness and identify pathways to better reach students. However, during a preliminary discovery call, the nonprofit shared that it does not have a central database with data from all the state affiliates. On the call, the Director of Evaluation also shared that it is a challenge for teachers at the nearly 2,000 affiliate schools to regularly and adequately provide reporting metrics, thus rendering some data inaccurate or missing. The sparsity of data, lack of data infrastructure, and nascent data mature culture would present challenges when executing a data science project. Since a data science project would not be recommended, the DataKinders instead provided resources to help the nonprofit get on the path toward creating a database and using data to make decisions.

2. The organization appears to demonstrate a minimal level of data maturity to sustain the potential project and solution. However, there are one or more risks that must be mitigated. Proceed with caution and keep this in mind during additional scoping.

In many cases, determining whether the organization’s data maturity could sustain a DataKind solution is nuanced and requires additional considerations. For example, you may find that the organization appears to exhibit a sufficient level of data maturity that could sustain a potential project - say, the existence of a reputable open dataset that could be used to build a predictive model. While access to data is important, be sure to also consider additional elements, like the organization’s ability to update the model with new data, interpret the model output, or even implement the model and tool within the team’s processes. In these situations, move forward with scoping while staying mindful of the potential limitations as you learn more about the scope and requirements to sustain the potential project in the Design Stage.

Example: A very large nonprofit proposes to collaborate with DataKind to develop a centralized data infrastructure with a goal of standardizing their data collection capabilities, as well as begin predicting key areas of need. While DataKind believes it can offer support on building such predictive models, they are uncertain if the current infrastructure and data practices could support the solution in the short-term. That is, previous data collection at the organization has not been standardized, such that it may take a year or more for sufficient data to be collected. In the interim, DataKinders have identified several open data sources that could complement existing internal data sources. DataKind will continue refining the project and scope, as well as mitigate any risks through a Data Audit. The team proceeds to the Design Stage, but will keep the data maturity level of the organization in mind and carefully continue to evaluate before deciding to move to the Prepare Stage.

If you are confident that the organization demonstrates a sufficient level of data maturity to support and sustain a potential project, you are ready to continue scoping! As a reminder, continue to identify and document potential risks, including those that you will further explore in the Design Stage and Data Audit.

Example: A global nonprofit organization working in microfinance and digital technology believes that data science and machine learning approaches would help make more informed decisions and personalize their service offerings to support marginalized populations. During scoping, DataKind confirmed that (1) the organization had access to local partners who could provide high quality data; and (2) the Project Champion and supporting team members understood the data and how the model would fit within their business practices. The DataKinders are confident that the organization’s data maturity level could support and sustain a solution, and therefore agree to continue refining and assessing the project and available data.

Additional Resources

Want to learn more about data maturity? Check out the resources below:

Contributer(s): Mitali Ayyangar, Benjamin Kinsella, Caroline Charrow, Emily Yelverton, Dulcie Vousden

Contact us

If you would like to learn more about us, partner with us, or get in touch, email us at community@datakind.org

Subscribe to our newsletter
Subscribe