Assessing Your Organization's Data Culture

• Review

  5 minutes

  Take a few minutes to review the following key concepts with the participants.

  • Data
  • Data for decision-making
  • Qualitative vs. quantitative data
  • Primary vs. secondary data
  • Dataset
  • Documentation
  • Data visualization
  • Stakeholders

  Pause to ask if anyone has questions so far.

  
  
  
  
  
  
  
  
  
  

• Introduction to Key Concepts

  10 minutes

  An important part of working with data is assessment. Understanding the importance of assessment and using data from your assessment is key to using data in decision-making.

  To begin, write the word assessment on the board. Ask the class if they have a definition in mind for this word. Then, provide the following definition:

  To assess something means to determine its merit, worth, or significance. At its most basic level, assessment answers the question: did it work? Adapted from Patton (2012)

  For organizational purposes, assessment describes critically looking at a program or policy by understanding the purpose of that program, what happened that was intended as a result of that program, how the program was implemented, and what outcomes were achieved. Then, that information and data that were collected during the process can be turned into items for future action or recommendations.

  Assessment answers the following questions:

  • What?

    What happens in the program? What services and experiences can the program offer? What outcomes and impacts result from the program? What happened that we didn’t anticipate?

  • So what?

    What do the findings mean? Why did the program results happen? What judgements can be made? Can we decide if the program was a success or failure?

  • Now what?

    What can we do with this information? What recommendations can we make? What improvements should be made?

• Providing Context

  25 minutes

  Tell the participants that assessment is something that we do in our everyday lives.

  Solicit the following responses:

  • What do the participants think makes a good football match?
  • What do the participants think makes a good student?
  • What do the participants think makes a good meal?

  Many people have different perspectives on what they perceive to be good or bad, and they assess things on a daily basis. When you ask the participants about their opinions on what makes up a good quality of something, you are assessing what concepts, ideas, or values are important.

  Provide the participants with one of the following two examples:

  1. Generic Example: A nonprofit in Uganda wanted to build a chicken coup and sell the eggs and chickens as a profit. However, once they built it, none of the chickens produced any eggs. When the chickens did not produce eggs, they asked themselves why did this happen? They then interviewed poultry experts in East Africa and found that the building they constructed for the chickens was too hot for them to lay any eggs. They used this information to draw a recommendation that the next building they build for the chickens should be structured to be cool enough for the chickens to lay eggs.
  2. Myanmar Specific Example: New Yangon Bus System (2017). Political and economic liberalization since 2010 has seen unprecedented population growth and activity in the nation’s largest city, Yangon. As a result, traffic congestion has increasingly become a major concern. In January of 2017, the NLD regional government reconfigured the city’s bus system, the primary means of transportation for the majority of the city’s population. They condensed the approximately 350 operating bus lines and over 4000 privately owned busses into a mere 71 lines and approximately 2000 busses. The idea was to improve congestion, safety, and environmental concerns; however, the first day of the new system in January was anything but improvement. Many individuals had to wait significantly longer for busses as many were full when they arrived at their bus stop. Many also discovered that their commute time increased significantly due to overcrowding and multiple transfers. In response, the Yangon regional government conducted surveys and studies on the faults and difficulties of the new bus system for the public. They found that they did not have enough busses for the public, people were unfamiliar with the routes, and many of the new routes were not as direct for passengers, resulting in longer commute times and multiple transfers. There was increasing public and political demand on the Yangon regional government to address these concerns and representatives from the regional government emphasize that they are making plans and drafting proposals to make changes based on survey data and public demand. (http://www.moi.gov.mm/moi:eng/?q=news/18/01/2017/id-9647) (http://frontiermyanmar.net/en/yangon-chief-minister-warns-bus-owners-over-ybs-failures)(http://www.mmtimes.com/index.php/business/24578-yangon-s-new-bus-system-struggles.html)

  Show the following image to the class:

  

  Apply the assessment cycle to the chicken coup story or the example of the bus system in Yangon.

  • Plan (use eggs the chickens lay for income) (create safer transportation options that result in less traffic congestion)
  • Do (build the chicken coup) (reconfigure the bus system)
  • Review (what happened)
  • Assess (why did it happen)

  Now, put the participants in groups of 2-3. Ask them to think of a time where they thought they had a good idea, but in practice the good idea they had failed. Have them share their experience with each other and how they assessed the situations. What data did they use to make those assessments? This does not have to be organization specific, but can be related to their everyday lives. If they struggle to think of a situation, provide some context. Is there an idea that they had or heard of like the chicken coup in Uganda or the new bus system in Yangon that failed in real life?

  When the participants are finished, ask for volunteers to share with the class. After each presentation is finished, apply the scenario to the assessment cycle.

  When the presentation is finished, have a discussion with the participants. Underscore the importance of using assessment for data-driven decision-making. Some guiding questions to ask include: Why is assessment important? What is the role of data used in assessment? How can that data you collected in the assessment be used for decision-making?

• Understanding Key Concepts

  10 minutes

  In assessing your own organizations, it is important to keep the vision and mission of your organization in mind. Ask the participants if they have heard of a vision/mission statement before. Then, provide the following definition (Adapted from Establishing Your Vision, Mission, and Strategy. Marshall Strategy, 2012):

  Vision statement: a postcard for how your organization sees the future.

  Mission statement: the role you want your organization to play in that future.

  As an analogy, provide the following image:

  

  Pass the image around or project it on the screen. Tell the participants that they can imagine their organization’s vision as the city they see in the picture. An organization’s mission is the path that they will take and the tools that they will use to get to that vision.

  For context, provide the following examples of mission/vision statements for organizations they might be familiar with. Feel free to use the generic examples, but if implementing this workshop in Myanmar, provide your own, locally specific examples for participants. Project these images on the screen or write them on the board.

  Generic Example: IBM

  Vision: A smarter planet.

  Mission: To invent, develop, and manufacture the most advanced information and technologies.

  Take time to discuss the mission with the participants. Some guiding questions include: how does IBM’s mission help them reach their vision? What tools will they use?

  Generic Example: BRAC

  Vision: A world free from all forms of exploitation and discrimination where everyone has the opportunity to realize their potential.

  Mission: Our mission is to empower people and communities in situations of poverty, illiteracy, disease, and social injustice.

  Take time to discuss the mission with the participants. Some guiding questions include: how does BRAC’s mission help them reach their vision? What tools will they use?

  Locally specific (Myanmar): MBAPF (Myanmar Book Aid and Preservation Foundation)

  Vision: To have a strong knowledge based society in Myanmar through libraries by transforming community libraries into community centres.

  Mission: To provide resources to build the capacity of libraries in Myanmar by bringing global library trends to Myanmar.

  Ask the class why a mission statement is important. Answer/discussion points: It helps to guide the work and decisions that are made. It also enables your specific project to use language that is meaningful or important to the broader goals of your organization.

  Ask the class to answer the following: My organization doesn’t have a mission statement – does this matter?

  Answer/discussion points: No. But, your organization and your specific projects likely have goals. These goals feed into an overall PURPOSE for why your organization exists, and that guides your actions.

• Application

  25 minutes

  Divide the class into groups of 2-3 and ask them to create their own vision/mission statements for their organization. Remind them to think of their vision statements as how they see the future that their organization helped build, and the mission as the path that they will take to get there.

  Some other guiding questions in crafting a mission statement include:

  • What is our cause?
  • What are our actions? What do we do? What do we provide?
  • What is our impact? Is it health? Stronger communities? Security? Opportunity?

  Have the participants write down their vision/mission statements in groups, and then ask for volunteers to share with the class. After each presentation, take time to ask others in the class how their vision/mission statement follows the picture analogy of the city and the road to get to the city.

  Debrief with the participants about why knowing your organization’s vision and mission statement is important in understanding your organization’s data culture, and how it can help you in data for decision-making. Some guiding questions include: Why do you need to know your organization’s vision/mission in order to make informed decisions about programming? How will knowing your organization’s mission/vision help you make these decisions? How can the vision/mission guide your organization in conducting assessments and collecting data?

  
  
  
  
  
  
  
  
  
  

• Linking Assessment and Stakeholder Analysis

  15 minutes

  Review the term “stakeholders”. Ask the participants for a definition, the remind them:

  Stakeholders: a person with an interest or concern in something. For example, who are the stakeholders in this class? Who are the stakeholders at a restaurant? Who are the stakeholders at a football match?

  Defining a problem in which to use data to make a decision is a crucial first step in data for decision-making. Another important piece of data for decision-making is understanding who your stakeholders are by performing a stakeholder analysis. Understanding all the actors and their relative influence over the problem will help you frame your plans within the existing government, civil society, academic, and other systems responsible for addressing a problem.

  When you think about your stakeholders, you should consider:

  • Communities: Individuals or groups who are affected by the problem.
  • Change agents: Individuals and groups directly working on the issue. These include community groups, NGOs, and others on-the-ground.
  • Support groups: Foundations, governments, NGOs, and others with resources to address the issue.
  • Policymakers: the people who can control government responses to the issue.

  Here, a locally-specific policy example would be helpful. If implementing this program in Myanmar, please feel free to use the example below.

  In 2015, Myanmar experienced severe flooding in many regions from heavy rainfall. Think about the various individuals and groups who could have been impacted and involved.

  Next, walk the class through who the stakeholders in the issue are. Some guiding questions: who are the communities? Who are the problem solvers? Who are the policy makers?

  The communities in this situation include (but are not limited to) the villages and populations affected by the flooding. The problem solvers include the aid and relief workers assisting affected populations. The policy makers include government officials and elected members of parliament.

  For each stakeholder, think through:

  • In what ways to they engage with the problem?
  • In what ways do their goals align with yours? In what ways are your interests in the issue opposing?
  • How much influence and power do they have to make change?
  • How do/can they use data to make decisions about the issue?
  • Where are the gaps in data that impact their decision-making? What would more/better data allow them to do?

  Before moving on, take the time to provide some space for discussion in order to link the key concepts of assessment, mission/vision, and stakeholder analysis together. Some guiding discussion questions include: How can knowing your organization’s mission/vision help guide your assessment process? How can conducting a stakeholder analysis help your assessment? How can frame your stakeholder analysis and assessments through the mission or your organization inform how you will use the results to make decisions?

  After discussion, dismiss the class for a ten-minute break.

• Introduction to Key Concepts

  15 minutes

  Welcome the class back and pause to answer any questions. Then, write “data producer” and “data consumer” on the board if possible, or display them on the projector.

  Underline the word “producer”. Ask the class what this word means. If the class is silent, feel free the guide them with the following questions: who makes the clothes they are wearing? Who cooks the meals in their household? Who makes their cell phone? These people are all producers.

  Then underline the word “consumer”. Ask the class what this word means. Again, if participants are slow to respond, guide them along by using the same examples. In the clothing example, who is wearing them? Who eats the meal that was cooked? Who uses the cell phone? These people are consumers.

  Then, connect this back to data for decision-making. State that much like in any marketplace, there are producers (those that make a good) and consumers (those that use the good). Offer the following definitions:

  • Data producer: A user or system that collects and/or provides data that are relevant to an organization.
  • Data consumer: A person or system that uses data.

  Many organizations are both data producers and consumers.

  Ask the class whether their organization is a data consumer, producer, or both. Ask for volunteers to explain how their organization produces or consumes data.

  For the participants whose organizations are data producers, ask them why they collect data? What can the data be used for? Who would want to use your data?

  For the participants whose organizations are data consumers, ask them why they use the data produced by the producing organizations? What are they using that data for? Who else might want that data?

  Discuss with participants potential sources of data. What kinds of things need to be considered when collecting data?

  Then, ask the participants what resources they might need in consuming data. Do they need human skills? Technology? The help of a decision-making authority?

  Finally, ask both the producers and consumers in the class what decisions they can make using the data that’s available to them at their organization. If the class is quiet, go back to some previous examples the participants offered. Some guiding questions include: How could that data be used to solve a problem? How could you use data available to you to make an internal decision at the organization? How could you use that data to make an external decision?

• Activity 2.1

  25 minutes

  Download Activity 2.1

  Adapted from: Serrat, Olivier (May 2010). The Critical Incident Technique. Knowledge Solutions, 86.

  Objectives:

  • Learn how to identify critical incidents at each participant’s organizations.
  • Analyze the incident and how the incident impacted their organization.
  • Review how data were collected, analyzed, and used throughout the critical incident.
  • Identify new ways to handle this incident in data for decision-making in the future.

  Materials Needed:

  • Paper
  • Pencils

  Introduction: (Use the following information to introduce and explain the activity to the class)

  Organizations are often challenged to identify and resolve both internal and external problems. Critical incidents should be highlighted because they are significant to the people that work at the organization, the organization’s programming, and the organization’s mission and vision.

  A critical incident is: an event that enables collecting direct observations of human behavior that have critical significance to participants. Think of this as a device that helps you to remember something particular about an event, and then use that particular memory to explain how the event, actions, or outcomes are representative of a broader concept.

  Example: Think back to the last book you read. What book was it? Why did you like the book? Where did you purchase the book?

  The critical incident in this instance is the last book that was read. By reflecting on that specific book, why it was chosen, and where it was purchased one can say whether this is the typical kind of book they read – and if it is not, they can then use the book as an explanation for why it differs from what they typically read.

  Remember that the word critical in this scenario means “important”, rather than in the negative sense. Therefore critical incidents are simply something really important that happened to an organization and can be called good or bad

  Both positive and negative critical incidents happen to almost every organization. After they happen, it is important to sit down and think through them. Why did this incident happen? How did it impact the organization? How will it affect our ability to continue to serve the community?

  In this activity, divide into groups of 2-3 participants. Then, think about a critical incident (good or bad) that happened at each organization that reflects something about that organization’s data culture. For example, was there an instance where the organization was asked to create a report using data that weren’t available to them? Or did the organization have to make a decision about what types of data to collect and what types of data not to collect?

  For each incident, think through the following questions and write down you answers:

  • What happened?
  • What were the events or circumstances that led to the critical incident?
  • What were the outcomes of the critical incident?
  • Did the incident improve a decrease organizational performance?
  • What could be the possible outcomes if behaviors remain unchanged?
  • What could be the possible outcomes if behaviors changed based on using lessons learned from that critical incident?
  • How can these lessons learned be used to make decisions to improve organizational performance based on data and evidence gathered?

  Once the participants have shared their stories, ask for volunteers to present to the class. After the presentations, debrief with the class and hold a discussion. What do they think their organization can gain from critical incidents in the future? What is the importance of data for decision making in these incidents? How can they teach others in their organizations about the importance of reviewing these incidents in order to gain insights about their organizations?

  Expand upon the incident to describe how this is typical, or atypical. Questions to ask include: How would this incident, and your response be useful to understand the broader topic that your organization or project works on? What lessons learned from this incident are applicable? What is missing from this incident that is more usual for your organization?

• Sharing Data

  10 minutes

  After the activity, take time to underscore to the participants the importance of self-reflection around data. Why do they think it is important to use self-reflection on evidence from critical incidents in their organization?

  Then, communicate to the participants that, just as self-reflection in their own organization is important, so are sharing these lessons learned from your organization with other organizations. This allows you to learn from each other’s successes and failures. Ask the participants why might this be important.

  This is a process known as knowledge sharing. Write the words “knowledge sharing” on the board or project them on the screen. Ask the participants what they think this means. Then, offer the following definition:

  Knowledge sharing: an activity through which knowledge is exchanged among people, friends, families, communities, or organizations. (Adapted from: Knowledge Sharing. (n.d.). In Wikipedia. Retrieved July 1, 2018, from https://en.wikipedia.org/wiki/Knowledge_sharing)

  Emphasize to the class that knowledge sharing helps create awareness among different organizations, helps facilitate faster solutions and improves response rates, can increase coordination, and can also provide ways for new ideas to be accepted and shared faster.

  Engaging with other organizations allows them to learn from each other. You can share approaches, methods, tools, or instruments with each other. You should try to be as open as possible about sharing your data, your analysis, and your conclusions from that analysis.

  Ask the participants, do they share knowledge in their everyday life? How does that impact the people around them? Does it help those people succeed? Then, link this back to their organization. Ask the class, how might one organization’s findings help others? If they are quiet or slow to provide a response, feel free to draw from the examples used in activity 2.1. How can the lessons learned from one organization help another?

  Remind the participants that it’s natural to be nervous sharing data. Criticism will come. But it’s important to move beyond that and realize the gains that you can make in sharing data.

  End with the point that when sharing data, privacy and security are important to consider. This course will not cover practices to protect privacy, but it is good for people to keep these issues in mind and obtain knowledge on them if they will be participating in sharing.

  
  
  
  
  
  
  
  
  
  

• Activity 2.2: Learning to Share Resources

  25 minutes

  Download Activity 2.2

  Data sources were created for the purpose of this activity

  Objectives:

  • Learn how to identify when your data alone cannot answer a specific research question or problem
  • Learn to identify and collaborate with institutions with relevant data
  • Understand why sharing data is important for decision making

  Materials Needed:

  • Research Questions (printed out)
  • Dataset printouts
  • Computer
  • Projector

  Introduction: (Use the following information to introduce and explain the activity to the class)

  Government and civil society institutions often collect data within a specific sector. For example, the Department of Education collects data on student demographics, student enrollment, and school facilities, whereas a health NGO might collect data on diseases and disabilities. Often, however, these institutions identify problems or research questions that require additional data from outside of the specific sector. For example, the Department of Labor might want to understand if unemployment is higher and average GDP lower in districts where there are more immigrants. In this instance, the Department of Labor wants to access or request information from the Department of Immigration. Collaboration and cooperation with regards to data between both government institutions and civil society organizations is important in ensuring that the data are available for making evidence-based and credible decisions.

  In this activity, participants should gather in six groups. Each group will be given a dataset and the relevant research question or problem. Each group will then identify what information they already have in their data to answer the question and what information they need. Groups then will find the respective organization with the relevant data.

  Participants should consider the following questions:

  • What data do you need to address the problem/answer the question?
  • What data do you have from your organization’s dataset?
  • What information are you missing?
  • How did you identify which organization had data you could use?
  • Which data did you use from the other organization and why?

  After the participants have completed the exercise, think about other situations in which you need to share data. Think about the data you organization collects or uses and which organizations you know who collect relevant data that they could collaborate with in the future. Brainstorm out loud as a group.

  See the annotated instructor guide for Activity 2.2 for more information

• Debrief

  5 minutes

  Review that today the class learned about the importance of assessment, understanding their organization’s mission/vision, how data are used in their organization, and the importance of sharing data. Remind the class of the following process of using data for decision-making:

  1. Identifying a problem or research question
  2. Assessing data available to you and your data needs
  3. Identify stakeholders
  4. Plan for data will be used, analyzed, and shared.

  State that number 4 will be part of tomorrow’s lesson. Then, ask the class that for homework each participant think of one dataset, table, or data visualization that they have at their own organization and can bring into class tomorrow.

  Take time to answer any questions, and then dismiss the class for the day.