Overcoming Data Collection/Management Challenges

Challenges are an inevitable part of life. No matter what field or profession you work in, you are bound to encounter some sort of roadblocks. Same thing goes for conducting applied evaluations or program implementation. Over the years, we’ve found that many organizations experience similar problems in regards to data collection and measurement. We’ve highlighted the two most common problems we encounter specific to data collection and survey design along with some recommendations. 

Survey Design Pitfalls

Surveys are often a topic of much consternation and can become problematic if not crafted with the target audience in mind. We often see surveys that include too much content, are not tailored to the appropriate audience, include items that don’t address the evaluation questions, or have double-barreled questions that limit the interpretability.

To solve survey design issues: look at the survey in a more objective manner and ask yourself whether the questions are “need to know” vs. “nice to know.” Only keep the items that are truly necessary and will be used in the analysis – so as not to waste the survey respondent’s time. Consider removing any unnecessary identifying or demographic information that you don’t intend to us. Don’t just collect data because you think you’re supposed to. Also, keep your survey items simple and to the point so the respondent can focus on selecting the most appropriate response, and doesn’t spend time trying to interpret/understand the question. Additionally, simple, small tweaks to phrasing or rating scales can instantly transform a problem survey to a successful one! For example, try using a five-star rating scale if collecting data from youth audiences. Using more visually interesting content can make the survey process more appealing.

Staffing Issues/Limited Time for Data Management

We often hear agencies say they don’t have enough staff to do the work that is required. Specifically, data collection and implementation often fall to wayside with the other daily demands. However, often times the data monitoring and collection is the most critical part of the job to ensure all the hard work is being documented.

We have found that streamlining data collection tracking tools into one centralized database can help save time in the long run, as all data can be found in one location. Similarly, as discussed above, re-evaluating the data that is being collected might help to identify pieces that are not relevant and can stopped being tracked. While re-structuring existing practices might be time consuming at the beginning, coming up with a better system can save your staff time and energy when entering, analyzing, and reporting out data. We also find it helpful to keep a codebook or example entry in your data collection system to ensure your data is entered accurately and efficiently even when time is limited or when there is staff turnover. Need assistance streamlining your existing tools? Talk to you evaluator to brainstorm ways to set up a system that works best for your organization.