Enhanced Survey Response Monitoring with the Qualtrics API
Researchers at Urban and elsewhere in policy, academia, and industry use the Qualtrics survey platform to build and deploy sophisticated web-based surveys without writing any code. Advanced users can even augment the platform’s point-and-click features with JavaScript and regular expressions to create custom question types, fine-tune response validation, and more.
Although Qualtrics is a powerful and effective tool for creating and distributing surveys, research teams may want more response-monitoring functionality than it offers out of the box. If teams want to meaningfully process incoming responses or cross-reference them with other data sources, for example, they will need to employ the Qualtrics API to create a custom process.
We built a custom solution, a response-tracking dashboard, using the qualtRics and Shiny R packages. qualtRics offers API-based functions for loading survey responses and metadata directly into R, and Shiny offers a user-friendly way to build interactive web applications. Together, these tools allow us to load incoming survey responses into R, process them, and display the results in an interactive dashboard.
Here, we introduce our demonstration dashboard and detail how other researchers can use qualtRics and Shiny to build survey data dashboards tailored to their project needs.
The demonstration dashboard
Our demonstration dashboard compares respondent demographics from incoming survey responses with American Community Survey (ACS) demographic estimates. This comparison allows research teams to adjust recruitment in real time to achieve a more representative final sample. If a certain population is underrepresented in responses to date, for example, the research team can prioritize recruiting more respondents from that community.
At a high level, the demo dashboard loads computer-generated data from an example Qualtrics survey, recodes race and ethnicity categories from the example survey into the ACS’s race and ethnicity categories, and compares the generated demographic data with 2020 ACS estimates for Washington, DC. (We chose DC for demonstration purposes, but ACS estimates are available for a wide range of geographies.) Let’s take a closer look at each of these steps.
First, the demo dashboard uses a qualtRics function to retrieve survey responses with the Qualtrics API and load those responses into R. Qualtrics offers two ways to authenticate calls to its API: API key and OAuth. We used key-based authentication for our dashboard because OAuth is not, as of writing, compatible with the qualtRics R package.
Next, the dashboard recodes race into ACS categories. This recoding is necessary because, while a survey may use different or more specific language than the ACS to talk about race, mapping respondent demographics to ACS categories allows us to assess a sample’s demographic representativeness via comparison with ACS demographic estimates.
Finally, the dashboard uses the tidycensus package together with the Census API to retrieve ACS data. The plot and data table that compare the recoded respondent demographics with ACS demographic estimates are what Shiny calls reactive output, because they automatically respond to user input. In this case, when a user clicks the “Refresh Qualtrics Data” button, it triggers a call to the Qualtrics API, which repopulates the dashboard with up-to-date response data.
Building a custom dashboard
While tracking respondent demographics against ACS demographics to inform recruitment efforts is one use case for a custom dashboard, other projects may call for something different. We created a template R script with code for the key feature of our dashboard, the “Refresh Qualtrics Data” button for updating response data, and placeholders for plots and tables to help other researchers get started with their own dashboards.
A Qualtrics account with API access is needed to track incoming responses, since the dashboard will update with new responses via calls to the Qualtrics API as described above. Users without API access, which is an additional paid feature, can still use the qualtRics and Shiny R packages to build interactive dashboards, like Urban’s Social Security reform explorer, from static datasets.
Whatever the purpose, qualtRics functions can help minimize the preprocessing necessary to get survey data dashboard-ready. The demo and template use the fetch_survey() function to call the Qualtrics API, but the package also offers a read_survey() function for loading manually exported survey data files neatly into R.
Once researchers have loaded, processed, and analyzed their data, they can reference Get Started with Shiny for help building their dashboards. To ensure users can interact with the dashboards — be it by refreshing data, specifying what outcomes to display, or something else — we recommend reading up on how to build applications that respond to user input.
Our demo dashboard shows one way of working with Qualtrics survey data in R, but the possibilities with qualtRics and Shiny are endless. We hope researchers will continue experimenting to see how these powerful packages can enhance survey research workflows.
-Francesca Vescia
