A  A

AD·VNVM·DATVM Down to a single bit of data : Latest Post Slides for a Presentation on "Non-Psychometric" Survey Design

Posted in || , , , , 2 min. to read

In short:

In August 2017, I gave a new presentation on "non-psychometric" survey design. I am releasing the slides for that presentation.

Please see my note about the scope of this website.

On August 31st, I gave a new presentation at the Wharton School for University of Pennsylvania staff. I called the presentation, "Some Thoughts on Non-Psychometric Survey Design, Or, Creating Surveys that Measure what you Actually Want to Measure." By "non-psychometric," I explained to the attendees, I meant survey design that isn't going to include validation steps or other psychometrics Psychometrics is the subfield of Psychology (and especially Personality Psychology) that deals with measure creation. tools -- put differently, survey design for authors who are not interested in making a "validated" measure, but who would still like to take lessons from professional psychometricians.

I'm making my slides and presenter notes available here, with animations and without animations.

The presentation's abstract is below:

Title: Creating Surveys that Measure what you Actually Want to Measure

People are often poor participants in research. They are inconsistent (e.g., reading questions differently than a survey author intended), complex (e.g., refusing to distill their thoughts about a topic into the one specific aspect a researcher is trying to measure), and biased (e.g., answering either to impress or to spite a researcher).

Unfortunately, people are also often poor authors of research surveys. They write items that are inconsistent (e.g., worded differently throughout a survey in a way that fatigues respondents), complex (e.g., asking multiple questions as part of a single item), and biased (e.g., leading a participant toward a particular response).

This workshop will address the latter in the hope of minimizing the effects of the former.

Topics we will cover:

  • Why people are bad at answering survey questions (impression management, being affected by moment-to-moment circumstances, different people having different baseline responses, and more!)
    • Preventing bias
    • Putting oneself in the shoes of the participant
    • Optimizing vs. satisficing
    • Thinking about test-retest stability
    • Acquiescence (https://www.qualtrics.com/blog/avoiding-the-yes-bias/)
    • Types of responses (covering topics like this: https://www.qualtrics.com/blog/optimal-scale-labels-why-they-matter/)
    • What does "measuring" psychological traits mean? What is a "valid" measure?
    • What makes for a "good" survey item?
    • Creating surveys using Qualtrics (https://upenn.qualtrics.com) (High-level overview of Qualtrics' interface and functionality)
    • (If we have time and there's interest) Advanced features of Qualtrics, which could include:
      • Complex survey flows
      • Using Qualtrics' API (using as an example some code I released several years ago for showing users customized graphs of their responses vs. the average response)

More Posts:

  1. Tag Handwritten Notes to Link them with Typed Notes // After four years of figuring it out and two years of testing it, I have a small tip for incorporating handwritten and typed project notes.
  2. An Introduction to the "Impost" Book Scanner, using Voussoir // Following several design principles of my "Non-Destructive Guillotine", I introduce a more portable single-camera book scanner design. The scanner uses a hinged platen that works with tightly-bound and paperback books, while allowing the book to sit spine-down at a variable angle. I also introduce a piece of software, "Voussoir," which semi-automates the process of de-keystoning and cropping images.
  3. The University of Pennsylvania has Issued a Press Release about my Appointment as Bollinger Fellow in Library Innovation // The University of Pennsylvania has issued a Press Release about the Bollinger Fellowship that I've started this month.
  4. I am a PhD Graduand // I have successfully defended my dissertation in Psychology at the University of Oregon, and am a PhD graduand (i.e., awaiting the conferral of the degree). My dissertation and its code and supporting files are freely available.
  5. I am Excited to be Working through CLIR in the University of Pennsylvania's Libraries // I am very excited to announce that I will be beginning work in Philadelphia at the University of Pennsylvania in January 2017 as the University Libraries' Bollinger Fellow in Library Innovation through the Council on Library and Information Resources (CLIR).
  6. An Introduction to the "Non-Destructive Guillotine" ("NDG") Book Scanner // I introduce a new portable, single-camera book scanner design. The scanner is small enough to fit into a backpack when disassembled, and can be re-assembled in under 10 minutes. It does not require any power tools to construct; the only required tools are a tape measure, a PVC cutting tool, and scissors. A full list of parts, sizes, and prices are below, along with a video explaining the scanner's construction and use.
  7. A Workflow for Using Plaintext Notes with Zotero // I've created an "export translator" and a CSL file to facilitate creating and using plaintext notes associated with citations in the cross-platform reference manager Zotero. The CSL file should also be compatible with Mendeley and other programs that use the CSL standard.
  8. A Video of 'Time Travel for Academics,' an Introduction to Git, from the University of Oregon // I've posted a video of a one-hour introduction to version control with Git for academics that I presented at the University of Oregon.
  9. Video Privacy Enhancer Plugin Updated // The Video Privacy Enhancer Plugin that I wrote for Pelican has been updated to allow embedding videos from Vimeo.