A perception experiment with time-critical graphics animation on the World-Wide Web

Realistic precision and accuracy of online experiment platforms, web browsers, and devices

Due to increasing ease of use and ability to quickly collect large samples, online behavioural research is currently booming. With this popularity, it is important that researchers are aware of who online participants are, and what devices and software they use to access experiments. While it is somewhat obvious that these factors can impact data quality, the magnitude of the problem remains unclear. To understand how these characteristics impact experiment presentation and data quality, we performed a battery of automated tests on a number of realistic set-ups. We investigated how different web-building platforms (Gorilla v.20190828, jsPsych v6.0.5, Lab.js v19.1.0, and psychoJS/PsychoPy3 v3.1.5), browsers (Chrome, Edge, Firefox, and Safari), and operating systems (macOS and Windows 10) impact display time across 30 different frame durations for each software combination. We then employed a robot actuator in realistic set-ups to measure response recording across the aforementioned platforms, and between different keyboard types (desktop and integrated laptop). Finally, we analysed data from over 200,000 participants on their demographics, technology, and software to provide context to our findings. We found that modern web platforms provide reasonable accuracy and precision for display duration and manual response time, and that no single platform stands out as the best in all features and conditions. In addition, our online participant analysis shows what equipment they are likely to use.

Comparability, stability, and reliability of internet-based mental chronometry in domestic and laboratory settings

The internet-based assessment of response time (RT) and error rate (ERR) has recently become a well-validated alternative to traditional laboratory-based assessment, because methodological research has provided evidence for negligible setting- and setup-related differences in RT and ERR measures of central tendency. However, corresponding data on potential differences in the variability of such performance measures are still lacking, to date. Hence, the aim of this study was to conduct internet-based mental chronometry in both poorly standardized domestic and highly standardized laboratory environments and to compare the variabilities of the corresponding performance measures. Using the Millisecond Inquisit4Web software, 127 men and women completed three different RT-based cognitive paradigms (i.e., go/no-go, two-back, and number-letter). Each participant completed all paradigms in two environments (i.e., at home and in the laboratory), with a time lag of seven days and in a counterbalanced order. Mixed-effects modeling was employed to estimate the between-setting variability across a comprehensive set of performance measures, including conventional measures of central tendency (i.e., mean RT and ERR) and further measures characterizing the joint distribution of RT/ERR. The latter measures were estimated using the diffusion model. The results suggested negligible differences between the domestic and laboratory settings. Thus, this study provides novel evidence suggesting that the statistical power of internet-based mental chronometry is commonly not compromised by increased environmental variance. The within- and between-session reliabilities were in a satisfactory range-that is, comparable to performance measures collected offline in laboratory settings. In consequence, our results support the broad applicability, robustness, and cost efficiency of mental chronometry assessment using the internet.

Auditory capture of visual apparent motion, both laterally and looming

Traditional tests of multisensory stimuli typically support that vision dominates spatial judgments and audition dominates temporal ones. Here, we examine if unambiguous auditory spatial cues can capture ambiguous visual ones in judgments of direction of apparent motion. The visual motion judgments include both lateral movement and movement in depth, each when coupled with auditory stimuli moving at one of four rates. Experiment 1 tested lateral visual movement judgments (leftward vs rightward) coupled with auditory stimuli that moved laterally. Experiment 2 tested depth visual movement judgments (approaching vs receding) coupled with auditory stimuli that got louder or quieter. Results of Experiment 1 revealed and replicated an overall leftward motion bias, but with additional acoustic capture to experience visual movement away from the side on which sound initially occurred, and no effect of auditory motion speed. Results of Experiment 2 revealed and replicated an approaching motion bias, but with no effect of initial sound intensity, and an additional systematic capture effect of auditory motion speed. Faster changes in acoustic intensity produced larger visual motion capture consistent with the direction of acoustic intensity change. Findings of both experiments generalized over conditions of listening device (head phones vs speakers) and test-setting (Laboratory vs Web-based data-collection). The leftward and approaching motion bias results replicate previous research. Our principal new findings, the auditory motion capture effects, confirm the multisensory nature of dynamic spatial perception and support that extent of inter-sensory capture is a function of the relative reliability of spatial information acquired by each sensory modality.

Reaction time effects in lab- versus Web-based research: Experimental evidence

Although Web-based research is now commonplace, it continues to spur skepticism from reviewers and editors, especially whenever reaction times are of primary interest. Such persistent preconceptions are based on arguments referring to increased variation, the limits of certain software and technologies, and a noteworthy lack of comparisons (between Web and lab) in fully randomized experiments. To provide a critical test, participants were randomly assigned to complete a lexical decision task either (a) in the lab using standard experimental software (E-Prime), (b) in the lab using a browser-based version (written in HTML and JavaScript), or (c) via the Web using the same browser-based version. The classical word frequency effect was typical in size and corresponded to a very large effect in all three conditions. There was no indication that the Web- or browser-based data collection was in any way inferior. In fact, if anything, a larger effect was obtained in the browser-based conditions than in the condition relying on standard experimental software. No differences between Web and lab (within the browser-based conditions) could be observed, thus disconfirming any substantial influence of increased technical or situational variation. In summary, the present experiment contradicts the still common preconception that reaction time effects of only a few hundred milliseconds cannot be detected in Web experiments.

Human Research and Data Collection via the Internet

Advantages and disadvantages of Web and lab research are reviewed. Via the World Wide Web, one can efficiently recruit large, heterogeneous samples quickly, recruit specialized samples (people with rare characteristics), and standardize procedures, making studies easy to replicate. Alternative programming techniques (procedures for data collection) are compared, including client-side as opposed to server-side programming. Web studies have methodological problems; for example, higher rates of drop out and of repeated participation. Web studies must be thoroughly analyzed and tested before launching on-line. Many studies compared data obtained in Web versus lab. These two methods usually reach the same conclusions; however, there are significant differences between college students tested in the lab and people recruited and tested via the Internet. Reasons that Web researchers are enthusiastic about the potential of the new methods are discussed.

Psychology and the internet: a social ecological analysis

This paper proposes a research strategy based on a social ecological analysis of the Internet and its psychological impact as an option to generate original research to answer the follow ing question: What is the psycho-environmental meaning of the Internet? This paper has two objectives: first, to analyze Computer Mediated Communication (CMC) linked to the use of the Internet from a social ecological approach, and second, to propose some relationships among variables from a social ecological perspective, which can help to clarify the variability and magnitude of the psychosocial effect associated with the Internet. This article is divided into three sections. The first briefly describes the origins and development of the Internet. It identifies some technological features and specifies some of the cutting-edge breakthroughs that have facilitated its expansion. The second section proposes a conceptual scheme from the social ecological perspective, which analyzes the subject-environmental binomial associated with the Internet. It identifies the basic assumption, the conceptual richness, and possibilities for research on the Internet, using a social ecological approach. Finally, the last section describes the scope and limitations of this perspective, and discusses its heuristic utility for the development and consolidation of a new area in psychological research: "digital psychology or cyber psychology."

The impact of spatiotemporal sampling on time-to-contact judgments

When motion in the frontoparallel plane is temporally sampled, it is often perceived to be slower than its continuous counterpart. This finding stands in contrast to humans' ability to extrapolate and anticipate constant-velocity motion. We investigated whether this sampling bias generalizes to motion in the sagittal plane (i.e., objects approaching the observer). We employed a paradigm in which observers judged the arrival time of an oncoming object. We found detrimental effects of time sampling on both perceived time to contact and time to passage. Observers systematically overestimated the time it would take a frontally approaching object to intersect their eye plane. To rule out artifacts inherent in computer simulation, we replicated the experiment, using real objects. The bias persisted and proved to be robust across a large range of temporal and spatial variations. Energy and pooling mechanisms are discussed in an attempt to understand the effect.

An inaccurate-timing filter for reaction time measurement by JAVA applets implementing Internet-based experiments

JAVA applets are not accurate enough to measure reaction time unless precautions are taken. A response-filtering technique is introduced that identifies inaccurate measurements by testing whether the client-side JAVA implementation produces false measurements of a time interval of known length during the measured reaction time. Reaction times that indicate such inaccuracy are discarded to increase the reliability of the remaining data. Three studies provide illustrative data on the effectiveness of the new technique, using 11 different computer systems. Study 1 showed differences in the accuracy of different clients and higher variability of the obtained means with applets than with native programs. Study 2 showed sharply increased variability when additional loads were imposed on the clients during reaction time measurement. The response-filtering technique diminishes this variability. Study 3 indicated more accurate measurement, with the new technique being able to reveal even smaller differences in difficult technical conditions.

Presentation accuracy of Web animation methods

Several Web animation methods were independently assessed on fast and slow systems running two popular Web browsers under MacOS and Windows. The methods assessed included those requiring programming (Authorware, Java, Javascript/Jscript), browser extensions (Flash and Authorware), or neither (animated GIF). The number of raster scans that an image in an animation was presented for was counted. This was used as an estimate of the minimum presentation time for the image when the software was set to update the animation as quickly as possible. In a second condition, the image was set to be displayed for 100 msec, and differences between observed and expected presentations were used to assess accuracy. In general, all the methods except Java deteriorated as a function of the speed of the computer system, with the poorest temporal resolutions and greatest variability occurring on slower systems. For some animation methods, poor performance was dependent on browser, operating system, system speed, or combinations of these.

Millisecond timing on PCs and Macs

A real-time, object-oriented solution for displaying stimuli on Windows 95/98, MacOS and Linux platforms is presented. The program, written in C++, utilizes a special-purpose window class (GLWindow), OpenGL, and 32-bit graphics acceleration; it avoids display timing uncertainty by substituting the new window class for the default window code for each system. We report the outcome of tests for real-time capability across PC and Mac platforms running a variety of operating systems. The test program, which can be used as a shell for programming real-time experiments and testing specific processors, is available at http://www.cs.dal.ca/~macinnwj. We propose to provide researchers with a sense of the usefulness of our program, highlight the ability of many multitasking environments to achieve real time, as well as caution users about systems that may not achieve real time, even under optimal conditions.