Using matlab for online experiments (visual perception)

We use matlab to run visual attention experiments in the lab but, of course, we are locked out of the lab. How would we take our code and put it on an online platform for remote participants to run?

[darova]

I use this sometimes

[Raymond MacNeil]

Most online platforms for hosting experiments seem to require a conversion to JavaScript. If you're able to convert your MATLAB code to Python, you can use pavlovia.org, which does this automatically apparently (converts the code to JS).

Other than the suggestion made by darova, it could be possible to implement something with MATLAB drive. If you figure something out, please let us know. My lab would be interested to know, as we primarily use Psychtoolbox. Our department recently purchased a license for Pavlovia, so most folks are going that route.

[Walter Roberson]

I would worry about the latency of timing of anything running server-side.

A javascript approach hints that the code might be sent to the client to execute. The question when then become whether javascript can do timing accurately enough for experimental purposes.

Hypothetically you could use MATLAB Compiler or the Web Apps facility to compile into an executable that the user could download. It just might be plausible that on Windows systems that you could do adequate-considering-the-alternatives timing using .NET, perhaps asking for cycle counters. But unfortunately the carefully designed psychtoolbox has a high installation cost and so is not practical to compile in to an executable for deployment, I would think.

(High installation cost: you usually have to install multiple special drivers, and know how to account for possibilities like retrace timing not being properly available. Psychtoolbox worries about issues like visual smear and knowing to start the timing when the frame becomes visible. I wonder how Palvovia handles those situations?)

[Raymond MacNeil]

"A javascript approach hints that the code might be sent to the client to execute. The question when then become whether javascript can do timing accurately enough for experimental purposes."

That might be the case. I just finished piloting a Pavlovia experiment that one of my lab-mates is running via PsychoPy/Pavlovia and there was an intiatilization phase where I had to 'download' stimuli, etc. I don't know if that means what you have said though. In any event, it comprised fairly complex animations, and everything ran very smoothly on my MacBook Air from early 2015.

I think in general if you've got a well–designed experiment where you expect a reasonable effect size, the loss of power due to variability in subjects (keyboard latency, testing environment) will be well made up for with easier access to larger sample. I know for a fact that in the case of well-established effects like Stroop interference, cue facilliation (Posner, 1980), and Mental Rotation (e.g., Shepard & Metzler, 1971), the variability in subject hardware and test environments will not be an issue with a reasonable sample size.

However, if an experimenter is running anything where timing is critical, and the meaninful effect to them is relatively small, he/she should definitely avoid running online. Variability in subjects will be too much of a problem. It goes without saying, you can't properly control stimulus size, timing, luminance, etc.

In terms of an online Psychophysics Toolbox, I think you're probably correct. I also think if you're running anything under the constraints I mention above, Psychopy/Pavlovia should perform fine. Translating your MATLAB code to Python will be an easier/safer project.

Another thing for vision and attention researchers to think about though is that comparing how data looks when collected from an experiment administered remotely over the Internet (untrolled testing environment) versus in the lab might make for interesting paper itself. Who knows when the next pandemic will strike, or for how long we'll be out of the lab.

Answer 1

[Peter April]

Hi Jeremy,

This is a rather late answer to your question, sorry in advance for that. VPixx Technologies has developed a tool called Pack & Go, which allows users to upload MATLAB/Psychtoolbox-3 experiments for online testing.

This is a streaming solution, and latency is indeed a concern; VPixx reports end-to-end latencies of <300ms, which is suitable for certain psychophysical applications. Trial metatada, including network latency, is provided for each testing session, offering some peace of mind and the ability to correct data for timing issues after the fact.

However, as Raymond pointed out, uncontrolled data collection environments produce noise for many reasons, and certain studies with small effects and precise timing requirements wouldn't be appropriate. On the other hand, access to larger samples, rapid data collection, and eliminating the need for dedicated testing equipment/space has its advantages.

For what its worth, Pack&Go only charges for data collection sessions. Hosting and testing studies is free, and formatting your scripts for upload is fairly simple. So, if you are still interested, it's easy to try it out: https://vpixx.com/products/labmaestro-packngo/