Posted by: Gabriele Paolacci | January 11, 2011

Synchronous Experiments

To what extent can researchers rely on AMT for experiments with simultaneous participation? Designs that require participants to interact with each other at the same time are very common (e.g., in experimental economics), yet it is difficult to implement them on the web: One needs to have a certain number of participants to be present at a certain time, to handle dropouts, etc. Although AMT has some features that might facilitate the implementation of synchronous designs, there are only a few available examples. Mason and Suri (2010) describe a possible procedure to run synchronous experiments on AMT, which was successfully used by Suri and Watts (2010). Another description  of how a synchronous study was conducted on AMT can be found in  Lydia Chilton’s Master’s Thesis. We welcome other examples of studies with simultaneous participation conducted in AMT (or elsewhere).

References

Chilton, Lydia B. (2009). Seaweed: A Web Application for Designing Economic Games. M.Eng. Thesis, Massachusetts Institute of Technology.

Mason, Winter and Suri, Siddharth, Conducting Behavioral Research on Amazon’s Mechanical Turk (October 12, 2010). Available at SSRN: http://ssrn.com/abstract=1691163

Suri, Siddharth and Watts, Duncan J. (2010). Cooperation and contagion in networked public goods experiments.  Available at: http://arxiv.org/abs/ 1008.1276

About these ads

Responses

  1. A HCOMP 2009 paper called KissKissBan needed to do some synchronous ESP-game like tests and paired players on MTurk. I think their strategy was similar to mine – lure people with a high-paying hit, and have them wait around for 30 seconds or so before forming a partnership.

    http://portal.acm.org/citation.cfm?id=1600153

    I’ve also heard about inviting turkers to come back at a specific time to form a group. This can get larger groups together, in theory, but I haven’t tried it myself.

    In general, I think synchronous tasks deserve a good technical solution – there is a demand for it.

  2. I don’t say much about posts often, but your article was begging me to say something. Your information is intelligent. We appreciate you writing this.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Categories

Follow

Get every new post delivered to your Inbox.

Join 32 other followers

%d bloggers like this: