English 中文(简体)
In mTurk, how can I use participation in a previous HIT (or series of HITs) as a qualification?
原标题:

I am using mTurk for surveys, and I need a way of making sure that people who have participated in a previous survey / HIT do not participate in certain future surveys / HITs. I am not sure whether I should do this as a qualification or in some other way.

I know there is some way to do this, but I have no idea how. I have very limited programming experience and would greatly, greatly appreciate specific instructions on how I might do this. My understanding is that I might need to use AWS? Many thanks!

问题回答

Mass rejections as suggested above are a really, really bad idea in terms of your reputation as a requester. You are much better off creating a Qualification for the new HIT, which automatically grants a score of 100 (or whatever) to anyone who takes it, and assigning scores of zero to everyone who has done the previous surveys. This prevents repeats but doesn t annoy any of your workers.

The easiest way to create a Qualification is at https://requester.mturk.com/qualification_types.

If you download the csv of workers from here https://requester.mturk.com/workers, you can assign scores to workers who have done the previous HIT(s).

To make the qualification grant scores to new workers automatically requires the API, though.

Here s a hacky way to do it:

  • When you accept HITs for surveys, save every participating worker s ID.
  • In the writeup, note that "if you ve done previous surveys w/ us, then you can t do this one (IE, you can, but we won t approve it)".
  • When you approve HITs, cross-reference the worker ids with anybody who participated in a previous survey, and reject the hits of any that match.

If you re doing enough surveys, then yes, you probably want to use AWS API for at least the approval part. otherwise, most things appear to be do-able from the requester interface.

Amazon Mechanical Turk service has this option for requesters to grant their workers by Qualification_Type. In this way by connecting your HITs to a qualification_type naming "A", then granting workers exactly the same qualification_type, only workers who have that qualification can see and work on HITs.

First, creating desired qualification types through mturk web UI.(it is only name and description) requester.mturk.com > manage > QualificationTypes. It will give you a qualification id after generating it. (you will need it soon)

Second, in HIT creation loop, you have to use QualificationRequirement class. (I am using java code and it looks like the below-mentioned code):

QualificationRequirement[] qualReq = new QualificationRequirement[1];
qualReq[0] = new QualificationRequirement();
qualReq[0].setQualificationTypeId(qualID);
qualReq[0].setComparator(Comparator.EqualTo);
qualReq[0].setIntegerValue(100);
qualReq[0].setRequiredToPreview(false);

then in HIT creation loop, I will use this:

try {
            hit = this.service.createHIT(null,
                    props.getTitle(),
                    props.getDescription(), 
                    props.getKeywords(),
                    question.getQuestion(),
                    new Double(props.getRewardAmount()), 
                    new Long(props.getAssignmentDuration()),
                    new Long(props.getAutoApprovalDelay()), 
                    new Long(props.getLifetime()),
                    new Integer(props.getMaxAssignments()),
                    props.getAnnotation(),
                    qualReq,
                    null);

Third is assigning the qualification type to the workers that you want them to work on your HITs. It is very straightforward, I usually use mturk UI to do it. https://requester.mturk.com/ > manage tab > Workers. You should download the CSV file if you want to assign this qualification to a bunch of workers. (Workers are who worked with you in the past)

you could notify workers by sending them an email after qualifying them

Notice: Some workers are very slow in answering your new HITs after qualifying them; so keep in mind that you should have some backup plan and time if you will not receive enough response in a certain amount of time.





相关问题
Mount windows shared drive to MWAA in bootscript

In MWAA startup script sudo yum install samba-client cifs-utils -y sudo mount.cifs //dev/test/drop /mnt/dev/test-o username=testuser,password= pwd ,domain=XX Executing above commonds giving error - ...

How to get Amazon Seller Central orders programmatically?

We have been manually been keying Amazon orders into our system and would like to automate it. However, I can t seem to figure out how to go about it. Their documentation is barely there. There is: ...

Using a CDN like Amazon S3 to control access to media

I want to use Amazon S3/CloudFront to store flash files. These files must be private as they will be accessed by members. This will be done by storing each file with a link to Amazon using a mysql ...

unable to connect to database on AWS

actually I have my website build with Joomla hosted on hostmonster but all Joomla website need a database support to run this database is on AWS configuration files need to be updated for that I ...

Using EC2 Load Balancing with Existing Wordpress Blog

I currently have a virtual dedicated server through Media Temple that I use to run several high traffic Wordpress blogs. Both tend to receive sudden StumbleUpon traffic surges that (I m assuming) ...

SSL slowness in EC2

We ve deployed our rails app to EC2. In our setup, we have two proxies on small instances behind round-robin DNS. These run nginx load balancers for a dynamically growing and shrinking farm of web ...

热门标签