05/30 - Wishful Wednesday!

Title: Rewrite 10 questions | PANDA
Requester: Andreas Vesalius [AN03RVG7KIWPL]
TurkerView: [ $8.26 / hour ]
Description: For 10 questions want you to rewrite the question as a closed-ended question.
Duration: 01:00:00
Available: 13
Reward: $0.75
Qualifications:

• Basic Qualifiication GreaterThanOrEqualTo 70

[37Z5ZKVZFVX06BRCPTY5XAKXIQIL3V]​

 

@gurlondrums

 

Jasmine Hill [A3JDYRWHCGHFSH] Give your opinion on 3 logo designs - $1.80 | PANDA

[3N9FD9XF48JS9A8QQA5D5GPDMRFEVU]​

Good

Unrated

Unrated

$12.86 / hour
00:08:24 / completion time

​

Pros:
Pretty simple, goes quick., can be done quicker. Pay is good.
Cons:
A tad bubbly after not seeming like it would be.

To read Jasmine Hill's full profile check out TurkerView!​

 

Title: Jury Decision Making in Mixed Tribunals [ $ | $ ] PANDA
Requester: Margaret Kovera [A2ZH21CP28SHAM]
TurkerView: [ $28.79 / hour ]
Description: Give your opinion on a trial
Duration: 04:00:00
Available: 0
Reward: $5.00
Qualifications:

• Location EqualTo US,

[3I2V0CCR7RKND8WOJHJNS6U27RH1JX]​

 

Title: Answer a survey about your retail experience [ $ | $ ] PANDA
Requester: Tyson [AYNEY6ADTO3LS]
TurkerView: [ $44.05 / hour ]
Description: We are trying to understand what elements of your shopping experience are most memorable.
Duration: 01:00:00
Available: 0
Reward: $1.50
Qualifications:

• HIT approval rate (%) GreaterThan 98,

[3H18QCSCZWFJZDPL22LEDDFOBMZXYO]​

 

Labor Experiment [AOLHHITHIDALG] Decision-making survey - $2.50 | PANDA

[3BQ1D491CE3X3X5BYD534I56792TRN]​

Unrated

Unrated

Unrated

$12.64 / hour
00:11:52 / completion time

​

Pros:

Cons:
Math & science you haven't used since high school.

To read Labor Experiment's full profile check out TurkerView!​

 

D. M. [A17KLBWRZ7M97G] 2-minute Decision Making Survey(~ 2 minutes) - $0.25 | PANDA

[35QUA2ABY6V11VVPOE40E3VMVV1EG6]​

Generous

Unrated

Unrated

$15.52 / hour
00:00:58 / completion time

​

Pros:

Cons:

To read D. M.'s full profile check out TurkerView!​

 

sdresearch [A22W7OU7XG46TW] Decision making survey study involving a simulated negotiation - $1.00 | PANDA

[35DAEVU8JHHCXDHMG1MUHJD83TD78B]​

Generous

Unrated

Unrated

$11.58 / hour
00:05:11 / completion time

​

Pros:

Cons:

To read sdresearch's full profile check out TurkerView!​

 

Nicole Hill [A1SKDWMFC3O8RF] Groupon customers will answer questions about deals. Survey ONLY WORKS on a laptop/desktop computer. Will not work on mobile. - $1.25 | PANDA

[3XHSLX0964T5XI6242CE6N56C9IQ8P]​

Good

Unrated

Unrated

$9.36 / hour
00:08:01 / completion time

​

Pros:

Cons:

To read Nicole Hill's full profile check out TurkerView!​

 

Joshua Knobe [A3INOEQSBNJES5] Fun, Easy Judgment Task(~ 7 minutes) - $0.40 | PANDA

[3TDV6MPLVN9SYIT47SBZVXZQNZJAAX]​

Good

Unrated

Unrated

$10.59 / hour
00:02:16 / completion time

​

Pros:

Cons:

To read Joshua Knobe's full profile check out TurkerView!​

 

Title: The How You Think About Sex Study - Recruiting MEN only(~ 30 minutes) [ $ | $ ] PANDA
Requester: Ron Rogge [A1S1Q6DLK91KEK]
TurkerView: [ $1.31 / hour ]
Description: Are you MALE and at least 18 years old? U of Rochester researchers are looking for people to complete a short (25-30min) survey on how you think about sex.
Duration: 01:30:00
Available: 1
Reward: $0.40
Qualifications:

• Location In AU, CA, GB, US,
• Exc: [2820363-89976] DoesNotExist [ Request | How to Use ]

[3WAI7IWAWUH3G2ZI0V8UP8A0WRJKCT]​

 

Jacob Lopez [A9RUFOL92QGZA] Short graduate research study about public health advertisements - $0.50 | PANDA

[3EGG3WLVA0D8VU58X5ZVOYIJTJ3IXC]​

Generous

Unrated

Unrated

$13.33 / hour
00:02:15 / completion time

​

Pros:

Cons:

To read Jacob Lopez's full profile check out TurkerView!​

 

Plenty of the really prolific reviewers are "slowpokes" lol.

But for the sake of clarity I do reject accounts from people who are unnecessarily slow in the same manner I'd reject people who were submitting times from surveys they scammed if I could tell the difference (I've tried 2x versus the "fast group", but its only been done w/ high bonus HITs, interestingly, no one seems inclined to oversell a Req lol). Either way at some point, the data becomes unusable to the general public (whatever you'd like to take that to mean). If someone is doing things at 1/2 the average pace, consistently, across a # of HITs and especially when compared solely to the "known slowpokes" they're probably doing something really wrong and their times, for the vast majority of purposes, are useless to display on the site. If they happen to be the first person to review a Req, that Req is super boned to all the people consuming data through the API & that's not really helpful to anyone.

Its just a matter that slow times are much more damaging to the ecosystem than a "whoopsie" on the other end so while I don't think I've ever restricted people from consuming data on TV (except through the API for people who abuse it) there are def lots of instances where I've rejected incoming data - and IMO I don't even do it enough.

 

Cocolab at UC Berkeley [A1N1YFYVJGJSBR] Read about animals and answer questions (~2.5 minutes) - $0.30 | PANDA

[344RGS4J7S7CNUWHQXL5C02NR58MP0]​

Good

Unrated

Unrated

$12.27 / hour
00:01:28 / completion time

​

Pros:

Cons:

To read Cocolab at UC Berkeley's full profile check out TurkerView!​

 

Title: Pinterest - Rate How Relevant a Pin Is to a Query | Accept Requester: Pinterest [AURHK491L2LH6] Contact TV: [Hrly=$28.22] [Pay=4.54] [Fast=4.20] [Comm=4.00] [Rej=0] [ToS=0] [Blk=0] TO: [Pay=3.00] [Fast=4.03] [Comm=2.88] [Fair=4.58] [Reviews=93] [ToS=15] Reward: 0.40 Duration: 1200 Available: 50 Description: You will log in to Pinterest to complete simple tasks. Qualifications: Scammers DoesNotExist

HIT exported from Mturk Suite v2.3.5​

'][\'

 

Not what I would call an accurate one on a single review, no.

Take the survey everyone got screened out of earlier.. if I were to run @AMindAtTurk 's time versus the average, she'd look like she fell asleep at the wheel because everyone else was screened out. That would be a pretty crap way to make any sort of judgement on her account.

Which is why it requires a few data points before I make a decision on it. Literally everyone I have looked at has at least one or two super duper messed up times vs the average in their history if they're done 10+ reviews lol. Its just how things go w/ being paid for screeners or multiple versions of a survey, etc etc. Even RA has a few HITs he's taken like 3-4x the average to complete, stuff happens lol

But once you've got 10-20 reviews done its pretty easy to get a snapshot of your times vs a lot of different controls.

1) The flat average of everyone (this also accounts for the # of total reviews, so if your time is versus just 1 other person its not that important, but if 14 other people did the exact same HIT and you did it in 5x the time, something is much more likely to be wrong)
2) The average of "known slowpokes"
3) The average of "known speedrunners"

2&3 usually cover your second point (time clusters) by excluding one control group or the other.

I also spot checks manually, and you'll see me often times do a survey when a new user comes in & posts a time (which is why the first handful of reviews are force-exported here) to check & see if its realistic or not.

2 months ago, I'll say the review criteria was pretty booty. But the # of queries I've built + the huge influx of extraordinarily HQ data from power users (slowpokes or not) has made it really solid IMO. That said, I'm not a statistician & I'm notoriously bad at math, so haha