09/14 - Still Saturday!

Title: Short survey about a coffee brand | Accept Requester: Christine [AZS9Y3KL6HWBS] Contact TV: [Hrly=$22.08] [Pay=Generous] [Approval=1-3 days] [Comm=Unrated] [Rej=0] [Blk=0] TO: [Pay=5.00] [Fast=4.00] [Comm=0.00] [Fair=5.00] [Reviews=1] [ToS=0] TO2: No Reviews Reward: 0.12 Duration: 1:00:00 Available: 1 Description: This HIT should take approximately 2 minutes Qualifications: Total approved HITs GreaterThan 100; HIT approval rate (%) GreaterThan 95; Location EqualTo US

HIT exported from Mturk Suite v2.6.13​

bad don't do it. it kept saying disconnected from server when it was time to get the completion code.

 

Good morning Turkers*

*Exclusions apply. "Good" not available in Canada, you lot only get morning. Due to cultural limitations greetings cannot be extended to Turkers in Cuba, Argentina, or France. No purchase necessary. 1 in 1,000,000 odds to upgrade your morning to Great.

 

Eva Stockigt [APJ2B8Y24PN84] Can You Make Good Predictions?(~ 50 minutes) - $6.50 | PANDA

Generous

Unrated

Unrated

$13.55 / hour
00:28:47 / completion time

​

Pros:

Cons:
-Writing
-Multiple Sections
-Watch timed videos

Qualifications: Total approved HITs >= 100; Location In AU, CA, GB, NZ, US; HIT approval rate (%) >= 95;

To read Eva Stockigt's full profile check out TurkerView!​

 

Of note:

How was age input controlled across the two studies? Was it a text entry field? Did respondents in the first study who reported an age of 42 later report 24, or 2, or 4, or 52, or another easy typo or transposition of a typo? Was it a slider input, the greatest and most heinous inhumanity which mankind has yet devised? Was it a drop down age range or age selection, which could be accidentally changed with careless clicking or keyboard input?

Did the study query respondents, asking if they still were employed in the same position at the same company (or, at least, holding a position with similar authority, either at the same company or in a new company) across both questionnaires?

Evidently not.


It seems to me that a simple solution would be to continue to use the pre-screen survey, but include the fictitious item in the pre-screen as well as the main survey. Exclude respondents that affirm experience with the fictitious item from the main survey. For respondents who pass the fictitious item test in the pre-screen, but then respond positively to the item in the main survey, drop these respondents from the survey and issue a missed attention check rejection.

If maintaining some state across surveys is critical for the data (respondents must have been and must still be workers with hiring authority), then include a question in the main survey that permits respondents to affirm they are still in the same or equivalent state.

 

They just read this Reddit thread.
https://www.reddit.com/r/mturk/comments/bon218/as_a_30_year_old_i_spend_way_too_much_time/

 

SmithOscar [AKL99MZHRN058] Decision making study - $2.50 | PANDA

Unrated

Unrated

Unrated

$15.73 / hour
00:09:32 / completion time

​

Pros:

Cons:

Qualifications: Already taken DoesNotExist ; Total approved HITs > 5000; HIT approval rate (%) >= 99; Location EqualTo US;

To read SmithOscar's full profile check out TurkerView!​

 

Morning all!

 

Sounds like they decided to go about creating a scenario that would affirm their already held belief. I but I could make a couple of surveys that PROVE that mTurk workers are all actually outside of the United States too... maybe even not from Earth.

This isn't rigorous scientific study, this is attention seeking (but one of them signed up for mTurk and got rejected and is still butt hurt about it).

In my untrained, non-qualified, opinion.

 

Genuinely surprised that it took this long, tbh, especially given how many people come from garbage sites like Swagbucks where the best way to complete surveys is prolific lying and clicking as many things as fast as possible. For a lot of people, doing surveys on mturk isn't that far removed from doing surveys on garbage sites like Swagbucks.

 

>However, changes from full to partial hiring may represent giving the question second thought or reflecting differently upon one's experiences. It is up to the individual researchers to choose, a priori, how stringent they would like to be in such cases.
Maybe you should actually read the paper before finding the one line that makes you angry and write up an entire post about it.

 

That's essentially what the last paragraph says, except they don't suggest rejections for people who are blatantly lying because they desperately need that $4/hr .50 cent survey to hit their $20 goal today, which they should. Requesters are way too afraid to reject because some people are also way too eager to start entire crusades over one mistaken rejection and drive off someone who made an honest mistake with personal insults and hatred.

 

How else do I keep my torches and pitchfork business afloat... hmmm... HOW I ASK YOU.

 

Maybe academics, who 100% should be held to a higher standard than some rando on the Internet, shouldn't be releasing papers with clickbait level titles.

 

I didn't even read the title, tbh.

 

I read the entire article and I saw that part as well. Just because they address a huge hole in their research doesn't mean it does not exist. I can't write a paper up that draws conclusions from faulty research methods and simple throw in, "but it's your call on how you use what I'm saying because my research methods have flaws" and have it be okay that I used carp methods to begin with when I could obviously identify them before writing the paper

 

leave my sisters and brothers alone!!!!

 

In principle I'm a much bigger proponent of kicking respondents to a "Thanks for your work, unfortunately you missed an attention check or failed to meet survey requirement or whatever. We aren't going to reject you, but piss off." instead of rejections, but for multi-phase studies with an unambiguous fraud detection question, rejection with a clear and unambiguous justification in the rejection note seems warranted.

Of course, then you get flooded with "But, I just mis-clicked the bright red 'I'M A LIAR PUNISH ME' button, why are you punishing me?! I am going to call my Uncle who works for Amazon and is also in this thing called IRB or whatever and you are in BIG TIME TROUBLE!!!"