I'm sending a third batch that my team wants classified. So, we've had a long discussion about the interface and the accuracy of the results. We ultimately decided to get rid of the second question because I've heard it was causing problems for the workers. Just to keep it simple, we just included one question and less options to confuse workers. I'm including the same batch from the first one because one of my coworkers want to compare the results from the first time. We are expecting more accurate results. I've also been told to only require workers to reside in the U.S. Sorry for those who live outside here but just doing what I'm told. Hopefully we get better results this time. I will be more strict this time. Unless there's some underlying technical thing that I'm not aware of, I will pay attention to answers this time and reject submissions. Hopefully, I can snuff out bad workers. And yes, I still don't know how to do a closed qual HIT. What my team wants to see if the changes I've made will yield much better results.
it seems that you use the following quals for this batch: 94%+ approval rate qual is useless, as pretty much every worker on the platform will be above 95% at least. you will get more quality results if you raise the approval rate minimum from 94 to 99. an analogy you can make is that in school, 90-99 is an A. on mturk however, 99 is an A, 98 is a B, and anything below 98 should be an F
The instructions are also fairly extensive. Ideally it would get better the more familiar you are with truck types, but as it is, the time it takes to look up what type of truck type it is and then select the proper option drives the pay way down, they're below $10/hr as it is right now.
Alright, but if I'm going to raise the pay next time, then I'll be approving/rejecting based on accuracy for sure. I'm just saying because I'm sure my guys in my team won't be very happy of me just giving more money for no better results in return. Fortunately, I have reviewed the results and we have gotten much, much better results so I do think higher pay is deserved on the condition that the answers given isn't nonsensical. The vast majority of answers I've seen on here are either correct or they're not necessarily right but I can see why they picked the answer and they more than likely couldn't pick the right answer due to unclear images. But a few of them were just straight-up wrong. Those I will reject if my team and I decide to go with higher pay. And next time, I will do that 99% qualification requirement. And I still have yet to figure out how to do closed qual. At this point, my team wants to handpick certain workers I think. I think that's what one of them would prefer.
Feedback would be helpful on some of those 50/50 hits. I submitted a couple where it was hard to tell if the truck was a hopper or had a skirt due to the truck just coming into frame of an image. Also this past batch had a bit more images with no truck in them unsure if that was normal or not. I found the original guide really useful compared to the compressed guide used in wave three.
Dang, that was fast. Hold on, that was my fault. I'll send another one and see what I can do about this one. Sorry
