I appreciate you taking the time to lay out your argument. In response, I realize we weren't clear when we talked about people doing hits too fast. 95-99% of the people who participate in our campaigns do as you say; they go about their own pace and provide quality work. However, we take issue with people who complete a disproportionate number of the HITs at speeds that are beyond humanly possible and provide invalid responses. To us, that is gaming the system. We understand that scripting and shortcuts are beneficial to workers. We don't take issue with the use of scripts, so long as workers honestly complete the work.
I also haven't done a geohive in probably a year or so. As I learned more about the mturk platform I also learned where the better money is, and haven't really looked back to you guys. I also am in a bit of a different category as I have a full time job, so I'm even MORE picky when it comes to what hits I will do since I have a real income to fall back on. Your hits don't pay enough for me to spend time on, and I can only imagine that people who do pay attention to them are providing less than stellar data because of that. I wholeheartedly agree with everything ezdz said. The way this market works you're going to get the new people who don't know where better requesters are, and the people who provide substandard work to meet whatever hourly they feel is worth it to them. While I can't speak for everyone, but myself (and many people I am aware of) use at least one simple script per hit we work on, and that is a wage calculator. We do a hit or two to get a feel for the pay, and then either dive in or move on. My advice to you is have a few of your workers (different people) sit down and complete a batch of your hits and gauge their times (tell them to pretend this is their only income), and their mess ups, and come up with a possible few things: 1) better pay scale 2) a system for errors to not cause qual revokes from mistakes (unless you want to be draconian, but this will drive people away if you rejects and revoke with no communication) 3) quality control questions 4) revoking of a qual when people don't meet your quality threshold (I don't think the threshold should be absolute 0 mistakes ever) 5) breaking up the hits, @Eisenpower mentioned an assembly line and it's spot on. You're going to get better overall data breaking up stuff and making it more linear per person. I am a great worker to my IRL job. I bust my ass and go above and beyond, but they also pay me higher than the minimum wage and I have lots of extra benefits. And if I happen to make a mistake, it's addressed appropriately and I am told what I've done, and I make sure to correct it and not cause it to happen again. Obviously if I didn't and kept repeating the mistake I would be fired and lose all this great stuff I have at this current job... so you can bet that if they say jump I'm asking how high and which way. I feel like sometimes people try to take the high road with this and be like "you should do your absolute best! Provide your best work ever!". I'm sorry, but for $10/hr you're getting teenager me and her attention span, not adult me with her experience and focus. Well to be frank for 10/hr you're not getting me at all because I rather pull an overtime shift at my IRL job, or work for the various 20/hr requesters that are up instead. I'm sorta just rambling at this point so I'll wrap it up. Those are my various two cents, and I hope it works out for y'all.
@James Wilson I'd love to get a reply to my email about those 19 rejections. I'd like to receive an explanation and if possible some examples of my mistakes. I will definitely edit/remove my feedback from turkerview after that. Thank you!
@James Wilson - Thanks for getting in the discussion and providing info. This comment is aimed more at the .10 batch, but I guess could be useful for all the batches. I think examples of edge cases would be helpful. Maybe a link in the instructions to an example page instead of filling up the hit with examples. For example, on the .10 hit, I wasn't always sure when to use the industry tag. Take warehouses for example... I know that a huge warehouse with semi trucks and/or rails leading to it would count (example 1), but what about smaller structures that may be a small warehouse, but could just as well be a strip mall or other commercial type center (example 2)? I wasn't sure about mines counting as industry, but they weren't listed so I wasn't marking them. Or if the image is all grassland, but there is one house somewhere (example 3), is that still marked as residential? I was marking those because I think that's want you wanted based on the instructions. tldr: A few typical examples and then some edge case examples may go a long way to helping you get some better data.
That's why I mentioned the subjectivity of it. The responsibility is on us to make a judgment in those cases. All we can do is make our best judgment given the information we have. Everything you mentioned, minus mines, is what also went through my head when I did them.
I agree with this, especially the fringe stuff. I have been marking am image with even 1 house as residential, even though that isn't residential Same for the industrial, if it looks like it could potentially be industrial I mark it, but that doesn't mean it is. It could just be a warehouse, green house, or any other large structure. Also in these latest hits, I can't tell the difference between the military base and an industrial park. The examples look interchangeable Also, if the image is a bunch of industrial buildings, do you want me to just toss down a bunch of markers?
In those recent batches, you did the right thing. If there was a single house and a single warehouse, we were looking for Residential and Industrial. We, GeoHIVE, need to do a better job of telling you our expectations and what to expect in the instructions. Several people have emailed us to ask for clarification and our expectations. In these batches, there are a lot of images that are hazy or dark and a lot of images that don't have any of the features present. Don't worry if there aren't any of the features present, and if the image is too poor quality, then tag Poor Image any where on the image. I agree, there's a lot of ambiguity between some industrial and military facilities. As the previous posts have said (@slothbear and @Eisenpower), a lot of this is subjective. Do the best you can and don't over think it. A single Industrial tag will work for an industrial area. No need to tag each facility. For this campaign there is really no need for more than 2-3 tags of each type.
Could you please tell us when the hits will be checked and hopefully approved for the last few days? The 2 batches that are up now.
I would just like the request to do geohives batches to be approved. I submitted back in december of last year. Should I try again? (I so can someone give me a link)?
I need my account on the geohive site unlocked. I took the qual test that recently went out but when trying to log in it says it can't with the information provided and of course I can't make a new one since the email is already in use. Any help would be appreciated.
Did you click the link to reset your password? I think you can use either your email or username to log in. That's what I did.
