Let's talk about all the things you love and hate about GeoHIVE HITs. If you ever wanted to get quick feedback, now's your chance! • Note that I do not run the 'DGIS' or 'PicsFromAbove' accounts, but the GeoHIVE side is all mine. • I want to make things better, so the HITs are more tolerable. Help me to help you.
I'll be blunt. The reason people hate the hits is for two reasons One is that they are pay a fraction of what they used to. (They used to pay 50 cents for 15 images or something like that) and they also have very crashy servers. The geohives do seem to do the 15 images for 15 cents though a lot more often and that is a lot better than what the other PFAs stuff does I'll assume that you don't have control over the budget so as long as you can get the server business sorted out I don't think that you really do much to make the hits better. I do enjoy doing them when the pay is right though as long as the servers are working.
First off thanks for the work and coming here to get feedback. Honestly the issues were summed up above me. Your department is the best of the bunch for sure and I have little to hate about the Geo's. Obviously we all want more money but when the servers are running smooth your hits are mostly decent enough to be min wage at the least. Bring you PFA coworkers on over for a chat, I have much to say to them.... Again thanks for the work!
Heya I think in general the server thing is going to be the biggest feedback you get. Its really bad, and its like coming to work to find out your boss trashed your office and set your computer on fire and he still expects you to work through it, haha. I'm also not sure how it works now since I have the qual, but I do remember back in the day it was very difficult to tell how to get the GeoHIVE qual. If I hadn't been told by another worker I would've never known. So if the outside site registration is still necessary I'd probably say to put a note about that in the HIT (if it isn't already, again sorry its been a LONG time since then) about it so new workers can find you and actually work on the HITs. Oh, the biggest pain point I can remember from you guys is swapping the keybinds between batches. Please, please don't do that. If 1 = yes, 2 = no, and 3 = cloudy on batch 1, and then you switch it in batch 2 it is inefficient to have to remap them in my head & if I hide the keybind thing after seeing it once I wont even know and end up submitting a lot of bad data.
Yeah Geohive does tend to have the best hits out of all of them. It just has always confused me to why they have 4 requester accounts to what are almost all the same. Or at least all 3 of the PFA accounts are basically the same plus the Geohive account.
Funnily enough that is likely why Geohive likely works better for a lot of reasons. The qual is spread by word of mouth but it isn't hard to get once you know you just need to sign up at https://geohive.digitalglobe.com/welcome/register.html Although if you ever find yourself not having enough workers to be able to get the work done you should try to move the qual to a more obvious location. I don't however suggest the auto granted qual that your co-workers use
First off Welcome to Turkerhub. Second your post has been moved to the Requester section so others can leave feedback as they wish. Server issues would be my complaint. Random black screens while not as bad as picsfromabove or dgis is still an issue with geohives. On cases like this maybe a forth option should be added to indicate the image didn't load, instead of using the third option which is usually cloudy or blurry image. Well someone has to say it so here it goes. Are these secretly being used for drone strikes?? As we have come to call these drone strike hits.
Oh jesus this.. how did I forget to ask this? Honestly @Bobby if you don't read anything else in this thread please give us the answer for this LOL. Its the source of much debate (and twice as many jokes) within the worker community.
Hi Bobby, Not sure if you guys all share the same server(s), but server issues are usually a problem. Sometimes we're fiddling with tabs of work, and images won't load for a few seconds. Other times, they'll load to a black screen. Concerning pay, it could be much better. There's got to be a ratio of pay to pics that will suit your budget, and line our wallets with that beautiful green.
The biggest negative I have with GeoHive hits is the server issues. When they load quickly, they are good hits, but when they don't, it's not worth it. The hits that are a simple multiple choice answer are usually good. I have felt leery about doing some batches when the object you're asking us to label is difficult to identify from satellite images. Some things can be clearly seen, but some of the batches have asked for things that are very difficult to accurately identify with a top down image. I want to give good data, but sometimes it doesn't seem possible to give accurate answers. The hits which require placing a tag vary wildly in the amount of work required. Some have been good but others have required far too much time to tag everything for the pay. I'm not sure what a good solution would be. Some requesters give bonuses for work. You could try a bonus system for number of items tagged beyond a certain amount, but you would need to have a system in place to ensure that the tags are valid so people don't try to scam the system. I'll also repeat what @ChrisTurk said about swapping the keybinds. It's much easier to work on the hits when the selections are consistent between batches. Thank you for stopping by. I feel like I've been very critical in this post, but overall I really like the work you post. I try to keep an eye on the GeoHive account because you do generally put out good work. I hope to continue working on them in the future.
Geohive, love that style of hits besides the random low quality image that I cross my fingers on (which you folks seem to understand) the server issue is the only thing. You're going to need a pretty big pipe though if the hits work rapidly to keep up. Instead of 30 people struggling to load, you'll have 200 people blazing through images. Clearly possible of course, with enough money, keeping hits uniform (number 1 to select yes etc.) is a good idea also. Last but not least, 15 for 15 cents is pretty nice, hits take less time than 50 for 50, but it doesn't really matter that much.
I was just curious how you guys handle workers that are just starting out on your hits. I have started your hits a few times, only to feel like I don't know how accurate I'm being and returning them in the end. So I just wanted to know what your policy is on rejections and qualification revoking, and if there is any leniency for new workers to gain the experience necessary to feel confident that the work is being done right.
I will second the idea that keeping the hotkeys the same throughout hits would be very beneficial to doing these more efficiently with better accuracy. Thanks for taking the time to listen to us it is really great when we can establish good worker and requester relationships through our work forums.
My only comment is on the qual. I went to the website everyone talked about and registered bus still don't have the qual. Most have said they had to send a message to you through Mturk to eventually get it and honestly I just have messed with it. My suggestion is to create a qual test that directs the user to register and then submit for an auto grant. You can always compare the two lists to make sure people are really registering and revoke the quals of those that don't.
Good News: We are currently re-designing the postgres tables that have existed since the platform was built. During this stage, we are all suffering through lag in the HITs. I say we, because it sucks for me as the admin too. We will likely see a completely different server and speed of image service within the next 6-8 months. As with anything in business, our demand is through the roof, and the HITs will not slow down. The three different requesters (PFA, GH, and DGIS) are in different offices. GH is my baby, and here in Tampa. We handle our crowd differently, we handle our training differently. The three of us have different directions, with different customers, and different bosses. We can't dictate each other's processes, but we do often share each other's crowd members. Why do I have you register through GeoHIVE's Website: I do this to track stats and progress of users in our crowd. I need to have the ability to define the top users in specific campaigns, should the requirement arise to invite them into a special set of campaigns, still served through MTurk. I also want to know who my long time users are, and who has earned their stripes. My number one in the crowd knows exactly who she is! (*wink*). Where I'm trying to take us: I want to establish a forum, much like this, that allows people to base their avatar and swag on the number of campaigns and votes they've worked. It would serve much the same purpose for this thread, only it would be solely a GeoHIVE HITs forum, and would be updated often, and between campaigns.
I made the requirement that people sign up through the website (I also made the website *Brushes Shoulders Off*). When we launch our feature editor, which is currently in development, we need a crowd of familiar faces to lead off the HITs. I need reliable users, who I can contact for feedback, send discreet invites to, ask questions about campaign progress... etc. For Example: We had one user in one campaign go through 400+ map views without a single tag being placed. I suspected there was a software error, and rather than revoking quals on a hard worker, I emailed him and asked what was going on. He told me that his tags were disappearing as soon as he placed them, but he kept working, hoping it was just a glitch. He's still in our crowd, and we communicate often. The problem has also been addressed.
