Hello guys, I'm sending probably one last test batch but for truck head types for my project. Finally, after all this time, I finally figured out how to do a closed qualification HIT but I won't do it here yet. My team wants to see how the truck head HITs will go first. But that's not the point of this post. I would like to thank everyone who took the time to do our tasks. It's been a great help....at least from what I can see. I don't know how my team feels about all of this but I'm sure they feel positive about it. I came here to make another thread because I would like those who have been keeping tabs with this to let them know that we're close to doing this for real. Now that I finally got the hang of this, I know....a little bit of what I'm doing, and now I know how to do a closed qualification HIT now, after this batch I just sent goes through, I think we will be ready. It's kinda been tough. Not only that, but I'm going to be leaving the team for the summer soon. So, I would like to get as much done as I can for my team to make my departure easier for them. What I'm planning on doing is since this batch is out, we'll analyze the results we'll see for this. We might pick out some workers from there if we spot some good workers maybe. If our results look good and we decide to pick out some workers, we might just be ready to send everything for the qualified workers to do. To be honest, I'm not even sure if we would want certain handpicked workers. I just don't know. It all depends what will happen in this next batch and what my team members have to say. But, if I had final say, I would pick workers from this batch because I can see that some workers have been really good to us and delivered some consistent results. However, I wasn't able to reject any submissions because we were on our "trial period" and I didn't know what I was doing at the time. Again, I don't know what will happen. My hopes are that we get good results again like last time and we have enough confidence to release a full closed qual HIT and release the full wave of stuff we want classified. So, thank you all.
your work has objective answers? can't see a reason why varied answers would be helpful here. as you've learned, there's a cost in time/money to find these people, don't lose them and keep track of their worker IDs so they can get quals. you can always do it in waves, release 10% of your project to those final qualified workers, look over what they've submitted before letting them have the rest.
Yes, varied answers won't help us. We get multiple workers to check for consistency mainly. The whole point of this is so they can deduce a model for identifying trucks like this later in the future. However, that isn't my job nor my concern. All I know is that my team is using this to ground truth data and my role in this team was to set this whole MTurk thing up and create the images they want for the tasks. That's all I know. As for the workers, I found a way to do it in the manner you've described. Apparently, there is some custom qualification that I can create in another tab. If download the csv file and put that qualification as a 1, then you're marked. Then, once I publish the full project, those who are marked, can do it. I plan on doing that before I leave.
