This is a large change, mostly aimed at moving TVJS into a slightly different role. Most likely moving forward TVJS will be more of a beta playground where I can test new features before pushing them upstream to more "stable" apps (Forker, MTS, etc). To that end, there is a lot of new stuff in here. By far the most important is: Review Approval Time Auto-Updating While we understand the old AA on www was being abused by some workers in order to submit poor quality work to requesters we don't think workers should be forced to fly completely blind. The gap between 1hr AA and 30 days is unacceptable for workers to make informed decisions. So we're going to work to responsibly fill in the gap for workers who prefer the peace of mind of knowing when (& how likely) a task is to approve given a good faith effort. Occasionally when you visit your dash or check in to leave a review TV will check the HITs you've reviewed & if they've approved fix the "fast" rating for you on TV by itself. Rejections will not be auto-reported (there are good reasons to leave this up to the discretion of the worker, especially if they made a mistake and may have "deserved" the rejection) but TVJS will alert you if a HIT you've reviewed has been rejected so you will know it for sure & can update the review if you want. Rejection reports will be accompanied by the Requester's feedback on the task so that other workers can have more information & make better decisions around these data points In testing w/ individual users this already accounts for about 75% AA time coverage which is an insane improvement versus the manual method TV has traditionally relied on. We'll be working on ways to back fill the existing data set as in many cases it has been tracked, just without the API to update the data. TVJS moving to the API servers means TV itself will have a lot more resources reserved for incoming data. This data will be returned to workers through the API in their script of choice, for TVJS10 an example is provided here: Misc Changes: TVJS has been moved to the new View API Please see the updated announcement for that here for plans moving forward w/ the API The data will be shown without a problem, but in order to submit reviews you'll need to grab your keys (its a simple process outlined in the app) & connect them there A handful of visuals have been changed, including the icons TV uses for its popovers Info boxes have been tweaked slightly to display information according to the new API standards. This isn't really important to end users, but TV's API will feed scripts data according to its ratings in order to help standardize metrics across the platform (such as approval time ratings, wages, etc). We'd like to see standards become the framework for interacting with requesters in ways that benefit both sides (quality data, quality pay). When we can send requesters things like "Hey, a 1-3 day approval time is "fair" we can help workers get paid quickly --- but all of those kinds of initiatives start with establishing what we want those metrics to be. If after community discussion we come to a consensus, and it doesn't match whats already there, this gives TV the opportunity to update information with scripts calling its data without forcing devs to dig into their code base & make adjustments/etc. Its complicated, we'll see how it goes. Info boxes will begin feeding individual user data back to the user. In TVJS10, that starts with your individual hourly wage average for any requesters you've reviewed through TV. There will likely be more information coming here in the future especially as we improve the way users are grouped/compared on the back end For now, you should be able to see at a quick glance if a requester is good for you even if it isn't for others (or vice-versa). A lot of the internals have been updated & fixed various errors/bad interactions around the script Reviews are now open to everyone Instead of filtering people before they join the site, we're now going to use the same metrics (approved HITs, approval rating) in generalized aggregate data to "group" users together & begin feeding more tuned aggregate data to groups. TV wont take your exact stats, instead it'll put you in "ranges" (ie, 1-5k, 10-25k, etc) This opens up more metrics to help users not only see the low/higher thresholds of tasks but also how they stack up with users similar to their experience/level on the platform. "Batches" are now tracked a lot better. Its still not perfect, and I don't really recommend reviewing them, but its at least better for folks who want to do so. "No Share" button should be more reliable. The opening agreement guidelines have been updated to help onboarding users have a clearer view of the kind of data TV generally takes in. Either way, TV will filter a lot of things itself on the back end now & as new issues crop up I'll adjust things as needed The client side qual blocks for submitting reviews are gone from the templates Return reviews are now available on the HIT Preview page. This should cover the major highlights to TVJS. As always let me know of any bugs that crop up or changes/additions/etc folks would like to see.
How do I leave feedback on a Requester once the hit has been Paid? I've been getting Requesters that pay within the last 5hrs or less (after completion) and would like to give them thier due atta-boy review. All that shows up now is a grayed out plus sign that expands when clicked and turns into a minus sign.
This page shows an overview of the entire TVJS script, the actual review process starts at ~2:50 into it if you just want to skip to the relevant part: https://turkerview.com/review.php Just let me know if any parts of it are unclear, or if you prefer a written overview of the process I have some posts bookmarked, I think this one does a pretty good job highlighting it: https://forum.turkerview.com/posts/1072252/
@ChrisTurk Is it possible to get the red/green/gray icons that we see in the queue somewhere on the task itself as well? I'm usually going back to the queue to see what color the icons are to see if the HIT is worth it.
If you update MTS you'll get a slick advanced task breakdown: https://forum.turkerview.com/threads/mturk-suite-turkerview-additions.2039/ But yeah Requester data will make it in as well soon-ish
@ChrisTurk hi ya hope you were able to Capre Diem, i have a stupid , im sure, question, i had to reset my pc and im wondering if that affected my API KEY, cuz on the mturk dashboard i was prompted by the green screen to get an API KEY, so i did, i didnt resubscribe to early supporter, so we dont run into the same problem as before, i went a head and copied the key provided to my clipboard is it the same key as before? no i didnt ever jot it down cuz im just not made that way lol
Yeah the key will never change, so you just have to reenter it and hit save in whichever script you're using
