A simple idea: reward good workers

One of the problems which immediately comes up in any open system is reputation--trust.  On the one hand, Amazon keeps track of your HIT approval rating, how many tasks you do are accepted vs. rejected.  That lets requesters know if you have a history of good work.  Oddly, sadly, that rating is not considered when Amazon decides to suspend your account.  That means a two-day old Turker who goes on a scamming binge is treated the same way as a multi-year Turker who accidentally accumulates one or two blocks before BAM, account gone, reputation irrelevant.  


Here are some telling comments from the TurkerNation.com forum:

• Yeah they don't consider ratings since those can be scammed by doing work for people who never reject but they should consider the whole account. I also had a lot of exclusive trusted worker type qualifications. So even if a couple requesters did block me and not want me working for them any more there were many that trusted me, liked my work, and wanted me to keep working for them. But they only saw I had blocks and banned me. If that is even the real reason since they changed their reason after a few emails.

• Everyone always lists their approval ratings when they get banned and I bet amazon doesn't consider it. Lets say you have been turking for a year, 99% and over 100,000 approved. You can scam the system and get 50,000 rejected and still be around 75%, which is where some new turkers fall by doing bad hits. Obviously that's a good way to scam the system. So, amazon probably looks at the past 7 days for low quality. If you have 500 rejections and 500 approvals in the past 7 days, thats pretty horrible work. And i bet amazon doesn't consider rotten requesters that will mass reject for no reason. The system is already set up that way that requesters can do no harm, and can create as many accounts as they want, but workers have 3 strikes and they won't tell us a damn thing.

• Lately I've only been doing hits I'm really sure about and I looked carefully at just those two. I only did 2 where the color in the description matched the picture, the size was stated in the description, etc just to be sure. And then I get warned over 2 hits......give me a break. After accumulating over 350,000 approvals with only a little over a hundred and thirty rejections I can't be doing that low of quality work. And about a hundred of those rejections were from produniverse when several posters were having probs with them. Its a shame anyone with a 98 or 99 approval rating could be banned so easily, but it seems like it could happen to anyone at any time. 

•  Really a few blocks shouldn't matter under the current system as iambrendalee said. Unless they send in along with the block something horrible "I want them banned" "Obvious scammer" then it should be treated as it is, a block from just that worker and not the whole site.  My past blocks were very old(over a year) and all misunderstandings but because of Amazon not making it easy for Requesters they didn't know how to lift the block. Even then an older answer from Amazon said even lifted ones do count against you.  If they are going to count so badly against you the least they could do is send a warning email when you are blocked but I never got any. 

• I say, unless someone is clearly abusing the system, then they shouldn't get banned. Requesters block people randomly, their approval / rejection systems go on the fritz and randomly reject, bad requesters reject for no reason at all...those things really shouldn't have an effect on our accounts. In what world should a sleazy (or dumb) requester or two...or three be able to determine whether I can continue working or not?  The thing is, we're disposable to mturk, so it's all moot. It's like they're looking for any excuse to get rid of us...which is just plain weird, considering they need us. 

• On a side note, stories like this are what make me paranoid as a turker. I have done this, on and off, since March of '08. It worries me that even with many thousands of hits completed properly, and an approval rating of over 99%, that a person can just arbitrarily be banned with absolutely no recourse at all. I know MTurk isn't a large priority for Amazon, but have they ever actually come out and explained why there is so little support for their work force? 

• Maybe you should take blocking more seriously. I mean, it only takes three irresponsible requesters blocking you before you are completely banned from mTurk. There isn't room for requesters who block even by mistake. 

I didn't post links to these, but you can just go to the search box and type in 'banned' and you'll see hundreds of posts come up.  Maybe, instead of scaring the pants off of loyal users, Amazon could leverage the reputation data that they have famously developed at Amazon.com and start treating Turkers more the customers and less like the commodities.