I started Turking a few months ago, made a quick couple hundred dollars, and then learned how MTurk's system leaves workers too often out in the cold. The site is not well maintained, and it functions far below Amazon's standards of customer service and user optimization. A Turker faces seemingly random blocks with little recourse. Spam tasks are not well screened. The interface needs basic improvements. Development is frustratingly slow. Amazon has heard these criticisms many times over but mysteriously has done little in response.
The point: a company as sophisticated and customer-service oriented as Amazon should pay more attention to its flagship crowdsourcing operation. MTurk should be nimble and trustworthy. It should not treat its best workers the same as its worst. Users should be given the benefit of the doubt, or an opportunity to establish a meaningful reputation. Site search should be robust. Task and profit tracking should be personalized and detailed. The site should be easy and efficient to use, and new features should be added regularly in response to user input.
The purpose of this blog is not to badmouth Mechanical Turk or Amazon.com, but to help them make a better system which will attract more people to use it, contribute to it, and benefit from it. Let me know what you think...
Four out of ten HITs are spam. Surprise.
So says NYU business professor, Panos Ipeirotis. Here's his finding:
Here's the link to Panos' full post describing his research: http://behind-the-enemy-lines.blogspot.com/2010/12/mechanical-turk-now-with-4092-spam.html
The results were disturbing. Out of the total of 5841 HITs, a total of 2390 HITs, or 40.92% were marked as spam HITs.Turkers have a feel for what to avoid. If it sounds too good to be true, if it looks fishy, if the requester is unknown and offering $10.13 for a ''quick HIT'', we try it once, and don't try it ever again. But why are these tasks still on the site at all? They're painfully obvious to notice (except for newcomers), and they bring down the efficiency and reputation of the site. Maybe Amazon will do something about it.
Here's the link to Panos' full post describing his research: http://behind-the-enemy-lines.blogspot.com/2010/12/mechanical-turk-now-with-4092-spam.html
Spamgirl speaks! An interview (part 1 of 3)
Spamgirl is the dutiful, no-nonsense moderator of http://turkers.proboards.com, the Turker forum. She was gracious and uncensored enough to talk about her experience with Mechanical Turk in great detail. Enjoy.
Where did you get the name Spamgirl? Why go there?
When I first started on the internet (oh man, it was TELNET LOL I'm old) I had a different name. Then my daughter was born and, well... she stole it from me and decided to use it for herself. I needed a new name, I was literally nameless. I went through a bazillion (Starla, Sunburnt Raisin, gosh I can't even remember them all) and then one day my friend was commenting on my anti-UCE activities and called me "The Spamgirl". It stuck. I also had a website at the time called "Sensible People Against Misinformation", so it fit even better. To be honest, it's been the bane of my existence - beyond the Hormel jokes, every time I sign up for anything I get an email before I'm approved asking me if I'm a spammer, or if I'm serious. Would a spammer truly call themselves Spamgirl? Uhm, no. I try to use antispamgirl now, as it's more appropriate, but I always forget and get locked out of my accounts when I try to log in.
What's it like being [presumably] female in the role of an internet forum moderator for a microtasking site. This is an area traditionally occupied by males. Or isn't it?
LOL Yes, female. I've been running forums for around a decade (RealityBBQ was mine until I sold it a few years ago, another site called KinzFans had a booming forum I ran) and on both it was mainly females running the show. When it comes to mTurk, I think the stats bear out a pretty even ratio of males and females working. That said, all the rest of the staff here (that are active, at least) are male. There are lots of female posters, though. I think it's pretty even, but I've never had a problem battling the boys so I haven't ever thought of it before.
Were you a Turker first? Are you still a Turker? What's your typical Turking like?
I've been a turker since November of 2005. I still turk, way too many hours a day, far too many days a week. I spend my life at my computer - at least until my husband finds a job and we can afford for me to take some breaks.
Tell me about any problems you have had as a Turker. What are the biggest complaints you would 'personally' raise?
Oh lordie, where do we start... international workers need to be paid in cash. Period. I know it's a huge problem, but every other site like this does it. It's ridiculous that after 5 years (happy anniversary, Spamgirl!) they still don't. I wish I knew why, I'd personally pledge to figure out a way around their problems.
No requester accountability for rejections, the amount of shitty requesters, no worker/requester forum, lack of communication between mTurk/requesters/workers, no way to rate requesters (and ditto for workers), insufficient help docs for workers and requesters... mTurk knows my complaints, they have a list
What are the biggest complaints you hear on the forum?
International payments, bad requesters ("spammers" and blackhats), being suspended with no recourse, no mediation between requesters and workers, no bonus tracking in My Account, and no way to know why your HIT was rejected.
If you could go into Amazon's system for a day and have free reign to prioritize their development schedule, what 5 changes to Mechanical Turk would you put at the top of the list?
1. International payments
2. Disallow new worker accounts for x months (i.e. until new workers are needed)
3. New requesters must put down $100 into their account via BANK ACCOUNT (NOT CC) before they're permitted to put up any HITs, and do a phone number confirmation, and submit a driver's license
4. Create a forum so requesters and workers can communicate (we'd still have this one because I'm sure the freedom on an official forum to speak your mind would be limited, and that's fair)
5. Create a mediation area of some sort so workers and requesters could communicate about blocks/rejections/etc. and mTurk would be better advised on bad requesters (and workers!)
read on to part 2 (of 3)
Where did you get the name Spamgirl? Why go there?
When I first started on the internet (oh man, it was TELNET LOL I'm old) I had a different name. Then my daughter was born and, well... she stole it from me and decided to use it for herself. I needed a new name, I was literally nameless. I went through a bazillion (Starla, Sunburnt Raisin, gosh I can't even remember them all) and then one day my friend was commenting on my anti-UCE activities and called me "The Spamgirl". It stuck. I also had a website at the time called "Sensible People Against Misinformation", so it fit even better. To be honest, it's been the bane of my existence - beyond the Hormel jokes, every time I sign up for anything I get an email before I'm approved asking me if I'm a spammer, or if I'm serious. Would a spammer truly call themselves Spamgirl? Uhm, no. I try to use antispamgirl now, as it's more appropriate, but I always forget and get locked out of my accounts when I try to log in.
What's it like being [presumably] female in the role of an internet forum moderator for a microtasking site. This is an area traditionally occupied by males. Or isn't it?
LOL Yes, female. I've been running forums for around a decade (RealityBBQ was mine until I sold it a few years ago, another site called KinzFans had a booming forum I ran) and on both it was mainly females running the show. When it comes to mTurk, I think the stats bear out a pretty even ratio of males and females working. That said, all the rest of the staff here (that are active, at least) are male. There are lots of female posters, though. I think it's pretty even, but I've never had a problem battling the boys so I haven't ever thought of it before.
Were you a Turker first? Are you still a Turker? What's your typical Turking like?
I've been a turker since November of 2005. I still turk, way too many hours a day, far too many days a week. I spend my life at my computer - at least until my husband finds a job and we can afford for me to take some breaks.
Tell me about any problems you have had as a Turker. What are the biggest complaints you would 'personally' raise?
Oh lordie, where do we start... international workers need to be paid in cash. Period. I know it's a huge problem, but every other site like this does it. It's ridiculous that after 5 years (happy anniversary, Spamgirl!) they still don't. I wish I knew why, I'd personally pledge to figure out a way around their problems.
No requester accountability for rejections, the amount of shitty requesters, no worker/requester forum, lack of communication between mTurk/requesters/workers, no way to rate requesters (and ditto for workers), insufficient help docs for workers and requesters... mTurk knows my complaints, they have a list
What are the biggest complaints you hear on the forum?
International payments, bad requesters ("spammers" and blackhats), being suspended with no recourse, no mediation between requesters and workers, no bonus tracking in My Account, and no way to know why your HIT was rejected.
If you could go into Amazon's system for a day and have free reign to prioritize their development schedule, what 5 changes to Mechanical Turk would you put at the top of the list?
1. International payments
2. Disallow new worker accounts for x months (i.e. until new workers are needed)
3. New requesters must put down $100 into their account via BANK ACCOUNT (NOT CC) before they're permitted to put up any HITs, and do a phone number confirmation, and submit a driver's license
4. Create a forum so requesters and workers can communicate (we'd still have this one because I'm sure the freedom on an official forum to speak your mind would be limited, and that's fair)
5. Create a mediation area of some sort so workers and requesters could communicate about blocks/rejections/etc. and mTurk would be better advised on bad requesters (and workers!)
read on to part 2 (of 3)
Spamgirl speaks! An interview (part 2 of 3)
continued from part 1.
You have the unique position as being a sort of mythical figure able to cross the customer-service moat and pass the automated banning dragons and actually speak with a human being at Amazon. How'd you get that gig?
I'm a loudmouth, so I was one of a few picked to do some surveys (I think, this is from memory and my memory is poor)... like focus groups. From there I struck up a conversation with the person at Amazon running the groups, and it went from there.
You've commented that a lot of the people you help through the TurkerNation forum deserved their blocks? Can you explain that. Has that changed recently?
Yes, it has changed, but only because I've made it blatantly clear that I'm not going to suffer fools. The same number of LEGITIMATE people are coming to me with bans, and the same number are being reinstated. I'd rather mTurk do something and screw up sometimes than do nothing and we all get screwed.
Specifically, and this is really the main issue: What must Amazon do to make its human resources management fair?
They can't. They don't have an applications process - they certainly can't afford to go through each worker and determine if they're good or bad, and even if they tried it wouldn't be accurate - so they're going to get more bad than good, it's natural. The only thing I'd recommend is some sort of appeals process as I can't imagine it's that hard to look at someone's work history and determine if their ban was a mistake or not (HIRE ME, AMAZON! I'LL DO IT!
Are you paid for your work with Amazon? What do you know that we don't? What kinds of things aren't you allowed to tell us?
ROFL *kisses*
On a scale of one to ten, how aware is Amazon of the issues that people raise on this forum (ten being more aware)?
100. They have spies, you know, who read this forum all the time. Plus I'm a bug in their ear about all this stuff... I'm the Hoffa of the Turkers! Trying to help the people (at least, the ones worth helping. Screw the scammers!)
On a scale of one to ten, how legitimate do you think those issues are, one being 'Turker Nation is a bunch of whiners' and ten being 'There are families starving because of poor user interface design'?
6ish. I think people tend to forget that Amazon and mTurk are separate. mTurk doesn't have a bottomless spending account, and the more we push on certain costly issues, the more likely mTurk will be shut down for not being profitable. We have to weigh the issues on a personal basis - what is intolerable? Where can I bend? Is it worth it for me to turk on eggshells or should I just walk? If it's truly causing upset, then it's best to just move on. That said, there is nothing wrong with a little protest, but we have to know that what we say and do has to be constructive and written professionally for mTurk to truly take us seriously. Whining, bitching, cussing (yeah, I know I cuss a lot LOL), flaming, and attacking don't get mTurk's attention. Petitions don't work, either, because (in my experience) Amazon employees don't like being told what to do (this isn't a personal thing, but seems like a general company policy.) The only "protest" that WOULD work would be the bulk of GOOD Turkers walking away from it, but that will *never* happen. If you have a complaint, post it - if there isn't already a thread for it, make a new one with a very clear title. They'll read it, and if you post a valid, intelligent argument, they WILL take it under advisement.
I know they're working hard on making it better for us, but I have a feeling that the infrastructure they have in place is ... uhm... not built well. I think it's probably hard for them to make major changes without crashing the entire system, I doubt it was built for expansion.
How would you describe the position that Mechanical Turk occupies in the Amazon.com universe? Do you think that Amazon ignores Mechanical Turk, and if so, do they do it intentionally? If they do it intentionally, why? Do they treat the site like a technological experiment, or is it just not that profitable, or is the profit sufficient but not worth further investment, or are they just waiting to blow our minds with MTurk 2.0 some time next year.
I honestly have no idea how the two connect... Amazon seems to know nothing about mTurk, and mTurk is it's own little planet to itself. I personally think Bezos must have some sort of stake in it for it to still be open (I felt the same way about Askville, but now I see it's just an SEO tool.)
Have you spoken to Jeff Bezos? Seriously.
LOL No, never I don't know if he knows what mTurk is!
What's your HIT approval rate? Have you ever been blocked? This is a little zen, but bear with me: Is there such a thing as a Mechanical Turk ban that even Spamgirl cannot unblock?
My HIT approval rate is 97.6%, further proof that a high approval rate doesn't mean anything. I had a HUGE number of HITs rejected back in the days of Enrovia, and everyone but mine were fixed (I was very vocal about my feelings about Enrovia.) So out of the 5000 rejections I have, I'd guess 4000 were from that situation. So I really have a 99.5% rejection rate, but what can I do?
I've never received an email saying I was blocked, but I don't dabble in unknown waters. I only do HITs for people with a smiley and many comments in the Hall of Shame/Fame.
There are bans I can't get unbanned, but they're the ones that were deserved and I wouldn't try to get them unbanned anyway. So I suppose no, never have I vouched for someone who didn't get their account back. I only vouch for people I know to be good workers, though. I submit an appeal for everyone else
How much contact do you have with other crowdsourcing operations, like CrowdFlower's Dolores Labs or the transcription service Casting words?
None. When I did Castingwords HITs I talked to them a smidge, but typically only about problems. I've never spoken with the people at Turkopticon, or Dolores Labs, or any requesters I'm not complaining to. Requesters ask me questions here, of course, but it tends not to go deeper. The only relationships I have are with Amazon Requester/mTurk, and even then it's only for certain HITs.
read on to part 3 (of 3)
You have the unique position as being a sort of mythical figure able to cross the customer-service moat and pass the automated banning dragons and actually speak with a human being at Amazon. How'd you get that gig?
I'm a loudmouth, so I was one of a few picked to do some surveys (I think, this is from memory and my memory is poor)... like focus groups. From there I struck up a conversation with the person at Amazon running the groups, and it went from there.
You've commented that a lot of the people you help through the TurkerNation forum deserved their blocks? Can you explain that. Has that changed recently?
Yes, it has changed, but only because I've made it blatantly clear that I'm not going to suffer fools. The same number of LEGITIMATE people are coming to me with bans, and the same number are being reinstated. I'd rather mTurk do something and screw up sometimes than do nothing and we all get screwed.
Specifically, and this is really the main issue: What must Amazon do to make its human resources management fair?
They can't. They don't have an applications process - they certainly can't afford to go through each worker and determine if they're good or bad, and even if they tried it wouldn't be accurate - so they're going to get more bad than good, it's natural. The only thing I'd recommend is some sort of appeals process as I can't imagine it's that hard to look at someone's work history and determine if their ban was a mistake or not (HIRE ME, AMAZON! I'LL DO IT!
Are you paid for your work with Amazon? What do you know that we don't? What kinds of things aren't you allowed to tell us?
ROFL *kisses*
On a scale of one to ten, how aware is Amazon of the issues that people raise on this forum (ten being more aware)?
100. They have spies, you know, who read this forum all the time. Plus I'm a bug in their ear about all this stuff... I'm the Hoffa of the Turkers! Trying to help the people
(at least, the ones worth helping. Screw the scammers!)On a scale of one to ten, how legitimate do you think those issues are, one being 'Turker Nation is a bunch of whiners' and ten being 'There are families starving because of poor user interface design'?
6ish. I think people tend to forget that Amazon and mTurk are separate. mTurk doesn't have a bottomless spending account, and the more we push on certain costly issues, the more likely mTurk will be shut down for not being profitable. We have to weigh the issues on a personal basis - what is intolerable? Where can I bend? Is it worth it for me to turk on eggshells or should I just walk? If it's truly causing upset, then it's best to just move on. That said, there is nothing wrong with a little protest, but we have to know that what we say and do has to be constructive and written professionally for mTurk to truly take us seriously. Whining, bitching, cussing (yeah, I know I cuss a lot LOL), flaming, and attacking don't get mTurk's attention. Petitions don't work, either, because (in my experience) Amazon employees don't like being told what to do (this isn't a personal thing, but seems like a general company policy.) The only "protest" that WOULD work would be the bulk of GOOD Turkers walking away from it, but that will *never* happen. If you have a complaint, post it - if there isn't already a thread for it, make a new one with a very clear title. They'll read it, and if you post a valid, intelligent argument, they WILL take it under advisement.
I know they're working hard on making it better for us, but I have a feeling that the infrastructure they have in place is ... uhm... not built well. I think it's probably hard for them to make major changes without crashing the entire system, I doubt it was built for expansion.
How would you describe the position that Mechanical Turk occupies in the Amazon.com universe? Do you think that Amazon ignores Mechanical Turk, and if so, do they do it intentionally? If they do it intentionally, why? Do they treat the site like a technological experiment, or is it just not that profitable, or is the profit sufficient but not worth further investment, or are they just waiting to blow our minds with MTurk 2.0 some time next year.
I honestly have no idea how the two connect... Amazon seems to know nothing about mTurk, and mTurk is it's own little planet to itself. I personally think Bezos must have some sort of stake in it for it to still be open (I felt the same way about Askville, but now I see it's just an SEO tool.)
Have you spoken to Jeff Bezos? Seriously.
LOL No, never
I don't know if he knows what mTurk is!What's your HIT approval rate? Have you ever been blocked? This is a little zen, but bear with me: Is there such a thing as a Mechanical Turk ban that even Spamgirl cannot unblock?
My HIT approval rate is 97.6%, further proof that a high approval rate doesn't mean anything. I had a HUGE number of HITs rejected back in the days of Enrovia, and everyone but mine were fixed (I was very vocal about my feelings about Enrovia.) So out of the 5000 rejections I have, I'd guess 4000 were from that situation. So I really have a 99.5% rejection rate, but what can I do?
I've never received an email saying I was blocked, but I don't dabble in unknown waters. I only do HITs for people with a smiley and many comments in the Hall of Shame/Fame.
There are bans I can't get unbanned, but they're the ones that were deserved and I wouldn't try to get them unbanned anyway. So I suppose no, never have I vouched for someone who didn't get their account back. I only vouch for people I know to be good workers, though. I submit an appeal for everyone else
How much contact do you have with other crowdsourcing operations, like CrowdFlower's Dolores Labs or the transcription service Casting words?
None. When I did Castingwords HITs I talked to them a smidge, but typically only about problems. I've never spoken with the people at Turkopticon, or Dolores Labs, or any requesters I'm not complaining to. Requesters ask me questions here, of course, but it tends not to go deeper. The only relationships I have are with Amazon Requester/mTurk, and even then it's only for certain HITs.
read on to part 3 (of 3)
Spamgirl speaks! An interview (part 3 of 3)
continued from part 2.
Personal Stuff
*How old are you? (definitely skip that one. rude!)
I'm in my 30s
*Where are you from? Where do you live now?
Toronto, born and bred.
*Do you have a college degree?
Ish... I have a certification in Web Design from a college, but I hate web design so it's expensive paper.
*Are you married?
Yups
*Does your family know about your Turking? What do they think about it?
ROFL! How could they not They've been bitter about it for years, but understand it's what brings in the cheddar. My husband had to buy a new cell phone 2 days ago after his got smushed by a tractor (*sigh*) and I said GREAT! THAT'S WHAT, A WEEK OF TURKING? He had no response.
*What have you purchased with your Turking earnings?
Uhm, other than the naughty stuff? LOL Everything from an Xbox 360 to garbage bags to clothing to bath stuff. I've spent thousands in my years, and I have a house full of Turkin' profits to show for it
*How would you describe the economic role of your Turking: lama skins - spa trips - ipads - restaurant dinners - credit card bills - electricity bills - food?
At the moment it's the only thing keeping us afloat. My husband lost his job in April and I haven't had a "real" job in a year (yesterday! Happy anniversary, Spamgirl, again.) I have medical issues that prevent me from working outside my home, so without mTurk... we'd be foreclosed and homeless. Before my husband lost his job it was a way to buy gifts and necessities and even treats (Coach handbag! Wut!)
Big Questions
What do you think about the role of crowdsourcing in the future of employment?
I see it as fairly insignificant. I don't think most work can be done this way, and I think the work that can be completed is, for the most part, too cheap for most to bother wanting to do it.
Is crowdsourcing fair?
Is life fair? No job is fair, IMHO. It is fairer than others in that you're free to do what you want, when you want, or nothing at all. Quitting a job can be detrimental to your reputation and difficult to do.
Is crowdsourcing a 'good thing' for workers?
As above.
What responsibilities does Amazon have to the task posters and task workers? Is Amazon meeting them?
Amazon, mTurk, seems to hold the belief that they will build the system and that's the extent of what they'll do. They've built a requester area which has a forum, so requesters receive some help and some documentation, but they complain it isn't enough. mTurk can't hold everyone's hand, I understand that, but the problems seem to repeat themselves and could probably be solved with a little effort (bad workers, no communication, how do I email a worker, how do I create a qualification test easily, etc.)
mTurk has little responsibility when it comes to workers. They just don't have the funds or time to put into creating a communication system, but they do what they can - they even post on the forum as necessary. I can't say I EXPECT more from them, but more could certainly be done.
How have you seen crowdsourcing change since it hit the scene in the middle of last decade?
I only know how it's changed at mTurk, as I don't visit elsewhere. In the beginning it was the wild frontier - few workers, lots of work, and experimentation. All the HITs were valid, even if the requesters were greedy baddies. Since then it's become rife with black hats and scammers who post on forums about how to screw us. It has also been inundated with non-English speakers who think their English is good enough to do writing HITs, which is a major complaint by requesters.
In ten years, what do you think you'll be doing in relation to Mechanical Turk?
Praise God, may I not even remember the place by then! If I'm still kickin' in 10 years, I'll have won the lottery and moved on I love mTurk, and I love the work I do, and I appreciate everything they have done for me, but I have to work so hard to make enough just to EAT... I can't wait until the debt is paid off, hubby gets a job, and I can spend some time with my family. I'm a little burnt out.
That's it! thank Spamgirl for her dedicated service and leave a comment or stop by the Turker Nation forum to let her know whatever you're thinking. Maybe she'll field follow up questions here, so leave 'em if you got 'em.
Personal Stuff
*How old are you? (definitely skip that one. rude!)
I'm in my 30s
*Where are you from? Where do you live now?
Toronto, born and bred.
*Do you have a college degree?
Ish... I have a certification in Web Design from a college, but I hate web design so it's expensive paper.
*Are you married?
Yups
*Does your family know about your Turking? What do they think about it?
ROFL! How could they not
They've been bitter about it for years, but understand it's what brings in the cheddar. My husband had to buy a new cell phone 2 days ago after his got smushed by a tractor (*sigh*) and I said GREAT! THAT'S WHAT, A WEEK OF TURKING? He had no response.*What have you purchased with your Turking earnings?
Uhm, other than the naughty stuff? LOL Everything from an Xbox 360 to garbage bags to clothing to bath stuff. I've spent thousands in my years, and I have a house full of Turkin' profits to show for it
*How would you describe the economic role of your Turking: lama skins - spa trips - ipads - restaurant dinners - credit card bills - electricity bills - food?
At the moment it's the only thing keeping us afloat. My husband lost his job in April and I haven't had a "real" job in a year (yesterday! Happy anniversary, Spamgirl, again.) I have medical issues that prevent me from working outside my home, so without mTurk... we'd be foreclosed and homeless. Before my husband lost his job it was a way to buy gifts and necessities and even treats (Coach handbag! Wut!)
Big Questions
What do you think about the role of crowdsourcing in the future of employment?
I see it as fairly insignificant. I don't think most work can be done this way, and I think the work that can be completed is, for the most part, too cheap for most to bother wanting to do it.
Is crowdsourcing fair?
Is life fair? No job is fair, IMHO. It is fairer than others in that you're free to do what you want, when you want, or nothing at all. Quitting a job can be detrimental to your reputation and difficult to do.
Is crowdsourcing a 'good thing' for workers?
As above.
What responsibilities does Amazon have to the task posters and task workers? Is Amazon meeting them?
Amazon, mTurk, seems to hold the belief that they will build the system and that's the extent of what they'll do. They've built a requester area which has a forum, so requesters receive some help and some documentation, but they complain it isn't enough. mTurk can't hold everyone's hand, I understand that, but the problems seem to repeat themselves and could probably be solved with a little effort (bad workers, no communication, how do I email a worker, how do I create a qualification test easily, etc.)
mTurk has little responsibility when it comes to workers. They just don't have the funds or time to put into creating a communication system, but they do what they can - they even post on the forum as necessary. I can't say I EXPECT more from them, but more could certainly be done.
How have you seen crowdsourcing change since it hit the scene in the middle of last decade?
I only know how it's changed at mTurk, as I don't visit elsewhere. In the beginning it was the wild frontier - few workers, lots of work, and experimentation. All the HITs were valid, even if the requesters were greedy baddies. Since then it's become rife with black hats and scammers who post on forums about how to screw us. It has also been inundated with non-English speakers who think their English is good enough to do writing HITs, which is a major complaint by requesters.
In ten years, what do you think you'll be doing in relation to Mechanical Turk?
Praise God, may I not even remember the place by then!
If I'm still kickin' in 10 years, I'll have won the lottery and moved on I love mTurk, and I love the work I do, and I appreciate everything they have done for me, but I have to work so hard to make enough just to EAT... I can't wait until the debt is paid off, hubby gets a job, and I can spend some time with my family. I'm a little burnt out.That's it! thank Spamgirl for her dedicated service and leave a comment or stop by the Turker Nation forum to let her know whatever you're thinking. Maybe she'll field follow up questions here, so leave 'em if you got 'em.
No-interference doesn't mean no-standards
Here's a quote:
What that quote shows off is the current slack excuse that Amazon uses to justify their lack of a coherent, common-sense, responsive policy. But as the poster highlights, it doesn't matter what happens between a Turker and a Requester until Amazon authorizes those actions as reasons for full site suspensions. This blog is not about defending bad workers; they exist and shouldn't be on the site. But when good workers are caught up in the same net and have to scrape and grovel to get back on to the site and work, it's a problem.
mturk wants it both ways- they say they're not involved, and it's between us and the requesters, yet they suspend our accounts for blocks. linkIt's from http://www.turkernation.com, the ProBoards forum where Turkers post about their experiences on MTurk. By and large the experiences are positive and the people there are eager for and appreciate good work and fair practices. For the record, this blog has no affiliation with that site.
What that quote shows off is the current slack excuse that Amazon uses to justify their lack of a coherent, common-sense, responsive policy. But as the poster highlights, it doesn't matter what happens between a Turker and a Requester until Amazon authorizes those actions as reasons for full site suspensions. This blog is not about defending bad workers; they exist and shouldn't be on the site. But when good workers are caught up in the same net and have to scrape and grovel to get back on to the site and work, it's a problem.
A reasonable idea, a telling commentary
This comment from the Turker Nation forum seems both reasonable and very telling:
I never understood why messages sent between worker and requesters were not done through Mturk. It would be much easier to log in and get your messages. I would also like to see blocks listed the way qualifications are listed. This way, workers know who has blocked them and how many blocks they have.
If we eliminate the issue of human decency and look at this purely from a business standpoint, Mturk is hurting itself with the way it is handling blocks and suspensions. Mturk gets a percentage from requesters for each HIT workers do. I understand they want to make sure requesters get good work so they continue to use Mturk. However, their current approach will not always get rid of bad workers. As many have said, scammers will use multiple accounts and computers. On the other hand, they are losing some good workers. Many workers sometimes use their earnings to buy things from Amazon. Each worker has family and friends who buy from Amazon. It is not good business to treat workers poorly. link
There's more where this came from
Spamgirl says:
I've been working for a long time just to get mTurk to set up a Requester/Turker communication portal of some sort, and they are all for it and yet it hasn't materialized. They don't have the people power nor the funding to add much to the site that doesn't directly increase revenue by pleasing requesters.
Like you said, it takes an enormous effort to get mTurk/Amazon to make changes based on Turker feedback. I've been working with them as much as possible and honestly, they're doing what they can with what they have and I respect their efforts... but the things we're asking for are relatively huge, and the platform mTurk is built on, code wise, is shaky. For example, it's broken right now! Who knows why...
I've slammed my head against the mTurk wall enough times to know that after a while it's not worth having lofty goals link
Exactly
A spot-on commentary:
This is ridiculous. I've only been turking for a couple of months (actually today is my two month anniversary), but I find it so hard to believe that a gigantic, well respected company like Amazon doesn't have a better system in place for Mturk.Yup. Exactly.
There is a severe lack of controls. At the very, very least, there should be a system in place to dispute a block or a ban with both sides submitting their sides of their story. It's a joke that Amazon doesn't look at a user's history and/or percentage rating before blocking someone's only source of income. link
Letter from a Turker to Amazon
Posted on the boards:
Hi,
I'm writing about a recent spate of account bans that have been reported at the turkernation.com forums. As you probably know, this site collects some of the most active, energetic, passionate, and committed users of MTurk.
Shockingly, many of them have been banned in the past few months because of the mistakes of only a few requesters. In the most recent case a user who had done 60k HITS with a 99.1% approval rating, and conservatively worked for the same few posters found herself banned and blocked.
Again, one of your MOST committed users is suddenly out of a job and without recourse except a polite letter from MTurk that you'll look into it later.
I don't know what's worse, that the people who might be your best advocates are getting spuriously banned or that Amazon seems to think this is OK! You are the world's preeminent customer service company and the level of customer service for the workers on this site is close to pathetic.
And I like the site! link
Too little, too late: The new Amazon way?
Amazon.com has set a standard for customer service that is literally etched into their corporate mission. So how do they explain this:
I've said it before on this site, but it needs repeating. Amazon makes the banning policy. Amazon issues the suspensions. Anything else is a deflection. "I understand that a future change in the program would not be of any assistance to you now, however it may stop another worker from having the same experience that you had." Does this sound like a company that cares about its users and is committed to offering the best technology and customer service?
How about they pull rank, apologize, reinstate the account immediately, apologize for the inconvenience, issue a bonus credit, and then actually fix the system. Again, this happens to long-time Turkers with approval ratings in the high 90's. That's like taking away someone's driver's license permanently, because of three speeding tickets... in their whole life!
Hello,
I am truly sorry for your concern about being blocked by a Requester, and I would be happy to help you. However, as stated on our site, all approvals, rejections, and blocks are at the sole discretion of the Requester.
I have passed along some feedback regarding implementing a dispute process for workers that feel that they should not have been blocked. I understand that a future change in the program would not be of any assistance to you now, however it may stop another worker from having the same experience that you had.
This is something that you will need to work out with the Requester directly. For your reference, here's how to contact the Requester again:
1. Click on the title of the HIT.
2. On the lower right-hand side of the box, click "Contact the Requester of this HIT"
3. Enter your question about the HIT and submit.
After you have sent your message, you may need to wait a few days for a reply. Some Requesters do not monitor their messages every day.
If you have any other questions or concerns, please use the following secure link to contact us back to ensure that we receive your message:
http://www.mturk.com/mturk/contactus
I apologize for the frustration that this experience has caused you, and thanks for participating in Mechanical Turk. link
I've said it before on this site, but it needs repeating. Amazon makes the banning policy. Amazon issues the suspensions. Anything else is a deflection. "I understand that a future change in the program would not be of any assistance to you now, however it may stop another worker from having the same experience that you had." Does this sound like a company that cares about its users and is committed to offering the best technology and customer service?
How about they pull rank, apologize, reinstate the account immediately, apologize for the inconvenience, issue a bonus credit, and then actually fix the system. Again, this happens to long-time Turkers with approval ratings in the high 90's. That's like taking away someone's driver's license permanently, because of three speeding tickets... in their whole life!
Bad customer service is never funny
The ban-monster strikes again:
As always, this blog is in no way affiliated with Turkernation.com or its posters.
Haha, this is pretty funny. Today I received two emails from Amazon apologizing for the warning emails, apparently only the first of three warnings were valid, the other two were Amazon mistakes. They say that the account has been reactivated... we'll see if they follow through.What does it mean when the world's premier technology corporation, which prides itself on customer service, lets this stand as the example for their flagship crowdsourcing platform? I think it means they're not paying attention.
EDIT: In true Amazon fashion, my account was never reactivated... and I still have no way of contacting them. I posted in the requester forum with no luck whatsoever. I honestly don't know how Amazon is in business, their support is by far the worst I have ever seen in any business yet. link
As always, this blog is in no way affiliated with Turkernation.com or its posters.
Subscribe to:
Posts (Atom)