About a year ago I wrote a blog post called Tagging Hall of Shame: Amazon. It pointed out that tagging books as “good” and “for dad” was not very helpful.
Well I am pretty impressed at this new step Amazon has taken. Amazon is running Amazon Mechanical Turk, a project in “artificial artificial intelligence.” Users (or “turkers”) perform small tasks for small pay. The tasks, called HITs (Human Intelligence Tasks), pay from 1 cent for answering simple questions to several dollars for things like transcribing podcasts. Anyone can make a HIT and anyone can sign up to answer them.
I’ve found this to be a profitable thing to do while sitting in front of the TV. Today, I ran into a task put forward by Amazon. They pay $0.01 for a user to look at a tag and a book, say if the tag is relavent or not, and (if the turker likes) suggest other tags. When I saw this, nearly 1,000 items were available to be checked.
This is a great idea! I will very happily take $0.01 to say “gift” is a bad tag. It’s a quick easy way to get rid of those bad tags, develop new ones, and get them community checked for a relatively low cost. If this improves the search on Amazon because tags become effective, then I think the very small investment will really pay off. Kudos to Amazon for creating mTurk and for using it to do this.
CLARIFICATION: Some commenters have been arguing that personal tags are useful. I agree. I mean - Duh. However, I am writing exclusively about collaborative tagging. Amazon was originally put into my hall of shame for sharing personal tags with the idea of using them for collaborative tagging. A review of the tags that were available on Amazon before showed they were all very personal, and thus completely useless in the collaborative sense. Note also that I’m not saying that I even believe collaborative tagging will work in the ideal world, but it certainly won’t work if all I see are other people’s personal notes.
I’d love to hear your comments on the evolution of collaborative tagging (including if you think it will even work), or on mTurk.
Won't it create a new business where companies use this service to increase their product placement within Amazon itself?
If that's so, I doubt it will improve the community as you suggest .
I think they should pay $0.02.
Let's say I give a tag one minute of thought (really less because if I'm doing it while watching Buffy whop on Spike yet again, I'll be distracted by her for at least ten seconds of that, distracted by the Cheetos for another ten seconds, my Diet Coke for another ten seconds, petting the Sheltie for a good 20 seconds, the wife and kids for another ten, so maybe I'll give it ten seconds while breathing deeply), maybe I can do sixty tags an hour. The quality of that thought won't be great, but I can choose yes or no or maybe. At the end of that hour, I am a bit fatter, the dog is happier, the wife and kids are the same, and I have $1.20. An $1.20 an hour for opinions about tags? That won't even pay for the Cheetos. This doesn't seem like much money, but heck, if Amazon wants my two cents on any topic, it's better than doing it for free which is what most user-generated content pays.
This is the most unhelpful thing. The problem is the tag it not for you. The tag is for the person that placed it there. By removing it you have utterly broken that person's ability to get back to that object. Either this is an utter lack of understanding of personal tagging or it is outright mean.
Amazon has done a rather good job providing excellent pivots to see what other objects a person has tagged, see who has tagged the item. etc. Amazon's approach to tagging is one of the more well rounded approaches to exploring tags.
Then it has to work like WikiPedia editing, Vanderwal.
The problem of group editing is choosing the members of the group. This has always been the problem. Then the issue is what is the agenda of the chooser of the choosers. The rules for selection have to be clear and unambiguous. Then the rules for the process of culling have to be clear and reasonably flexible. At that point, a machine can *almost* do it but not quite. So a well-selected, practiced and diligent team is needed depending on the value of the information. That won't come cheaply, thus the comment about putting in one's two cents worth.
The advantage of mTurk comes down purely to what Dr. Werhner Von Braun once said about human astronauts over fully-competerized space craft: "People are the best computers one can produce with unskilled labor."
As long as the value of the information is not high, say only important to that one author, then the mTurk approach can be justified by its cost. That that may be perceived as 'mean' is not part of the cost justification. Those are the values and one chooses one's choosers based on values. That is a two way street if the so-called 'collaborative' act is balanced. Otherwise, it is power over content as created by the customer and the branding of the service as collaborative is simply another web scam.
Tagging "good" and "dad" are incredibly helpful. Those are tags that mean a lot to ME. That's the value of these folksonomies: that they provide value to an individual first, and secondarily to the group. The purpose of tags on sites that do it well, like delicious or Flickr, is to let me categorize my things my way first. Actually, if I cannot tag a book "dad" how would I remember that it was something he'd recommended to me?
I hope that Amazon isn't using turk to prune out these personal tags in favor of some lowest-common-denominator effort at just getting more descriptive words for ad engines to attach to.
len, the point is not to reach one set of tags that are right. The point is that each and every tag is there for a reason for the person that placed the tag there. Tags, as Amazon is using them is personal. Every tag is sacred, and every person is an expert in their own vocabulary (what they call things and how they retrieve them).
People also add tags for social interaction, just look at the Kfed tag pages.
Unlike wikipedia which aims to get at one truth or a representative understanding of the truths on a subject, tagging by the consumer as it builds a folksonomy is about placing hooks for refindability and can also be for sociality.
Jennifer, how do you separate the personal tags from the collaborative tags? You can not, as the personal hooks that people place on items are there for personal and/or for social interaction. Amazon's interface already does show your tags for personal retrieval and it shares them, just as del.icio.us and most other social bookmarking tools show them. The research around collaborative tagging systems shows these systems work best when people are tagging for themselves. The personal tagging and the collaborative tagging are the same! Collaborative tagging with out personal interest is what Cory Doctorow appropriately labeled metacrap.
On Amazon there are many reasons that people are tagging, including irony and expression of taste. The brilliant thing that Amazon does is it shows an identity so you can start looking at other things an identity has tied to it. This is good as it will allow following people's taste you agree with or ignoring those you do not agree with.
This display of identity is what separated del.icio.us from all the tagging systems that came before it that were an utter mess. Having identity, object being tagged (by the consumer of the content), and tag was the concept I coined folksonomy around. You can have people who do not agree on taste (this is the core of Amazon's recommendation service) and having tagging interfaces that can discern tag likes and dislikes intelligently will be helpful. But, in the testing of the current tag interface that Amazon uses (I have been doing a bit of this, along with other tagging sites) people are finding other people's tags helpful, including the "gift for x" tags. People are consuming the tags are a wide variety of ways. The key is to discern how people are discerning tags and have a tool that can adapt to the various ways people use them.
Let me say in response that for many of the reasons you argue that this kind of personal-as-public tagging is good, I am utterly unconvinced that there is any benefit in doing it from the perspective of other users. I do not believe that there are "right" and "wrong" tags as you implied in your first comment. However, I do believe there are tags that are worthless to the general public. I think there are ways tags can work (using a social network / community filter), and when I have time I will write about that.
Basically, though, I think all those personal tags shared with everyone will converge to something that is no more useful than keywords that we could probably already get from somewhere else. I have yet to see anything that convinces me that "folksonomy" is anything beyond a cute buzzword.
So: Personal tags - obviously useful to a person. Everyone's personal tags all put together - not so useful. Everyone's personal tags all put together as something more useful than keywords: I totally don't buy it.
I agree that there are ways to make tagging more useful, much more useful, using a more granular social network, but to do that involves pairing understanding of interest and compatibility of the person tagging and the person reading the tags. When doing research around tagging I and most other researchers are initially surprised at what people find valuable and useful out in the long tail.
I just returned from a symposium of researchers looking at this exact topic and the terms they find of their own value and their subjects of research projects value regularly included items out at the tail. The distinction of understanding and community around variations of the tag "socialnetwork", social.network", "social_network", and "social-network" were very noticeable. If these get normalized to one of those solutions the communities break and there have been some heated discussions around this subject where people believe their is one true term. To nearly a person the tags were part of self selection based on definition derived from the object and the tag applied. People learned to use the tag that brought a community together.
I would love to hear how you can have collaborative tagging without personal tagging as nearly every paper I have read uses collaborative tagging as a result of personal tagging. All the tagging researchers at the symposium use that as their premise.
There is value that can be derived, if understanding the cost, when the single tags are not shown. But the scaling effects have not kicked in for most of Amazon's products that have been tagged.
There is value in the mTurk approach, but the value is mostly for the person that is rating the tag as valuable or not. The tag as applied to the object as a combination will help Amazon improve its modification for that person and others whose similar vocabulary and taste could be derived through collaborative filtering. This approach could be used to modify a "messy" interface of undesired tags.
Generalizing the value of the tag beyond that person reviewing or those who have similar tendencies would start damage the value that can be derived from the tagging.
This is about Mturk tasks, though not about tagging. I think the pay offered for some of the tasks are too low... for writing a 200 words comment, the task offer a mere $0.10 and there are several other similar postings too. These HITs could be an interesting diversions, but don't think could be treated as a 'profitable' venture!.