Stage-oe-small.jpg

Inproceedings3179: Unterschied zwischen den Versionen

Aus Aifbportal
Wechseln zu:Navigation, Suche
(Die Seite wurde neu angelegt: „{{Publikation Erster Autor |ErsterAutorNachname=Bermbach |ErsterAutorVorname=David }} {{Publikation Author |Rank=2 |Author=Robert Kern }} {{Publikation Author |Ra…“)
 
 
Zeile 30: Zeile 30:
 
{{Publikation Details
 
{{Publikation Details
 
|Abstract=Micro-task markets like Amazon MTurk enable online workers to provide human intelligence as Web-based on demand services (so called "people services"). Businesses facing large amounts of knowledge work can benefit from increased flexibility and scalability of their workforce but need to cope with reduced control of result quality. While this problem is well recognized, it is so far only rudimentarily addressed by existing platforms and tools. In this paper, we present a flexible research toolkit which enables experiments with advanced quality management mechanisms for generic micro-task markets. The toolkit enables control of correctness and performance of task fulfillment by means of dynamic sampling, weighted majority voting and worker pooling. We demonstrate its application and performance for an OCR scenario building on Amazon MTurk. The toolkit however enables the development of advanced quality management mechanisms for a large variety of people service scenarios and platforms.
 
|Abstract=Micro-task markets like Amazon MTurk enable online workers to provide human intelligence as Web-based on demand services (so called "people services"). Businesses facing large amounts of knowledge work can benefit from increased flexibility and scalability of their workforce but need to cope with reduced control of result quality. While this problem is well recognized, it is so far only rudimentarily addressed by existing platforms and tools. In this paper, we present a flexible research toolkit which enables experiments with advanced quality management mechanisms for generic micro-task markets. The toolkit enables control of correctness and performance of task fulfillment by means of dynamic sampling, weighted majority voting and worker pooling. We demonstrate its application and performance for an OCR scenario building on Amazon MTurk. The toolkit however enables the development of advanced quality management mechanisms for a large variety of people service scenarios and platforms.
 
+
|Download=CSP-WMV-Paper.pdf,
 
|Forschungsgruppe=Ökonomie und Technologie der eOrganisation
 
|Forschungsgruppe=Ökonomie und Technologie der eOrganisation
 
}}
 
}}

Aktuelle Version vom 2. September 2011, 08:29 Uhr


An Extendable Toolkit for Managing Quality of Human-based Electronic Services


An Extendable Toolkit for Managing Quality of Human-based Electronic Services



Published: 2011 August

Buchtitel: Proceedings of the 3rd Human Computation Workshop (HCOMP 2011)
Verlag: AAAI Press
Organisation: AAAI

Referierte Veröffentlichung

BibTeX

Kurzfassung
Micro-task markets like Amazon MTurk enable online workers to provide human intelligence as Web-based on demand services (so called "people services"). Businesses facing large amounts of knowledge work can benefit from increased flexibility and scalability of their workforce but need to cope with reduced control of result quality. While this problem is well recognized, it is so far only rudimentarily addressed by existing platforms and tools. In this paper, we present a flexible research toolkit which enables experiments with advanced quality management mechanisms for generic micro-task markets. The toolkit enables control of correctness and performance of task fulfillment by means of dynamic sampling, weighted majority voting and worker pooling. We demonstrate its application and performance for an OCR scenario building on Amazon MTurk. The toolkit however enables the development of advanced quality management mechanisms for a large variety of people service scenarios and platforms.

Download: Media:CSP-WMV-Paper.pdf



Forschungsgruppe

Ökonomie und Technologie der eOrganisation


Forschungsgebiet