<<Back to RESEARCH

Crowdsourcing Optimization

Crowdsourcing is a promising form of internet-based service provision, where customers outsource tasks to a crowd of human workers.

I focus on expert crowdsourcing, which involves complex tasks, like knowledge synthesis, product design and idea generation, and workers with specialized skills, such as content writing or innovation capability. ​A basic problem here is: How can we provide customers with performance guarantees (high task quality, adherence to budget and timeliness), while ensuring that the workers will be allocated to the tasks according to their individual skills and preferences?

To ensure these performance guarantees, I develop people-to-task matching mechanisms, which combine machine scheduling, traditionally found in operational research, with non-deterministic behavioral user modelling, which captures the uncertainty of human behavior based on statistical analyses of the crowdsourcing population.

I experimentally validate the developed mechanisms through crowd simulations and real-user experiments. This line of research has practical implications for ensuring Quality of Service in crowd work platforms, allowing them to offer cost, quality and time guarantees for expert tasks.

Related Publications

It's about time: Online Macrotask Sequencing in Expert Crowdsourcing​

Schmitz, H., Lykourentzou, I., (2016), ArXiv e-prints, arXiv: 1601.04038

In this work we introduce the problem of Task Assignment and Sequencing (TAS), which adds the timeline perspective to expert crowdsourcing optimization. Current studies do not take into account the element of time, assuming crowdsourcing models with known worker and task arrivals. Realistically however, time is critical: tasks have deadlines, expert workers are available only at specific time slots, and arrivals are not known a-priori. Our work is the first to address the problem of optimal task sequencing for online, heterogeneous, time-constrained macrotasks.

​We propose TAS-Online, an online algorithm that aims to complete as many tasks as possible within budget, required quality and a given timeline, without future input information regarding job release dates or worker availabilities. Results, comparing the algorithm to four typical benchmarks, show that it achieves more completed jobs, lower flow times and higher job quality.

Task assignment optimization in knowledge-intensive crowdsourcing

Basu Roy S., Lykourentzou I., Thirumuruganathan S., Amer-Yahia S., Das G., (2015),
​VLDB Journal, 24(4), 467-491.

This work formalizes the problem of worker-to-task assignment in knowledge-intensive crowdsourcing (KI-C) for the first time as an optimization problem. We present a theoretical analysis of the KI-C task assignment optimization problem and propose optimal and approximation algorithms with guarantees to solve it. The algorithms rely on task assignment index pre-computation and adaptive maintenance, and they take into account human factors, such as worker expertise, wage requirements, and availability. Experiments with real and simulated data show that the proposed algorithms can echieve efficient task assignments of high-quality under guaranteed cost.

Related Projects

aCCoRdO: Computational methods for human use optimization in complex crowdsourcing 
​Funding: National Research Fund of Luxembourg (FNR)