Introduction
In spite of the nebulousness inherent in determining and measuring expertise, crowdsourcing systems in research and practice that seek to acquire distributed knowledge (e.g. citizen science) indicate a preference for expert contributors. To ensure information quality (IQ), consumers – users of data acquired from such crowdsourcing projects – implement gatekeeping strategies that vary in degree of subtleness and intentionality. These gatekeeping strategies stem from the underlying premise of a positive correlation between IQ and contributor’s level of expertise in a given domain. The assumption therefore, is that expertise leads to better contributions and the expert crowd is the better crowd. A gatekeeping strategy popular in citizen …show more content…
A contributor’s level of expertise compared to others in the group is aptly termed “relative expertise”. According to Budescu and Chen (cite), relative expertise can be measured objectively, through the assignment of weights to contributor’s level of education, seniority, professional status and historical track record (e.g. Biros et al. 2002). In addition, expertise can be measured subjectively using ratings provided by the experts themselves or by others like their peers or superiors (e.g. Lukyanenko et al. 2014). However, measuring relative expertise by a contributor’s performance on tests similar to the intended task has been highlighted as the most efficient method (Clemen 2008, Lin and Cheng 2009, Budescu & Chen 2015). Accordingly, we also adopt the idea of performance-based measurements for expertise. In the next section, we explore classification theory as our overarching …show more content…
rule-based classification, or their instances i.e. exemplars and prototypes (Murphy 2002, Kloos & Sloutsky 2008). In rule-based classification, it is usually necessary to pay selective attention to only relevant features necessary for identifying instances of the class while ignoring irrelevant features (not useful for predicting class membership). Although selective attention leads to efficient learning, especially when making connections between instances with very sparse similarities and dense dissimilarities, it has its cost. The primary cost of selective attention is a learned inattention to features that are not “diagnostic” in the present context (Hoffman & Rehder 2010). If these features however become diagnostic in another context then the ability to make predictions and transfer knowledge is lost. Adapting Hoffman and Rehder’s example, an observer tasked with distinguishing rose bushes from raspberry bushes who considers the presence of berries as the most diagnostic feature may ignore any other feature of both plants (e.g. thorns and leaves). However, if the observer must later distinguish raspberry from cranberry bushes, thorns are suddenly diagnostic as both have red berries and only the raspberry has thorns. The observer may therefore have difficulty distinguishing both bushes due to selective attention. The cost of selective attention is also invariably a cost of
In spite of the nebulousness inherent in determining and measuring expertise, crowdsourcing systems in research and practice that seek to acquire distributed knowledge (e.g. citizen science) indicate a preference for expert contributors. To ensure information quality (IQ), consumers – users of data acquired from such crowdsourcing projects – implement gatekeeping strategies that vary in degree of subtleness and intentionality. These gatekeeping strategies stem from the underlying premise of a positive correlation between IQ and contributor’s level of expertise in a given domain. The assumption therefore, is that expertise leads to better contributions and the expert crowd is the better crowd. A gatekeeping strategy popular in citizen …show more content…
A contributor’s level of expertise compared to others in the group is aptly termed “relative expertise”. According to Budescu and Chen (cite), relative expertise can be measured objectively, through the assignment of weights to contributor’s level of education, seniority, professional status and historical track record (e.g. Biros et al. 2002). In addition, expertise can be measured subjectively using ratings provided by the experts themselves or by others like their peers or superiors (e.g. Lukyanenko et al. 2014). However, measuring relative expertise by a contributor’s performance on tests similar to the intended task has been highlighted as the most efficient method (Clemen 2008, Lin and Cheng 2009, Budescu & Chen 2015). Accordingly, we also adopt the idea of performance-based measurements for expertise. In the next section, we explore classification theory as our overarching …show more content…
rule-based classification, or their instances i.e. exemplars and prototypes (Murphy 2002, Kloos & Sloutsky 2008). In rule-based classification, it is usually necessary to pay selective attention to only relevant features necessary for identifying instances of the class while ignoring irrelevant features (not useful for predicting class membership). Although selective attention leads to efficient learning, especially when making connections between instances with very sparse similarities and dense dissimilarities, it has its cost. The primary cost of selective attention is a learned inattention to features that are not “diagnostic” in the present context (Hoffman & Rehder 2010). If these features however become diagnostic in another context then the ability to make predictions and transfer knowledge is lost. Adapting Hoffman and Rehder’s example, an observer tasked with distinguishing rose bushes from raspberry bushes who considers the presence of berries as the most diagnostic feature may ignore any other feature of both plants (e.g. thorns and leaves). However, if the observer must later distinguish raspberry from cranberry bushes, thorns are suddenly diagnostic as both have red berries and only the raspberry has thorns. The observer may therefore have difficulty distinguishing both bushes due to selective attention. The cost of selective attention is also invariably a cost of