An Experimental Approach to the Construction of Binary Decision Classes from Card Sort Data

Date
2013-07
Authors
Almestadi, Emad Hamdan
Journal Title
Journal ISSN
Volume Title
Publisher
Faculty of Graduate Studies and Research, University of Regina
Abstract

This thesis presents work done towards understanding the data collected from a card sorting study of facial photographs. In that study, 25 participants sorted 356 photos (178 Caucasian and 178 First Nations) into piles based on similarity. Photos placed in the same pile are deemed to be similar, photos in different piles are deemed to be dissimilar. Looking to establish binary decision classes is reasonable because the underlying question that participants answered was “Are these photos similar or not?”. There may also be more than two decision classes to describe all the behaviours. For example, an initial split into decision classes may be thought of as “doing something” and “not doing something”. The latter could be split into two, and the whole process repeated. Differences amongst the sorting behaviours of participants are evident, but the reason for these differences is difficult to determine. An early hypothesis was that perceived race was being used a criterion for some participants but not for others. An analysis that looked at the ratio of Caucasian and First Nations photos in each pile was used determine a pair of decision classes from which accurate classifiers could be built. Open questions from that earlier work include the basis for participants making those decisions and whether the behaviour supported by a small amount of carefully chosen data would be supported by all the data. There are several million possible decision class pairs that could be used to split those 25 participants into 2 groups. This work applies a knowledge discovery approach to find other candidate decision classes for this data, for which accurate classifiers can also be built. Each participant made a relatively small number of direct comparisons and a large number of indirect comparisons to determine whether each pair of photos (63,190 in all) was similar or dissimilar. For each pair, a binary vector was used to record the judgement of each participant (0 if the participant thought the pair was similar, 1 if dissimilar). These vectors were used as the basis for the present study. Each pair of photos can be said to have a certain power to discern between participants. If all participants gave the same judgement for a pair, the pair has no power to discern between participants. Conversely, a pair which 12 or 13 participants had rated similar (or dissimilar) has the highest power to discern between participants, because for this pair there will be the most disagreement when considering pairs of participants. This work focuses on those pairs with maximum discernibility. To generalize earlier work on a heuristic for evaluating candidate decision classes, it is hypothesized that a t-test could be used to give a better indication about the suitability of a decision class pair. To this end, some experimental analysis of the card sorting study data was undertaken and the rough set attribute reduction methodology was used to evaluate the findings from the computational experiment. ii

Description
A Thesis Submitted to the Faculty of Graduate Studies and Research In Partial Fulfillment of the Requirements for the Degree of Master of Science in Computer Science, University of Regina.x , 108 l.
Keywords
Citation
Collections