Vania Dimitrova, Lydia Lau, Dhavalkumar Thakker, Fan Yang-Turner, and Dimoklis Despotakis. 2013. Exploring exploratory search: a user study with linked semantic data. In Proceedings of the 2nd International Workshop on Intelligent Exploration of Semantic Data (IESD '13). Association for Computing Machinery, New York, NY, USA, Article 2, 1–8. https://doi.org/10.1145/2462197.2462199
ABSTRACT
... Linked semantic data appears to offer a great potential for exploratory search, which is open-ended, multi-faceted, and iterative in nature. However, there is limited insight into how browsing through linked semantic data sets can support exploratory search.
This paper presents a user study with a uni-focal semantic browsing interface for exploratory search through several data sets linked via domain ontologies. The study, which is qualitative and exploratory in nature and uses music as an illustrative domain, examines (i) obstacles and challenges related to user exploratory search in LOD and (ii) the serendipitous learning effect and the role semantics plays in that.
[Avaliação qualitativa baseada em uma interface de navegação]
1. INTRODUCTION
In contrast to regular search, exploratory search gives a more complete overview of a topic. Exploratory search is open-ended, multi-faceted, and iterative in nature, and is commonly used in scientific discovery, learning, and sense making [18, 27]. Exploration demands more time, effort and creativity from the user, but rewards the user with deeper knowledge [18]. Exploratory search is particularly beneficial for ill-structured problems and more open-ended goals, with persistent, opportunistic, iterative query processes. Exploratory tasks inherently have uncertainty, ambiguity and discovery as common aspects [18].
[Busca Exploratória requer mais tempo, esforço de busca, esforço de avaliação e interpretação]
The main contribution of this paper to interactive exploration of semantic data is the multi-faceted evaluation study which follows methodologies from the research community on exploratory search. The findings indicate that semantic facets support exploratory search and facilitate serendipitous learning; and indicate issues that need further attention to enable exploratory search with LOD (e.g. avoiding ‘empty links’ and sensing the quality of information the user will see from a link).
[Aprendizado com a busca é um efeito colateral]
2. RELATED WORK
Semantic data browsers have emerged from a collective effort in the semantic research community. Such browsers operate on semantically augmented data (e.g. tagged content) and lay out browsing trajectories using relationships in the underpinning ontologies.
[Não achei exemplosfuniconando no site da W3C]
A key component of exploration is human learning, a topic studied extensively by cognitive psychologists [16]. An evaluation study of a faceted search interface [15] showed that facets played a major role in the browsing process, accounting for about half of the time spent looking at actual results. This underscores the importance of facets, which is adopted in the design of the semantic data browser presented in this paper. We further examine the role of semantic facets (facts, related terms, and content) in exploratory search tasks by looking into how users complete exploratory search tasks in an unfamiliar domain.
3. CASE STUDY: MUSICPINTA
4. EXPERIMENTAL STUDY
To get an insight of how MusicPinta can support exploratory search through linked semantic data, we conducted an experimental study following methodological recommendations for evaluating exploratory search systems [27].
The experimental study followed with two exploratory search tasks as the within avoid bias by the ordering of tasks equally into two groups; one group performed task 1 then task 2, and the other group performed task 2 then task 1.
[Metodologia]
Method. Each participant attended an individual session, conducted and observed by an experimenter, about an hour, with:
• Pre-study questionnaire [5 min] - collecting information about the user profile and test his/her domain awareness.
• Introduction to MusicPinta [10 min] – the participants followed a script which introduced the main features of the system using the instrument tenor saxophone as an example;
• Task 1 [20 min] - identify distinctive characteristics of the musical instrument bouzouki [15min], after which complete a task difficulty questionnaire [5 min].
• Task 2 [20 min] - usage and features of the musical instrument electric guitar [15min], after which complete a task difficulty questionnaire [5 min].
• Post-study questionnaire [10 min] – test again the participant’s domain awareness and gather usability feedback.
• Brief interview [5 min] – overall impression of MusicPinta.
Domain awareness test. Testing domain awareness is crucial for identifying whether there was any learning effect from the browsing behavior. A free word association test, seen as a reliable measure of prior knowledge in reading comprehension studies [29], was used. We asked participants to write any associated words to a list of 12 musical instruments.
[Como medir o conhecimento antes de interagir com o KG]
Tasks. To design the study tasks, we have followed the main characteristics of exploratory search tasks summarised in [28]: the main goal is learning and/or investigation of a musical instrument; there is a low level of specificity about the information needed and how to find it; search is open ended, requires finding several items and involves a degree of uncertainty; tasks are ‘not too easy’ and include multiple facets.
[Tarefas analíticas que requerem comparar para identificar similaridades e diferenças além de identificar características "interessantes" dos itens buscados.]
The completion of Task 1 required mainly browsing through the musical instrument classification (in both DBTune and DBpedia) and reading descriptions provided from DBpedia. The task was analytical in nature, as users had to perform comparison and identification of distinctive features. In contrast, Task 2, which required browsing through content about music albums and artists, and reading through Amazon reviews, was more ambiguous and involved some creative thinking and imagination.
Task difficulty. After each task, the users were asked to fill-out a short questionnaire to rate their subjective level of cognitive load using a modified version of the NASA-TLX questionnaire [6]. In addition, the participants were asked to think aloud; the experimenter kept notes of any interesting comments made.
Data collected. The data collected in the study includes: (i) the forms with the participants’ outputs for tasks 1 and 2; (ii) the pre and post-experiment questionnaires and word association tests; (iii) system log data; (iv) experimenter notes. The data was analysed using qualitative and quantitative methods (including non-parametric statistical test); the results are presented below.
5.2 Browsing Behaviour
The interaction log files, which recorded the user clicks when browsing data sets in MusicPinta, provided an insight into the browsing behaviour. The log data was pre-processed and each link was assigned an abstraction level based on the ontology
There was a high number of ‘empty clicks’ - the user clicks on a link and is taken to a page with no information, sees that this link is not helpful and quickly returns to the previous page. ... ‘Empty clicks’ leading to pages with no information was seen as one of the main causes for frustration
5.3 Learning
An inherent effect of exploratory search is learning [18]. Therefore, we expected that while conducting the two investigatory tasks the participants, who were unfamiliar with the musical instruments they browsed in MusicPinta, would gain some knowledge necessary to complete the tasks.
Overall, the participants’ word association post-test included more facts related to task 1 than to task 2
The new facts which participants acquired were examined to identify possible origin for the knowledge gained. Facts about the instruments related to task 1 mainly refer to instrument characteristics (e.g. origin, shape, sound), which were gained from the pages about the instrument. Knowledge gained while performing task 2 included interesting associations (e.g. flute was associated to electric guitar and acoustic guitar, as they could be used together in performances), adding instrument classification (e.g. plucked string instrument was added for ukulele and acoustic guitar), or adding a link to a concept (e.g. associating American invention and 1931 invention to electrical guitar). While the successful completion of task 2 required browsing mainly through content level, the new facts associated to the shown focus and context words related this task came from the links to classification level seen in the browser. Although the users did not click on these links, when skimming through the page they noticed interesting facts (e.g. looking for electric guitar albums a participant noticed the link to American invention and commented ‘Well, I have learned something new today – did not know electric guitar was American invention’).
It was interesting to observe that some participants formulated more abstract facts about the instruments, which are based on the knowledge overview gained from the semantic links,
6. DISCUSSION
6.1 Use of semantic data browsers for exploratory search
The study provides evidence that semantic data browsers are suitable for exploratory search (which is in line with current trends in knowledge-enriched browsing environments – see Section 2). Users without domain knowledge could complete investigation tasks with MusicPinta fairly quickly and create meaningful answers
Showing the three facets together (facts, terms and content) provided a flexible way to explore the richness of data types and embedded information in linked data sets. Facts and terms gave starting points for classification/content exploration, while descriptions from DBpedia or from Amazon reviews offered additional details. There were different ways people consume knowledge enriched information.
When browsing at content level, people looked for ‘important’ links or anything that was ‘interesting’. A way to support this process is to add some intelligent functionality which measures the value of the content that a link may refer to and to take this into account for filtering the links shown to the user.
[A interface faz bastante diferença no processo de exploração]
6.2 Learning effect and the role of semantics
The study results suggest that exploratory search with semantic data browsers can be beneficial for domain learning (which is in line with studies in exploratory search [18]). New knowledge came from browsing through the classification level, reading instrument descriptions, and reading instrument reviews.
This gives support that the display of semantics, i.e. the knowledge structures underpinning the linked data sets, plays a key role for facilitating learning.
An interesting effect of serendipitous learning [5] was observed where semantic links enabled discovering new facts ‘by chance’, unrelated to the task at hand
6.3 Reflection on the experimental design
Study results are inevitably influenced by experimental design choices.
7. CONCLUSIONS
This paper starts from the position that linked semantic data can provide a rich source of knowledge that can be exposed to end users via semantic data browsers. In a study based on a Music domain, we created representative exploratory search conditions, including a more knowledge-rich analytical task (compare, find similarities & differences) and a more content-based creativity task (find something interesting). The study confirmed that the overview of the knowledge structure presented with the classification level tags is beneficial for the success of the analytical tasks and can facilitate serendipitous learning in the creativity tasks.
[As tarefas não são somente de pergunta e resposta. Outras ações precisam ser tomadas com as informações.]
Comentários
Postar um comentário
Sinta-se a vontade para comentar. Críticas construtivas são sempre bem vindas.