HCIR 2007: Workshop on Human-Computer Interaction and Information Retrieval | EvaluationDiscussion

December 16, 2007, at 12:25 PM by 18.95.2.77 -

Changed lines 9-15 from:

Daniel Tunkelang opened the session with three topics of interest:

how to use sub-document pieces in the retrieval process;
"IR in the small" - what changes when you want to look at small sets of data, as opposed to large corpora;
HCIR evaluation - how to do it? what directions should we look in for solutions?

The recency effect ensured that the last item would be the primary one discussed.

to:

Daniel Tunkelang opened the session by introducing a topic of interest: HCIR evaluation. How can we do it? What directions should we look in for solutions?

November 02, 2007, at 04:09 PM by 128.30.44.143 -

Changed lines 46-48 from:

Interesting related work from 1996 (!) is in the Mira Workshop.

to:

Interesting related work from 1996 (!) is in the Mira Workshop.

November 02, 2007, at 04:09 PM by 128.30.44.143 -

Added lines 44-46:

Interesting related work from 1996 (!) is in the Mira Workshop.

November 01, 2007, at 09:28 PM by 18.95.2.77 -

Changed lines 40-41 from:

Michael Bernstein: Ryen White identified two important use cases. Once you can identify what people are doing, then you can start to improve their lives in that way. "Is this interface better?" is always going to be a question because certain interfaces are better for different things. The important question is under what circumstances are these interfaces useful.

to:

Michael Bernstein: Ryen White identified two important use cases. Once you can identify what people are doing, then you can start to improve their lives in that way. "Is this interface better?" is always going to be a question because certain interfaces are better for different things. The important question is under what circumstances are those interfaces useful.

November 01, 2007, at 09:28 PM by 18.95.2.77 -

Changed lines 38-39 from:

Mark Maybury: I was struck by how simple the usage models were in the papers presented today - what about building causal models of what user is doing? To me, the stressful part is how to make sense of all of the information coming in. This is very different than "what web page should i go to next." I think there are a spectrum of models capturing different levels of complexity.

to:

Mark Maybury: I was struck by how simple the usage models were in the papers presented today - what about building causal models of what users are doing? To me, the stressful part is how to make sense of all of the information coming in. This is very different than "what web page should i go to next." I think there are a spectrum of models capturing different levels of complexity.

November 01, 2007, at 09:26 PM by 18.95.2.77 -

Changed lines 24-27 from:

Shiry Ginosar: It's important to be able to directly compare interfaces, rather than just reporting "happiness," "success," etc. In IR, there is a standard set of documents that different IR systems can be compared on. It's important to have such a "box" that you can hook up to different interfaces in order to compare them. In this case, the box should include users; we can evaluate the system and the user together [see abstract for more detail].

Yifen Huang: We need to have a feedback loop including the user that can be translated back into updating the function that the computer is optimizing for [see abstract for more detail].

to:

Shiry Ginosar: It's important to be able to directly compare interfaces, rather than just reporting "happiness," "success," etc. In IR, there is a standard set of documents that different IR systems can be compared on. It's important to have such a "box" that you can hook up to different interfaces in order to compare them. In this case, the box should include users; we can evaluate the system and the user together [see abstract "Human Computation for HCIR Evaluation" for more detail].

Yifen Huang: We need to have a feedback loop including the user that can be translated back into updating the function that the computer is optimizing for [see abstract "Reasoning and Learning in Mixed-Initiative Tasks" for more detail].

November 01, 2007, at 09:25 PM by 18.95.2.77 -

Changed lines 14-15 from:

The recency effect ensured that the third item would be the primary one discussed.

to:

The recency effect ensured that the last item would be the primary one discussed.

November 01, 2007, at 09:25 PM by 18.95.2.77 -

Changed lines 5-8 from:

to:

What follows is not a transcript but a very brief summary of the viewpoints expressed during the closing discussion session at HCIR'07. If something you said is not accurately represented, please let us know.

November 01, 2007, at 09:23 PM by 18.95.2.77 -

Changed line 7 from:

sub-document scope in retrieval process

to:

how to use sub-document pieces in the retrieval process;

November 01, 2007, at 09:21 PM by 18.95.2.77 -

Changed lines 7-10 from:

to:

sub-document scope in retrieval process
"IR in the small" - what changes when you want to look at small sets of data, as opposed to large corpora;
HCIR evaluation - how to do it? what directions should we look in for solutions?

November 01, 2007, at 09:21 PM by 18.95.2.77 -

Added lines 1-40:

EvaluationDiscussion

Main.EvaluationDiscussion History