forked from wsdm-cup-2017/wsdm-cup-2017.github.io
-
Notifications
You must be signed in to change notification settings - Fork 0
/
vandalism-detection.html
347 lines (310 loc) · 24.7 KB
/
vandalism-detection.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>WSDM Cup 2017</title>
<link href="css/bootstrap.min.css" rel="stylesheet" />
<link href="css/prettify.css" rel="stylesheet" />
<style>
.navbar .navbar-nav {
font-weight: bold;
}
</style>
<!-- HTML5 Shim and Respond.js IE8 support of HTML5 elements and media queries -->
<!-- WARNING: Respond.js doesn't work if you view the page via file:// -->
<!--[if lt IE 9]>
<script src="js/html5shiv.js"></script>
<script src="js/respond.min.js"></script>
<![endif]-->
<link rel="shortcut icon" href="img/icon-wsdm.png">
<!--
<link rel="apple-touch-icon-precomposed" sizes="144x144" href="ico/apple-touch-icon-144-precomposed.png">
<link rel="apple-touch-icon-precomposed" sizes="114x114" href="ico/apple-touch-icon-114-precomposed.png">
<link rel="apple-touch-icon-precomposed" sizes="72x72" href="ico/apple-touch-icon-72-precomposed.png">
<link rel="apple-touch-icon-precomposed" href="ico/apple-touch-icon-57-precomposed.png">
-->
</head>
<body>
<nav class="navbar navbar-inverse navbar-static-top" style="margin-bottom:0px;">
<div class="container-fluid">
<div class="navbar-header">
<button type="button" class="navbar-toggle" data-toggle="collapse" data-target="#bs-example-navbar-collapse-1">
<span class="sr-only">Toggle navigation</span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="index.html">WSDM Cup 2017</a>
</div>
<div class="collapse navbar-collapse" id="bs-example-navbar-collapse-1">
<ul class="nav navbar-nav navbar-right">
<li><a href="index.html">Home</a></li>
<li><a href="about.html">Organization</a></li>
<li><a href="about.html#important-dates">Important Dates</a></li>
<!--<li><a href="proceedings.html">Proceedings</a></li>-->
<li class="dropdown active">
<a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-haspopup="true" aria-expanded="false">Tasks <span class="caret"></span></a>
<ul class="dropdown-menu">
<li><a href="vandalism-detection.html">Vandalism Detection</a></li>
<li><a href="triple-scoring.html">Triple Scoring</a></li>
</ul>
</li>
</ul>
</div>
</div>
</nav>
<div class="container">
<div class="row">
<div class="col-xs-12">
<div class="clearfix">
<h1 id="task-description" class="page-header">
Vandalism Detection
<div class="thumbnail pull-right" style="text-align:right;margin-left:15px;"><a href="http://www.adobe.com/" target="_blank"><img src="img/logo-adobe.png" alt="Adobe" style="max-height:150px"></a><div style="font-size:7pt;margin-right:10px;margin-top:2px;">Sponsor</div></div>
<div class="thumbnail pull-right" style="text-align:right;margin-left:15px;"><a href="http://www.wikimedia.de/" target="_blank"><img src="img/logo-wikimedia-germany.svg" alt="Wikimedia Germany Logo" style="height:150px"></a><div style="font-size:7pt;margin-right:85px;">Supporter</div></div>
</h1>
<p><a href="https://www.wikidata.org/">Wikidata</a> is the new, large-scale knowledge base of the Wikimedia Foundation which can be edited by anyone. Its knowledge is increasingly used within Wikipedia as well as in all kinds of information systems, which imposes high demands on its integrity. Nevertheless, Wikidata frequently gets vandalized, exposing all its users to the risk of spreading vandalized and falsified information.</p>
</div>
<div class="panel panel-default">
<div class="panel-heading">Task</div>
<div class="panel-body">
<p>Given a Wikidata revision, compute a vandalism score denoting the likelihood of this revision beeing vandalism (or similarly damaging).</p>
</div>
</div>
<div class="panel panel-default">
<div class="panel-heading">Rules</div>
<div class="panel-body">
<p>The goal of the vandalism detection task is to detect vandalism nearly in real time as soon as it happens. Hence, the following rules apply:</p>
<ul>
<li>Use of any additional data that is newer than the provided training data is forbidden. In particular, you <b>may not</b> scrape any Wikimedia website, use the API, the dumps, or any related data source to obtain data that is newer than February 29, 2016.</li>
<li><b>You may</b> use sources of publicly available external data having to do with geographical information, demographic information, natural language processing, etc. This data must not relate to the specific revision label (vandalism vs regular).</li>
</ul>
</div>
</div>
<div class="panel panel-default" id="corpus-wdvc-16">
<div class="panel-heading">Wikidata Vandalism Corpus 2016</div>
<div class="panel-body">
<p>To develop your software, we provide you with a training corpus that consists of Wikidata revisions and whether they are considered vandalism.<p>
<p>The Wikidata Vandalism Corpus 2016 contains revisions of the knowledge base Wikidata. The corpus comprises manual revisions only, while all revisions by official bots were filtered out. For each revision, we provide the information whether it is considered vandalism (ROLLBACK_REVERTED) or not. Unlike the Wikidata dumps, revisions are ordered chronologically by REVISION_ID (i.e., in the order they arrived at Wikidata). For training, we provide data until February 29, 2016. The evaluation will be conducted on later data.</p>
<p>The provided training data consists of 23 files in total. You can check their validty via their <a href="http://www.uni-weimar.de/medien/webis/corpora/corpus-wdvc-16/checksums.md5">md5</a> or <a href="http://www.uni-weimar.de/medien/webis/corpora/corpus-wdvc-16/checksums.sha1">sha1</a> checksums.</p>
<h4>Revision Data Files (21 files)</h4>
<ul>
<li><a href="http://www.uni-weimar.de/medien/webis/corpora/corpus-wdvc-16/wdvc16_2012_10.xml.7z">wdvc16_2012_10.xml.7z</a> (18 MB)</li>
<li><a href="http://www.uni-weimar.de/medien/webis/corpora/corpus-wdvc-16/wdvc16_2012_11.xml.7z">wdvc16_2012_11.xml.7z</a> (98 MB)</li>
<li><a href="http://www.uni-weimar.de/medien/webis/corpora/corpus-wdvc-16/wdvc16_2013_01.xml.7z">wdvc16_2013_01.xml.7z</a> (129 MB)</li>
<li><a href="http://www.uni-weimar.de/medien/webis/corpora/corpus-wdvc-16/wdvc16_2013_03.xml.7z">wdvc16_2013_03.xml.7z</a> (367 MB)</li>
<li><a href="http://www.uni-weimar.de/medien/webis/corpora/corpus-wdvc-16/wdvc16_2013_05.xml.7z">wdvc16_2013_05.xml.7z</a> (466 MB)</li>
<li><a href="http://www.uni-weimar.de/medien/webis/corpora/corpus-wdvc-16/wdvc16_2013_07.xml.7z">wdvc16_2013_07.xml.7z</a> (451 MB)</li>
<li><a href="http://www.uni-weimar.de/medien/webis/corpora/corpus-wdvc-16/wdvc16_2013_09.xml.7z">wdvc16_2013_09.xml.7z</a> (442 MB)</li>
<li><a href="http://www.uni-weimar.de/medien/webis/corpora/corpus-wdvc-16/wdvc16_2013_11.xml.7z">wdvc16_2013_11.xml.7z</a> (482 MB)</li>
<li><a href="http://www.uni-weimar.de/medien/webis/corpora/corpus-wdvc-16/wdvc16_2014_01.xml.7z">wdvc16_2014_01.xml.7z</a> (576 MB)</li>
<li><a href="http://www.uni-weimar.de/medien/webis/corpora/corpus-wdvc-16/wdvc16_2014_03.xml.7z">wdvc16_2014_03.xml.7z</a> (589 MB)</li>
<li><a href="http://www.uni-weimar.de/medien/webis/corpora/corpus-wdvc-16/wdvc16_2014_05.xml.7z">wdvc16_2014_05.xml.7z</a> (1,022 MB)</li>
<li><a href="http://www.uni-weimar.de/medien/webis/corpora/corpus-wdvc-16/wdvc16_2014_07.xml.7z">wdvc16_2014_07.xml.7z</a> (1,349 MB)</li>
<li><a href="http://www.uni-weimar.de/medien/webis/corpora/corpus-wdvc-16/wdvc16_2014_09.xml.7z">wdvc16_2014_09.xml.7z</a> (1,430 MB)</li>
<li><a href="http://www.uni-weimar.de/medien/webis/corpora/corpus-wdvc-16/wdvc16_2014_11.xml.7z">wdvc16_2014_11.xml.7z</a> (1,237 MB)</li>
<li><a href="http://www.uni-weimar.de/medien/webis/corpora/corpus-wdvc-16/wdvc16_2015_01.xml.7z">wdvc16_2015_01.xml.7z</a> (1,291 MB)</li>
<li><a href="http://www.uni-weimar.de/medien/webis/corpora/corpus-wdvc-16/wdvc16_2015_03.xml.7z">wdvc16_2015_03.xml.7z</a> (1,582 MB)</li>
<li><a href="http://www.uni-weimar.de/medien/webis/corpora/corpus-wdvc-16/wdvc16_2015_05.xml.7z">wdvc16_2015_05.xml.7z</a> (1,266 MB)</li>
<li><a href="http://www.uni-weimar.de/medien/webis/corpora/corpus-wdvc-16/wdvc16_2015_07.xml.7z">wdvc16_2015_07.xml.7z</a> (1,335 MB)</li>
<li><a href="http://www.uni-weimar.de/medien/webis/corpora/corpus-wdvc-16/wdvc16_2015_09.xml.7z">wdvc16_2015_09.xml.7z</a> (1,752 MB)</li>
<li><a href="http://www.uni-weimar.de/medien/webis/corpora/corpus-wdvc-16/wdvc16_2015_11.xml.7z">wdvc16_2015_11.xml.7z</a> (3,130 MB)</li>
<li><a href="http://www.uni-weimar.de/medien/webis/corpora/corpus-wdvc-16/wdvc16_2016_01.xml.7z">wdvc16_2016_01.xml.7z</a> (3,332 MB)</li>
</ul>
<h4>Meta File (1 File)</h4>
<ul>
<li><a href="http://www.uni-weimar.de/medien/webis/corpora/corpus-wdvc-16/wdvc16_meta.csv.7z">wdvc16_meta.csv.7z</a> (168 MB)</li>
</ul>
<table class="table table-condensed rule" style="font-size:small;">
<thead>
<tr><th>Name</th><th>Types</th><th>Description</th></tr>
</thead>
<tbody>
<tr><td>REVISION_ID </td><td>Integer </td><td> The Wikidata revision id </td></tr>
<tr><td>REVISION_SESSION_ID</td><td>Integer </td><td> The Wikidata revision id of the first revision in this session </td></tr>
<tr><td>USER_COUNTRY_CODE </td><td>String </td><td> Country code for IP address (only available for unregistered users) </td></tr>
<tr><td>USER_CONTINENT_CODE</td><td>String </td><td> Continent code for IP address (only available for unregistered users)</td></tr>
<tr><td>USER_TIME_ZONE </td><td>String </td><td> Time zone for IP address (only available for unregistered users) </td></tr>
<tr><td>USER_REGION_CODE </td><td>String </td><td> Region code for IP address (only available for unregistered users) </td></tr>
<tr><td>USER_CITY_NAME </td><td>String </td><td> City name for IP address (only available for unregistered users) </td></tr>
<tr><td>USER_COUNTY_NAME </td><td>String </td><td> County name for IP address (only available for unregistered users) </td></tr>
<tr><td>REVISION_TAGS </td><td>List<String> </td><td> The Wikidata revision tags </td></tr>
</tbody>
</table>
<h4>Truth File (1 File)</h4>
<ul>
<li><a href="http://www.uni-weimar.de/medien/webis/corpora/corpus-wdvc-16/wdvc16_truth.csv.7z">wdvc16_truth.csv.7z</a> (44 MB)</li>
</ul>
<table class="table table-condensed rule" style="font-size:small;">
<thead>
<tr><th>Name</th><th>Types</th><th>Description</th></tr>
</thead>
<tbody>
<tr><td>REVISION_ID </td><td>Integer</td><td>The Wikidata revision id </td></tr>
<tr><td>ROLLBACK_REVERTED </td><td>Boolean</td><td>Whether this revision was reverted via the rollback feature </td></tr>
<tr><td>UNDO_RESTORE_REVERTED</td><td>Boolean</td><td>Whether this revision was reverted via the undo/restore feature </td></tr>
</tbody>
</table>
<p>The ROLLBACK_REVERTED field encodes the official ground truth for this competition. The UNDO_RESTORE_REVERTED field serves informational purposes only.</p>
<p>The truth file will only be available for the training dataset but not for test datasets.</p>
<p> The corpus can be processed, for example, with <a href="https://www.mediawiki.org/wiki/Wikidata_Toolkit">Wikidata Toolkit</a>.</p>
</div>
</div>
<div class="panel panel-default">
<div class="panel-heading">Output</div>
<div class="panel-body">
<p>For each Wikidata revision in the test corpus, your software shall output a vandalism score in the range [0,1]. The output shall be formatted as a CSV file in the format <a href="https://www.ietf.org/rfc/rfc4180.txt">RFC4180</a> and consist of two columns: The first column denotes Wikidata's revision id as an integer and the second column denotes the vandalism score as a float32. Here are a few example rows:</p>
<table class="table table-condensed rule" style="font-size:small;">
<thead>
<tr><th>Revision Id</th><th>Vandalism Score</th></tr>
</thead>
<tbody>
<tr><td>123</td><td>0.95</td></tr>
<tr><td>124</td><td>0.30</td></tr>
<tr><td>125</td><td>12.e-5</td></tr>
</tbody>
</table>
</div>
</div>
<div class="panel panel-default">
<div class="panel-heading">Performance Measures</div>
<div class="panel-body">
<p>For determining the winner, we use ROC-AUC as primary evaluation measure.</p>
<p>For informational purposes, we might compute further evaluation measures such as PR-AUC and the runtime of the software.</p>
</div>
</div>
<div class="panel panel-default">
<div class="panel-heading">Test Corpus</div>
<div class="panel-body">
<p>Once you finished tuning your approach to achieve satisfying performance on the training corpus, you should run your software on the test corpus.</p>
<p>During the competition, the test corpus will not be released publicly. Instead, we ask you to submit your software for evaluation at our site as described below.</p>
<p>After the competition, the test corpus is available including ground truth data. This way, you have all the necessities to evaluate your approach on your own, yet being comparable to those who took part in the competition.</p>
<!--<p>
<a class="btn btn-default" target="_blank" href="http://www.uni-weimar.de/medien/webis/corpora/corpus-pan-labs-09-today/pan-14/pan14-data/">Download corpus</a>
</p>-->
</div>
</div>
<div class="panel panel-default">
<div class="panel-heading">Submission</div>
<div class="panel-body">
<p>We ask you to prepare your software so that it can be executed via a command line call without any parameters.</p>
<pre class="prettyprint lang-c" style="overflow-x:auto">
> mySoftware
</pre>
<p>You can choose freely among the available programming languages and among the operating systems Microsoft Windows and Ubuntu. We will ask you to deploy your software onto a virtual machine that will be made accessible to you after registration. You will be able to reach the virtual machine via ssh and via remote desktop. More information about how to access the virtual machines can be found in the user guide below:</p>
<p><a class="btn btn-default" href="wsdm-cup-17-virtual-machine-user-guide.pdf">Virtual Machine User Guide »</a></p>
<p>Once deployed in your virtual machine, we ask you to access TIRA at <a href="http://www.tira.io">www.tira.io</a>, where you can self-evaluate your software on the test data.</p>
<p><strong>Note:</strong> By submitting your software you retain full copyrights. You agree to grant us usage rights only for the purpose of the WSDM Cup 2017. We agree not to share your software with a third party or use it for other purposes than the WSDM Cup 2017.</p>
<p>Your software will receive the uncompressed Wikidata revisions on <code>stdin</code>, and the uncompressed meta data on the named pipe <code>meta</code>. Your program must produce the results on <code>stdout</code> in form of a RFC4180 CSV file containing the two columns REVISION_ID and VANDALISM_SCORE. You will only receive new revisions on <code>stdin</code> and <code>meta</code> after having reported your vandalism score on <code>stdout</code>. To enable fast and concurrent processing of data, we will introduce a backpressure window of <var>k</var> revisions, i.e., you will receive revision <var>n + k</var> on <code>stdin</code> as soon as having reported your result for revision <var>n</var> on <code>stdout</code> (the exact constant <var>k</var> is still to be determined but you can expect it to be around <var>16</var> revisions). We will provide you a demo program in the following days enabling you to test your software against it.
</div>
</div>
<!--
<div class="panel panel-default">
<div class="panel-heading">Results</div>
<div class="panel-body">
<p>The following table lists the performances achieved by the participating teams:</p>
<table class="table table-condensed rule" style="font-size:small;">
<thead>
<tr class="top"><th colspan="5" style="text-align:center">Source Retrieval Performance</th></tr>
<tr class="top2"><th colspan="2" style="text-align:center">Workload to 1st Detection</th><th colspan="2" style="text-align:center">Downloaded Sources</th><th>Team</th></tr>
<tr class="mid"><th>Queries</th><th>Downloads</th><th>Precision</th><th>Recall</th><th></th></tr>
</thead>
<tbody>
<tr><td> 54.5</td><td> 33.2</td><td>0.40</td><td>0.39</td><td>Victoria Elizalde<br/>Private, Argentina</td></tr>
<tr><td> 83.5</td><td>207.1</td><td>0.08</td><td>0.48</td><td>Leilei Kong, Yong Han, Zhongyuan Han, Haihao Yu, Qibo Wang, Tinglei Zhang , Haoliang Qi<br/>Heilongjiang Institute of Technology, China</td></tr>
<tr><td> 60.0</td><td> 38.8</td><td>0.38</td><td>0.51</td><td>Amit Prakash and Sujan kumar Saha<br/>Birla Institute of Technology, India</td></tr>
<tr><td> 19.5</td><td>237.3</td><td>0.08</td><td>0.40</td><td>Šimon Suchomel and Michal Brandejs<br/>Masaryk University, Czech Republic</td></tr>
<tr><td>117.1</td><td> 14.4</td><td>0.57</td><td>0.48</td><td>Kyle Williams, Hung-Hsuan Chen, and C. Lee Giles<br/>Pennsylvania State University, USA</td></tr>
<tr><td> 37.0</td><td> 18.6</td><td>0.54</td><td>0.45</td><td>Denis Zubarev° and Ilya Sochenkov*<br/>°Institute for Systems Analysis of Russian Academy of Sciences and *Peoples’ Friendship University of Russia, Russia</td></tr>
</tbody>
</table>
<p>A more detailed analysis of the retrieval performances can be found in the overview paper accompanying this task.</p>
<p>
<a class="btn btn-default" href="http://www.uni-weimar.de/medien/webis/publications/papers/stein_2013h.pdf#page=11">Learn more »</a>
</p>
</div>
</div>-->
<div class="panel panel-default">
<div class="panel-heading">Related Work</div>
<div class="panel-body">
<ul><li>
Stefan Heindorf, Martin Potthast, Benno Stein, and Gregor Engels. <a href="https://is.uni-paderborn.de/fileadmin/Informatik/AG-Engels/Publikationen/2016/heindorf2016_CIKM.pdf">Vandalism Detection in Wikidata</a>. In <i>Proceedings of the 25th ACM International Conference on Information and Knowledge Management (CIKM 16) (to appear) </i> , October 2016. ACM.
<a href="https://is.uni-paderborn.de/fileadmin/Informatik/AG-Engels/Publikationen/2016/heindorf2016_CIKM.pdf">[Paper]</a>
</li><li>
Stefan Heindorf, Martin Potthast, Benno Stein, and Gregor Engels. <a href="https://is.uni-paderborn.de/fileadmin/Informatik/AG-Engels/Publikationen/2015/heindorf2015_SIGIR.pdf">Towards Vandalism Detection in Knowledge Bases: Corpus Construction and Analysis</a>. In Ricardo Baeza-Yates, Mounia Lalmas, Alistair Moffat, and Berthier Ribeiro-Neto, editors, <i>38th International ACM Conference on Research and Development in Information Retrieval (SIGIR 15)</i>, pages 831-834, August 2015. ACM. ISBN 978-1-4503-3621-5
<a href="https://is.uni-paderborn.de/fileadmin/Informatik/AG-Engels/Publikationen/2015/heindorf2015_SIGIR.pdf">[Paper]</a>
<a href="https://is.uni-paderborn.de/fileadmin/Informatik/AG-Engels/Publikationen/2015/heindorf2015_SIGIR_poster.pdf">[Poster]</a>
<a href=" http://www.uni-weimar.de/en/media/chairs/webis/corpora/corpus-wdvc-15/">[Corpus]</a>
</li><li>
Martin Potthast, Benno Stein, and Robert Gerling. <a href="http://www.uni-weimar.de/medien/webis/publications/papers/stein_2008c.pdf">Automatic Vandalism Detection in Wikipedia</a>. In Craig Macdonald et al, editors, <i>Advances in Information Retrieval. 30th European Conference on IR Research (ECIR 08)</i> volume 4956 of Lecture Notes in Computer Science, pages 663-668, Berlin Heidelberg New York, 2008. Springer. ISBN 978-3-540-78645-0. ISSN 0302-9743.
<a href="http://www.uni-weimar.de/medien/webis/publications/papers/stein_2008c.pdf">[Paper]</a>
<a href="http://www.uni-weimar.de/medien/webis/publications/posters/stein_2008c.pdf">[Poster]</a>
</li></ul>
</div>
</div>
</div> <!-- col -->
</div> <!-- row -->
<div id="task-committee" class="row" style="padding-top:30px;">
<div class="col-xs-12">
<h1 class="page-header">Task Chairs</h1>
</div>
</div>
<div class="row">
<div class="col-xs-6 col-sm-3">
<div class="thumbnail" style="text-align:center;">
<a href="http://is.uni-paderborn.de/en/research-group/fg-engels/people/visitenkarten/heindorf/business-card.html" target="_blank"><img src="img/stefan.jpg" class="img-rounded" alt="Stefan Heindorf"></a>
<p style="white-space:nowrap"><a href="http://is.uni-paderborn.de/en/research-group/fg-engels/people/visitenkarten/heindorf/business-card.html" target="_blank">Stefan Heindorf</a></p>
<p style="font-size:10pt">Paderborn University</p>
</div>
</div>
<div class="col-xs-6 col-sm-3">
<div class="thumbnail" style="text-align:center;">
<a href="http://www.uni-weimar.de/medien/webis/people" target="_blank"><img src="img/martin.jpg" class="img-rounded" alt="Martin Potthast"></a>
<p style="white-space:nowrap"><a href="http://www.uni-weimar.de/medien/webis/people" target="_blank">Martin Potthast</a></p>
<p style="font-size:10pt">Bauhaus-Universität Weimar</p>
</div>
</div>
</div>
<div class="row">
<div class="col-xs-12">
<h2>Task Committee</h2>
</div>
</div>
<div class="row">
<div class="col-xs-6 col-sm-3">
<div class="thumbnail" style="text-align:center;">
<a href="http://is.uni-paderborn.de/" target="_blank"><img src="img/gregor.jpg" class="img-rounded" alt="Gregor Engels"></a>
<p style="white-space:nowrap"><a href="http://is.uni-paderborn.de/" target="_blank">Gregor Engels</a></p>
<p style="font-size:10pt">Paderborn University</p>
</div>
</div>
<div class="col-xs-6 col-sm-3">
<div class="thumbnail" style="text-align:center;">
<a href="http://www.webis.de" target="_blank"><img src="img/benno.jpg" class="img-rounded" alt="Benno Stein"></a>
<p style="white-space:nowrap"><a href="http://www.webis.de" target="_blank">Benno Stein</a></p>
<p style="font-size:10pt">Bauhaus-Universität Weimar</p>
</div>
</div>
</div>
</div> <!-- /container -->
<script src="js/jquery.js"></script>
<script src="js/bootstrap.min.js"></script>
<script src="js/prettify.js"></script>
<script>
!function ($) {
$(function(){
window.prettyPrint && prettyPrint()
})
}(window.jQuery)
</script>
<script>
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','https://www.google-analytics.com/analytics.js','ga');
ga('create', 'UA-19597677-4', 'auto');
ga('send', 'pageview');
</script>
</body>
</html>