Live E-Assessment via ARS

Introductory phase: Activation of prior knowledge
At the beginning of the event, core content from the previous event is addressed in a short question and answer session. If there are serious gaps in knowledge, a call can be made for targeted self-study, with reference to the relevant learning resources, or the relevant material can be briefly recapitulated. Tip: Answering the questions should only take a short time and the level of ambition should be low.

Work phase: Reflection of the content through peer instruction according to Eric Mazur
Questions that were answered incorrectly by many are briefly discussed in partner work or small groups and then asked again (2-round voting). A possible workflow would be:

1. Presentation of learning content of a topic block (approx. 20min), followed by presentation of the assignment.
2. Acquisition and solution of the task by the students (max. 5 min.)
3. Evaluation of the answers and initiation of the next phase
4. Procedure adapted to the result
   1. Answers predominantly correct --> Present results, explanation of remaining ambiguities and continuation of the event
   2. Answers predominantly wrong --> Adapted repetition of contents, if necessary renewed voting
   3. Answers diffuse --> Initiate peer discussion: time-limited discussion of own answer with someone who chose a different answer ("Convince your neighbor of your answer") and repeat task (2-round voting). Presentation of the results comparing the first and second round.

Final phase: Uncovering ambiguities through anonymous question collection
Already during the working phase, students are encouraged to ask comprehension questions anonymously. In the final phase of the course, the teacher review the questions, deletes duplicate or nonsensical ones if necessary, and then presents the remaining ones. The answers can be given directly or afterwards, e.g. via the learning platform.

Particify, the successor to ARSnova, is available for lecturers at the University of Bremen. This is hosted by the ZMML, can be used with the university account and is fully DSGVO-compliant. The Cliqr plug-in offers an alternative within Stud.IP. In addition, there are a number of free ARS systems on external servers, including PINGO (developed by Paderborn University, supports the peer induction method in particular).

• Consulting on didactic design and technical realization
• Provision of the software

Examples from the summer semester 2015 (results of guideline-based interviews in September 2015):

Prof. Dr. Manfred Fahle (Department of Biology/Chemistry, Biology major):
 Professor Fahle has been using the hardware clickers awarded by the ZMML in two of his 90-minute lectures in the bachelor's program in biology since 2010: "Animal Physiology and Human Biology 2 (NHZ2)" with about 120 students and "Basic Principles of Neurophysiology and Neuroanatomy (PM1-6)" with about 30 students. Since only 40 units are available, small groups of 2 to 3 students each are formed in NHZ2 to answer. During the course of the lecture, approximately 15-minute teaching sessions are followed by one question each (2 to 3 minutes per clicker session). The total time required for ARS use is about 15 minutes It is predominantly single-choice questions of the type "Which of the following 5 statements is false?" The questions represent the primary learning objectives of the course and are similar to the questions on the e-exam. After time elapses, the voting result appears automatically. If the answers are predominantly correct, a brief explanation is given as to why the statement in question was incorrect and then the lecture continues. If the answers are only 50% or less correct, the material is repeated in approx. 2 to 3 minutes in an adapted, concentrated form. Technically, there have been no problems so far and no devices have been stolen. The time required to integrate the clicker questions into the PowerPoint presentation is low. The unrest in the lecture hall, which arises during the discussion phases at NHZ2, ends immediately after the clicker session is over and the result is presented. On average, despite non-trivial questions, over 70% of the answers were correct. The attention in the lecture could be kept throughout not only thanks to the clicker use, but also many illustrative case studies. Thought experiments such as "What would it mean to you if your excretory system did not reabsorb 99% of your primary urine?" help maintain attention. There was no detectable attrition effect at the end of the semester, and participation in voting was consistently high at about 90%. The need for group voting in NHZ2 proved to be beneficial, as reflection on the content was reinforced through discussion among students. In the teaching evaluation, the use of clickers was rated on a scale from 1 (very positive) to 5 (very negative) with an average of 1.4. One student commented: "With the help of the clicker questions, it was easy to assess whether you had understood the subject matter. In addition, you were "woken up" if your thoughts wandered. Otherwise, the lecture material was well absorbed through nice anecdotes and a relaxed style." A good prerequisite for the use of ARS was the reduction of the amount of material by approx. 25% in favor of a lecture atmosphere conducive to learning, which was achieved in the course of many years of teaching.

Prof. Dr. Martin M?hrle (Department 7: Economics):
 The 90-minute lecture "Marketing" is offered in the summer semester for students of business administration in the 2nd semester and all other business-related courses of study both in attendance and as a mobile lecture (mlecture.uni-bremen.de). The number of participants was 900 in 2014 and over 1000 in 2015. 700 students were present at the beginning and 200 at the end of each semester. In both semesters, PINGO was used to ask a few knowledge or assessment questions after about 45 minutes, which related to the content of the first half of the lecture. This timing was chosen because, as expected, student attention drops then and restlessness increases in the lecture hall. The multiple-choice questions were designed in such a way that the correct answer was not obvious, but rather the solution triggered an "aha" effect in the students. Example: "The relevant market for gummy bears includes ..." with the answer options fruit gums, chocolate products, play shapes for the sandbox and pacifiers (multiple choice possible). As expected, almost 70% of the students chose fruit gums, but all four answers were correct, depending on the underlying concept. Despite the large audience, there were neither technical nor organizational problems; even with more than 200 participants, the WLAN capacity was sufficient. The time required for the ARS deployment was only 3-5min. The goal of recapturing the attention of the students in this way was successful at the beginning, the participation rate was about 30 to 50%. However, the desired effect decreased significantly over the course of the semester, so that in the summer semester 2015 not every lecture was enriched with ARS questions. Prof. M?hrle comments: "In large events, ARS are like spices: They are useful if you use them sparingly."
 
 Tobias Tkaczick (Department of Social Sciences, Geography):
 Tobias Tkaczick is a lecturer in Department 8 (Geography program) and has regularly used PINGO for the 90-minute Introduction to Cartography lecture since winter semester 2012/2013. The 100 to 160 participants are full majors for the B.Sc. and the B.A. in the 1st semester, but also student teachers and complementary majors in the 3rd semester. At the beginning of the lecture, two review questions serve to recall the material of the last lecture and to find an introduction to the new topic. In the further course, each 20-minute teaching unit ends with a PINGO session in which the students can check whether they have processed the previously heard material by means of application and comprehension questions. If, for example, cartographic information systems were discussed separately in the teaching unit, they are to be identified in the question round in a complex map via a multiple choice task. Discussions in the approx. 1-2 minute voting phase are explicitly desired. After presentation of the result, silence returns immediately and the solution is, depending on the answer behavior, only briefly explained or discussed with the plenum. A mix of simple and more complex questions creates a balance between a regular sense of achievement and a challenge. The depth of the ARS questions is limited compared to those of the written exam, as they have to be oriented to the short voting time. While initially about 70% of students participate in the polls, by the end of the semester the percentage is about 40%. The reasons for this decline are unclear. In addition to providing feedback on learning success and interaction with students, ARS use also serves to break up the otherwise long presentation periods. The lecturer also receives immediate feedback on whether the content has been successfully conveyed. The reduction of the lecture material to the essentials, which is necessary due to the time required for the PINGO application (approx. 15 minutes), is perceived as a positive side effect. In the course evaluation, the students were enthusiastic about the use of PINGO, which will be continued in the coming semesters.