Effectiveness Check Report

This report provides a detailed evaluation of responses to the effectiveness check

Key Performance Indicators

The effectiveness check report analyzes the following KPIs in the order mentioned.

This KPI shows the average score for all effectiveness questions (as a percentage) by quarter. The average score is calculated as follows:

Yes/No: (number of users who responded yes / total number of replies) * 100.

Single response: (average value of the responses / highest value) * 100.

Multiple choice: (average value of the responses / highest value) * 100.

Free text: (number of participants who responded positively / total number of replies) * 100.

Matrix: sum of the average values for each row / number of rows

For the Arrange feedback form question type, the average score is N/A.

This KPI displays the number of questions with a discrepancy of more than 50% between supervisor and learner feedback.

This KPI displays the number of questions with an average score below 50%, indicating that the corresponding courses were not well received by participants and require further improvement.

This KPI also displays the number of single-response questions that received at least one response lower than or equal to 20% of the maximum value, thereby helping to identify outliers.

This overview displays the question's name, text, type, published version, number of addressees (participants—enrolled, in progress, passed, failed, or cancelled if a response has been submitted—and their supervisor), number of replies, response rate, average response value in % (learner and supervisor combined), average response value (supervisor), average response value in % (supervisor), and average response value (learner).

It also displays the average response value in % (learner) and the question ID.

Drill-down Page

The drill-down page analyzes the following KPIs in the order mentioned.

This card is shown when the specific question is selected. It displays the question text, name, type, minimum value, maximum value, number of feedback forms, courses to which the question has been assigned, number of addressees, and number of replies.

This overview displays the course templates and courses to which the selected feedback question has been assigned, the addressees, the number of replies, and whether the feedback evaluation period is over.

This KPI displays the average effectiveness score (in %) provided by learners and supervisors for the selected feedback question.

This KPI displays the courses with the lowest average score (in %) provided by learners and supervisors for the selected feedback question.

This KPI displays the courses with the highest average score (in %) provided by learners and supervisors for the selected feedback question.

This KPI displays the trend of the combined average score (in %) for the learner and supervisor, compared across the course templates (including versions) for the selected question.

This KPI displays the number of participants who selected Yes and the number who selected No for the selected Yes/No feedback question, compared across the courses.

This KPI displays the number of participants who selected the specific option for the selected single-response or multiple-choice feedback question, compared across the courses.

This KPI displays the number of participants who selected the respective option in each row of the selected matrix feedback question, compared across the courses.

This KPI displays the number of participants who ranked the specific item first, second, and so on for the selected Arrange feedback question, compared across the courses.

This KPI displays the highest average-ranked option for the selected Arrange question, compared across the courses.

This KPI displays responses to the selected free-text feedback question, classifies them as positive, negative, or neutral, and compares the results across the courses.

Metrics

This overview displays the course name, response rate, average respondent score, score discrepancy (learner versus supervisor), average score (supervisor), average percentage score (supervisor), average score (learner), average percentage score (learner), and course card.

Metrics

The effectiveness check report allows the report results to be filtered by the following metrics in the listed order.

This metric filters the report by the desired feedback submission period during which the feedback form was submitted. The default time period is 6 months.

This metric lists all feedback form templates and feedback forms assigned to courses to which the reporter has access. Selecting a feedback form template also selects all listed feedback forms for that template.

When the “Feedback evaluation period is over” option is selected, only questions from feedback forms assigned to date-dependent courses for which the feedback evaluation period is over are considered.

N/A is shown if no such feedback question is available.

This metric lists all courses to which the reporter has access and their course templates. Selecting a course template also selects all listed courses for that template.

This metric lists all learning modes assigned to courses to which the reporter has access.

This metric lists all providers assigned to courses to which the reporter has access.

This metric lists all tutors assigned the tutor role in the course templates and courses to which the reporter has access.

This metric filters the report by the desired classification assigned to the feedback form question belonging to the effectiveness check feedback form for courses to which the user has access.