Overview of Automated Grading Systems
If bridges and buildings were made like we make software, then we would have disasters happening daily. I have heard this several times from many people. It is sad but true. Buggy software is the bane of the software industry. One of the ways of increasing software quality is by proper education. Several professionals from the software industry also attest to this. They believe that a greater emphasis should be given to quality and testing in university courses. But simply explaining the principles of software quality is not sufficient. Students tend to forget theoretical principles over time. Practical exposure and experience is equally important. Students should be put in an environment where they can appreciate the importance of quality software and can experience the benefits of processes that enhance quality. Many universities have a period of internship for the students in which they work in a software company and experience these factors first hand. However because the internship usually is of a duration of 3-6 months, it is not sufficient to instill the importance of quality. Emphasis on code quality should be made a part of the entire software curriculum for it to have proper impact. Every assignment that the students submit should be subjected to the same quality standards that an industrial project would be subjected to.
Having university assignments adhere to industrial standards will result in the faculty having to spend more time grading the assignments. The faculty can no longer just give an assignment, wait for the students to submit it, and grade them. The faculty must be more like a project manager who constantly mentors the students and helps them improve the quality of their work. Along with spending a good amount of time mentoring students off class hours another challenge is timely evaluation of student assignments. Faculty members are already overloaded with the task of teaching, designing projects, grading, and research. Once we incorporate testing and quality into the curricula, each assignment will have to be graded along many more dimensions, such as quality of the tests, coverage of the tests, etc. This can be very time consuming. We need a mechanism which will automatically grade student assignments to the best possible extent, so that students are given a timely feedback, and faculty can focus more on providing feedback on the style, design, and documentation of the project. Such a system will also bring consistency to the grading process and will eliminate discrepancies due to instructors bias and lethargy. A good automated grading system should be capable of executing the test cases written by students as well as the faculty on the project, determining the coverage of the test cases, and compiling and executing the submitted programs. It should be configurable so that faculty can determine the importance of various factors that make up the final grade.
Several efforts have been made to design and implement automated grading systems in universities. Some existing systems are: 1. WEB-CAT
In this article I will briefly explain two such automated grading systems - WEB-CAT, and the Praktomat systems, and propose a system that contains useful features from them as well as some new features.
WEB-CAT was created at Virginia Tech university to address the need for incorporating software testing as an integral part of all programming courses. The creators realized the need for a software to automatically grade student assignments to enable faster feedback to students and to balance the working load of faculty members.
Since Test Driven Development (TDD) was to be used for all the assignments, the students had to be graded not only on the quality of code, but also on the quality of their test suite. WEB-CAT grades students on three criteria. It gives each assignment...
Please join StudyMode to read the full document