On this page
The assignment submission system accepts assignments over the internet. Any assignment that can be saved as a file can be submitted through the system.
Adding a course to the submission system
The lecturer or teaching assistant provides the administrators with the course number and groups, for example course 89-999, groups 01, 02, 03. Administrators create an account named after the course, if one does not exist, and give its password to the lecturer.
Course directory structure
- A subdirectory for each course group stores its submissions.
- WWW holds the course website.
- data holds configuration and grades, including submit.txt, which records submissions, and, for automatically checked assignments, grades.txt, which records submissions and grades.
Accessing the course directory and website
To access the course directory and run the following commands, connect to planet by SSH and sign in as the course account, for example 89-999. The course website is at u.cs.biu.ac.il/~course_name.
Adding an assignment
Each assignment’s files are stored in a directory named after the assignment, inside each course group’s directory. Create these directories with:
make_targil targil_name
The assignment directories are scanned hourly. New assignments then appear in the web submission interface.
Assignment path
course_name/group_name/targil_name/user_name-reduction~
Accessing previous years’ courses
cd ../courses_73/course_name or cd /u/courses/courses_73/course_name
courses_73 stores the courses and assignments submitted during the Hebrew year 5773 (2012–2013).
Setting submission deadlines
Set deadlines in ~course_name/lastdates. This file requires -rw-r--r-- permissions.
To set a deadline for all students:
targil_name DDMMYY reduction
reduction is the percentage deducted for late submission. Use a one- or two-digit integer.
To set a deadline for a group or an individual student:
targil_name DDMMYY reduction [ user_name | group_name ]
Separate fields with spaces or tabs. Blank lines and lines starting with # can separate entries.
Example lastdates file
targil1 010113 0
targil1 020113 0 03
targil1 200213 0 student
targil1 100313 15
The example sets these deadlines for targil1:
- Everyone, without a deduction: 1 January 2013.
- Group 03, without a deduction: 2 January 2013.
- The specified student, without a deduction: 20 February 2013.
- Everyone, with a 15-point deduction: 10 March 2013.
Of the applicable deadlines, the system chooses the one with the smallest deduction. After all applicable deadlines have passed, submissions are rejected.
Extracting ZIP and RAR files
Students can submit multiple files in a ZIP or RAR archive. By default, all files are extracted into a single, flat directory.
To preserve the directory hierarchy, for example for Ant projects or assignments requiring evidence of Git use, create an empty file named ~course_name/.nonflat.
This applies to all assignments in the course.
Automatic checking
Automatic checking follows these steps:
- Compile the submitted program.
- Run it with inputs supplied by the teaching assistant.
- Compare its outputs with the expected outputs.
- Award the score assigned by the teaching assistant for each test case.
The system supports C, C++, Java, Python, Assembly, SCIM, SML, Prolog, Perl, and MATLAB.
Prepare two kinds of files in data:
- auto.cfg, specifying properties of the course’s assignments. Its presence marks the course for automatic checking.
- targil_name.io, containing test inputs and expected outputs, one file per assignment.
The auto.cfg file
This uses INI format. Each assignment has a section starting with [targil_name], followed by parameter=value lines. The name must match the one passed to make_targil. Parameters may appear in any order.
Example
[targil1]
required parameter#
compiler = java
all other parameters are optional#
input = myinput
output = myoutput
timeout = 60
timing = 1
memleak = 1
feedback = 1
data_files = targil1_data
weight = 25
message = A bonus will be given for efficient programs
[targil2]
compiler = cpp
input = stdin
output = stdout
[targil3]
compiler = make
alias=<<END_ALIAS
targil3.1
targil3.2
targil3.3
END_ALIAS
Parameters and values
compiler = # required parameter
c — The file "targil_name.c" is compiled with the command
gcc targil_name.c -lm -ansi -pedantic; "a.out" is run
cpp — The file "targil_name.cpp" is compiled with the command
g++ targil_name.cpp -lm -ansi -pedantic; "a.out" is run
s — The file "targil_name.s" is compiled with the command
gcc -m32 targil_name.s; "a.out" is run
java — The files are compiled with the command
javac *.java"; "java targil_name" is run
py — No compilation; "python targil_name.py" is run
scm — No compilation; "guile -s targil_name.scm" is run
sml — No compilation; "sml @SMLload=heapimage.x86-linux" is run
pl — No compilation
swipl -q -g "consult('$exercise.pl'), $input_string" -t "halt" is run
plx — No compilation; "perl targil_name.plx" is run
make — The files are compiled according to a "makefile" supplied by student
a.out" is run"
dm — dummy; submitted files may be checked latter
m — No compilation; 'matlab -nodisplay -r "check(),quit"' is run
(check.m is supplied by metargel to check the .m files sent by students)
sh - bash
ant – "ant compile" for compilation and "ant" to run. It sets appropriate ANT_HOME environment variable.
ant_compile – just compile with "ant compile" but do not run it.
net — Submitted files will be copied to and executed on virtual machines to test
network connectivity
input = # default = stdin
.stdin — the program should read its input from standard input
.standard input will contain the test input
Any "file_name" except "stdin" — the program should read its input from
.a file named "file_name". The file "file_name" will contain the test input
output = # default = stdout
.stdout — the program should write its output to standard output
.standard output will be compared to the correct output
Any "file_name" except "stdout" — the program should write its output to a file
.named "file_name". The file "file_name" will be compared to the correct output
timeout = # default = 12
.A timeout in seconds that limits the execution of checked programs
timing = # default = 0
1 — timing is enabled, execution time in seconds is recorded in "grades.txt" next
to the mark of each test case, in the format "mark/time".
.Execution time is reported to student if feedback = 1
.The command "make_csv_long" will display this information
memleak = # default = 0
1 — memleak is enabled, memory leak in bytes is recorded in "grades.txt" next
to the mark of each test case, in the format "mark/time/lost/not-freed".
.Memory leak is reported to student if feedback = 1
.The command "make_csv_long" will display this information
feedback = # default = 0
1 — email sent as a response to submission of checked programs
.will include feedback
exact = # default = 1
1 — In both program output and correct output, whitespace is trimmed
before and after each line and consecutive spaces and/or tabs within
line are replaced by one space. Also, empty lines are trimmed too.
To avoid these behaviors set exact to 1.
data_files = # default = none
dir_name — a copy of the files within the directory
course_name/data/dir_name" will be available to the program~"
.at execution time for reading and writing
.The files will be copied again for each test case
weight = # default = 0
This field will be used by make_csv to calculate the average grade
.for the course assignments
alias = # default = none
alias_name — input/output for checking will be taken from
.the file alias_name.io
Files within the directory "~course_name/data/alias_name" will be
.compiled together with the submitted files
.The grade will be given for targil alias_name
More than one alias can be denoted using the shell "here document syntax":
alias=<<END_ALIAS
alias1
alias2
END_ALIAS
To set a weight to aliases, each alias needs a section with a "weight" parameter
.(see example above)
.Example using aliases, submit t1.c and t2.c to test
:auto.cfg
[test]
compiler = c
alias=t1
alias=t2
:t1.io
END_INPUT
one
END_OUTPUT 50
:t2.io
END_INPUT
two
END_OUTPUT 50
.or: submit one.c and two.c to test
:auto.cfg
[test]
compiler = make
alias=t1
alias=t2
:t1/makefile
a.out: one.c
gcc one.c
:t2/makefile
a.out: two.c
gcc two.c
message = # default = none
This field can be used to send a message to students,
the message will be attached to the email sent after submission.
gitcheck = # default = 0
Make sure the student made use of git (for that contents of .git directory must be sent).
P.s. make sure to disable flat extraction of archive (zip, rar) submitted files (to make sure .git directory will be in place).
giturl = # default = none
This is a link to a repository given to the students. If student’s git url (in .git/config) equals this one its implies that the student made a local copy (clone) of the repository instead of making a separate copy of it on git server (forking).
Giturl implies gitcheck.
The targil_name.io file
This contains inputs and expected outputs for each test case. A line beginning END_INPUT ends an input, and a line beginning END_OUTPUT ends the expected output. A score follows END_OUTPUT. If the tested program’s output matches the expected output, that score is added to the assignment grade.
In this example, the program reads two numbers and prints their average. The first case inputs 3.0 and 4.0, expecting The average is 3.5, worth 60 points. The second inputs 1.0 and 0.5, expecting The average is 0.75, worth 40 points.
3.0
4.0
END_INPUT
The average is 3.50
END_OUTPUT 60
1.0
0.5
END_INPUT
The average is 0.75
END_OUTPUT 40
Command-line arguments in targil_name.io
To pass command-line arguments to the program, append them to each test case’s END_INPUT line.
The first test case passes two command-line arguments, arg and 20. The second passes myfile, which can be used as the name of a file for the program to open. The second test case has no input, so its END_INPUT myfile line immediately follows the preceding test case’s END_OUTPUT 50 line.
input1
input2
END_INPUT arg 20
output1
output2
END_OUTPUT 50
END_INPUT myfile
output3
output4
END_OUTPUT 50
Line comparison in targil_name.io
Normally the checker compares the program output with the expected output and awards the test case’s points if they match. Before comparing, it replaces sequences of spaces and tabs with a single space in both outputs.
You can instead award points according to the number of differing lines. Put several space-separated numbers after END_OUTPUT. They define the score for n differing lines:
END_OUTPUT 20 15 10
Here, 20 is awarded for zero differing lines, 15 for one, and 10 for two. More differences receive no points.
Line comparison uses the diff algorithm. Each added, removed, or changed line counts as an error. If m lines are changed to n lines, the error count is max(m,n). A moved line counts as one error.
Adding teaching-assistant files at compilation time
To compile additional files, such as main.cpp, together with submitted files, create a directory named after the assignment inside data:
~course_name/data/targil_name
Place the additional files there. If a submission contains a filename also present in that directory, it is rejected. If the assignment uses aliases, you can create a directory named after each alias.
Adding teaching-assistant files at runtime
To make data files available at runtime, set data_files to a directory name in auto.cfg:
data_files=data_dir
Create that directory inside data:
~course_name/data/data_dir
Put the data files there. If a submission contains a filename also present in this directory, it is rejected. The tested program can read and write the data files. Fresh copies are made for each test case.
Temporary files
The tested program may create temporary files in its current directory.
Alternative files for manual checking
For manual checking, you can create an alternative IO file and/or teaching-assistant file directory, separate from automatic-checking files:
~course_name/data/targil_name.io.manual
~course_name/data/targil_name.manual
Detailed checking of a submission
The following command rechecks a submission, displays its input, and compares outputs:
check_ex full_path_to_targil_dir
or
check_ex targil_name/user_name
For example:
check_ex /u/courses/89-999/01/targil1/student-0
or
check_ex targil1/student
Use it when preparing test inputs or investigating a grade. It does not write a grade to the grades file and does not send email.
Rechecking all submissions for an assignment
This command rechecks all submissions. If you edited targil_name.io, it uses the updated inputs:
check_again [group/]targil_name [ nomail | mail | feedback | debug ]
- No extra parameter: displays grades, without updating the grades file or sending email.
- nomail: displays grades and updates the grades file, without sending email.
- mail: displays grades, updates the grades file, and emails the submitter.
- feedback: displays grades, updates the grades file, and emails the submitter with test inputs and output comparisons.
- debug: displays grades, test inputs, and output comparisons, without updating the grades file or sending email.
The grades file
The checker appends the grade for each checked assignment to:
~course_name/data/grades.txt
Do not edit this file while the system is running. A mirrored backup is kept elsewhere.
Creating a grades table
This command reads grades.txt and creates a table using the most recent submission of each assignment:
make_csv [ targil1 ... ]
Without parameters, it includes all assignments. With parameters, it includes only the listed assignments. The file course_name.csv is created in course_name/data. Each row contains the student’s name, ID number, login, assignment grades, and weighted average, separated by commas.
Other command variants:
- make_csv_leak: includes memory-leak information.
- make_csv_long: lists each test case’s grade separately.
- make_csv_submissions: includes grades for all submissions.
- make_csv_timing: includes execution times.
Checking for plagiarism
The following command sends submitted files to moss.stanford.edu, which checks similarities between programs:
moss targil_name [ s | t | l | y | m | c ]
The server returns a URL with the results. It needs to know the programming language. The command infers this from auto.cfg, so no additional language parameter is needed for automatically checked C, C++, Java, or Python assignments. Assignments configured as make are sent as C++.
If the language cannot be inferred, add one of these parameters:
- s:
.sfiles (8086 assembly). - t:
.txtfiles (checked as text). - l:
.lexfiles (checked as text). - y:
.yfiles (checked as text). - m:
.mfiles (MATLAB). - c: an assignment configured as make that contains C files.
To exclude particular files:
moss targil_name -b filename1 -b filename2 lang

