Due: 11am Monday 3 September (Week 6)
Worth: 10%
This assignment asks you to develop a lexical analyser, parser and tree builder for a version of the Java programming language. We will build on these components in later assignments to build a full implementation of this Java version.
Building this implementation will give you insight into the way that programming language implementations work in general, as well as specific experience with how Java is compiled and executed.
This kind of task often arises in programming situations other than language implementation. For example, many applications have configuration files that are written in simple languages. The application must be able to read these files reliably and understand their structure, just as a compiler must read program files and understand them.
Aspects such as checking the validity of names or types and translating the program into an executable form are beyond the scope of syntax analysis. We will address them in Assignments Two and Three.
Java is a very widely used object-oriented programming language. Building a compiler for the whole of Java is not feasible for a unit like COMP332. Instead, we will implement a cut-down version called MiniJava. This exercise will still require you to come to terms with the fundamental issues for progarmming language implementation, but within a more tractable setting.
MiniJava is a strict subset of Java where programs contain a
main class and some number of other classes. Classes can contain
private field and public instance method declarations, but more advanced
features such as visibility control, static methods and overloading
are not present.
The data types other than class instances are just integers, Booleans
and integer arrays.
Expressions are restricted to basic operations on integers, Booleans
and arrays, plus calling instance methods. Allowed statements are
assignment, conditionals, while loops and blocks. There is no
mechanism for input, so data must be encoded in the program. Output
is restricted to calling System.out.println.
As an example, here is a simple factorial calculation implemented in MiniJava:
class Factorial {
public static void main () {
System.out.println (new Fac ().ComputeFac (10));
}
}
class Fac {
public int ComputeFac (int num) {
int num_aux;
if (num < 1)
num_aux = 1;
else
num_aux = num * (this.ComputeFac (num - 1));
return num_aux;
}
}
You can find more information about MiniJava at the
web site of the Modern Compiler Implementation in Java textbook.
In particular, that site features some more MiniJava programs
that you might find useful. They are also included in the
test directory of the assignment skeleton,
modified to remove the main argument array, since we are not
handling that in this assignment.
Note that the implementation approach used in that textbook is not the same as the one we will be using in our assignments. In particular, that textbook follows an approach that uses some tools and hand-codes the rest of the compiler in Java. In contrast, we will write our implementations in Scala using the Scala and Kiama libraries, without any tools other than sbt and the Scala compiler.
You have to write, document and test a Scala lexical analyser, parser and tree builder for MiniJava as described below. There is one part for each of two passing assessments standards: a) P and Cr, and b) D and HD, with part (b) requiring more independent work than part (a).
You are strongly advised to complete each part of the assignment before moving onto the next one. In fact, within each part it is advisable to solve small portions at a time rather than trying to code the whole solution in one go.
To guide your implementation, here is a complete context-free grammar for the MiniJava language
Program : MainClass ClassDeclaration*.
MainClass : "class" Identifier "{"
"public" "static" "void" "main" "(" ")" "{"
Statement
"}"
"}".
ClassDeclaration : "class" Identifier ("extends" Identifier)? {"
VarDeclaration*
MethodDeclaration*
"}".
VarDeclaration : Type Identifier ";".
MethodDeclaration : "public" Type Identifier "(" Arguments ")" "{"
VarDeclaration*
Statement*
"return" Expression ";"
"}".
Arguments : Type Identifier ("," Type Identifier)*
| /* empty */.
Type : "int"
| "boolean"
| "int" "[" "]"
| Identifier.
Statement : "{" Statement* "}"
| "if" "(" Expression ")" Statement "else" Statement
| "while" "(" Expression ")" Statement
| "System.out.println" "(" Expression ")" ";"
| Identifier "=" Expression ";"
| Identifier "[" Expression "]" "=" Expression ";".
Expression : Expression ("&&" | "<" | "+" | "-" | "*") Expression
| Expression "[" Expression "]"
| Expression "." "length"
| Expression "." Identifier "(" ExpressionList ")"
| Integer
| "true"
| "false"
| "this"
| Identifier
| "new" "int" "[" Expression "]"
| "new" Identifier "(" ")"
| "!" Expression
| "(" Expression ")".
ExpressionList : Expression ("," Expression)*
| /* empty */.
Note: in this syntax definition, the comment /* empty */
denotes an empty alternative.
The symbols Integer and Identifier are terminals
in this grammar. Both integer literals and identifiers match the longest
sequence of characters that they can.
An Integer comprises a sequence of one or more digits.
An Identifier comprises a sequence of characters that starts
with a letter and continues with zero or more letters, digits or underscores.
For the purposes of this assignment, you don't need to worry about identifiers
that look exactly like keywords. Just assume that they will not occur in the
input.
Whitespace does not contribute to the program structure, except to separate symbols that would otherwise be merged without the whitespace. E.g., the characters "onetwo" denote a single identifier, whereas the characters "one two" denote two consecutive identifiers.
The binary operators &&, <, +,
- and * are all left associative, except for
< which is not associative. The binary operators have a precedence
order given by the previous sentence, from lowest to highest, except that
+ and - have the same precedence.
The first part of the assignment involves building a syntax analyser and tree builder for the MiniJava language as defined above.
Your code must use the Scala parsing library as discussed in lectures and practicals. You should use the expression language syntax analyser and tree builder as a guide for your implementation.
A skeleton sbt project for the assignment has been provided on
BitBucket as the inkytonik/comp332-ass1 repository.
The modules are very similar to those used in the practical exercises
for Week 2 and 3. The skeleton contains the modules you will need.
Some of the parsing and tree construction is given to you as an
illustration; you must provide the rest (look for FIXME in the
code).
As well as lexing and parsing the input, your program should
construct a suitable source program tree to represent the parsed
result. See MiniJavaTree.scala in the skeleton for
the full definition and description of the tree structures that
you must use. For this part of the assignment, you should not
have to modify the tree classes, just create instances in your
parser code.
As an example of the desired tree structure, here is the factorial program from above in source program tree form. Make sure you understand this structure and the nodes in it, before you start coding the assignment. This program does not use all of the tree node classes.
Program (
MainClass (
"Factorial",
Println (
CallExp (NewExp ("Fac"), "ComputeFac", List (IntExp (10))))),
List (
Class (
"Fac",
None,
Nil,
List (
Method (
IntType (),
"ComputeFac",
List (Argument (IntType (), "num")),
List (Var (IntType (), "num_aux")),
List (
If (
LessExp (IdnExp ("num"), IntExp (1)),
VarAssign ("num_aux", IntExp (1)),
VarAssign (
"num_aux",
StarExp (
IdnExp ("num"),
CallExp (
ThisExp (),
"ComputeFac",
List (
MinusExp (
IdnExp ("num"),
IntExp (1)))))))),
IdnExp ("num_aux"))))))
The second part of the assignment entails extending the language. The aim is to give you some exposure to researching a language feature and its syntax, then adding it to the syntax analyser and tree builder implementation.
Specifically, you should devise an extension that supports the use
of
As well as determining an appropriate syntax for MiniJava interfaces and recognising it in your parser, you must augment your tree definition to contain whatever new classes you need to appropriately represent interfaces in the source program tree. You should re-use existing classes where it makes sense, but you will at least need a new class to represent interfaces themselves.
Consult the teaching staff of the unit if you are in doubt about what to include in your extension.
test/factorial.java you use the command
run test/factorial.java
Assuming your code compiles and runs, this will print the tree that has been constructed (for correct input), or will print a syntax error message (for incorrect input).
The project is also set up to do automatic testing. See the file
ParsingTests.scala which provides the necessary
definitions to test the syntax analyser on some sample inputs. Note
that the tests we provide are not sufficient to test all of
your code. You must augment them with other tests.
You can run the tests using the test command in sbt. This
command will build the project and then run each test in turn,
comparing the output produced by your program with the expected
output. Any deviations will be reported as test failures.
All of the code for your syntax analyser. To make this clear, submit every file that is needed to build your program from source, including files in the skeleton that you have not changed. Do not add any new files or include multiple versions of your files. Do not include any libraries. We will compile all of the files that you submit using sbt, so you should avoid any other build mechanisms.
Your submission should include all of the tests that you have used to make sure that your program is working correctly. Note that just testing one or two simple cases is not enough for many marks. You should test as comprehensively as you can.
A type-written report that describes how you have achieved the goals of the assignment. Your report must contain the following components or sections:
Submit your code and report electronically as a single zip file
called ass1.zip using the appropriate submission link on
the COMP332 iLearn website by the due date and time. Your report
should be in PDF format. DO NOT SUBMIT YOUR ASSIGNMENT OR DOCUMENTATION
IN ANY OTHER FORMAT THAN ZIP and PDF, RESPECTIVELY.
The assignment will be assessed according to the assessment standards for Learning Outcomes 2, 3 and 4 as specified in the Unit Guide.
Marks will be allocated equally to the code and to the report. Your code will be assessed for correctness and quality with respect to the assignment description. Marking of the report will assess the clarity and accuracy of your description and the adequacy of your testing. Marks allocated to testing will be 30% of the marks for the assignment.