This document describes the Misty parse tree. This is not a part of the Misty System Standard. A conforming implementation does not have to conform to this. This is descriptive of one particular implementation.
The tree can be fully realized as a JSON object.
The subprogram tokenize.mst takes a source text and produces an array of tokens. The tokens are made up of these fields:
The kind and text fields will be meaningful to the parser that consumes the token list. The other fields are intended for error reporting.These token records are treated as nodes that are woven into a parse tree.
"name", "text", "number",
"comment", "newline", "space",
and all of the intrinsics, functinos, and punctuators, including:
. : ( ) [ ] { } + - * / // & && = <> < > <= >= /\ \/ |
The characters making up the token.
The kind: "text" tokens have already had escape sequences decoded and outer quotes removed.
A contigous run of spaces is made into a single kind: "space" token with a text field containing the entire run of spaces.
The numeric value of kind: "number" tokens.
The character position in the source text where the token begins, starting at 0.
The line number of the the token, starting at 0.
The column number of the start of the token, starting at 0.
The line number if the last character of the token.
The column number of the character past the token.
The number of spaces as an integer. This is in space tokens with a from_column of 0, and long texts.
If the token contains an error, then it will also contain these fields:
A brief description of the error. Numbers, texts, and functinos can contain errors.
The character position in the source text where the error was identified.
The column where the error was identified.
The row at which the error was identified.
The subprogram parse.mst turns a list of tokens into an abstract parse tree by adding new fields. The source notations are retained for error reporting and debugging.
The root of the tree is a record. Its fields are
An array of endowment names.
An array of functions. For each function there is a record in this array, plus function 0, the program or subprogram.
An array of intrinsics that are used. This information might be used by a linker.
"misty"
An array of log names.
"program" or "subprogram"
The name of the file.
An array of pattern definitions.
An array of scopes. This array is parallel with the functions array. It contains records containing the names that are used in each function. The names can be the function name and inputs, and additional names created with def, use, var. The scope also contains names that are found in outer functions and intrinsics.
A scope is used to determine what a name in a function refers to.
An array of subprograms that are mentioned in use statements. This information might be used by a linker.
Names are use to represent functions, parameters, variables, and constants.
"name"
The number of the function in which this name is defined.
0 if a variable is declared in the current function
1 if a variable is declared in the outer function
2 or after and so on
"def" "endowment" "function" "intrinsic" "parameter" "use" "var"
The name, as a simple text.
More information about the name token can be found in the scope.
A functino is an operator function. It looks like an operator, but it behaves as an intrinsic function. It has a name token, with the operator as the name.
Number tokens have their values in text form so that the interpretation of the number can be delayed as long as possible. This is to minimize rounding and other representation errors.
"number"
A number in text form. This is important during bootstrapping because JavaScript is not able to exactly represent Misty numbers.
A number.
"text"
A text.
On long text literals, this is the numeric indentation of the close quote.
"array"
An array (possibly empty) of expression nodes.
"record"
An array (possibly empty) of pair records. A pair record is
first
A simple text.
second
An expression node.
"space"
The spaces.
The number of spaces.
The left operand node.
The right operand node.
| * / // + - & && = <> < <= > >= /\ \/
. operator"."
The left operand node.
A text token.
"["
The left operand node.
The right operand node. If right is missing, then [] are representing the push/pop operator.
"("
The left operand node, which should resolve to a function.
An array of expression nodes, an argument list.
then operatorThis is the ternary operator.
"then"
A condition node.
The expression node if true.
The expression node if false.
The first and second may also have a box field, which if true, indicates that the first result receives the last element of the destination array, removing it from the destination array, reducing its length by 1, or that the second result is appended to the destination array.
"assign"
The destination expression node.
The value expression node.
true if first ended with [].
true if second ended with [].
"break"
If the statement is labelled, this is a simple text.
"call"
An invocation expression.
"def"
A name token.
An expression node.
"disrupt"
"do"
An array of statement nodes.
If the do statement is labelled, this is a simple text.
"if"
A conditional expression node.
An array of statement nodes.
An array of if statement nodes (without else_if or else) that are from else if.
an array of statement nodes (optional).
"jump"
An invocation expression.
"return"
An optional expression node.
"send"
The address expression node.
The message expression node.
The optional reply function expression node.
"use"
A name token.
An invocation expression.
"var"
A name token.
An optional expression node. The default is null.
The function operator does two things: It makes a function definition that it appends to the functions array, and it makes a reference to that function definition.
"function" "program" "subprogram"
The location of the function definition in the functions array.
If it is an operator function, then name will be name of the operator (functino), such as "'+'". The name will also be noted in the intrinsics.
An array of statement nodes, the function's disruption handler. (Optional.)
"function"
A list of name nodes, the parameters. Each has a parameter_nr field, 0 or greater. If a parameter has a default value, it will be in the name token's first field.
If the function is named, this is a simple text.
The number of the outer function that made this one.
An array of statement nodes.
The number of slots that are protected during memory reclamation.
The number of slots required for the this function's inputs, vars, and defs.
The root node contains an array of scopes. The scopes array is indexed by function_nr, like the functions array. Each function definition has a scope record containing all of the names used in that function.
A scope record contains name fields, where the key is a variable name, and the field is a record containing:
If the name is used by an inner function, then this is true.
The number of the this function in the functions array..
If this is an intrinsic functino, then true.
The distance to the outer scope.
The maker of the name:
def endowment function parameter intrinsic use var
Only names that were made by the var statement can have their values replaced by the assign statement.
The name of the variable.
The number of appearences where the variable is actually used.
The pattern operator does two things: It makes a function definition that it appends to the patterns array, and it makes a reference to that pattern definition.
"pattern"
The location of the pattern definition in the patterns list.
Coming soon.