Creating a language project

This tutorial gets you started with language development by creating a language project and changing various aspects of the language. First follow the installation tutorial if you haven't done so yet.

Creating a new project

In Eclipse, open the new project dialog by choosing File ‣ New ‣ Project from the main menu. In the new project dialog, select Spoofax LWB ‣ Spoofax language project and press Next. In this wizard, you can customize the various names your language will use. However, for the purpose of this tutorial, fill in HelloWorld as the name of the project, which will automatically fill in the other elements with defaults. Then press Finish to create the project. There should now be a project named helloworld in the Package Explorer.

Adding syntax

First we will add some syntax to the language. Open the main SDF3 file helloworld/src/start.sdf3 file by expanding the folders and double-clicking the file. SDF3 is a meta-language (i.e., a language to describe languages) for describing the syntax of a language, from which Spoofax will derive the parser of your language. Under the context-free syntax section, replace the Start.Empty = <> line with Start.Program = <<{Part " "}*>>, indicating that the language accepts programs which consists of zero or more parts.

Part is a sort and must be defined by adding its name to the context-free sorts section on a new line.

Now we will add syntax productions to Part to the context-free syntax section. Add Part.Hello = <hello> on a new line, indicating that one sort of Part is the word hello. Then add Part.World = <world> on a new line, indicating that one sort of Part is the word world.

src/start.sdf3 full contents

module start

context-free start-symbols

  Start

context-free sorts

  Start
  Part

context-free syntax

  Start.Program = <<{Part " "}*>>
  Part.Hello = <hello>
  Part.World = <world>

lexical syntax

  LAYOUT = [\ \n\v\f\r]

context-free restrictions

  LAYOUT? -/- [\ \n\v\f\r]

To observe our changes, build the project by clicking on the project in the Package Explorer and choosing Project ‣ Build Project from the main menu, or by pressing Cmd+B on macOS or Ctrl+B on others. To see when the build is done, open the progress window by choosing Window ‣ Show View ‣ Progress. If the progress view is empty, the build is done. The initial build can be a bit slow because there is a lot of code to compile in the background. Subsequent builds will be faster due to incrementalization.

Create an example file for your language by right-clicking the project and choosing New ‣ File, filling in example/example1.hel as file name, and pressing Finish. Type a valid sentence such as hello world hello hello world in this file, and it will highlight purple indicating that hello and world are keywords.

We can also run a debugging command on the example file to check the AST that the parser produces. There are three ways to run this debugging command:

• Make sure the example1.hel editor is open and active (i.e., has focus), and choose from the main menu: Spoofax ‣ Debug ‣ Show parsed AST.
• Make sure the example1.hel editor is open and active (i.e., has focus), right-click in the editor and choose: HelloWorld ‣ Debug ‣ Show parsed AST.
• Right-click the example/example1.hel file in the Package Explorer and choose: HelloWorld ‣ Debug ‣ Show parsed AST

Running this command will open a new editor with the AST of the program, such as:

Program([Hello(), World()])

There are also (continuous) variants of these debugging commands which will keep updating the debugging view when the source program changes. You can drag tabs to the sides of the screen to keep multiple editors open simultaneously, for example to keep the continuous debugging view visible, and to keep the syntax definition files editable.

Warning

There is currently a bug where continuous debugging views are not updated any more after the language is rebuilt. In that case, you have to open the continuous debugging view again.

Testing the new syntax

We can also systematically test the syntax (and other facets) of the language with the Spoofax Testing Language (SPT). Open the SPT file helloworld/test/test.spt. This file contains one test which tests that the empty program parses successfully, which is still the case because a program can consist of 0 parts.

Add a new test case to the test suite by adding:

test hello world parse [[
  hello world
]] parse succeeds

which tests that hello world parses successfully.

You can also add negative tests such as:

test gibberish [[
  asdfasdfasdf
]] parse fails

test/test.spt full contents

module test

test parse empty [[]] parse succeeds

test hello world parse [[
  hello world
]] parse succeeds

test gibberish [[
  asdfasdfasdf
]] parse fails

If you keep the SPT file open and rebuild your language, the SPT tests will be re-executed to provide feedback whether your change to the language conforms to your tests. You can also run all SPT tests by right-clicking a directory with SPT files, or by right-clicking the language project, and choosing SPT ‣ Run SPT tests. This will (once the tests are done executing) pop up a SPT Test Runner view with the results of testing.

Changing syntax highlighting

Now we will change the syntax highlighter of the language. Open the main ESV file helloworld/src/main.esv. ESV is a meta-language for describing the syntax highlighter. Change the keyword : 127 0 85 bold line to keyword: 0 0 150 bold and build the project. Then check your example1.hel example file, it should now be highlighted blue.

To make iteration easier, you can drag the example1.hel tab to the side of the screen to open the language definition and example file side-by-side. You can play around with the coloring a bit and choose a style to your liking. Remember to rebuild the project after making a change to the language definition.

Troubleshooting

Check the troubleshooting guide if you run into problems.