Building a Javascript Compiler for Maven
January 25th, 2012
This post is one in a series where I slowly assemble a Maven workflow for large javascript projects. If you came here via google, I recommend starting at the beginning: Designing a Javascript Maven Plugin. Alternatively, you can just go directly to the js-maven-plugin project on github.
Since there’s no sense in rebuilding something that already exists, I decided to start with google’s excellent closure compiler as a baseline, and simply write a maven wrapper around it. There’s nothing inherently wrong with the YUI Compressor; It’s quite a nice compressor. In the light of supporting more use cases, however, I feel that a compiler with more optimization options would be a better choice, especially since all of those options are individually configurable.
However before I wrote the plugin, there are a few things I had to take into consideration:
- Developers should be able to debug while developing. This means that the compiled code needs to be human readable.
- Optimization may impact code function, so the compiled code should be optimized.
- Once packaged, the code should be minified.
In practice, this means two compiler passes over the source code. The first is to perform code optimizations, yet provide a human readable source file for debugging (in jetty, for instance). The second is to minify, but not optimize, shortly before the asset is packaged. This suggests the following steps for my Javascript Library lifecycle.
- compile
- net.krotscheck:js-maven-plugin:optimize-js-lib
- prepare-package
- net.krotscheck:js-maven-plugin:minify-js-lib
Given that with Google Closure two steps are practically identical, most of the functionality will be abstracted. This means I’ll effectively be running the compiler twice, not a problem unless a browser decides to throw errors on whitespace usage.
The Compiler Plugin
There are three steps to compiling using Closure. First we need to gather all of our source files, then we need to configure the compiler, and then we’ll run the compiler and output our source. There is a fourth step- gathering dependencies, which I’ll cover in a subsequent post. None of these steps are especially difficult. Here’s step one, finding all of our source code.
Full source available on github: JSLibraryCompilerMojo.java
/**
* Location of the source directory.
*
* @parameter expression="${project.build.sourceDirectory}"
* @required
*/
protected File sourceDirectory;
/**
* This method scans the configured source directory of the Mojo and returns all
* javascript documents, filtering by extension.
*/
protected List<JSSourceFile> generateJavascriptSourceList() {
ArrayList<JSSourceFile> sourceList = new ArrayList<JSSourceFile>();
String[] extensions = { "js" };
getLog().debug("Discovering Source Files");
Collection<File> sourceFiles = FileUtils.listFiles(sourceDirectory, extensions, true);
for (Iterator<File> iterator = sourceFiles.iterator(); iterator.hasNext();) {
File file = (File) iterator.next();
sourceList.add(JSSourceFile.fromFile(file));
getLog().debug(" + " + file.getAbsolutePath());
}
return sourceList;
}
Here’s step two, writing our compiler configuration. Note that I consciously made the choice to reduce the complexity of what my users can alter for the sake of illustration; This is primarily done out of lazyiness- If I allow complete individual configuration of all properties, this would have taken me a lot longer (and this post would be long, too).
You’ll notice that I’ve parameterized the method- this is to make its reuse easier once we move on to the minify step.
Full source available on github: AbstractClosureMojo.java
/**
* The input language to validate with. Valid options are ec5, ec5-strict,
* and ec3.
*
* @parameter default-value="ec3"
*/
private String inputLanguage;
/**
* Parse the parameters that have been configured for this mojo and generate
* the appropriate compiler options object.
*
* @return
*/
protected CompilerOptions buildCompilerOptions(String level,
Boolean prettyPrint) {
// Create a new options construct.
CompilerOptions options = new CompilerOptions();
// Determine the input language switch: EC5, EC5-Strict, or EC3. Note
// that most browsers don't support EC5.
switch (inputLanguage) {
case "ec5-strict":
getLog().debug("Compiling for ECMAScript5 - Strict Mode");
options.setLanguageIn(LanguageMode.ECMASCRIPT5_STRICT);
break;
case "ec3":
getLog().debug("Compiling for ECMAScript3");
options.setLanguageIn(LanguageMode.ECMASCRIPT3);
break;
case "ec5":
default:
getLog().debug("Compiling for ECMAScript5");
options.setLanguageIn(LanguageMode.ECMASCRIPT5);
break;
}
/**
* Determine the compiler level.
*/
switch (level) {
case "simple":
getLog().debug("Optimization Level: Simple");
CompilationLevel.SIMPLE_OPTIMIZATIONS
.setOptionsForCompilationLevel(options);
break;
case "advanced":
getLog().debug("Optimization Level: Advanced");
CompilationLevel.ADVANCED_OPTIMIZATIONS
.setOptionsForCompilationLevel(options);
break;
case "whitespace":
default:
getLog().debug("Optimization Level: Whitespace");
CompilationLevel.WHITESPACE_ONLY
.setOptionsForCompilationLevel(options);
break;
}
if (prettyPrint) {
options.inputDelimiter = "// [%num%] %name%";
options.printInputDelimiter = true;
options.prettyPrint = true;
}
return options;
}
Our last step then is to invoke the closure compiler, which is also verbose but pretty straightforward. I’ve omitted some of my logging statements for simplicity.
Full source available on github: JSLibraryCompilerMojo.java
/**
* The compile level for google closure. Valid options are
* "whitespace", "simple" and "advanced".
*
* @parameter default-value="whitespace"
*/
private String compilationLevel;
/**
* This configures, resolves, and compiles the source for this project.
*/
public void execute() throws MojoExecutionException {
// Generate compiler options
CompilerOptions options = buildCompilerOptions(compilationLevel, true);
// Compiler option, not yet used.
List<JSSourceFile> externs = new ArrayList<JSSourceFile>();
// Generate the list of source files to include.
List<JSSourceFile> source = new ArrayList<JSSourceFile>();
source.addAll(generateJavascriptSourceList());
getLog().info("Compiling Javascript Library...");
// Generate the compiler instance and compile.
Compiler.setLoggingLevel(Level.OFF);
Compiler compiler = new Compiler();
compiler.compile(externs, source, options);
// Output the compiled code.
try {
// Helper method to make sure the output file exists first.
File newArtifactFile = getOutputFile();
getLog().info(" -> " + newArtifactFile.getAbsolutePath());
FileUtils.writeStringToFile(newArtifactFile, compiler.toSource());
} catch (IOException e) {
getLog().error("Unable to create artifact file: " + e.toString());
}
}
Done! Now all we have to do is compile our mojo and use the pom reference from my post about designing a javascript maven plugin, and then invoke maven.
No comments yet.