This post is one in a series where I slowly assemble a Maven workflow for large javascript projects. If you came here via google, I recommend starting at the beginning: Designing a Javascript Maven Plugin. Alternatively, you can just go directly to the js-maven-plugin project on github.

Since there’s no sense in rebuilding something that already exists, I decided to start with google’s excellent closure compiler as a baseline, and simply write a maven wrapper around it. There’s nothing inherently wrong with the YUI Compressor; It’s quite a nice compressor. In the light of supporting more use cases, however, I feel that a compiler with more optimization options would be a better choice, especially since all of those options are individually configurable.

However before I wrote the plugin, there are a few things I had to take into consideration:

1. Developers should be able to debug while developing. This means that the compiled code needs to be human readable.
2. Optimization may impact code function, so the compiled code should be optimized.
3. Once packaged, the code should be minified.

In practice, this means two compiler passes over the source code. The first is to perform code optimizations, yet provide a human readable source file for debugging (in jetty, for instance). The second is to minify, but not optimize, shortly before the asset is packaged. This suggests the following steps for my Javascript Library lifecycle.

compile

net.krotscheck:js-maven-plugin:optimize-js-lib

prepare-package

net.krotscheck:js-maven-plugin:minify-js-lib

Given that with Google Closure two steps are practically identical, most of the functionality will be abstracted. This means I’ll effectively be running the compiler twice, not a problem unless a browser decides to throw errors on whitespace usage.

The Compiler Plugin

There are three steps to compiling using Closure. First we need to gather all of our source files, then we need to configure the compiler, and then we’ll run the compiler and output our source. There is a fourth step- gathering dependencies, which I’ll cover in a subsequent post. None of these steps are especially difficult. Here’s step one, finding all of our source code.

/**
 * Location of the source directory.
 *
 * @parameter expression="${project.build.sourceDirectory}"
 * @required
 */
protected File sourceDirectory;

/**
 * This method scans the configured source directory of the Mojo and returns all
 * javascript documents, filtering by extension.
 */
protected List<JSSourceFile> generateJavascriptSourceList() {
    ArrayList<JSSourceFile> sourceList = new ArrayList<JSSourceFile>();

    String[] extensions = { "js" };
    getLog().debug("Discovering Source Files");

    Collection<File> sourceFiles = FileUtils.listFiles(sourceDirectory, extensions, true);

    for (Iterator<File> iterator = sourceFiles.iterator(); iterator.hasNext();) {
         File file = (File) iterator.next();

        sourceList.add(JSSourceFile.fromFile(file));

        getLog().debug(" + " + file.getAbsolutePath());
    }

    return sourceList;
}

Here’s step two, writing our compiler configuration. Note that I consciously made the choice to reduce the complexity of what my users can alter for the sake of illustration; This is primarily done out of lazyiness- If I allow complete individual configuration of all properties, this would have taken me a lot longer (and this post would be long, too).

You’ll notice that I’ve parameterized the method- this is to make its reuse easier once we move on to the minify step.

/**
 * The input language to validate with. Valid options are ec5, ec5-strict,
 * and ec3.
 *
 * @parameter default-value="ec3"
 */
private String inputLanguage;

/**
 * Parse the parameters that have been configured for this mojo and generate
 * the appropriate compiler options object.
 *
 * @return
 */
protected CompilerOptions buildCompilerOptions(String level,
                Boolean prettyPrint) {

        // Create a new options construct.
        CompilerOptions options = new CompilerOptions();

        // Determine the input language switch: EC5, EC5-Strict, or EC3. Note
        // that most browsers don't support EC5.
        switch (inputLanguage) {
        case "ec5-strict":
                getLog().debug("Compiling for ECMAScript5 - Strict Mode");
                options.setLanguageIn(LanguageMode.ECMASCRIPT5_STRICT);
                break;
        case "ec3":
                getLog().debug("Compiling for ECMAScript3");
                options.setLanguageIn(LanguageMode.ECMASCRIPT3);
                break;
        case "ec5":
        default:
                getLog().debug("Compiling for ECMAScript5");
                options.setLanguageIn(LanguageMode.ECMASCRIPT5);
                break;
        }

        /**
         * Determine the compiler level.
         */
        switch (level) {
        case "simple":
                getLog().debug("Optimization Level: Simple");
                CompilationLevel.SIMPLE_OPTIMIZATIONS
                                .setOptionsForCompilationLevel(options);
                break;
        case "advanced":
                getLog().debug("Optimization Level: Advanced");
                CompilationLevel.ADVANCED_OPTIMIZATIONS
                                .setOptionsForCompilationLevel(options);
                break;
        case "whitespace":
        default:
                getLog().debug("Optimization Level: Whitespace");
                CompilationLevel.WHITESPACE_ONLY
                                .setOptionsForCompilationLevel(options);
                break;
        }

        if (prettyPrint) {
                options.inputDelimiter = "// [%num%] %name%";
                options.printInputDelimiter = true;
                options.prettyPrint = true;
        }

        return options;
}

Our last step then is to invoke the closure compiler, which is also verbose but pretty straightforward. I’ve omitted some of my logging statements for simplicity.

/**
 * The compile level for google closure. Valid options are
 * "whitespace", "simple" and "advanced".
 *
 * @parameter default-value="whitespace"
 */
private String compilationLevel;

/**
 * This configures, resolves, and compiles the source for this project.
 */
public void execute() throws MojoExecutionException {

        // Generate compiler options
        CompilerOptions options = buildCompilerOptions(compilationLevel, true);

        // Compiler option, not yet used.
        List<JSSourceFile> externs = new ArrayList<JSSourceFile>();

        // Generate the list of source files to include.
        List<JSSourceFile> source = new ArrayList<JSSourceFile>();
        source.addAll(generateJavascriptSourceList());

        getLog().info("Compiling Javascript Library...");

        // Generate the compiler instance and compile.

        Compiler.setLoggingLevel(Level.OFF);
        Compiler compiler = new Compiler();
        compiler.compile(externs, source, options);

        // Output the compiled code.
        try {
                // Helper method to make sure the output file exists first.
                File newArtifactFile = getOutputFile();

                getLog().info(" -> " + newArtifactFile.getAbsolutePath());
                FileUtils.writeStringToFile(newArtifactFile, compiler.toSource());

        } catch (IOException e) {
                getLog().error("Unable to create artifact file: " + e.toString());
        }
}

Done! Now all we have to do is compile our mojo and use the pom reference from my post about designing a javascript maven plugin, and then invoke maven.

How will this impact Web Development?

This largely depends on what kind of web development you do. In most
cases you and your enterprise won’t be affected in the slightest.
Chrome has a very fast and robust rendering and JavaScript engine, and
much like any newly released browser (remember Firefox 1.0?) loads up
in no time flat. The rule of thumb is that if you’re already supporting
Safari, you can safely assume you’re supporting Chrome.

Why is this? What you may not know is that the underlying  engine that Safari runs on is a package called WebKit,
which is the same engine which powers Google Chrome. There are some
revision based incompatibilities (Since Safari’s already a few versions
ahead), but practically speaking they’re identical. The downside of
this is that if your agency is one of the rare islands left that only
support the “Two Major Browsers” (Firefox and Internet Explorer), you
no longer have an excuse to not support them.

If you’re doing Rich Internet Application development, you’ve just
been presented with a very interesting way of taking your application
to the desktop. You might not have heard of Google Gears
before now, or might not have considered it to be a viable option.
Gears is a browser extension framework that allows desktop-application
like interaction between your RIA and the client’s computer. Sounds
neat, right? It is, and it’s directly integrated into Chrome and is
available as a plugin for both IE and Firefox. Unfortunately, the major
restriction of Gears up to this point was that you were still
restricted to the browser’s UI, but as I point out later in this
article this is no longer entirely the case.

What about Mobile?

If you’re doing Mobile Web Development, you may be able to target
WebKit directly from this point forward. See, Safari is the exclusive
browser on the iPhone, and with the upcoming release of the T-Mobile HTC Dream,
you can bet that Chrome will be the default browser for Android. What
this means is that WebKit will become the de-facto web development
standard for mobile devices. While mobile UI patterns and application
frameworks will shake themselves out over the next few years, the
writing’s on the wall: If you want to take RIA’s to mobile devices
without bothering with a native application, WebKit is the platform to
build for.

So What’s The Big Deal™?

At this point you’re probably asking yourself: "What’s the big
deal"? If Chrome behaves much like the other major browsers out there,
why is there so much buzz about it? Is this just Google Hype?

Without going into a lot of gritty detail  about it (The Comic Book
published by Google does that really well), the major big deal is that
Chrome is not just a Browser: Chrome is an Application Platform.

Much like AIR, Chrome attempts to blur the lines between the desktop
and the web by creating a wrapper for previously developed content.
They even do it in very similar ways: AIR allows the execution of
JavaScript RIA’s in an integrated WebKit Browser running within the
ActionScript Virtual Machine, while Chrome allows the execution of
Flash RIA’s running in the Flash Player. The difference is simply the
technology stack used- Chrome is based around JavaScript and HTML,
while AIR is based on ActionScript and MXML.

The Google Engineers are quite explicit about this. The Comic Book
talks about it, and one of the primary features is "Create Application
Shortcut". While functionally this really just creates a direct link to
a specific website, the integration of Google Gears allows some
websites to move almost entirely to your desktop. It even goes so far
as to use the favicon for your application icon, giving you a Desktop
Application experience for any website you choose (try it with Google
Calendar or Gmail).

So what’s the Big Deal? It’s a concept change, a different way of
looking at the Web. It’s not particularly new- Microsoft tried to do
this with the close Windows/Internet Explorer integration in the late
90′s and .chm/.hta applications, but it is the first time that the
browser’s been turned into a (soon to be) platform agnostic application
wrapper.

A Future Vision

Not to be a crazy futurist or anything, but consider the following
possibility: Both Google and Adobe have now firmly cast their lot in
with an ECMAScript/DOM-like technology stack, and we already know that
there is a close relationship between the two companies both from
YouTube and from the indexable headless player. Personally, I think
it’d be pretty interesting if the future held a technological
convergence of all ECMAScript languages. Compiling HTML to a desktop
application? Converging JavaScript and ActionScript into a single
ECMAScript language? It’s all possible.

So lets talk about the consumer for a bit, in particular the consumer described in The Long Tail. Here is an individual who knows his/her own lifestyle and needs, who like particular applications over others, whose preferred experience with one service might be different from his/her neighbor’s experience for the same. This is an individual who, when presented with a Smart Phone, wants to have features and applications behaving a particular way, and will only put up with the provided options of the telco because there’s no other choice. In short, they have a craving for personalization, and will gladly move to any platform that has applications that meet their needs.

Deploying a mobile application to meet these needs is, in its simplest form, the following: First, you must have a developer able to access your SDK, platform and testbed with minimal fuss. Then, that developer must be able to deploy his application to a location where a potential user may download it onto their device. Lastly, the device must be able to run said application. These three tenets of Develop, Deploy, Consume can be easily seen on the regular web. Developers have their SDK’s, Deployment happens on a web server, and browsers allow consumption of the offered services.

In the mobile world this isn’t quite as simple. Yes, there are many websites now that have mobile components, yet lets face it: The form factor and memory constraints of mobile computing devices make it so that few applications can offer full functionality and experience while constrained by regular browser controls. Additionally, we are back to (for the time being) the computing world of the early 90′s, where applications are restricted by processing and memory constraints. Back then, however, we could load applications from Floppy disks, something that’s not so easy on a mobile device. The development platforms exist, but deployment (as noted above) is extremely difficult, and the broad diversity of devices makes your target installable base problematic at best.

So let us take a look at the various players in the mobile market right now. First of all, let us look at the telecommunication companies, those that have the aforementioned problem with getting something On Deck. From their perspective, they have control over their delivery platform, and getting something from their platform to a mobile device is simple enough, however their process is such that they have completely lost their developer base. There are too many devices, too many different platforms and form factors, and deploying an application is simply too difficult for any developer to bother building something. Imagine the computing world in the mid-80′s, and you’ll understand what I mean.

Next let’s take a look at Microsoft, which in our case will also double as any company that has a mobile application which they want to deploy. Microsoft has historically been brilliant in its developer support tools, so much so that it’s stupidly easy to develop an application for their operating system. Similarly, the fact that they are in the software and not the hardware business lets them sell a platform rather than a device, which allows mobile manufacturers to create devices that can run any application built for it. The place where they have failed, however, is in securing and simplifying the delivery platform, which remains under the control of the telco’s.

Third, let us look at Google and Android. Here, the development platform is easy to access and has some brilliant development tools. And… then what? Well, frankly, we don’t know a whole lot about how Android will be delivered, because while many manufacturers and providers have jumped on the bandwagon, we as of yet have no idea whether Android will use an open application deployment platform, or even whether phones built by their manufacturing partners will come with Android pre-installed. The suggested promise of Android is Build Once, Deploy To Any Phone, but if it means that a developer still has to get something On Deck with a Telco, you will once again be in the same boat as Microsoft- a common platform, but no control over application distribution.

I should note here that I doubt Google will not have some central application service, it simply hasn’t been announced yet.

Lastly, let us come to Apple. In this case I have to make a special note, in that while I recognize the SDK is publicly available, it is unfortunately not available for Windows (No, Aptana junkies, that’s just a web browser). Thus while they have alienated a good percentage of computer users overall, the additional cost of an OSX development platform is comparatively minor, and won’t detract a company interested in building an iPhone application. Taking that into consideration, they have everything buttoned up. Developers can access the SDK, they have an established and largely ubiquitous deployment platform not tied to the telco, and they have a device that will run any software built. This end-to-end solution is exactly the same thing that made the iPod so successful, because it got a user from Purchase to Play in one smooth experience.

And no, I don’t own an iPhone.

As a result I was practically giddy when I recognized a real world example of Judo Economics, a situation where a player uses the strength of their opponent against them. Consider a simple case: There exist two markets for a single product and two companies that can produce said product. The first market is large, however serving it comes with a high cost and therefore a low margin. The second market is small, but serving it is cheap and therefore comes with a high margin. The first of the companies must serve both markets, has deep pockets, while the second is considering market entry, but can only serve one of the two. Each company can only set one price against both markets. The question posed is: What will happen?

The answer is as follows: The entrant will enter the smaller, more profitable market and undercut the larger company to the point where the larger company would suffer a net loss by matching their price. Why? Well, without drawing the decision tree it turns out that net profit for the system will be larger for the big company because the entrant can continue to undercut it until the margin in the smaller market disappears entirely. Quite simply, they cannot match the entrant because they are forced to serve the both markets. Ergo: Their strength is used against them.

The Real World

Now lets take this example to the real world: Microsoft is currently serving the vast majority of the desktop market. In fact, it’s difficult to imagine a desktop computer that doesn’t run Windows, and even in the laptop market they command a substantial segment. Microsoft has gone to great lengths to solidify their position within that market, releasing their own languages, developer toolkits, IDE’s and whatnot. Buy Microsoft, by Microsoft, for Microsoft, anything you want to do on a computer you can do with Microsoft.

Unfortunately, by winning the war for the desktop market they’ve now locked themselves into serving it, and as any marketer knows you can’t really segment a market that size perfectly, at least not without severely diluting your brand. At most we’ve seen three versions of Windows, usually targeted at the Office User, the Power User, and the Home User. From both a product management perspective and from a branding perspective they’re pushing the limit of how many versions of windows they can support, especially if you take into account all the legacy versions still in circulation.

The number of versions actually doesn’t really matter- what matters is that Microsoft has to charge a price that the majority of said segment is willing to pay while at the same time accommodating for the development cost of meeting the needs of the entire market. Some consumers might be willing to pay more than they’re charging, others less, and the end result is that the margins average out to something marginally profitable.

The Competition

This is where the case for Judo Economics begins to take shape. Microsoft in the late 90′s and early 00′s could serve the entire market and did so, winning a dominant position. Since then the competition has identified users’ specialized needs and focused on them, effectively fragmenting the formerly homogenous user base of computer users into smaller segments: gamers, office workers, mobile computing, social networking, online productivity and so forth. They’ve each chosen their target market segment, have specialized their product offering for that particular segment, and are reaping the benefits of customers who are willing to pay more for a more specialized offering.

In the meantime, Mr. Gates still has to serve everyone else. The long and short is all the geeks who were angry about Microsoft’s antitrust practices vis-a-vis netscape and IE/browser integration/exclusion should have been cheering them on. A monolithic operating system and environment cannot be truly customized to the individual anymore, and they incur the massive cost of having to develop not only all the functionality they’ve supported up until now, but any new features that are identified in the market. In the world of the Long Tail their ability to meet all of a users general needs rather than all of a users’ specialized needs puts them at a significant disadvantage which their competition is busily exploiting.

So who are the competition?

Adobe: You might not think of Adobe as much of a competitor to Microsoft, however the recent introduction of Silverlight shows this is an area that Microsoft has identified as profitable (they’re good at identifying opportunity, I’ll give them that). Unfortunately Adobe has a lock on the real driving force behind the online experience: Designers. As long as they continue to focus on supporting them, the decisions of what technology to use will remain firmly in their court. Yes, purely technically designed websites and RIA’s will likely fall to Microsoft, but quite frankly I don’t think Adobe cares about serving the online accountant.

Apple: In 1999, the book on this fight seemed to have been written. Apple soundly lost the battle for the desktop, and was classified even lower than Linux, a computing platform for rabid advocates that would eventually follow the Amiga into obscurity. And now where are they? Apple is sexy. Apple is hip. Apple is targeted at a consumer segment that is affluent and to some extent image conscious, taking those users away from Microsoft. All those fat margins that could have been earned by Bill are now wandering off to the upmarket offerings provided by Apple, who doesn’t ever want to be in the position of having to serve everyone… or do they? (more on this in another post).

Nintendo: The XBox 360 is everywhere, Halo was a huge deal. We know that Microsoft was succesful here, and will continue to be so for the forseeable future. Yet this market shows some of the same signs as Apple’s position in the Desktop and Laptop market, because the XBox has become a commodity- everybody has one. Everyone does not have a Wii, and the speed at which they fly off the shelves shows a level of demand much like that of the iPod. The Wii is cool, while the XBox has become the console equivalent of a beige box. How this will play out in the long run is hard to say at this time, however given Redmond’s recent strategic moves into community gaming I feel they won’t give up their share so readily.

Google: It’s hard not to mention Google here, and even though I think they’ve jumped the shark and will lose their innovative edge in the very near future, their dominance in the search market is too big to ignore (just look at the Yahoo hostile takeover- Microsoft’s not ignoring them). Here we once again have a company that has to do everything: browser, search, ads, etc, competing with a company that’s just really, really good at one or two of those. Guess which one is dominant? I thought so.

So that’s the argument in a nutshell: Microsoft has to serve the entire market, and as a result is too crippled to actually compete for the plumb market shares. Looking forward we can already see them becoming more aware of their situation though: The next version of windows is supposedly modular and thus more customizable to individuals’ needs. They are attempting to acquire Yahoo, and opening up the XBox for community developers is a dangerously O.P.E.N. tactic. I’m calling the score tied, lets see what happens in round three.

Usually this isn’t really a problem- flash applications have not been too complex and not many people care where on the banner you clicked, just that you left the site as a result. Yet now with the strong growth of Flex and Ajax our web applications are becoming more and more complex, and marketers and usability experts are now demanding this tracking data in spite of the paradigm limitations.

If you really think about it, what we really are interested in tracking is a user action, rather than the page loads we are collecting right now, which means that the largest part of an analysts job is turning these page events into meaningful user actions, rather than interpreting those users. There’s been some attempt to set metrics to individual places within a page flow, yet nobody has yet thought to rethink the paradigm. But I digress…

Most Metrics providers have since opened their API’s enough to allow a developer to pretend like a new page refresh has occurred. While this is hardly optimal, it does allow us to track user events from inside of flash, but the fact that each provider has implemented their API a different way means that implementing metrics for each is still a string of unique problems to solve.

To that end I’ve written the Metrics package, which is intended to provide a common metrics proxy that any developer may use, which relies on a common library of connectors that can be swapped out as needed. In this article I go over the details of how to use it, and touch on how individual connectors might be written. You can access and download the actual library here.

Step 1: Initialize the Metrics Engine

The first step is to initialize the metrics controller with the connectors you’d like it to use.

import com.practicalflash.metrics.connectors.GoogleConnector;
import com.practicalflash.metrics.connectors.UrchinConnector;
import com.practicalflash.metrics.Metrics;

// Add the connectors
Metrics.addConnector( com.practicalflash.metrics.connectors.GoogleConnector);
Metrics.addConnector( com.practicalflash.metrics.connectors.UrchinConnector);

Note that the package can take an arbitrary number of connectors, yet will only instantiate those which successfully autodetect their environment. Should you want to deploy an application and don’t know in advance what environment you’ll be reporting to, you can hedge your bets and add several, as I have above. The only caveat is that any provided class must implement the IMetrics interface.

Step 2: Dispatch a Metrics Event

Once you’ve initialized your connector, you may dispatch metric events from any point in the application by creating a MetricsEvent object. The following example demonstrates a case where you might have an event listener waiting for a particular user action.

import com.practicalflash.metrics.MetricsEvent;

function someEventHandler( event : Event ) : void
{
        var metricEvent : MetricsEvent = new MetricsEvent("/this/is/a/test.html?foo=bar");
        metricEvent.dispatch();
}

Additional options include setting the event type, the dispatching target, or adding an arbitrary number of parameters to be parsed and passed along. Exact behavior for these parameters is described in the documentation, and may be dependent on their level of support within the various connectors.

metricEvent.target = event.target;
metricEvent.type = MetricsEvent.RECORD;
metricEvent.addParameter( "bar" , "foo" );

Further Customization

If any of the provided connectors don’t suit your purpose, the task of writing your own is actually fairly simple. The only requirement for the connector is that it implements the IMetrics interface, which prescribes two functions: detect(), which runs any autodetection script you’d care for, and send(), which should process and deliver any received information appropriately.

Additionally, the AbstractConnector class is provided with a few useful utility features, including a singleton instance of the JavascriptConnector (accessible via the jsBridge parameter) as well as getDOMBounds(), which returns an objects’ rectangular bounds with respect to the HTML DOM (just in case your metrics package supports the reporting of a page’s click area).

A sample connector might be as simple as the following:

package com.practicalflash.metrics.connectors
{
        import com.practicalflash.metrics.MetricsEvent;
        import com.practicalflash.metrics.interfaces.IMetrics;

        public class UrchinConnector extends AbstractConnector implements IMetrics
        {
                public function detect() : Boolean
                {
                        if( available )
                                return jsBridge.hasParameter( "urchinTracker" );
                        else
                                return false;
                }

                public function send( event : MetricsEvent ) : void
                {
                        jsBridge.urchinTracker( event.page + "?" + event.getURLString() );
                }
        }
}

The Metrics package was designed to provide a simple, reusable, extensible and flexible interface to any web metrics package, including Google Analytics, Urchin, Omniture, WebTrends and more. It supports multiple connector management, instantiating only those registered connecters that are successful at autodetecting their environment. If a developer does not know in advance which metrics package his client is using (or whether his client might change his mind at a later time), he may safely add several connectors secure in the knowledge that the package will do the heavy lifting. Additional functionality allows you to “hot-swap” connectors should that become necessary, though I myself don’t exactly know why you’d want to do that.

At this point in time, the only connector published with this library is the UrchinConnector. Google Analytics and WebTrends will follow shortly, however due to my employers’ client agreements I find it prudent to not release connectors for API’s that are considered proprietary (such as Omniture) unless I have a formal letter granting me legal permission to do so.