Dojo from the Air

While I'm sitting here 35,000 feet above the Tasman I thought it'd be a good time to update you all on the goings on in the world of Dojo from my point of view.

1.1!

Dojo released 1.1 after a lot of hard work. The team worked really hard to get this release out, and it adds a lot of polish to 1.0 and a number of cool new features. Some quick snippets from me:

dojo.attr()

We now have a smooth & nice way to access attributes on DOM elements, normalising access much in the way dojo.style() does. And, just like dojo.style() can now, if you pass in an object as the name, it'll set a bunch of attributes for you in one go:

// get the current value of the "foo" attribute on a node
dojo.attr(dojo.byId("nodeId"), "foo");

// we can just pass the id:
dojo.attr("nodeId", "foo");

// use attr() to set the tab index
dojo.attr("nodeId", "tabindex", 3);

// set multiple values at once, including event handlers:
dojo.attr("buttonId", {
 "name": "myButton",
 "value": "123",
 "onclick": function(e){
     // an onclick handler connected via dojo.connect()
 }
});

api.dojotoolkit.org

Our API browser is back, newer and better. You can search and browse all the code in Dojo, see method arguments and examples, and dig into the code if necessary. And when you figure out a tip you can comment in the appropriate place to help others out. We think this is a fantastic addition, and I've been using it on a daily basis for a while now.

dojox.help

Not really part of 1.1, but happened about the same time. The talented Neil Roberts whipped up an awesome demo based on some suggestions from a dojo cookie. Basically it accesses the API site live and allows you to get method arguments and descriptions directly in Firebug. I often get some code semi-working and test and experiment using the Firebug console while I finalise it, so this is a real boon.

Let's have a go:

1. load this page in Firefox with Firebug enabled
2. in the Firebug console, type the following:
   • dojo.require("dojox.help.console") // loads the help infrastructure
   • dojo.attr // prints the method arguments for dojo.attr()
     
   • dojo.attr.help() // prints the method description, examples, etc
3. revel in its coolness, dig in if you're using trunk, or wait patiently for it to appear in 1.2

Summer of Code

On Monday the Summer of Code students for 2008 will be announced. Dojo received a huge pile of great applications, and we had a really hard time evaluating and ranking them. Unfortunately the funding isn't infinite and neither is our mentoring capacity, so we've chosen what we think is a pretty exciting mix of projects. More to follow next week after the announcements!

The coding exercise we set went down pretty well. I suspect many of the students spent a lot more time on it than we asked, but there are some damn cool demos in there, which I'll try to show off over the next few weeks. Next year we'll try something a little more focussed, but its actually fairly tricky to think up the right balance.

Anyway, time to land now. Better be off. :)

Prepping for the Summer of Code

So, it was announced today that Dojo will be part of the Google Summer of Code in 2008. w00t! For students, you get a modest amount of fortune (US$4500) and some degree of fame (TBA), and Dojo gets your nice shiny code. This is our third year and the second time I've been involved. Looking at the quality of some of the applications that didn't make it this year, I'm a little scared that we only just scraped in.

Since the programme was announced on 25 February, we've achieved quite a bit, so I thought I'd jot it down:

• Figured out who's organising it: me, since I volunteered again ;)
  
  
• Thought about what was good and bad from 2007 and 2006 and what we'd change
  
  
• Write up a pile of documentation:
  
• An overview of SoC and Dojo for mentors and students
  
• A FAQ for students about what we and they should expect
  
  
• Created our project ideas list - basically we need to list some projects we think are both interesting and achievable by a new student in a couple of months. This involved a huge brain dump from me, followed by trying to herd our contributors into reviewing it and adding their own ideas, notes, and corrections. We've got some pretty good stuff down, but I know there's more great ideas out there still.
  
  
• Posted to the blog and our mailing lists looking for mentors and more ideas. We're trying to get 2 mentors per student this year, with one from the "core" Dojo team and someone else from the Dojo community who wants to get more involved - we have a lot of smart & talented people and we want to pull them closer.
  
  
• Answered a pile of emails, forum posts, IRC, and IM conversations with students and potential mentors.
  
  
• Wrote our application to Google. Only 175 out of 505 organisations have been accepted, so it needed to be good.
  
  
• Wrote a coding test - shamelessly ripping off the idea from my good friends at SilverStripe we decided that our students should do something practical as part of their application too. Still some more tweaking to do there too.
  

That took until this morning. Phew. What's next?

• Start matching enquiries to mentors
• Hassle some more of the core committers to get involved
  
• Figure out how we're going to deal with version control & ticketing for the SoC projects
  
• Sketch out our mentor 'teams' and fill in any gaps
  
• Decide how many projects we can handle
• Keep on top of the emails and other enquiries

Student applications start flowing in on the 24th, then they need to be reviewed and voted on by the 11th. Then I can relax for a bit (unless I end up mentoring a project as well) and just make sure everyone is communicating and making progress. Until evaluation-time.

Automated Dojo testing - DOH & Selenium-RC

Before the 0.9 release, Dojo replaced its old unit-testing framework with a new one named D.O.H. that is miles nicer and cleaner. It'll run in both the browser & under Rhino, supports defferrables and is relatively easy to customise.

At One Track Mind we've been using Selenium-RC to drive unit-testing of JS code since Dojo 0.4 (of our stuff mostly, not Dojo itself) and it wasn't ever particularly clean. The basic idea is to run a suite of unit tests across a series of browser/version/OS combinations and collate the results. So you can find that some irrelevant little change b0rked Opera but only on Windows.

Selenium's API for writing tests (for JS stuff anyway) is pretty awful compared with DOH and we'd done ours in Python, which was then launched from Ant after the Dojo build was done. But what Selenium-RC does do well is fire up browsers on multiple operating systems, do magic proxying, and allow you to interrogate the pages.

A thread on dojo-interest this week sparked me back into looking at it again. I noticed that Selenium-RC now supports running from Javascript. (to be honest, the java bindings would have always worked but someone's actually spelt it out in small words now). Since the Dojo build system is now Javascript-based, why not make the Selenium test runner as well?

Plan:

• Have a config file defining which browsers to launch, which machines they're on, and what tests to run.
• Launch each browser via Selenium-RC
  
• Run the tests via the normal DOH browser runner.
• Use Selenium to extract the results from DOH.
  
• Collate the results from the various browsers and produce something useful.

So, after a bit of toying I came up with this:

• Drop seleniumRunner.js, seleniumRunner.config.js, seleniumRunner.sh (or the .bat if you're on Windows), and selenium-java-client-driver.jar into util/doh/ in your Dojo install.
  
• Put selenium-server.jar on each test machine, then run java -jar selenium-server.jar -multiWindow so it listens for the browser-control messages.
• Edit seleniumRunner.config.js and change browsers and rootUrl to match your setup. The rootUrl needs to be reachable from each test machine.
• run ./seleniumRunner.sh seleniumRunner.config.js from util/doh/ on your workstation
• It'll load the config, fire up the browsers on each machine, run the unit tests from Dojo core, and print the pass/fail/error stats for each.
  
• Each browser is kicked off and monitored in a separate thread (not strictly necessary but too cool to resist doing).

Some issues I ran into:

• unless I ran the selenium server in multiWindow mode Safari and Firefox would pop up Print dialogs (!?!) whenever the test page was loaded. But Safari never initialised the test page if it was in multiWindow mode. On OSX and Windows. gah.
  
• Opera on OSX didn't set up the Selenium proxy properly (localhost:4444 for reference).
• IE didn't like doing a dojo.connect() via the selenium javascript commands for some reason.
  

The next step is to improve the reporting. This will probably necessitate writing another browser runner that produces a JSON report (or changing the existing one). Modifying DOH to have an isComplete flag would make life simpler as well. I'd want to know which tests were run on each browser (since not all tests run everywhere), which ones passed and failed and had errors. A hunk of JSON which could be loaded into a Dojo Grid for exploration might make an interesting way of viewing the results.

Cool data in the browser with Dojo

This is basically what I talked about on Saturday at Barcamp (which was heaps of fun and big props to Ludwig for organising it all!).

Cool data in the browser with Dojo

We're now at a place with Javascript development where we can handle structured data really well in the browser. And then we can start doing some cool stuff - grids, charts, maps, and so on. The Dojo toolkit calls it dojo.data.

Basically dojo.data is an abstraction - sitting between where the data comes from (tables, lists, xml, csv, json, Flickr...) and what you want to do with it (querying, widgets, graphs, grids, maps, ...).

Stores

A data store describes where the data comes from. This could be from in the HTML page (from inline lists or tables), from static sources (XML, CSV, JSON), dynamic sources (database-backed querying) or external APIs (like Flickr). Different stores can implement different sections of the dojo.data API depending on what the source supports - reading, writing, identities, and so on. Its pretty easy to create your own store implementation if none of the existing ones suit.

Data Widgets

Data widgets present data from a store. The idea is that we can create kick-ass widgets once that work with data from any source. The widgets interface to the data via the dojo.data APIs and then do cool things to present it. Data-driven widgets include form components like ComboBox, the tree, the grid, charting, image galleries, and so on. We'll just talk about a couple of them.

The Dojo Grid was released in September by SitePen, Mozilla, and others. It is the Javascript table to end all tables, with a truly impressive feature list - paging, sorting, filtering, subgrids, events, formatting, internationalisation, and the list goes on. Of course, the grid can consume data from a store, so we can have all these impressive features for CSV data, XML data, or anything else. SitePen has a two part tutorial on a few of the grid features, or you can check out the documentation (still in-progress) and the tests.

Dojo Charting provides vector-based charting in 2D, and now 3D thanks to the work of Kun Xi and Neil Joshi during this years Google Summer of Code. Underneath it uses dojox.gfx which leverages SVG, VML, Silverlight, or Canvas depending on the browser to draw the vector graphics. Charting is pretty new but already has an impressive set of features: lines, columns, areas, bars, pies, grids, axes, themes, ... see the tests for some examples. The dojo.data implementation in charting isn't fully complete yet but its still easy enough to get it working.

Demo Time

Enough talk - lets see all this in action. I've grabbed some 10-minute weather readings in CSV format from 18 Auckland stations for a 24 hour period in November from NIWA's climate database. Then built a simple data explorer where you can select a station in the table and see a line chart of the temperature and wind speed over the 24 hour period for that station. And there's a map, because that's what I do.

Play with the demo

You'll notice that there is no page-loading involved (AJAX goodness), you can sort the grid by clicking on the column headings, and that it's all very smooth and responsive. View the source and read the comments to see what is happening underneath.

Enjoy!