Modern Javascript in GNOME – GUADEC 2017 talk

I gave a presentation at GUADEC 2017 this morning on modern Javascript in GNOME, the topic of the last few posts on this blog. As I promised during the talk, here are the slides. There is a beefy appendix after the questions slide, with details about all the new language features, that you are welcome to use as a reference.

Thanks to the GNOME Foundation for the travel sponsorship, to my employer Endless for paying for some of this work, and especially to Rob McQueen for the last-minute loan of a USB-C video adapter!

Official badge from the GUADEC website: "I'm going to The GNOME Conference GUADEC Manchester, United Kingdom"

Inventing GObject ES6 classes

Hello again! If you’re a GJS user, I’d like your opinion and ideas. After my last post where I talked about new features coming in GNOME 3.26 to GJS, GNOME’s Javascript engine, I’m happy to say that the patches are nearly ready to be landed. We just need to figure out how to build SpiderMonkey 52 consistently even though Mozilla hasn’t made an official standalone release of it yet.

A top view of a latte next to a notebook with a pen, with coffee beans strewed artfully around.

A better literal depiction of JAVA SCRIPT I could not ask for… (Public domain image courtesy of Engin_Akyurt)

As I reported last time:

After that is done, I will refactor GJS’s class system (Lang.Class and GObject.Class). I believe this needs to be done before GNOME 3.26. That’s because [we will] gain ES6 classes, and I want to avoid the situation where we have two competing, and possibly incompatible, ways to write classes.

That’s what I’m busy doing now, in the run-up to GUADEC later this month, and I wanted to think out loud in this blog post, and give GJS users a chance to comment.

First of all, the legacy Lang.Class classes will continue to work. You will be able to write ES6 classes that inherit from legacy classes, so you can start using ES6 classes without refactoring all of your code at once.

That was the good news, now the bad

However, there is not an obvious way to carry over the ability to create GObject classes and interfaces from legacy classes to ES6 classes. The main problem is that Lang.Class and its subclasses formed a metaclass framework. This was used to carry out certain activities at the time the class object itself was constructed, such as registering with the GType system.

ES6 classes don’t have a syntax for that, so we’ll have to get a bit creative. My goals are to invent something (1) that’s concise and pleasant to use, and (2) that doesn’t get in the way when classes gain more features in future ES releases; that is, not too magical. (Lang.Class is pretty magical, but then again, there wasn’t really an alternative at the time.)

Here is how the legacy classes worked, with illustrations of all the possible bells and whistles:

The metaclass magic in Lang.Class notices that the class extends GObject.Object, and redirects the construction of the class object to GObject.Class. There, the other magic properties such as Properties and Signals are processed and removed from the prototype, and it calls a C function to register the type with the GObject type system.

Without metaclasses, it’s not possible to automatically carry out magic like that at the time a class object is constructed. However, that is exactly the time when we need to register the type with GObject. So, you pretty much need to remember to call a function after the class declaration to do the registering.

The most straightforwardly translated (fictional) implementation might look something like this:

The fictional GObject.registerClass() function would take the role of the metaclass’s constructor.

This is a step backwards in a few ways compared to the legacy classes, and very unsatisfying. ES6 classes don’t yet have syntax for fields, only properties with getters, and the resulting static get syntax is quite unwieldy. Having to call the fictional registerClass() function separately from the class is unpleasant, because you can easily forget it.

On the other hand, if we had decorators in the language we’d be able to make something much more satisfying. If you’re familiar with Python’s decorators, these are much the same thing: the decorator is a function which takes the object that it decorates as input, performs some action on the object, and returns it. There is a proposed decorator syntax for Javascript that allows you to decorate classes and class properties. This would be an example, with some more fictional API:

This is a lot more concise and natural, and the property decorators are similar to the equivalent in PyGObject, but unfortunately it doesn’t exist. Decorators are still only a proposal, and none of the major browser engines implement them yet. Nonetheless, we can take the above syntax as an inspiration, use a class expression, and move the registerClass() function around it and the GObject stuff outside of it:

Here, the body of the class is almost identical to what it would be with the decorator syntax. All the extra stuff for GObject is contained at the top of the class like it would be with the decorators. We don’t have the elegance of the property decorator, but this is quite an improvement on the first iteration. It’s not overly magical, it even acts like a decorator: it takes a class expression, and gives back a GObject-ized class. And when decorators eventually make it into standard Javascript, the basic idea is the same, so converting your code will be easy enough. (Or those who use transpiling tools can already go ahead and implement the decorator-based API.)

This is the best API I’ve been able to come up with so far. What do you think? Would you want to use it? Reply to this post or come talk to me in #javascript on GNOME IRC.

Next steps

Note first of all that none of this code exists yet. Depending on what feedback I get here, I hope to have a draft version working before GUADEC, and around the same time I’ll post a more detailed proposal to the javascript-list mailing list.

In addition, I will be speaking about this and more at GUADEC in my talk, “Modern Javascript in GNOME“. If you are attending, come talk to me there! Thanks to the GNOME Foundation for sponsoring my travel and accommodations.

Official badge from the GUADEC website: "I'm going to The GNOME Conference GUADEC Manchester, United Kingdom"

Javascript news from GNOME 3.24

Welcome back to the latest news on GJS, the Javascript engine that powers GNOME Shell, Endless OS, Polari, GNOME Documents, and many other apps.

GNOME 3.24 has been released for about three weeks now, and with it went GJS 1.48.0. Here’s what’s new!

Javascript upgrade!

First of all, we have a more modern Javascript engine. GJS is based on Mozilla’s SpiderMonkey, the same Javascript engine that runs in the Firefox browser. Back in GNOME 3.22, GJS was based on version 24, which was released in September 2013. Now we’ve moved to version 38, which although still old, was released almost two years later in May 2015.

(The number of each SpiderMonkey release increases by 7 each time, because they make a standalone SpiderMonkey release for each Extended Support Release of Firefox, which is one out of every 7. That’s why you might also hear them referred to as “ESR 38”, etc.)

This brings a lot of new Javascript language features with it. Here are some of the ones I’m most excited about.

Promises

Promises allow you to do asynchronous operations (like reading files, or waiting, or fetching things from the network) in a much more intuitive way. With Promises, the code reads from top to bottom as if it were synchronous, instead of from nested level to nested level (often called “callback hell“.)

Here’s an example, a Promises version of examples/gio-cat.js that’s included in GJS’s source distribution:

This is much longer than the original program, but only the lower part of the program is actually the equivalent of the old callback-based code. The top part would ideally be provided by GJS itself. I’m still figuring out what is the best API for wrapPromise but it’s definitely a candidate for including in a future version of GJS.

This code calls loadContents, prints the contents, and exits the main loop. If an exception is thrown anywhere in the chain before .catch, then the function provided to the catch call will log the error message. In any case, no matter whether the operation succeeded or not, the last then call will make sure the main loop exits.

Template literals

Template literals will change your life if you work with text in your GJS program. They are regular strings in backticks, with interpolation. Say goodbye to this:

const Format = imports.format;
String.prototype.format = Format.format;
log("%s, %s!".format(greeting, name));

Also say goodbye to this:

log(greeting + ", " + name + "!");

Instead, from now on you’ll do this:

log(`${greeting}, ${name}!`);

It’s a lot more readable and intuitive.

Template literals can also cover more than one line, and they do real interpolation of expressions too, not just variable names:

const CSS = `
label {
    font-size: ${fontdesc.get_size()};
}`;

You can also “tag” templates which is out of scope of this blog post, but there is one built-in tag which serves the same purpose as r'' string literals do in Python:

String.raw`I'm writing some \LaTeX\ code here
and I \textbf{don't} want to deal with escaping it:
\[ E = mc^2 \]`

Generators

Generators are a great addition to the Javascript toolbox. They were actually already available in GJS, but only in Mozilla’s nonstandard extension form. They are introduced with the function* keyword instead of function, and they work a lot like Python’s generators. Here’s an example, implementing the xrange() function similar to the one in Python using a generator:

function* xrange(limit) {
    for(let count = 0; count < limit; count++)
        yield count;
}

The yield statement returns control back to the caller, while preserving the state of the generator until the next call. You can get all the values one by one, calling a generator’s next() method, but for...of loops will also deal with generators:

for (let ix of xrange(5))
    print(`Counting from 0 to 4: ${ix}`);

If you want to empty a generator into an array, you can also use the spread operator: [...xrange(5)] will give you an array of numbers from 0 to 4.

Here’s a more complicated example showing the yield* statement which allows you to compose more than one generator:

This code prints looks at the directory that it’s given, and prints all the files in it that are not themselves directories (the “leaf nodes”.) If one of the files is a directory, it will descend into that directory and repeat the process, thanks to yield*.

Want to know more?

Since there’s a lot more than I can cover in a comfortably readable blog post, I made a slide deck. I tried to put it together in such a way that you can use it as reference material.

For more information on all of these cool things, I highly recommend this “ES6 Explained” series of posts from the Mozilla Hacks blog. Some of these features, such as classes and modules, are still to come in GJS.

Maintainer life

The Javascript engine upgrade was the major feature, but I also spent some time on making things easier for myself as the maintainer. A well-tended garden will hopefully attract more gardeners. Happily, some other people joined in for this part.

I cleaned up the build system, using more modern and concise Autotools code. I also spent some time cleaning up compiler warnings, both on GCC and Clang. Now the build and test runs are faster, and the cleaner output makes it much easier to see when something goes wrong. I also made sure that GJS builds on macOS, or at least it did until my Apple hardware broke down. Chun-wei Fan made some improvements that ensure GJS builds on Windows with MSVC. Claudio André implemented continuous integration in a Docker container, with the intention to run it on Travis CI, but sadly we do not have permission to flip the bit to get Travis to build it.

Having written Jasmine GJS in order to bring some of that convenient unit testing technology from the Node world into GJS applications, I also wanted to use it for writing GJS’s own unit tests. I couldn’t use it directly because that would have been a circular dependency, of course, but I embedded a copy of upstream Jasmine plus a very stripped-down version of Jasmine GJS, and called it “Minijasmine”. It’s now a lot easier, and dare I say less of a drag, to write unit tests for GJS. Accordingly, we’ll now try to cover every bug fix with a regression test.

And I worked on getting the bug tracker down to a less daunting number of bugs. It was fun to make the bug chart in my last post, so here’s another one: this is the number of open bugs during the release cycle from 1.46.0 to 1.48.0.

Graphical report results

You can definitely see that November Bug Squash Month had an effect

Unfortunately the chart will not look like this again next time around. The big drop was me closing all the obsolete or already-fixed bugs during November Bug Squash Month. We are down from about 160 to about 100 bugs, but those were all the easy ones; there are only hard ones left now.

Thanks

Thanks to everyone who participated to bring GJS to GNOME 3.24: Chun-wei Fan, Claudio André, Florian Müllner, Alexander Larsson, Iain Lane, Jonh Wendell, and Lionel Landwerlin.

As well, this release incorporated a lot of patches that people contributed a long time ago, even up to 8 years, that for various reasons had not been reviewed yet. (Many from emeritus GJS maintainers!) Thanks to those people for participating in the past, and I’m glad we were able to finally bring your contributions into the project: Giovanni Campagna, Jasper St. Pierre, Sam Spilsbury, Havoc Pennington, Joe Shaw, Paolo Borelli, Shawn Walker, and Tim Lunn.

Luke Jones and Hussam Al-Tayeb identified a serious memory leak right before the final 1.48.0 release and without their contribution, it would have been a different and much sadder story. As it was, 1.48.0 still contained another serious bug that made GNOME Shell quite unusable for an unlucky few people. Thanks to Georges Stavracas for rewriting a happy ending for 1.48.2.

Special thanks to Cosimo Cecchi, for reviewing almost every single line of the code I wrote for this release: about 20000 lines, many of them boring and repetitive.

Thanks also to my employer Endless which sponsored most of the Javascript engine upgrade, and a good chunk of miscellaneous bug fixing time.

Looking forward

My next post will be about what’s to come in GJS for GNOME 3.26.

November Bug Squash Month: GJS

Here’s what happened during November Bug Squash Month in GJS.

First off, I didn’t really get on the ball to promote Bug Squash Month and I didn’t take pictures of any bug squashing activity… which I regret. I hope this post can make up for some of that.

During November I finally took the leap and offered to become a maintainer of GJS. My employer Endless has been sponsoring work on bugs 742249 and 751252, porting GJS’s Javascript engine from SpiderMonkey 24 to SpiderMonkey 31. But aside from that I had been getting interested in contributing more to it, and outside of work I did a bunch of maintenance work modernizing the Autotools scripts and getting it to compile without warnings. From there it was a small step to officially volunteering.

With not much of November remaining and a holiday and family visit coming up (life always is more important than bug squashing!) I decided to start out my bug-squashing campaign with what would get me the most results for the time spent: going through GJS’s bug tracker and closing obsolete or invalid bugs. This I managed to do, closing about 1/4 of all open bugs!

Then I made a list of all open bugs with attached patches and intended to review them to see if they still applied and why they hadn’t been committed yet. I got through a few, and had the dubious distinction of fixing up and committing patches from a 7 year old bug yesterday. But as you can see in the list, there are still 54 remaining. A good to-do list for the next Bug Squash Month, or whenever I feel like working on GJS but don’t know what to work on!

Did you know Bugzilla could generate graphs? I didn’t! Here’s a graph of the total bug count in GJS during November Bug Squash Month:

chart

The clunkiness of this chart kills me though…

My plans now that Bug Squash Month is over are to concentrate on fixing things that make it more pleasant to use and contribute to GJS:

  • Find an active co-maintainer so that we can review each other’s patches (could this be you?)
  • Make ES6 Promises available (this work is also being sponsored by Endless)
  • Rework the test suite to use an embedded copy of Jasmine so that writing automated tests becomes less of a pain
  • Find ways to bring in some of the conveniences that Node developers are used to

I’ve also decided to try an experiment: I’ve just made the Trello board public on which I keep track of what I’m working on and what I’d like to work on. Let me know if this is interesting to you and what features you might like to see on there! (It’s made possible by a Chrome extension, Bug 2 Trello.)

All in all November Bug Squash Month was a success, though next time I will get started earlier in the month. Come join me next time!

gnome_bugsquash_nov_2016

 

Wave at the camera

You have probably seen the fake advertisement for Wave, the new way of charging your iOS 8 phone in any standard household microwave. (Although I would venture that some of the responses with fried microwaves and phones are hoaxes as well.)

I admit I did giggle when I first read it — some chump microwaved their expensive phone and blew it up, funny, right? Only I realized that it’s not funny at all.

Why shouldn’t people believe that a new technology would allow them to charge their phone by microwaving it? It’s no more or less magical than any other new technology being invented every day. It just happens not to have been invented yet.

Yes, people need to think critically, check sources, use common sense, and become less science-illiterate. Is microwaving your phone a smart thing to do? No. Could the average person probably have known better? Yes. But if you are lucky enough to be in the minority for whom this is obvious, you don’t have any right to laugh at those for whom it is not.