Taking Out the Garbage

From the title, you might think this post is about household chores. Instead, I’m happy to announce that we may have a path to solving GJS’s “Tardy Sweep Problem”.

For more information about the problem, read The Infamous GNOME Shell Memory Leak by Georges Stavracas. This is going to be a more technical post than my previous post on the topic, which was more about the social effects of writing blog posts about memory leaks. So first I’ll recap what the problem is.

Garbage, garbage, everywhere

At the root of the GNOME desktop is an object-oriented technology called GObject. GObjects are reference counted, but not garbage collected. As long as their reference count is nonzero, they are “alive”, and when their reference count drops to zero, they are deleted from memory.

GObject reference counting

Graphical user interfaces (such as a large part of GNOME Shell) typically involve lots of GObjects which all increase each other’s reference count. A diagram for a simple GUI window made with GTK might look like this:

A typical GUI would involve many more objects than this, but this is just for illustrating the problem.

Here, each box is an object in the C program.

Note that these references are all non-directional, meaning that they aren’t really implemented as arrows. In reality it looks more like a list of numbers; Window (1); Box (1); etc. Each object “knows” that it has one reference, but it knows nothing about which other objects own those references. This will become important later.

When the app closes, it drops its reference to the window. The window’s reference count becomes zero, so it is erased. As part of that, it drops all the references it owns, so the reference count of the upper box becomes zero as well, and so on down the tree. Everything is erased and all the memory is reclaimed. This all happens immediately. So far, so good.

Javascript objects

To write the same GUI in a Javascript program, we want each GObject in the underlying C code to have a corresponding Javascript object so that we can interact with the GUI from our Javascript code.

Javascript objects are garbage collected, and the garbage collector in the SpiderMonkey JS engine is a “tracing” garbage collector, meaning that on every garbage collection pass it starts out with objects in a “root set” that it knows are not garbage. It “traces” each of those objects, asking it which other objects it refers to, and keeps tracing each new object until it hits a dead end. Any objects that weren’t traced are considered garbage, and are deleted. (For more information, the Wikipedia article is informative: https://en.wikipedia.org/wiki/Tracing_garbage_collection)

We need to integrate the JS objects and their garbage collection scheme with the GObjects and their reference counting scheme. That looks like this:

The associations between the Javascript objects (“JS”) and the GObjects are bidirectional. That means, the JS object owns a reference to the GObject, meaning the reference count of every GObject in this diagram is 2. The GObject also “roots” the JS object (marks it as unable to be garbage collected) because the JS object may have some state set on it (for example, by writing button._alreadyClicked = false; in JS) that should not be lost while the object is still alive.

The JS objects can also refer to each other. For example, see the rightmost arrow from the window’s JS object to the button’s JS object. The JS code that created this GUI probably contained something like win._button = button;. These references are directional, because the JS engine needs to know which objects refer to which other objects, in order to implement the garbage collector.

Speaking of the garbage collector! The JS objects, unlike the GObjects, are cleaned up by garbage collection. So as long as a JS object is not “rooted” and no other JS object refers to it, the garbage collector will clean it up. None of the JS objects in the above graph can be garbage collected, because they are all rooted by the GObjects.

Toggle references and tardy sweeps

Two objects (G and JS) keeping each other alive equals a reference cycle, you might think. That’s right; as I described it above, neither object could ever get deleted, so that’s a memory leak right there. We prevent this with a feature called toggle references: when a GObject’s reference count drops to 1 we assume that the owner of the one remaining reference is the JS object, and so the GObject drops its reference to the JS object (“toggles down“). The JS object is then eligible for garbage collection if no other JS object refers to it.

(If this doesn’t make much sense, don’t worry. Toggle references are among the most difficult to comprehend code in the GJS codebase. It took me about two years after I became the maintainer of GJS to fully understand them. I hope that writing about them will demystify them for others a bit.)

When we close the window of this GUI, here is approximately what happens. The app drops its references to the GObjects and JS objects that comprise the window. The window’s reference count drops to 1, so it toggles down, dropping one direction of the association between GObject and JS object.

Unlike the GObject-only case where everything was destroyed immediately, that’s all that can happen for now! Everything remains in place until the next garbage collection, because at the top of the object tree is the window’s JS object. It is eligible to be collected because it’s not rooted and no other JS object refers to it.

Normally the JS garbage collector can collect a whole tree of objects at once. That’s why the JS engine needs to have all the information about the directionality of the references.

However, it won’t do that for this tree. The JS garbage collector doesn’t know about the GObjects. So unfortunately, it takes several passes of the garbage collector to get everything. After one garbage collection only the window is gone, and the situation looks like this:

Now, the outermost box’s JS object has nothing referring to it, so it will be collected on the next pass of the garbage collector:

And then it takes one more pass for the last objects to be collected:

The objects were not leaked, as such, but it took four garbage collection passes to get all of them. The problem we previously had, that Georges blogged about, was that the garbage collector didn’t realize that this was happening. In normal use of a Javascript engine, there are no GObjects that behave differently, so trees of objects don’t deconstruct layer by layer like this. So, there might be hours or days in between garbage collector passes, making it seem like that memory was leaked. (And often, other trees would build up in the intervening time between passes.)

Avoiding toggle references

To mitigate the problem Georges implemented two optimizations. First, the “avoid toggle references” patch, which was actually written by Giovanni Campagna several years ago but never finished, made it so that objects don’t start out using the toggle reference system. Instead, only the JS objects hold references to the GObjects. The JS object can get garbage collected whenever nothing else refers to it, and it will drop its reference to the GObject.

A problem then occurs when that wasn’t the last reference to the GObject, i.e. it’s being kept alive by some C code somewhere, and the GObject resurfaces again in JS, for example by being returned by a C function. In this case we recreate the JS object, assuming that it will be identical to the one that was already garbage collected. The only case where that assumption doesn’t hold, is when the JS code sets some state on one of the JS objects. For example, you execute something like myButton._tag = 'foo';. If myButton gets deleted and recreated, it won’t have a _tag property. So in the case where any custom state is set on a JS object, we switch it over to the toggle reference system once again.

In theory this should help, because toggle references cause the tardy sweep problem, so if fewer objects use toggle references, there should be fewer objects collected tardily. However, this didn’t solve the problem, because especially in GNOME Shell, most JS objects have some state on them. And, sadly, it made the toggle reference code even more complicated than it already was.

The Big Hammer

The second optimization Georges implemented was the affectionately nicknamed “Big Hammer”. It checks if any GObjects toggled down during a garbage collector pass, and if so, restart the garbage collector a few seconds after. This made CPU performance worse, but would at least make sure that all unused objects were deleted from memory within a reasonable time frame (under a minute, rather than a day.)

Combined with some other memory optimizations, this made GNOME 3.30 quite a lot less memory hungry than its predecessors.

An afternoon at Mozilla

Earlier this year, I had been talking on IRC to Ted Campbell and Steve Fink on the SpiderMonkey team at Mozilla for a while, about various ins and outs of being an external (i.e. not Firefox) user of SpiderMonkey’s JS engine API. Early September I found myself in Toronto, where Ted Campbell is based out of, and I paid a visit to the Mozilla office one afternoon.

I had lunch with Ted and Kannan Vijayan of the SpiderMonkey team where we discussed the current status of external SpiderMonkey API users. Afterwards, we made the plans which eventually became this GitHub repository of examples and best practices for using the SpiderMonkey JS engine outside of Firefox. We have both documentation and code examples there, and more on the way. This is still in progress, but it should be the beginning of a good resource for embedding the JS engine, and the end of all those out-of-date pages on MDN!

I also learned some good practices that I can put to use in GJS. For example, we should avoid using JS::PersistentRooted except as a last resort, because it roots objects by putting them in a giant linked list, which is then traced during garbage collection. It’s often possible to store the objects more efficiently than that, and trace them from some other object, or the context.

Ending the tardy sweeps

In the second half of the afternoon we talked about some of the problems that I had with SpiderMonkey that were specific to GJS. Of course, the tardy sweep problem was one of them.

For advice on that, Ted introduced me to Nika Layzell, an engineer on the Gecko team. We looked at the XPCOM cycle collector and I was surprised to learn that Gecko uses a scheme similar to toggle references for some things. However, rather than GJS sticking with toggle references, she suggested a solution that had the advantage of being much simpler.

In “Avoiding toggle references” above, I mentioned that the only thing standing in the way of removing toggle references, is custom state on the JS objects. If there is custom state, the objects can’t be destroyed and recreated as needed. In Gecko, custom state properties on DOM objects are called “expandos” or “expando properties” and are troublesome in a similar way that they are in GJS’s toggle references.

Nika’s solution is to separate the JS object from the expandos, putting the expandos on a separate JS object which has a different lifetime from the JS object that represents the GObject in the JS code. We can then make the outer JS objects into JS Proxies so that when you get or set an expando property on the JS object, it delegates transparently to the expando object.

Kind of like this:

In the “before” diagram, there is a reference cycle which we have to solve with toggle references, and in the “after” diagram, there is no reference cycle, so everything can simply be taken care of by the garbage collector.

In cases where an object doesn’t have any expando properties set on it, we don’t even need to have an expando object at all. It can be created on demand, just like the JS object. It’s also important to note that the expando objects can never be accessed directly from JS code; the GObject is the sole conduit by which they can be accessed.

Recasting our GUI from the beginning of the post with a tree of GUI elements where the top-level window has an expando property pointing to the bottom-level button, and where the window was just closed, gives us this:

Most of these GObjects don’t even need to have expando objects, or JS objects!

At first glance this might seem to be garbage-collectable all at once, but we have to remember that GObjects aren’t integrated with the garbage collector, because they can’t be traced, they can only have their reference counts decremented. And the JS engine doesn’t allow you to make new garbage in the middle of a garbage collector sweep. So a naive implementation would have to collect this in two passes, leaving the window’s expando object and the button for the second pass:

This would require an extra garbage collector pass for every expando property that referred to another GObject via its JS object. Still a lot better than the previous situation, but it would be nice if we could collect the whole thing at once.

We can’t walk the whole tree of GObjects in the garbage collector’s marking phase; remember, GObject references are nondirectional, so there’s no generic way to ask a GObject which other GObjects it references. What we can do is partially integrate with the marking phase so that when a GObject has only one reference left, we make it so that the JS object traces the expando object directly, instead of the GObject rooting the expando object. Think of it as a “toggle reference lite”. This would solve the above case, but there are still some more corner cases that would require more than one garbage collection pass. I’m still thinking about how best to solve this.

What’s next

All together, this should make the horrible toggle reference code in GJS a lot simpler, and improve performance as well.

I started writing the code for this last weekend. If you would like to help, please get in touch with me. You can help by writing code, figuring out the corner cases, or testing the code by running GNOME Shell with the branch of GJS where this is being implemented. Follow along at issue #217.

Additionally, since I am in Toronto again, I’ll be visiting the Mozilla office again this week, and hopefully more good things will come out of that!

Acknowledgements

Thanks to Ted Campbell, Nika Layzell, and Kannan Vijayan of Mozilla for making me feel welcome at Mozilla Toronto, and taking some time out of their workday to talk to me; and thanks to my employer Endless for letting me take some time out of my workday to go there.

Thank you to Ted Campbell and Georges Stavracas for reading and commenting on a draft version of this post.

The diagrams in this post were made with svgbob, a nifty tool; hat tip to Federico Mena Quintero.

Advertisement

The Parable of the Code Review

Last week’s events, with Linus Torvalds pledging to stop behaving like an asshole, instituting a code of conduct in Linux kernel development, and all but running off to join a monastery, have made a lot of waves. The last bastion of meritocracy has fallen! Linus, the man with five middle fingers on each hand, was going to save free software from ruin by tellin’ it like it is to all those writers of bad patches. Now he has gone over to the Dark Side, etc., etc.

There is one thing that struck me when reading the arguments last week, that I never realized before (as I guess I tend to avoid reading this type of material): the folks who argue against, are convinced that the inevitable end result of respectful behaviour is a weakening of technical skill in free software. I’ve read from many sources last week the “meritocracy or bust” argument that meritocracy means three things: the acceptance of patches on no other grounds than technical excellence, the promotion of no other than technically excellent people to maintainer positions within projects, and finally the freedom to disrespect people who are not technically excellent. As I understand these people’s arguments, the meritocracy system works, so removing any of these three pillars is therefore bound to produce worse results than meritocracy. Some go so far as to say that treating people respectfully, would mean taking technically excellent maintainers and replacing them with less proficient people chosen for how nice¹ they are.

I never considered the motivations that way; maybe I didn’t give much thought to why on earth someone would argue in favour of behaving like an asshole. But it reminded me of a culture shift that happened a number of years ago, and that’s what this post is about.

Back in the bad old days…

It used to be that we didn’t have any code review in the free software world.

Well, of course we have always had code review; you would post patches to something like Bugzilla or a mailing list and the maintainer would review them and commit them, ask for a revision, or reject them (or, if the maintainer was Linus Torvalds, reject them and tell you to kill yourself.)

But maintainers just wrote patches and committed them, and didn’t have to review them! They were maintainers because we trusted them absolutely to write bug-free code, right?² Sure, it may be that maintainers committed patches with mistakes sometimes, but those could hardly have been avoided. If you made avoidable mistakes in your patches, you didn’t get to be a maintainer, or if you did somehow get to be a maintainer then you were a bad one and you would probably run your project into the ground.

Somewhere along the line we got this idea that every patch should be reviewed, even if it was written by a maintainer. The reason is not because we want to enable maintainers who make mistakes all the time! Rather, because we recognize that even the most excellent maintainers do make mistakes, it’s just part of being human. And even if your patch doesn’t have a mistake, another pair of eyes can sometimes help you take it to the next level of elegance.

Some people complained: it’s bureaucratic! it’s for Agile weenies! really excellent developers will not tolerate it and will leave! etc. Some even still believe this. But even our tools have evolved over time to expect code review — you could argue that the foundational premise of the GitHub UI is code review! — and the perspective has shifted in our community so that code review is now a best practice, and what do you know, our code has gotten better, not worse. Maintainers who can’t handle having their code reviewed by others are rare these days.

By the way, it may not seem like such a big deal now that it’s been around for a while, but code review can be really threatening if you aren’t used to it. It’s not easy to watch your work be critiqued, and it brings out a fight-or-flight response in the best of us, until it becomes part of our routine. Even Albert Einstein famously wrote scornfully to a journal editor after a reviewer had pointed out a mistake in his paper, that he had sent the manuscript for publication, not for review.

And now imagine a future where we could say…

It used to be that we treated each other like crap in the free software world.

Well, of course we didn’t always treat each other like crap; you would submit patches and sometimes they would be gratefully accepted, but other times Linus Torvalds would tell you to kill yourself.

But maintainers did it all in the name of technical excellence! They were maintainers because we trusted them absolutely to be objective, right? Sure, it may be that patches by people who didn’t fit the “programmer” stereotype were flamed more often, and it may be that people got sick of the disrespect and left free software entirely, but the maintainers were purely objectively looking at technical excellence. If you weren’t purely objective, you didn’t get to be a maintainer, or if you somehow did get to be a maintainer then you were a bad one and you would probably run your project into the ground.

Somewhere along the line we got this idea that contributors should be treated with respect and not driven away from projects, even if the maintainer didn’t agree with their patches. The reason is not because we want to force maintainers to be less objective about technical excellence! Rather, because we recognize that even the most objective maintainers do suffer from biases, it’s just part of being human. And even if someone’s patch is objectively bad, treating them nonetheless with respect can help ensure they will stick around, contribute their perspectives which may be different from yours, and rise to a maintainer’s level of competence in the future.

Some people complained: it’s dishonest! it’s for politically correct weenies! really excellent developers will not tolerate it and will leave! etc. Some even still believe this. But the perspective has shifted in our community so that respect is now a best practice, and what do you know, our code (and our communities) have gotten better, not worse. Maintainers who can’t handle treating people respectfully are rare these days.

By the way, it may not seem like such a big deal now that it’s been around for a while, but confronting and acknowledging your own biases can be really threatening if you aren’t used to it… I think by now you get the idea.

Conclusion, and a note for the choir

I generally try not to preach to the choir anymore, and leave that instead to others. So if you are in the choir, you are not the audience for this post. I’m hoping, possibly vainly, that this actually might convince someone to think differently about meritocracy, and consider this a bug report.

But here’s a small note for us in the choir: I believe we are not doing ourselves any favours by framing respectful behaviour as the opposite of meritocracy, and I think that’s part of why the pro-disrespect camp have such a strong reaction against it. I understand why the jargon developed that way: those driven away by the current, flawed, implementation of meritocracy are understandably sick of hearing about how meritocracy works so well, and the term itself has become a bit poisoned.

If anything, we are simply trying to fix a bug in meritocracy³, so that we get an environment where we really do get the code written by the most technically excellent people, including those who in the current system get driven away by abusive language and behaviour.

---

[1] To be clear, I strive to be both nice and technically excellent, and the number of times I’ve been forced to make a tradeoff between those two things is literally zero. But that’s really the whole point of this essay ↩

[2] A remnant of these bad old days of absolute trust in maintainers, that still persists in GNOME to this day, is that committer privileges are for the whole GNOME project. I can literally commit anything I like, to any repository in gitlab.gnome.org/GNOME, even repositories that I have no idea what they do, or are written in a programming language that I don’t know! ↩

[3] A point made eloquently by Matthew Garrett several years ago ↩

JavaScript news from GNOME 3.30

Welcome back to the latest news on GJS, the Javascript engine that powers GNOME Shell, Endless OS, and many GNOME apps.

I haven’t done one of these posts for several versions now, but I think it’s a good tradition to continue. GNOME 3.30 has been released for several weeks now, and while writing this post I just released the first bugfix update, GJS 1.54.1. Here’s what’s new!

If you prefer to watch videos rather than read, see my GUADEC talk on the subject.

JavaScript upgrade!

GJS is based on SpiderMonkey, which is the name of the JavaScript engine from Mozilla Firefox. We now use the version of SpiderMonkey from Firefox 60. (The way it goes is that we upgrade whenever Firefox makes an extended support release (ESR), which happens about once a year.)

This brings a few language improvements: not as many as in 2017 when we zipped through a backlog of four ESRs in one year, but here’s a short list:

• Asynchronous iterators (for await (... in ...))
• Rest operator in object destructuring (var {a, b, ...cd} = ...)
• Spread operator in object literals (obj3 = {...obj1, ...obj2})
• Anonymous catch (catch {...} instead of catch (e) {...})
• Promise.prototype.finally()

There are also some removals from the language, of Mozilla-specific extensions that never made it into the web standards.

• Conditional catch (catch (e if ...))
• For-each-in loops (for each (... in ...))
• Legacy lambda syntax (function (x) x * x)
• Legacy iterator protocol
• Array and generator comprehensions ([for (x of iterable) expr(x)])

Hopefully you weren’t using any of these, because they will not even parse anymore! I wrote a tool called moz60tool that will scan your source files and hopefully flag any uses of the removed syntax. It’s also available as a shell extension by Andy Holmes.’

Time for your code to get a checkup… Photo by rawpixel.com on Pexels.com

ByteArray

A special note about ByteArray: the SpiderMonkey upgrade made it necessary to rewrite the ByteArray class, since support for intercepting property accesses in C++-native JS objects was removed, and that was what ByteArray used internally to implement expressions like bytearray[5].

The replacement API I think would have made performance worse, and ByteArray is pretty performance critical; so I took the opportunity to replace ByteArray with JavaScript’s built-in Uint8Array. (Uint8Array didn’t exist when GJS was invented.) For this, I implemented a feature in SpiderMonkey that allows you to store a GBytes inside a JavaScript ArrayBuffer object.

The result is not 100% backwards compatible. Some functions now return a Uint8Array object instead of a ByteArray and there’s not really a way around that. The two are not really unifiable; Uint8Array’s length is immutable, for one thing. If you want the old behaviour back, you can call new ByteArray.ByteArray() on the returned Uint8Array and all the rest of your code should work as before. However, the legacy ByteArray will have worse performance than the Uint8Array, so instead you should port your code.

Technical Preview: Async Operations

The subject of Avi Zajac’s summer internship was integrating Promises and async functions with GIO’s asynchronous operations. That is, instead of this,

file.load_contents_async(null, (obj, res) => {
    const [, contentsBytes, etag] = obj.load_contents_finish(res);
    print(ByteArray.toString(contentsBytes));
});

you should be able to do this:

file.load_contents_async(null)
.then((contentsBytes, etag) => print(ByteArray.toString(contentsBytes)));

or this:

const [contentsBytes, etag] = await file.load_contents_async(null);
print(ByteArray.toString(contentsBytes));

If you don’t pass in a callback to the operation, it assumes you want a Promise instead of a callback, and will return one so that you can call .then() on it, or use it in an await expression.

This feature is a technology preview in GNOME 3.30 meaning, you must opt in for each method that you want to use it with. Opt in by executing this code at the startup of your program:

Gio._promisify(classPrototype, asyncMethodName, finishMethodName);

This is made a bit extra complicated for file operations, because Gio.File is actually an interface, not a class, and because of a bug where JS methods on interface prototypes are ignored. We also provide a workaround API for this, which unfortunately only works on local (disk) files. So the call to enable the above load_contents_async() code would look like this:

Gio._promisify(Gio._LocalFilePrototype, 'load_contents_async', 'load_contents_finish');

And, of course, if you are using an older GNOME version than 3.30 but you still want to use this feature, you can just copy the Promisify code into your own program, if the license is suitable. I’ve already been writing some code for Endless Hack in this way and it is so convenient that I never want to go back.

Debugger

At long last, there is a debugger. Run it with gjs -d yourscript.js!

The debugger commands should be familiar if you’ve ever used GDB. It is a bit bare-bones right now; if you want to help improve it, I’ve opened issues #207 and #208 for some improvements that shouldn’t be too hard to do.

The debugger is based on Jorendb, a toy debugger by Jason Orendorff which is included in the SpiderMonkey source repository as an example of the Debugger API.

Performance improvements

We’ve made some good improvements in performance, which should be especially apparent in GNOME Shell. The biggest improvement is the Big Hammer patch by Georges Stavracas, which should stop your GNOME Shell session from holding on to hundreds of megabytes at a time. It’s a mitigation of the Tardy Sweep problem which is explained in detail by Georges here. Unfortunately, it makes a tradeoff of worse CPU usage in exchange for better memory usage. We are still trying to find a more permanent solution. Carlos Garnacho also made some further improvements to this patch during the 3.30 cycle.

The other prominent improvement is better memory usage for GObjects in general. A typical GNOME Shell run contains thousands or maybe ten-thousands of GObjects, so shaving even a few bytes off per object has a noticeable effect. Carlos Garnacho started some work in this direction and I continued it. In the end we went from 128 bytes per GObject to 88 bytes. In both cases there is an extra 32 byte penalty if the object has any state set on it from JavaScript code. With these changes, GNOME Shell uses several tens of megabytes less memory before you even do anything.

I have opened two issues for further investigation, #175 and #176. These are two likely avenues to reduce the memory usage even more, and it would be great if someone were interested to work on them. If they are successful, it’s likely we could get the memory usage down to 56 bytes per GObject, and eliminate the extra 32 byte penalty.

Eventually we will get to that “well-oiled machine” state… Photo by Celine Nadon on Unsplash

Developer Experience

I keep insisting it’s no coincidence, that as soon as we switched to GitLab we started seeing an uptick in contributors whom we hadn’t seen before. This trend has continued: we merged patches from 22 active contributors to GJS in this cycle, up from 13 last time.

Claudio André landed many improvements to the GitLab CI. For one thing, the program is now built and tested on more platforms and using more compile options. He also spent a lot of effort ensuring that the most common failures will fail quickly, so that developers get feedback quickly.

From my side, the maintainer tasks have gotten a lot simpler with GitLab. When I review a merge request, I can leave the questions of “does it build?” and “are all the semicolons there?” to the CI, and concentrate on the more important questions of “is this a feature we want?” and “is it implemented in the best way?” The thumbs-up votey things on issues and merge requests also provide a bit of an indication of what people would most like to see worked on, although I am not really using these systematically yet.

We have some improvements soon to be deployed to DevDocs, and GJS Guide, a site explaining some of the more basic GJS concepts. Both of these were the subject of Evan Welsh’s summer internship. Evan did a lot of work in upstream DevDocs, porting it from the current unsupported CoffeeScript version to a more modern web development stack, which will hopefully be merged upstream eventually.

It’s about time we had a signpost to point the way in GJS. Photo by Jens Johnsson on Pexels.com

We also have an auto formatter for C++ code, so if you contribute code, it’s easier to avoid your branches failing CI due to linter errors. You can set it up so that it will correct your code style every time you commit; there are instructions in the Hacking file. It uses Marco Barisione’s clang-format-hooks. The process isn’t infallible, though: the CI job uses cpplint and the auto formatter uses clang-format, and the two are not 100% compatible.

There are a few miscellaneous nice things that Claudio made. The test coverage report for the master branch is automatically published on every push. And if you want to try out the latest GJS interpreter in a Flatpak, you can manually trigger the “flatpak” CI job and download one.

What’s coming in 3.32

There are a number of efforts already underway in the 3.32 cycle.

ES6 modules should be able to land! This is an often requested feature and John Renner has a mostly-working implementation already. You can follow along on the merge request.

Avi Zajac is working on the full version of the async Promises feature, both the gobject-introspection and GJS parts, which will make it no longer opt-in; Promises will “just work” with all GIO-based async operations.

Also related to async and promises, Florian Müllner is working on a new API that will simplify calling DBus interfaces using some of the new ES6 features we have gained in recent releases.

I hope to land Giovanni Campagna’s old “argument cache” patch set, which looks like it will speed up calls from JS into C by quite a lot. Apparently there is a similar argument cache in PyGObject.

Finally, and this will be the subject of a separate blog post coming soon, I think we have a plausible solution to the Tardy Sweep problem! I’m really excited to write about this, as the solution is really ingenious (I can say that, because I did not think of it myself…)

Contributors

Thanks to everyone who participated to bring GJS to GNOME 3.30: Andy Holmes, Avi Zajac, Carlos Garnacho, Christopher Wheeldon, Claudio André, Cosimo Cecchi, Emmanuele Bassi, Evan Welsh, Florian Müllner, Georges Basile Stavracas Neto, James Cowgill, Jason Hicks, Karen Medina, Ole Jørgen Brønner, pixunil, Seth Woodworth, Simon McVittie, Tomasz Miąsko, and William Barath.

As well, this release incorporated some old patches that people contributed in the past, even up to 10 years ago, that were never merged because they needed some tweaks here or there. Thanks to those people for participating in the past, and I’m glad we were finally able to land your contributions: Giovanni Campagna, Jesus Bermudez Velazquez, Sam Spilsbury, and Tommi Komulainen.

GUADEC 2018 Reminiscences

This year’s GUADEC in Almería, Spain, was over two months ago, and so here is a long overdue post about it. It was so long ago that I might as well call it a reminiscence! This will be a different kind of post than the ones I’ve done in past years, as plenty of other bloggers have already posted summaries about the talks.

Board of Directors

I didn’t even get to see that many talks anyway, as this was my first GUADEC after being elected to the GNOME Foundation board of directors and I found myself doing a lot of running around to complete things. The board has to prepare for a number of meetings including the GNOME Foundation’s Annual General Meeting that’s always held at GUADEC, and so there was plenty of preparation to be done.

So, except for a few sessions, I mainly followed the “hallway track” this year.

It’s an exciting time to be on the board; it’s been in the news recently that the GNOME Foundation has received two substantial donations, and is hiring some new roles. If you want more information and background, Rosanna Yuen, director of operations, explains all about it in this GUADEC talk.

Interns

Somehow I found myself mentoring two interns this summer, Avi Zajac and Evan Welsh, and both of them were able to attend GUADEC. (I co-mentored Evan with my coworker Manuel Quiñones, who unfortunately could not be there.) I had not done a good job introducing them to each other, but they connected with each other and realized that they were both working with me! If you haven’t already, make sure to read Avi’s blog post and Evan’s blog post for their perspectives on how it went. I was glad to have both of them there and really enjoyed meeting up in person.

Both internships have finished up in the meantime. Evan’s website is viewable here, as well as some improvements to DevDocs which I hope to deploy soon. Avi’s project was released as a technical preview in GNOME 3.30 and will be covered in my next blog post.

JavaScript Talk

I gave one scheduled talk, on GJS and JavaScript, and one unscheduled talk, on Endless Code.

I will cover the material from the JavaScript talk in my next post about the new features in GJS, but for now I wanted to post the slides for everyone’s reference. The video of the talk is here.

Endless Hack Talk

I was voted into one of the conference’s open talk slots with a proposal to talk about Endless Code (since then, renamed to Endless Hack). This is a new (well, it was new at the time of the conference) project at Endless. It’s a continuation of this feature which (I didn’t work on, but) my coworkers demoed about two years ago:

Want to hack on an app in @endlessglobal? Simple, just flip over the window and start coding. All code, build tools, and deps are there. pic.twitter.com/T3maWdw2eI

— Jono Bacon (@jonobacon) January 5, 2017

The Endless Hack product generalizes this idea to the whole desktop. The idea is that you should be able to tinker with everything, and there’s a narrative that guides you along the way and teaches you programming concepts. It’s aimed at children and young teenagers. Although this product hasn’t been released yet, and although some of the source code is currently open it’s not in a finished or usable state yet, I did want to talk about it at GUADEC because I think the ability to learn by tinkering is an important part of the free software experience and a direct consequence of one of the Four Software Freedoms, and it’s something the GNOME community should be aware of.

We also made a survey asking people about their experience learning how to program, or not learning how to program, and it’s still open, because I did not do a very good job in the talk of publicizing the link! You can fill it out here.

I haven’t dared to watch the video of me talking completely unrehearsed, but you can watch it here if you want.

Unconference

I had high hopes for organizing a GJS unconference session like last year, but after a certain point I was just completely tired out. We did eventually have a GJS session that consisted of people hacking on their favourite thing. Happily, I was able to convince Georges Stavracas to fix a regression that was preventing GNOME Shell from starting. I got a chance to work with Meg Ford on testing with JavaScript, and I also got some work done on the GJS debugger, a new feature in GNOME 3.30. I will talk about all this and more in my next post!

We also used some of the unconference time for a kickoff session for the GNOME Developer Center. Bastian Ilsø is leading this initiative and has a lot of material for you to read on what’s happened in the meantime.

Acknowledgements

I’d like to thank the GNOME Foundation for making it possible for me to attend the conference and the board meetings.

Thank you to my coworker Lisette Silva for convincing me to submit the open talk and giving some last-minute feedback beforehand.

Geek tip: g_object_new and constructors

tl;dr Don’t put any code in your foo_label_new() function other than g_object_new(), and watch out with Vala.

From this GJS bug report I realized there’s a trap that GObject library writers can fall into,

Avoid code at your construction site.

that I don’t think is documented anywhere. So I’m writing a blog post about it. I hope readers from Planet GNOME can help figure out where it needs to be documented.

For an object (let’s call it FooLabel) that’s part of the public API of a library (let’s call it libfoo), creating the object via its foo_label_new() constructor function should be equivalent to creating it via g_object_new().

If foo_label_new() takes no arguments then it should literally be only this:

FooLabel *
foo_label_new(void)
{
    return g_object_new(FOO_TYPE_LABEL, NULL);
}

If it does take arguments, then they should correspond to construct properties, and they should get set in the g_object_new() call. (It’s customary to at least put all construct-only properties as arguments to the constructor function.) For example:

FooLabel *
foo_label_new(const char *text)
{
    return g_object_new(FOO_TYPE_LABEL,
        "text", text,
        NULL);
}

Do not put any other code in foo_label_new(). That is, don’t do this:

FooLabel *
foo_label_new(void)
{
    FooLabel *retval = g_object_new(FOO_TYPE_LABEL, NULL);
    retval->priv->some_variable = 5;  /* Don't do this! */
    return retval;
}

The reason for that is because callers of your library will expect to be able to create FooLabels using g_object_new() in many situations. This is done when creating a FooLabel in JS and Python, but also when creating one from a Glade file, and also in plain old C when you need to set construct properties. In all those situations, the private field some_variable will not get initialized to 5!

Instead, put the code in foo_label_init(). That way, it will be executed regardless of how the object is constructed. And if you need to write code in the constructor that depends on construct properties that have been set, use the constructed virtual function. There’s a code example here.

If you want more details about what function is called when, Allison Lortie has a really useful blog post.

This trap can be easy to fall into in Vala. Using a construct block is the right way to do it:

namespace Foo {
public class Label : GLib.Object {
    private int some_variable;

    construct {
        some_variable = 5;
    }
}
}

This is the wrong way to do it:

namespace Foo {
public class Label : GLib.Object {
    private int some_variable;

    public Label() {
        some_variable = 5;  // Don't do this!
    }
}
}

This is tricky because the wrong way seems like the most obvious way to me!

This has been a public service announcement for the GNOME community, but here’s where you come in! Please help figure out where this should be documented, and whether it’s possible to enforce it through automated tools.

For example, the Writing Bindable APIs page seems like a good place to warn about it, and I’ve already added it there. But this should probably go into Vala documentation in the appropriate place. I have no idea if this is a problem with Rust’s gobject_gen! macro, but if it is then it should be documented as well.

Documented pitfalls are better than undocumented pitfalls, but removing the pitfall altogether is better. Is there a way we can check this automatically?

Announcing Flapjack

Here’s a post about a tool that I’ve developed at work. You might find it useful if you contribute to any desktop platform libraries that are packaged as a Flatpak runtime, such as GNOME or KDE.

Flatpak is a system for delivering desktop applications that was pioneered by the GNOME community. At Endless, we have jumped aboard the Flatpak train. Our product Endless OS is a Linux distribution, but not a traditional one in the sense of being a collection of packages that you install with a package manager; it’s an immutable OS image, with atomic updates delivered through OSTree. Applications are sandboxed-only and Flatpak-only.

Flatpak makes the lives of application developers much easier, who want to get their applications to users without having to care which Linux distribution those users use. It means that as an application developer, I don’t have to fire up three different virtual machines and email five packaging contributors whenever I make a release of my application. (Or, in theory it would work that way if I would stop using deprecated libraries in my application!)

This is what flapjacks are in the UK, Ireland, Isle of Man, and Newfoundland. Known as “granola bars” or “oat bars” elsewhere. By Alistair Young, CC BY 2.0, https://commons.wikimedia.org/w/index.php?curid=5331306

On my work computer I took the leap and now develop everything on an immutable OSTree system just like it would be running in production. I now develop everything inside a Flatpak sandbox. However, while Flatpak works great when packaging some code that already exists, it is a bit lacking in the developer experience.

For app developers, Carlos Soriano has written a tool called flatpak-dev-cli based on a workflow designed by Thibault Saunier of the Pitivi development team. This has proven very useful for developing Flatpak apps.

But a lot of the work I do is not on apps, but on the library stack that is used by apps on Endless OS. In fact, my team’s main product is a Flatpak runtime. I wanted an analogue of flatpak-dev-cli for developing the libraries that live inside a Flatpak runtime.

Flapjack

…while this is what flapjacks are everywhere else in Canada, and in the US. Also known as “pancakes.” By Belathee Photography, CC BY-SA 3.0, https://commons.wikimedia.org/w/index.php?curid=15167594

Flapjack is that tool. It’s a wrapper around Flatpak-builder that is intended to replace JHBuild in the library developer’s toolbox.

For several months I’ve been using it in my day-to-day work, on a system running Endless OS (which has hardly any developer tools installed by default.) It only requires Flatpak-builder, Git, and Python.

In Flapjack’s README I included a walkthrough for reproducing Tristan’s trick from his BuildStream talk at GUADEC 2017 where he built an environment with a modified copy of GTK that showed all the UI labels upside-down.

That walkthrough is pretty much what my day-to-day development workflow looks like now. As an example, a recent bug required me to patch eos-knowledge-lib and xapian-glib at the same time, which are both components of Endless’s Modular Framework runtime. I did approximately this:

flapjack open xapian-glib
flapjack open eos-knowledge-lib
cd checkout/xapian-glib
# ... make changes to code ...
flapjack test xapian-glib
# ... keep changing and repeating until the tests pass!
cd ../eos-knowledge-lib
# ... make more changes to code ...
flapjack test eos-knowledge-lib
# ... keep changing and repeating until the tests pass!
flapjack build
# ... keep changing and repeating until the whole runtime builds!
flapjack run com.endlessm.encyclopedia.en
# run Encyclopedia, which is an app that uses this runtime, to check
# that my fix worked
git checkout -b etc. etc.
# create branches for my work and push them

I also use Flapjack’s “devtools manifest” to conveniently provide developer tools that aren’t present in Endless OS’s base OSTree layer. In Flapjack’s readme I gave an example of adding the jq tool to the devtools manifest, but I also have cppcheck, RR, and a bunch of Python modules that I added with flatpak-pip-generator. Whenever I need to use any of these tools, I just open flapjack shell and they’re available!

Questions you might ask

Why is it called Flapjack?

The working title was jokingly chosen to mess up your muscle memory if you were used to typing flatpak, but it stuck and became the real name. If it does annoy you, you can alias it to fj or something.

Flatpak-builder is old news, why does Flapjack not use BuildStream?

I would like it if that were the case! I suspect that BuildStream would solve my main problem with Flapjack, which is that it is slow. In fact I started out writing Flapjack as a wrapper around BuildStream, instead of Flatpak-builder. But at the time BuildStream just didn’t have enough documentation for me to get my head around it quickly enough. I hear that this is changing and I would welcome a port to BuildStream!

As well, it was not possible to allow --socket=x11 during a build like you can with Flatpak-builder, so I couldn’t get it to run unit tests for modules that depended on GTK.

Why are builds with Flapjack so slow?

The slowest parts are caching each build step (I suspect here is where using BuildStream would help a lot) and exporting the runtime’s debug extension to the local Flatpak repository. For the latter, this used to be even slower, before my colleague Emmanuele Bassi suggested to use a “bare-user” repository. I’m still looking for a way to speed this up. I suspect it should be possible, since for Flapjack builds we would probably never care about the Flatpak repository history.

Can I use Flapjack to develop system components like GNOME Shell?

No. There still isn’t a good developer story for working on system components on an immutable OS! At Endless, the people who work on those components will generally replace their OSTree file system with a mutable one. This isn’t a very good strategy because it means you’re developing on a system that is different from what users are running in production, but I haven’t found any better way so far.

Epilogue

Thanks to my employer Endless for allowing me to reserve some time to write this tool in a way that it would be useful for the wider Flatpak community, rather than just internally.

That’s about it! I hope Flapjack is useful for you. If you have any other questions, feel free to ask me.

Where to find it

Flapjack’s page on PyPI: https://pypi.python.org/pypi/flapjack
The code on GitHub: https://github.com/endlessm/flapjack
Report bugs and request features: https://github.com/endlessm/flapjack/issues

Wrapping up GUADEC 2017

I’m now back home after attending GUADEC 2017 in Manchester, followed by a week of half-vacation traveling around the Netherlands and visiting old friends. It was a fantastic opportunity to meet others in the GNOME community once again; having gone to a few hackfests and conferences in the past two years, I now recognize many friendly faces that I am happy to get a chance to see from time to time.

Talks

Here’s what I attended during the conference; I’ll link to the videos and provide a sentence or two of commentary.

• The GNOME Way, Allan Day (video) — for me, one of the two highlights of the conference, a great statement of what makes GNOME tick, and a great opener for its 20th birthday.
• Limited Connectivity, Endless Apps, Joaquim Rocha (video) — although already familiar to me, it was a nice overview of the product that I work on at Endless.
• State of the Builder, Christian Hergert — one of these days I will start using Builder as soon as I can find some time to get it to learn my preferred keybindings.
• The Battle over Our Technology, Karen Sandler (video) — the second of the two conference highlights, a timely reminder of why free software is important.
• Seamless Integration to Hack Desktop Applications, Simon Schampijer (video) — my coworker and fellow-person-whose-last-name-gets-pronounced-wrong Simon showed off one of the most empowering features that I have ever seen.
• Progressive Web Apps: an Opportunity for GNOME, Stephen Pearce (video) — I have been reading a lot about progressive web apps recently and am both excited and skeptical. (Stephen also made a webkit game in GJS in just one day.)
• Different Ways of Outreaching Newcomers, Julita Inca (video) — it was fantastic to see this legendary GNOME mentor and organizer speak in person.
• Lightning talks by the GSoC and Outreachy interns (video) — I always admire the intern sessions because I would have soiled myself had I had to speak to a 300-person conference room back when I was an intern. Hopefully next year the interns will have a session earlier in the day so their audience is fresher though! Also a shout out to my coworkers Kate Lasalle-Klein and Robin Tafel who are not interns but also gave a lightning talk during this session about working together with the GNOME design team. (If you’re looking for it in the other lightning talk video, you’re not finding it because it was in this session.)
• Fantastic Layouts and Where to Find Them, Martin Abente Lahaye (video) — a great introduction to Emeus, the constraint layout manager, with a surprise appearance from an airplane hack.
• Replacing C Library Code with Rust: What I Learned, Federico Mena Quintero (slides) — I am mentoring a Summer of Code student, Luke, who is doing some investigation into converting parts of GJS into Rust, and this talk really helped me understand some things from his work that I’ve been struggling with.
• Continuous: Past, Present, Future, Emmanuele Bassi (video) — this talk made me want to help out on that lonely, lonely build sheriff mountain.
• A Brief History of GNOME, Jonathan Blandford (video) — I had seen it before, but an hour well spent.
• GNOME to 2020 and Beyond, Neil McGovern (video) — by turns optimistic and pessimistic, the new GNOME executive director talked about the future.
• What’s Coverity Static Analysis Ever Done for Us?, Philip Withnall (video) — my coworker and fellow-person-with-an-excellent-first-name Philip talked about static analysis, which I cannot wait to start using on GJS.
• Building a Secure Desktop with GNOME Technologies, Matthew Garrett (video) — the excellent “raise your hand if your system is bugging you to install an update right now” line made this talk for me.
• GNOME Build Strategies and BuildStream, Tristan Van Berkom (video) — not quite what I expected, but luckily I got a better idea of what BuildStream does from the unconference session.
• Bringing GNOME Home to Ubuntu, Tim Lunn (video) — it was a pleasure to meet Tim in person, who did the SpiderMonkey porting work on GJS before me, and whose commits I have often referred to.
• GitLab, Carlos Soriano — I’m really excited to kick Bugzilla out of my workflow as soon as I can.
• Microsoft ❤️ Linux, Julian Atanasoae — If nothing else Julian is brave to get up in front of this audience and sing the praises of Microsoft. I am skeptically optimistic; sure, Microsoft is doing some great things for open source, I even had a slide about some Microsoft tools in my talk, but on the other hand let’s not forget they were still trying to undermine and destroy our community not too long ago.
• How to Get Better Mileage out of Glade, Juan Pablo Ugarte (video) — Slides created in Glade, what more could you ask for?
• Lightning talks (video) —The highlight for me of the second lightning talk session was Sri’s self-described “rant.” There were a few talks in the lineup that I felt it was too bad didn’t get any time.

There were also so many talks that were programmed opposite the talks that I decided to go see. It seemed like that happened more often than last year! (Either my interests have broadened, or the quality of the talks is increasing…) I will be watching many videos in the coming days, now that they have been released, but I was especially sad not to see the two talks on animations by Jakub Steiner and Tobias Bernard because they were opposite (and immediately after, respectively) my own talk!

And the video of my talk is now published as well, although like many people I find it excruciating to watch myself on video; the rest of you can watch it, I’ll watch this instead.

Unconference

The unconference part of the conference (where people propose topics, get together with like-minded attendees in a conference room, and talk or work together) was held in a nice workspace. I had one session on GJS on Monday where we first discussed how the Ubuntu desktop team (I got to meet Ken VanDine, Iain Lane, and Chris Coulson, as well as connect with Tim Lunn again) was going to deploy Mozilla’s security updates to Javascript (and therefore GJS and GNOME Shell) in Ubuntu’s long-term support releases. Then Stephen Pearce joined and suggested a GJS working group in order to make development more visible.

Later I joined the GNOME Shell performance session where I rebased Christian Hergert’s GJS profiler code and showed Jonas Adahl how it worked; we profiled opening and closing the overview.

On the second day I joined the Continuous and Builder sessions. Builder was looking good on a giant TV set!

On the third day I attended the BuildStream session and I’m quite excited about trying it out for a development workflow while hacking on a component of a Flatpak runtime, which is a shaky task at best using the current Flatpak tools.

In the afternoon I technically “had another GJS session” though it’s my experience on the third unconference day that all the sessions melt into one. This time many people went on a hike in the afternoon. I was very sad to have missed it, since I love hiking, but I was having an allergy attack at the time which made it difficult for me to be outside. However, I spent the afternoon working on the GObject class stuff for GJS instead, and chatting with people.

Social events

This GUADEC conference had the best social event on Saturday night: a GNOME 20th birthday party, complete with cake, old farts Q&A panel, trivia quiz, raffle, and a blast from the past dance floor with music from back when GNOME started. There was even an afterparty way into the small hours … which I did not go to because my talk was in the first slot on Sunday morning!

Apart from that there were many opportunities to connect with people, from day 1 through 6. One thing I like about GUADEC is that the days are not stuffed full of talks and there is plenty of time to have casual conversations with people. One “hallway session” that I had, for example, was a conversation with Marco Barisione, talking about the reverse debuggers RR and UndoDB. Another was with with Sri Ramkrishna, following on from his lightning “rant” on Sunday, about what kind of help beginning app writers are looking for, whether they can get it from tutorials or Stack Overflow, and what kinds of things get in their way.

Thanks

Many thanks to the GNOME Foundation for sponsoring my attendance. I’m really glad to have been able to join in this year.

Modern Javascript in GNOME – GUADEC 2017 talk

I gave a presentation at GUADEC 2017 this morning on modern Javascript in GNOME, the topic of the last few posts on this blog. As I promised during the talk, here are the slides. There is a beefy appendix after the questions slide, with details about all the new language features, that you are welcome to use as a reference.

Thanks to the GNOME Foundation for the travel sponsorship, to my employer Endless for paying for some of this work, and especially to Rob McQueen for the last-minute loan of a USB-C video adapter!

Inventing GObject ES6 classes

Hello again! If you’re a GJS user, I’d like your opinion and ideas. After my last post where I talked about new features coming in GNOME 3.26 to GJS, GNOME’s Javascript engine, I’m happy to say that the patches are nearly ready to be landed. We just need to figure out how to build SpiderMonkey 52 consistently even though Mozilla hasn’t made an official standalone release of it yet.

A better literal depiction of JAVA SCRIPT I could not ask for… (Public domain image courtesy of Engin_Akyurt)

As I reported last time:

After that is done, I will refactor GJS’s class system (Lang.Class and GObject.Class). I believe this needs to be done before GNOME 3.26. That’s because [we will] gain ES6 classes, and I want to avoid the situation where we have two competing, and possibly incompatible, ways to write classes.

That’s what I’m busy doing now, in the run-up to GUADEC later this month, and I wanted to think out loud in this blog post, and give GJS users a chance to comment.

First of all, the legacy Lang.Class classes will continue to work. You will be able to write ES6 classes that inherit from legacy classes, so you can start using ES6 classes without refactoring all of your code at once.

That was the good news, now the bad

However, there is not an obvious way to carry over the ability to create GObject classes and interfaces from legacy classes to ES6 classes. The main problem is that Lang.Class and its subclasses formed a metaclass framework. This was used to carry out certain activities at the time the class object itself was constructed, such as registering with the GType system.

ES6 classes don’t have a syntax for that, so we’ll have to get a bit creative. My goals are to invent something (1) that’s concise and pleasant to use, and (2) that doesn’t get in the way when classes gain more features in future ES releases; that is, not too magical. (Lang.Class is pretty magical, but then again, there wasn’t really an alternative at the time.)

Here is how the legacy classes worked, with illustrations of all the possible bells and whistles:

	const MyClass = new Lang.Class({
	Name: 'MyClass',
	GTypeName: 'MyNamespaceMyClass',
	Extends: GObject.Object,
	Implements: [Gio.Initable, MyCustomInterface],
	Properties: {
	'prop': GObject.ParamSpec.int( /* etc., etc. */ ),
	},
	Signals: {
	'signal': { param_types: [ /* etc., etc. */ ] },
	},
	_init(props={}) {
	this.parent(props);
	// etc.
	},
	get prop() { /* … */ },
	method(arg) { /* … */ },
	});

view raw

legacy.js

hosted with ❤ by GitHub

The metaclass magic in Lang.Class notices that the class extends GObject.Object, and redirects the construction of the class object to GObject.Class. There, the other magic properties such as Properties and Signals are processed and removed from the prototype, and it calls a C function to register the type with the GObject type system.

Without metaclasses, it’s not possible to automatically carry out magic like that at the time a class object is constructed. However, that is exactly the time when we need to register the type with GObject. So, you pretty much need to remember to call a function after the class declaration to do the registering.

The most straightforwardly translated (fictional) implementation might look something like this:

	class MyClass extends GObject.Object {
	static get GTypeName { return 'MyNamespaceMyClass'; }
	static get Implements { return [Gio.Initable, MyCustomInterface]; }
	static get Properties {
	return {
	'prop': GObject.ParamSpec.int( /* etc., etc. */ ),
	};
	}
	static get Signals {
	return {
	'signal': { /* etc. */ },
	};
	}
	constructor(props={}) {
	super(props);
	// etc.
	}
	get prop() { /* … */ }
	method(arg) { /* … */ }
	}
	GObject.registerClass(MyClass);

view raw

verbose.js

hosted with ❤ by GitHub

The fictional GObject.registerClass() function would take the role of the metaclass’s constructor.

This is a step backwards in a few ways compared to the legacy classes, and very unsatisfying. ES6 classes don’t yet have syntax for fields, only properties with getters, and the resulting static get syntax is quite unwieldy. Having to call the fictional registerClass() function separately from the class is unpleasant, because you can easily forget it.

On the other hand, if we had decorators in the language we’d be able to make something much more satisfying. If you’re familiar with Python’s decorators, these are much the same thing: the decorator is a function which takes the object that it decorates as input, performs some action on the object, and returns it. There is a proposed decorator syntax for Javascript that allows you to decorate classes and class properties. This would be an example, with some more fictional API:

	@GObject.Class('MyNamespaceMyClass')
	@GObject.implements([Gio.Initable, MyCustomInterface])
	@GObject.signal('signal', { /* etc. */ })
	class MyClass extends GObject.Object {
	constructor(props={}) {
	super(props);
	// etc.
	}
	@GObject.property.int('Short name', 'Blurb', GObject.ParamFlags.READABLE, 42)
	get prop() { /* etc. */ }
	method(arg) { /* etc. */ }
	}

view raw

decorators.js

hosted with ❤ by GitHub

This is a lot more concise and natural, and the property decorators are similar to the equivalent in PyGObject, but unfortunately it doesn’t exist. Decorators are still only a proposal, and none of the major browser engines implement them yet. Nonetheless, we can take the above syntax as an inspiration, use a class expression, and move the registerClass() function around it and the GObject stuff outside of it:

	var MyClass = GObject.registerClass({
	GTypeName: 'MyNamespaceMyClass',
	Implements: [Gio.Initable, MyInterface],
	Properties: { 'prop': GObject.ParamSpec.int( /* etc. */ ) },
	Signals: { 'signal': { /* etc. */ } },
	}, class MyClass extends GObject.Object {
	constructor(props={}) {
	super(props);
	// etc.
	}
	get prop() { /* … */ }
	method(arg) { /* … */ }
	});

view raw

class.js

hosted with ❤ by GitHub

	var MyInterface = GObject.registerInterface({
	GTypeName: 'MyNamespaceMyInterface',
	Requires: [Gio.Initable],
	Properties: { 'prop': GObject.ParamSpec.int( /* etc. */ ) },
	Signals: { 'signal': { /* etc. */ } },
	}, class MyInterface {
	get prop() { /* … */ }
	method(arg) { /* … */ }
	});

view raw

interface.js

hosted with ❤ by GitHub

Here, the body of the class is almost identical to what it would be with the decorator syntax. All the extra stuff for GObject is contained at the top of the class like it would be with the decorators. We don’t have the elegance of the property decorator, but this is quite an improvement on the first iteration. It’s not overly magical, it even acts like a decorator: it takes a class expression, and gives back a GObject-ized class. And when decorators eventually make it into standard Javascript, the basic idea is the same, so converting your code will be easy enough. (Or those who use transpiling tools can already go ahead and implement the decorator-based API.)

This is the best API I’ve been able to come up with so far. What do you think? Would you want to use it? Reply to this post or come talk to me in #javascript on GNOME IRC.

Next steps

Note first of all that none of this code exists yet. Depending on what feedback I get here, I hope to have a draft version working before GUADEC, and around the same time I’ll post a more detailed proposal to the javascript-list mailing list.

In addition, I will be speaking about this and more at GUADEC in my talk, “Modern Javascript in GNOME“. If you are attending, come talk to me there! Thanks to the GNOME Foundation for sponsoring my travel and accommodations.

The GJS documentation is back

We have once again a set of accurate, up-to-date documentation for GJS. Find it at devdocs.baznga.org!

Many thanks are due to Everaldo Canuto, Patrick Griffis, and Dustin Falgout for helping get the website back online, and to Nuveo for sponsoring the hosting.

In addition, thanks to Patrick’s lead, we have a docker image if you want to run the documentation site yourself.

If you find any inaccuracies in the documentation, please report a bug at this issue tracker.