More Memory, More Problems

In GJS we recently committed a patch that has been making waves. Thanks to GJS contributor Georges Basile “Feaneron” Stavracas Neto, some infamous memory problems with GNOME Shell 3.28 have been mitigated. (What’s the link between GNOME Shell and GJS? GNOME Shell uses GJS as its internal Javascript engine, in which some of the UI and all of the extensions are implemented.)

There is a technical explanation, having to do with toggle-refs, a GObject concept which we use to interface the JS engine’s garbage collector with GObject’s reference counting system. Georges has already provided a fantastic introduction to the technical details so I will not do another one here. This post will be more about social issues, future plans, and answers to some myths I’ve seen in various comments recently. To read this post, you only need to know that the problem has to do with toggle-refs and that toggle-refs are difficult to reason about.

Not a Memory Leak

I really don’t want to call this a memory leak, much less “the GNOME memory leak” that’s become common in the press coverage lately. I find that that sets the wrong expectations for users suffering from these memory problems. You might say that for the end user it makes no difference, their computer’s memory is being occupied by GNOME Shell, so what’s the point in not calling it a memory leak? And you would be partially right. The effect is no different. The expectations are different though, especially for users who have some technical knowledge. A memory leak is a simple problem to fix. When you have one, you run your software under Valgrind or ASAN, you get a backtrace that shows where the memory was allocated that you didn’t free, and you free it. Problem solved. You can even run Valgrind in your automatic tests to prevent new leaks. That’s not the case here, and if we refer to it as a memory leak then it can only cause frustration on the part of users who are aware how simple it is to fix a memory leak.

This problem is different. It’s not a leak in the traditional sense. The memory does eventually get freed, but GNOME Shell holds onto it for too long; long enough to cause problems on some systems. As GJS contributor Andy Holmes put it, it’s a “tardy GC sweep.” I think that has a catchy ring to it, so I’ll call it the “tardy sweep problem” from now on.

Scruffy the janitor from Futurama, relaxing in a chair, captioned

Meme by Andy Holmes, used with kind permission

To be honest, I found that the OMG!Ubuntu article about “the memory leak” attracted a lot of comments that don’t sit well with me, and I think that the wrong expectations set by calling it a “memory leak” are partly to blame. With this post, I hope to give a better idea of what GNOME users can expect.

On the bright side, due to the recent publicity and especially the OMG!Ubuntu article, more GNOME developers are talking about the memory problems and suggesting things, which is causing an exciting confluence of ideas that I couldn’t have come up with on my own.

Edit: I want to be absolutely clear that with the above I’m not blaming bug reporters for not knowing whether something is a “proper” memory leak or not. This is intended to bring some attention to the wrong expectations that arise, especially among technically savvy users, when GNOME developers and the tech press use the term “memory leak,” and illustrate why we ourselves should not use the term here.

 

The Big Hammer

Hammer smashing a peanut

CC0 licensed image, by stevepb

I’ve been calling this patch the “Big Hammer” because it’s a drastic measure: starting a whole new garbage collection cycle in order to clean up some objects that we already know should be cleaned up.

The tardy sweep problem has now been mitigated with the Big Hammer, but reducing GNOME Shell’s memory usage has been a battle for years, and it has very little to do with memory leaks.

There are many other causes of high memory usage in GNOME Shell. Some are real memory leaks, that generally get fixed before too long. GNOME Shell developers have had their suspicions about NVidia drivers for years. Another cause is JS memory leaks in GNOME Shell. (Contrary to popular belief, it is possible to leak memory in pure Javascript code. Andy’s new heapgraph tool is useful when tracking these down, but throughout most of the life of GNOME Shell this tool didn’t exist.) There’s also memory fragmentation, which can look like a memory leak in a resource monitor.1 In addition, when diagnosing reports from users, configurations vary wildly. Memory usage simply differs from system to system. Finally, people have different configurations of Shell extensions, some of which leak memory as well.

The Sad Lifecycle of a GNOME Shell Memory Leak Bug Report

  1. User reports “I have a memory leak”
  2. Developer runs Valgrind, sifts through Valgrind trace, finds a small leak and fixes it
  3. Problem isn’t fixed
  4. Repeat 2 and 3 until no leaks shown in the Valgrind trace
  5. Problem still isn’t fixed
  6. User eagerly awaits each point release hoping for relief, and is disappointed each time
  7. Bug report gains popularity, accretes followers like a katamari, some of whom vent about unrelated bugs, hound the developer, or become abusive, until the original point of the bug report is lost
  8. Developer can’t do anything productive with the bug report at this point. They know there’s still a memory problem and it’s not a traditional leak, but the bug report is not helping them find it
  9. Users don’t accept that answer
  10. Developer closes the bug report and makes users even angrier. (Or, developer ignores the bug report until it fizzles out, and makes users sad.)

Here are some examples of long-running bug reports where you can see this dynamic in action. It’s quite sad to observe, because everybody involved is doing what makes perfect sense from their perspective (except for a few people behaving badly), yet the result is a mess.

I hope this illustrates why it’s important to assume that people are acting in good faith.

I know some people will argue that the developer mustn’t close the bug report until the bug is “fixed”, meaning that there is no more unnecessary memory usage. But in my opinion that’s just not a useful way to think of bug reports. GNOME Shell developers know (and so do I, from the GJS side) that GNOME Shell uses a lot of memory. I agree it’s nice to keep a bug report open so that users know that we’re aware of it and it’s on our to-do lists somewhere, but very soon the time we spend dealing with the noise on the bug report eclipses whatever benefit it might bring to the community.

I hope GitLab will improve things a bit here, since if you feel strongly about an issue in the bugtracker, you can upvote it or add an emoji reaction to it. This is a good way for users to show that an issue is important to them, and if enough people use it then it’s a good indicator for me to see which issues are prioritized highest by users and contributors.

5 Myths About GNOME Shell’s Memory Problems (Paraphrased)

“GNOME developers never cared about the tardy sweep problem until OMG!Ubuntu reported on it. They don’t care about users until someone makes them.”

Carlos Garnacho, of GNOME Shell fame, pointed out to me that in a more global perspective there has been a very active hunt for actual memory leaks across many of GNOME Shell’s dependencies for quite some time, and he has personally patched leaks in IBus, AccountsService, libgweather, gnome-desktop, and more.

It seems the tardy sweep problem has gotten worse in recent versions of GNOME, although it’s hard to measure between different systems with different configurations. I don’t know why it’s gotten worse.2

It seems to have been known for a long time, though: for history buffs, it was alluded to in a comment in commit ae34ec49, back in 2011. That knowledge was apparently lost when GJS was without a maintainer for a couple of years. To be honest, it has taken me well over a year to get familiar enough with the toggle-ref code (which integrates the JS garbage collector with GObject’s refcounting system), that I feel even remotely comfortable making or reviewing changes to it. I only fully realized the implications of the tardy sweep problem after talking to Georges and seeing his memory graph.

It seems that a previous GJS maintainer, Giovanni Campagna, was trying to mitigate the tardy sweep problem already five years ago, with a patch that allowed objects to escape the tardy sweep by opting out of the whole toggle-ref system in some cases. Unfortunately, as far as I can tell from my bug tracker archaeology, his patch went through a few reviews and the answer was always “Wow, this is really complicated, I need to study it some more in order to understand it.” Then it fell by the wayside when he stepped down from GJS maintainership.

I picked the patch up again late last year and fixed up most of the bit-rot. It still had a few problems with it. I never found the time to fix it up completely, which Georges kindly took over for me. I initially preferred Giovanni’s patch above the Big Hammer, but unfortunately for me, Georges proved that it wasn’t as effective as we thought it would be, only clearing up about 5% of the tardy sweep memory.3

“GNOME fixed the problem with a shoddy solution. This will decrease performance but they won’t notice because they all have top-of-the-line machines.”

I don’t call it the Big Hammer for nothing. We were concerned about performance regressions too, so that’s reasonable. However, as you can read in the bug tracker, we did actually do some testing on lower-end hardware before merging the Big Hammer, and it was not as bad as I expected. Carlos has been doing some measurements and found that garbage collection accounts for about 2‚Äď3% of the time that GNOME Shell occupies the CPU.

However, it’s exactly because we want to be cautious that the Big Hammer has only been committed to master, which will first be released in the unstable GNOME 3.29.2 snapshot. I don’t plan to release it on a stable branch until we’ve run it some more.

Ubuntu has already put the Big Hammer in their LTS version. That’s more of a risk than I would have recommended, but it’s not my decision to make, and I am grateful that we will be getting some testing through that avenue. Endless is also considering putting the Big Hammer in their stable version.

(And alas, I don’t have a top-of-the-line machine. Feel free to donate me one if that’s what’s required to make me conform to some stereotype of GNOME developers. ūüėá)

“The problem should be fixed now. GNOME Shell will run smooth from now on.”

No, it’s not. GNOME Shell still isn’t that great with memory.

Carlos is working on some merge requests which are approaching being ready to merge, which should make things a bit more memory-efficient. He’s also had some success with experiments trying to reduce memory fragmentation, taking better advantage of SpiderMonkey’s compacting garbage collector.

We are also bouncing around some ideas for making the Big Hammer into a smaller hammer. In particular, we’re trying to see if the extra garbage collections can be restricted to only the JS objects that represent GObjects, since those are the only objects that are affected by the tardy sweep problem. We’re also trying to see if there’s a way to return black-marked (reachable) objects to their original white-marked (eligible for collection) state when a GObject is toggled down in the middle of a garbage collection.

Another approach to investigate is to make better use of incremental garbage collection. SpiderMonkey offers this facility but we don’t use it yet. The idea is, instead of pausing and doing a big garbage collection, we do a slice of a few milliseconds whenever we have time. I don’t know yet whether this will have a large or small effect, or even render the Big Hammer unnecessary.

We’re also going to update to SpiderMonkey 60 in GNOME 3.30 which will hopefully bring in another year’s worth of Mozilla’s garbage collector research and optimization.

Finally, I’m gradually working on another unfinished merge request left over from Giovanni’s tenure as GJS maintainer, that should drastically increase the performance of GNOME Shell’s animations (though not necessarily help with memory.)

“GNOME has no business releasing any new versions until this problem is fixed.”

GNOME has a fixed release schedule, so they release new versions on the release dates, with whatever is aboard the train at that time. That’s not going to change.

“This version of GNOME is going into Ubuntu LTS! GNOME needs to work harder to fix this.”

Of course, I want whatever version of GNOME ships with any Linux distribution to be as good as possible. But as the upstream GJS maintainer, I have no say over what a downstream Linux distribution chooses to ship. The best way for a Linux distro to make sure their release is shipshape, is to contribute resources towards fixing whatever they consider a blocker.

That sounds a bit callous, as if I refuse to fix any bugs that Ubuntu wants fixed; that’s not what I mean at all. But my free time is limited. I’m paid for a part of my GJS maintainer work, but only for specific features. I can’t work to anyone’s external deadlines in my free time, because otherwise I’ll burn out and that’s not good for anyone with any interest in GJS either. Sometimes I have other priorities besides sitting at the computer; sometimes I do have time but no ideas about a particular problem; sometimes my brain isn’t up to fixing a difficult memory problem and I choose to work on something easier.4 Bugfixing work isn’t fungible.

I picked Ubuntu to illustrate this example, because contributing is exactly what the Ubuntu team has done; Ubuntu contributors fixed stability bugs in GJS, as well as GNOME Shell and Mutter, for GNOME 3.28. To say nothing of contributors from other downstreams, as well. That’s great and I’m looking forward to more of it! Some commenters seem to see downstreams fixing bugs as something that GNOME developers should be ashamed of, but I believe everyone is better off for it when that happens!

Acknowledgements

Thanks to Carlos Garnacho and Andy Holmes, who commented on a draft version of this blog post. Thanks in addition to Andy who coined the term “tardy sweep” and provided Scruffy as the mascot; Heartbleed has branding, why shouldn’t we? And of course, thanks to Georges who kicked off the whole research in the first place!


[1] and has often made people angry in bug reports when told it’s not a memory leak ‚Ü©

[2] I have a hunch, though. When I updated SpiderMonkey to version 38 in GNOME 3.24, we went from a conservative collector to an exact-rooted, moving one ‚ÄĒ see this Wikipedia article for definitions of those terms. It may be that the old garbage collector, though generally considered inferior, did actually mitigate the tardy sweeps a little, because I think back then it would have been possible for more objects to make their way into an ongoing sweep. It’s also possible that it was made worse earlier than that, by some adjustments in GNOME Shell that adjusted how often the garbage collector was called. ‚Ü©

[3] Technical explanation: Tweener, which is the animation framework used by GNOME Shell, renders many objects ineligible to opt out of the toggle-ref system. I would like to see Tweener replaced with Clutter implicit animations in GNOME Shell, which would make Giovanni’s patch much more effective, but that’s a big project. ‚Ü©

[4] Like writing a blog post about a difficult memory problem. Joke’s on me, that’s actually really hard ‚Ü©

Advertisements

Weekend Website Experiment

As you may know if you read this blog via Planet GNOME, the GNOME project is busy switching to GitLab for its code hosting and bug tracking. I like GitLab! It’s a large step up from Bugzilla, which was what GNOME used for the last 20 years. Compared to GitHub, GitLab is about equal, with a few nicer things and a few less nice things.

The one thing that I miss from Bugzilla is a dashboard showing the overall status of the bugs for your project. I thought it would not be too hard to use the GitLab API to do some simple queries and plop them on a web page. So, last weekend I gave it a try. The final result is here. Click the button to log into GitLab, and you’ll be redirected back to the page where you’ll get the results of the queries.

I’d like to write up what I did because I learned a new thing, and I think more writeups illustrating the trial and error of learning a new thing are always good.

My goals were to write something without being too meticulous, and write something that was not intended to scale. (Both are things that I normally disapprove of. It’s good to try the other side once in a while.) So, I was just going to mix the HTML, CSS, and JS all in one file. I used the base CSS and overall page structure from my personal website.

I decided to use GitHub Pages for hosting. My personal website is already hosted there. GitLab also offers GitLab Pages, which is very similar, but GNOME hasn’t enabled it yet on their GitLab instance. So, I created a fork of GNOME/gjs on GitHub, and created a gh-pages branch. Whatever you commit to that branch shows up as your project’s GitHub Pages site on the web.

The first thing I figured out is that you have to be authenticated to query issues in the API, even if the same information is publicly available on the web. That’s too bad! My little project suddenly went from “easy” to “figuring out something I’ve never done before.” But that’s also exciting.

First, I got the queries running on a local webpage, using a temporary personal GitLab access token. Each item looked kind of like this:

Number of crashers: 16
Number of bugs: 25
etc.

Next, I decided to tackle the authentication problem. I did some searches on variations of “gitlab authentication plugin”, “add gitlab authentication to webpage”, etc., to see if there was something ready-made I could drop in. No such luck. I did find NodeJS modules that I could have used, had I been writing the site using NodeJS. I weighed the unknown cost of implementing the authentication in plain old browser JS against the unknown cost of setting up tools that I was unfamiliar with in order to use the ready-made module (I wasn’t even sure what tools I would have to use ‚ÄĒ Webpack, I think?) and decided to keep looking.

I next searched for things like “gitlab oauth2 in browser”, “gitlab oauth2 example”, since I knew that the login used the OAuth2 protocol. Eventually I landed on this page and figured out that the magic words I wanted were “implicit flow” or “implicit grant”:

Implicit Flow – This flow is designed for user-agent only apps (e.g. single page web application running on GitLab Pages).

That sounded like exactly my use case, so I read further. You have to send the user to a certain page on the GitLab site, and send a redirect URL which the user will be sent back to with the authorization token in the URL hash. I managed to keep everything on the same webpage. In pseudocode, the flow looks like this:

if we have the authorization token:
    fetch the numbers with the API calls
else if there is a hash in the URL:
    token = get the token from the hash
    store the token
    fetch the numbers with the API calls
else:
    show a button that links to the authorization URL on GitLab

For storing the token so that you don’t have to log in every time, I used localStorage. I have no idea if that’s good practice or not, but from what I could read online it seems that it’s at least not bad practice. It’s quite easy to retrieve the token, but only if you have access to the browser where it is stored. I don’t think localStorage can leak out over the web, but with the recently discovered vulnerabilities who knows…

Last, I made it look nicer. I had a pretty good idea of what I wanted it to look like: I wanted the numbers to be large, in colored boxes with rounded corners and thick borders. I tried a few things with floating <div>s before giving up and using a CSS flex layout. This makes the page probably unviewable on older browsers, but I was seriously done with CSS positioning.

The code is here, or just “View Source” while you’re on the page.

What I would do differently

Writing HTML, CSS, and JS directly for the web is tedious and repetitive. I wish my younger self had used some sort of framework like Bootstrap to make my personal site. Failing that, I wish I had decided not to reuse components from my personal site to do this, and instead started with a fresh site using a framework. Bootstrap or Semantic UI are two that I know of, and maybe should have tried out. The code ended up being 263 lines of HTML, CSS, and JS, much of it just repeated items in order to do the boxes in different colors.

Reuse of this code

You may notice I did not release this code under an open source license. That’s because it’s probably full of bad practices, so I don’t want people to copy it. If you can convince me that it’s done right or tell me what I did wrong, then I’ll (fix it and) open-source it, and other GitLab maintainers might find it useful.

Geek tip: g_object_new and constructors

tl;dr Don’t put any code in your foo_label_new() function other than g_object_new(), and watch out with Vala.

From this GJS bug report I realized there’s a trap that GObject library writers can fall into,

Avoid code at your construction site.

Avoid code at your construction site.

that I don’t think is documented anywhere. So I’m writing a blog post about it. I hope readers from Planet GNOME can help figure out where it needs to be documented.

For an object (let’s call it FooLabel) that’s part of the public API of a library (let’s call it libfoo), creating the object via its foo_label_new() constructor function should be equivalent to creating it via g_object_new().

If foo_label_new() takes no arguments then it should literally be only this:

FooLabel *
foo_label_new(void)
{
    return g_object_new(FOO_TYPE_LABEL, NULL);
}

If it does take arguments, then they should correspond to construct properties, and they should get set in the g_object_new() call. (It’s customary to at least put all construct-only properties as arguments to the constructor function.) For example:

FooLabel *
foo_label_new(const char *text)
{
    return g_object_new(FOO_TYPE_LABEL,
        "text", text,
        NULL);
}

Do not put any other code in foo_label_new(). That is, don’t do this:

FooLabel *
foo_label_new(void)
{
    FooLabel *retval = g_object_new(FOO_TYPE_LABEL, NULL);
    retval->priv->some_variable = 5;  /* Don't do this! */
    return retval;
}

The reason for that is because callers of your library will expect to be able to create FooLabels using g_object_new() in many situations. This is done when creating a FooLabel in JS and Python, but also when creating one from a Glade file, and also in plain old C when you need to set construct properties. In all those situations, the private field some_variable will not get initialized to 5!

Instead, put the code in foo_label_init(). That way, it will be executed regardless of how the object is constructed. And if you need to write code in the constructor that depends on construct properties that have been set, use the constructed virtual function. There’s a code example here.

If you want more details about what function is called when, Allison Lortie has a really useful blog post.

This trap can be easy to fall into in Vala. Using a construct block is the right way to do it:

namespace Foo {
public class Label : GLib.Object {
    private int some_variable;

    construct {
        some_variable = 5;
    }
}
}

This is the wrong way to do it:

namespace Foo {
public class Label : GLib.Object {
    private int some_variable;

    public Label() {
        some_variable = 5;  // Don't do this!
    }
}
}

This is tricky because the wrong way seems like the most obvious way to me!

This has been a public service announcement for the GNOME community, but here’s where you come in! Please help figure out where this should be documented, and whether it’s possible to enforce it through automated tools.

For example, the Writing Bindable APIs page seems like a good place to warn about it, and I’ve already added it there. But this should probably go into Vala documentation in the appropriate place. I have no idea if this is a problem with Rust’s gobject_gen! macro, but if it is then it should be documented as well.

Documented pitfalls are better than undocumented pitfalls, but removing the pitfall altogether is better. Is there a way we can check this automatically?

Modern Javascript in GNOME – GUADEC 2017 talk

I gave a presentation at GUADEC 2017 this morning on modern Javascript in GNOME, the topic of the last few posts on this blog. As I promised during the talk, here are the slides. There is a beefy appendix after the questions slide, with details about all the new language features, that you are welcome to use as a reference.

Thanks to the GNOME Foundation for the travel sponsorship, to my employer Endless for paying for some of this work, and especially to Rob McQueen for the last-minute loan of a USB-C video adapter!

Official badge from the GUADEC website: "I'm going to The GNOME Conference GUADEC Manchester, United Kingdom"

Inventing GObject ES6 classes

Hello again! If you’re a GJS user, I’d like your opinion and ideas. After my last post where I talked about new features coming in GNOME 3.26 to GJS, GNOME’s Javascript engine, I’m happy to say that the patches are¬†nearly ready to be landed. We just need to figure out how to build SpiderMonkey 52 consistently¬†even though Mozilla hasn’t made an official standalone release of it yet.

A top view of a latte next to a notebook with a pen, with coffee beans strewed artfully around.

A better literal depiction of JAVA SCRIPT I could not ask for…¬†(Public domain image courtesy of Engin_Akyurt)

As I reported last time:

After that is done, I will refactor GJS’s class system (Lang.Class and GObject.Class). I believe this needs to be done before GNOME 3.26. That’s because [we will] gain ES6 classes, and I want to avoid the situation where we have two competing, and possibly incompatible, ways to write classes.

That’s what I’m busy doing now, in the run-up to GUADEC later this month, and I wanted to think out loud in this blog post, and give GJS users a chance to comment.

First of all, the legacy Lang.Class classes will continue to work. You will be able to write ES6 classes that inherit from legacy classes, so you can start using ES6 classes without refactoring all of your code at once.

That was the good news, now the bad

However, there is not an obvious way to carry over the ability to create GObject classes and interfaces from legacy classes to ES6 classes. The main problem is that Lang.Class and its subclasses formed a metaclass framework. This was used to carry out certain activities at the time the class object itself was constructed, such as registering with the GType system.

ES6 classes don’t have a syntax for that, so we’ll have to get a bit creative. My goals are to invent something (1) that’s concise and pleasant to use, and (2) that doesn’t get in the way when classes gain more features in future ES releases; that is, not too magical. (Lang.Class¬†is pretty magical, but then again, there wasn’t really an alternative at the time.)

Here is how the legacy classes worked, with illustrations of all the possible bells and whistles:

The metaclass magic in Lang.Class notices that the class extends GObject.Object, and redirects the construction of the class object to GObject.Class. There, the other magic properties such as Properties and Signals are processed and removed from the prototype, and it calls a C function to register the type with the GObject type system.

Without metaclasses, it’s not possible to automatically carry out magic like that at the time a class object is constructed. However, that is exactly the time when we need to register the type with GObject. So, you pretty much need to remember to call a function after the class declaration to do the registering.

The most straightforwardly translated (fictional) implementation might look something like this:

The fictional GObject.registerClass()¬†function would take the role of the metaclass’s constructor.

This is a step backwards in a few ways compared to the legacy classes, and very unsatisfying. ES6 classes don’t yet have syntax for fields, only properties with getters, and the resulting static get syntax is quite unwieldy. Having to call the fictional registerClass() function separately from the class is unpleasant, because you can easily forget it.

On the other hand, if we had decorators in the language we’d be able to make something much more satisfying. If you’re familiar with Python’s decorators, these are much the same thing: the decorator is a function which takes the object that it decorates as input, performs some action on the object, and returns it. There is a proposed decorator syntax for Javascript that allows you to decorate classes and class properties. This would be an example, with some more fictional API:

This is a lot more concise and natural, and the property decorators are similar to the equivalent in PyGObject, but unfortunately it doesn’t exist. Decorators are still only a proposal, and none of the major browser engines implement them yet. Nonetheless, we can take the above syntax as an inspiration,¬†use a class expression, and move the registerClass()¬†function around it and the GObject stuff outside of it:

Here, the body of the class is almost identical to what it would be with the decorator syntax. All the extra stuff for GObject is contained at the top of the class like it would be with the decorators. We don’t have the elegance of the property decorator, but this is quite an improvement on the first iteration. It’s not overly magical, it even acts like a decorator: it takes a class expression, and gives back a GObject-ized class. And when decorators eventually make it into standard Javascript, the basic idea is the same, so converting your code will be easy enough. (Or those who use transpiling tools can already go ahead and implement the decorator-based API.)

This is the best API I’ve been able to come up with so far. What do you think? Would you want to use it? Reply to this post or come talk to me in #javascript on GNOME IRC.

Next steps

Note first of all that none of this code exists yet. Depending on what feedback I get here, I hope to have a draft version working before GUADEC, and around the same time I’ll post a more detailed proposal to the javascript-list mailing list.

In addition, I will be speaking about this and more at GUADEC in my talk, “Modern Javascript in GNOME“. If you are attending, come talk to me there! Thanks to the GNOME Foundation for sponsoring my travel and accommodations.

Official badge from the GUADEC website: "I'm going to The GNOME Conference GUADEC Manchester, United Kingdom"

The GJS documentation is back

Aside

We have once again a set of accurate, up-to-date documentation for GJS. Find it at devdocs.baznga.org!

Many thanks are due to Everaldo Canuto, Patrick Griffis, and Dustin Falgout for helping get the website back online, and to Nuveo for sponsoring the hosting.

In addition, thanks to Patrick’s lead, we have a docker image if you want to run the documentation site yourself.

If you find any inaccuracies in the documentation, please report a bug at this issue tracker.

GJS: What’s next?

In my last post, I went into detail about all the new stuff that GJS brought to GNOME 3.24. Now, it’s time to talk about the near future: what GJS will bring to GNOME 3.26.

Javascript engine

The highest priority is to keep upgrading the Javascript engine. At the time of writing, I’ve got SpiderMonkey 45 almost, but not quite, working, and Mozilla is on the verge of releasing the standalone version of SpiderMonkey 52. If we can get there, then we’ll finally be on a supported release which means we can have a closer collaboration with the Mozilla team. (During the past six months, they’ve been patient with me asking questions about old, unsupported releases, but it’s not fair to ask them to continue doing that.)

I plan to upgrade to 45 but not merge it, and then immediately continue upgrading to 52 on the same branch, then merge it all in at the same time. That way, we won’t have an interregnum where everyone has to build SpiderMonkey 45 in JHBuild and Continuous and the Flatpak SDK. Subscribe to bug 781429 and its offshoots if you want to follow along.

The main language features that this will bring in are: classes (45) and ES7 async/await statements (52). At that point, the only major ES6 feature that we will still be missing is modules.

ES6 Classes

After that is done, I will refactor GJS’s class system (Lang.Class and GObject.Class). I believe this needs to be done before GNOME 3.26. That’s because in SpiderMonkey 45, we gain ES6 classes, and I want to avoid the situation where we have two competing, and possibly incompatible, ways to write classes.

ES6 Modules

Full ES6 module support is still missing in SpiderMonkey 52, but at least some parts of it are implemented. I’ll need to investigate if it’s possible to enable them in GJS already. Although, we will definitely not enable them yet if there’s no way to keep the existing modules working; we don’t want to break everyone’s code.

Developer tools

Next comes a debugger. There are not one, not two, but three existing implementations of a GJS debugger sitting unattended in Bugzilla or a Git branch. None of them will apply to the codebase as is, so my task will be to fix them up, evaluate the merits of each one, and hopefully come up with one patchset to rule them all.

Christian Hergert is planning to add a profiler, so that you can profile your Javascript code with Sysprof, inside Builder.

Documentation

I would very much like to get the GJS documentation browser back online. I hosted it on EC2, but I have run out of free hosting. If you have a server where it can be parked, let me know! (It’s a web app, not static pages, so I can’t just put it on GitHub Pages.) If you want to run the web app locally yourself, you can find instructions here for how I set it up on EC2, on a RHEL 7 box.

Misc.

All that is probably more than I’ll have time for, but here are some of the things that I’d like to get done after that:

  • Update the tutorials on developer.gnome.org to use more modern GJS
  • Better integration with Builder
  • Use structured logging to clean up the “debug topics” mechanism
  • Reduce the list of unreviewed patches in Bugzilla down to 0
  • Find ways to bring in some of the conveniences that Node developers are used to

Chun-wei Fan is working on converting some of the codebase to use C++ smart pointers so that we get the memory leak safety advantages of g_autoptr without losing portability to MSVC.

Build system

The question is inevitable: are we going to switch the build system to Meson? I’m looking forward to it, but no, not until Meson is more mature and some of the open questions about distribution and autobuilding have been answered.

Help!

I think it’s great that once I started contributing, other people soon started contributing too. The 1.48.0 release had way more patches and contributors than 1.46.0, even if you don’t count all the stale patches that I souped up. GNOME’s #javascript IRC channel is starting to be a lively place, compared to how deserted it was last year.

What I’d most like to encourage is for more people to contribute major features so that the above list doesn’t read like¬†a to-do list that’s mostly for me. I’m happy to provide guidance. I think it would be great for GJS to become a more competitive development language for apps using the GNOME technology stack1 and we won’t get there with just me.

Another way you can help is by using the development version of GJS while developing your apps or GNOME Shell, thereby helping to try out the new features. We had some serious bugs up to, or even past, the last minute in GNOME 3.24, and this seems like the best way to prevent that.

Finally, you can help by sharing your experiences with GJS: good and bad. Talk¬†on the mailing list¬†or IRC, or file a bug on bugzilla.gnome.org¬†if there’s something wrong.


[1] In that regard I’d love to prove wrong Michael Catanzaro’s opinion about using GJS: “there‚Äôs no way to change the reality that JavaScript is a terrible language. It has close to zero redeeming features, and many confusing ones.” There is a way! In my opinion ES6 and ES7 have gone a long way towards filling in those potholes. To name just a few, arrow functions mean you can almost always stop caring about the pitfall of what this¬†refers to, and the prospect of doing asynchronous I/O with Promises instead of callbacks actually makes me want to use JS. Of course, in-browser JS is still a terrible language because it has to support the lowest common denominator of Javascripts so that people who haven’t upgraded their browser since Internet Explorer 8 can still visit your website, and that’s why modern web developers preprocess and transpile it to high heaven. But we don’t have to care about all those browser users! ‚Ü©