Last weekend, I wrote my third (2.5th?) MediaWiki extension, Lua Branches. The goal of the extension is to allow you to add “branches” (alternate versions) to your Lua modules, and then easily access them when using invokes during sandboxing.

Why is this useful?

Imagine you have a module that has 5 imports (dependent modules called via require). Each of these imports has another 2 imports, and so forth. There may be some overlap, but you have a pretty large DAG (directed acyclic graph) of files that your parent module is dependent on. Say you want to modify a leaf module. You have a couple choices:

1. Just make the change and hope it works. If it does, great! If not then yikes you just broke stuff in production, also you caused a bunch of jobs, and each time you try to fix it, you cause more jobs, etc.
2. Have an entire clone of your wiki (a “development” or “staging” environment) and sandbox changes there. Maybe your code namespaces are mirrored, but you don’t keep all of the content up-to-date, so the effects of the jobqueue are minimal, and if you break stuff it’s not the end of the world. For huge changes this can be a good idea, but for minor everyday fixes this is quite a bit of effort to maintain, especially since importing the right amount of data but not too much is quite possibly a very difficult proposition.
3. Create /Sandbox versions of all of your modules chaining up to the parent module, and make each one import the /Sandbox child appropriately. Then invoke the /Sandbox parent module in a sandbox user page. If two people are using two different sandboxes at the same time, do this twice! When you’re done, remember to clean up after yourself! If you had to change code in multiple levels of children & parents, remember to undo the /Sandbox part of the requires at every stage when you’re copying your code back to live! (Maybe you can do some kind of abuse filter for this, like /Sandbox cannot appear inside of a require if /Sandbox isn’t in the title of a page.)

But what if we could make a /Sandbox branch without all the hassle, and effortlessly switch between live & sandbox branches of a module? This is the goal of Lua Branches: you can specify a branch via the parameter _branch= in the #invoke parser function, and it will automatically use the branched modules when they exist, falling back to the default/master/main branch when they do not.

Current state of the extension

As is, Lua Branches cannot be used unless you slightly fork Scribunto. I had to add two hooks as well as give the engine constructor the ability to send a branch name from the #invoke parser function. Overall, it’s a pretty minimal set of changes (although I couldn’t swear that this is the precise set of arguments to the hooks that would be wanted if this were to be merged into the actual live Scribunto extension).

I would love to see a patch like this merged into Scribunto so that Lua Branches could be realized! But as-is, it’s more of a proof of concept than anything else, unless you want to semi-fork Scribunto locally.

Config

There’s one config variable, $wgLuaBranchesBranchPattern. The default value is %name%/%branch% but for example you might want to do something like zzzz/%name%/%branch% so that all of your branches appear at the very end of Special:AllPages. Or if you’re not into subpages you could do %name%-%branch%.

If you’re having users name branches after themselves, you also might want to do something like %branch%/%name% or Sandbox/%branch%/%name% (or Zzz_Sandbox/%branch%/%name%), inverting the categorization.

Usage example

Let’s put the following at Module:Kittens:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

local printer = require('Module:KittenPrinter')
local anotherModule = require('Module:HasNoBranch')

local p = {}

function p.main()
     return printer.print()
end

return p

At Module:KittenPrinter:

1
2
3
4
5
6
7

local p = {}

function p.print()
    return "Kittens are very cute and this is master branch"
end

return p

And at Module:KittenPrinter/my-branch (I’m using the default branch config):

1
2
3
4
5
6
7

local p = {}

function p.print()
    return "Kittens are very cute and this is another branch"
end

return p

At Module:HasNoBranch, I just have the following:

1
2
3

local p = {}

return p

As its name implies, there is no branch for this module (i.e. Module:HasNoBranch/my-branch does not exist); this is to ensure that the extension doesn’t crash when importing a requirement without a branch.

Now, we have two tests:

Okay, not quite what we wanted. But because the caching layer for Scribunto imports is actually done inside the Lua engine, and not in PHP, this is the best we can do. And, since this extension is just for prototyping, this is kinda okay - you will be making one sandbox page per test, anyway, right?

Further development

As of publication of this article, the extension actually doesn’t look for a branch of the module that you are invoking, only its imports. I made a ticket for this, but it seems less important and probably requires more edits to Scribunto, I’m not sure.

If you read the repo’s readme, you’ll also see a section about limitations - for example, I have to do an existence check per import to determine whether the branch exists or not. It would be nice to be able to batch these database calls, but that would probably require a much bigger incursion onto what Scribunto does (and possibly even editing the Lua engine, I’m not sure). And for what’s again just for prototyping, I don’t think this is that big of a performance concern.

I also have not super deeply tested this extension, since it’s more of a proof of concept than anything else, but I’m mostly sure it works. If you are willing to make the needed changes to Scribunto in your local MediaWiki installation, you should be good to use Lua Branches. I’d love to hear about your experience with it (and feel free to give it a star on github if you like it)!