Last weekend, I wrote my third (2.5th?) MediaWiki extension, Lua Branches. The goal of the extension is to allow you to add “branches” (alternate versions) to your Lua modules, and then easily access them when using invokes during sandboxing.
Why is this useful?
Imagine you have a module that has 5 imports (dependent modules called via
require). Each of these imports has another 2 imports, and so forth. There may be some overlap, but you have a pretty large DAG (directed acyclic graph) of files that your parent module is dependent on. Say you want to modify a leaf module. You have a couple choices:
- Just make the change and hope it works. If it does, great! If not then yikes you just broke stuff in production, also you caused a bunch of jobs, and each time you try to fix it, you cause more jobs, etc.
- Have an entire clone of your wiki (a “development” or “staging” environment) and sandbox changes there. Maybe your code namespaces are mirrored, but you don’t keep all of the content up-to-date, so the effects of the jobqueue are minimal, and if you break stuff it’s not the end of the world. For huge changes this can be a good idea, but for minor everyday fixes this is quite a bit of effort to maintain, especially since importing the right amount of data but not too much is quite possibly a very difficult proposition.
/Sandboxversions of all of your modules chaining up to the parent module, and make each one import the
/Sandboxchild appropriately. Then invoke the
/Sandboxparent module in a sandbox user page. If two people are using two different sandboxes at the same time, do this twice! When you’re done, remember to clean up after yourself! If you had to change code in multiple levels of children & parents, remember to undo the
/Sandboxpart of the requires at every stage when you’re copying your code back to live! (Maybe you can do some kind of abuse filter for this, like
/Sandboxcannot appear inside of a require if
/Sandboxisn’t in the title of a page.)
But what if we could make a
/Sandbox branch without all the hassle, and effortlessly switch between live & sandbox branches of a module? This is the goal of Lua Branches: you can specify a branch via the parameter
_branch= in the
#invoke parser function, and it will automatically use the branched modules when they exist, falling back to the default/master/main branch when they do not.
Current state of the extension
As is, Lua Branches cannot be used unless you slightly fork Scribunto. I had to add two hooks as well as give the engine constructor the ability to send a branch name from the
#invoke parser function. Overall, it’s a pretty minimal set of changes (although I couldn’t swear that this is the precise set of arguments to the hooks that would be wanted if this were to be merged into the actual live Scribunto extension).
I would love to see a patch like this merged into Scribunto so that Lua Branches could be realized! But as-is, it’s more of a proof of concept than anything else, unless you want to semi-fork Scribunto locally.
There’s one config variable,
$wgLuaBranchesBranchPattern. The default value is
%name%/%branch% but for example you might want to do something like
zzzz/%name%/%branch% so that all of your branches appear at the very end of
Special:AllPages. Or if you’re not into subpages you could do
If you’re having users name branches after themselves, you also might want to do something like
Zzz_Sandbox/%branch%/%name%), inverting the categorization.
Let’s put the following at
Module:KittenPrinter/my-branch (I’m using the default branch config):
Module:HasNoBranch, I just have the following:
As its name implies, there is no branch for this module (i.e.
Module:HasNoBranch/my-branch does not exist); this is to ensure that the extension doesn’t crash when importing a requirement without a branch.
Now, we have two tests:
Okay, not quite what we wanted. But because the caching layer for Scribunto imports is actually done inside the Lua engine, and not in PHP, this is the best we can do. And, since this extension is just for prototyping, this is kinda okay - you will be making one sandbox page per test, anyway, right?
As of publication of this article, the extension actually doesn’t look for a branch of the module that you are invoking, only its imports. I made a ticket for this, but it seems less important and probably requires more edits to Scribunto, I’m not sure.
If you read the repo’s readme, you’ll also see a section about limitations - for example, I have to do an existence check per import to determine whether the branch exists or not. It would be nice to be able to batch these database calls, but that would probably require a much bigger incursion onto what Scribunto does (and possibly even editing the Lua engine, I’m not sure). And for what’s again just for prototyping, I don’t think this is that big of a performance concern.
I also have not super deeply tested this extension, since it’s more of a proof of concept than anything else, but I’m mostly sure it works. If you are willing to make the needed changes to Scribunto in your local MediaWiki installation, you should be good to use Lua Branches. I’d love to hear about your experience with it (and feel free to give it a star on github if you like it)!