Everything about cargo_attach

#cargo_attach is a parser function that helps Cargo know when to re-parse a page when you recreate a table. Most wikis using Cargo will never need to use #cargo_attach because #cargo_declare includes an inherent attachment, but if you ever want to store to a table from a template that doesn’t do its own declaration, you will need a #cargo_attach.

For ease of reading, from now on I will talk about #attach and #declare but please keep in mind these are abbreviations; the actual parser functions you will need to use are #cargo_attach and #cargo_declare.

Our example

Our example setup starts like this:

We have a template called Template:Infobox Item which declares a table called Items
We also have a template called Template:Infobox Special Item that needs to store to the same Items table
Therefore, Infobox Special Item will need to attach the table Items

When is attach useful?

#attach is useful when the following two conditions are met:

You have a template storing to a table it did not declare (in this case Infobox Special Item storing to Items)
You need to recreate the table

If you are in this situation and don’t have the #attach set up properly, then the following will happen:

You will need to run blank edits over a portion of your wiki to repopulate the data in the table

In the vast majority of cases this is not a big deal and getting your #attach right is not that important. So if you feel overwhelmed, don’t worry about it! But when recreating a table, if something isn’t populating, you should ask whether your attaches are set up properly.

How to use attach

In the example we’ve set up above, we would need one single #attach call and it would be in Template:Infobox Special Item. The template will look like this:

<includeonly><!--

some infobox code goes here

-->{{#if:{{NAMESPACE}}||{{#cargo_store:_table=Items
<!-- the fields go here -->
}}<!-- /if -->}}</includeonly><noinclude>{{#cargo_attach:_table=Items}}</noinclude>

If you don’t have any documentation etc and you save this page, it’ll look like this:

But why?

When you recreate a table, Cargo does the following (in order):

Create a new, empty table with the current schema as specified by the declaring template (possibly a replacement table)
Discover what templates either declare or attach that table
Find all of the backlinks (“WhatLinksHere” report) of each of those templates
Reparse those pages and write the rows to a brand new table

Specifically, the #attach lets Cargo automatically rediscover your page during recreate.

Most people can stop reading now. But if you have a large number of records (think 50,000+) or ever need to store to multiple tables from the same template, you should continue reading.

Interaction with conditionals

Sometimes, a template might write to a table only in certain circumstances. For example, you may want to store only in the main namespace and otherwise do nothing.

It’s worth noticing that Cargo will still attempt to store data from sandbox pages during the recreate as described above. This will rarely matter, but for wikis with hundreds of thousands of records where half the time the store does not happen, or for wikis with very slow job queues, you might want to optimize this a bit.

I cannot stress enough how this is a micro-optimization in 99% of cases. You probably don’t need to care about the rest of this section, but I’m including it for completeness (and also because it helps to introduce later ideas).

Say your template looks like this and is used 200,000 times in your wiki:

<includeonly><!--

some infobox code goes here

-->{{#if:{{{is_special|}}}|<!-- half the time, is_special will NOT be defined -->
|{{#cargo_store:_table=Items
<!-- the fields go here -->
}}<!-- /if -->}}</includeonly><noinclude>{{#cargo_attach:_table=Items}}</noinclude>

So during each recreate, we have 100,000 pages being skipped over. Especially in wikis with slow job queue performance, this could result in several minutes or even hours wasted on waiting for the job queue to get past these “empty” pages.

Dedicated attaching templates

The way to remove that extra waiting is to use a dedicated attaching table. First, we’ll create a new template called Template:Items/attach. That template will look like this:

<noinclude>{{#cargo_attach:_table=Items}}</noinclude>

Every page that transcludes Template:Items/attach will then be added to the recreate queue, but Template:Items/attach does not output anything or have any other side effects. All it does is sit there as a marker for Cargo to include the page it’s used on in the recreate queue.

So, we can rewrite Template:Infobox Special Item like this:

<includeonly><!--

some infobox code goes here

-->{{#if:{{{is_special|}}}|<!-- half the time, is_special will NOT be defined -->
|{{#cargo_store:_table=Items
<!-- the fields go here -->
}}{{Items/attach}}<!-- /if -->}}</includeonly>

Notice that Template:Items/attach is placed inside the #if statement, so it’s only transcluded if is_special is defined. Now the pages are added to the recreate queue only when necessary!

There will be a bit more discussion about this scenario later and tradeoffs you might make, but first let’s move onto the “Cargo attach trick.”

The cargo_attach trick

Let’s modify our example above. We’re now going to have:

A template called Template:Infobox Item which declares a table called Items
Another table called ItemEffects that’s stored to from Infobox Item
Because a template can only declare one table, ItemEffects is declared from Template:ItemEffects/CargoDeclare

We will now need to employ the “Cargo attach trick”:

Make a template called Template:ItemEffects/attach
This template will attach the ItemEffects table inside the noinclude, and do nothing else
Template:Infobox Item will make a call to Template:ItemEffects/attach, so that the attaching template is transcluded to content pages as needed and everything gets tagged appropriately for the recreate process.

Here’s how that looks in code at Template:ItemEffects/attach:

<noinclude>{{#cargo_attach:_table=ItemEffects}}</noinclude>

If you read the previous section, this is pretty much the same as how Template:Items/attach worked there. Every attachment template looks almost identical to this, just change out the table name.

And at Template:Infobox Item:

<includeonly><!--

some infobox code goes here

-->{{#if:{{NAMESPACE}}||{{#cargo_store:_table=Items
<!-- the fields go here -->
}}{{#cargo_store:_table=ItemEffects

}}{{ItemEffects/attach}}<!-- /if -->}}</includeonly><noinclude>{{#cargo_attach:_table=Items}}</noinclude>

So we have:

The actual infobox code that displays things (which I’ve omitted)
If we are not in a sandbox/template namespace then:
1. A store to Items
2. A store to ItemEffects
3. A transclusion of the ItemEffects/attach template, so that the content page gets tagged by Cargo for reparse when we recreate

And that’s it! That’s the Cargo attach trick. You can include as many /attach templates as you’d like in an infobox.

How to do this from Lua

If you need to do the attach trick from Lua, do mw.getCurrentFrame():expandTemplate on your attaching template. You do not need to return this template, just make sure you’re expanding it somewhere in your module code. You can read more about frame objects if you want.

Keep in mind that the nature of the attach trick does NOT allow you to call the #attach parser function within your Lua code. You MUST include the template that does the attach.

Cargo also does not (currently) allow you to attach via a Module documentation page, although it would be cool if it did. Maybe a future patch will allow that, but attaching from Template namespace is pretty deeply entrenched in the codebase.

Yay!

The final section will be a very niche topic that most people can skip. Congrats, you made it through all the important stuff!

A bit more on conditionals

Above, we talked about improving performance by putting an attachment inside of a conditional. But that’s not always the best idea. When deciding whether your attachment template should go inside or outside of a conditional, you should ask yourself the following question: If I need to recreate, could the result of the conditional change?

First, why do you recreate a table? Generally three reasons, but the first two are the ones we’ll consider:

Schema change
You are storing data that was pulled from somewhere else and just did a major data update (I really hope you don’t do this ever, please script your page updates instead if you must use Cargo)
You accidentally populated a mistake throughout the entire table and need to fix it
- Possibly where all your records were deleted
  - This situation is slightly impossible to guard against entirely, and you may want to assume blank edits will be needed if it happens

Now, why might we have a conditional that determines whether we store?

Dependent on where the page is on wiki (if NAMESPACE etc)
Different types of your entity get different additional data
Storing inside an arraymap and there could be 0, 1, or many items (your life will be easier if you use arraymaptemplate here and not worry about the attach trick)
- Although if you are using a lot of Lua, it might be harder to have separate templates

So, if you recreate your table because of a schema change or because of a data source change, how likely is it that a page that previously was storing 0 rows, should now be storing 1 or more rows?

Keep in mind that this is still specifically about recreating. Inputting new values and then saving the page triggers a new store no matter what’s going on with your attaches, so that operation doesn’t matter here. We’re specifically talking about recreating the table without edits.

And again keep in mind that this is usually an extremely minor performance question. If you have under 500 pages total using this template, just put the attach outside the conditional, no reason not to.

But let’s jump up a couple orders of magnitude and say you have 500,000 pages using this template. Then we need to think about what might change for a rebuild. Is it possible that a page that previously had {{#var:is_special}} set to false now has it set to true due to a change in a Lua module, so now we need to store to the SpecialItems table? Is it possible that an arraymap that previously had 0 iterations now has 5? If the answer is always no or extremely unlikely then you should probably put the attachment template inside the conditional. But if these types of changes happen a lot, then consider keeping the attachment template outside the conditional and waiting a bit more.

(And again, please just use arraymaptemplate to store and never arraymap, that’ll make your life a lot easier.)

Conclusion

#cargo_attach helps the wiki know how to rebuild tables automatically without blank edits needed. If you mess up your attaches, just issue blank edits! It’s not the end of the world! But it’s a good idea to try to get it right, and if your wiki has hundreds of thousands of rows of data (or a really slow job queue), you should probably put some effort into doing the attaches efficiently.