XML Preprocessing proposals/ideas

[Pablete1234]

1 Introduction

What is xml preprocessing? what is it useful for?

XML preprocessing means that the map xml you've made won't be passed as-is to pgm modules, instead, it will be pre-processed by pgm, adding some sections, removing others, or changing some values. This is useful for maps which may play differently on some occasions or different versions. For example the tournament edition of a map may have longer respawn time, or the christmas variant may use a different map and maybe moves spawn locations.

2 Current state of xml preprocessing

What current pgm xml preprocessing is supported as of now (24cf489):

• Variants: A map XML may define a set of variants for the map, eg: A christmas variant, or a tournament-edition variant.
• Conditionals & includes: A map XML may use if and unless to conditionally (depending on the variant) use or not use certain sections, and includes to "copy-paste in" a different xml.
• Constants: A map XML can define some constant values, which are then "search-replaced" on the rest of the xml

Example:

<map>
  <name>Base Name</name>
  <variant id="tm">TE</variant>
  <variant id="christmas" world="christmas" override="true">Christmas Name</variant>
  <constants>
    <constant id="team-size">32</constant>
  </constants>

  <if variant="tm">
    <constant id="team-size">8</constant>
    <respawn auto="true"/>
    <include id="void-death"/>
  </if>

  <team id="red" max="${team-size}">Red</team>
  <team id="blue" max="${team-size}">Blue</team>
</map>

This would "generate" 3 maps in pgm: Base Name (32v32), Base Name: TE (8v8, and auto-respawn, and instant death when falling to the void) and Christmas Name (32v32), the last one using a different world folder, named christmas.

2.1 How are all these applied?

1. Read the XML, assume variant is default if we're not reading a specific variant.
2. Replace conditionals (if and unless) and includes
   • if and unless are either replaced with their inner content, or removed
   • includes are replaced with their inner content
   • New content added by this step, also goes thru this step, in other words, you can use includes inside ifs and vice-versa.
3. Search for all <constant> elements defined, and build the map of key->constant.
   • constant tags can be wrapped around none or multiple parent constants elements, similar to most common xml features.
   • constants defined after override constants defined earlier. This means to override default values inside an include you should define the include above your constants within your map xml.
4. Perform post-process step, replacing any texts with ${whatever} with their respective constant
5. Search all variant elements defined. For each variant, do steps 1 to 4 and register as a diff map.

"Consequences" of this system are that constants can easily be used anywhere (even if defined at the bottom, can be used at the top), and conditional constants are pretty trivial to make, just define a default above, then an if with overrides for them, however it also means that using constants as if-conditionals isn't viable, and that's a relevant ask for includes (an include that can, depending on some override-able constant, include a certain section or not).
Also note that variants are read from the default-variant XML, so if you were to put variants inside a conditional that doesn't get included in the default variant, they wouldn't be registered.

3 Missing features and proposals

The current state is not bad, it is very much usable and solves a lot of the pain for mapmakers having to maintain completely different versions of maps just because of a few different details between one edition and another, however, there's still a ton of room for improvement:

3.1 Support different worlds (eg: christmas or halloween editions)

This has been solved in #1204, using solution 2. Expand to see original text

This is the main missing feature right now, we have no doubt about if this should be added, the only question is how, there's three potential solutions:

1. <if variant="x"><terrain world="christmas"/></if> terrain module wrapped in a conditional, syntax that already existed in 1.9+ pgm
   • Pros: backwards-compatibility with 1.9+ PGM maps, continues PGM's modular approach
   • Cons: longer to write, 3 xml lines for <if><terrain world="a"></if>
2. <variant id="christmas" world="christmas">..., allow the world to be defined directly in the variant
   • Pros: simple, lets you just with one attribute say the world you want to use
   • Cons: breaks backwards-compat, and it's not too modular. If a map wants a diff world (odd?) it requires a variant, unless we allow the default variant to be defined (eg: variant id="default" world="x").
3. Allow both terrain and variant to define a world
   • Pros: maximum flexibility
   • Cons: added complexity both code-wise (needing to search in 2 diff places), and mapmaker/user complexity, as they may see it done in two different ways and are unsure as to which is the correct, or may be confused about it.

Opinion: leaning towards option 2, as the xml would require having been changed regardless from legacy if syntax to current, and option 3 is overcomplicated.

3.2 Alternative types of conditionals

It has been requested to be able to make conditionals based on other factors, like constants. Making them specifically based on constants is tricky (see section 2.1 for more insight).

To my understanding, the main problems aimed to solve here are:

• Being able to conditionally enable/disable behavior that usually lives within an include.
• Gaining the ability to have server-family-like behavior, so the server may have a family: ranked in the config, and maps can tweak functionality based on it.
• Being able to "group-up" behaviors of variants, eg: maybe TM version and Ranked version share traits, instead of needing to always use if variant="tm, ranked you could use if type="ranked" where the map previously defines both tm and ranked to be of "type ranked".

This is the main hot-topic as it is what can add the most flexibility, and also the most complexity to the whole thing.

3.2.1 Potential solution: allow constants in conditionals

If we start enabling the use of xml to define things that conditionals can use, it means order of applying can become weird, see:

<map>
  <name>Map</name>
  <description>This map is a ${type} map</description>
  <variant id="ranked">Ranked</variant>
  
  <if constant="type" constant-value="ranked">
    <team id="red" max="8"/>
    <team id="blue" max="8"/>
  </if>

  <constant id="type">public</constant>
  <if variant="ranked">
    <constant id="type">ranked</constant>
  </if>

  <if constant="type" constant-value="ranked">
    <respawn auto="true"/>
  </if>
</map>

Notice that ${type} currently can be used anywhere, even if the constant hasn't been defined yet, because they occur at different steps (see section 2.1). If we were to allow constants in conditionals, that would have to change, you'd need constants handled at a different step.

• If you process the constants before conditionals, the example would not enable auto-respawn, as <constant id="type">ranked</constant> wouldn't have been read as a constant (ifs not passed yet!)
• If you process constants at the same time as conditionals, then the teams wouldn't be processed, as the constant is only defined later!
  • See implementation of this in Implement conditional constants #1373
• You can process ifs that depend only on variants first, then constants, then ifs that depend on constants, but that adds its own whole layer of complexity to understanding weird behaviors and edge-cases, especially to the mapdevs that aren't well versed in the system.

Opinion: unsure on this whole thing, we need to find a good solution and this just seems like a patch.

3.2.2 Potential solution: implement "tags" (needs a better name) as a separate type of constants

This idea shares most of what is said above, but instead of re-using constants as what to use in conditionals, it simply adds a new type that would be in charge of that. One name for it could be "tags", and the xml could define certain tags then use them for if-checking, and default "tags" could also be defined in the pgm yml config. This idea doesn't deviate too much from the above, it mostly adds clarity as to how constants and tags would be different, handled differently, and processed at different times (they'd be processed at the same time as conditionals/includes, and back-references would not be supported.)

Opinion: unsure on this whole thing, we need to find a good solution and this just seems like a patch.

3.2.3 Potential solution: Parameters for includes

Map XML:

<map>
  <name>Map</name>
  <include id="some-include" do-something="false" mode="mercy"/>
</map>

Include XML:

<map do-something="true" mode="limit" max="5">
  <if do-something="true">
    ...
  </if>
  
  <score>
    <if mode="mercy">
      <mercy>${max}</mercy>
    </if>
    <if mode="mercy">
      <limit>${max}</limit>
    </if>
  </score>
</map>

This has some big benefits in terms of complexity: it avoids the whole issue of 3.2.1 as parameters are specifically defined when passed to includes, and there is no way to overwrite them later on in other places of the xml, or have potential back-references etc. Additionally, it also means you could potentially use the same import multiple times with different parameters, which could have other uses (eg: import once per team a set of definitions that you know are repeated for each team).
The main drawback is it only enables extra conditionals inside includes, doing nothing to help the standalone xml. That however, may be enough.

Opinion: Leaning towards. I think it adds quite a bit of flexibility and the technical-side shouldn't be too hard to figure out.

3.3 Alternative ways of handling variants

At the moment all maps have an "internal" default (the map without any variant applied), which takes on the main "Map Name".
You're not allowed to define a variant with id default, but maybe we should allow that, and change our approach to the "Map Name" being "one of the variants", which, depending on the pgm config.

Let's see an example:

<map>
  <name>Map Name</name>
  <variant id="default">Public</variant>
  <variant id="ranked">Ranked</variant>
  <variant id="halloween" override="true">Spooky-Map Name</variant>
</map>

Currently, this would be invalid xml (default isn't allowed), but if we were to allow it, this would create 2 maps: Map Name: Public, Map Name: Ranked and Spooky-Map name, those would be the three actual maps that exist here.

Additionally, Map Name would be a "proxy" or "fake map" that redirects to one of the other maps. Usually to the default, but we could allow PGM config to define a different default. Doing /map Map Name would show a map named Map Name but what you're actually seeing is the "contents" of Map Name: Public, or if the config is different it could be the contents of Map Name: Ranked.

Such structure would allow for maps to freely have their ranked variant without having to migrate stuff (eg: map names in pools) for many, many maps over to a separate version in specific servers.

It could however also lead to some user-side complexity as they could no longer know what map they're really playing, and especially with variants that override the name it could become weird. If during Halloween the default variant is set to that, then cycling to Map Name would lead you to what you would expect should be called Spook-Map Name, however because of what we've explained above, this version would be "proxied" by the plain map name. Maybe we could consider override names to also override the proxy's display name, so if you /cycle Map Name (or /map) during Halloween what you'd see is just Spooky-Map Name as the response.

Opinion: Not against it, but it may complicate the whole model for how maps are handled, so it needs to be looked at with care.

3.4 Extra features for constants

Constants are nice as they are, but their functionality could be expanded

3.4.1 Support deleting attributes using the constant

A consideration was that constants maybe should support "deleting" the attribute, example:

<map>
  <name>Map</name>
  <variant id="tournament-edition">TE</variant>
  
  <constants>
    <constant id="size">32</constant>
    <constant id="overfill" delete="true"/>
  </constants>

  <if variant="tournament-edition">
    <constant id="size">8</constant>
    <constant id="overfill">8</constant>
  </if>

  <team id="red" max="${size}" max-overfill="${overfill}"/>
  <team id="blue" max="${size}" max-overfill="${overfill}"/>
</map>

In here we want the ranked version to have max=8, max-overfill=8, and the default to have max=32, and no max-overfill at all.
Being able to set the constant to "no value" (via delete="true" to signal that you clearly are doing this on purpose, it's not that you forgot the value), it would strip the whole attribute.

This whole feature is mostly relevant due to what's explained in 3.2 not being a thing, so this could be a way for includes to delete some stuff based on parameters set on the map (given the lack of features like what's proposed in 3.2.3).

Opinion: Leaning towards. I think it could simplify stuff and the complexity added is small.

3.4.2 Support deleting whole elements using the constant

This is an additional to the previous one, which would also add support for the elements being deleted:

<map>
  <name>Map</name>
  <variant id="tournament-edition">TE</variant>
  
  <constants>
    <constant id="mercy">3</constant>
    <constant id="limit">10</constant>
  </constants>

  <if variant="tournament-edition">
    <constant id="mercy" delete="true"/>
  </if>
  
  <score>
    <limit>${limit}</limit>
    <mercy>${mercy}</mercy>
  </score>
</map>

This would be the whole previous option, plus allowing for whole elements to be deleted (see, mercy rule being deleted in the TE), but this is a bit of a more complicated one as you may not want the whole element gone, maybe you want just the inner text removed?

Opinion: Leaning against. If you want to do it for an element, wrap it around a conditional, and let's figure out how to make conditionals usable for this (section 3.2, and probably since this is most relevant for includes, see 3.2.3).

3.4.3 Supporting "mathematical" evaluation in constants (not happening)

<map>
  <name>Map</name>
  
  <constants>
    <constant id="timelimit">30</constant>
  </constants>

  <alert time="${timelimit-5}m">Match ends in 5m!</alert>
  <timelimit>${timelimit}m</timelimit>
</map>

This would "resolve" the time in alert to timelimit-5, which would be 25m.

This is simply not happening anytime soon. Constants should be considered a plain string, and nothing else.

3.4.4 Priorities or conditional overriding in constants

Constants currently always override, so re-defining a constant always re-sets its value. It could make sense to allow for override="false" or a more complex priority="0" system where the one with most priority prevails, independent of the order they're defined in. This would particularily be helpful for includes being able to define their constants as override="false" so the constants defined in the map always take priority (even if declared before the include).

Opinion: I think an override boolean tag is simple and would be helpful, a priority may be way too much for what we want.

4 Wrap-up

I've you've made it here without skipping any of the document, i'm surprised. Please leave your feedback and ideas about this whole thing. We NEED others' opinions for this to proceed.

So go ahead, show us what your problems when XMLing are, and what features or ideas would help solve them!