Technical Devlog: Evolution of Scripting

In Beta 1.4 I'm changing how scripts are implemented in Ero Dungeons. If you aren't a modder, you probably have no idea what these are, and what they do. As such, it seemed a neat idea to give a short explanation here. Scripts are how everything, from moves to equipment to curios, does stuff. But the easiest way to explain is to go back to Ero Witches.

How it Worked in the Previous Game

 Ero Witches didn't have an in-game data manager (this was added to Ero Dungeons last Christmas), so all data was stored in external Excel files which were then converted to csv files to be read in by the game. Here's a screenshot of part of the list of moves:

The damage calculation of moves worked like Pokemon, but everything else was done in the script field. For example, when the player uses the move glitterdust, the game goes through every line of the script field and parses it. So in this case it would read "add_trait,glitterdust" split it at the comma, and look what code it should execute when it encounters "add_trait" and in that code try to create the trait "glitterdust". But it wasn't limited to that, you had a script "fixed_damage" which would override the resulting damage. You had "always_available", which wouldn't have any effect during the move, but comes into play when moves were selected. You also had "self_add_trait" to add a trait to the user of the move. All in all, there were only about 8 different scripts, so it was pretty manageable.

Traits worked similarly. But here the scripts were read whenever the game wanted to know the stats of the player. So if the game wanted to know what her Attack was, it would run over all traits (and equipment), parse the scripts, and add up all relevant scripts. If you think this is a lot of work and can't possibly be performant, you are underestimating the strength and power of modern computers.

The application of these scripts was very broad. You had scripts that change movement speed in the overworld map, force the usage of moves, or change the way the puppets are drawn. It was a very versatile system, and in Ero Dungeons scripts still somewhat work like that. So what needed to change?

Problem 1: Typos

So "self_add_trait" is a script, but I would often write "add_self_trait". Or I would write "ATK_stages" instead of "ATK_stage".  Or I would write "force,repeating_shotgun" instead of "force,repeater_shotgun". Then, when the game would encounter this script it would either crash or do nothing. This required a lot of testing from my part, and slowed down development too much.

To counter it, I added a verification system. For every script that could exist I'd add a key in a dictionary and link it to a hint about the values it could take. Then, when starting the game, the game would look through every single script and check whether it exist and whether the arguments I put behind it made sense. Again, thanks to the power of modern computers, this took less than a second but saved many hours of testing time.

I'd do the same for every sort of object that could take scripts. So you had movescript_verification, consumablescript_verification, aiscript_verification, generalscript_verification, playerscript_verification, timescript_verification (to determine how long traits would last), titletrigger_verification, expression_verification, actor_trigger/effect/activation_verification (this is for overworld objects, what are now curios), dialog_verification, and so on...

It wasn't very pretty, but it worked.

Problem 2: Tooltips

Tooltips had to be written manually. This caused issues when I would rebalance things. A move might apply a trait that lowers stats by 2 stages, then I rebalance that trait to only lower it by 1. If I forget to change the tooltip of the move (and the trait), there is no way I'll notice until a player would wonder why the tooltip was incorrect.  Additionally, writing all those tooltips was mind numbing. Having to write "lowering the <STAT> of the target by <AMOUNT> stages." for every move isn't very interesting.

Problem 3: Bloat and Limitations

It wasn't that much of a problem for Ero Witches, since that game didn't have as many scripts, but it was still silly to have things like add_trait and self_add_trait and add_value and self_add_value.  The game also wasn't able to let things happen at the start of combat or turn (the WHEN clauses in Ero Dungeons), or add traits conditionally.  Many of the more complex behaviours were just hardcoded, because adding it "correctly" would be too much work.

How it Worked in the Current Game

Solving Problem 2

Instead of storing the scripts in one large dictionary hardcoded into the game. I moved them to their own files. Additionally, I gave each of them a description. The tooltip of an item is just the descriptions of the scripts of that item pasted under eachother. So if one item gets rebalanced, the tooltip gets automatically corrected. And as a player you always know what an item does exactly (unless I explicitly hide the scripts like with the Bad Luck Charm).

This, combined with moving the scripts from Excel to an in-game editor, also fully removes Problem 1. With the data readily available, I can verify scripts in real time. So if I make a typo in the effects of an item, the editor will tell me immediately instead of waiting until I converted the data to csv and started the game.

Finally, it also helps modders. Now they immediately know both what scripts are available, and why exactly their script doesn't work.

Solving Problem 3

For moves I split up the script part into two halves. One that got applied to the attacker, and one on the targets (provided the move hits). This significantly lowered the bloat.

For other scripts, which I started calling "complex_scripts", I created three new type of scripts. IF, allowing conditionals which if negatively evaluated stops any scripts after fom being executed. WHEN, allowing any scripts behind it to be executed only at a specific time (start of turn, end of day, etc...). FOR, which multiplies any scripts behind it by a calculated multiplier. Additionally the old scripts were split up in two different categories "scriptablescripts" which are passive and always active and "whenscripts" which are active and happen after a WHEN clause (hence the name).

This sounds simple, but the implementation was an absolute mess. The code was an absolutely disgusting mess. In practice it was a dictionary which connects each script to all relevant clauses (FOR, WHEN, IF), however since this breaks for ELSE and ELIF, and doesn't keep any order of the clauses, it required a bunch of extra code to barely hold it together.

I also added an AT-clause later. Which allows a script to be evaluated for a different character. However it was notoriously buggy, and only barely worked for the few items in the game that used it.

Problem 4: Overlap

It was already clear in Ero Witches, but there are just so many types of scripts. As the amount of scripts for each type grows, there is going to be a growing overlap between them. A move requirement has a has_token script, conditionals of course have the same one, as do requirements for curios or quirks. Obviously, the earliest scripts for all of these had no overlap, so it wasn't clear that the same scripts could be used. In fact, some scripts may only work for one type, but eventually those grow few and far between. This becomes even worse due to small naming changes, tokens in curio requirements becomes, has_tokens for equipment, and self_has_tokens in move requirements. Disgusting.

Problem 5: Limitations and Bloat

For example, you had the following conditionalscripts, has_parasite, self_has_parasite, not_has_parasite, not_self_has_parasite. Not only does each of these needs its own implementation and description, it also unnecessarily bloats the list of all scripts, making it harder to find the ones you're looking for. They also aren't consistent, some may have the default referring to the target, some have it refer to the user which leads to a lot of confusion.

As mentioned before, the AT-clauses were also broken, so applying effects to allies would sometimes break or be unusable.

Problem 6: Complex Tooltips

Due to the spaghetti code earlier, tooltips tend to break when too many IF clauses are used. It also won't keep the order between different clauses, which - certainly with the AT-clauses - can turn the tooltip in something different than intended. This mostly affects mods, since the basegame doesn't really have very complex scripts, but the code to make the tooltips was still very ugly and overly long.

Problem 7: Non-Tooltip Descriptions

Next to the tooltips, there are also the floating text in combat, as well as the descriptions after dungeons or after interacting with curios. These are all hardcoded. If I'll have to rework the scripts to minimize bloat and overlap, I don't want to rewrite these hundreds of lines of code, and I certainly don't want to expand them for the new scripts that become available for these instances.

How it Works Now

Solving Problem 4 and 5

There are two types of main scripts. Those that are passive (scriptablescript) and those that are active (whenscript). Then you have three scripts that can influence those scripts: conditionals (conditionalscript) defining whether a part of code can run; temporals (temporalscript) defining when an active script has to be run; and counters (counterscript) multiplying scripts by a certain value.

All scripts for items in the game are in theory comprised of these scripts. Equipment uses all of them. A curio doesn't use passive scripts, but can use all others, in fact it can be seen as an action after a WHEN:curio_interacted clause. So instead of having curioscripts, a curio can use a mix of when/conditional/temporal and counterscripts. A curioreqscript, which defines what choices a character can make when interacting with a curio is actually just a list of conditionalscripts. A move is merely a complex script that only runs while the move is actively being performed, so applying a token can use a whenscript, while damage scaling based on the amount of tokens is a counterscript and a scriptablescript. The requirements of a move doesn't have to be a movereqscript but is again a series of conditionalscripts.

Finally, the scripts within each script type can be further reduced by using Scopes. Scopes define on which target the script has to be applied. The default scope is SELF, but anything is possible. As a benefit, these scopes are very easy to implement. 

Solving Problem 6:

The solution to the spaghetti codebase was actually quite simple. Parsing stuff and interpreting it is a solved problem, after all it's what all compilers do. The first step is tokenization. A text like:

FOR:token,divine
DMG,30
ENDFOR
WHEN:move:HIT_TARGETS
IF:save,WIL
dot,fire,4,3

Gets read line by line which each part getting converted to a Token and a value. So you get:

TOKEN.FOR
TOKEN.COUNTERSCRIPT token,divine
TOKEN.OPEN
TOKEN.SCRIPT DMG,30
TOKEN.CLOSE
TOKEN.WHEN
TOKEN.TEMPORALSCRIPT move
TOKEN.SCOPE HIT_TARGETS
TOKEN.OPEN
TOKEN.IF
TOKEN.CONDITIONALSCRIPT save,WIL
TOKEN.OPEN
TOKEN.WHENSCRIPT dot,fire,4,3
TOKEN.CLOSE
TOKEN.CLOSE

Which then gets compressed, with parts between OPEN and CLOSE becoming their own section. These blocks can then be handled independent of all other stuff.

TOKEN.FOR
TOKEN.COUNTERSCRIPT token,divine
TOKEN.BLOCK <lots of stuff here>
TOKEN.WHEN
TOKEN.TEMPORALSCRIPT move
TOKEN.SCOPE HIT_TARGETS
TOKEN.BLOCK <lots of stuff here>

This can then easily be handled by the game, the blocks can be ignored or not depending on the tokens in front of them, and making tooltips just means you have to step through each line. The code for creating nice looking tooltips went from 1000+ lines to around 200, and the strange bugs are now solved.

Solving Problem 7:

Any whenscript can lead to a set of actions, these actions are very concrete things that the game has to do. So a whenscript of token_chance,50,block can lead to an action add_tokens,block or no action at all. When adding the floating text during combat, or the descriptions at the end of dungeons, or after interacting with curios, I'm actually listing the results of those actions. So the solution was to make these actions explicit, and give each action a description.

This further reduces the amount of code needed, since scripts can reuse these actions. It also means I no longer have to hardcode any of these descriptions. Additionally, it allows translators to translate these.

Conclusion:

I think the scripting followed a very natural evolution. Sure, the current implementation is much much better than it was in the days of Ero Witches. But on the other hand, it would also have been overkill for that game. In fact, it might even be overkill for this game. However, looking at the Patreon, it seems like I'll be developing games for a while. Creating content for games is always the hardest and most time consuming part (compared to prototyping, implementing, and polishing), and good tooling is vital for that. Furthermore, wedging this into an existing game also helps tremendously, since it shows all usecases and gives immediate feedback on stuff that is broken.