This was an interesting post. Your suggested solutions, though, imo sound too close to "have systems in place that detect when damage is doubled", which has lots of potential failure modes (e.g. it might detect when damage numbers double, but not when damage is applied twice).
One much more general thing you can do in any software where inputs aren't highly time-sensitive (i.e. not real-time strategy or something) is to have an auto-replay functionality: While playing through a level, record your inputs and the resulting outputs (level finished, boss defeated, etc). Then, after doing any significant code changes, let the game play through your recordings, and check whether your inputs still result in the expected outputs.
For instance, you can find stream recordings on Youtube of Jonathan Blow developing an isometric puzzle game, and after any major code changes, he'd let the game play through all level recordings (at 8x speed or higher) and check whether they still behaved as expected, both in terms of "is this level still getting solved" and in terms of fuzzier things like "does anything look off, e.g. in terms animations".
Good to see there's a whole blog about this problem, of deluding oneself into thinking black boxes are all that reliable. The documentary Hypernormalisation, and Adam Curtis' work generally, is/are good exercises in holistic/gestalt thinking, as opposed to the linear thinking of domain experts.
On a similar vein, look up ubersqueeze in the same game. I found out about the bug, being a new (to octopode) player, picking it, meleeing things and being like "wow the squeeze is crazy strong", talking about it with the community, then a more veteran player spectated my game and figured out it was super bugged lol, fun times.
So, how did the bug get discovered?
Y'know, I should go see if the person doing the reverting commit still remembers how they found it. Everyone I know just saw it from the commit.
This was an interesting post. Your suggested solutions, though, imo sound too close to "have systems in place that detect when damage is doubled", which has lots of potential failure modes (e.g. it might detect when damage numbers double, but not when damage is applied twice).
One much more general thing you can do in any software where inputs aren't highly time-sensitive (i.e. not real-time strategy or something) is to have an auto-replay functionality: While playing through a level, record your inputs and the resulting outputs (level finished, boss defeated, etc). Then, after doing any significant code changes, let the game play through your recordings, and check whether your inputs still result in the expected outputs.
For instance, you can find stream recordings on Youtube of Jonathan Blow developing an isometric puzzle game, and after any major code changes, he'd let the game play through all level recordings (at 8x speed or higher) and check whether they still behaved as expected, both in terms of "is this level still getting solved" and in terms of fuzzier things like "does anything look off, e.g. in terms animations".
Good to see there's a whole blog about this problem, of deluding oneself into thinking black boxes are all that reliable. The documentary Hypernormalisation, and Adam Curtis' work generally, is/are good exercises in holistic/gestalt thinking, as opposed to the linear thinking of domain experts.
On a similar vein, look up ubersqueeze in the same game. I found out about the bug, being a new (to octopode) player, picking it, meleeing things and being like "wow the squeeze is crazy strong", talking about it with the community, then a more veteran player spectated my game and figured out it was super bugged lol, fun times.