This page gives a high-level description of ISerializer and IDeserializer.

Rough design notes

We want to handle the following serialization-related features:

Out-of-sync detection: Based on hash of simulation state. Must be fast.
Debugging OOS failures: Human-readable output of simulation state, that can be diffed against other states.
Saving and loading games to disk: Machine-readable output of state. Should remain usable after patches to the game.
Joining in-progress network games: Machine-readable output of state. Must be compact (minimise network traffic). Must be secure (no remote code execution from invalid network data).

All cases must be portable across OSes, CPU architectures (endianness, bitness), compilers, compiler settings, etc.

This is designed for serializing components, which we consider to have three main kinds of state:

Basic deterministic state: Stuff that's critical to the simulation state, and is identical between all players in a network game. Always needs to be saved and loaded and hashed and dumped etc.
Derived deterministic state: Stuff that can be identically recomputed from the basic deterministic state, but typically is stored in memory as an optimisation (e.g. caches). Shouldn't be saved to disk/network (to save space), and should be recomputed on loading. Should be included in hashes and debug dumps, to detect errors sooner.
Non-deterministic state: Stuff that's not part of the simulation state, and may vary between players in a network game, but is kept for graphics or for optimisations etc. Shouldn't be saved or hashed or dumped or anything.

(TODO: Versioning (for saved games to survive patches) is not yet implemented.)

We have separate serialization and deserialization APIs (thus requiring components to implement separate serialize/deserialize functions), rather than a single unified API (that can either serialize or deserialize depending on the particular instance). This means more work for simple components, but it simplifies complex components (those which have versioning, or recompute derived state) since they don't need lots of "if (IsDeserializing()) ..." checks.

Callers must use the same sequence of IDeserializer calls as they used ISerializer calls. For efficiency, serializations typically don't encode the data type, and so mismatches will make them read bogus values and hopefully hit an out-of-bounds exception.

The ISerializable interface used by some code in the engine is irrelevant here. Serialization of interesting classes is implemented in this module, not in the classes themselves, so that the serialization system has greater control (e.g. the human-readable debug output can handle classes very differently to binary output).

To encourage portability, only fixed-size types are exposed in the API (e.g. int32_t, not int). Components should use fixed-size types internally, to ensure deterministic simulation across 32-bit and 64-bit architectures.

TODO: At least some compound types like lists ought to be supported somehow.

To encourage security, the API accepts bounds on numbers and string lengths etc, and will fail if the data exceeds the bounds. Callers should provide sensible bounds (they must never be exceeded, but the caller should be able to safely cope with any values within the bounds and not crash or run out of memory etc).

For the OOS debugging output, the API requires field names for all values. These are only used for human-readable output, so they should be readable but don't have to be unique or follow any particular pattern.

The APIs throw exceptions whenever something fails (all errors are fatal). Component (de)serialization functions must be exception-safe.