Pyrogenesis trunk
ISerializer.h
Go to the documentation of this file.
1/* Copyright (C) 2020 Wildfire Games.
2 * This file is part of 0 A.D.
3 *
4 * 0 A.D. is free software: you can redistribute it and/or modify
5 * it under the terms of the GNU General Public License as published by
6 * the Free Software Foundation, either version 2 of the License, or
7 * (at your option) any later version.
8 *
9 * 0 A.D. is distributed in the hope that it will be useful,
10 * but WITHOUT ANY WARRANTY; without even the implied warranty of
11 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
12 * GNU General Public License for more details.
13 *
14 * You should have received a copy of the GNU General Public License
15 * along with 0 A.D. If not, see <http://www.gnu.org/licenses/>.
16 */
17
18#ifndef INCLUDED_ISERIALIZER
19#define INCLUDED_ISERIALIZER
20
21#include "maths/Fixed.h"
22#include "ps/Errors.h"
24
26ERROR_TYPE(Serialize, OutOfBounds);
27ERROR_TYPE(Serialize, InvalidCharInString);
28ERROR_TYPE(Serialize, ScriptError);
29ERROR_TYPE(Serialize, InvalidScriptValue);
30
31/**
32 * @page serialization Component serialization
33 *
34 * This page gives a high-level description of ISerializer and IDeserializer.
35 *
36 * @section design Rough design notes
37 *
38 * We want to handle the following serialization-related features:
39 * - Out-of-sync detection: Based on hash of simulation state.
40 * Must be fast.
41 * - Debugging OOS failures: Human-readable output of simulation state,
42 * that can be diffed against other states.
43 * - Saving and loading games to disk: Machine-readable output of state.
44 * Should remain usable after patches to the game.
45 * - Joining in-progress network games: Machine-readable output of state.
46 * Must be compact (minimise network traffic).
47 * Must be secure (no remote code execution from invalid network data).
48 *
49 * All cases must be portable across OSes, CPU architectures (endianness, bitness),
50 * compilers, compiler settings, etc.
51 *
52 * This is designed for serializing components, which we consider to have three main kinds of state:
53 * - Basic deterministic state: Stuff that's critical to the simulation state,
54 * and is identical between all players in a network game.
55 * Always needs to be saved and loaded and hashed and dumped etc.
56 * - Derived deterministic state: Stuff that can be identically recomputed from
57 * the basic deterministic state,
58 * but typically is stored in memory as an optimisation (e.g. caches).
59 * Shouldn't be saved to disk/network (to save space), and should be recomputed on loading.
60 * Should be included in hashes and debug dumps, to detect errors sooner.
61 * - Non-deterministic state: Stuff that's not part of the simulation state,
62 * and may vary between players in a network game,
63 * but is kept for graphics or for optimisations etc.
64 * Shouldn't be saved or hashed or dumped or anything.
65 *
66 * (TODO: Versioning (for saved games to survive patches) is not yet implemented.)
67 *
68 * We have separate serialization and deserialization APIs (thus requiring components
69 * to implement separate serialize/deserialize functions), rather than a single unified
70 * API (that can either serialize or deserialize depending on the particular instance).
71 * This means more work for simple components, but it simplifies complex components (those
72 * which have versioning, or recompute derived state) since they don't need lots of
73 * "if (IsDeserializing()) ..." checks.
74 *
75 * Callers must use the same sequence of IDeserializer calls as they used
76 * ISerializer calls. For efficiency, serializations typically don't encode
77 * the data type, and so mismatches will make them read bogus values and
78 * hopefully hit an out-of-bounds exception.
79 *
80 * The ISerializable interface used by some code in the engine is irrelevant here. Serialization
81 * of interesting classes is implemented in this module, not in the classes themselves, so that
82 * the serialization system has greater control (e.g. the human-readable debug output can handle
83 * classes very differently to binary output).
84 *
85 * To encourage portability, only fixed-size types are exposed in the API (e.g. int32_t,
86 * not int). Components should use fixed-size types internally, to ensure deterministic
87 * simulation across 32-bit and 64-bit architectures.
88 *
89 * TODO: At least some compound types like lists ought to be supported somehow.
90 *
91 * To encourage security, the API accepts bounds on numbers and string lengths etc,
92 * and will fail if the data exceeds the bounds. Callers should provide sensible bounds
93 * (they must never be exceeded, but the caller should be able to safely cope with any values
94 * within the bounds and not crash or run out of memory etc).
95 *
96 * For the OOS debugging output, the API requires field names for all values. These are
97 * only used for human-readable output, so they should be readable but don't have to
98 * be unique or follow any particular pattern.
99 *
100 * The APIs throw exceptions whenever something fails (all errors are fatal).
101 * Component (de)serialization functions must be exception-safe.
102 */
103
104/*
105 * Implementation details:
106 *
107 * The current implementation has lots of nested virtual function calls,
108 * to maximise flexibility. If this turns out to be too inefficient,
109 * it'll need to be rewritten somehow with less virtuals (probably using
110 * templates instead). (This is a more generic design than will be needed
111 * in practice.)
112 *
113 * The public API functions do bounds checking, then pass the data
114 * to the Put* methods, which subclasses must handle.
115 */
116
117/**
118 * Serialization interface; see @ref serialization "serialization overview".
119 */
121{
122public:
123 virtual ~ISerializer();
124
125 /**
126 * Serialize a number, which must be lower <= value <= upper.
127 * The same bounds must be used when deserializing the number.
128 * This should be used in preference to the Unbounded functions, to
129 * detect erroneous or malicious values that might cause problems when used.
130 * @throw PSERROR_Serialize_OutOfBounds if value is out of bounds
131 * @throw PSERROR_Serialize for any other errors
132 * @param name informative name for debug output
133 * @param value value to serialize
134 * @param lower inclusive lower bound
135 * @param upper inclusive upper bound
136 */
137 void NumberU8(const char* name, uint8_t value, uint8_t lower, uint8_t upper);
138 void NumberI8(const char* name, int8_t value, int8_t lower, int8_t upper);
139 void NumberU16(const char* name, uint16_t value, uint16_t lower, uint16_t upper); ///< @copydoc NumberU8
140 void NumberI16(const char* name, int16_t value, int16_t lower, int16_t upper); ///< @copydoc NumberU8
141 void NumberU32(const char* name, uint32_t value, uint32_t lower, uint32_t upper); ///< @copydoc NumberU8
142 void NumberI32(const char* name, int32_t value, int32_t lower, int32_t upper); ///< @copydoc NumberU8
143
144 /**
145 * Serialize a number.
146 * @throw PSERROR_Serialize for any errors
147 * @param name informative name for debug output
148 * @param value value to serialize
149 */
150 void NumberU8_Unbounded(const char* name, uint8_t value)
151 {
152 // (These functions are defined inline for efficiency)
153 PutNumber(name, value);
154 }
155
156 void NumberI8_Unbounded(const char* name, int8_t value) ///@copydoc NumberU8_Unbounded()
157 {
158 PutNumber(name, value);
159 }
160
161 void NumberU16_Unbounded(const char* name, uint16_t value) ///@copydoc NumberU8_Unbounded()
162 {
163 PutNumber(name, value);
164 }
165
166 void NumberI16_Unbounded(const char* name, int16_t value) ///@copydoc NumberU8_Unbounded()
167 {
168 PutNumber(name, value);
169 }
170
171 void NumberU32_Unbounded(const char* name, uint32_t value) ///@copydoc NumberU8_Unbounded()
172 {
173 PutNumber(name, value);
174 }
175
176 void NumberI32_Unbounded(const char* name, int32_t value) ///@copydoc NumberU8_Unbounded()
177 {
178 PutNumber(name, value);
179 }
180
181 void NumberFloat_Unbounded(const char* name, float value) ///@copydoc NumberU8_Unbounded()
182 {
183 PutNumber(name, value);
184 }
185
186 void NumberDouble_Unbounded(const char* name, double value) ///@copydoc NumberU8_Unbounded()
187 {
188 PutNumber(name, value);
189 }
190
191 void NumberFixed_Unbounded(const char* name, fixed value) ///@copydoc NumberU8_Unbounded()
192 {
193 PutNumber(name, value);
194 }
195
196 /**
197 * Serialize a boolean.
198 */
199 void Bool(const char* name, bool value)
200 {
201 PutBool(name, value);
202 }
203
204 /**
205 * Serialize an ASCII string.
206 * Characters must be in the range U+0001 .. U+00FF (inclusive).
207 * The string must be between minlength .. maxlength (inclusive) characters.
208 */
209 void StringASCII(const char* name, const std::string& value, uint32_t minlength, uint32_t maxlength);
210
211 /**
212 * Serialize a Unicode string.
213 * Characters must be in the range U+0000..U+D7FF, U+E000..U+FDCF, U+FDF0..U+FFFD (inclusive).
214 * The string must be between minlength .. maxlength (inclusive) characters.
215 */
216 void String(const char* name, const std::wstring& value, uint32_t minlength, uint32_t maxlength);
217
218 /**
219 * Serialize a JS::MutableHandleValue.
220 * The value must not contain any unserializable values (like functions).
221 * NOTE: We have to use a mutable handle because JS_Stringify requires that for unknown reasons.
222 */
223 void ScriptVal(const char* name, JS::MutableHandleValue value);
224
225 /**
226 * Serialize a stream of bytes.
227 * It is the caller's responsibility to deal with portability (padding, endianness, etc);
228 * the typed serialize methods should usually be used instead of this.
229 */
230 void RawBytes(const char* name, const u8* data, size_t len);
231
232 /**
233 * Returns true if the serializer is being used in debug mode.
234 * Components should serialize non-critical data (e.g. data that is unchanged
235 * from the template) only if this is true. (It must still only be used
236 * for synchronised, deterministic data.)
237 */
238 virtual bool IsDebug() const;
239
240 /**
241 * Returns a stream which can be used to serialize data directly.
242 * (This is particularly useful for chaining multiple serializers
243 * together.)
244 */
245 virtual std::ostream& GetStream() = 0;
246
247protected:
248 virtual void PutNumber(const char* name, uint8_t value) = 0;
249 virtual void PutNumber(const char* name, int8_t value) = 0;
250 virtual void PutNumber(const char* name, uint16_t value) = 0;
251 virtual void PutNumber(const char* name, int16_t value) = 0;
252 virtual void PutNumber(const char* name, uint32_t value) = 0;
253 virtual void PutNumber(const char* name, int32_t value) = 0;
254 virtual void PutNumber(const char* name, float value) = 0;
255 virtual void PutNumber(const char* name, double value) = 0;
256 virtual void PutNumber(const char* name, fixed value) = 0;
257 virtual void PutBool(const char* name, bool value) = 0;
258 virtual void PutString(const char* name, const std::string& value) = 0;
259 // We have to use a mutable handle because JS_Stringify requires that for unknown reasons.
260 virtual void PutScriptVal(const char* name, JS::MutableHandleValue value) = 0;
261 virtual void PutRaw(const char* name, const u8* data, size_t len) = 0;
262};
263
264#endif // INCLUDED_ISERIALIZER
ERROR_GROUP(Serialize)
ERROR_TYPE(Serialize, OutOfBounds)
A simple fixed-point number class.
Definition: Fixed.h:120
Serialization interface; see serialization overview.
Definition: ISerializer.h:121
void NumberU16(const char *name, uint16_t value, uint16_t lower, uint16_t upper)
Serialize a number, which must be lower <= value <= upper.
Definition: ISerializer.cpp:42
void NumberI8(const char *name, int8_t value, int8_t lower, int8_t upper)
Definition: ISerializer.cpp:35
void NumberI32(const char *name, int32_t value, int32_t lower, int32_t upper)
Serialize a number, which must be lower <= value <= upper.
Definition: ISerializer.cpp:63
virtual void PutNumber(const char *name, uint32_t value)=0
void NumberI8_Unbounded(const char *name, int8_t value)
Serialize a number.
Definition: ISerializer.h:156
void String(const char *name, const std::wstring &value, uint32_t minlength, uint32_t maxlength)
Serialize a Unicode string.
Definition: ISerializer.cpp:82
void NumberU8_Unbounded(const char *name, uint8_t value)
Serialize a number.
Definition: ISerializer.h:150
void RawBytes(const char *name, const u8 *data, size_t len)
Serialize a stream of bytes.
Definition: ISerializer.cpp:100
virtual void PutNumber(const char *name, double value)=0
virtual void PutNumber(const char *name, fixed value)=0
void NumberFloat_Unbounded(const char *name, float value)
Serialize a number.
Definition: ISerializer.h:181
void ScriptVal(const char *name, JS::MutableHandleValue value)
Serialize a JS::MutableHandleValue.
Definition: ISerializer.cpp:95
void NumberI16_Unbounded(const char *name, int16_t value)
Serialize a number.
Definition: ISerializer.h:166
virtual void PutNumber(const char *name, int8_t value)=0
virtual bool IsDebug() const
Returns true if the serializer is being used in debug mode.
Definition: ISerializer.cpp:105
virtual void PutNumber(const char *name, uint8_t value)=0
virtual void PutRaw(const char *name, const u8 *data, size_t len)=0
void NumberU32(const char *name, uint32_t value, uint32_t lower, uint32_t upper)
Serialize a number, which must be lower <= value <= upper.
Definition: ISerializer.cpp:56
void NumberU32_Unbounded(const char *name, uint32_t value)
Serialize a number.
Definition: ISerializer.h:171
virtual ~ISerializer()
Definition: ISerializer.cpp:24
virtual void PutScriptVal(const char *name, JS::MutableHandleValue value)=0
void NumberI32_Unbounded(const char *name, int32_t value)
Serialize a number.
Definition: ISerializer.h:176
void NumberI16(const char *name, int16_t value, int16_t lower, int16_t upper)
Serialize a number, which must be lower <= value <= upper.
Definition: ISerializer.cpp:49
virtual std::ostream & GetStream()=0
Returns a stream which can be used to serialize data directly.
virtual void PutNumber(const char *name, uint16_t value)=0
virtual void PutNumber(const char *name, int32_t value)=0
void NumberFixed_Unbounded(const char *name, fixed value)
Serialize a number.
Definition: ISerializer.h:191
void Bool(const char *name, bool value)
Serialize a boolean.
Definition: ISerializer.h:199
virtual void PutNumber(const char *name, int16_t value)=0
virtual void PutNumber(const char *name, float value)=0
void StringASCII(const char *name, const std::string &value, uint32_t minlength, uint32_t maxlength)
Serialize an ASCII string.
Definition: ISerializer.cpp:70
void NumberU16_Unbounded(const char *name, uint16_t value)
Serialize a number.
Definition: ISerializer.h:161
void NumberDouble_Unbounded(const char *name, double value)
Serialize a number.
Definition: ISerializer.h:186
virtual void PutBool(const char *name, bool value)=0
void NumberU8(const char *name, uint8_t value, uint8_t lower, uint8_t upper)
Serialize a number, which must be lower <= value <= upper.
Definition: ISerializer.cpp:28
virtual void PutString(const char *name, const std::string &value)=0
Definition: SerializeTemplates.h:64
uint8_t u8
Definition: types.h:37
unsigned short uint16_t
Definition: wposix_types.h:52
unsigned int uint32_t
Definition: wposix_types.h:53
short int16_t
Definition: wposix_types.h:38
unsigned char uint8_t
Definition: wposix_types.h:51
signed char int8_t
Definition: wposix_types.h:37