EFT internals research: Response caching

Hello everyone!

Today, I would like to cover 4 aspects of EFT's internals in 4 different articles: secured comminucation, payload encryption, response caching and file integrity scanning. These articles are mostly aimed at SuperMod author and anyone else interested in making modding more secure.

In this article, I'll explain how a somewhat misunderstood system actually functions. Implementing it correctly helps tremendously in speeding up game loading.

Response caching

Since the early days of EFT (before 0.11.0, which is more than 5 years ago!) BSG had the idea that sending over large requests often is expensive. Because of this, the data send by the server was stored on the client and verified by the server. However, with everything EFT, this got exploited to oblivion when hackers realised EFT was far more lax in validation than it should've been.

Since 0.12.9 or so, they by default no longer use this system. The code still persists in the game, so the server can still use it! The great thing about it is that loading into the game and after a raid is much faster.

After working more than 4 years on this project, this sytem also marks the first time EFT actually properly implements a standard. Somewhat shocking, honestly.

How it works in more detail is as follows:

– Sending a request that should be cached but isn't? The HTTP header “If-None-Match” is set to empty.
– Sending a request that should be cashed and is? The HTTP header “If-None-Match” is “123456” (what the hash is)

The hash format send is CRC32.

– If the server responds with HTTP code: 304 and no payload, the client will start to load the cache.
– If the server responds with HTTP code: 200, and a payload containing {“err”:0,”errmsg”:null,”data”:{/*…*/},”crc”:123456}, it will load the data from the response and regenerate the cache file.

Cache files are RFC1950 compressed with an XOR encryption key (13) on top. The filenames are MD5 hashes of the full request path (example: https://prod.escapefromtarkov.com/client/items). Since AKI and the research server use other endpoints, cache generated is not compatible.

Previously it was assumed that sending {“err”:0,”errmsg”:null,”data”:{/*…*/},”crc”:0} was the way to go and that the hash should be ADLER32. While it works, this is not the proper implementation because the client still has to parse the payload from the server.ADLER32 has a chance of generating invalid hashes.

The code for this server-side is suprisingly easy to write.

In the place where we store the data, regenerate the hash whenever the data is loaded or modified.

Whenever we receive a request, check the hash. Is the hash the same? Just return the status code. Otherwise, send the full response.

…as seen here!

Wait a second, that's it?

Yup, that's it! Introducing support for this in AKI wouldn't be too difficult either, but not within the aim of the server. Problem is that in AKI, mods might do tricky things with requests that can invalidate the hash while it's being send. Not ideal! This is why I design my research server in a way this isn't possible.

Future work

There is an interesting obervation to be made: not all requests that previously were cached, are today. For example, customization info used to be. It might be interesting to re-enable caching for them and see how much loading time gain there will be.