• 注册
  • SPTarkov 动态 SPTarkov 动态 关注:243 内容:59

    EFT internals research: File integrity scanning

  • 查看作者
  • 打赏作者
  • 当前位置: ODDBA社区 > SPTarkov 动态 > 正文
  • 10
  • SPTarkov 动态
  • SPT主创
    初窥堂奥
    VIP2
    SPTarkov项目组

    Hello everyone!

    Today, I would like to cover 4 aspects of EFT's internals in 4
    different articles: secured comminucation, payload encryption, response
    caching and file integrity scanning. These articles are mostly aimed at
    SuperMod author and anyone else interested in making modding more
    secure.

    In this last article, I'll be talking about another completely unexplored territory; file integrity validation.

    File integrity validation

    Ever had the game instantly close on you when you start the game the wrong way? If so, that was file integrity kicking in and detecting you run AKI.

    In EFT, there are two validation checks: one by BsgLauncher, another by the game itself (introduced in 0.12.11). BsgLauncher uses the info stored in a file called ConsistencyInfo. The game on the other hand has the info hard-coded inside FilesChecker.dll. By having these two checks, it detected early on alot of cheaters who made use of simple timely injection scripts. These days the tricks are long bypassed.

    AKI opts for completely disabling the checks by making them always return true. My research server on the other hand reimplements the checks so the server can do custom verification.

    Here is what the data structure roughtly looks like:

        [DataContract]
        public struct ConsistencyMetadata
        {
            [DataMember]
            public string Path;

            [DataMember]
            public long Size;

            [DataMember(EmitDefaultValue = false)]
            public string Hash;

            [DataMember(EmitDefaultValue = false)]
            public int? Checksum;

            [DataMember(EmitDefaultValue = false)]
            public bool? IsCritical;
        }

    There is a slight problem; BsgLauncher and the game itself use two different methods for validation. The launcher checks a MD5 hash, whereas the game uses a checksum. because of this, I opted for implementing both in case the need arises for supporting both.

    Some files (like Assembly-CSharp.dll) might be marked as critical. Non-critical files are only checked for if they exist and if the filesize is correct. Critical files however also validated for the hash/checksum. While implementing these checks itself is not difficult, the real challenge is to make it performant. To get a rough idea how slow critically-checking all files is: 6200
    files (some of which 200+ MB) take 1 minute on a 6-core laptop processor
    with multi-threaded checking.

    In dotnet 6 (A language runtime for C#), we can use Parallel.ForEachAsync() to run the code multi-threaded. However, this is not available in EFT's C# runtime. My only option here is to re-implement the functionality myself. Welcome to my own thread scheduler!

            // https://github.com/dotnet/runtime/blob/main/src/libraries/System.Threading.Tasks.Parallel/src/System/Threading/Tasks/Parallel.ForEachAsync.cs
            public static Task ForEachAsync<T>(
                IEnumerable<T> source,
                Func<T, CancellationToken, Task> body)
            {
                if (source == null)
                {
                    throw new ArgumentNullException(“source”);
                }

                if (body == null)
                {
                    throw new ArgumentNullException(“body”);
                }

                if (Options.MaxDegreeOfParallelism < 1)
                {
                    throw new IndexOutOfRangeException(“Options.MaxDegreeOfParallelism”);
                }

                var threadCount = Options.MaxDegreeOfParallelism;
                var threads = new Task[threadCount];
                var scheduler = Options.TaskScheduler ?? TaskScheduler.Current;

                var ct = Options.CancellationToken;
                var cts = CancellationTokenSource.CreateLinkedTokenSource(ct);
                var enumerator = source.GetEnumerator();
                var semaphore = new SemaphoreSlim(1, 1);

                for (int i = 0; i < threadCount; ++i)
                {
                    threads[i] = Task.Factory.StartNew(async () =>
                    {
                        try
                        {
                            while (true)
                            {
                                if (cts.IsCancellationRequested)
                                {
                                    ct.ThrowIfCancellationRequested();
                                    break;
                                }

                                // continue on captured context.
                                await semaphore.WaitAsync();

                                T item;

                                try
                                {
                                    if (!enumerator.MoveNext())
                                    {
                                        break;
                                    }

                                    item = enumerator.Current;
                                }
                                finally
                                {
                                    semaphore.Release();
                                }

                                // continue on captured context.
                                await body(item, cts.Token);
                            }
                        }
                        catch
                        {
                            cts.Cancel();
                            throw;
                        }
                    }, CancellationToken.None, TaskCreationOptions.DenyChildAttach, scheduler).Unwrap();
                }

                var continuationOptions = TaskContinuationOptions.DenyChildAttach
                                        | TaskContinuationOptions.ExecuteSynchronously;

                return Task.WhenAll(threads).ContinueWith(t =>
                {
                    // Clean up (dispose all disposables)
                    enumerator.Dispose();
                    cts.Dispose();
                    semaphore.Dispose();
                    return t;
                }, CancellationToken.None, continuationOptions, TaskScheduler.Default).Unwrap();
            }

    This is all really complicated, so in short: I grab the threads (except the primary thread) available, then offload tasks (things to do) onto them. In between tasks I need to synchronize, which the semaphore is used for.

    There is a nice benefit to this implementation; while Parallel.ForEachAsync adds threads to use over time, my implementation immediately uses all threads. This makes the code more performant in scenarios like this. The result of this is much faster file validation than EFT currently provides.

            public async Task<Dictionary<string, ConsistencyResult>> ValidateFiles(
                string basepath,
                ConsistencyMetadata[] metadatas)
            {
                var results = new ConcurrentDictionary<string, ConsistencyResult>();

                await Scheduler.ForEachAsync(metadatas, async (metadata, ct) =>
                {
                    var result = await ValidateFile(basepath, metadata);
                    results.TryAdd(metadata.Path, result);
                });

                return new Dictionary<string, ConsistencyResult>(results);
            }

    This is especially nice for bundle loading. With a bit of code reworking, it would be possible to run most of the bundle loading code on separate threads, instead of asynchroniously on the same thread, resulting in faster loading of the game.

    How do modders benefit from this?

    With this implementation, it is now possible to have server-side validation of your files. If a person tampered with your mod, you can now prevent loading of the game and warn the user not to do it or to redownload the mod. One more protection layer for modders to use!

    Do only modders benefit from this?

    No, normal users benefit too. Since the code is faster to run, the client will load faster too. And you now got a way to validate that your client is set-up correctly, making it easier to diagnose issues.

    略有小成

    Do you think updating the Unity 2023 engine will significantly increase the frame rate of the game?

    你认为更新 Unity 2023 引擎会显着提高游戏的帧率吗?

    回复
    初来乍到

    6666666666666666

    回复
    登堂入室

    所以海关刷killa是正式版本的问题嘛?现在有啥活动啊让killa来海关?

    回复
    炉火纯青
    2021

    [S-10型]

    回复
    登堂入室

    看不明白

    回复
    渐入佳境
    赠送了礼物[赞]
    回复
    初窥堂奥
    赠送了礼物[赞]
    回复
    初来乍到
    赠送了礼物[赞]
    回复
    初来乍到
    打赏了1金币
    回复
    登堂入室
    赠送了礼物[赞]
    回复

    请登录之后再进行评论

    登录
  • 签到
  • 任务
  • 发布
  • 模式切换
  • 偏好设置
  • 帖子间隔 侧栏位置: