HelloEOS

Short description

EOS is Epic Online Services. They are free to use, i.e. work without pay. The main goal is to escape from “NAT”, which allows you to connect clients that have access to the network, but do not have an external IP address. This technology is called p2p, which means peer-to-peer network. We will call a network node accessible to all clients a “relay”. And it’s better to call the clients themselves “peers”. The project below shows the history of a fairly complete study of the functionality of EOS. The language used is C++ standard version 17, C++17. A short analysis of the Epic EOS SDK and its tools is also given. Useful for indie multiplayer games.

Background

It all started when I studied part of the internals of the UnrealEngine kernel in the UePrjCreator project (repo) , I decided to apply my knowledge in some small gaming project. I’ve always been interested in what Epic has to offer in this regard. Unsuccessful searches seem to be hiding information, but in fact we all know that this is due to the lack of keywords to search for. I asked on the forum “forums.unrealengine.com”, here link. User and member of community answered, thanks to him, here he is GarnerP57. I Googled it, took the keywords, and began, as one should always do, to look for alternatives.

The rest of the cute little critters

This is necessary in order not to stumble upon obvious nonsense that no one needs and will soon be abandoned or already abandoned. I first met “AccelByte”, it seemed more publicized and more accessible because the descriptions of network interaction, no additional information or noise is mixed in. As in the case of Epic there was a lot about Ue interspersed with EOS. But difficulties arose with “AccelByte”; the portal and the community were not allowed into the blog. Then, using keywords, I found several more similar non-technologies, here they are:

• I abandoned “Steam” because it is potentially paid, and now the topic is “indie games”.
• I’ll try “Edgegap” later.
• Refused the large “Amazon/AWS” or whatever it was, it seemed suspicious …

… or just wanted to get closer to the Epic product! Very decent source code, no crutches, haven’t come across them. Many IT giants stand out from open source. Here are the key words: games p2p, p2p relay, hosting multiplayer online game, networking and multiplayer, Unreal Engine "p2p". Everyone understands that the main thing that everyone needs from multiplayer in terms of the network is going beyond the “NAT”.

From simple to complex

So the fun begins!

Here are the steps from simple to complex.

From learning the basics, to taking full control, and until the second rebirth. This is probably the standard for “R&D”… But it is not exactly :)

The following are links to the entry point code, and it should be taken into account that main.cpp: is normal, and just a wrapper in try/catch:

int main(int argc, char *argv[]) {
	namespace1::clazz impl;
	try {
		impl.run( argc );
	} 
	catch (std::exception &exc) {
		LOG( "[E] std::exception::what = '%s'", exc.what( ) );
		LOG( "[~] press [Enter] to exit" );
		getchar( );
	}
	return 0;
}

First steps

Let’s introduce a new concept “tick”, this is an SDK function that needs to be called to give the EOS network internals processor time to work with the network. Those who are involved in game development are familiar with the concept of a tick, and yes, this is the same game tick.

At first, simple authorization/authentication was needed, everything was synchronous and with long waits. The topic is very broad and boring, there are many providers using which you can log in to EOS. But still, because it’s boring, I’ll skip this section altogether, it’s better to read code, besides, now I only have access through string tokens of the EOS_DevAuthTool tool.

The code provided in the “SDK” is good, but personally, in order to fully understand the technology, I need to try to break it or improve it. And the second point, the connection to game ticks, as an architectural solution is absolutely correct for game development, in the implementation from the “SDK”, it puts a lot of load on one core, while reducing the load on the core reduces the responsiveness of the provided “UI”, which is unacceptable. In addition, useless use of the processor leads to indiscriminate consumption of electrical energy, which indirectly leads to environmental pollution. Besides, you don’t always want to have a uniform UI for all projects – well, you understand :) The simplest and at least somewhat functional is chat. It was possible to implement it quite quickly, and the examples from the “SDK” are quite convenient. In the “SDK” they are located along the path Samples/P2PNAT/

Here is a link to the entry point MainSynchronously.h. Other useful sources for this part are in the Synchronously directory.

Not multithreaded.

After learning how to work in synchronous mode in those areas that seemed necessary for minimal multiplayer, it was time to experiment. It turned out that indeed the ticks should go strictly in the “EOS” initialization thread. The documentation says this, but does not describe the reason. Some “API” calls explicitly report that there is an error and an invalid “GameThread”, some ignore this.

[LogEOSRTC] RTCPlatform Failed to tick LibRTC. ResultCode=[NotGameThread] Description=[The current thread is wrong, this API must be used only from the game thread]
[LogEOSRTC] LibRTCCore: 'InGameThreadChecker' has been alarmed. ExpectedGameThread=[Id=[7024]] UsedThread=[Id=[6808] Name=[]]

There was hope that they would work from another thread, but such exceptions arise.

Unhandled exception at 0x00007FF9F7DDE1BA (EOSSDK-Win64-Shipping.dll) in HelloEOS.exe: 0xC0000005: 
Access violation reading location 0x00000198C72FA058.

For example in ::EOS_Presence_SetPresence( PresenceHandle, &SetOpt, this, SetPresenceCallbackFn );. I was a little disappointed and decided to switch to something big and beautiful for a while, so that I could return later with a fresh look. Here is a link to the entry point MainAnchronously.h. But this file will be deleted in the release due to its uselessness, like the Anchronously directory.

gRPC over UDP

It was tempting to try to implement the simplest, so-called unary, gRPC channel/call. This venture was a success, grpc::reflection helped a lot with its google::protobuf::DynamicMessageFactory in conjunction with InProcessChannel and grpc::experimental::ClientInterceptorFactoryInterface. Thus, the “helloworld” example from gRPC on the p2p network Epic EOS worked.

Everything according to the gRPC canons:

protoc --grpc_out=src --plugin=protoc-gen-grpc=grpc_cpp_plugin resource/protos/helloworld.proto
protoc --cpp_out=src resource/protos/helloworld.proto

Next, the files helloworld.pb.h, helloworld.pb.cc, helloworld.grpc.pb.h, helloworld.grpc.pb.cc are added to the project.

When the client is running, it creates a channel through grpc::experimental::CreateCustomChannelWithInterceptors in which the wrapper is fired. The UnaryCall wrapper passes a string with “FullySpecifiedMethod” and data on EOS. Pir accepts him, parses him, and calls itself via InProcessChannel, and returns the result via EOS. It turned out to be a kind of “grpcOverUdp”. Of course UDP here is correct and secure, the datagram sequence is guaranteed by EOS. As for re-sending in case of loss, this is a big question. And rather to WebRTC as a base. I carried out experiments, this is below in the section about “stand/bench”.

Here is a link to the entry point Main_gRpc.h. Other useful sources for this part are in the gRpc directory.

Approach: deferred calls

After a small victory, we should address the elephant in the room, or as the French say, let us return to our sheep. I decided to gnaw on the granite of science and solve the issue of user comfort. The first thing that came ready was to encapsulate tick calls. Provide the user with functionality in which he called the necessary methods for exchanging data without worrying about ticks. We construct the data exchange logic first, and therefore launch ticks. This brought reassurance that everything would be good and convenient, most importantly modern - after all, we managed to get away from simple C from the SDK.

Here is an example of the simplest p2p interaction:

    bool isServer = ( argc > 1 );
    Ctx ctx = createContext( isServer );
    QueueCommands::init( ctx.m_PlatformHandle );
    ConnectionRequestListener::AcceptEveryone acceptEveryone( ctx );
    const std::string text0 = "ping1", text1 = "ping2";
    if ( isServer ) {
        Receiving receiving( ctx );
        receiving.text( text0.length( ) );
        receiving.text( text1.length( ) );
        auto incoming = QueueCommands::instance( ).ticksAll( );
        assert( ( text0 == std::string( incoming[ 0 ].begin( ), incoming[ 0 ].end( ) ) ) );
        assert( ( text1 == std::string( incoming[ 1 ].begin( ), incoming[ 1 ].end( ) ) ) );
    } else {
        Sending sending( ctx );
        sending.text( text0 );
        sending.text( text1 );
        QueueCommands::instance( ).ticksAll( );
    }

Here is a link to the entry point MainDeferred.h. Other useful sources for this part are in the Deferred directory.

Nice features

It is very convenient that there are several data transmission channels and you can set them directly and are not afraid that some will intersect with others. Perhaps then it will seem unnecessary to also “name the sockets”, but I always lack features and therefore such variety makes me happy.

Having made a naive ping meter, I was surprised by the responsiveness of the p2p network. A packet travels very quickly from a peer through a relay to another peer. Still, UDP was chosen for a reason; by the way, EOS based on WebRTC works.

Then, as a result of communication with the community, I realized that the idea of adding a feature to measure the EOS channel width would violate the agreement after all, such software is not a gaming application in any way. In general, the maximum possible amount of data: this is the maximum buffer size multiplied by the maximum number of elements in the internal queue. 1170*~4481=~5 Megabytes. This amount of data is sent instantly

Finally, it was interesting to create a mechanism for waiting for a friend to come online.

Approach: asynchrony and multithreading

As a result of development, the code review helped a lot, they suggested how to move from delayed calls to asynchronous ones. It was a wonderful idea and I immediately started developing it! The key turned out to be a tip from Mr. G. Sliepen (from codereview.stackexchange.com) that it is better to use std::packaged_task. It’s so simple that I can’t imagine how I could have missed this opportunity. Instead of std::function, the queue now contains std::packaged_task and results can be obtained from std::future. Since results can be obtained asynchronously, it turns out that ticks can be placed in a separate thread. Of course, we place the EOS initialization in the same thread, and of course with waiting for initialization success. Waiting for the results of the p2p network EOS takes place using any convenient well-known methods from std::future. I’m glad that we managed to fit into the C++17 framework without the slightest loss.

This is what happened, again an example of the simplest interaction:

    bool isServer = ( argc > 1 );
    auto oes = Async::Thread::Factory::create( isServer );
    if ( !oes ) 
        return ECONNRESET;
    auto socketNameChat = "CHAT";
    std::string text0 = "Hello Asy", text1 = "nc World!";
    if ( isServer ) {
        Async::Transport::Receiver recvChat = oes ->createReceiver( socketNameChat );
        Async::command_t command0 = recvChat.byLength( text0.length( ) );
        Async::command_t command1 = recvChat.byLength( text1.length( ) );
        Networking::messageData_t incoming0 = command0.get( );
        Networking::messageData_t incoming1 = command1.get( );
        assert( text0 == std::string( incoming0.begin( ), incoming0.end( ) ) );
        assert( text1 == std::string( incoming1.begin( ), incoming1.end( ) ) );
    } else {
        Async::Transport::Sender sendChat = oes ->createSender( socketNameChat );
        sendChat.text( text0 ).wait( );
        sendChat.text( text1 ).wait( );
    }

It’s now very convenient to use EOS in many threads, I’m glad and got a lot of pleasure. It’s hard to imagine a more comfortable use of the network, but coroutines beckon the developer somewhere far away.

Here is a link to the entry point MainAsynchronously.h. Other useful sources for this part are in the Async directory.

Test bench idea

After some time, I started thinking about how to ensure a restful sleep: so that the end user does not have to worry about network errors. After all, the network is not predictable. It is not deterministic, chaotic, which is why it is attractive to pure logicians; it poses a challenge. In addition, authorization/authentication does not take milliseconds, which can be tiring at the development stages when everything is on fire and you want to go faster. We needed a mechanism as close as possible to p2p and allowing not only to connect peers. But also to emulate network failures, from simple packet loss to a terrible incident - a sharp increase in ping duration. Even reducing the channel width can affect a highly dynamic game. It is convenient to have a tool that, depending on the project, allows you to configure network requirements. With such a tool it would be possible to track the current channel width and peak loads depending on the situation, It would be absolutely gorgeous. So I started the project BenchEosP2p, as soon as I finished it, checked it as a codereview, immediately came back here and wrote this text.

Approach: coroutines

It can be transferred to “coroutines TS” without any problems, but then we will have to upgrade C++ to C++20. Maybe we’ll do it a little later, when everyone is ready.

Details

While the project uses the building system from Microsoft, it is MSBuild from “MSVC 2019”. A little later, after connecting CI there will be “cmake” of course.

It’s a pity that for unverified applications you have to launch the tool from the SDK, SDK/Tools/EOS_DevAuthTool-win32-x64-X.X.0.zip, where X.X.0 is your SDK version. But during development and testing the stand will help you: BenchEosP2p.

There should be screenshots of how to create free accounts, it’s not very simple and obvious. Add your project. Add friends. But wisely, I cheated a little. And I post links to Epic resources, which will change over time, and change depending on changes in the product, and I won’t have to redo unnecessary work.

Here is a table with EOS blogs on the official repo (there are also examples in C#): https://github.com/EpicGames/EOS-Getting-Started?tab=readme-ov-file#c-code

We are also interested in this blog, everything is there: Blog post: “4. Authenticating with Epic Account Services (EAS)” https://dev.epicgames.com/en-US/news/player-authentication-with-epic-account-services-eas

Of course, the application can then be placed on markets, but verification by domain is required. There is not enough strength to describe this process now.

Thanks

• Mr. G. Sliepen for participating in code review.
  • code review about deferred
  • code review about asynchronous
• Community “forums.unrealengine.com”
  • a long way to gaming p2p
• Community “eoshelp.epicgames.com”
  • about bandwidth-meter
• Authors/writers “dev.epicgames.com”
• Authors/developers “Epic OES SDK”

Conclusion

Link to repo