Server works with Gamelift-local, breaks on fleet activation

I’m using Unreal Engine 4, the C++ server SDK and YetiTech-Studios’ Client SDK plugin

I managed to get all my projects’ dependencies compiled for Linux and I can run the server from linux using gamelift-local and connect to it and play with no problem, but when I upload it to gamelift all I get is a bunch of events like the following:

SERVER_PROCESS_CRASHED | Server process exited without calling ProcessEnding(), exitCode(1) …

I’ve downloaded the output from the extraction process, and only some files seem to be missing, but it’s just a few .ini files that are generated. All of the .so files are there and in the same locations.

Any help at all would be greatly appreciated, and if there’s some way I can get logs out during this phase?

Sorry, this can be a little a painful at time.

To get logs, when you call create-fleet ensure you open a SSH port for your local machine:

FromPort=22,ToPort=22,IpRange=<your external ip>/32,Protocol=TCP

You can also do this via update-fleet-port-settings on any fleet thats in ERROR or ACTIVE but if you want your logs during ACTIVATING then you have to remember to do it during fleet creation (for now).

Then you can call describe-instances to get an instance id from your fleet, and then call get-instance-access with that instance id
https://docs.aws.amazon.com/gamelift/latest/developerguide/fleets-remote-access.html

Note: You may have to reformat the pem file to ensure its a valid pem file and call chmod 400 on it. You may want to write a small python script using boto3 to help automate this as it can be time consuming to repeat.

Then you can ssh into your instance and your logs should be in /local/game.

Other things to check:

  • Look at the fleet events tab in the Gamelift console and ensure that if you ran an install.sh file then its output matches what you expect
  • Look at the other fleet events in case they hint at the problem
3 Likes

Thank you so much, this is incredibly valuable information that will probably save me a few headaches! Cheers

So after I ssh’d into the failed fleet instance, I found the two errors that were causing this issue, and I’m a bit surprised.

/usr/lib64/libcurl.so.4: no version information available (required by local/game/<MyGameName>/Binaries/Linux/libaws-cpp-sdk-core.so)
/lib64/libc.so.6: version `GLIBC_2.25’ not found (required by libcrypto.so.1.1)

The GLIBC_2.25 part is the one that bothers me (the other one seems to be more of a warning), since when I run

lld -v

the system GLIBC version printed out is 2.17, which is 7 years old (2012). Am I supposed to downgrade my tools until I can build libraries that link to GLIBC 2.17 or is there some preferred simpler method?

As you note GameLift currently uses AmazonLinux in EC2 instances. Theses have an older version of the glib c libraries (GNU C Library (GNU libc) stable release version 2.17) which is what the dynamic linker is finding (see ldd --version)

May want to try ldd to see where its looking for libraries: https://gcc.gnu.org/onlinedocs/libstdc++/faq.html#faq.how_to_set_paths

For unreal builds I believe you should should have *.so files for these libs in the same folder of your executable, so I would check how you are building.

See:


Can I ask why you are including the UE4GameLiftClientSDK in your server build? Just curious about the use of the ibaws-cpp-sdk-core.so

I was under the impression that the core, cognito-identity and gamelift .so files were needed for server builds too (I actually followed the process from chrisgong’s post that you linked to get to this point) . I’ll have a look at my Unreal build.cs file and remove the Client SDK from server builds.

I’m not sure I understand you correctly. Can I just copy over my libc.so and bundle it, and then add it to the library path as part of the install script?

For some reason the older version of the gamelift client sdk was configured so that it was built with the client AND server. However, we are currently working on updating the client sdk so that it only builds the client sdk with the client only. Therefore, the gameliftsdk server .so file seems to only depend on the libssl and libcrypto files from my understanding. The .so files for cognito identity, gamelift, and core can be excluded from the server build once we update the client sdk, which should be coming very soon.

1 Like

Hi Pip, could you expand on the part where you mentioned ssh’ing into the server during fleet creation? I seem to be getting this message when calling update-fleet-port-settings right after creating the fleet:

An error occurred (InvalidRequestException) when calling the UpdateFleetPortSettings operation: Fleet fleet-… (NEW) is not ACTIVE or ERROR

What worked for me was when opening ports, as you do with 7777 in UDP mode, you can open port 22 in TCP mode for “YourPublicIPAddress/32”

However you will still only be able to ssh into the machine at the end of the activation stage or when it changes to error state.

But is this when the fleet has finished activating? Or is this when the fleet is still activating/validating?

You can enter the setup when creating the fleet, but you aren’t able to actually ssh into it until later on. And I believe it’s when it finished the activation state, just the aws console being slow might say it’s still activating.

1 Like

In the end I had to completely remove the GameliftClientSDK for the server build, otherwise it would want to link aws core and gamelift into the executable even if I excluded the ClientSDK in the build.cs for server builds.

I managed to play a few rounds there and now I have to fix stuff in my code that caused it to crash, but this is now back in familiar territory. Thank you so much for all your help @Pip @chrisgong