A process unrecognized by GameLift is attempting to make GameLift Server SDK API calls. Are you using a process launcher?

Hi,

I am unable to run my executable on amazon gamelift. I’ve seen another post with this error, but it didn’t help me resolve it.

context:

I am building a single C++ executable on an Amazon Linux 2, along with an install.sh script to install some third parties.
I am calling InitSDK from the main executable.
I have set it up like this in the fleet:
/local/game/MyExecutable

It is not a shortcut, and I am not running a process launcher.

I would like some help with this, as I have no clue why this is happening.

Thanks!
David

Hi @David_St-Louis

We typically see this error message when a process that tries to communicate with GameLift (via initSDK, processReady, etc.) is not the process in the runtime configuration.

  1. Does you fleet eventually go active? If so, how often do you see this error message?
  2. Is it possible that you are running other processes that are invoking GameLift SDK calls? (e.g. something that started from the install.sh?)
  3. It might be possible to find out which process is invoking GameLift SDK and generated this error message:
    a. Make sure you create your fleet with TCP port 22 open (you cannot update fleet port settings until the fleet is ACTIVE), so you can ssh into the fleet (see example script), run sudo su, and go to /local/whitewater/AuxProxy/logs to find the GameLift process manager logs.
    b. Look for SERVER_PROCESS_SDK_INITIALIZATION_TIMEOUT, prior to that, there should be a log SDK connected:... immediately followed by SDK disconnected:...; You can also use the timestamp of the fleet event error to narrow your search.
    c. In the SDK connected... message, see if you can find any information identifying the process
  4. Are you using the latest versions of the C++ SDK? (If you downloaded the SDK after 2020-09-17 then you should have the latest, see release notes) If not, please try using the latest version of the C++ SDK.

If you need further assistance, please provide your region and fleet id to help us investigate.

Related: Launch bash script or jar - is it possible?

Hi James,
Thanks for the quick reply.

  1. It never goes active.
  2. No other processes. I install stuff first using install.sh, I see the log and looks fine. I am calling my executable and that’s it. First thing I do is call InitSDK. I know the executable is called, because I played with return code and it changed.
  3. Ok I will attempt SSH into it. I was wondering how to get the log
  4. I am using the C++ SDK version 3.4.1

More info:
my fleet Id is : fleet-33aee5db-583a-4719-8029-b2461d9416b2

Everything looks good until that point.

(edit): For the fail to report healthy, it’s the InitSDK that takes forever to run. I think it tries to contact amazon gamelift and it fails to do so. Not sure if there is another outbound port I have to bind? I’ve bound TCP 22 and TCP 9313

Have you tried to test the integration with GameLift local? If not, please give that a try also: Testing Your Integration - Amazon GameLift

I am calling my executable and that’s it.

Do you mean your install.sh script is calling your game server executable? If so, it would make sense why you are seeing this error. You should be passing the server launch path in CreateFleet’s RuntimeConfiguration argument, see “Server Process Allocation” in Deploy a GameLift fleet with a custom game build - Amazon GameLift. On instance start up, GameLift takes care of running the game server process that you specified in the runtime configuration. This is the only process that can call SDK method, any other processes that tries to invoke SDK calls are not allowed.

Not sure if there is another outbound port I have to bind? I’ve bound TCP 22 and TCP 9313

Nope, GameLift takes care of reserving the outbound port that it needs to communicate with the service, so that shouldn’t be the issue.

Please let us know if you have any luck with the SSH approach.

Sorry, by “calling my executable” I mean I’ve set up in the fleet the executable and nothing else. /local/game/MyExecutable
I am not calling it by the intermediate of a shortcut or other process.

I’ll be looking at the ssh and local today. Could be hard to setup, because I am on windows, and building my code inside a docker container (amazonlinux:latest)

From the doc,

Only instances in active fleets can be remotely accessed

Mine never get active, so I’ll try the other solution to gamelift local…

So I was still able to SSH into it (Doc is wrong :slight_smile: ) but I cannot find the specified log file. There is no logs folder under AuxProxy. There is a logs folder on the parent but none called gamelift process manager.

I am unsure if those other logs are related to this. I cannot find SERVER_PROCESS_SDK_INITIALIZATION_TIMEOUT in them.

If you can help me look into it a bit more would be great:
Region: us-west-2
Fleet id: fleet-33aee5db-583a-4719-8029-b2461d9416b2

I also tried to run the executable directly from SSH. Here is the log I see if it can help track down:

[2021-06-23 12:18:07] [connect] Successful connection
[2021-06-23 12:18:07] [connect] WebSocket Connection 127.0.0.1:5757 v-2 "WebSocket++/0.5.1" /socket.io/?EIO=4&transport=websocket&t=1624450687&pID=20533&sdkLanguage=Cpp&sdkVersion=3.4.1 101
[2021-06-23 12:18:07] [connect] Successful connection
[2021-06-23 12:18:07] [connect] WebSocket Connection 127.0.0.1:5757 v-2 "WebSocket++/0.5.1" /socket.io/?EIO=4&transport=websocket&t=1624450687&pID=20533&sdkLanguage=Cpp&sdkVersion=3.4.1 101
[12:18:07:0522] Failed to GameLift::InitSDK():

that last line is from my code, and I am attempting to print the error message but it’s empty:

auto gamelift_outcome = Aws::GameLift::Server::InitSDK();
if (!gamelift_outcome.IsSuccess())
{
    auto error = gamelift_outcome.GetError();
    log("Failed to GameLift::InitSDK(): %s\n", gamelift_outcome.GetError().GetErrorMessage());
    return 1;
}

Ok i ran gamelift local, then ran my executable. And I get this, several times, after calling InitSDK

[2021-06-23 14:15:25] [info] asio async_connect error: system:111 (Connection refused)
[2021-06-23 14:15:25] [info] Error getting remote endpoint: system:107 (Transport endpoint is not connected)
[2021-06-23 14:15:25] [error] handle_connect error: Underlying Transport Error

Could there be a bug with this C++ SDK version? Or did I skip a step somewhere :thinking:
There is no other log, no other calls done. It mirrors pretty much what we were seeing in the fleet. InitSDK fails then keep retrying for a few minutes.

Okay, good to know that it’s reproducible on GameLift local, at least we can iterate much more quickly on the fix.

I have submitted a request for the GameLift team to take a look

P48992119

@David_St-Louis

We’ve checked on our end that 3.4.1 C++ SDK is working as expected, i.e. no hangs after initSDK. I’ll DM you the test app source code.

If you tried this and still have issues, could you please provide us some code snippet for your server executable?

I will try to build your unit tests using the same environement. Could my problem be because I don’t build it on Ubuntu, but I build it inside a AmazonLinux2 env directly? (I thought that would be the only way to build this)

Hi @JamesM_Aws ,

the code you gave me, are we sure we’re talking about the exact same SDK? I see you have code like this:

// Init AWS Sdk for usage
Aws::SDKOptions options;
Aws::InitAPI(options);

Which I don’t seem to have. I am supposed to call InitSDK first, and the documentation specifies it’s the first thing I should call.
I confirm that the version of the SDK I am building is: GameLift-Cpp-ServerSDK-3.4.1
In that SDK folder, I have no such file as Aws.h.

I’ll try to compare your CMakeLists.txt with mine and see if there is something I missed. Could that file be generated in any other way? I am building the SDK as per the documentation:

cmake3 -DCMAKE_BUILD_TYPE=Release VERBOSE=1 ..
make -j16

P.S.: I saw this InitAPI in the game example on github too, but I didn’t make much of it, thinking it was probably using a much older API.

(edit) It appears I am missing “AWSSDK”. I don’t recall seeing this as requirement, I knew I missed an important step. Trying to figure out how to install that one. Will marked this post as the solution if this works.

Aws::InitAPI is used to send testing data to SQS (to help us verify that players joined, game session created, etc.), it shouldn’t have anything to do with the iniSDK() issue that you are seeing.

You could really strip the code down to only calling initSDK() and processReady(), similar to aws-gamelift-sample/GameLiftManager.cpp at master · aws-samples/aws-gamelift-sample · GitHub

If your code is simple enough but initSDK() is still hanging, then it’s possible that your SDK is the root cause. Could you verify that you followed all the steps correctly in the README that came with the C++ SDK?

Hi James, Ok thanks. Good to know, so I only need server SDK.
I took a different approach this time. Simple project, 1 file and I am now building it in Ubuntu instead of directly on AmazonLinux2 EC2.

This is the simplest test I could do, and it still fails with:

A process unrecognized by GameLift is attempting to make GameLift Server SDK API calls. Are you using a process launcher?

I first build the Server SDK. Then I create a CMake for my code like so:

cmake_minimum_required(VERSION 3.0.0)
project(simple)
add_executable(${PROJECT_NAME} main.cpp)

# Setup includes and linked libraries
target_include_directories(${PROJECT_NAME} PUBLIC 
    ${CMAKE_SOURCE_DIR}/GameLift-Cpp-ServerSDK-3.4.1/build/prefix/include
)

set(GAMELIFT_BIN_DIR ${CMAKE_SOURCE_DIR}/GameLift-Cpp-ServerSDK-3.4.1/build/prefix/lib)
target_link_libraries(${PROJECT_NAME} PUBLIC 
    ${GAMELIFT_BIN_DIR}/libaws-cpp-sdk-gamelift-server.a
    ${GAMELIFT_BIN_DIR}/libprotobuf.a
    ${GAMELIFT_BIN_DIR}/libsioclient.a
    ${GAMELIFT_BIN_DIR}/libboost_date_time.a
    ${GAMELIFT_BIN_DIR}/libboost_random.a
    ${GAMELIFT_BIN_DIR}/libboost_system.a
)

The main.cpp file:

#include <stdio.h>
#include <aws/gamelift/server/GameLiftServerAPI.h>

#define PORT 9313
#define log(...) { printf(__VA_ARGS__); }

int main(int argc, char *argv[])
{
    log("Hello Gamelift!\n");

    // InitSDK
    Aws::GameLift::GenericOutcome initOutcome = Aws::GameLift::Server::InitSDK();
    if (!initOutcome.IsSuccess())
    {
        auto error = initOutcome.GetError();
        log("Failed to GameLift::InitSDK(): %s\n  %s\n  %s\n", 
            error.GetErrorName(), 
            error.GetErrorMessage(), 
            error.GetDefaultNameForErrorType(error.GetErrorType()));
        return 1;
    }

    // ProcessReady
    Aws::GameLift::Server::ProcessParameters processParameters(
        [](Aws::GameLift::Server::Model::GameSession gameSession, void* cbObject)
        { // onStartGameSession
            // game properties
        }, nullptr,

        [](Aws::GameLift::Server::Model::UpdateGameSession, void* cbObject)
        { // onUpdateGameSession
        }, nullptr,

        [](void* cbObject)
        { // onProcessTerminate
        }, nullptr,

        [](void* cbObject) -> bool
        { // onHealthCheck
            return true;
        }, nullptr,

        PORT, // port
        {} // logParameters
    );

    auto gamelift_outcome = Aws::GameLift::Server::ProcessReady(processParameters);
    if (!gamelift_outcome.IsSuccess())
    {
        auto error = gamelift_outcome.GetError();
        log("Failed to GameLift::ProcessReady(): %s\n", gamelift_outcome.GetError().GetErrorMessage());
        return 2;
    }

    // ProcessEnding
    Aws::GameLift::Server::ProcessEnding();
    return 0;
}

I have a feeling it’s not the code, but it’s a permission thing maybe?

Fleet ID: fleet-4a65212e-be37-4bfc-8f2c-6d817adf9e23

Side note:
I found a bug. While testing with gamelift local, calling InitSDK will return Aws::GameLift::GAMELIFT_ERROR_TYPE::ALREADY_INITIALIZED. IsSuccess is false before of this. I added an extra check:

        if (error.GetErrorType() != Aws::GameLift::GAMELIFT_ERROR_TYPE::ALREADY_INITIALIZED)

Is this normal? I am calling InitSDK once.

Ok, I think found the issue. It seems to be a bug in the SDK 3.4.1

Calling InitSDK will return Aws::GameLift::GAMELIFT_ERROR_TYPE::ALREADY_INITIALIZED . IsSuccess() returns false. I added an extra check:

if (error.GetErrorType() != Aws::GameLift::GAMELIFT_ERROR_TYPE::ALREADY_INITIALIZED)

Pretty much all SDK calls seem to behave this way. ProcessReady() also returns ALREADY_INITIALIZED as an error.

Adding this extra check fixed it for me it seems, and now my simple test fleet is “active”. I still need to make this work with my bigger project, but I think this was probably the issue here.

Marking this as the solution. But the SDK needs to be checked for this bug. I am building the SDK without STD.

Glad that you are unblocked. I’ll cut a ticket for the GameLift team to investigate.

Could you share your build process with us if you wouldn’t mind? E.g. commands to run CMAKE. Thanks!

P48992119

Sure.

All commands below are performed inside WSL (Ubuntu on windows)

  1. Created an empty folder for the new project
  2. Copied GameLift-Cpp_ServerSDK-3.4.1/ into it
  3. Build GameLift-Cpp_ServerSDK-3.4.1
cd GameLift-Cpp_ServerSDK-3.4.1
mkdir build
cd build
cmake ..
make
cd ../..
  1. Create main.cpp
#include <stdio.h>
#include <aws/gamelift/server/GameLiftServerAPI.h>

#define PORT 9313
#define log(...) { printf(__VA_ARGS__); }

int main(int argc, char *argv[])
{
    log("Hello Gamelift!\n");

    // InitSDK
    Aws::GameLift::GenericOutcome initOutcome = Aws::GameLift::Server::InitSDK();
    if (!initOutcome.IsSuccess())
    {
        auto error = initOutcome.GetError();
        log("Failed to GameLift::InitSDK(): %s\n  %s\n  %s\n", 
            error.GetErrorName(), 
            error.GetErrorMessage(), 
            error.GetDefaultNameForErrorType(error.GetErrorType()));
        return 1;
    }

    // ProcessReady
    Aws::GameLift::Server::ProcessParameters processParameters(
        [](Aws::GameLift::Server::Model::GameSession gameSession, void* cbObject)
        { // onStartGameSession
            // game properties
        }, nullptr,

        [](Aws::GameLift::Server::Model::UpdateGameSession, void* cbObject)
        { // onUpdateGameSession
        }, nullptr,

        [](void* cbObject)
        { // onProcessTerminate
        }, nullptr,

        [](void* cbObject) -> bool
        { // onHealthCheck
            return true;
        }, nullptr,

        PORT, // port
        {} // logParameters
    );

    auto gamelift_outcome = Aws::GameLift::Server::ProcessReady(processParameters);
    if (!gamelift_outcome.IsSuccess())
    {
        auto error = gamelift_outcome.GetError();
        log("Failed to GameLift::ProcessReady(): %s\n  %s\n  %s\n", 
            error.GetErrorName(), 
            error.GetErrorMessage(), 
            error.GetDefaultNameForErrorType(error.GetErrorType()));
        return 2;
    }

    // ProcessEnding
    Aws::GameLift::Server::ProcessEnding();
    return 0;
}
  1. Create CMakeLists.txt
cmake_minimum_required(VERSION 3.0.0)
project(simple)
add_executable(${PROJECT_NAME} main.cpp)

target_include_directories(${PROJECT_NAME} PUBLIC 
    ${CMAKE_SOURCE_DIR}/GameLift-Cpp-ServerSDK-3.4.1/build/prefix/include
)

set(GAMELIFT_BIN_DIR ${CMAKE_SOURCE_DIR}/GameLift-Cpp-ServerSDK-3.4.1/build/prefix/lib)
target_link_libraries(${PROJECT_NAME} PUBLIC 
    ${GAMELIFT_BIN_DIR}/libaws-cpp-sdk-gamelift-server.a
    ${GAMELIFT_BIN_DIR}/libprotobuf.a
    ${GAMELIFT_BIN_DIR}/libsioclient.a
    ${GAMELIFT_BIN_DIR}/libboost_date_time.a
    ${GAMELIFT_BIN_DIR}/libboost_random.a
    ${GAMELIFT_BIN_DIR}/libboost_system.a
)
  1. Build simple server
mkdir build
cd build
cmake ..
make
cd ..
  1. Copy executable to a finalbuild folder
mkdir finalbuild
cp ./build/simple ./finalbuild/
  1. Upload build
aws gamelift upload-build --name simple --build-version 1.0.0 --build-root ./finalbuild --operating-system AMAZON_LINUX_2 --region us-west-2

You will run into issues where you need openssl-1.0 for your SDK to build. And if you build this inside an EC2 Linux2, you need to use cmake3 command instead.

1 Like

Hi David-

I think I see what’s going on here, though I’m not positive on the nuances. I believe what’s happening is that the two methods you refer to are returning a GenericOutcome(nullptr). Then that is picking up the constructor for GenericOutcome(GameLiftError) instead of GenericOutcome(void*). GameLiftError is an enum and the first value (ordinal 0) is ALREADY_INITIALIZED. This lines up with what you’re seeing for ProcessReady, for which that return shouldn’t even be possible.

This is not really good coding convention on our side and I’ve cut a tracking item to ensure that we follow a better pattern (not relying on this sketchy implicit conversion path) in the next revision of the SDK. On your side, I suspect there could either be a compiler bug or incomplete C++ 11 support that treats nullptr as 0 instead of a pointer. Updating to a more recent compiler might fix that. If not, the workaround you’ve specified above should be harmless and sufficient.

1 Like