Timeout trying to connect to RealTime server with certificate generation enabled

Kip · February 7, 2020, 10:57pm

What else must be done to get WSS working? I’ve enabled certificate generation on the fleet, but I can’t seem to open a connection. Any insight into what I might be missing?

Here are the server logs as it starts up:

[INFO] (rt-logger.js) 59: Game session (PID: 13531) configured with logger: {"logDirPath":"/local/game/logs/13531","logLevelFilter":"*.info","filename":"server.log"}
[INFO] (index.js) 123: Game server was constructed with {"test_dist":false,"script":"../../../local/game/etag-005d5904bf82113cb0e612377793a8ab/src/server.js","port":1900,"end":2000,"udp_port":33400,"udp_port_end":33500,"devargs":"dev","enable_security":true}
[INFO] (gamelift.js) 195: Starting GameLift Realtime server process. PID: 13531...
[INFO] (gamelift.js) 200: Calling GameLiftServerAPI.SdkVersion...
[INFO] (gamelift.js) 205: GameLiftServerAPI.SdkVersion succeeded with result: 3.3.0
[INFO] (gamelift.js) 208: Calling GameLiftServerAPI.InitSDK...
[INFO] (gamelift.js) 213: GameLiftServerAPI.InitSDK succeeded
[INFO] (gamelift.js) 216: Waiting for Realtime server to start...
[INFO] (index.js) 149: Loading game server script at path: /local/game/etag-005d5904bf82113cb0e612377793a8ab/src/server.js
[INFO] (index.js) 177: Fetching credentials to establish secure connections...
[INFO] (gamelift.js) 170: Calling GameLiftServerAPI.GetInstanceCertificate
[INFO] (gamelift.js) 175: GameLiftServerAPI.DescribePlayerSessions succeeded with optional result: [object Object]
[INFO] (index.js) 185: Credential paths fetched: {"CertificatePath":"/local/gamemetadata/certificates/certificate.pem","PrivateKeyPath":"/local/gamemetadata/certificates/privateKey.pem","CertificateChainPath":"/local/gamemetadata/certificates/certificateChain.pem","HostName":"u2fp3zjxdovmooutoxufgldo641gkzfay986yrvaejfoyq5fsmv3w04sl24cfqd.whji3yx30fjwq55416k67pyobo9oi02c.us-west-2.amazongamelift.com","RootCertificatePath":"/local/gamemetadata/certificates/rootCertificate.pem"}
[INFO] (index.js) 202: Parsed credentials signed for hostname: u2fp3zjxdovmooutoxufgldo641gkzfay986yrvaejfoyq5fsmv3w04sl24cfqd.whji3yx30fjwq55416k67pyobo9oi02c.us-west-2.amazongamelift.com
[INFO] (index.js) 209: Initializing Realtime server event handlers...
[INFO] (dtls.js) 32: Creating DTLS server...
[INFO] (ws.js) 47: Creating WebSocket server over HTTPS...
[INFO] (ws.js) 64: Available cipher suite on host: aes128-gcm-sha256,aes128-sha,aes128-sha256,aes256-gcm-sha384,aes256-sha,aes256-sha256,dhe-psk-aes128-cbc-sha,dhe-psk-aes128-cbc-sha256,dhe-psk-aes128-gcm-sha256,dhe-psk-aes256-cbc-sha,dhe-psk-aes256-cbc-sha384,dhe-psk-aes256-gcm-sha384,dhe-psk-chacha20-poly1305,dhe-rsa-aes128-gcm-sha256,dhe-rsa-aes128-sha,dhe-rsa-aes128-sha256,dhe-rsa-aes256-gcm-sha384,dhe-rsa-aes256-sha,dhe-rsa-aes256-sha256,dhe-rsa-chacha20-poly1305,ecdhe-ecdsa-aes128-gcm-sha256,ecdhe-ecdsa-aes128-sha,ecdhe-ecdsa-aes128-sha256,ecdhe-ecdsa-aes256-gcm-sha384,ecdhe-ecdsa-aes256-sha,ecdhe-ecdsa-aes256-sha384,ecdhe-ecdsa-chacha20-poly1305,ecdhe-psk-aes128-cbc-sha,ecdhe-psk-aes128-cbc-sha256,ecdhe-psk-aes256-cbc-sha,ecdhe-psk-aes256-cbc-sha384,ecdhe-psk-chacha20-poly1305,ecdhe-rsa-aes128-gcm-sha256,ecdhe-rsa-aes128-sha,ecdhe-rsa-aes128-sha256,ecdhe-rsa-aes256-gcm-sha384,ecdhe-rsa-aes256-sha,ecdhe-rsa-aes256-sha384,ecdhe-rsa-chacha20-poly1305,psk-aes128-cbc-sha,psk-aes128-cbc-sha256,psk-aes128-gcm-sha256,psk-aes256-cbc-sha,psk-aes256-cbc-sha384,psk-aes256-gcm-sha384,psk-chacha20-poly1305,rsa-psk-aes128-cbc-sha,rsa-psk-aes128-cbc-sha256,rsa-psk-aes128-gcm-sha256,rsa-psk-aes256-cbc-sha,rsa-psk-aes256-cbc-sha384,rsa-psk-aes256-gcm-sha384,rsa-psk-chacha20-poly1305,srp-aes-128-cbc-sha,srp-aes-256-cbc-sha,srp-rsa-aes-128-cbc-sha,srp-rsa-aes-256-cbc-sha
[INFO] (ws.js) 89: Using secure options for websocket: {"ciphers":"TLS_AES_256_GCM_SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384","minVersion":"TLSv1.2","dhparam":"2048"}
[INFO] (index.js) 223: Initializing UDP connector...
[INFO] (index.js) 129: Attempting to open port 33400...
[INFO] (dtls.js) 56: DTLS server listening on 0.0.0.0:33400
[INFO] (index.js) 229: Initializing TCP connector...
[INFO] (index.js) 129: Attempting to open port 1900...
[INFO] (ws.js) 163: HTTP/S server listening on port: 1900
[INFO] (server.js) 605: Ready to host games...
[INFO] (index.js) 241: onProcessStarted success. Process ready for games.
[INFO] (index.js) 261: Game session initialized with port: 1900
[INFO] (gamelift.js) 223: Realtime server started! Calling GameLiftServerAPI.ProcessReady with processParameters: {"Port":1900,"LogParameters":{"LogPaths":["/local/game/logs/13531"]}}
[INFO] (gamelift.js) 229: Process advertised to AuxProxy! GameLiftServerAPI.ProcessReady succeeded
[INFO] (gamelift.js) 231: GameLift Realtime server process started successfully.

Edit:

The GameLift service is also continuously complaining about this fleet:

SERVER_PROCESS_SDK_INITIALIZATION_TIMEOUT
Server process started correctly but did not call InitSDK() within 5 minutes, launchPath(/local/NodeJS/bin/node), arguments(-- /local/game/src/gamelift.js --enable-security --script ../../../local/game/etag-005d5904bf82113cb0e612377793a8ab/src/server.js --devargs "dev"), instanceId(i-09f6e5e76f757c572)

Edit:

Connecting to an instance without certificate generation enabled:

$ telnet 35.166.244.249 1901
Trying 35.166.244.249...
Connected to ec2-35-166-244-249.us-west-2.compute.amazonaws.com.
Escape character is '^]'.

Trying to connect to an instance with certificate generation enabled:

$ telnet 34.223.223.14 1900
Trying 34.223.223.14...

I can, however, connect to the secure fleet from another fleet:

[gl-user-remote@ip-10-172-206-82 ~]$ telnet 34.223.223.14 1900
Trying 34.223.223.14...
Connected to 34.223.223.14.
Escape character is '^]'.

Do I need to configure some security groups or something with the secure fleet?

Pip · February 10, 2020, 6:34pm

I am sorry you are having problems with this.

I’m going to let the GameLift team know, to see if they have any further insight.

I would though:

Make sure your Client side SDK is up-to-date to support secure fleets as I believe there is some handshake code required

The bigger worry is this warning:

Server process started correctly but did not call InitSDK() within 5 minutes, launchPath(/local/NodeJS/bin/node), arguments(-- /local/game/src/gamelift.js --enable-security --script ../../../local/game/etag-005d5904bf82113cb0e612377793a8ab/src/server.js --devargs "dev"), instanceId(i-09f6e5e76f757c572)

This means that GameLift will probably terminate your instances, this looks like you have a bug in your server script causing it fail. This may be the reason you can’t connect connect to your secure fleet.

I would:

Double check the documentation
Lint/validate your scripts
Simplify your script until you have a working fleet and then add in your logic in small sections.

Kip · February 12, 2020, 11:06pm

Any new information on this? We have a functional fleet without certificate generation enabled.

I tried launching a secure fleet with a minimal, functional script. I’ve even attempted it with this example script from the Realtime docs – just to see:

/*
* All or portions of this file Copyright (c) Amazon.com, Inc. or its affiliates or
* its licensors.
*
* All use of this software is governed by the terms and conditions governing AWS
* Content in the AWS Customer Agreement at aws.amazon.com/agreement. Do not
* remove or modify any license notices. This file is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
*
*/

// Example minimal server file with no overridden callbacks or configuration

var gameSession;

// Called when game server is initialized, is passed server object of current session
function init(session) {
    gameSession = session;
}

exports.ssExports = {
    init: init
};

Unfortunately, the server does not respond and any connection attempt times out.

As I said previously, I am able to establish a connection from one fleet instance to another fleet instance. So, something is working in that regard. That’s why I asked about security groups.

We may need to set up a proxy server as a workaround for this issue, but I was hoping all the GameLift certificate generation / SSL bits would just work.

Looking forward to any insight on this issue, thanks!

Pip · February 20, 2020, 8:02pm

Apologies, I forgot to follow up with you.

IMHO The minimal script isn’t that useful, as it will get you an active fleet but does very little. You ideally want something that logs around major lifecycle events including player connections.

The two obvious issues are that:

Server is listening on a port that you didn’t open in the gamelift fleet, but that seems unlikely because you’ve gotten non tls version working.
Your use of telnet to test connections is interesting (I had assumed that Telnet didn’t support TLS but as it sort of works for you it could be a root ca issue when the TLS handshake is timing out and failing.
- BTW Have you tried other tools such as telnet-ssh and ```openssl s_client -connect :<993> to connect?
- EC2 instances may have different telnet client than yours/version so that could be why they work. They also have all the right rootCAs installed from the beginning for ACM TLS certs.

If you’re still having problems could you provide fleet ids + region of the realtime fleets you couldn’t connect to? Can ask the GameLift service team to investigate this particular failure.