New 3rd Party SDK integration - CPPREST

I am trying to include cpprest sdk (https://github.com/Microsoft/cpprestsdk/) into my project. Can somebody please advice me what is the best way to achieve this?

I tried adding it into 3rdParty folder and configuring it in SetupAssistantConfig.json. I used the example file to guide me, but after I open Setup Assistant I get an error saying cpprest requires source. Here is what I added into the json

    "SDKs" :
[
{ "comment" : "CppREST",
"identifier" : "cpprest",
"name" : "cppREST",
"version" : "2.10",
"optional" : 0,
"description" : "cpp sdk for connecting with REST API",
"detailedInstructions" : "The C++ REST SDK is a Microsoft project for cloud-based client-server communication in native code using a modern asynchronous C++ API design. This project aims to help C++ developers connect to and interact with services.",
"tags" : ["compilegame", "compileengine", "compilesandbox"],
"symlinks" :
[
{
"source" : "cpprest",
"destination" : "$CODEFOLDERNAME$/cpprest",
"exampleFile" : "cpprest/http_client.h"
}
]
}
],

To be honest I am not sure what to put into the symlinks so these are just guesses. I tried adding ‘source’ parameter into the main structure of the SDK item which made the Setup Assistant open, but cpprest wasn’t there.

I also tried adding into include directories of my cpp solution, but even though it makes the code look ok and I can access the libraries, it doesn’t compile (because it’s not in the waf file). It tells me it cannot find the included files when I build it.

I also tried adding the wscript file in ‘include’ parameter as another item. After building, it doesn’t error on the includes, but gives me warnings treated as error about no overrides for virtual members.

Severity	Code	Description	Project	File	Line	Suppression State
Warning	C4266	'Concurrency::task<size_t> Concurrency::streams::details::streambuf_state_manager<unsigned char>::_putn(const _CharType *,size_t,bool)': no override available for virtual member function from base 'Concurrency::streams::details::streambuf_state_manager<unsigned char>'; function is hidden	_WAF_	c:\BitBucket\vcpkg\installed\x64-windows\include\cpprest\containerstream.h	496

I for fact know that the library is working as I am using it in different (not Lumberyard) projects.

I tried following documentation on Amazon website, but most of the stuff are old and even the ones presented as current are missing bits and are simply not working.

I would hugely appreciate any kind of help/pointers guys.

One thing that might be useful here is to make CppRest part of a gem 3rd Party instead of a global 3rd Party. If a 3rd Party library is only used in reference to a particular gem, then it needs to be located entirely within the gem’s folder. It won’t need to be setup in the setup assistant configuration since it is not managed there.

Gem 3rd Party Advantages

  •  Promotes better modularity. Won’t clutter up the
    

    engine with 3rd party libraries unless it is actually activated within a
    game project

  •  Version control for the library is maintained by the
    

    gem’s version control, not the engine. 3rd Party library updates
    will require a version update for the gem.

  • Allows easier integration for 3rd party Gems.
    Gem 3rd Party Disadvantages

  •  Without central management, conflicts may occur if
    

    different gems use the same 3rd party library

  •   Problem will surface in
    

    monolithic builds. Normally gems protect themselves with unique
    namespaces when building monolithically, but we don’t have that control
    over 3rd party libraries (without modification)
    Global 3rd Party Library

If you want to keep this a global 3rd Party to be used everywhere, then you’ll also need to be sure to add a JSON into dev/WAF/3rdParty folder; have you done this step yet? I can help step through that part if you’d like :slight_smile:

As for the symlink section in SetupAssistantConfig, this will be the folder path relative to 3rdParty containing the version of CppRest you want to use. For example, suppose your folder structure looks something like:

   /3rdParty /CppRest /v1.0.0
/vc140
/some_file.lib /vc120
/some_other_file.cpp
/v1.5.1
/vc140
/some_file.lib /vc120
/some_other_file.cpp

If you want to use version 1.5.1 then “source” : “CppRest/v1.5.1”

“exampleFile” is a used as a small validation to make sure the contents of cpprest actually exists where they ought to exist. “exampleFile” should be a path relative to “source” to any file such as…

“exampleFile” : “vc140/some_file.lib”

One you’ve configured everything correctly, the wscript which requires CppRest won’t need to add any “includes”, but you’ll have something like… “uselib = [‘CPPREST’]” Uselib will automatically bring in the includes which you’ve defined inside of dev/WAF/3rdParty/CppRest.json.

Let me know if anything comes up!

I’ve discussed this a bit more with a resident expert and they recommend adding the 3rd Party to the Gem you’re building if possible. Here’s a breakdown of how to add a Gem 3rd Party Library, hope it helps!

Add a Gem 3rd Party Library

The gem module will look in a specific folder, 3rdParty, for the 3rd party sdk definition file. Consider the following folder structure:

\MyGem
+---\3rdParty
+---\Assets
+---\Code
| +---wscript
+---\External
+---gem.json

The 3rd party library definition file(s) are required to be located in the \3rdParty folder. The actual location of library can be anywhere, but its recommended that it be placed within the Gem’s folder structure, particularly under an ‘External’ folder (with a folder for the SDK as a subfolder under this).

The following steps will assume we are adding 3rd party library ‘SuperLibrary’ to a gem called ‘MyGem’. SuperLibrary will be a static library that support windows and mac.

Step 1. Determine the 3rd party library location

The first step is to choose the location where to copy the third party library. For this example, we will place the ‘SuperLibrary’ folder underneath the ‘External’ subfolder described above.

    |
+---\External
+---\SuperLibrary
+---\includes
+---superlibrary.h
+---\lib
+---\win32
+----superlibrary.lib
+---\osx
+----superlibrary.a

Step 2. Create a 3rd party definition file

The next step to to create a 3rd party definition json file. Based on the folder structure from step 1, the 3rd party definition file (superlibrary.json) may look like this

	{
"name": "SUPERLIBRARY",
"source": "@GEM@/External/SuperLibrary",
"description": "My Super Library",
"includes": [
"includes"
],
"defines": [],
"lib_required": "True",
"platform": {
"win_x64_vs2015": {
"libpath": [
"lib/win32"
],
"lib": [
"superlibrary.lib"
]
},
"win_x64_vs2017": "@win_x64_vs2015",
"darwin_x64": {
"libpath": [
"lib/osx"
],
"lib": [
"superlibrary.a" ]
}
}

Note that there is an alias above called ‘@GEM@’. This alias will represent to root path of the current Gem. The ‘source’ key represents the base of the 3rd party library folder, from which the ‘includes’, ‘libpath’, etc are based from.

Step 3. Apply the library to modules/gems that need it

By default, once the 3rd party library definition is set in the Gem, it is automatically available to that gem. In order for this gem to be available in other modules, those modules need to add ‘SUPERLIBRARY’ to its ‘uselib’ list. Its important to note that this scenario is meant for Gem-scoped 3rd party libraries, ie engine modules should not rely on it. If there is a need for an engine library to consume a 3rd party library, then it needs to follow the ‘Adding a 3rd Party Library to the Engine’ section.

In order for other gems (or the game gem) to have access to this Gem-scoped 3rd party library, it needs to establish a Gem dependency on it. For example, if we want another Gem called ‘OurGem’ to have access to the new SuperLibrary library, it will need to define its dependency in its gem.json file.

OurGem.gem.json

 	{
"Dependencies": [
{
"Uuid": "981435f1646a4ccfbfd7733920c011b6",
"VersionConstraints": [
"~>0.1"
],
"_comment": "MyGem"
}
],
"GemFormatVersion": 3,
"Uuid": "1daafa0eaa544f64befb74a1cc719a9c",
"Name": "OurGem",
"DisplayName": "Our Gem",
"Version": "0.1.0",
"LinkType": "Dynamic",
"Summary": "Our Gem",
"Tags": ["Animation"],
"IconPath": "OurGem.png",
"EditorModule" : true
}

And then in the wscript for OurGem:

OurGem/Code/wscript

    def build(bld):
bld.DefineGem(
uselib = ['SUPERLIBRARY'],
file_list = ['our_gem.waf_files']
)

Step 4. Configure and validating its definition

Once everything is set properly, its time to test it. 3rd party definitions are processed during the ‘configure’ WAF command. Before starting, make sure that the gem is enabled for the enabled game projects. First run the configure command from the engine root path

>lmbr_waf configure

If configure completes successfully, you will see the configured values in the waf variant cache files under BinTemp/c4che. The following is an example of the new entries in the variant cache file for visual studio 2015 with the profile configuration:

win_x64_vs2015_profile_cache.py

...
INCLUDES_SUPERLIBRARY = ['C:\\MyProj\\Lumberyard-1.16\\Gems\\MyGem\\External\\Includes']
...
STLIBPATH_SUPERLIBRARY = ['C:\\MyProj\\Lumberyard-1.16\\Gems\\MyGem\\External\\lib\\win32']
...
STLIB_SUPERLIBRARY = ['superlibrary']
...

Thank you @gFunc, I managed to add the library into the Gem which is a better solution for me anyway. However, this only revealed that my previous warnings and errors were not caused by wrong linking, but something else.

I am still getting the following warnings and errors when I try to build my project:

couple of these warnings:

SeverityCodeDescriptionProjectFileLineSuppression State
WarningC4266'Concurrency::task<size_t> Concurrency::streams::details::streambuf_state_manager<char>::_putn(const _CharType *,size_t,bool)': no override available for virtual member function from base 'Concurrency::streams::details::streambuf_state_manager<char>'; function is hidden_WAF_c:\Amazon\Lumberyard\1.15.0.0\dev\ExplorationEmpty\Gem\3rdParty\cpprest\include\cpprest\containerstream.h496

and couple of these warnings:

SeverityCodeDescriptionProjectFileLineSuppression State
WarningC4266'web::json::value &web::json::details::_Value::index(web::json::array::size_type)': no override available for virtual member function from base 'web::json::details::_Value'; function is hidden_WAF_c:\Amazon\Lumberyard\1.15.0.0\dev\ExplorationEmpty\Gem\3rdParty\cpprest\include\cpprest\json.h1812

and then I get this error:

SeverityCodeDescriptionProjectFileLineSuppression State
ErrorC2220warning treated as error - no 'object' file generated_WAF_c:\Amazon\Lumberyard\1.15.0.0\dev\ExplorationEmpty\Gem\3rdParty\cpprest\include\cpprest\json.h1812

As stated previously, I use the exact same library in different projects outside LY and it works correctly. Do you have any idea what I might be missing here please?

Here is my cpprest.json located in myProject/Gem/3rdParty folder. The library is placed in the same folder as indicated in the source parameter:

    {
"name": "CPPREST",
"source": "@GEM@/3rdParty/cpprest",
"description": "cpprest SDK",
"includes": [
"include"
],
"lib_required": "True",
"platform": {
"win_x64_vs2015": {
"importlibpath": ["lib"],
"sharedlibpath": ["debug/bin"],
"import_debug": [
"cpprest_2_10.lib"
],
"import_release": [
"cpprest_2_10.lib"
],
"shared_debug": [
"cpprest_2_10d.dll"
],
"shared_release": [
"../../bin/cpprest_2_10.dll"
]
},
"win_x64_vs2017": "@win_x64_vs2015"
}
}

Also, when I run command

lmbr_waf configure

it creates only LumberyardSDK_vs15.sln, but not the solution for the project I am working on. This is one of the reasons why I was confused by the 3rdParty library inclusion. The project solution is marking it as not found (it’s not in the WAF file), yet in the LumberyardSDK_vs15.sln it’s linking it fine and I can see the library in the WAF file. Is there a way of building project solutions and not just the main LubmberyardSDK solution?

Sorry, I posted this last week, but the forums rejected it as spam, ha!

When you create the .SLN file using lmbr_waf configure, it doesn’t include the Launcher for the project you’re working on? In ProjectConfigurator.exe have you set the your project to the default project? After setting your project as the default, and re-running lmbr_waf configure, your SLN should contain your game project.

Thank you gFunc, that was a very thorough answer. I checked everything I was doing against it and I didn’t miss anything. The library seemed to be linked properly and loading, but I was still getting the errors. I focused on a different part of the project but in a meanwhile I got an answer from developers of the CPPREST SDK. Apparently the issue was on their end, they are working on some fixes, but for the moment I have to include pragma warning push/pop around cpprestsdk headers. In case anybody else gets stuck on it as well:

#pragma warning( push )
#pragma warning( disable : 4266 ) #include <cpprest/filestream.h>
#include <cpprest/http_client.h>
#pragma warning( pop )