Lmbr_waf, cofigured parameters or commandline parameters not used

Hello again,
I use Visual Studio 2019 to build the project. One thing that is annoying is, I use lmbr_waf configure with solution name, output folder and extension, I can see that in the project settings of Visual Studio the Output Path is set to the desired path, but compilated files go in the Bin64vc142. I assume the configured parameter is not used if the build is startet from Visual Studio itself. The --max_cores parameter is also not used, but with the max cores it doesn’t matter if it is set in the user settings or command line. Incredibuild also not works. I think there is more, but these are the most annoying at the moment.

Greetings,
Joerg

Maybe I missed something and someone with more knowledege could give advise. I wanted to create a gem for my Multiplayer Code, but every change in Source Code of Gem, requires at least 30% of new build of the entire Game, Engine and Editor. This is not acceptable. Its more time compiling than coding. I had the impression that the gems are dynamic linked libraries, that are dynamic linked in. But I have the feeling that the libraries are statically linked. Someone has more insight and can explain this a bit?

Joerg

Hi Joerg,

Hmm that definitely doesn’t sound right to me. If a Gem is working as it’s supposed to, changes in it should be pretty quick (changing a .cpp. file and then compiling/linking should be a minute or so).

Could you share any more information like your wscript file that describes your Gem? There might be a gotcha you’re running into.

If you take a look at the White Box Gem wscript file as an example it’s pretty simple and might have some clues.

Let me know if you can share the wscript file and I’ll see if I can offer any suggestions :+1:

Cheers!

Tom

Hello,
it’s not changing something and compile, this is not fast, but I could life with it (you see in CryEngine tutorial how fast they compile?). I talk about changing project, in my case to multiplayersample to debug. I had to recompile most of multiplayersample and going back to my project i must recompile again. Even trying to build the projects in different folders doesn’t work most of the time. I think there are dependencies in BinTemp or cache.

Hi @masm32,

Okay I understand. Yes changing projects can lead to longer build times unfortunately depending on what Gems get enabled/disabled (if there’s a big delta it’ll take a lot longer). For a lot of game teams switching projects is not a super common operation (usually one is selected and then used for most things) so this is likely an area that just hasn’t had as much focus to optimize and improve for build times.

Switching between Debug and Release also means everything has to be rebuilt which is unfortunately unavoidable (though not every single time you switch).

The CMake conversion is still well underway (as mentioned in the 1.23 Release Notes) and should hopefully improve some of these build artifact issues and provide more flexibility around build folders, but it’s still a little way off I’m afraid.

One thing you could try to improve build times is right clicking on the Editor project in the Visual Studio solution, and the going to the Project only option and try just building Editor. Hopefully that should actually build less things than just building the whole solution. It might not make much difference but worth a shot!

You can also try enabling Uber files too which might help speed things up.

Sorry not to be more help, I hope you can find something that works for you.

1 Like

Hello,

You are right, changing projects is for a dev team not really required, but changing the build configuration is a common thing to do. I want to debug code in debug build, but do prefer to work with the editor in profile.
You mentioned that build time depends on Gems that are enabled, but my understanding was that Gems are linked dynamically into the game, so switching from my project with more Gems enabled causes a rebuild, even the MultiplayerSolution has not as much Gems. It is clear that Gems that are not in my project have to be build, but it makes no sense to build them every time. This should only happen if they are statically linked.
Even using the system for different folders for debug, profile or release changes not much for rebuilds, so this is mostly useless.

I hope that cmake will change this a bit. By the way, I activated uber files.

Joerg

Hi Joerg,

You are totally right about changing between Debug and Profile. I often find myself in the exact same predicament. One trick I use regularly is to put:

#pragma optimize("", off) 
#pragma inline_depth(0)

at the top of a .cpp file. This will disable optimizations for this file only, and makes debugging a lot easier.

Note: With Uber files turned on, you might want to put…

#pragma optimize("", on)
#pragma inline_depth()

at the bottom of the file, otherwise disabling optimizations will spill over into other files in that one Uber file (which you might not want). You still have to recompile which is a bit of a nuisance but it should be much faster if you want to debug something in Profile and don’t want to build the whole Debug configuration.

You’re right, Gems are linked dynamically. I think enabling and disabling Gems within a project is usually fairly fast but when changing projects (perhaps due to the build system), whatever cache that existed before can’t be used. I have to admit I’m not as familiar with this but I totally understand your frustration that (in theory at least) what you’re describing should be possible. There’s likely some fly in the ointment that’s contributing to more being rebuilt than should be.

The CMake team has made some huge strides and we hope we can share what we’ve been working on before too long.

Thanks for your understanding and I hope the optimize trick might come in handy!

Tom

Hi Tom,

thank you for your kind reply. The frustration level is high working with Lumberyard. The problem here is the lack of in depth documentation (or I have not found it yet), so learning all things digging through source code and mostly doing try and error.

If I understand you right, you use the preprocessor flags to deactivate optimization and debug your code in profile configuration? I have to test this.

1 Like

Hi Joerg (I do apologize for missing the r before :see_no_evil: - I’ve now corrected it!)

Not at all, and yes I do know the feeling :slight_smile: 100%, we are trying to improve the docs and expand on them, it’s just unfortunately quite a slow and painstaking process (especially with things changing as fast as they do too). We have two internal pages for our team, one called ‘Quirkflows and Gotchas’ and another called ‘General Tips and Tricks’ which we often point people towards. It might be worth us trying to clean them up a bit and make them publicly available (it’s always easier said than done but I’ll reach out and see).

That’s absolutely right yes - the useful thing about this is you just disable the optimizations for your code, so the Editor/Game still runs fast but you can step through your code. The inline_depth directive is useful to stop lambdas getting inlined.

Let me know if you have any luck with that!

Thanks,

Tom

Thank you Tom,

do you know how the number of max-cores parameter is used, because it would be nice if my system keeps more responsible if I compile the engine again.

No worries at all! Glad to be of some help :slight_smile:

Hmm, that’s a good question. I do think it should be respected, so the fewer number of cores you specify the less will be used for compilation (this should give you some responsiveness back). The trade-off of course being the fewer you have the longer your code will take to compile :weary:

Hopefully, you can find a reasonable balance between the two. You can try the classic halve it/double it (with respect to the number) approach to see if there’s a big difference in either compile-time or responsiveness. There’s also a little bit of info about the parameter on this page - https://docs.aws.amazon.com/lumberyard/latest/userguide/waf-user-options-and-settings.html

Cheers!

Tom

Hello,
this is the problem, even if I change the number of max-cores in user settings or commandline parameter it is not respected. It uses 8 cores on my system. Its the same with IncrediBuild. I have a personal license to use IncrediBuild on my system, but if I set it in UserSettings to True it is not used. If I compile other Visual Studio projects IncrediBuild is used.

Hmm I see… I’m not sure what would be causing that I’m afraid.

With Incredibuild make sure you build by using the normal build command in Visual Studio, not the Incredibuild option. Waf will invoke Incredibuild when using the normal build command.

There’s a chance something broke with max-cores at some point, I’m afraid I can’t say for sure. I don’t think anyone is going to have a chance to look at this in the near future, hopefully you can work around it for now. I’ll check with the build team if they have any suggestions.

Hello Tom,

thank you. I always build with the build command from Visual Studio. So at the moment I have to live with it.

Joerg

1 Like

Hi @masm32

Someone on the team mentioned you might be able to improve the situation with your CPUs by tweaking the settings directly in Incredibuild (see the image below)

IMG_20200916_213846_253

Try setting it to one less than your number of CPUs

Hi Tom,

somehow I never send my answer, so I have to apologize.
I got this message: 1>Required Make && Build Tools Package not found. Build will not be accelerated through Incredibuild

I have installed IncrediBuild, but Lumberyard Documentation says that it searches in x86 Program Folder. I have installed the x64 Version of IncrediBuild. If you know where to change the path to the IncrediBuild tools it would be nice. If not I have to search if I got the time.

Joerg

Hi Joerg,

No worries, sorry I’m just getting back to you.

Hmm, that sounds like Incredibuild isn’t being used. I think the safest bet is probably to uninstall and reinstall in the correct location. I’ll double-check my machine tomorrow to see where it is (I had to recently uninstall and reinstall Incredibuild as my Build Monitor stopped displaying correctly :see_no_evil:, fortunately, that did the trick!). It was pretty fast so I’d recommend giving that a whirl.

Thanks,

Tom