Compiling time tests on code tiny changes : Ly 1.20 vs 1.22

fluffy · January 15, 2020, 7:01pm

Hi Lumberyard devs!
I want to take your attention on one very importat thing for me on the - compiling time.
I made little test for you, so you can see the difference on compiling same project with using difference versions of engine.

First test: Lumberyard 1.20 + my GameProject + VS2017Community 15.9.19

1>------ Build started: Project: _WAF_, Configuration: [Game and Engine] profile x64 ------
1>[WAF] Engine Root: c:\Lumberyard\dev\
1>[WAF] Executing 'build_win_x64_vs2017_profile' in 'c:\Lumberyard\dev\BinTemp\win_x64_vs2017_profile'
1>[INFO] Regenerating 3rd Party settings file...
1>[WARN] Missing JDK path from Setup Assistant detected. Target platforms that require java will be disabled. Please re-run Setup Assistant with "Compile For Android" enabled and run the configure command again.
1>[SETTINGS] msvs_version = 15 (default 14)
1>[SETTINGS] Target Output folder (win_x64_vs2017/profile): Bin64vc141
1>[WARNING] Incredibuild disabled by build option
1>[WARN] Invalid module 'LyIdentity_shared' in spec 'game_and_engine'
1>[WARN] Invalid module 'LyIdentity_static' in spec 'game_and_engine'
1>[WARN] Module use dependency 'CrashHandler' for target 'CrySystem' refers to an invalid module
1>[WARN] Module use dependency 'ToolsCrashHandler' for target 'EditorLib' refers to an invalid module
1>Cannot copy source 'C:\Lumberyard\3rdParty\PVRTexTool\2016_r1.1\PVRTexLib_License.txt' File not found.  Skipping.
1>[3836/5992] cxx (win_x64_vs2017|profile): GameProject\Gem\Code\Source\Components\RTSCameraComponent.cpp -> BinTemp\win_x64_vs2017_profile\GameProject\Gem\Code\Source\Components\RTSCameraComponent.cpp.1363050.obj
1>[5903/5992] cxxshlib (win_x64_vs2017|profile): BinTemp\win_x64_vs2017_profile\GameProject\Gem\Code\GameProject_rc\GameProject.auto_gen.1363050.res BinTemp\win_x64_vs2017_profile\GameProject\Gem\Code\Source\Components\GameCursorComponent.cpp.1363050.obj BinTemp\win_x64_vs2017_profile\GameProject\Gem\Code\Source\Components\RTSCameraComponent.cpp.1363050.obj BinTemp\win_x64_vs2017_profile\GameProject\Gem\Code\Source\Components\SimpleFPSController.cpp.1363050.obj BinTemp\win_x64_vs2017_profile\GameProject\Gem\Code\Source\Components\VisibleTagComponent.cpp.1363050.obj BinTemp\win_x64_vs2017_profile\GameProject\Gem\Code\Source\GameProjectModule.cpp.1363050.obj BinTemp\win_x64_vs2017_profile\GameProject\Gem\Code\Source\GameProjectSystemComponent.cpp.1363050.obj BinTemp\win_x64_vs2017_profile\GameProject\Gem\Code\Source\GameProject_precompiled.1363050.obj -> Bin64vc141\Gem.GameProject.ee43c3a070a94d369c664215508971cf.v0.1.0.dll Bin64vc141\Gem.GameProject.ee43c3a070a94d369c664215508971cf.v0.1.0.pdb Bin64vc141\Gem.GameProject.ee43c3a070a94d369c664215508971cf.v0.1.0.dll.manifest
1>   Creating library c:\Lumberyard\dev\Bin64vc141\Gem.GameProject.ee43c3a070a94d369c664215508971cf.v0.1.0.lib and object c:\Lumberyard\dev\Bin64vc141\Gem.GameProject.ee43c3a070a94d369c664215508971cf.v0.1.0.exp
1>[WAF] 'build_win_x64_vs2017_profile' finished successfully (18.676s)
1>[WAF] Executing 'package_win_x64_vs2017_profile'
1>[WAF] 'package_win_x64_vs2017_profile' finished successfully (1.613s)
1>[WAF] Executing 'deploy_win_x64_vs2017_profile'
1>[INFO] Regenerating 3rd Party settings file...
1>[WAF] 'deploy_win_x64_vs2017_profile' finished successfully (1.658s)
2>------ Build started: Project: GameProject (GameProject\GameProject), Configuration: [Game and Engine] profile x64 ------
2>[WAF] Engine Root: c:\Lumberyard\dev\
2>[WAF] Executing 'build_win_x64_vs2017_profile' in 'c:\Lumberyard\dev\BinTemp\win_x64_vs2017_profile'
2>[INFO] Regenerating 3rd Party settings file...
2>[WARN] Missing JDK path from Setup Assistant detected. Target platforms that require java will be disabled. Please re-run Setup Assistant with "Compile For Android" enabled and run the configure command again.
2>[SETTINGS] msvs_version = 15 (default 14)
2>[SETTINGS] Target Output folder (win_x64_vs2017/profile): Bin64vc141
2>[WARNING] Incredibuild disabled by build option
2>[WARN] Invalid module 'LyIdentity_shared' in spec 'game_and_engine'
2>[WARN] Invalid module 'LyIdentity_static' in spec 'game_and_engine'
2>[WARN] Module use dependency 'CrashHandler' for target 'CrySystem' refers to an invalid module
2>[WARN] Module use dependency 'ToolsCrashHandler' for target 'EditorLib' refers to an invalid module
2>[WAF] 'build_win_x64_vs2017_profile' finished successfully (2.958s)
2>[WAF] Executing 'package_win_x64_vs2017_profile'
2>[WAF] 'package_win_x64_vs2017_profile' finished successfully (1.603s)
2>[WAF] Executing 'deploy_win_x64_vs2017_profile'
2>[INFO] Regenerating 3rd Party settings file...
2>[WAF] 'deploy_win_x64_vs2017_profile' finished successfully (1.651s)
========== Build: 2 succeeded, 0 failed, 0 up-to-date, 0 skipped ==========
TOTAL COMPILE TIME = 18.6+3 ~ 20..25sec

Second test: Lumberyard 1.22 + my GameProject + VS2017Community 15.9.19

1>------ Build started: Project: _WAF_, Configuration: [Game and Engine] profile x64 ------
1>[WAF] Engine Root: c:\Lumberyard\dev\
1>[WAF] Executing 'build_win_x64_vs2017_profile' in 'c:\Lumberyard\dev\BinTemp\win_x64_vs2017_profile'
1>[WARN] Missing JDK path from Setup Assistant detected. Target platforms that require java will be disabled. Please re-run Setup Assistant with "Compile For Android" enabled and run the configure command again.
1>[SETTINGS] Target Output folder (win_x64_vs2017/profile): Bin64vc141
1>[WARNING] Incredibuild disabled by build option
1>[WARN] Invalid module 'LyIdentity_shared' in spec 'game_and_engine'
1>[WARN] Invalid module 'LyIdentity_static' in spec 'game_and_engine'
1>[WARN] Module use dependency 'CrashHandler' for target 'CrySystem' refers to an invalid module
1>[WARN] Module use dependency 'ToolsCrashHandler' for target 'EditorLib' refers to an invalid module
1>Cannot copy source 'C:\Lumberyard\3rdParty\PVRTexTool\2016_r1.1\PVRTexLib_License.txt' File not found.  Skipping.
1>[4135/6379] cxx (win_x64_vs2017|profile): GameProject\Gem\Code\Source\Components\RTSCameraComponent.cpp -> BinTemp\win_x64_vs2017_profile\GameProject\Gem\Code\Source\Components\RTSCameraComponent.cpp.1363050.obj
1>[4138/6379] cxx (win_x64_vs2017|profile): GameProject\Gem\Code\Source\GameProjectModule.cpp -> BinTemp\win_x64_vs2017_profile\GameProject\Gem\Code\Source\GameProjectModule.cpp.1363050.obj
1>[6284/6379] cxxshlib (win_x64_vs2017|profile): BinTemp\win_x64_vs2017_profile\GameProject\Gem\Code\GameProject_rc\GameProject.auto_gen.1363050.res BinTemp\win_x64_vs2017_profile\GameProject\Gem\Code\Source\Components\GameCursorComponent.cpp.1363050.obj BinTemp\win_x64_vs2017_profile\GameProject\Gem\Code\Source\Components\RTSCameraComponent.cpp.1363050.obj BinTemp\win_x64_vs2017_profile\GameProject\Gem\Code\Source\Components\SimpleFPSController.cpp.1363050.obj BinTemp\win_x64_vs2017_profile\GameProject\Gem\Code\Source\Components\VisibleTagComponent.cpp.1363050.obj BinTemp\win_x64_vs2017_profile\GameProject\Gem\Code\Source\GameProjectModule.cpp.1363050.obj BinTemp\win_x64_vs2017_profile\GameProject\Gem\Code\Source\GameProjectSystemComponent.cpp.1363050.obj BinTemp\win_x64_vs2017_profile\GameProject\Gem\Code\Source\GameProject_precompiled.1363050.obj -> Bin64vc141\Gem.GameProject.ee43c3a070a94d369c664215508971cf.v0.1.0.dll Bin64vc141\Gem.GameProject.ee43c3a070a94d369c664215508971cf.v0.1.0.pdb Bin64vc141\Gem.GameProject.ee43c3a070a94d369c664215508971cf.v0.1.0.dll.manifest
1>   Creating library c:\Lumberyard\dev\Bin64vc141\Gem.GameProject.ee43c3a070a94d369c664215508971cf.v0.1.0.lib and object c:\Lumberyard\dev\Bin64vc141\Gem.GameProject.ee43c3a070a94d369c664215508971cf.v0.1.0.exp
1>[WAF] 'build_win_x64_vs2017_profile' finished successfully (2m18.624s)
1>[WAF] Executing 'package_win_x64_vs2017_profile'
1>[WAF] 'package_win_x64_vs2017_profile' finished successfully (2.909s)
1>[WAF] Executing 'deploy_win_x64_vs2017_profile'
1>[WAF] 'deploy_win_x64_vs2017_profile' finished successfully (1.957s)
2>------ Build started: Project: GameProject (GameProject\GameProject), Configuration: [Game and Engine] profile x64 ------
2>[WAF] Engine Root: c:\Lumberyard\dev\
2>[WAF] Executing 'build_win_x64_vs2017_profile' in 'c:\Lumberyard\dev\BinTemp\win_x64_vs2017_profile'
2>[WARN] Missing JDK path from Setup Assistant detected. Target platforms that require java will be disabled. Please re-run Setup Assistant with "Compile For Android" enabled and run the configure command again.
2>[SETTINGS] Target Output folder (win_x64_vs2017/profile): Bin64vc141
2>[WARNING] Incredibuild disabled by build option
2>[WARN] Invalid module 'LyIdentity_shared' in spec 'game_and_engine'
2>[WARN] Invalid module 'LyIdentity_static' in spec 'game_and_engine'
2>[WARN] Module use dependency 'CrashHandler' for target 'CrySystem' refers to an invalid module
2>[WARN] Module use dependency 'ToolsCrashHandler' for target 'EditorLib' refers to an invalid module
2>[WAF] 'build_win_x64_vs2017_profile' finished successfully (48.837s)
2>[WAF] Executing 'package_win_x64_vs2017_profile'
2>[WAF] 'package_win_x64_vs2017_profile' finished successfully (2.616s)
2>[WAF] Executing 'deploy_win_x64_vs2017_profile'
2>[WAF] 'deploy_win_x64_vs2017_profile' finished successfully (1.867s)
========== Build: 2 succeeded, 0 failed, 0 up-to-date, 0 skipped ==========
TOTAL COMPILE TIME 2.18+48.8 ~~ 3min

As you can see the difference of compiling time is very huge:
25sec Vs 3min! (1.20 vs 1.22) (VS2017 used ver 15.9.19)
Please fix this new WAF changes, it’s very annouing to wait too much to test code changes.

Amadis · January 27, 2020, 11:33pm

I have the same exact problem. Please fix, this is completely unacceptable. Mine goes from 30 seconds to 15 minutes.

tom-hh · February 1, 2020, 10:29am

Hi @fluffy and @Amadis,

First of all we’re very sorry this issue is affecting you and leading to increased iteration/build times. We all know how frustrating this is!

The good news is we’re aware of the problem and are currently working on a fix. The issue appears to have been introduced by modifications to a waf build script to improve consistency of whether files should be rebuilt or not. Unfortunately this came at a significant cost and an improved solution is being worked on as I type.

We really hope to have a solution in the form of a patch very soon,

Thank you for your patience!

Tom

fluffy · February 1, 2020, 4:56pm

Thanks @tom-hh for reply, i’m very glad to hear, what Ly team are working on solving this issue. I’m looking forward to see the patch or new ver of engine 1.23 ))

tom-hh · February 1, 2020, 10:30pm

No worries at all @fluffy, my understanding is it’s the infrastructure team, we’ll definitely have more news next week! Thanks again!

tom-hh · February 6, 2020, 11:02am

Hi @fluffy,

Apologies we haven’t been in touch sooner but I can confirm we have a fix for this issue and as far as I’m aware it will be in the 1.23 Release.

Thank you for your understanding,

Tom

fluffy · February 6, 2020, 6:08pm

Thanks @tom-hh. I’ll be waiting till Ly v1.23 be released. I hope it will be released soon. While I’m use Ly v1.21

Softmints · February 9, 2020, 9:15pm

Hi folks, my team just upgraded to 1.22 last week and while we expected some big builds as part of that, this is now hammering our workflow.

If a patch could be made available in the meantime, that would be appreciated. At a minimum, please document the changes for the fix so smaller teams like us can apply it ourselves, and aren’t forced to commit resources to an immediate 1.23 upgrade.

JamesKythera · February 21, 2020, 2:16pm

Hi Tom,

I’m trying out Lumberyard 1.23 now, and it looks like the problem is not fixed. It still takes >2 minutes to launch the Editor, even when not a single file needs compiled.

Did the fix not make it into 1.23?

Best wishes,
James

tom-hh · February 24, 2020, 9:04am

Hi @JamesKythera and others on this thread,

I am so sorry and I don’t know quite how this happened but I can confirm the fix did not make it into 1.23.

I am chasing this up now and will let you know as soon as we have a solution.

Kind regards,

Tom

tom-hh · February 24, 2020, 7:11pm

Hey eveyone!

We have a patch available - compile-time-patches.zip (4.6 KB)

It’s in two parts so you’ll need to apply patch-01 and then patch-02 but that should dramatically improve iterative compile times.

Please let us know how you get on, thank you for your patience!

Cheers!

Tom

fluffy · February 25, 2020, 7:02am

thx @tom-hh for the patch, now it’s took around 37sec to rebuild project after I made little code change - I add one variable exposed to editor. (ly ver 1.23)

tom-hh · February 25, 2020, 8:22am

Great to hear @fluffy! Thank you for letting us know!

wcb · April 11, 2020, 8:51pm

Was this added to 1.23.1

tom-hh · April 14, 2020, 8:14am

Hi @wcb,

That is an excellent question, I’m afriad I’m not sure…

Let me see if I can find out and I’ll get back to you

In the meantime I included the patch file just above which should resolve the issue before the change makes it into an official release.

Thanks!

Tom

tom-hh · April 14, 2020, 8:41am

Hi @wcb,

I had a quick look and it doesn’t seem this fix made it into 1.23.1 - do keep an eye out for it in an upcoming release though!

Have you been able to test out the fix locally yet? Beyond Compare is a great tool for applying the patch files (or this should be possible with Git for Windows too).

Thanks!

Tom

fluffy · December 15, 2020, 6:56pm

hi there!

it’s kinda of tip of day that I found today:

I found another solution that allow dramatically increase compiling time for GameProject. (game)
Note: this do not works for “first build” for new project, but only for future compilations (if you are use c++ and write code only in game project)

After you made you compilation and configure all gemes/ third libs you can open “configuration manager” uncheck “waf” and check “gameproject” this allows to incredible increase speed of run/test code even if you are using ssd.
if you’ll needed add some new geme to project probably you’ll need turn back “waf option” for project recompilation, then you can disable it again to speedup compiling