Reducing Size of Release Build/Only Including Necessary Assets

Lumberyard Discussion Other Support
, , ,
#1

I have been working on a project that has required me to use different assets one time (think A-B testing). As a result of this use, my final build size has increased significantly (specifically the gameproject_pc_paks/gameproject/misc.pak file). I now want to remove all of the unnecessary bits from the release build, without going through and deleting the assets from the project.

Essentially, I am looking for any ways to cut down on my build size while keeping the game project relatively intact.

  • Is there an option to only include certain levels in the final build of a project (and their associated assets)?
  • Also, how does one go about reducing the size of the misc.pak file?
    I have already made sure not to include the .pdb file in the gameproject_pc_paks/executable directory, and I already just delete the levels folder for the levels I donā€™t want to include in the final build (those are on the order of kilobytes, however).
#2

I got a similar problem. Got a simple test project build from
the default project and the packaged release folder is 230Mb
Thatā€™s exactly Unreal level and would put LY in the same
bulky category. That canā€™t be right, right?
(pdb files, debugging, qt5 etc are deleted alreadyā€¦)

The same test project in most other engines is between 15Mb (Armory3D) and 60Mb (Unity). In Unreal it is also exactly 230Mb.

I was told the minimal project size in LY is ~33Mb.
My launcher.exe alone is already 78Mb. Did I make an error somewhere? Or is it really that bulky?

(I hope some people here actually also got some release build going so that I can get an answer to thisā€¦ Looks to me like most people are just toying around or are already gone for goodā€¦

Build size to me is important and the only reason I donā€™t use Unreal for everything. If LY really plays in the same ballparkā€¦ hmmmā€¦)

#3

I cannot comment in any way, because I have not yet reached the creation of the release version of the game. I am currently only designing levels and creating the necessary assets. I am sorry, I cannot help you.

#4

More details:
Again, full release build of a bare bones unity-style ā€œroll-a-ballā€
with no external assets/content is ~230 MB (!) :scream:

Comparison of the same test project in other engines:

  • Unreal Engine: 230Mb (!)
  • Unity = 60Mb
  • Godot = 40Mb
  • Away3D = 20Mb
  • Heaps = 20Mb
  • Armory3D = 16Mb

Biggest Files in the packaged release in Lumberyard:

The Launcher.exe itself = 71Mb <-- Why is the launcher so large?
PhysXGPPU.dll = 27Mb
gamedata.pak = 77Mb <ā€” What is in there?
misc.pak = 24Mb
shadercache.pak = 20Mb
engine.pak = 10Mb
d3d11 cache folder = 8 MB
These all look pretty essential to me.

So looks like itā€™s actually quite bloatedā€¦
I mean Unreal is the king of build sizes and LYā€™s build is exact
the same MB down to the last digit hereā€¦ Any thoughts on that?

People who want to release on mobile at some point are in for quite the rideā€¦
Releasing casual games on that build size might also
be difficult to ā€œsellā€ to the end userā€¦

#5

Hi @Lytz1,

Since the original post was written Lumberyard released a dependency based asset management system called Asset Bundler to help with this problem from the Assets side - https://docs.aws.amazon.com/lumberyard/latest/userguide/asset-bundler-overview.html. Thereā€™s a tutorial for using this in StarterGame at https://docs.aws.amazon.com/lumberyard/latest/userguide/asset-bundler-tutorial-release.html. Those pak files are just zip files full of shaders and assets (7 zip or other should be able to show you whatā€™s inside). You should be able to use the Asset Bundler tool to at least be sure youā€™re only including the Assets your game needs - the old system was a wildcard/glob system which would over include assets which were in your cache but not necessarily used by your game.

Hope that helps!

#6

Thanks. Good to know.
However, my main concern right now is the high
build size for a tiny test project as minimalistic as it gets.
This is the (230Mb release packaged) test project:

(If this is the Baseline then how huge does everything get when
more stuff is included?)

(I always do Unitys classic ā€œRoll-A-Ballā€ right away in every engine as it is fast to do, gives me something to compile and letā€™s me test the build system/capacities of any engine quickly. It also gives me a pretty good basic comparison for exactly this stuff I am talking about here: Build sizesā€¦ :wink: )

#8

Did you try the new asset bundler system?

#9

No, not yet. Iā€™ve just learned about that today.
Still, donā€™t see how this would help to reduce ~ 200Mb
when there is nothing but some cubes and a ball in one single level?

I am gonna look into that very soon though.

(If so, that would imply that there is a ton of unnecessary additional
stuff packed into the build by default. But that still would be great
because this certainly is better than a 230Mb baseline for a minimal project-physX-shader-engine bundled build and could be thrown out.)

#10

No guys. No. Baseline is ~230Mb. Period.
Assetbundler looks useful for Assets on top of that.
The seed lists make no sense here I think. Because they are for, well, Assetsā€¦ Here are no Assets involved.

Everything I pointed out here is already ~200Mb:
PhysiX and Launcher alone together already ~100Mb.

The Launcher.exe itself = 71Mb <-- Why is the launcher so large?
PhysXGPPU.dll = 27Mb
gamedata.pak = 77Mb <ā€” What is in there?
misc.pak = 24Mb
shadercache.pak = 20Mb
engine.pak = 10Mb
d3d11 cache folder = 8 MB

IMO: None of these have anything to do with the Assetbundler, no?

Again, I am really curious: Why is the launcher bloated to 70Mb in the release build but only 17Mb in debug? And what is in gamedata.pak?

(Also: Anyone up for a comparison challenge and prove me wrong? Do create newproject ā€”> select default ā€”> just build it ā€”> then package it right away for release with nothing more in it or added and letā€™s see what you getā€¦ :slightly_smiling_face:)

I think I will do another run and package an empty project with NOTHING in it and see what I get.

#11

The launcher is only 17 MB because it is dynamically linked to all the gems and such. The release build is statically linked. You can probably reduce the size if you disable gems you donā€™t need.

Iā€™m not sure if you actually need that PhysX DLL or not. Possibly? I donā€™t know. It runs without it but may be needed for people that have GPU physx capabilities.

Also, Iā€™m really not sure if they strip all the debug info out of the release build or not. I havenā€™t checked into it.

#12

The launcher is only 17 MB because it is dynamically linked to all the gems and such. The release build is statically linked.

Of course, makes sense.

You can probably reduce the size if you disable gems you donā€™t need.

Indeed. I am sure that this might be a big factor here. i just checked.
The create new ā€”> default has quite some stuff in there that isnā€™t really necessary in most cases. So this is clearly a good point to start.

Iā€™m not sure if you actually need that PhysX DLL or not. Possibly? I donā€™t know. It runs without it but may be needed for people that have GPU physx capabilities.

Good point. This would be good to know in general.

I am also right now looking through the .pak files myself.
There is indeed a lot of stuff in there that isnā€™t necessary but clearly
optional. Not sure how it ended up there but there is a huge amount.
I started this project from the default not from the empty, so there is a lot of stuff in there that isnā€™t obvious that it is there.

So looks like once again I was too fast to judge.
I think I need to go through quite some rebuilds to figure things out
so this might take some time but I will try to get as low as possible.
Letā€™s see what I can come up with.
I am also building another one based on an empty right now.

So I officially take this :point_down: statement back. Sorry 'bout thatā€¦ : :sweat_smile:

No guys. No. Baseline is ~230Mb. Period.

1 Like
#13

When you figure it out, will you be able to record a video lesson on how to properly and most profitably release a project with various details? It would be very interesting, because now there are no video lessons on this topic, at all.

1 Like
#14

I think I might be able to do that. This might take a while though.
Itā€™s unfortunately indeed a bit complicated. :sweat_smile:

I want to go through this at least a couple of times so that
I get a firmer grasp of everything.

Iā€™m also sure I missed a thing or two on the first two runs,
so I need to figure these things out first as well.
But I think I should already be around ~85% there.

1 Like
#15

Hey @Lytz1,

I am also right now looking through the .pak files myself.
There is indeed a lot of stuff in there that isnā€™t necessary but clearly
optional.

Yep thatā€™s the piece the Asset Bundler should be able to help with - Rather than using the old system for packing up assets which just grabbed everything from the cache matching some pre-defined patterns it allows you to walk a dependency graph of only things that live within your level (But it does have options as well for any additional ā€œstuffā€ you may want to include which doesnā€™t live within the graph). The old system did indeed pack up much more than necessary generally.

To give it a quick try to see what it would include you can run it from dev with these options (This is an example, your level name and folders may be slightly different)

(Important that youā€™ve let asset processor run before you do this)

d:\P4\dev>Bin64vc142\AssetBundlerBatch assetlists --addseed levels\game\singleplayer\level.pak --print

And you should see a big long list of assets that it found walking the dependency graph under level.pakā€¦

- objects/airship/airship_crane_01_ddna.dds.4a - objects/airship/airship_crane_01_ddna.dds.5a - objects/airship/airship_crane_01_ddna.dds.3a - objects/airship/airship_crane_01_ddna.dds.2a - objects/airship/airship_crane_01_ddna.dds.6a - objects/airship/airship_crane_01_ddna.dds.7a Total number of assets for Platform ( pc ): 3921.

So those assets would then be used to populate your gameā€™s version of the game data pak, hopefully saving you a lot of wasted space :slight_smile:

3 Likes
#16

Awesome, now I get it. I actually did look into the cache folder already and was thinking to myself why do I have stuff in the .paks from the cache in there.

When I found out about the Asset Bundler I basically just ran it without thinking too much about it so probably thatā€™s why it didnā€™t change anything at the end in the first place.

(Important that youā€™ve let asset processor run before you do this)

Good to know. I have to admit I didā€™t take my time with the AB and rushed it a bit.

Anyway, that helps. Thanks a lot.