Engine_GPUMemory_Testing/How do YOU debug?

Greetings my forum peoples. I’m an Indie developer and a QA Engineer. I’m thinking of migrating my Unity game over to Lumberyard. I wanted to maybe start a constructive thread for performance issues and ways to debug them. I’ve used Visual Studio for a lot of my debugging as it’s pretty easy to use (Just attach :slight_smile:). The idea is to list a relatively common issue around performance and provide some ways to debug them to find the cause. Ill start us off with two of the most common performance issues that I’ve seen in probably 9/10 titles. I’ve tested well over 50-100 titles in the past 6+years I’ve been testing. Not to mention the ones I’ve encountered as an avid gamer. To start this shindig off, I’ll name Out of Memory Stability and Memory Leaks. Also, ill be attaching some slides with screenshots of Visual Studio Diagnostic tools using Lumberyard as an example. It will contain: Launching the editor, creating a level, generating terrain mesh, creating an entity, and assigning a mesh component.

Out of Memory Stability Issues:
-Performing an action when too many GPU allocations are being made to the heap without a returned to the platform. This can cause the software to:
-'Soft Lock" or become unresponsive.
-“Chug” or perform very slowly.
-Crash and possibly cause a loss of data.

How do you catch it in action? I’m going to refer to my slideshow pictures for the bulk of the explaining but the one thing you need to watch out for is an increase in the size of the heap. I’m not talking about just the heap growing because that will always happen. I’m talking about an increase in heap size WITHOUT returning the memory to the platform. Example: “Software Greg needs to borrow 500kb to allow the user to perform an action. Greg will say “YO, Petey platform, hook me up with 500kb on graphical memory, I’m good for it!” Petey says “Ok that’s cool, just make sure you return the 500kb to me as i don’t have an endless supply”” If Software Greg doesn’t give that memory back, eventually, Petey will run Out Of Memory. This is how an Out of Memory issue works in a nutshell.

Memory Leak:
-Memory leaks are more ambiguous in nature and are often not caused by the user. This is typically found when a nonfunctional action is occurring e.g. sitting on a menu or entering a different game state than the software planned for. This can cause:
-Slowly performing worse over any given amount of time
-The ultimate fate of a memory leak is a crash and possible loss of data
-Good coding practices in memory management and cleanup help to avoid these issues btw. I’m looking at you Front-End built in C#!

Memory leaks are a slow drain on the graphical memory. Meaning, that small allocations are being made but not used or they are about to be used but are stopped by a ‘WaitCall’. So what happens when the platform is not aware of this? It will keep allocating memory without knowing something has occurred to stop the software from using it.

It’s on us to catch these issues and make sure we are always pushing the envelope to find new creative ways to debug our issues and to also make sure our software is doing what we expect.
What are some issues and debugging tactics that you folks use? Let’s keep this thread alive!

5 Likes

Welcome to the community JMoney, this is handy information for optimizing and keeping memory and those frames as high as possible. Better yet, preventing crashes. :clap:t4:

1 Like

No problem @JKingpin. Not only that but this would be a good spot for developers to talk about issues and solutions to their issues. Ideally I would want some more people discussing some of their debugging tricks on this thread. My scope is around performance issues and I know quite a bit about networking issues. What kind of issues do you Lumberyard guys run into and how do you deal with them?