For this month’s Dev Diary, I would like to introduce the Atom team! And a special thanks to Chanelle Mosquera for being the author of this month’s dev diary. See below for their Dev Diary.
Atom: Meet the Team
As more graphics technology advances year-over-year and customer demands for high-fidelity visuals swoops skyward, a small group of game developers and engineers in AWS Lumberyard found themselves looking at a steep challenge to deliver a renderer to meet these celestial expectations. Together, they researched and brainstormed what the future of rendering might look like.
They laid out the tenets for a new graphics engine:
- A renderer built with the latest advanced graphics features.
- A renderer that can seamlessly integrate new rendering techniques.
- A renderer that supports physically-based shading models, and material and shader tools.
- A renderer with a customizable, flexible render pipeline.
- A renderer for engineers, game developers, technical artists, and everyone in between.
- And a renderer that can withstand the next generation of graphics.
These tenets defined the early goals for what is now known as the Atom Renderer.
In a previous blog post, Splitting the Atom: Introducing Lumberyard’s New Photorealistic Renderer, we introduced Atom and gave an overview of its technical components. In short, Atom is a modern, real-time renderer that features a forward+ rendering pipeline, physically-based rendering, and global illumination. Of course, every game development team has their own unique rendering needs, so Atom is being designed with a modular, data-driven foundation for easy customization. It supports modern graphics APIs like Vulkan, DirectX 12, and Metal; but with Atom, there is no need to worry about these low level graphics APIs because it provides a platform-independent API for developers to build with. Furthermore, render passes can be authored and configured in JSON, allowing direct and intuitive customization of the render pipeline. No matter your project, ambition, or the size of your team — Atom is an adaptable renderer.
Today, Atom delivers the latest graphics features, a new shader language, a material editor, and a flexible API for easy feature integration. To build it, the Atom team is not only made up of engineers, but game developers and technical artists as well who collectively leverage their individual motivations and experiences to make Atom a unique rendering platform.
To dive into the philosophies that drove the development of Atom, I spoke with a few members of the Atom team who shared their experiences, challenges, and excitement while working on this project. Here are their stories!
Qing Tao - Sr. Graphics Engineer
Qing, a Sr. Graphics Engineer of the Atom team, is the lead developer of the Render Pipeline Interface (RPI), a platform-independent library that provides APIs for graphic feature developers. For the ambitious community of Atom’s future developers, the RPI library is key to developing items such as system tools, render scenes, and advanced graphics features. The RPI was designed so features are independent of each other, saving developers from untangling code and stepping over existing features. Modularity and customizability have been Atom’s core values since the beginning, but it’s the thoughtful assessments and data-driven developments that pushed Atom towards these goals. With many years of experience as a game developer at Gameloft and Glu Mobile, Qing understands the efforts and obstacles game developers face when working with a render engine. “Usually working on games, you just want to see something fast, but you have to deal with a lot of things that are already developed somewhere else,” she told me.
Making Atom modular has not been easy, but receiving customer feedback, and using the renderer as a customer herself, has helped facilitate this. To demonstrate the RPI’s usability, Qing quickly developed ray tracing ambient occlusion and checkerboard rendering. She notes: “It was a fast implementation time to include into the engine.” By spending a lot of time with the engine, resolving edge cases in the architecture and design, and repeatedly answering the question “how do I keep the features separate and independent” across many use cases, Atom has grown into a mature renderer.
With Atom, Qing hopes to provide and improve on the APIs that allow developers to make features more easily. She hopes that “customers will be able to iterate faster and try out the newest techniques”. Even beyond games, Atom’s ray tracer and physically-based capabilities can be helpful across many other fields. Teams can adopt Atom as their full renderer, or they can just grab the graphics features that they need. That’s the beauty of Atom’s modular design.
Anton Michels - Sr. Graphics Engineer
From the early “hello triangle” days, Anton, a Sr. Graphics Engineer, developed Atom’s Pass System, and now leads a team of six to develop features such as lighting, shadows, and post-processing effects to help Atom see its rendering potential. The Pass System manages the many render passes that take simple geometry and other data to output a beautiful image on the screen. Especially for features, passes provide crucial instructions that direct what the feature does and how it behaves. Recently, Anton has been working on character skin rendering, one of the more challenging materials to accurately simulate and have interact with the surrounding environment. Anton explains: “Skin has so much going on. For example, your skin has a thin layer of oil on top of it, so light interacts with that, then the skin part, then some of the light goes beneath the skin’s surface and bounces around under the skin before coming back out… modeling that in an efficient way is very difficult.”
Like the rest of the team, Anton has had grueling past experiences working with complex renderers, often trying to implement a single feature within a large amount of spaghetti code. “Modifying any part of the renderer, you usually have to touch a main renderer file, which creates merge conflicts… it’s very hard to maintain,” he said. To avoid these difficulties, many of Atom’s asset data are data-driven, like materials, passes, and shaders, which means you can create new materials, shaders, lighting models, and other custom features. Anton adds: “You can do all of that without touching any C++ code,” and regarding our data-driven Pass-System, “our main render pipeline file is less than 500 lines of JSON code” — it’s impressively compact!
Communication and innovation are key in developing a system as unified, yet independent, as Atom. There are a number of guidelines and books on how to build the multiple parts of a renderer, but no guide can teach you how to put those parts together and create harmony within a team to collaborate on difficult design decisions. Anton muses on how far Atom has come, noting that “two years later, we have very complex materials… global illumination… post-process effects… PBR shading model… multi-scattering compensation…” and many more. Customers can run with these pre-built features, or build on top of it — a welcome ambition with Atom’s modular design. Aton adds: “I would really like to see somebody take the modularity and flexibility in a direction that we wouldn’t expect them to.” For example, a scene made of a photorealistic environment with cartoon-like characters, two contrasting features that thanks to Atom’s physically-based rendering capabilities, can blend quite nicely together.
Adi Bar-Lev - Sr. Graphics Engineer
Adi migrated from Lumberyard to become Atom’s Sr. Graphics Engineer to initiate development across many domains including shaders, materials, features, and the render pipeline interface. As the team grew and these responsibilities dispersed, Adi was able to focus on developing graphics features for game developers. With his prior experience working on feature tools at New World and Ubisoft, and his current work across Lumberyard and Atom, Adi brings a refreshing game engineer mindset, which inspires his work on developing features that bring depth and quality to any simulated environment.
During his first days, Adi excitedly worked on a “welcome project” that would bring hazy skies and misty grounds to the ol’ Lumberyard: specifically, a deferred volumetric fog effect with noise turbulence. For another project, Adi and a few others tested Atom’s physically-based rendering capability and technical art tool usability by creating a pearlescent car paint material type with metallic flakes, transparent windows, accurate reflection, and anisotropic response. Nowadays, Adi is brushing the loose strands of Atom’s hair technology, which is re-architected from AMD’s TressFX library. With 2 million vertices, fully dynamic rendering in under 3 milliseconds, there’s no windstorm that Atom’s, um, hair styles can’t handle! With that said, feature performance is of utmost importance. Adi is only one of many on the Atom team who dedicates a large portion of development time into testing and improving the performance of all aspects of Atom.
From initiating the early designs of Atom to developing state-of-the-art graphics features, Adi is glad to help shape Atom throughout its many phases, telling me “I’m proud of everything I do now. The fog can be enhanced to something more, (like physically-based interaction with light) … Hair [will] be a blockbuster for every game company.” There are outstanding modern render engines out there, and Atom surely rises to their level. Atom is designed by a team who empathize with the users and developers who hope to build upon Atom. “We have a system that I am not ashamed of,” Adi says. “The ease of development on the system is what we want the developer to feel. They should feel that they are not blocked from the engine.” Adi looks forward to seeing people pick up Atom and use it to create “multiplanting tools”, or big world generation of assets, such as trees, rocks, vegetation, and ground.
Chris Santora - Sr. Engineer
Chris, a Sr. Engineer on the Atom team, is the lead developer of Atom’s Material System. He mainly focuses on the development of its backend, while also working with folks on other material-related aspects, such as developing the physically-based rendering material types and consulting on the development of material tools, like the Material Editor and the Material Component in the game engine. Since materials interface with shaders to create captivating visual surfaces, Chris also oversees the development of the Shader System from a customer perspective. Recently, Chris finished work on a multi-layer PBR material type, which contains three sets of PBR material properties to create complex, multi-layered materials. For example, the first layer could be a metallic surface with peeling paint, and the second layer could be a brick wall – two completely different materials, which you can blend and transition between.
Atom is a physically-based renderer, but that shouldn’t keep the creatives from developing quirky, stylized visuals. Far from realism, but still demonstrating the mathematical foundations of lighting and shading, Chris nods to Arc System Works and their incredible visual styles in Guilty Gear Xrd (pronounced “ecks-erd” — an Arc System Works thing!) and Dragon Ball FighterZ, noting that “you don’t even notice it’s 3D because it looks hand crafted.” Using data-driven methods, Atom’s Material and Shader Systems are designed with the flexibility to make these creative ventures not only possible, but an intentional experience for technical artists. Chris adds, “It would be really cool if, for example, Arc System Works used our engine… it would answer the question of, `did we succeed in making something that is easy and customizable?’” No matter who uses the engine, direct customer feedback is critical to every developer on the team. It guides and affirms the direction of Atom’s development.
The early stages of development were a challenging time since customer feedback was very limited, so the team had to leverage their experiences in the game industry to speculate what customers want and need. Soon, customers and partners were able to provide continuous constructive feedback, which provided critical direction in Atom’s development.
Galib Arrieta - Sr. Software Engineer
Galib is the Sr. Software Engineer behind Atom’s Shader Build Pipeline, which encompasses the Amazon Shader Language compiler (AZSLc) and the many processes that build shader code into shader assets, resulting in incredible visual effects. A lot of thought had gone into the decision on whether to develop a new shader language for Atom. Ultimately, the pros of being able to introduce useful and optimal shader techniques, while also reserving the familiarity of HLSL opened more possibilities for Atom. However, a shader language is only as good as its compiler. Over the past year, Galib accelerated and simplified it, so shader writers can focus on their creative pursuits. Currently, Galib is focused on streamlining the shader build pipeline further by reducing three builders to a single builder and by off-loading the work onto CMake.
A fast and robust shader compiler is only the first milestone. Atom’s modular and customizable philosophies apply to the shader build pipeline as well. Atom’s Shader System provides the option to use shader variants , which creates versions of the same shader with minor differences based on conditional branches in the code without relying on pre-processor flags. This prevents shader code from branching at runtime, saving computation cost per pixel. For common screen resolutions with 2-4K pixels, that’s a big difference! As a sub-feature of the shader variant system, Galib introduces the idea of a shader supervariant system, a new design that will allow users to specify multiple sets of instructions for compiling the same AZSL code. For example, a developer can write multisampling texture code and with a single flag to the AZSLc, they can automatically generate a non-multisampling version of the same code — the same shader can be compiled in two different ways. “With the shader supervariant system, the user will have total power to customize exactly and with which arguments on how to compile a shader,” Galib explains.
Galib is most impressed by the fact that Atom’s Pass System is completely data driven, making it relatively straightforward to customize the rendering pipeline. He tells me: “As a pet project, I built a clone of ShaderToy, [that I called] RenderJoy, and it would have been almost impossible to do on any other game engine. But the data driven nature of the Atom Pass System made it possible to develop RenderJoy in a few days.” Like his teammates, Galib is excited to see what customers will create with Atom, and looks forward to playing an Atom-based game.
Jonny Galloway - Sr. Design Technologist
Jonny is a Sr. Design Technologist who focuses on Atom’s realtime 3D rendering, content tools, and workflows. He has a background in creative direction and game production, and previously worked on areas such as the “Large World” feature set for Lumberyard (used in New World), which allows users to create procedural biomes, terrain, and world design. Originally joining the Atom team as a Technical Product Manager, Jonny brought a wealth of perspectives and expertise to the Atom team. Over the years, Jonny’s finger prints have been all over both Lumberyard and the new Atom renderer, but nowadays, he focuses more on the content data pipelines, tools, and workflows for creative professionals. “Because you have to feed the engine great data to render pretty pictures… once we got Atom integrated into the editor switching to a Design Technologist role made sense, as it allowed me to be more holistically focused on the product and feature design aspects and connecting the overall customer experience.”
“Look development“ is a key component of every game, which includes materials, lighting, and post-fx, and Jonny maintains high, industry-level standards for the creative workflow experience that Atom delivers. “It’s not just important that a modern 3D renderer has a high bar for visual quality, but the experience needs to engage the creative flow, the tools need to be not just robust, but intuitive and allow content creators to iterate rapidly.” These values were the guiding principles of the Material System’s user experience, where material data centers around physically-based rendering, packaged with a Material Editor that is look-development-oriented, focusing on the artistic tasks and experiences. As a technical artist, it’s very important for Jonny to have complete control over the look and feel, which is why the team implemented a new shading language and designed the Material and Shader Systems, so that new material types and shaders can be written far more easily and quickly.
One of the challenges of building an entirely new rendering engine is wrangling the scope of Atom. Atom is made of many complex, interlaced systems and tooling, and figuring out what order to work on these systems, and which of its aspects need the most love has kept Jonny up late many nights. “I am a tinkerer at heart, so generally I just find this work so challenging and rewarding both artistically and technically, [and] I am generally very proud of the fact that we were successfully able to do this at all.” Eventually, Jonny would love to improve Atom’s rendering and tools for procedural large world design: bringing to life large open worlds with beautiful terrain and vegetation, and coupling the artistic and design side with Python automation in the pipelines and editor tools.
Preview: Developers’ Technical Dive into Atom
In a later post, the developers from the Atom Team returns to discuss the technical details of the features they’re working on. We’ll cover topics such as the following:
- Material System
Learn about Atom’s Material System, the data-driven design behind materials and material types, and what material types will be coming with Atom.
- Shader System
Learn about shader build pipeline optimizations and how the shader variant system can help you iterate on shader development more effectively.
- Render Pipeline Interface
Learn about a few of the key components of the RPI: data assets, render contexts, and profiling tools.
Chanelle Mosquera , a programmer writer for AWS GameTech. Working alongside the Atom team, I get to learn and play with the many systems and features across the renderer — a perk that encourages my enthusiasm for computer graphics.