Compute Farm Cloud Gem - How to use?

My project needs a large open world with a size of 64 km ^ 2, however, it is impossible to make such a size of a terrane with great detail and high quality using standard tools. So I came across this technology(Compute Farm Cloud Gem), which in theory can help with this problem. But the question is how to use this gem in the engine and how to create a terrain accordingly? As far as I understand, in order to reach a size of 64 km ^ 2, you need to create 4 height maps with dimensions of 4096 * 4096 and merge them together using gem. How to do it?

Additional materials:

https://docs.aws.amazon.com/lumberyard/latest/userguide/cloud-canvas-cloud-gem-compute-farm.html

Hi @didzey, I’ve got some suggestions from the developers on how to build large, hi-fi terrain in Lumberyard (see below). Let us know if it helps.

General Steps to Highest Quality

If you would actually like to author a 2k world @ 16k color:

Create a new Level in Lumberyard (ctrl+N or File > New)

Heightmap Resolution: 2048 x 2048

The next screen a llows you to select 'Texture Dimensions. However, the default world it generates does not utilize a 16k megaterrain color map!

Note: If you immediately open the Export/Import megaterrain tool, you will see the default set up:

  1. 2 x 2 tiles
  2. @ 512
  3. Effectively, this gives you:
  4. 2k world / heightmap
  5. 1k megaterrain color

So the default setting always goes to 1K megaterrain and not 16K in the “Select Texture Dimensions” screen.

  1. IF you would actually like to author a 2k world @ 16k color try the following:

After creating your new level…

  • Run this step (twice):
  • Game > Terrain > Refine Terrain Texture Tiles
  • Game > Terrain > Refine Terrain Texture Tiles
  • Then open the ‘Export / Import Megaterrain’ tool:
  • Select all of the tiles in the tool interface
  • Then use ‘Change Tile resolution’
  • Select Resolution ‘2048 x 2048’

If you check the Export/Import Megaterrain Tool, the result should look like:

Tools > Terrain Editor

Terrain Editor > Tools > Export/Import Megaterrain Color

Now you can import / export, and verify that after exporting your megaterrain color texture that if you open it in Photoshop it is 16k.

Importing World Data into Lumberyard

Open the Terrain Editor:

Tools > Terrain Editor

Import Height

In the Lumberyard Terrain Editor there are tools for working with import/export of world data:

To import your heightmap:

Terrain Editor > File > Import Heightmap

We support a variety of image file types for heightmap import

Anything 16bit or higher is generally adequate

8bit formats will have stairstepping (due to lack of precision)

BT is generally the hgihest quality precision

Currently, even though there is support for importing color data to individual tiles within the terrain, there is only support for importing a single heightmap for the entire terrain.

Suggestion: Down sample your heightmap to the world resolution before importing; If you have a 2k world and try to import a 4k heightmap, it will ask if you want to clip or resize. But because the internal tools for resizing data are not very good and actually just skips rows of data, the results are subpar. So a down sample to the target res will give you the best results.

Importing Color Map

We also support importing color data for terrain

You actually can import color data for individual tile within the terrain, but it’s often easier to simply work with a single megaterrain color image

Note: Lumberyard does not fix-up along tile edges, so you need to be careful and make sure your tile data is authored in a way that all the adjacent tile edges match up (so there are no seams)

Currently only the BMP file type is supported for color

To get to the megaterrain color import:

Terrain Editor > Tools > Export/Import Megaterrain Color

You can select tiles, or ranges of tiles.

Suggestion: select ALL of the tiles and import a single 16k color map, this has given me the best results overall.

Notes:

You can actually go higher then 16k, it seems that you can continue to refine the number of tiles, then alter their resolution.

FAQ

Q: If 1m = 1 height texel, does this mean a single terrain quad = 1m?
A: YES. If I place primitve_cube (which is -1/+1m) onto the terrain, it appears to cover 2x2 quads.
Notes: This is only true when 1 texel height = 1m

Q: If I author a 16k color map, I should have 8x8 texels of color per 1m quad?
(Is this a hard code limit, why couldn’t it be 16 or 32, etc?)
A: YES . Smaller terrain with large color texture = higher res per meter. 16K mega texture is largest supported, not because a hard limit, but more because things start breaking down - it wasn’t really built for 16k+ texture.

Q: It appears to be generating color tiles? What is a tile? Does this define a atlas?
A: NO . It’s not really an atlas (but it’s similar), it’s a quadtree of image resolutions.

Q: Do tiles have sectors? Are the sectors what are packed into a tile atlas?
A: NO. Terrain is split into tiles internally.
These tiles are really an authoring concept. Each tile is the root of a megaterrain quadtree.

Q: When I perform ‘Generate Terrain Texture’, what does this do?

  • Does this build sectors, then pack them into an atlas?
  • Does this build the mips? How are the mips generated?

A: NO. it’s a quadtree of images at increasing resolution.
Not really mips (per dds), but the quad tree generates and stores multiple resolutions much like mips.
Notes: It’s authored for the continuous terrain LOD system.

Q: When I create a 2k world, this tool shows me 2x2 tiles @ 512 (what does the 512 mean?)
A: 512 IS the actual number of megatexels in this case.
It seems to be ignoring the level setup, even though you ask for a 16k texture resolution.
It’s really building 2k world, 2x2 tiles, each tile 512 megatexels (by default).
Note: As stated in the steps above, you can increase the number of tiles (refine) and the resolution (up to 2k)
This allows you to get to 16k megaterrain color.

Q: If I use ‘Change tile resolution’ what does this do?

  • The largest size available is 2048 x 2048
  • How does the default 512 relate to this change in resolution?

A: YES , each tile will max at 2048

  • 512 is the default tile resolution (for megatexels)
  • But you can have more tiles, by refining the number of tiles

Q: What does the 512 … 2048 resolution mean in relation to:

  • The 2k size of the world/height?
  • The 16k size of the megaterrain?

A: Heightmap is independent, this purely changes the resolution of the color map per-tile.

  • Resolution displayed in the megaterrain export/import IS the color megatexel resolution for that tile. (effectively ignoring level setup and the resolution you set when you generate terrain texture)
  • So I was able to get to 16k color maps by:
    • Refining my 2k world twice, to 8 x 8 tiles
    • Then setting each tile to 2k
  • And increase visual quality by setting in game.cfg:
    • e_TerrainTextureStreamingPoolItemsNum = 4096

Q: How are splat map weights calculates and applied?

  • Are they per-quad, or per-vertex? Or are they textures?

A: They appear to be per-vertex. Only the triangles that have weights > 0 are rendered for a given layer. Each layer is rendered as a separate draw call, plus the megatexture is it’s own draw call per texture block.

Notes: Your splatmaps should be authored to match the heightmap resolution. 2k world, would use 2k splats.

Q: How do mesh sectors and continual LOD operate for these?

  • If the weighting is on the mesh vertex, then splat weights should LOD with the terrain mesh density?

A: Yes, as the mesh changes resolution, so does the weight map, so you may see details from certain layers suddenly change or vanish.

Q: What is the optimal resolution for imported splat maps?

  • Should they be authored to match the 2k height? (yes, probably … please verify)
  • Or the 16k megaterrain?
  • Or something else?

A: Match the heightmap (ie mesh) density for splat map resolutions.

if your terrain is 1k (1024 x 1024), use the same size image for splatmaps.

Q: At runtime, how does terrain decide what texture resolution (mip) to fetch and draw (for the mega terrain color)?

A: How Mesh LOD is calculated : Each terrain sector at various LODs keeps track of a float error value representing the greatest distance between a vertex in that particular LOD and the highest detail terrain sector. When determining the LOD for a given sector, an “allowedError” is calculated based on the zoom factor of the camera, the distance of the terrain sector to the camera and the terrain LOD ratio. Then the lowest detail terrain sector with an error that is less than the allowedError is chosen. See BuildErrorsTableForArea() in terrain_edit.cpp.

  • How Terrain LOD is calculated : Some magic numbers are used along with the number of pixels per meter in the texture, the distance to the terrain sector, the zoom factor of the camera, and the e_TerrainTextureLodRatio CVar to determine the correct LOD. Unfortunately this does not take screen resolution in to consideration, but at least you can modify this with the CVar.

Q: If I import a 16k megaterrain color texture, I get pretty sharp results but …
A: When terrain has been modified in the editor, a different system is used to render the dirty chunk rather than runtime, so it could look different.

Q: As soon as I ‘generate terrain textures’ the visual quality gets really blurry, and the sectors appear to start fetching mips at various resolutions … it’s hard to determine what is going on (and thus how to improve things)
A: Texture Pool is used for terrain textures, it defaults to 64 in size, and each texture is 256x256. A 16kx16k texture would actually contain 5461 individual textures, so not nearly enough can stay resident in memory at any given time. To increase the pool size, you can use e_TerrainTextureStreamingPoolItemsNum , the current max is 4096. This CVar must be set in the level or project cfg files, since it needs to be present as soon as the system loads, and changes made in runtime will not affect it.

Q: Are the sectors currently fetching megaterrain color based on their mesh LOD?
A: NO

Q: I don’t Understand How terrain.layer materials (in layers) are applied to the terrain and what detail repeat does?

  • For instance, if I author a dirt map as 2m x 2m, what is the proper detail repeat value to use in my material?
  • Are UVs 0…1 per meter/quad?
  • How does detail repeat values relate to overall terrain?

A: YES. If I author a terrain detail material as a 2m patch, then set the TileU and TileV in the material to 0.5, it will cover 2m.

  • Additional info: The shader calculates UV coordinates from world space. Data is passed into the terrain shader that tells it the x-y offset and scale for the texture uvs, then it uses that to transform the world space coordinate into a uv coordinate.
2 Likes