Shaders seem to be locked somehow

Lumberyard Discussion Art & Graphics
,
#1

Hi Guys,

Got a really strange one. In GameSDK.

I’ve started trying to get foveated rendering working, but all my shader changes do nothing.

I mean nothing at all.

I’ve tried all sorts of things.

I’ve modified ShadowMaskGenPS to always return 1.

I’ve modified the vegetation pixel shader to always return purple.

I’ve tried dozens of things… and nothing happens.

I’ve checked that the shaders are being compiled by typing rubbish into a shader and looking for compiler errors.

I’ve tried deleting and regenerating all the pack files.

I’ve checked that the opengl renderer is not being run.

I’ve tested both in the editor and stand alone.

I’m out of ideas.

Has anyone, anyone, got any clue where I should look next.

Cheers

Paul

#2

I’m a total noob here just early evaluating, but to me it sounds like cached platform asset doesn’t get updated on changes.

My first try will be to delete manually platform assets instead of paks this before to compile shaders.Maybe they compile, but do not overwrite cached platform data.

This way you can also understand if some app keep file handlers to your files preventing update as windows will warn you.

Then try to run any process with the most verbose output you can and with admin rights as so many app fails silently on denied access.I don’t know if it is the case but It’s the best advice I can give without a more specific knowledge of the subject matter.

I also believe you can just focus on the editor as stand alone runs on pak files that are presumably built from the actually cached data you are getting in editor, just in a more load friendly archive file.

#3

No it’s not that.

I’ve searched for all instances of shaders and made sure they are up to date.

It’s something far more hidden, I’m going to see if I can run filemon and check where the shaders get loaded from

#4

aaaaagggggghhhhhhh

Added code to see what shaders actually are loaded and the ones I was changing are not loaded.

It seems the GameSDK uses fixed pipeline emulation.

Loading shader 'Fallback'
Loading shader 'FixedPipelineEmu'
Loading shader 'Fallback'
Loading shader 'FixedPipelineEmu'
Loading shader 'Light'
Loading shader 'ShadowMaskGen'
Loading shader 'HDRPostProcess'
Loading shader 'Hud3D'
Loading shader 'PostEffects'

Can we have emojis please, I need the head meets wall one … and beer… lot’s of beer

#5

Hey @StainlessBeer, keep your head away from the wall… I’m checking on this one for ya.

#6

Any joy?

I am assuming at the moment that a load of shaders get loaded through a different code path.

I made some changes to the vegetation system that got me a 5 FPS gain with foveated rendering, but that was in the c++ code not the shaders.

The shaders are still proving to be a major headache as you are never sure which ones are loaded, and even when you are sure they are loaded, it’s impossible to work out which techniques are used and which #defines are set when they are compiled

#7

I’m getting closer to working out what is going on, but running out of time.

I’ve found that a lot of the shader #defines are stored in files which exist in cvargroups stored all over the place.

That allowed me to write a tool which handles all the #defines and generates the text that actually gets compiled by the shader compiler.

This then really screwed my head up.

For example. The Vegetation shader includes CommonShadowGenPass.cfi and ShadowCommon.cfi

I assumed ShadowMaskGen would be run, but it looks like the shadow pass that is actually run is dependant on the flags set in the vegetation shader.

I think the actually technique executed is defined in the script string.

So for vegetation the technique would be ShadowGen.

Assuming the #defines have been scanned correctly, this would then execute the Common_SG_PS pixel shader, which is very basic.

It does explain why my shader changes had absolutely no effect though.

I know the shader system is being re-written, any idea when that will be rolled out to the public?

#8

This issue is assigned to one of our engineers right now and he should have an update for you shortly.

#9

I am more and more convinced that my changes are not being executed.

I am now pointing my blame finger on the asset management system.

All assets come form “@assets@” which seems to be in the engine as a way of allowing remote file systems.

I suspect this is broken for stand alone apps.

Instead of getting the shaders from gamesdk_pc_paks\GameSDK*.pak it is getting them from somewhere else.

It’s all that makes sense by this stage.

#10

Actually I was wrong.

Before I make the pak files, I have to build the pak files.

@"/C .\bin64\AssetProcessorBatch.exe /gamefolder={0} /platforms=pc", game

That’s the one that actually updates the shader source code…

#11

Well I think I have found one problem.

I have two commands. One builds the entire game. One just updates the shaders.

If I place the code “b*stard” into a shader and update the shaders, I get no errors.

If I then build the entire game, I get a shader compiler error.

So the shaders that get compiled by

@"/C .\Bin64\ShaderCacheGen.exe /BuildGlobalCache /ShadersPlatform=PC"

Are not the ones I have been editing.

So what I think happens is that the makepaks command …

@"/C .\bin64\rc\rc.exe /job=bin64\rc\RCJob_Generic_MakePaks.xml /p=pc /game={0}", game.ToLower()

Actually copies the source files into pak files which are then compiled into the shader cache.

man this is a mess.

#12

You have to …

  1. Build pak files
  2. Make pak files
  3. Build shader paks
    Then changes will be propagated to the game.
#13

Yes, I have now written by own tool to update the shaders and place them in the correct directory.

That did catch me out a long time ago.

#14

It sounds like you’ve delved deeper than I have so far but I did run into a problem related to shaders. Just in case it helps: I was using the “BuildShaderPak_DX11.bat” file to build the shaders but it is hard coded to work on “SamplesProject”. I had to edit it to point to the project I was working on.