[axmol-2.0] Support GLES3/OpenGL3

[halx99]

Released

UPDATE

GLES3/OpenGL3 support was moved to milestone 2.0, and still support GLES2.0 for old android device and android simulator marketplace of china

workflow

1. Write shader language by ESSL 310 or GLSL 450
2. Add all shader files into project_root/Source/shaders, default shader file extensions: vertex(.vert, .vsh), fragment(.frag, .fsh)
3. Engine will auto add shader files as glscc compiling files
4. glscc compile source shaders to target platforms: Desktop GL(GLSL330), GLES3(ESSL300), GLES2(GLSL100), Apple Metal(MSL)
5. Engine auto sync compiled runtime shader folder(${CMAKE_BINARY_DIR}/runtime/axslc) to target app app_res_root/axslc, and app_res_root/axslc will be added to search path by engine FileUtils implementation.
6. Start app, load compiled shaders from app_res_root/axslc by shader name.

checklist

• Mesh instancing draw, implemented by @DelinWorks and adpte metal backend by @halx99
• axmolengine/glslcc 1.9.0 by @halx99
• AXGLSLCC.cmake cmake tool by @halx99
• Migrate engine shaders to ESSL 310 for glslcc spirv happy to compile them to target platforms: ESSL 300, GLES 330, MSL
• Shader migrate tool: amol-migrate-1.1 by @delin and @halx99, because glslcc(spirv-cross) only accept ESSL310 or GLSL450 as input shader.
• Android: switch GLES loader to glad for auto loading GLES3 APIs , i.e. glDrawElementsInstanced
• OpenGL UBO support by @delin @halx99
• Metal load compiled binary shader files (sgs format specified by glslcc) by @halx99
• Since use glslcc, the thirdparty glsl-optimizer can be removed, and it's not support compile essl 3.1 to msl, not compatible with new shader workflow.
• Auto sync compiled shaders ${CMAKE_BINARY_DIR}/runtime/axslc to apps by @halx99
  • win32/linux, use symlink
  • winuwp
  • android
  • macos/ios/tvos, draft works, waiting cmake patch by @halx99 to be merge by kitware
• Migrate all tests shaders for making cpp_tests works on new shaders
• Engine Material load needs reimplement due to shader load flow changed
• Refactor shader load managment: ProgramManager support load cusom shader program immediately and improve register mechanism
• Text rendering, SDF outline support
• Remove mat3->mat4 convert, instead, provide math::Mat3 to initialize a GPU layout compatible mat3, both GL3/GLES3/Metal have same mat3 layout, identical to mat3x4
• Check cpp-tests
• GLESv2 compatible

Compatibility

• Since shaders needs compiled by glslcc offline, no longer support embed c++ shaders
• Don't worry, all changes are in dev branch, and will take long time, the main is the lts for 1.0.x currently. even through, we want merge dev in to main in the future, but before that, we will create a lts branch 1.0.x

New shader looks like

vertex shader:

#version 310 es

layout(location = 0) in vec4 a_position;
layout(location = 1) in vec2 a_texCoord;

layout(location = 0) out vec2 v_texCoord;

layout(std140) uniform vs_ub {
    mat4 u_MVPMatrix;
};

void main()
{
    gl_Position = u_MVPMatrix * a_position;
    v_texCoord = a_texCoord;
}

fragment shader:

#version 310 es
precision highp float;
precision highp int;

layout(location = 0) in vec2 v_texCoord;
layout(binding = 0) uniform sampler2D u_tex0;
layout(location = 0) out vec4 FragColor;

void main()
{
    FragColor =  texture(u_tex0, v_texCoord);
}

Notes

• all non-sampler uniforms must in uniform block, because glslcc(spirv) limits
• recommanded use macro for layout location to ensure them are same between vert and frag shader, oterwise, metal render will crash due to layout location between vert and frag shader mismatch

layout(location = POSITION) in vec4 a_position;
layout(location = TEXCOORD0) in vec2 a_texCoord;

layout(location = TEXCOORD0) out vec2 v_texCoord;

layout(std140) uniform vs_ub {
    mat4 u_MVPMatrix;
};

void main()
{
    gl_Position = u_MVPMatrix * a_position;
    v_texCoord = a_texCoord;
}

#version 310 es
precision highp float;
precision highp int;

layout(location = TEXCOORD0) in vec2 v_texCoord;
uniform sampler2D u_tex0;
layout(location = SV_Target0) out vec4 FragColor;

void main()
{
    FragColor =  texture(u_tex0, v_texCoord);
}

Shader syntax migrate summary

• both vertex & fragment shader should insert version decl #version 310 es in header
• fragment shader needs precision decls for float & int
• the quailfier attribute in vertex shader change to in
• the qualifier varying in vertex shader change to out
• the quaifier varying in fragment shader change to in
• the gl_FragColor need replaced by a defined out vec4 variable in fragment shader

Shader restrict or recommands

• please only write 1 uniform block per shader stage, multi uniform blocks not implemented in metal backend

performance preview (release build)

axmol-dev:

axmol-main:

cocos2dx-3.17.1

[solan-solan]

Remove glsl-optimizer please at the final step. Really, it is not good that axmol does not support glsl 1.0 at all. Is there some other game engine that supports only glsl 3.0 specification? Why such limitation?
There were customers on the forum that goes from cocos to axmol. And I am one of them. It is more simple if there is opportunity to migrate custom shaders one by one, and check that everything works after each shader. Think please that shaders could not be debuged. Since that, if you really decided to reject glsl 1.0, please don't do it right away.
Besides, can I have sub directories inside project_root/Source/shaders?
And such question, how glsl 1.0 will be restricted on Android? glslcc can not compile it?
And guys, why so importance to delete things that worked many years before? Is it somehow interferers with new platforms? Embedded to c++ shader feature was very good and were achieved using glsl optimizer for ios/Mac.
And one more. How shaders could be compiled offline if they depend from macroses which passed in runtime? Or I missed something?

[DelinWorks]

@solan-solan I don't think we're dropping support for gles 1.0 , just creating a new version of the engine that supports gles 3.0, if you want to migrate right away just pick the gles 1.0 version, otherwise you have to manually change your project to work with gles 3.0

[solan-solan]

@DelinWorks @halx99
Clarify please the following. If I properly understand, glslcc will compile shaders to spir-v in build time. What will happen with macroses? What if the certain macroses should be configured for some shaders in runtime in the game, like in my case? Does this spir-v will be converted back to the platform shader language in runtime where it could be configured?

[halx99]

The glslcc use spirv-cross decompile spirv-bytecode to MSL(metal), ESSL300(GLES), GLSL330(OpenGL)

[halx99]

glslcc compile workflow:

input shader(ESSL310/GLSL450)
  --> spirv byte code
       --> MSL for ios/macos/tvos
       --> ESSL300 for GLES3
       --> GLSL330 for DesktopGL

[rh101]

What if the certain macroses should be configured for some shaders in runtime in the game, like in my case?

Any chance you can show an example of your current usage related to macros and shaders?