volumetric: Add volumetric lighting

[vlj]

This PR adds volumetric lighting for sunlight and point lights.

The volumetric effect is raymarch based and is based on physical transmittance equation
The fog is "thicker" around z = 0, and thickness follow an exponential curve
sunlight has shadow support (but it seems quite costly)

[past-due]

Doing some very preliminary testing:

• Volumetric lighting no longer respects tile visibility / "fog of war" (in campaign mode, for example, where unexplored tiles should be completely dark and/or not revealing of their topology for the most part - can test with "classic" fog enabled on Beta1 to see what this should look like with fog too)
• FPS absolutely tanks when zooming in close to terrain and positioning the camera different ways
• Even on "high", the sunlight shadowing support appears to be very... stepped / blocky? (Was testing on Gamma 1 for this with that initial transport)

[ManManson]

Something seems to be broken in this PR with regard to OpenGL rendering backend. Even with "volumetric lighting = off" (but also reproduces with other value for the setting), when I start beta 1 campaign, I consistently see null pointer dereference exception at gfx_api_gl.cpp:2601:

void gl_context::bind_pipeline(gfx_api::pipeline_state_object* pso, bool notextures)
{
	gl_pipeline_id* newPSOId = static_cast<gl_pipeline_id*>(pso);
	// lookup pipeline
	auto& pipelineInfo = createdPipelines[newPSOId->psoID]; //< `newPSOId` is null!
	gl_pipeline_state_object* new_program = pipelineInfo.pso;
...

Also attaching the stacktrace:

>	warzone2100.exe!gl_context::bind_pipeline(gfx_api::pipeline_state_object * pso, bool notextures) Line 2601	C++
 	warzone2100.exe!gfx_api::pipeline_state_helper<gfx_api::rasterizer_state<2,0,255,1,2,0>,2,0,std::tuple<gfx_api::Draw3DShapeInstancedGlobalUniforms,gfx_api::Draw3DShapeInstancedPerMeshUniforms>,std::tuple<gfx_api::vertex_buffer_description<12,0,gfx_api::vertex_attribute_description<0,1,0>>,gfx_api::vertex_buffer_description<12,0,gfx_api::vertex_attribute_description<3,1,0>>,gfx_api::vertex_buffer_description<16,0,gfx_api::vertex_attribute_description<1,2,0>>,gfx_api::vertex_buffer_description<16,0,gfx_api::vertex_attribute_description<4,2,0>>,gfx_api::vertex_buffer_description<96,1,gfx_api::vertex_attribute_description<5,2,0>,gfx_api::vertex_attribute_description<6,2,16>,gfx_api::vertex_attribute_description<7,2,32>,gfx_api::vertex_attribute_description<8,2,48>,gfx_api::vertex_attribute_description<9,2,64>,gfx_api::vertex_attribute_description<10,3,80>,gfx_api::vertex_attribute_description<11,3,84>>>,std::tuple<gfx_api::texture_description<0,3,0,0>,gfx_api::texture_description<1,0,0,0>,gfx_api::texture_description<2,3,0,0>,gfx_api::texture_description<3,3,0,0>,gfx_api::texture_description<4,2,2,3>,gfx_api::texture_description<5,0,0,0>>,2>::bind() Line 541	C++
 	warzone2100.exe!drawInstanced3dShapeTemplated_Inner<2,gfx_api::pipeline_state_helper<gfx_api::rasterizer_state<2,0,255,1,2,0>,2,0,std::tuple<gfx_api::Draw3DShapeInstancedGlobalUniforms,gfx_api::Draw3DShapeInstancedPerMeshUniforms>,std::tuple<gfx_api::vertex_buffer_description<12,0,gfx_api::vertex_attribute_description<0,1,0>>,gfx_api::vertex_buffer_description<12,0,gfx_api::vertex_attribute_description<3,1,0>>,gfx_api::vertex_buffer_description<16,0,gfx_api::vertex_attribute_description<1,2,0>>,gfx_api::vertex_buffer_description<16,0,gfx_api::vertex_attribute_description<4,2,0>>,gfx_api::vertex_buffer_description<96,1,gfx_api::vertex_attribute_description<5,2,0>,gfx_api::vertex_attribute_description<6,2,16>,gfx_api::vertex_attribute_description<7,2,32>,gfx_api::vertex_attribute_description<8,2,48>,gfx_api::vertex_attribute_description<9,2,64>,gfx_api::vertex_attribute_description<10,3,80>,gfx_api::vertex_attribute_description<11,3,84>>>,std::tuple<gfx_api::texture_description<0,3,0,0>,gfx_api::texture_description<1,0,0,0>,gfx_api::texture_description<2,3,0,0>,gfx_api::texture_description<3,3,0,0>,gfx_api::texture_description<4,2,2,3>,gfx_api::texture_description<5,0,0,0>>,2>>(ShaderOnce & globalsOnce, const gfx_api::Draw3DShapeInstancedGlobalUniforms & globalUniforms, const iIMDShape * shape, gfx_api::buffer * instanceDataBuffer, unsigned __int64 instanceBufferOffset, unsigned __int64 instance_count, gfx_api::texture * lightmapTexture) Line 1472	C++
 	warzone2100.exe!??@f9127b39a461518384315cddb5480d8f@(ShaderOnce & globalsOnce, const gfx_api::Draw3DShapeInstancedGlobalUniforms & globalUniforms, const iIMDShape * shape, int pieFlag, gfx_api::buffer * instanceDataBuffer, unsigned __int64 instanceBufferOffset, unsigned __int64 instance_count, gfx_api::texture * lightmapTexture) Line 1551	C++
 	warzone2100.exe!pie_Draw3DShape2_Instanced(ShaderOnce & globalsOnce, const gfx_api::Draw3DShapeInstancedGlobalUniforms & globalUniforms, const iIMDShape * shape, int pieFlag, gfx_api::buffer * instanceDataBuffer, unsigned __int64 instanceBufferOffset, unsigned __int64 instance_count, bool depthPass, gfx_api::texture * lightmapTexture) Line 1602	C++
 	warzone2100.exe!InstancedMeshRenderer::Draw3DShapes_Instanced(unsigned __int64 currentGameFrame, ShaderOnce & globalsOnce, const gfx_api::Draw3DShapeInstancedGlobalUniforms & globalUniforms, int drawParts, bool depthPass) Line 1633	C++
 	warzone2100.exe!InstancedMeshRenderer::DrawAll(unsigned __int64 currentGameFrame, const glm::mat<4,4,float,0> & projectionMatrix, const glm::mat<4,4,float,0> & viewMatrix, const ShadowCascadesInfo & shadowCascades, int drawParts, bool depthPass) Line 1437	C++
 	warzone2100.exe!pie_DrawAllMeshes(unsigned __int64 currentGameFrame, const glm::mat<4,4,float,0> & projectionMatrix, const glm::mat<4,4,float,0> & viewMatrix, const ShadowCascadesInfo & shadowMVPMatrix, bool depthPass) Line 1336	C++
 	warzone2100.exe!drawTiles(iView * player, LightingData & lightData, LightMap & lightmap, ILightingManager & lightManager) Line 1522	C++
 	warzone2100.exe!draw3DScene() Line 1018	C++
 	warzone2100.exe!displayWorld() Line 1179	C++
 	warzone2100.exe!renderLoop() Line 320	C++
 	warzone2100.exe!gameLoop() Line 698	C++
 	warzone2100.exe!runGameLoop() Line 1172	C++
 	warzone2100.exe!mainLoop() Line 1308	C++
 	warzone2100.exe!wzEventLoopOneFrame(void * arg) Line 3708	C++
 	warzone2100.exe!wzMainEventLoop(std::function<void __cdecl(void)> onShutdown) Line 3728	C++
 	warzone2100.exe!realmain(int argc, char * * argv) Line 2229	C++
 	warzone2100.exe!SDL_main(int argc, char * * argv) Line 94	C++
 	warzone2100.exe!main_getcmdline() Line 80	C
 	warzone2100.exe!WinMain(HINSTANCE__ * hInst, HINSTANCE__ * hPrev, char * szCmdLine, int sw) Line 110	C

Additional info:

• Target OS: Windows 11
• MSVC toolchain

[ManManson]

Some minor comments, plus the PR crashes for OpenGL rendering backed. Otherwise, looks good.

Also, please rebase the branch to resolve the conflicts against master.

[ManManson]

No need to keep commented code. We can always bring it back via git, if needed.

[ManManson]

Can you please elaborate why in this particular place the condition is != 1, when in other places (e.g. tcmask_instanced.frag) it's != 0? If this is expected, a comment about this would be really helpful.

[ManManson]

Ditto.

[past-due]

Something seems to be broken in this PR with regard to OpenGL rendering backend. Even with "volumetric lighting = off" (but also reproduces with other value for the setting), when I start beta 1 campaign, I consistently see null pointer dereference exception at gfx_api_gl.cpp:2601:

@ManManson Can you try running again with command-line option --debug=3d and then check the log .txt file? I suspect you’ll see issues with shader compilation, and the specific output for that would be helpful.

[ManManson]

Sure, I'll try to check in a few days.

[ManManson]

Here are some relevant details from the log file:

G:\repos\warzone2100\build\src\warzone2100.exe caused an Access Violation at location 0xlX in module G:\repos\warzone2100\build\src\warzone2100.exe Reading from location 00000010.

Log message: 3d      |16:00:16: [gl_pipeline_state_object::fetch_uniforms::<lambda_dfc68f02ba1f05ed9d4aa1088a6cd335>::operator ():1887] [Component program]: Did not find uniform: fogEnabled
Log message: error   |16:00:16: [khr_callback:3394] last message repeated 8 times (total 16 repeats)
Log message: 3d      |16:00:16: [gl_pipeline_state_object::fetch_uniforms::<lambda_dfc68f02ba1f05ed9d4aa1088a6cd335>::operator ():1887] [Component program]: Did not find uniform: PointLightsPosition
Log message: 3d      |16:00:16: [gl_pipeline_state_object::fetch_uniforms::<lambda_dfc68f02ba1f05ed9d4aa1088a6cd335>::operator ():1887] [Component program]: Did not find uniform: PointLightsColorAndEnergy
Log message: 3d      |16:00:16: [gl_pipeline_state_object::fetch_uniforms::<lambda_dfc68f02ba1f05ed9d4aa1088a6cd335>::operator ():1887] [Component program]: Did not find uniform: bucketOffsetAndSize
Log message: 3d      |16:00:16: [gl_pipeline_state_object::fetch_uniforms::<lambda_dfc68f02ba1f05ed9d4aa1088a6cd335>::operator ():1887] [Component program]: Did not find uniform: PointLightsIndex
Log message: 3d      |16:00:16: [gl_pipeline_state_object::fetch_uniforms::<lambda_dfc68f02ba1f05ed9d4aa1088a6cd335>::operator ():1887] [Component program]: Did not find uniform: viewportWidth
Log message: 3d      |16:00:16: [gl_pipeline_state_object::fetch_uniforms::<lambda_dfc68f02ba1f05ed9d4aa1088a6cd335>::operator ():1887] [Component program]: Did not find uniform: viewportHeight
Log message: 3d      |16:00:16: [gl_pipeline_state_object::fetch_uniforms::<lambda_dfc68f02ba1f05ed9d4aa1088a6cd335>::operator ():1887] [Component program]: Did not find uniform: cameraPos
Log message: 3d      |16:00:16: [gl_pipeline_state_object::fetch_uniforms::<lambda_dfc68f02ba1f05ed9d4aa1088a6cd335>::operator ():1887] [Component program]: Did not find uniform: tcmask
Log message: 3d      |16:00:16: [gl_pipeline_state_object::fetch_uniforms::<lambda_dfc68f02ba1f05ed9d4aa1088a6cd335>::operator ():1887] [Component program]: Did not find uniform: normalmap
Log message: 3d      |16:00:16: [gl_pipeline_state_object::fetch_uniforms::<lambda_dfc68f02ba1f05ed9d4aa1088a6cd335>::operator ():1887] [Component program]: Did not find uniform: specularmap
Log message: 3d      |16:00:16: [gl_pipeline_state_object::fetch_uniforms::<lambda_dfc68f02ba1f05ed9d4aa1088a6cd335>::operator ():1887] [Component program]: Did not find uniform: hasTangents
Log message: last message repeated 10 times (total 26 repeats)
Log message: error   |16:00:16: [khr_callback:3394] GL::API(Error:High) : GL_INVALID_OPERATION error generated. <program> has not been linked, or is not a program object.
Log message: error   |16:00:16: [khr_callback:3394] GL::API(Error:High) : GL_INVALID_OPERATION error generated. <program> object is not successfully linked, or is not a program object.
Log message: error   |16:00:16: [khr_callback:3394] last message repeated 2 times
Log message: error   |16:00:16: [khr_callback:3394] last message repeated 2 times (total 4 repeats)
Log message: 3d      |16:00:16: [gl_pipeline_state_object::getLocs:1526] Missing expected sampler uniform: shadowMap
Log message: 3d      |16:00:16: [gl_pipeline_state_object::getLocs:1526] Missing expected sampler uniform: lightmap_tex

Is this output sufficient to pinpoint the problem?

[ManManson]

I'll take another stab at this in a few days and will try to fix the remaining issues mentioned above so we can proceed with merging this to upstream.