@timon

Last Devlog (as im finished) FINALLY 😁

Whats new:

I tested alot with the vibrancy values and changed the sky color values to:

baseSky.r *= 0.1;baseSky.g *= 0.45;baseSky.b *= 9.0;
The 9.0 wont change anything at daytime, but makes the sky way more blue on night time.

I used the same lighting setup for both day and night. Now I use seperate Light values making the night natively darker while keeping block lighting the same as before.

float skyLight = day ? lm.y : lm.y * 0.45;vec3 lighting = skyLight * (vec3(0.5, 0.62, 0.75) + vec3(1.0, 0.92, 0.78) * sun * shadowCol)+ vec3(1.0, 0.6, 0.3) * lm.x * lm.x + 0.03;

Also, before the entites were glowing very hard. This was because the lightmap was applied directly on the entities. I removed it:

lmdata = vec4(lmcoord, 0.0, 1.0); (Picture of the glowing pig also here)

I made some small and easy updates. I tested alot and wrote alot and deleted most things. Gemin had the Fog Idea :)

First Change:
I changed the gbuffers_skybasic.fsh color grading for the sky. It makes the sky way more light blue isntead of the dark blue I had before (image included)

baseSky.r *= 0.1;
baseSky.g *= 0.65;
baseSky.b *= 1.8;

Second Change:
I added Exposure to the final.fsh. One Problem was that it got so bright, that lighter texutres like sand looked basically white. So i went in the composite.fsh and multiplied the lighting by 0.85 so it will get darker and compensates the Exposure (image included before and after):

final.fsh:
float exposure = 1.3;color.rgb *= exposure;

composite.fsh:
vec3 finalColor = albedo.rgb * lighting * 0.85;

Third Change:
I added a fog near the render distance. It was Geminis Idea to add fog. It was only 2 lines to add:

float fog = smoothstep(far * 0.7, far, length(viewPos.xyz));finalColor = mix(finalColor, fogColor, fog);

This is a devlog with much time because first i forgot to devlog and second because i had one huge problem and i was working on it for like two and a half hour (also much with ai on that problem, ill admit that i was furious after 1,5 hours) So yeah.

Whats new: Shadow Rewrite, Color Grading, Lighting, Wave update.

Starting Small: I made Waves faster to make them more visible.

Before:
float wave = sin(p.x * 1.20 + frameTimeCounter * 2.50) * 0.01;

Now:
float wave = sin(p.x * 1.20 + frameTimeCounter * 2.50) * 0.05;

Like seen in the last Devlogs, the shadow never really looked nice. Before i used like manual 5 samplig grid to soften the shadows which is okay for performance but it looks weird.
In the new version i completely threw that out. Instead I decided to move over to PCF Percentage Closer FIltering (was claudes idea). It makes the shadow look soft and way smoother.

vec3 pcf(vec4 clip) {
ivec2 s = ivec2(texcoord * vec2(viewWidth, viewHeight)) % 64;
float a = texelFetch(noisetex, s, 0).r * 6.2831853;
mat2 r = mat2(cos(a), -sin(a), sin(a), cos(a));
vec3 sum = vec3(0.0);
float n = 0.0;

for (int x = -PCF_RANGE; x <= PCF_RANGE; x++) {
    for (int y = -PCF_RANGE; y <= PCF_RANGE; y++) {
        vec4 p = clip + vec4(r * (vec2(x, y) * (PCF_RADIUS / 8092.0)), -0.002, 0.0);
        p.xyz = warpShadowClipPos(p.xyz);
        p.xyz = p.xyz / p.w * 0.5 + 0.5;
        if (all(greaterThanEqual(p.xy, vec2(0.0))) && all(lessThanEqual(p.xy, vec2(1.0)))) {
            sum += sh(p.xyz);
            n += 1.0;
        }
    }
}

return n > 0.0 ? sum / n : vec3(1.0);

}

While rewriting the composite I had multiple transparent bugs (seen in the pictures below). Basically my palyer hand and mobs were see through.
The mob bug was pretty easy fix. gbuffers_entities.vsh wasnt actually passing down normal vectors to the fragment shader. (This was also found by gemini not me)
For the Hand fix, i ran into a problem becuase it was the fault of composite.fsh. I was doing some weird depth buffer checks that basically told the shader to ignore the hand and draw the world behind it. (And again this was found by gemini)

In the composite shader, I had this depth check to filter out water and sky:
if (d0 != d2) {
color = albedo;
return;
}

By using if (d0 == 1.0 || d0 < d1 || d0 != d2) the shader now correctly renders it. The Problem was becase how Miencraft renders the hand. The hand renders into depthtex2 but it usually doesnt render into depthtex0. Because the hand depth in d2 was different freom the sky block depthh in d0 it triggered and the composite shader skipped drawing the hand. ( I hope I understood it right).

Then came the 2.5 hour problem 😩:
Random blocky shadows appeated on water (seen in pictures). Because the problem didnt occured before and because I just reworked the composite i tried everything. I changed things back to old. asked AI nothing helped. In the end it was because gbuffers_water.vsh was calcuilating the shadow coordiantes without applying the shadow map distorion. It was trying to read warped shadow map using undistoreted coordinates. nce I added the #include “libs/distort.glsl” it worked… (again found by gemini, i troubleshooted ALOT with it).

Lasty i addewd Color Grading tweaks in final.fsh. Just more vibrance and contras.

I spent some time remaking the water. Before the Waves were pretty symmetrical. I added more randomness by adding more cos and sin waves with different speed and direction.

Before:

wavePos = floor(wavePos * 16.0) / 16.0;
float wave = sin(wavePos.x * 1.5 + frameTimeCounter * 2.0) * 0.08 +
cos(wavePos.z * 1.5 + frameTimeCounter * 1.8) * 0.08;

After:

float wave = 0.0;
wave += sin(wavePos.x * 1.2 + frameTimeCounter * 1.6) * 0.07;
wave += sin(wavePos.z * 1.4 + frameTimeCounter * 1.9) * 0.05;
wave += sin((wavePos.x + wavePos.z) * 1.0 + frameTimeCounter * 1.1) * 0.04;
wave += cos((wavePos.x - wavePos.z) * 1.1 + frameTimeCounter * 1.3) * 0.045;

Then I went to the gbuffer_terrain.fsh and gbuffer_terrain.vsh. It is there fore the foundation for shadow mapping by calculating the shadow coordinates in the vertex shader and getting the shadow map.

When i finnished that i went to the composite.fsh to actually getting the shadows to run.
But first i began working on the overall lighting to make it feel much brighter and cooler.

vec3 sunLight = vec3(1.0, 0.92, 0.78) * shadow * lm.y;vec3 skyAmbient = vec3(0.3, 0.42, 0.55) * lm.y;vec3 blocklight = vec3(1.0, 0.6, 0.3) * lm.x * lm.x;vec3 lighting = sunLight + skyAmbient + blocklight + 0.03;

Now the shadows. The shadowSample() function takes the reconstructed player position and normal ( reconstructed player = 3d world position from a pixel, normal = where a surface is facing to) and transforms it into the shadow map.

I also added 6 shadow samples around the pixel (geminis advice) to make the shadows softer instead of sharp (Though i might change that because as you will se in the picture the shadow is ugly… It might be because of that).

float shadow = step(sp.z, texture2D(shadowtex0, sp.xy).r) * 0.4+ step(sp.z, texture2D(shadowtex0, sp.xy + vec2( r, r)).r) * 0.15+ step(sp.z, texture2D(shadowtex0, sp.xy + vec2(-r, r)).r) * 0.15+ step(sp.z, texture2D(shadowtex0, sp.xy + vec2( r, -r)).r) * 0.15+ step(sp.z, texture2D(shadowtex0, sp.xy + vec2(-r, -r)).r) * 0.15;

The shadow itself is actiavated in main() with:

float shadow = shadowSample(playerPos, normal);

After all that though, the shadow didnt work and i was looking for it like 30 minutes and also troubleshootign with AI and after changeing the code like 10 times it also didnt work so I reverted the changes and noticed that:

uniform float shadowDistance; is wrong because it of course needs a number :/

-> uniform float shadowDistance = 128.0;

Im making a shader for Minecraft and after upgrading the template from 1.21.5 to 26.2 I implemented waves to water.. It took long 😭 but it wasnt that difficult. I also added a pciture here.

With mc_Entity i added blockID for specifically Water to the IDs 10034 and 10033. For the animation I used sine and cosine functions.
After i styled the Water by blending its texture with a blue tint:

vec3 waterColor = vec3(0.1, 0.4, 0.8);

Though, because it was all the same color, you couldnt really see the waved that were further away than 3 blocks, so I implented a bit of shadow:

shadow = texture(shadowtex0, shadowCoord.st).r;albedo.rgb *=mix(0.5, 1.0, shadow);

I improved the GPU readback so the CPU no longer has to wait for the GPU every frame (Gemini noticed that this happend and with its help I fixxed it). Before the resul buffer was copied and read immediately which could cause the CPU to block while the GPU was still processing shader.

This is difficult because getting the CPU and GPU sync right is tough. As the CPU should only read a buffer when the GPU finished writing on it.
I used ID3D11Query events for it:

if (r.pending &&
context->GetData(r.query.Get(), nullptr, 0, 0) == S_OK)
{
context->Map(r.buffer.Get(), 0, D3D11_MAP_READ, 0, &m);
serial.send((const UINT*)m.pData);
context->Unmap(r.buffer.Get(), 0);
r.pending = false;
}

Fixed a few issues in the shader. First of all the border mapping wasnt complete and I forgot the top and left side of the LEDs.. Also removed Typos. I also added sRGB and improved the saturation handling as before they didnt match accurate. I fixed it by the colors to linear space before processing them.

Before:

float luma = dot(rgb, float3(0.2126, 0.7152, 0.0722));
rgb = saturate(lerp(luma.xxx, rgb, saturationBoost) * brightness);

After:

sum += ToLinear(desktop.Load(int3(pixel, 0)).rgb);

float3 rgb = ToSrgb(sum / (samplesAcross * samplesDeep));
rgb = saturate(BoostSaturation(rgb) * brightness);

Using an HLSL compute shader and [numthreads(90, 1, 1)] the GPU spawns 90 parralel threads which are calculating the average color for every LED simaultaneously.

As only picking one pixel at each led would be pretty inaccurate, we are taking zones and use the average color of ever zone.

(For now the average of every zone is picked, if I am testing it in real
life, we might change it - maybe the outer pixels are doubled for a more accurate color that looks better at the end).

I had an issue where some LEDs were using the wrong part of the screen because of incorrect index calculation. I fixed it by correcting the LED and screen coordinates:

(in the picture)

Before doing the PC side and video capture code, I made the the code for the physical ahrdware (the esp32). I set up a Arduino sketch using FastLED library to manage the output signal on GPIO (in my case DATA_PIN 2), I configured the setup for the 90 LEDS I have on the RGB Strip (ws2812b usually have 60 LEDS per 1m and i need 1,5m). I set the communication speed to 921.600 baud.

void setup() {    FastLED.addLeds<WS2812B, DATA_PIN, GRB>(leds, NUM_LEDS);    FastLED.setBrightness(BRIGHTNESS);    FastLED.clear(true);    Serial.begin(921600);}

To avoid lagging, I built a simple step by step parser that handles incoming data byte by byte:

1. Ready how many Leds are sent
2. It Checks if the data is valid and mathces the 90 LEDS and if anything is wrong it resets and waits for the next frame.
3. Once all 270 bytes (90 lights * 3 colors) arrive, it calls FastLED.show() to light up the strip

I started working on the communication between the PC and the LED hardware with serial. Serial is the channel used to send binary color data directly to the esp32.
To make sure the lights dont stay lit when the program closes the destructor sends a black frame on exit to turn of all lights. I didnt notice that it would be a problem but when troubleshooting for another proble, gemini noticed it:

~Serial() { black(); if (handle_ != INVALID_HANDLE_VALUE) CloseHandle(handle_); }

I am using DXGI Desktop duplication that can capture the frames directly from the gpu. This is good so the frames dont need to be loaded again for the LEDs.

I started with creating the circuit in KiCad. I noticed that there was not the right ESP32 for me so i looked online and found this:
https://github.com/davidkleymann/doit-esp32-devkit-kicad

I build everything in KiCad just to see that the libraries are probably outdated (or i just did something wrong but i dont believe that)
After about 20 of trouble shooting I got it working but I noticed that KiCad is not the best place for what im trying to do so I gave up. I searched for a new programm and found https://wokwi.com.

It worked well. Though, they did not have splicing connectors / wago connectors so i made it with a Breadboard in this circuit instead. In the real version I will do it instead with splicing connectors.
Also the LED strip name for the negative power supply pin was VSS, i thought it should be GND and learned the difference then.