Beginnings of player input

Because shader code is run entirely on the GPU, I can’t just directly listen for keyboard input. The only thing that the shader code actually reads is the colors rendered to the screen by minecraft, so I needed to make a sort of “translation layer” that listens for player input in minecraft and outputs a color on the screen so it can be read and interpreted by the shader code.

My really jank test rig

As shown in the video, the color of the square updates depending on the pressed keys. The inputs are gathered using minecraft’s “predicate” system and are translated to the background color of a text display entity. The player is mounted to an armor stand to stop them from moving away, keeping the text display in the center of the screen always.

How I keep track of data in code that can’t store variables

If a shader can only read colors, how does it remember where the player is?
This question was one that I had until I discovered minecraft’s support for the persistent render target. This allows the shader to sample from the pixels it renders in the current frame in the next frame. A floating point number is stored using thirty two bits. Each pixel’s color is stored with eight bits of precision each for red, green, blue, and alpha channels, which also happens to total 32 bits.
So I wrote what is possibly the strangest utility I have ever written, that can translate a float to an rgba color and an rgba color to a float. Now, using a persistent render target that is simply never rendered to the screen, I can store the player’s x, y, and camera position in colors, and update them based on the color in the middle of the screen from the minecraft render.

I suppose the persistent render target sorta counts as the game’s “RAM.” I’ll be using it to store a lot more data in the future, but next I’ll be fixing up some issues I found with rendering and player control.

I finally got some actual visual output!! After fixing many compiler errors, and then another few that kept causing the loaded data to reset, I got this output. The current system renders walls based on the pixel’s distance from a wall defined by a texture (visualized in the picture). This created a sort of fisheye distortion that I’ll try and work out later, but I’m happy with my progress for now, and would like to focus on getting player movement functional

Minecraft has in recent snapshots greatly improved the capabilities of its shader system, now allowing postprocessing shaders “post effects” to be applied using commands. This has wildly changed what is now possible in vanilla minecraft and I immediately was overcome with excitement about what was now possible. After playing around with it for a while and discovering the “persistent” mode on custom render targets, I realized that even more than expected was now possible, and found a way to make posteffects that have fade-in animations by updating the value in a persistent render target over the course of a few frames. To make it consistent across different frame rates, I had to figure out a way to get a deltatime (time between frames), which I was able to do using the GameTime uniform in minecraft.

Having gotten a deltatime, I began wondering about the possibilities of creating a fully functional game contained entirely in a minecraft postprocessing shader…