4D raymarching engine - 06/05/21

---

So for one of my university modules, we had to create some code in C++/OpenGL that would demonstrate some kind of interesting graphical effect or simulation. I was torn for what to do for this for a while, until I remembered some work I had done nearly 2 years ago. I had become somewhat interested in 4D geometry, and wanted to make a visualiser.

At this point, I'm going to have to explain the 4th dimension yet again, like I had to do so many times over the course of this project. Imagine a 2D game. In this game, you can go left/right, and also jump (up/down). These are the 2 dimensions in the "2D" part. In a 3D game, you can go left/right, forwards/backwards and up/down. In this 4D engine, there is an extra axis of movement. I have only found "ana" and "kata" as alternatives for the usual left/right style of names.

Another name for the axis is W, coming from left/right being X, up/down being Y and in/out being Z (I know several people will take umbrage with my ordering of Y and Z. This order makes more sense to me). To those that say the 4th dimension is time, I'm talking about a 4th spatial dimension, as otherwise most game engines are 4D, and the programming becomes trivial.

Looking on the internet, the only substantial project in a 4D engine seems to be Miegakure. This game has been in development for around 10 years, with no release date in sight, however in that time they have done a great job making visualisations. If you don't understand my explanation of 4D space, they have lots of nice videos about it.

When I did my work over the summer, exploring how this could be done, I tried to compute the intersection between the 4D world, and the 3D view that the player has, to generate new geometry that could be displayed in a standard 3D engine. However, I became stuck after some small prototypes, and let the project go. I believe this is the system that Miegakure uses for their renderer, if im understanding their protional video correctly.

When I revisted it for this university module I again tried to struggle on with this approach, but again found the maths too tricky. This was when I had a stroke of luck. I follow the youtube channel 3Blue1Brown, and was browsing his website, and then his twitter. He shared a video by Inigo Quilez where they used a technique known as raymarching to create a 3D scene of a girl taking a selfie.

I was already familiar with raymarching as a rendering technique, from watching CodeParade's video on the topic, so I started looking at Inigo's other videos. In some of them, he does mathematical tutorials on raymarching itself, and one of them was on deriving the SDF of a box. Right at the end he just offhandedly mentioned "However, the single expression we derived does work in 4 dimensional space without any changes, and actually makes it trivial to work in such a bizarre space". This completely blew my mind, as it was the solution I had been looking for.

I started looking into raymarching in a lot more depth, and while OpenGL itself did not support it, I could write a fragment shader that just rendered it onto two triangles, which are always placed in front of the camera. This also solved the fact that I wasnt particularly familiar with OpenGL in C++, and was struggling to get objects to display properly in 3D. I knew a decent amount of how shader code worked, due to previous little experiments in things like Shadertoy (Which I later found out is run by Inigo Quilez himself. It's a small world). Getting some basic 3D rendering wasn't too hard, once I fixed some irritating bugs that warped the view near the edges fo the screen.

Once I had realtime raymarching working inside the shader (Running at around 2000 fps on my gpu at full speed, causing it to give off some interesting noises), most of the work required to convert it to a 4D renderer was just changing all the vec3 to vec4 in the shader, and updating the SDFs to work with 4D primitives. This worked surprisingly well and I had that up and running within a day. I had now technically satisfied the main point of the assignment, but there were a lot of sub-points i needed to satisfy as well.

One line of the marking spec was the bane of my existence: "– Loading of meshes from a standard 3D object file format". This was a required part of the submission, and posed several problems. First, 3D wasnt really a thing in this context. Second, there was no standard format for 4D models. Thirdly, how on earth was I going to load a mathematical formula from a file, into C++, then over to the GPU, which would then render them properly.

After discussing with my much more clever friend, I decided that the best way to do it was probably to bave a static set of primitives (Cube, Sphere etc) that I could create in the scene at a certain position, and essentially perform a boolean union on. Unioning multiple objects in an SDF is fine, since you simply take the minimum of all of the individual SDFs. Then I just had to write a file handler to pass the data into the shader, and hope that "A file format with a defined standard" would count as a "Standard file format" in the eyes of the assessor.

It took a little while to get all of that working, since string parsing is not exactly C++'s forté, but I did manage to get it working, finally. At that point, all I had left to do was collision detection and some "advanced graphical effects". Collision detection was a little annoying to do, since I effectively had to rewrite a lot of the shader in raw C++, to be able to cast a ray into the scene, since the entire representation of the scene was inside the shader. It isn't a particularly in-depth collision system, simply casting a ray out of the camera, and if the scene is too close, you arent allowed to do that movement. However, this worked perfectly to also stop you walking along to 4th dimension too far. Oddly, it didn't work in the negative X and Z directions, so I simply made my demonstration scene only be in the positive X and Z quadrant.

All that I had left to do was some sort of "advanced graphical effect", so first I added a glow around all objects (CodeParade mentioned how to do this in a video a while back, basically keeping track of the closest the ray ever got to the scene, and adding glow based on that), and then decided to add some lighting. After some attempts which crashed my shader, I got shadows to render properly, even getting soft shadows as a bonus thanks to some code I found on Inigo's website. I attempted to add in Phong Reflections to make the lighting look a little better, but all the example code I could find seemed to break in weird ways. Rewatching the Selfie Girl video, I saw a simple shading technique being used that was similar to Phong, but slightly different. This was quick to implement, and made much more sense to me.

At this point, the deadline was starting to approach, so I needed to start doing parts of the project not directly related to programming. One of the most important parts was profiling the code you had written. Unfortunately, all of the important parts of my code were in the shader. Everything else executed pretty much exactly the same code each frame, so it was a little pointless to try and profile it. Unfortunately, I was unable to profile the actual innards of the shader code. The built in NSight extension in visual studio showed the execution time of my frames and the draw calls, but that didn't help. I was advised to try the standalone Nsight graphics application from nvidia, which seemed promising, however, shader profiling was only support on Turing (20xx) and Ampere (30xx) cards, whereas I use a Pascal (10xx) card, and was unable to use those features. I explained this to the assessor, so hopefully they will be leniant in this regard.

In conclusion to the project, I really dislike C++. I always felt like I was fighting it to do what I wanted when programming, so much so that I preferred working on the GLSL side much more. Having previously programming in assembly and python, extreme ends of the computing level, C++ feels like a weird middle ground, where you have to worry about really low level concepts like memory addresses, but with high level features like polymorphism thrown in. It just feels weird to me.

On the other hand, now I have a nice base raymarching setup that I can experiment with to do all sorts of stuff. I just have to remove the weird features that I had to shoehorn in to meet the marking criteria, and I think I'll be left with a decent engine for future use.

The code can be downloaded here, and I dont even know what you need to run it anymore. Probably C++ and OpenGL.