Mapping OpenJK

[BruceJohnJenner]

Sooo,

 

This is a bit of a broad question. I now have a fork of OpenJK working on Ubuntu linux, so I can now tinker with the source at will. But, not unexpectedly, the source code for the project is massive, quite a bit for me to take in all at once. For comparison, my largest project would be this:

 

https://github.com/BruceJohnJennerLawso/Ignition_Engine

 

which is a minnow compared to OpenJK. I also see from browsing the code, that a lot of it is written in functional style as opposed to object oriented. This again isn't particularly surprising given that the game is over 10 years old, and speed was probably a fairly important factor in its design, but the code is proving rather difficult to make sense of. What I was hoping to do, is create a quick roadmap of where everything is in OpenJK, and a rough description of what it does.

 

So far the only things I have been able to figure out is this:

 

-win32 and macosx under the code folder (only looking at JKA code here) are obviously related to lower level stuff on the given operating system (creating a window, getting a handle to it, drawing stuff to that window, etc.). If that is the case, I dont follow why sdl is also there, since sdl would cover all of those tasks on its own wouldnt it? Or maybe SDL is only there to provide support for low level stuff on linux that wasnt available when the commercial version was released?

 

-I think ghoul2 is somehow related to in game meshes, more specifically taking a static model that a modeller creates in an external program like blender, 3dsmax, what have you, and making it interface with the game physics and animations? Is it some sort of external library that handles things like this?

 

-No clue what icarus is, If anyone could comment on that, it would be very helpful.

 

-game and cgame are fairly obvious, some game constants were defined in the headers in game, which allowed for changing a few things like the speed of weapons fire, although much of that is a mystery to me at the moment.

 

Beyond that, Im not sure what most of the source does. Is there a good starting point in the code to look at that would help me get a better understanding of how it all works?

 

 

[eezstreet]

SDL is used on Linux and I think also Mac for window management. It isn't used on Windows (yet). There's other platform-specific code as well. SDL was never in the commercial release of the game.

 

GHOUL2 (.glm) is a proprietary format created by Raven for SOF2 and later adapted for JK2/A. It was rather controversial for its time due to the level of graphic violence that was portrayed in SOF2 (but was toned down for JK2/A). The ghoul2 folder is the renderer and physics code for it. The game still uses MD3 models for a few things.

 

ICARUS is the singleplayer scripting system (.IBI files)

 

As for game/cgame/ui, Quake 3 runs up to three concurrent virtual machines. These were originally in QVM files, which were more LLVM-like in nature. JA (specifically) alters this so they are compiled as native binaries. In multiplayer, these compile into three different DLLs: cgamex86.dll, jampgamex86.dll and uix86.dll. The cgame module is essentially the client-side virtual machine, while game is the server-side. Each mod can have its own cgame/game DLLs so it can have its own behavior without having to modify the engine.

In singleplayer, the UI module is absent (it's part of the engine), and the game and cgame modules are combined as simply jaspgamex86, since the client/server separation is not necessary in SP. OpenJK modifies the loading procedure for SP so that the jaspgamex86 can be used from a different folder.

 

Are you more interested in MP or SP?

[Futuza]

I highly suggest you jump on the JACoders IRC if you haven't been visiting already, but yes agree a roadmap for the code would be beneficial.

[BruceJohnJenner]

SDL is used on Linux and I think also Mac for window management. It isn't used on Windows (yet). There's other platform-specific code as well. SDL was never in the commercial release of the game.

 

GHOUL2 (.glm) is a proprietary format created by Raven for SOF2 and later adapted for JK2/A. It was rather controversial for its time due to the level of graphic violence that was portrayed in SOF2 (but was toned down for JK2/A). The ghoul2 folder is the renderer and physics code for it. The game still uses MD3 models for a few things.

 

ICARUS is the singleplayer scripting system (.IBI files)

 

As for game/cgame/ui, Quake 3 runs up to three concurrent virtual machines. These were originally in QVM files, which were more LLVM-like in nature. JA (specifically) alters this so they are compiled as native binaries. In multiplayer, these compile into three different DLLs: cgamex86.dll, jampgamex86.dll and uix86.dll. The cgame module is essentially the client-side virtual machine, while game is the server-side. Each mod can have its own cgame/game DLLs so it can have its own behavior without having to modify the engine.

In singleplayer, the UI module is absent (it's part of the engine), and the game and cgame modules are combined as simply jaspgamex86, since the client/server separation is not necessary in SP. OpenJK modifies the loading procedure for SP so that the jaspgamex86 can be used from a different folder.

 

Are you more interested in MP or SP?

 

Would it not make some sense to port everything to SDL, or are the Win32 and OSX parts just there to smooth over small areas where SDL doesnt fit quite perfectly due to differences in how operating systems handle things? Just a wild thought here, but would porting things to SFML be a possibility? I am a big fan of SFML personally.

 

I would imagine that ICARUS and Ghoul2 dont really need much modification at this point, more just adjusting how the game uses them internally in game and cgame? Is ghoul2 basically just a filetype that normal mesh files like 3ds and obj can be ported to?

 

Why exactly is LLVM used to run virtual machines? (ie what problem requires setting one up instead of just writing normal C code to do it?)

 

Are all users in a multiplayer session required to use the same versions of dlls though? If a mp session was using different code for cgame, things would behave differently for everybody wouldnt they?

 

Generally more interested in SP, but Im more interested in just tinkering with game mechanics to do interesting stuff. Even just modifying the speed that weapons fire moves at to be faster has a big effect on SP gameplay. But changes like that are obviously not part of the OpenJK branch, so I think Im going to just make that into a separate repository that isnt related to OpenJK.

 

I would love to tinker with integrating bullet physics into the game though. I think somebody already experimented with that before, and it looked really cool with crates & objects bouncing around.

 

 

I highly suggest you jump on the JACoders IRC if you haven't been visiting already, but yes agree a roadmap for the code would be beneficial.

 

Ill make sure to stop by the IRC, maybe sometime on this weekend if I have time.

 

What I was thinking of in place of a roadmap, would be to create a quick comment at the top of each file in OpenJK that describes what it does. Sooo, I think Ill refork OpenJK, and make pull requests to master with the commented files, and whoever is in charge of the main branch can review the comments to make sure theyre accurate, then add them to master.

 

Does that sound like a decent idea?

[Futuza]

Documentation of source code is always good.  So I'm not going to disagree with you.

[BruceJohnJenner]

Documentation of source code is always good.  So I'm not going to disagree with you.

Well I didnt think you would argue against it, but is it okay to submit requests directly to master this way or would it be better to do on another branch?

[Xycaleth]

Would it not make some sense to port everything to SDL, or are the Win32 and OSX parts just there to smooth over small areas where SDL doesnt fit quite perfectly due to differences in how operating systems handle things? Just a wild thought here, but would porting things to SFML be a possibility? I am a big fan of SFML personally.

That's exactly what I'm in the middle of doing right now - porting the Windows build to SDL. There's a fair amount of code behind the SDL integration into JKA, so moving to SFML just for the sake of it isn't really worth it. SDL already does everything we need (and more).

 

I would imagine that ICARUS and Ghoul2 dont really need much modification at this point, more just adjusting how the game uses them internally in game and cgame? Is ghoul2 basically just a filetype that normal mesh files like 3ds and obj can be ported to?

Both the Ghoul2 and ICARUS code are a bit messy and it would be good to see them cleaned up/streamlined, as long as all the same functionality was kept. Ghoul2 is more of a collection of formats - there's GLM and GLA for models and animations respectively. GLM stores the model data and GLA stores the skeletal animation data - so yes, normal meshes can be converted to GLM.

 

 

Why exactly is LLVM used to run virtual machines? (ie what problem requires setting one up instead of just writing normal C code to do it?)

LLVM has nothing to do with it actually. Originally in Q3, SoF2 and JK2, the cgame, game and ui modules used to be compiled down to byte code using a special C compiler (LCC - Little C Compiler I think it was called) and stored in "QVM" files. When the .qvm files got loaded, they were then JIT compiled into native machine code. This made it possible to compile the cgame/game/ui code once and it could be run on any platform. The downside is that you're completely sandboxed in - you have no standard libraries, and you rely solely on the engine to provide all of your interaction with the outside world. So you couldn't even use C libraries etc. JKA did away with all of that and just use DLLs.

 

Are all users in a multiplayer session required to use the same versions of dlls though? If a mp session was using different code for cgame, things would behave differently for everybody wouldnt they?

Players can use different cgame DLLs. All "behaviour" code is run by the server; players have extremely limited input into what happens in the game. In fact, the only input the player can send to the server are movement directions, their camera look angles, and which "fast-tracked" buttons they're pressing. There's a sideband channel for 'reliable' commands as well which are where the console commands get sent to. The client is essentially a dumb visualizer for the server.

 

Generally more interested in SP, but Im more interested in just tinkering with game mechanics to do interesting stuff. Even just modifying the speed that weapons fire moves at to be faster has a big effect on SP gameplay. But changes like that are obviously not part of the OpenJK branch, so I think Im going to just make that into a separate repository that isnt related to OpenJK.

 

 

I would love to tinker with integrating bullet physics into the game though. I think somebody already experimented with that before, and it looked really cool with crates & objects bouncing around.

That was possibly be me except I tried integrating the Newton Dynamics physics engine into JKA before the engine source code was even made available. I stopped because I couldn't get the level of integration I wanted through just the cgame/game modules. It's a big job - I think I estimated around 2 months for someone who actually knew what they were doing.

 

What I was thinking of in place of a roadmap, would be to create a quick comment at the top of each file in OpenJK that describes what it does. Sooo, I think Ill refork OpenJK, and make pull requests to master with the commented files, and whoever is in charge of the main branch can review the comments to make sure theyre accurate, then add them to master.

 

Does that sound like a decent idea?

Sounds good