Open World?

[katanamaru]

How possible would it be to do an open world/sandbox type map for sp and mp?

I know everythng is possible with the code, but how practical would it be?

[Corto]

I think the engine would need a complete overhaul, starting with realtime data streaming. I'm gonna let the actual experts talk about this, but I've read a few papers on this, and it sounds great, very few actual engines employed data streaming over the years (I remember the Gecko engine used for the Soul Reaver 1 and 2 games for example).

[katanamaru]

<-- Not a mapper.

What about even faking it with smaller maps that linked together someway?

Or any other way to simulate an open world?

 

I don't know any mapping techniques, so when I talk about open world I mainly mean from the players point of view. Any technical trickery is allowed to make it work behind the scenes.

[Xycaleth]

UE3 supports level streaming. From what I understand, the whole level is split up into a grid, and each grid cell is streamed in as you get near to it, and streamed out as you get further away. This happens in the background, so you don't notice the loading (at least in theory). It would take quite a lot of effort I think to get this kind of support in JKA, and the .bsp format isn't really suited to efficient level streaming either. There's probably ways to fake an open world (within reason), like making a REALLY huge map but then you run into problems with poor frame rate.

[eezstreet]

Essentially mapping right now works off of a system that has existed since the days of Doom, called Binary Space Partitioning (BSP). Quake 3 and beyond use a slightly different BSP system, instead of WAD like Doom does (I have no clue how id Tech 2 or id Tech 4 handle it). Here's a small listing of the formats:

 

- IBSP -- Quake 3, Tremulous, Urban Terror (?)

- RBSP -- Elite Forces, Jedi Knight 2 + Academy, SoF II

- CBSP -- Call of Duty games up to (and possibly including) Ghosts

- VBSP (sometimes referred to as SBSP) -- games made using Valve's Source Engine

 

Each variant of the BSP uses different lumps, which constitute different data, such as lighting and so forth. There's also PVS, which helps determine which sections of the map are visible from different sections.

 

Now, open world games use different forms of level streaming. Gamebyro and Skyrim's engine construct a giant grid, and only the cell containing the player and a few surrounding cells are loaded during load screens. Whenever the player moves in a certain direction, additional cells become preloaded based on position, and old cells are cached and compressed in a small chunk of memory for later use. Stuff like vegetation position and so forth is tossed out completely and reloaded or regenerated I think.

 

The basic problem with open world games and attempting them in JKA is based off of three core problems:

• You cannot use level streaming, period. I've spent a lot of time trying to develop level streaming to a way that works, and frankly it isn't possible without extensive edits
• PVS is incredibly nasty to deal with in an open world game. @@Pande will know of the woes associated with using a -lomem vis compile.
• I lied, there's only two reasons. Just making sure you're paying attention here.

JKG managed to fake this pretty well. Essentially what JKG did was build each map into a series of layers. Using the -cs and -rs on different func_groups, you are able to control which func_groups are able to cast and receive shadows. Each layer would have its own cs and rs values, starting from 2 and working its way up. Using maps in the shape of valleys and using small cutscenes, the player would transition between the layers. This works pretty well, actually, despite being static. You can achieve a map with a maximum of 5 miles x 5 miles (approximately) using this method.

 

I have another theory about space and having massive star destroyers and such, but I don't want to spoil too many state secrets.

[eezstreet]

To elaborate more on what @@Xycaleth said and to tie it into what I said, you could in theory use level streaming by tossing out the PVS system and lighting and instead opting for more portal-based level stuff and dynamic lighting ala Doom 3 and use one BSP for each cell. But that leaves the texture loading, which needs less compression in order to lower load times, and the actual system itself. Lots of coding no matter what.

[ChalklYne]

What all can you currently do with loading screens? Can you change the loading screen into a RoQ instead of a texture? I know you can with the shaders, but I recall having some sort of problem before. Is there a way you can make one huge map and then separate it into a couple BSP's and transition with cutscenes instead of loading screens? Can the loading screen be a map instead? Like can you call a blank map with just the player running around during loading? I'm asking for help on my own personal project and was thinking it was relevant. 

[eezstreet]

I already pretty much said what was possible.

As far as loadscreens are concerned, ROQs in place of textures are possible and to some extent already used (Chicago streets used an animmap, which would have a similar effect). ROQs would likely be very jittery/stuttery though due to the way it works.

If you're referring to JKG though, we're completely reworking the loadscreens and also working on SWFs being used in place of shaders, so I suppose that SWFs might be used during a load screen (playing a minigame or complex animation during a load screen? COOL.)

Blank maps during loading = impossible without extensive changes to the code (think what we're doing with JKG -- asynchronous loading, but having to load another map on top of that which is incredibly inefficient on this engine)

[ChalklYne]

Thank you.

[Xycaleth]

The reason the loading screen is static is because the game is "locked up" - every time the screen changes is when the game updates the screen. Since there's no multitasking involved it happens sequentially: screen gets shown, load bsp, screen gets shown, load textures (which takes a while), screen gets shown, etc. This same problem prevents maps from being streamed in/out.

[CaptainChar]

its possible in theory, though you would need several copies of the "overworld" with slight variences, for branching storyline, its a basic theory but it should work but I don't know the max render size JKA will do before the void leaks into the map, ive had several maps that were too large and the void was shown threw the skybox, I had to change the culldistance to fix that, but ive wondered how well a sandbox style city map would fair eg a Grand Theft auto esque city

[eezstreet]

its possible in theory, though you would need several copies of the "overworld" with slight variences, for branching storyline, its a basic theory but it should work but I don't know the max render size JKA will do before the void leaks into the map, ive had several maps that were too large and the void was shown threw the skybox, I had to change the culldistance to fix that, but ive wondered how well a sandbox style city map would fair eg a Grand Theft auto esque city

This is way too much of a hassle to do something even remotely complex. If you're searching for a branching storyline, do us all a favor and code it proper. Use cull distance + fog to occlude distance and use models to display stuff beyond the max grid. Players can't go beyond the max grid but models with misc_model_static (read: not misc_model) can. Or, you can use a sky portal, but I'm not a big fan of skyportals as you can clearly see their obvious lack of detail.

[Pande]

Few things: theres no -lomem switch for vis eez, thats for lighting. I think what you are refering to is the hilarious bugs I got with the game when the lightmap data was too big (because I was covering 1/2 the X Y grid x 16 layers with lightmap on about 80k brushes). Lmao that was a funny sight... never again.

[Futuza]

I remember you wailing on the jkg irc channel for hours about that...

 

I will add this folk: if you truly desire better maps you should just find a better engine, its already written in, and there's a lot less work involved than trying to remake this poor old engine into a new one.  That being said, we at JKG are kind of hypocrits. 

[eezstreet]

Few things: theres no -lomem switch for vis eez, thats for lighting. I think what you are refering to is the hilarious bugs I got with the game when the lightmap data was too big (because I was covering 1/2 the X Y grid x 16 layers with lightmap on about 80k brushes). Lmao that was a funny sight... never again.

-lomem actually does exist for vis as I remember, but I dunno if it works properly. I think the parser for it is in place but the keyword physically does nothing.

[Pande]

No -lomem but there is -fast, which should never ever be used for a final compile (unlike -fast for light).