[Concept/Explanation] LMO Mapping

[eezstreet]

This concept needs to be started on early, because it's important, and affects over 90% (or more) of how the game functions. And I also need a lot of input and direction.

 

So let's start off with the basics.

 

 

The world, as defined by the server, will exist in dimensions, or Realms.

Each dimension has a Realm associated with it. The Realm can be either a Planet (more on this) or a static bsp. For all intents and purposes, the game as it stands right now, is a single dimension, static bsp realm. There are a large number of realms, and a near infinite number of players on the server at once, but capped to a reasonable amount (2000?) to keep server taxing low. A realm is classified as an instance (may be destroyed and has a limited number of player slots) or a static world (infinite number of players).

 

An example server, with its Realms:

• Realm 0: Tatooine
• Realm 1: Upper Coruscant
• Realm 2: Lower Coruscant
• Realm 3: Coruscant Undercity
• Realm 4: Korriban
• Realm 5: Dantooine
• Realm 6: Interspace Zone
• Realm 7: Tatooine Arena Instance ID#FE0 (players: 3/16)
• Realm 8: Jawa Fortress Instance ID#A4B (players: 1/16)
• Realm 9: Player Housing Tatooine ID#00D (players: 1/16)

When a realm has no players in it, logic is not performed in that realm, to conserve server performance.

 

 

A realm can be either a Planet or a static BSP.

Realms can contain planets or static BSPs, which are instances, player housing, ...

But a Planet is really a misnomer. A planet can represent the vastness of space, or the planet of Tatooine. What a planet really is, is a collection of new JBSP files, loaded into a grid. These JBSPs are modular in nature and can have specific lumps, including:

• PVS data
• Lighting
• Geometry
• AI Waypointing/Navmesh
• Sound Information
• Physics information

They'd be created with a new jkgmap program that operates similar to q3map2.

 

After a JBSP has been created (actually, you'll need a lot. probably 81 for a 9x9 grid), the artist can place the grid pieces into the JKGalaxies World Editor program. From there, interior locations (which are JBSPs, but have open leaf PVS nodes that can connect to the outside world, so places like Mos Eisley won't lag your machine to pieces with all the interiors) are linked to the world grid.

 

When a player changes Realms, it may trigger a map change.

But only for the client. The server doesn't need to change maps...it has them all loaded!

The client keeps a cache of JBSP files that have been loaded and cycles through them. Map loading times are dramatically lower because levels are effectively streamed from the hard drive. When you get close to an instance gate, the other level will start loading in the background so it can load quicker if the user decides to go into that level.

 

Outdoor lighting is dynamic.

But not the broken kind like in JKA - the good kind like most modern game engines do. The server can set the time of day in the world (per realm!), and since all planets are using the same clock, an effort can be made to keep all of the planets in correct sync.

 

 

What are the benefits?

Much faster map loading. Vastly increased FPS in outdoor environments. No joining servers for each map.

This is probably the biggest one here, but we're not talking a vast array of servers running all kinds of different maps here, we're talking one server running all of the maps at the same time. In some respects this means a much easier thing to create, since all servers can have their own account system and not have to tap into some kind of shared JKGalaxies account, but in some respects this is a much harder thing to create simply because of how much insane nonsense is going to need to be programmed.

 

Let's be clear, this is a pretty bold plan, but it's based on how MMOs operate already. It'll require a significant amount of retooling to do, and it's a daunting task for sure.

 

You can (in theory) fly a ship directly into space.

Yeah. That. No more need for "shuttles" - space battles could potentially just be handled by a series of interspace realms, such as "Coruscant Airspace" or "Tatooine Airspace", with a few linking ones between them. That's of course, if we decide to pull the trigger with space combat, which will require quite a bit of work, but it could be pulled off.

 

Much faster asset creation.

No more 8+ hour compiles for huge ass desert maps. No more running out of memory when compiling maps.

 

Cross-planet communication.

You can for instance, open up the possibility for features such as a Looking for Group feature, or perhaps an announcer for the Tatooine Arena. Or whisper to a friend on another planet. All of those would be difficult if not impossible to do on a cross-server setup.

 

[ooxavenue]

You may want to attempt to factor in a 3D map grid a lets say 9x9x9  which may be exsive and complicated but would waste less space me thinks, considering that maps RBSP maps may waste a ton of airspace, depends really if RBSPs are going to be cubed^3 in nature or a just on a plane of RBSPs on coordinate table, with the individual bsp being a 3d world. I am going to be most intrested in seeing how you can cook this up.

The only draw back i can see is on the server initialize, cause that load time may be massive, but once everything is loaded i dont really see a problem other then hitting the ent limit, but if you can get the certain areas of rbsp multithreaded safely ent limts might not be a proplem.

 

RBSP would also be a concept of multiple processes of JKG (local and cross server maybe?) all sharing the same RBSP grid type with each node in the grid having fields of IP, X, Y, (Z? if 3d) and would connect or "move" you cross local process or server. the RBSP, could maybe also be generated/hosted by a JKG master server, and the master server would dictate what is the universal RBSPs and control what BSPs the server would load, so any server running JKG would be added to the RBSP master server and provide a viable realm for people to play on

[eezstreet]

You may want to attempt to factor in a 3D map grid a lets say 9x9x9  which may be exsive and complicated but would waste less space me thinks, considering that maps RBSP maps may waste a ton of airspace, depends really if RBSPs are going to be cubed^3 in nature or a just on a plane of RBSPs on coordinate table, with the individual bsp being a 3d world. I am going to be most intrested in seeing how you can cook this up.

 

The only draw back i can see is on the server initialize, cause that load time may be massive, but once everything is loaded i dont really see a problem other then hitting the ent limit, but if you can get the certain areas of rbsp multithreaded safely ent limts might not be a proplem.

 

RBSP would also be a concept of multiple processes of JKG (local and cross server maybe?) all sharing the same RBSP grid type with each node in the grid having fields of IP, X, Y, (Z? if 3d) and would connect or "move" you cross local process or server. the RBSP, could maybe also be generated/hosted by a JKG master server, and the master server would dictate what is the universal RBSPs and control what BSPs the server would load, so any server running JKG would be added to the RBSP master server and provide a viable realm for people to play on

You're right, the ent limit might be a problem. Which is why it should be removed.

 

A vertical element is not really necessary I think, you aren't really going to need to load things vertically, but maybe.

[Futuza]

I'll be honest...this sounds very very difficult.  Almost easier to just switch engines, where most of the work is already done for us.  But I guess we're kind of crazy.  2000 players is also a very high server cap.  Something like 400-900 is probably more reasonable.  As ooxavenue mentions, it doesn't necessarily need to always be 9x9 in all cases.  For example you might only have a southern and western grid open to mos eisley with no path available to the east and north (also going down is not always going to be an option).  Getting the server to also process for that many players at once will need not only severe optimizations to the current way the game is run, but also a very powerful machine.  Most MMOs have server farms doing this kind of work.  

 

I still think a simpler route is to limit each "planet" (eg: Tatooine, Coruscant, etc.) to a separate server and have information handed off or permanently saved to a master server, which only has to deal with logins, client handoffs and database information saves/loads, not running the game.  I still think having dimensions or realms that occur on that planet is a good idea, if it is all limited to each planet, based on the setting.  Having one machine responsible for the entire game, seems kind of like putting your eggs all in one basket (yes I know the master server idea is still this way, but its much easier to make a database server redundant, then rely on one machine for everything).  While space is cool, I think its an idea that should definitely wait till later, after the major designs are worked out and actually created and not worried about yet at this point.  JBSPs should probably be one map (easier for the artist to make it that way, if it's all together), but be divided up evenly into grids automatically by the server, with the ability to manually specify divisions possible.  

 

I'd also like to build some autogeneration tools into the map builder.  For example, making terrain in the game engine right now is a pain in the butt.  It should be more like Cryengine: create a B&W bumpmap, import it, generate terrain, paint terrain with texture and modify in realtime as necessary with height/curve adjust painting tools.  Then you add details like models and entities.  Also simply a random map terrain creator, which uses a rand seed to generate a bump map and then texture paint it and smooth it out to a reasonable level.  Additionally there ought to be an easy way to add boundaries to maps that create the illusion of being infinite.  For example, on Tatooine, Mos Eisley is surrounded by desert.  If you walk to the edge of the desert, eventually you run into either a wall or artificial barrier.  To help extend the illusion on certain maps, you could for example procedural generate a small amount of terrain extending the edge in real time as you approach closer to it.  That might create extreme memory requirements though, so another option is to simply allow edges to stealthily teleport the character backwards toward the true edge, after the edge boundary is crossed, while making it appear that the character is actually moving forward toward an unreachable horizon.  Music will also no longer be tied to the map, but to in game LUA events (eg: during fights play battle music, then fade to map ambiance music a few moments after the fight finishes, etc).  Destructible terrain is also fun.

[Tempust85]

Can the ent limit be removed just like that? If that's the case, why is it in JKA in the first place?

[Ramikad]

I approve everything that can get this crazy.

 

 

 

 

 

[Xycaleth]

Can the ent limit be removed just like that? If that's the case, why is it in JKA in the first place?

One problem with having too many ents is that your network load will increase a lot as well. Imagine if each player adds an extra 100 bytes to each data packet, and you have 1000 players, that's 100kb per snapshot. Then multiply that by 20 (because you're running with sv_fps 20), every client now needs to have at least a 2mb/s downstream link and your server needs a 2gb/s upstream since it needs to send the updates to all 1000 players. Realistically a player probably only adds a few tens of bytes but the same principle applies. If you have different instances on the same server then things become a bit more manageable as you would effectively be running multiple servers (or that's how it would look from the outsiders point of view). Each instance is isolated from the other so you only need to send data about the current instance to the players in that instance.

[eezstreet]

Answering a few random questions:

- The ent limit in Q3 is because the devs went with a static limit because it was easier to work with.

 

- Mos Eisley won't be a separate planet, it will be part of Tatooine.

 

- Game music won't be tied to Lua (that much..), the client will figure out what music is best for them based on the situation.

 

- New terrain tools are possible (but b/w maps can already be used, see Easygen)

 

- Why autogenerate terrain when you can just have it be part of the planet? (Albeit inaccessible)

 

- Each planet being a separate server is not good, as what happens if a server goes down? Your character becomes stuck. It's more logical to handle it all on one server. Also it means that you need to disconnect from one server and reconnect to another, which is no bueno.

[Xycaleth]

If each server is a planet and the server goes down then players can continue to play on the other servers You can have a recovery system that starts up the crashed server again

[Futuza]

I'd rather have some players able to play, then no players at all.  Sure characters can get stuck, but only as long as the server they are on is down, better than the entire player population being unable to play.  I'm just not sure I see the benefit of putting it all on one machine, other then you don't have to have a handoff system and wait/loading times between planets is probably going to be a bit shorter.

 

- New terrain tools are possible (but b/w maps can already be used, see Easygen)

- Why autogenerate terrain when you can just have it be part of the planet? (Albeit inaccessible)

Easygen is not easy to use.  It's clunky and the functionality should have just been built into the map editor in the first place, which is why I bring it up.

 

??? Because then somebody has to do extra work to make that part of the planet? More work done by auto generate tool means artists have time to focus on other areas.  Not sure exactly what you mean.

[UniqueOne]

Man this really is the dream. Star Wars the way it should be. I have a few thoughts that might make this a little easier, but i'll talk to you on irc next time your there.

 

I use a simple random generator for my grass system that given the same coordinates as seed will give back the same random values each time. Might be easier to do the whole universe using these random numbers based on the coordinates. You could use something like a universe map image to specify "climates" across the universe. I'm not sure how one would fit old maps into this system though. Maybe using pre-allocated locations.

[Xycaleth]

I'm completely against using random numbers for environments unless it's used in very minimal ways. You have no control over what the environment will look like if you use it for everything and it will almost always look unnatural. Using them as a starting point to work from is a good idea though - that I'd be very much interesting in seeing.

[UniqueOne]

I'm completely against using random numbers for environments unless it's used in very minimal ways. You have no control over what the environment will look like if you use it for everything and it will almost always look unnatural. Using them as a starting point to work from is a good idea though - that I'd be very much interesting in seeing.

 

That's why I suggest using image maps for static info like the climates across the length of the universe. Use random to get terrain info and where planets are located, use image lookups for the rest. I suggest this mostly because it takes a lot of the work out of getting things working. It could be replaced or overridden by static images at any place too I guess. In essence it creates the generic clutter of stuff in the universe and you throw whatever you choose on top of it.

 

I expect vis and collision to be the hard parts. Maybe both could be done using heightmaps generated on the fly combined with object bounds for the models placed? heightmaps wouldn't work well from above (occlusion culling required?), but on the ground they could be used to quickly cull a lot of terrain stuff I would think.

[Onysfx]

About the entity limit. Will the limit be reset for every new map you go into on the 9x9x9 grid? Or do they all share the same entity limit?

 

Also I honestly don't think we're going to have to worry about having 2000+ players (could be wrong though).

 

I really like the idea of splitting houses onto different bsps. So basically the player will be teleported there when he enters a house? Or would it be possible to implement a portal like feature where he can see the inside of the house through the doorway from another bsp?

[Tempust85]

This could be really cool, and even cooler if mixed with a finished rend2 for visual candy.

[eezstreet]

I'd rather have some players able to play, then no players at all.  Sure characters can get stuck, but only as long as the server they are on is down, better than the entire player population being unable to play.  I'm just not sure I see the benefit of putting it all on one machine, other then you don't have to have a handoff system and wait/loading times between planets is probably going to be a bit shorter.

 

Easygen is not easy to use.  It's clunky and the functionality should have just been built into the map editor in the first place, which is why I bring it up.

 

??? Because then somebody has to do extra work to make that part of the planet? More work done by auto generate tool means artists have time to focus on other areas.  Not sure exactly what you mean.

I don't know, I think it could be in theory possible to do both a one-server, all worlds/multi-server, one world each, and just have an instance be either hosted on the same server or on a different one...I will have to think on this some more.

 

Easygen is bad, you're right, but the game actually did have a terrain entity for RMG (unused feature) which could do heightmaps in the game itself.

 

Auto generate is not going to look great, and it's more programmer time to develop.

 

Man this really is the dream. Star Wars the way it should be. I have a few thoughts that might make this a little easier, but i'll talk to you on irc next time your there.

 

I use a simple random generator for my grass system that given the same coordinates as seed will give back the same random values each time. Might be easier to do the whole universe using these random numbers based on the coordinates. You could use something like a universe map image to specify "climates" across the universe. I'm not sure how one would fit old maps into this system though. Maybe using pre-allocated locations.

I'm not on IRC at all anymore, I use Discord now.

 

As far as random generation:

I'm completely against using random numbers for environments unless it's used in very minimal ways. You have no control over what the environment will look like if you use it for everything and it will almost always look unnatural. Using them as a starting point to work from is a good idea though - that I'd be very much interesting in seeing.

This.

 

About the entity limit. Will the limit be reset for every new map you go into on the 9x9x9 grid? Or do they all share the same entity limit?

 

Also I honestly don't think we're going to have to worry about having 2000+ players (could be wrong though).

 

I really like the idea of splitting houses onto different bsps. So basically the player will be teleported there when he enters a house? Or would it be possible to implement a portal like feature where he can see the inside of the house through the doorway from another bsp?

There is basically no entity limit. If there is one, it's per-Realm.

 

You're probably correct, the 2000 figure is just a random number I made up. But the system should at least be scaleable.

 

Basically yes, to both of those questions. You can preview a house if the client has loaded the JBSP fully, but some stuff (furniture etc that are entities) will probably not be visible. It would internally be teleporting you to somewhere else, and if the house is pretty large, you'll likely get a loading screen before the map loads up all the way.

 

This could be really cool, and even cooler if mixed with a finished rend2 for visual candy.

It would only be started after rend2 is in a good spot where I can use it. A lot of the things I want to do (not terrible dynamic lighting, ..) pretty much require rend2. So it's even cooler.

[Onysfx]

I personally think tatooine should be the first world to work on. Big and open, lots of flat desert to occupy the grid.

[Ramikad]

I personally think tatooine should be the first world to work on. Big and open, lots of flat desert to occupy the grid.

 

Would probably be a good test of the system. Imagine a properly scaled Tatooine planet map...

 

 

 

 

 

[Onysfx]

@@Ramikad

 

I don't see the Boonta Eve race track on that map.

[Futuza]

You know it's kind of funny because the cannon doesn't match up and contradicts itself.  This is the race course, no such formations exist on the other map near Mos Espa.

[Ramikad]

You know it's kind of funny because the cannon doesn't match up and contradicts itself.  This is the race course, no such formations exist on the other map near Mos Espa.

 

Perhaps the place has changed in 32 years

[Futuza]

Unless there was some crazy seismic activity I doubt three decades is enough to change it that much.  Maybe if we were talking the Dune Sea where everything is literally just moving piles of sand, but this is solid mountain/rock formation area.  It's kind of good though, contradictions like this gives us more room to be more creative with what JKG's Tatooine will look like, since canon itself isn't consistent no one can get mad if we add a canyon here or there.

[Ramikad]

Unless there was some crazy seismic activity I doubt three decades is enough to change it that much.  Maybe if we were talking the Dune Sea where everything is literally just moving piles of sand, but this is solid mountain/rock formation area.  It's kind of good though, contradictions like this gives us more room to be more creative with what JKG's Tatooine will look like, since canon itself isn't consistent no one can get mad if we add a canyon here or there.

 

The Clone Wars, maybe. Lots of explosions and blaster fire. Or the Tusken vs Jawa. Or a small meteorite + sandstorms. Or abuse of rock, ore and mineral extraction for low-quality houses to rebuild after the Clone Wars. Or Krayt Dragons.

 

Or...