code FEATURE: OpenGL 3 Renderer

[SomaZ]

Good stuff guys.

 

Looks like tangents for the older models are fucked (funky shading definitely visible), might be worth taking the attitude of: older models load, and work, but look bad... start working in fbx from now on guyzzzz. or something along those lines.

 

Also, there's something fucky going on with the fresnel term on the spheres in the background, it's all kinds of weird. Let me know if you want me to take a look at the shader code.

 

Question: how are you handling the importance sampling? in realtime? or using the split-sum equation Epic posted up in 2014?

Shader is a mess right now. Actually I want to rewrite some parts of it, but first I want to add some other stuff to the renderer. Else we do things twice or trice. I'd love someone else to check the shader. Let's say, I will mention you if the renderer is, let's say, prepared. Right now it uses a wild mix of stuff, like a mixed specular brdf from Renaldas siggraph15 and frostbite. (super gross).

 

@@DT85 Thank you for the kind words. Besides that. Have you tried compiling your maps with deluxe mapping? This kind of works now. It saves light directions per surface. If there is more than one light affecting the surface, the light direction gets averaged (this looks shitty, but maybe your maps can work like that)

[minilogoguy18]

@@AshuraDX, it will be a lot of work but since this is just a SP mod and not the base game in which people are going to be adding a hundred or so extra characters into we really just have to worry about our own content supporting PBR. If someone wants to add say Kylo Ren or something to the mod well then yeah, they'll be shit outta luck unless they update the maps to work. Right now we have around 20 or so characters if you include civilians and droids. The maps textures will need to be updated which wont be too bad since we only have a few finished maps and the weapons which were going to get an overhaul anyway.

 

Why not just help us out? =p

[Tempust85]

Shader is a mess right now. Actually I want to rewrite some parts of it, but first I want to add some other stuff to the renderer. Else we do things twice or trice. I'd love someone else to check the shader. Let's say, I will mention you if the renderer is, let's say, prepared. Right now it uses a wild mix of stuff, like a mixed specular brdf from Renaldas siggraph15 and frostbite. (super gross).

 

@@DT85 Thank you for the kind words. Besides that. Have you tried compiling your maps with deluxe mapping? This kind of works now. It saves light directions per surface. If there is more than one light affecting the surface, the light direction gets averaged (this looks shitty, but maybe your maps can work like that)

 

I'll give deluxe mapping another go since your update, it never did anything before.

[AshuraDX]

@@minilogoguy18 because I allready have more than enough WIPS running. I also have offered to help with some textures in the past.

[Almightygir]

Well the problem is, you'll want the GLM format to be able to store custom tangents, that's how games engines currently sync normal maps (using the mikkt-space normal format anyway). They do this by storing whatever tangents (either user created, or auto-generated) the mesh has in the fbx file format, this way the baker can bake normals using those exact tangents, and the renderer can render using the same tangents.

 

If the mesh tangents change at any point in the chain after normal maps are baked, they will never be in sync, and that's what leads to the rendering errors seen above.

 

The FBX documentation tells you how they store tangent and binormal data (you will need both). With that data stored in the mesh, you can use the info here to make sure that the renderer is synced to substance, knald and toolbag's normalmap bakers: https://github.com/tcoppex/ext-mikktspace

[Almightygir]

Also, hi Psyk0!

[Almightygir]

Shader is a mess right now. Actually I want to rewrite some parts of it, but first I want to add some other stuff to the renderer. Else we do things twice or trice. I'd love someone else to check the shader. Let's say, I will mention you if the renderer is, let's say, prepared. Right now it uses a wild mix of stuff, like a mixed specular brdf from Renaldas siggraph15 and frostbite. (super gross).

 

@@DT85 Thank you for the kind words. Besides that. Have you tried compiling your maps with deluxe mapping? This kind of works now. It saves light directions per surface. If there is more than one light affecting the surface, the light direction gets averaged (this looks shitty, but maybe your maps can work like that)

 

Sure man, i don't check here that often so you're probably better off emailing me if you need to get hold of me:

lee.devonald[at]gmail[dot]com

Edited February 23, 2017 by Circa
saved your inbox's life

[Psyk0Sith]

Also, hi Psyk0!

 

Oh hei thar!!!

[Tempust85]

@@Almightygir

 

The issue above was due to a coding error where tangents weren't being zero'd.

 

Afaik, both MD3 & GLM do not store tangent or bitangets in the file, only normals. We're shit outta luck there unless someone wants to upgrade the formats and then the exporters.

[minilogoguy18]

Hmm, I'm sure a lot of work would be involved in switching the file format to something different, we at least don't have to worry about compatibility since this is an SP mod but at this point it's moving in the direction of being a stand alone indie title.

 

The upside is that we'd be able to ditch some of the other limitations that come with GLM/GLA and probably use 1 format for every 3d object, static or animated.

[Tempust85]

Alot of work is an understatement lol. Not only would you need to add the format to the renderer, you then also have to change the game code to load it and ditch ghoul2 loading. You can't even use MD3 as a player model (ala quake3) - that's how much Ghoul2 is tied into the game code.

 

Honestly I'm quite happy with what we got now, which is ALOT better than poor ol' rend2 MP with its borked GLM tangents.

[minilogoguy18]

So am I correct to assume by what you say that the DF2 engine in ways has surpassed the standard OJK w/Rend2? If so that's pretty great.

[Archangel35757]

Well the problem is, you'll want the GLM format to be able to store custom tangents, that's how games engines currently sync normal maps (using the mikkt-space normal format anyway). They do this by storing whatever tangents (either user created, or auto-generated) the mesh has in the fbx file format, this way the baker can bake normals using those exact tangents, and the renderer can render using the same tangents.

 

If the mesh tangents change at any point in the chain after normal maps are baked, they will never be in sync, and that's what leads to the rendering errors seen above.

 

The FBX documentation tells you how they store tangent and binormal data (you will need both). With that data stored in the mesh, you can use the info here to make sure that the renderer is synced to substance, knald and toolbag's normalmap bakers: https://github.com/tcoppex/ext-mikktspace

 

 

@@Almightygir

 

The issue above was due to a coding error where tangents weren't being zero'd.

 

Afaik, both MD3 & GLM do not store tangent or bitangets in the file, only normals. We're shit outta luck there unless someone wants to upgrade the formats and then the exporters.

 

Suggesting to modify these widespread, popular, legacy formats is a non-starter in my opinion-- that would break legacy MD3 and GLM model compatibility as used by other games... (and not to mention every legacy exporter/importer for MD3 and GLM would also get broken).  MD3 and GLM file formats both support custom vertex normals. 

 

Maybe in the future there could be a supplemental file format (*.TAN), that stores only the tangent-basis for an MD3 or GLM normal-mapped model, which Rend2 could use (if the *.TAN file was found in the model file's directory-- e.g., myModel.md3, myModel.glm, myModel.tan).  But even this would be unnecessary as long as it is clearly documented on how normal maps should be created for OpenJK and Rend2.

 

For now, the artists will need to be educated on what methods, procedures, and normal map baking tools and their settings (as stipulated by 3ds Max, Substance Designer, ZBrush, Blender, XNormal, etc., etc.) are required to properly bake normal maps for MD3 and GLM models so that they generate a compatible tangent-basis for use in the Rend2 Renderer.

 

I am one that will have to educate myself on the proper normal-mapping and Physically-Based Rendering (PBR) workflows for 3ds Max and the OpenJK SP Rend2 Renderer (being developed in DFII and merged back into OpenJK Rend2 for others to use).  I'm sure @@Psyk0Sith and @@AshuraDX could provide great links for this re-education... but here are a few I found:

 

http://wiki.polycount.com/wiki/Normal_map

 

http://wiki.polycount.com/wiki/Normal_Map_Technical_Details#Tangent_Basis

 

https://docs.google.com/document/d/1Fb9_KgCo0noxROKN4iT8ntTbx913e-t4Wc2nMRWPzNk/edit

 

http://www.hard-light.net/wiki/index.php/PBR_Workflow#Albedo

[Almightygir]

Allow me:

 

There is no proper normal map rendering workflow for 3ds max. 3ds max isn't even synced to itself... or Stingray, which is hilarious. Maya might be possible. But why sync to either of those when the core/popular baking softwares are using mikktspace?

 

As for PBR workflows... it's really simple: texture the way you always did, if you're a good texture artist you'll have no problem. All that's really changed about the way things are rendered in PBR compared to legacy methods is that energy is now conserved, so as a surface becomes more reflective, it becomes less diffuse, simples.

 

Personally, i would suggest scrapping the modification of older formats all-together. leave them in as legacy, so old models will "work". But add FBX support and be done with it. If you're overhauling an entire renderer, at some point you HAVE to draw a line and say "this old shit won't work".

 

edit:

HAH! i'm referenced in one of those links.

[minilogoguy18]

@@Almightygir, man, I missed you lol.

 

Despite being a lot of work it would be nice to implement FBX on top of keeping existing G2 although I don't know exactly how we'd handle the player animation part since for MP compatibility all characters must use the same animation file but maybe there is some way around that. I'm pretty sure FBX supports letting the model read from a separate animation file so perhaps there could be a clone of the _humanoid bank but who knows, I'm just a monkey on a treadmill doing what I can to get that banana hanging from the fishing pole taped to my back.

 

I've been bringing myself up to speed on the PBR workflow and it really isn't that big of a deal, you just have more maps to allow the surface to react to lighting in more dynamic ways giving a much higher detailed result.

 

G2 has a shit ton of draw backs, the coordinate system is wonky as hell and not having proper tangent support is going to be a problem on a render engine that we're  trying to use to take advantage of a property that allows proper normal map rendering.

[Archangel35757]

I think the results that have been achieved with @@SomaZ's latest fixes using @@AshuraDX's Clone Trooper as a test case are really great for this older game engine upgrade with the existing MD3 and GLM model file formats.

 

@@Almightygir, @Xycaleth - Am I correct to say that if the normal maps are created for GLM and MD3 models in a manner compatible with the latest Rend2 SP fixes (...as shown by Ashura's Clone Trooper) then it is acceptable to leave the tangent basis vector calculations within the Renderer itself, correct?  Sure it would be more ideal if the tangent basis vectors were included in the model file format... but it's not necessary.  I think trying to replace Ghoul2 or add FBX would be nigh impossible-- that would require major, extensive code re-writing that no experienced coder here would even begin to contemplate seriously... and they would likely say it doesn't fit within the tenets of the OpenJK project.

[Xycaleth]

Firstly it doesn't really matter what the OpenJK project can/can't do in this situation. You're making something standalone and separate from it so it doesn't make sense to restrict yourselves unnecessarily.

 

For the tangents, I don't know. AlmightyGir is more likely to know more about that than me as that's his area of work.

[Almightygir]

Well, the tangent calculations should be done by the renderer, but the tangents need to come from somewhere for those calculations to happen. for each vertex you should be able to read a tangent, normal, and binormal.

 

I'm not sure why it would be so difficult to store tangents in GLM format, you have access to the format, you know how to write it and you know how to read from it. So write in a new boolean which is only activated on exporting if the user desires it to be so, and write the tangent and binormal info into the end of the file. When reading, check for that bool, if it's true, find the new tangent info, if false or not present treat it like an older GLM file. From my own (limited) experience of writing mesh importers, this should be do-able. That said, i don't know the inner workings of the engine, or rather, how organized or disorganized it is.

[SomaZ]

@@Almightygir Can't we simply take the mikktspace sources you provided to calculate the tangents and bitangents in the renderer with it? The result should be the same, or am I mistaken?

[Almightygir]

well, the problem is this:

 

in order to bake the maps out, you'll have to bake in max/maya/substance/whatever, right? outside of max and maya (which don't use mikkt), nothing will support the GLM format. so you will always be baking with one set of tangents (because obj handles them differently to fbx, and certainly differently to glm), and then rendering with another set of tangents.

 

BUT, it will look a lot closer than it does now.

[Archangel35757]

@@Almightygir, @@Xycaleth- what's wrong with the Clone Trooper results @@SomaZ posted in the video? It appears that the normal maps have now been sorted out and fixed for both MD3 and GLM models... this is a huge achievement enabling a PBR workflow in this upgrade effort; things appear to be synced up now in rend2-- good to go now, right?

[minilogoguy18]

Well changing the GLM format seems like it would be the way to go since the game can already read that but some things seem like they would need some changing, for instance the carcass compiler. If someone is going to go so far as to alter how it outputs a GLM file they may as well add FBX file reading by Assimilate/Carcass and keep dotXSI around only for legacy purposes. This will also open up a lot of possibilities for pretty much any software the user desires to be used to make content for this game.

 

While slightly off topic we could also use MD3 support for Maya, I'd like to see Softimage support for it but since the software is no longer being made people may not be interested in writing a plugin for MD3.

[Almightygir]

@@Almightygir, @@Xycaleth- what's wrong with the Clone Trooper results @@SomaZ posted in the video? It appears that the normal maps have now been sorted out and fixed for both MD3 and GLM models... this is a huge achievement enabling a PBR workflow in this upgrade effort; things appear to be synced up now in rend2-- good to go now, right?

 

This image is probably the best way to explain the problem here:

 

Being aware that this isn't Unity, the problem is largely the same. Because at the time that shot was taken, there were (and still are, because legacy bullshit) many different methods of authoring normal maps for unity, and also the tangents for the meshes. You can see that not only do different bakers calculate tangent space differently, but having the mesh tangents imported and read correctly by both the baker and the renderer is important for a proper/good looking result.

 

In the turnaround posted earlier:

 

The most obvious error is on the central plate on the butt, there were a few others i spotted. And while you might say "eh... it's not THAT bad...", it's a problem that will cause artists to pull their hair out over "Y IT LOOK BAD?!" for until the end of time, simply because of a rendering mis-match, they'll always wonder if it's them or the renderer that fucked up. Don't give them that doubt, fix the renderer so they know it's their shitty art work

 

I'm not trying to belittle the accomplishment, 90% of the work is done... It's time for the last 90% now

[Archangel35757]

@@Almightygir - you said, "...Because at the time that shot was taken, there were (and still are, because legacy bullshit) many different methods of authoring normal maps..."

 

So please forgive my ignorance... and be patient with me, but isn't this resolved/mitigated by artist education on what methods, techniques, tools and settings should be used to create compatible normal maps for the Rend2 renderer (and tweak the calculations in the rend2 code as needed for the accepted pipeline)? It would seem that way to me. Yes, perfection would be as you suggest... have the tangent basis vectors stored in the mesh, but this isn't going to happen on this engine upgrade-- as no experienced coders want to take this on; and isn't this just idiot-proofing the workflow... that could be mitigated by education as I stated above, no?

 

The dominant model authoring tools are Softimage or its free XSIModTool, 3ds Max, and Blender.

 

For Ghoul2 GLM models... there is only one tool that correctly compiles the GLM and creates a Ghoul2 GLA animation file-- and that is Raven's Ghoul2 Compiler called Carcass (I don't know if @@mrwonko's GLM exporter for Blender can compile GLA files). And @@Scooper created a direct GLM exporter for 3dsMax but again I don't think it compiles a GLA either.

 

I'm sure you know... Carcass uses dotXSI files exported from 3dsMax, Softimage, or XSIModTool as input to compile the GLM/GLA files.

 

I fixed the dotXSI exporters for 3dsMax and also added support for custom, explicit normals. And even though the XSI format can add custom user data as you suggest, Carcass would not make use of that information.

 

For MD3 format, @@DT85 and I updated the 3dsMax exporters so that custom normals are correctly stored in the MD3 format (the format already had the potential it was just that exporters didn't compute vertex normals correctly)... this is all solved now in the v2.2 MD3 exporters. There is no MD3 exporter for Softimage and I'm not sure if Blender MD3 exporters write out custom normals. And as we already discussed... MD3 and GLM formats don't support this tangent space data.

[Almightygir]

okay, i'll try to simplify further...

 

in order to bake your normalmaps, you have to either export to fbx, or obj first, because no baker supports GLM. This is mesh "a". Mesh a has a set of tangents that can largely be reliably predicted because the obj format is simple as balls, and fbx is well documented. In either case, we know what the mesh tangents are on export, but we also know they won't be the same as GLM (or can be reasonably sure).

 

once you've baked, you then have to export to GLM, which has a different set of tangents.

 

the renderer would be trying to use a normalmap baked with tangents a, on a mesh whos vertices now have tangents b. the math wouldn't add up.