Archangel35757

@@minilogoguy18 - I doubt Carcass would use the tangent data stored as COLORs. Since normal maps were not originally supported. Carcass might even fail with the extra data in there... because they wrote their own custom XSI parser and did not use the XSIFTK. Did you do a Carcass compile to see? Please email me your dotXSI 3.0 & 3.5 test files. Because Softimage does have a feature to compute tangent basis vectors... it can write that data to the XSI file. Not sure if 3dsMax has a similar feature for mesh objects-- @@Psyk0Sith? It may be something I would have to add in the dotXSI exporter code. If I have to update the dotXSI exporters I would leave 3.0 alone and simply add radio buttons to choose between 3.0 and 3.5 formats. @@Psyk0Sith - would you export a dotXSI 3.0 file and let me compare that to mini's results? Assuming you have diffuse, AO, and Normal map applied? @@Almightygir, yes... but we have the 3.17.16 SDK. So maybe it is a trivial matter for an experienced coder like yourself to add a feature that would allow xNormal to also write out the tangent basis vectors (it calculated from mikktspace algorithms) for an imported mesh to a *.TAN file (my extension suggestion) or your suggested *.GTB extension. In my little mind, this would be the cleanest solution since it would not require updating any exporters or tools and would work for any supplied mesh format... no? @@minilogoguy18 - I believe the consensus is to put the tangents in a supplementary file (if this is even necessary-- would still like to see dotXSI/xNormal-->Carcass-->GLM results) and not dick with the GLM format (it doesn't need to be in there anyway since it's not computed from the GLM mesh).

@@Psyk0Sith - xNormal 3.17.16 supports dotXSI format. Later versions removed dotXSI support and also removed all x86 DLLs and plugins. The SDK installer is included in the zip archive but it's only ~3Mb. The installer exe is what is large. The 3.19.2 (64-bit only) installer is ~64Mb... 3.17.16 x86/x64 installer is 155Mb.

Yes, I stated this earlier in the discussion. I did a deep-dive into the Wayback Machine... and I found xNormal v3.17.16 and its SDK. I have submitted it to the Utilities section for a permanent home (it's still awaiting approval)... but you can grab it from this link in the short term: http://ge.tt/8LfJ2zi2 Make sure you do a dotXSI "ExportSelected" and omit all the bolts and Stupidtriangle_off... because it will complain about meshes without textures. I've also written to Santiago and asked if he'd add back support for the dotXSI file format-- no response yet.

@@Xycaleth - Thank you, so my understanding is correct... and this supplementary tangent data could be readily stored in a separate file format as I suggested much earlier in this discussion. Yes, FBX stores them (as @@Almightygir originally pointed out) and so does XSI 6.x. But I don't know if xNormal supports XSI 6.x format (I doubt it)-- it may be only the more commonly supported XSI 3.x format. I've never used xNormal-- is there a built-in method to have xNormal store or write out its computed tangent basis vectors for an imported mesh?

@@Almightygir - we posted at same time... so maybe you missed my last question. I would still like for you or @@Xycaleth to clear away my confusion.

@@Xycaleth - thanks for the clarification. But my question remains... what is the point of storing tangent basis vectors in a GLM mesh file-- if the GLM mesh file itself is not used for the normal map baking process? If the normal map is baked using FBX, OBJ, XSI... isn't it those formats where it's desirable to compute or store the tangent basis vectors? Then those FBX, XSI, etc. tangent basis vectors would need to be carried over and written into the GLM mesh (and not computed from GLM directly), or am I misunderstanding?

@@Almightygir - All the tools that would require updating affect everybody-- not just DF2. And respectfully, you didn't answer my question. From where will the tangent basis vectors be computed/written into the GLM file? If you compute it from its own GLM mesh and UV data... how does the half-precision compression factor into that? Secondly, the GLM mesh would not be used to bake the normal map, right? That would be either OBJ, FBX, XSI, right? So is it not one of those formats where it is crucial to have the tangent basis vectors written? @@minilogoguy18 - look a little more carefully at the link you provided... it clearly stated, "...The following tables list the features supported when importing into and exporting from Softimage using the dotXSI 6 (*.xsi) format." Carcass supports dotXSI 1.1, 1.3, 3.0 and 3.5 formats-- and not 5.x or 6.x. So export out Gorc, with tangent data applied, as dotXSI 3.5 and 6.x and compare the two resulting files-- send them to me to check also.

@@minilogoguy18 - Softimage's native format stores the tangent basis vectors but I don't think the XSI format does-- I can't find any native template; but it would be possible to store the data in a generic custom user data template. You could export a dotXSI and see if it is in the resulting file.

@@Scooper wouldn't give me the source code to compile the GLM exporter for Max6 thru Max2011. I fully understand his position (and reluctance). I imagine he would have to be the one to change it... and keep his source closed. How about we thoroughly investigate the dotXSI/xNormal baking --> GLM conversion... and then determine if the results are spectacular or not? And then go from there... are the results from Ashura's workflow unacceptable as-is now? It looks great to my untrained eye. I still want @@Xycaleth or @@Almightygir to answer my questions above... especially about if it's the FBX or XSI that needs the tangent basis vectors (because those are the formats the normal maps will be baked from).

@@minilogoguy18 -- It's easy for you to say "...it's no big deal to change GLM importers and exporters and update Carcass..." but this isn't on my ToDo list. The Carcass source code was intrusted to an individual in confidence... and any such major change would likely not go un-noticed; shouting "we have the code" may get that source in trouble (and kill ever getting any further resources)-- so please try to be discreet. Please take in-game screenshots of @@AshuraDX's clone trooper and point out the normal map flaws (you too @@Almightygir). Everyone was thrilled with the Clone Trooper results until @@Almightygir comes along and says, "...it's good but not perfect." And we haven't even thoroughly investigated the dotXSI/xNormal-->GLM workflow. Furthermore, am I correct to say that Normal maps are NOT going to get baked using GLM meshes? Right? Baking would use FBX, OBJ, or XSI, yes? So it's those formats that need to store the tangent basis vectors (FBX already does)... yes? So you would need to copy the tangent basis vectors from the FBX or XSI files and carry that data thru the GLM conversion and write it into the GLM format, right? And like I said earlier Ghoul2 is a half-precision format (based on XSI single-precision format). So how does that compression affect the mesh data? With ALL the work that DF2 mod has to do... it seems like vanity to pursue this course until the other workflows are shown to be inadequate.

@@Xycaleth, @@DT85 - I for one am against modifying the GLM format. It breaks compatibility with Carcass, and all legacy importers & exporters for all 3D packages. We don't even know the results of exporting XSI files and baking normal maps in xNormal from the XSI files-- this could yield near-perfect results; and if that pipeline works out then modifying the GLM file is unnecessary. I think the results of @@AshuraDX's Clone Trooper are shit hot and it's very difficult to see any flaws... I dare say the standard player [and myself included] would be able to find the flaws that experts like @@Almightygir can see after the model spins around several times-- especially in a fast paced shooter game like JK. I still don't "see" the flaw he referred to on the clone trooper ass.

There is no TANGENT data in the dotXSI or GLM formats, right? So the only way would be if XSIModTool has a method to show tangent space normals... I suppose you or @@Xycaleth could modify his new ModView code to give an option to display tangent space normals for the loaded GLM model as the rend2 renderer would compute them in-game (I think this should definitly be a feature in the next version of ModView). I found a Maxscript in this CGTalk thread that computes and shows the tangents and binormals in the 3ds Max viewport-- I have no idea if this method matches how xNormal or rend2 computes them so most likely the script would have to be updated. The full script is listed for viewing... maybe you could take a look at it? http://forums.cgsociety.org/showthread.php?t=407814 EDIT: I had to dive into the Wayback Machine to find the referenced terathon.com webpage. Looks like it's based on the method in this book: http://web.archive.org/web/20061119113206/http://www.terathon.com/code/tangent.php "Mathematics for 3D Game Programming & Computer Graphics", 2nd ed., sec. 6.8.3, by Eric Lengyel.

Well, with the latest dotXSI exporters which fixed vertex normals... the GLM vertex normals (as displayed in ModView) exactly match the dotXSI and native 3dsMax mesh vertex normals... and the UVs appear unharmed as well. We will just have to do some test cases and see. Perhaps you can use your new Gorc mesh for this?

@@Almightygir - why use the .SBM format over importing the dotXSI file into XNormal and then using mikktspace? Carcass takes the dotXSI files and compiles the GLM. Since we export the models to dotXSI anyways isn't that the best? I guess we can do some comparison test with both formats. XNormal 3.17.16 is uploaded and awaiting approval.

So I have submitted XNormal version 3.17.16 (and its SDK) to the Files > Utilities section. Version 3.17.16 was the last version to support x86 apps and provide x86 plugins for Max7-2013 and Maya 8.5-2013. Newer version is 64-bit only. Also, 3.17.16 is the last version to support the dotXSI file format. If you needed to use XNormal's .SBM mesh format, I'm pretty sure that the 3.19.2 64-bit 3ds Max and Maya .SBM exporter plugin's mesh output will still work for importing with 3.17.16.

xNormal has exporter plugins that support 3dsMax, Photoshop, and Maya that exports XNormal's own mesh format (.SBM). The older version of xNormal (v3.17.16 ...no longer available to download from xNormal.net) supports older versions of 3ds Max (Max7-2013) and Maya (8.5-2013). I will upload it to Utilities shortly. XNormal also supports the dotXSI file format (...but it doesn't like mesh objects without texture coordinates-- meaning you must do an ExportSelected from 3ds Max and omit the bolts/tags and Stupidtriangle_off). @@Almightygir, @@Psyk0Sith - since XNormal directly supports importing dotXSI mesh files, and we export dotXSI files to be compiled into Ghoul2 models using Carcass... I would recommend that we create the normal maps using the exported High-/Low-Poly meshes in dotXSI format and import the dotXSI files into XNormal for normal map baking... it has options to use exported normals (as written in the dotXSI file). Seems like the best option since Carcass compiles all the dotXSI files into Ghoul2 GLM/GLA, yes (...at least for 3ds Max, Softimage, Maya workflows)? XNormal v3.17.16 supports the following mesh formats: FBX, Silo ASCII/Binary, DirectX, Milkshape3D, Orge Binary, Geomview OFF (ascii), Collada 1.4, OVB, DXF, dotXSI, Stooge, 3DS, SBM (native XNormal format), OBJ, OpenCTM, ASE, PLY, LWO, Modo. @@minilogoguy18 - did you use XNormal (and dotXSI files) for your AT-ST normal maps? Or did @@AshuraDX bake those using Substance Tools? I also have the source files for the Maya 8.5 dotXSI exporter... which could be updated to support newer versions of Maya. Blender has a dotXSI exporter in v2.42a, 2.45, 2.46) ...and I have the XSI FTK/SDK that anyone could use-- if they need it to update the Blender exporters for newer versions of Blender (@@mrwonko).

...but is it half-ass now? I don't think so-- I think it's very acceptable now. And if you undertook to update all the exporters and file formats and get an update of Carcass, etc., you have to sync all that between Rend2 and all modeling packages and their exporters... (what coders are volunteering to undertake all of this?) ...whereas it seems simpler (KISS) to let the renderer compute the tangent space normals given proper MD3 and GLM exported files with current exporters and Carcass compiler v2.2. Ghoul2 is a half-precision file format and XSI (and FBX?) are single precision file formats so does that compression impact tangent space normals?

@@Almightygir - so every LOD (LOD0, LOD1, LOD2, etc.) has to be exported out and baked against the hi-res mesh? Or do folks just accept the error and apply the LOD0 baked normal map onto the lower LODs? "...just about every piece of software out there calculates tangents for obj files the same way." And since both MD3 and GLM are well known formats to this engine and renderer, can't Rend2 compute the tangents the same for them? Given people follow a proper workflow?

@@Almightygir- I'm not sure what you mean by exported mesh tangents... yes, FBX stores them but OBJ does not-- you only get vertex normals, correct? With the latest v2.2 MD3 and v1.9 dotXSI3.0 exporters-- the vertex normals in the MD3 and dotXSI files exactly match what is stored inside 3dsMax for the mesh. But like you said, to take the mesh into another normal map baking tool it must be exported to FBX... and then the normal map is brought back into 3dsMax, and then applied to a lower res mesh right? But the low res mesh wouldn't have the same vertex normals (or tangent space normals) as the higher-res mesh anyways, right? @@AshuraDX, @@DT85, @@Psyk0Sith? I've never baked a normal map for a model yet for this game engine... But it's the lower-res mesh LOD0 that is exported for the game as either MD3 or GLM and not the hi-res mesh used to bake the normal map, right? So how does that factor in?

@@Almightygir - you said, "...Because at the time that shot was taken, there were (and still are, because legacy bullshit) many different methods of authoring normal maps..." So please forgive my ignorance... and be patient with me, but isn't this resolved/mitigated by artist education on what methods, techniques, tools and settings should be used to create compatible normal maps for the Rend2 renderer (and tweak the calculations in the rend2 code as needed for the accepted pipeline)? It would seem that way to me. Yes, perfection would be as you suggest... have the tangent basis vectors stored in the mesh, but this isn't going to happen on this engine upgrade-- as no experienced coders want to take this on; and isn't this just idiot-proofing the workflow... that could be mitigated by education as I stated above, no? The dominant model authoring tools are Softimage or its free XSIModTool, 3ds Max, and Blender. For Ghoul2 GLM models... there is only one tool that correctly compiles the GLM and creates a Ghoul2 GLA animation file-- and that is Raven's Ghoul2 Compiler called Carcass (I don't know if @@mrwonko's GLM exporter for Blender can compile GLA files). And @@Scooper created a direct GLM exporter for 3dsMax but again I don't think it compiles a GLA either. I'm sure you know... Carcass uses dotXSI files exported from 3dsMax, Softimage, or XSIModTool as input to compile the GLM/GLA files. I fixed the dotXSI exporters for 3dsMax and also added support for custom, explicit normals. And even though the XSI format can add custom user data as you suggest, Carcass would not make use of that information. For MD3 format, @@DT85 and I updated the 3dsMax exporters so that custom normals are correctly stored in the MD3 format (the format already had the potential it was just that exporters didn't compute vertex normals correctly)... this is all solved now in the v2.2 MD3 exporters. There is no MD3 exporter for Softimage and I'm not sure if Blender MD3 exporters write out custom normals. And as we already discussed... MD3 and GLM formats don't support this tangent space data.

@@Almightygir, @@Xycaleth- what's wrong with the Clone Trooper results @@SomaZ posted in the video? It appears that the normal maps have now been sorted out and fixed for both MD3 and GLM models... this is a huge achievement enabling a PBR workflow in this upgrade effort; things appear to be synced up now in rend2-- good to go now, right?

@@UniqueOne - You might like this one too... Kasper A. Andersen and Christian Bay:Effects for a Game Engine. 2006 http://image.diku.dk/projects/media/christian.bay.kasper.andersen.06.pdf Our approach introduces an infinitely large landscape with constant framerate and no stalls at runtime. With more computational power, more resources will be used for dynamic lightning, and natural effects like water. We present simple and fast shader approaches for both of these effects

VS .NET 2003 Pro installs and works perfectly on Windows 7. I have it installed on my Win7 64-bit workstation. ...but installing VS2015 Community afterwards will break the 2003 project wizard (which can be rectified by uninstalling VS2015-- maybe there's a smarter solution which would leave VS2015 installed).

@@UniqueOne - Kasper A. Andersen and Christian Bay:A survey of algorithms for construction of optimal Heterogeneous Bounding Volume Hierarchies. 2006 In this paper we describe algorithms for automatic object hierarchy construction. We develop and evaluate a Branch and Bound algorithm. The algorithm is controlled by a heuristic- and cost function. The hierarchy data structure built by the algorithm is the Approximating Hybrid Bounding Volume Hierarchy. It has nodes with varying branching factors, multiple bounding geometry types and approximating geometries in the leaf nodes. Maybe that could be helpful for your automatic collision bounding volumes?

I think the results that have been achieved with @@SomaZ's latest fixes using @@AshuraDX's Clone Trooper as a test case are really great for this older game engine upgrade with the existing MD3 and GLM model file formats. @@Almightygir, @Xycaleth - Am I correct to say that if the normal maps are created for GLM and MD3 models in a manner compatible with the latest Rend2 SP fixes (...as shown by Ashura's Clone Trooper) then it is acceptable to leave the tangent basis vector calculations within the Renderer itself, correct? Sure it would be more ideal if the tangent basis vectors were included in the model file format... but it's not necessary. I think trying to replace Ghoul2 or add FBX would be nigh impossible-- that would require major, extensive code re-writing that no experienced coder here would even begin to contemplate seriously... and they would likely say it doesn't fit within the tenets of the OpenJK project.